====================================================================== DUNE HD STB -- BASIC DEVELOPMENT INFORMATION ====================================================================== Important notes about latest ATV and Android 11+ AOSP platforms: ================================================================ On ATV and Android11+ AOSP platforms, please prepend the following FS_PREFIX to the paths /config, /tmp, /flashdata, /firmware, /firmware_ext: /data/data/com.dunehd.app I.e.: /data/data/com.dunehd.app/tmp/ /data/data/com.dunehd.app/config/ /data/data/com.dunehd.app/flashdata/ /data/data/com.dunehd.app/firmware/ /data/data/com.dunehd.app/firmware_ext/ (In other documentation sections, this FS_PREFIX may be omitted; please prepend it on your own.) In PHP plugins and CGI programs, you can use the following env.variables to write code compatible with all platforms: FS_PREFIX: /data/data/com.dunehd.app on ATV and Android 11+ AOSP platforms; unset otherwise. HD_APK: 1 on ATV and Android 11+ AOSP platforms; unset otherwise. HD_FW_APK: 1 on Anroid 11+ AOSP platforms; unset otherwise. HD_HTTP_LOCAL_PORT: 11080 on ATV platforms; empty (old firmwware versions) or 80 (new firmware versions) on all other platforms. Notes on adopting PHP plugins for latest ATV and Android 11+ AOSP platforms: The PHP engine adds FS_PREFIX to all paths when needed automatically, except commands passed as arguments to such as shell_exec() and system(), so there are good chances that the old PHP plugin code will fully work on new platforms w/o any code changes. On Android 11+ AOSP platforms, /tmp folder is available both as $FS_PREFIX/tmp and /tmp. Other differences in the latest ATV and Android 11+ AOSP platforms: The process name of the main Dune HD GUI process is not "com.dunehd.shell", but "com.dunehd.app". If the plugin contains executable shell scripts, they can not be executed directly; please call these scripts via "sh" or "bash" (example: "bash /path/to/script"). The plugin can not contain binary executable files. If the plugin requires some specific binary executable file, please send a request to Dune HD to consider including this executable file in next software updates. Platforms ========= New Android 11 AOSP platforms: Android 11, SoC Amlogic S905W2 Android 11, SoC Amlogic S905X4 Android 11, SoC Amlogic S905Y4 (operator models) Android 11, SoC Amlogic S928X Android TV platforms (Dune HD Media Center APK running on Google-certified Android TV OS): Android 11, SoC Amlogic S905X4 Android 12, SoC Amlogic S905X4 Old Android AOSP platforms: Android 6, SoC Amlogic S905L/X/D Android 7, SoC Amlogic S905L2 Android 9, SoC Amlogic S905X3 Android 7, SoC Realtek RTD1295 Android 7, SoC Realtek RTD1395 Android 9, SoC Realtek RTD1619/RTD1619DR Old Sigma Designs platforms: Sigma Designs 8642 platform: STB models: Base 3.0, Prime 3.0, Smart D1/H1/B1, Max, Duo CPU: MIPS 74Kf 667 MHz (little endian) Linux kernel: 2.6.22.19 Sigma Designs 8652 platform: STB models: Lite 53D CPU: MIPS 24Kf 500 MHz (little endian) Linux kernel: 2.6.22.19 Sigma Designs 8670 platform: STB models: TV-101, TV-102p, TV-201*, TV-301. CPU: MIPS 24Kf 700 MHz (little endian) Linux kernel: 2.6.29 Sigma Designs 8674 platform: STB models: TV-102, TV-202*, Connect CPU: MIPS 24Kf 700 MHz (little endian) Linux kernel: 2.6.32 Sigma Designs 8672 platform: STB models: TV-303D, Base 3D, Connect 3D* CPU: MIPS 74Kf 800 MHz (little endian) Linux kernel: 2.6.32 SIgma Designs 8756 platform: STB models: TV-204 CPU: ARMv7 Cortex A9 1000 MHz (little endian) Linux kernel: 3.4.39.13 SIgma Designs 8758 platform: STB models: TV-205, TV-206, Solo4K CPU: ARMv7 Cortex A9 1200 MHz, dual core (little endian) Linux kernel: 3.4.39.13 NOTE: the models marked with "*" are not officially launched. Toolchain ========= Toolchain for SMP86xx MIPS platforms ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Toolchain version: gcc 4.3.3 (CodeSourcery mips-4.3-154) Libc version: GNU libc 2.8 (Sourcery G++ Lite 4.3-154) Download: http://www.codesourcery.com/sgpp/lite/mips/portal/package4432/public/mips-linux-gnu/mips-4.3-154-mips-linux-gnu-i686-pc-linux-gnu.tar.bz2 Usage instructions: Use "-EL" compile flag. E.g.: /path/to/mips-4.3/bin/mips-linux-gnu-gcc -EL -o a a.c Toolchain for SMP87xx ARM platforms ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Toolchain version: gcc 4.8.1 (Mentor arm-2013.11-33) Libc version: GNU libc 2.18 (Sourcery G++ Lite 2013.11-33) Download: http://sourcery.mentor.com/public/gnu_toolchain/arm-none-linux-gnueabi/arm-2013.11-33-arm-none-linux-gnueabi-i686-pc-linux-gnu.tar.bz2 Toolchain for Android platforms ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Android NDK version: r14b Android NDK API level: 19 Download: https://dl.google.com/android/repository/android-ndk-r14b-linux-x86_64.zip Usage instructions: $NDK/build/tools/make_standalone_toolchain.py --arch arm --api 19 --install-dir $TOOLCHAIN $TOOLCHAIN/bin/arm-linux-androideabi-gcc -mcpu=cortex-a7 -mfloat-abi=softfp -mfpu=neon -pie -o a a.c Telnet access to STB ==================== Sigma Designs platforms: http://files.dune-hd.com/partners/sdk/misc/dune_service_telnetd.dsf Android platforms: http://files.dune-hd.com/partners/sdk/misc/dune_service_android_telnetd.dsf Put this file onto a USB flash drive and "launch" it on the STB (go to "Sources" menu, navigate into the USB flash drive, choose the file, and press "ENTER" RC button). This will enable Telnet service on the STB (kept till reboot). Android platforms, r22+ firmare versions: You can also go to Setup - Information, then press UP, RIGHT, DOWN, LEFT, ENTER. This will enable Telnet access, and will make it persistent. On some platforms, telnet work w/o login prompt, on some platforms, the following login and password should be used: login: root password: IP-address of the STB: see Setup / Information. To make Telent access persistent (automatically enable Telnet service on each STB boot): touch $FS_PREFIX/config/telnetd sync To remove automatic enabling of Telnet service: rm -f $FS_PREFIX/config/telnetd sync On non-ATV platforms, the standard telnet port 23 is used. On ATV platforms, non-standard port 10023 is used, so you need to use command like: telnet IP 10023 On ATV platforms, telnet access is limited, it works on behalf the user of "Dune HD Media Center" app and gives access only to the internals of this app, access to the entire OS is very limited. NOTE: When an STB is used to provide access to an operator's service, and the operator does not want end users to be able to enable telnet access to the STB (e.g. to ensure the requirements of content owners or DRM vendors, or to protect from STB software/firmware modification by end users), the operator's firmware version is typically configured to limit or completely disable the described above possibility to enable telnet access to the STB and the telnet service on the STB. NOTE: The use of telnet access to STB is not intended to be and should not be used by regular end users for the regular use an STB, it is intended to be used only by specially qualified persons who perform software development related works or, in some cases, technical support related works, in contact with and under guidance of engineers of STB vendor or an operator, based on official recommendations and instructions by the STB vendor, thus ensuring that all implementation details of the STB hardware and firmware (in particular, including all the STB security mechanisms) are fully and properly taken into account. The use of telnet access to an STB may potentially lead (immediately or later) to unexpected work of the STB or even impossibility to use the STB at all, and is performed at the risk of the person who does it. The use of telnet access to an STB means that the person who does it understands all of the above and accepts this risk. If any problem occurs as a direct or indirect consequence of the use of telnet access to the STB or any commands performed in the telnet session or any software installed or modified with the help of such commands, the standard way to recover the STB firmware or STB settings may or may not work, and a request to the technical support to restore the STB operation may or may not help, and the STB warranty may be void. The use of any other way (if available for a particular STB model and a particular firmware version) to execute any Linux commands in the Linux OS running on the STB (e.g. via DSF files, via PHP plugins, etc) or access any other debug/development interfaces of the STB (if available for a particular STB model and a firmware version), or exploit any bugs in the hardware or the software (if such a possibility is identified) lead to the same risk. SSH access to STB ================= For Android platforms, this is not supported. All SDK firmware versions (Sigma Designs platforms) include also a built-in SSH server, which is active by default, and support key-based authentication. For more information, please see here: http://files.dune-hd.com/partners/sdk/sdk.txt => "JS STB SDK" section => Download any js_stb_sdk-*.zip package => See js_stb_sdk-*/doc/README.txt => "SSH access to device" section Custom firmware builds can include built-in SSH server too. Standard public (retail) firmware versions do not include built-in SSH server. ADB access to STB ================= On Android platforms, you can enable ADB in the following ways: 1) Standard way (as on any Android device): in Setup / System menu, enable Developer settings, then enable USB debugging in Developer settings. After that, you can use ADB to connect to the STB via network. 2) Use the following DSF file: http://files.dune-hd.com/partners/sdk/misc/dune_service_adb_enable.dsf Using standard ADB push/pull commands, you can easily copy files between the PC and the memory of the STB. Some examples of ADB usage: adb connect dune-IP-address # Here, you may need to confirm ADB access on TV screen of the STB adb push file /tmp/file adb pull /tmp/file file adb shell NOTE: When an STB is used to provide access to an operator's service, and the operator does not want end users to be able to enable ADB access to the STB (e.g. to ensure the requirements of content owners or DRM vendors, or to protect from STB software/firmware modification by end users), the operator's firmware version is typically configured to limit or completely disable ADB access to the STB. Filesystem and partitions overview ================================== Sigma Designs platforms: / /bin/ /cdrom/ /dev/ /drivers/ /etc/ /fconfig/ /home/ /init* /lib/ /linuxrc -> bin/busybox* /opt/ /proc/ /root/ /sbin/ /sys/ /tmp/ /usr/ /var/ RAM disk. Initialized on Linux kernel boot (the initial image of the RAM disk is a part of the Linux kernel). Changes are lost on STB reboot. /tango -> /tango3/ /tango3/ /firmware -> /tango3/firmware/ /firmware_ext -> /tango3/firmware_ext/ Main read-only partition (CRAMFS file system) storing the main parts of the firmware. /app (Only for models based on SMP8674 and SMP8672.) Special "container" flash memory partition which stores "tango.cramfs" file containing the CRAMFS image of the "tango" filesystem. This CRAMFS image is loop-mounted to /tango. The /app partition is by default mounted as read-only. 87xx platforms: SQUASHFS filesystem and "tango.squashfs" file are used. /mnt/ /config -> /mnt/config/ Small read-write flash memory partition. Is used mostly to keep STB settings. Is always available. /flashdata/ "Flash Memory Storage" mount point, automatically mounted by Dune. A bigger flash memory read-write partition. May not be available depending on the STB model and STB settings/state. More info: http://dune-hd.com/firmware/usb_flash_drive/ 87xx platforms: /mnt, /config and /flashdata reside in the same big flash memory partitions (/flashdata -> /mnt/flashdata). /persistfs/ "System Storage" mount point, automatically mounted by Dune. May be absent. More info: http://dune-hd.com/firmware/usb_flash_drive/ /tmp/mnt/storage/* Local storages (USB flash drives etc), automatically mounted by Dune. When a new local storage is connected to Dune, it is automatically mounted by Dune (note, it may be required to navigate in the GUI into "Sources" menu in order to force Dune to detect and automatically mount the storage). /D/ "Main Storage" mount point, automatically created by Dune. May be absent. Just an alias to one of mount points in /tmp/mnt/storage. /tmp/mnt/network/*, /tmp/mnt/smb/* Network shares (NFS and SMB), automatically mounted by Dune. Dune mounts network shares when the user navigates into the share. Android platforms: The following directories are used the same way as on Sigma Design platforms: $FS_PREFIX/tmp # resides in RAM disk (or in /data on ATV platforms) $FS_PREFIX/config # resides in /data or in $FS_PREFIX $FS_PREFIX/flashdata # resides in /data or in $FS_PREFIX $FS_PREFIX/firmware # resides in /system or in $FS_PREFIX $FS_PREFIX/firmware_ext # resides in /system or in $FS_PREFIX The following directories are used the same way as in any Android device: /system # read-only partition, is rewritten on firmware updates /data # read-write parition, is kept on firmware updates Storing your files on STB ========================= Sigma Designs plaforms: If you need to keep your own files in the flash memory of Dune, put them into /mnt (/config) partition. This partition typically has about 2MB of available space. If more space is needed, consider using /flashdata (if available) or /persistfs (if available). You can also use any user-visible local storage device (/tmp/mnt/* or /D). Also you can mount a network share and use it. NOTE (IMPORTANT!): Do not forget to do "sync" after each write into flash memory partition or local storage device. Android platforms: You can use $FS_PREFIX/config, $FS_PREFIX/flashdata, /sdcard directories. Manually mounting NFS share =========================== On platforms with root access (non-ATV): mkdir /mntpoint mount -o nolock ip-address:/path/to/dir /mntpoint NOTE: You can also mount NFS/SMB shares via STB menu (go to "Sources", press "POP UP MENU" RC button, choose "Add network folder"). Access from network to STB storage devices ========================================== Sigma Designs platforms: All storage devices (USB HDD, USB flash drive, etc) attached to the STB are automatically exported by the STB to network via SMB and FTP protocols, and can be accessed from network via: FTP: ftp://stb-ip-address SMB: \\stb-ip-address Android platforms: FTP access is not available. SMB access is available the same way as for Sigma Designs platforms (ATV platforms: on non-standard port 10445). Boot process ============ Sigma Designs platforms: See here: http://files.dune-hd.com/partners/sdk/dune_boot_process.txt Running some code automatically on STB boot =========================================== Sigma Designs platforms: Create the folder /config/boot/ and put executable files (e.g. shell scripts) into it. All executable files in this folder will be automatically launched by Dune on boot. (More exactly, "root" process will launch these executables on boot, before starting "shell".) Be careful: if these executables hang up or do other bad things, STB may become unusable (and emergency settings reset may be needed). Terminating Dune GUI ==================== Sigma Designs platforms: 1. Put the STB into "software standby" mode by pressing "POWER" RC button. 2. Terminate Dune GUI processes: killall root shell NOTE: Killing just "shell" process is not enough, as "root" process will automatically restart it. Log files ========= Most log files are written into $FS_PREFIX/tmp/run/ and $FS_PREFIX/tmp/. Linux kernel log messages are written into /var/log/messages (and are also accessible via "dmesg"). Android platforms: Android logs are available in standard way via "logcat" command. Built-in HTTP server ==================== The "httpd" process (simple HTTP-server based on busybox) listens on TCP port 80 (11080 in ATV models) and serves incoming HTTP requests on this port. Relevant locations in the filesystem: $FS_PREFIX/tmp/www WWW root folder. Static files $FS_PREFIX/tmp/www/path/to/file are accessible as: http://stb-ip-address/path/to/file $FS_PREFIX/tmp/www/cgi-bin Folder where CGI executables can reside. Executable CGI files $FS_PREFIX/tmp/www/cgi-bin/path/to/cgi-exe are accessible as: http://stb-ip-address/cgi-bin/path/to/cgi-exe Sigma Designs platforms: By default, the HTTP server listens on all network interfaces of the STB and is accessible not only via the loopback interface (using 127.0.0.1 IP address) but also via real network interface (using real IP address (e.g. Ethernet) of the STB). If if it desired to limit the HTTP server to listen on the loopback interface only, it can be configured using a custom HTTP server configuration file (see below). During STB boot, the HTTP server is typically launched by "root" process (note, this may take place before the STB has IP address from DHCP server), using the command like: /usr/sbin/httpd -h $FS_PREFIX/tmp/www It is possible to customize the behavior of the HTTP server by creating "/etc/httpd.conf" configuration file. The file should be created before the HTTP server is launched (or HTTP server should be restarted after creating/changing the configuration file). An appropriate place where this can be done is a boot script in "/config/boot/" directory (see "Running some code automatically on STB boot" section) or "/firmware/config/start.sh" in the custom firmware, see "Custom boot script" section here: http://files.dune-hd.com/partners/sdk/dune_custom_firmware.txt See here for additional information about HTTP server and "httpd.conf" file format: http://files.dune-hd.com/partners/sdk/httpd.txt Modifying files in root filesystem ================================== Sigma Designs platforms: The root filesystem (/) is a RAM-disk, whose content is initialized on Linux kernel boot (the initial RAM-disk content is part of Linux kernel image). In runtime, it is possible to apply any modifications to the root filesystem. These modifications will be stored in RAM and will be lost on STB reboot. If you want to preserve the modifications on STB reboots, you can put the code which applies these modifications into autostart-on-boot scripts (see "Running some code automatically on STB boot". Modifying files in read-only /tango filesystem ============================================== Sigma Designs platforms: The /tango filesystem (aka /tango3, also contains /firmware and /firmware_ext folders) is read-only CRAMS filesystem. So it is impossible to directly modify its content. In runtime, it is possible to apply temporary modifications to this filesystem using "bind mount". Examples: 1) Replacing a single file: mount --bind /tmp/shell /firmware/bin/shell 2) Replacing the entire folder (with all subfolders and files): mount --bind /tmp/skin /firmware/skin 3) Adding modifications to certain folder: cp -a /firmware/config /tmp/firmware_config echo "..." >/tmp/firmware_config/some_file mount --bind /tmp/firmware_config /firmware/config These temporary modifications will be lost on STB reboot. If you want to preserve the modifications on STB reboots, you can put the code which applies these modifications into autostart-on-boot scripts (see "Running some code automatically on STB boot". NOTE: the presence of such "bind mounts" may prevent /tango from unmounting and thus lead to firmware upgrade function to fail with an error. If you are affected by this, either first remove all bind-mounts before performing firmware upgrade, or use "emergency firmware recovery" procedure (which works before /tango is mounted and before autostart-on-boot scripts are launched). Alternatively, you can build you own version of /tango CRAMFS filesystem and write it directly into the flash memory of the STB. For more information on this, see "Extracting and overwriting tango filesystem image" section. Extracting and overwriting tango filesystem image ================================================= Sigma Designs platforms: NOTE: This instruction correponds to 86xx platforms. To extract /tango CRAMFS filesystem image from the flash memory of the STB, perform the following commands on the STB: # For models based on SMP8670: dd if=/dev/nblock8 of=/path/to/tango.cramfs # For models based on SMP8674 and SMP8672: cp /app/tango.cramfs /path/to/tango.cramfs To modify the /tango CRAMFS filesystem image, perform the following commands on a Linux PC: mkdir /path/to/modified-tango-dir mount --loop /path/to/tango.cramfs /tmp-dir cp -a /tmp-dir/* /path/to/modified-tango-dir/ umount /tmp-dir # here, do modifications in modified-tango-dir mkfs.cramfs -b 4096 /path/to/modified-tango-dir /path/to/tango.cramfs To write the modified /tango CRAMFS filesystem image back to the flash memory of the STB, perform the following commands on the STB: fuser -mk -9 /tango umount /tango # For models based on SMP8670: dd if=/path/to/tango.cramfs of=/dev/nblock8 # For models based on SMP8674 and SMP8672: losetup -d /dev/loop0 mount -o remount,rw /app cp /path/to/tango.cramfs /app/tango.cramfs umount /app sync reboot Creating custom firmware images =============================== Sigma Designs platforms: See here: http://files.dune-hd.com/partners/sdk/dune_custom_firmware.txt General requirements for custom code running on STB =================================================== - Avoid writing to flash memory too often. It is generally recommended to ensure that writing to flash memory is performed not often than ~100 times per day in average in typical end user's scenarious. So, when needed, accumulates changes in RAM (or in RAM-disk) and delay writing them to flash memory till a suitable moment. - When writing to flash memory, avoid unneeded writes: if there are good chances that the new data is the same as the old data, first compare the new data and the old data and skip overwriting the old data if they are equal. - When writing to flash memory, try to do 'sync' as soon as possible, to ensure that changes are flushed into the hardware and will not be lost on STB reboot or power-off. - Note that writing to flash memory (or 'sync' command which flushes possibly previously buffered changes in flash memory) may cause short (up to 1 sec) freezes of various STB activities. In particular, playback may freeze. So, when possible, try to delay writes to flash memory to a suitable moment. E.g., when writing to flash memory is needed during playback, it may be reasonable to delay it till playback stop or channel switch. - Avoid writing too much data into /config (/mnt) partition. This partition is not big and it may contain other files besides of your files. - When writing into /config (/mnt) or /flashdata partitions, be prepared that there may not be enough space in these partitions (in case when there are other files stored there besides of your files). When possible, try to handle failed writes in as robust way as possible. - When customizing /tango, avoid putting too much extra stuff into /firmware folder, and use firmware_ext when reasonable. When preparing for firmware upgrade, the system copies the content of /firmware folder (but not /firmware_ext folder) into /tmp (RAM-disk), and if there is too much stuff there, the available RAM may be exhausted. So, everything that is not needed during firmware ugrade, is better to put into /firmware_ext rather than /firmware. - When running background processes and doing extra mounts (especially bind-mounts), take into account possible influence on firmware upgrade. During firmware upgrade, the system needs to unmount /tango partition; if this fails for some reason (e.g. some bind-mount is blocking this), the firmware upgrade will fail. - When running background processes, take into account that their CPU and RAM consumption may have influence on other processes running on the STB (especially if your background processes run concurrently with web browser, or concurrently with native Dune UI). When possible, try to minimize CPU and RAM consumption of background processes. When reasonable, start background processes using "nice" (e.g. "nice -n 19 /path/to/your/exe"). When reasonable, shutdown your processes when they are not needed. Debugging tools =============== Sigma Designs platforms: GDB and GBD server: http://files.dune-hd.com/partners/sdk/misc/gdb.867x http://files.dune-hd.com/partners/sdk/misc/gdbserver.867x http://files.dune-hd.com/partners/sdk/misc/gdbserver.864x strace: http://files.dune-hd.com/partners/sdk/misc/strace.86xx tcpdump: http://files.dune-hd.com/partners/sdk/misc/tcpdump.86xx.tar.gz DSF file for launching custom Linux code ======================================== The following DSF file allows to run an arbitrary executable/shell-script w/o the need to use telnet/SSH access to the STB, directly from the native Dune UI. http://files.dune-hd.com/partners/sdk/misc/dune_service_exec.dsf Put this file onto a USB flash drive and "launch" it on the STB (go to "Sources" menu, navigate into the USB flash drive, choose the file, and press "ENTER" RC button). It will check for "dune_exec" executable file in the same directory where the DSF file is located, and execute it. Launching executables from user storages on Android platforms ------------------------------------------------------------- Android platforms mount user storages with "noexec" mount flag (for security reasons), which means that executable files on these storages can not be directly executed. To fix this problem, for example, the following command can be used (should be executed under root user -- i.e. in telnet/ADB session on Realtek Android platforms, or in telnet session on Amlogic Android platforms): mount -o remount,exec `readlink /tmp/mnt/D` Or the following utility DSF file can be used (does the same): http://files.dune-hd.com/sdk/misc/dune_service_android_main_storage_allow_exec.dsf The effect is preserved till STB reboot.