Building DVSDK 4 for Gumstix DM3730

<!-- /* Font Definitions */ @font-face {font-family:Times; panose-1:2 0 5 0 0 0 0 0 0 0; mso-font-charset:0; mso-generic-font-family:auto; mso-font-pitch:variable; mso-font-signature:3 0 0 0 1 0;} @font-face {font-family:"MS 明朝"; mso-font-charset:78; mso-generic-font-family:auto; mso-font-pitch:variable; mso-font-signature:1 134676480 16 0 131072 0;} @font-face {font-family:"Cambria Math"; panose-1:2 4 5 3 5 4 6 3 2 4; mso-font-charset:0; mso-generic-font-family:auto; mso-font-pitch:variable; mso-font-signature:3 0 0 0 1 0;} @font-face {font-family:Cambria; panose-1:2 4 5 3 5 4 6 3 2 4; mso-font-charset:0; mso-generic-font-family:auto; mso-font-pitch:variable; mso-font-signature:-536870145 1073743103 0 0 415 0;} /* Style Definitions */ p.MsoNormal, li.MsoNormal, div.MsoNormal {mso-style-unhide:no; mso-style-qformat:yes; mso-style-parent:""; margin:0in; margin-bottom:.0001pt; mso-pagination:widow-orphan; font-size:12.0pt; font-family:Cambria; mso-ascii-font-family:Cambria; mso-ascii-theme-font:minor-latin; mso-fareast-font-family:"MS 明朝"; mso-fareast-theme-font:minor-fareast; mso-hansi-font-family:Cambria; mso-hansi-theme-font:minor-latin; mso-bidi-font-family:"Times New Roman"; mso-bidi-theme-font:minor-bidi;} a:link, span.MsoHyperlink {mso-style-noshow:yes; mso-style-priority:99; color:blue; text-decoration:underline; text-underline:single;} a:visited, span.MsoHyperlinkFollowed {mso-style-noshow:yes; mso-style-priority:99; color:purple; mso-themecolor:followedhyperlink; text-decoration:underline; text-underline:single;} p {mso-style-noshow:yes; mso-style-priority:99; mso-margin-top-alt:auto; margin-right:0in; mso-margin-bottom-alt:auto; margin-left:0in; mso-pagination:widow-orphan; font-size:10.0pt; font-family:Times; mso-fareast-font-family:"MS 明朝"; mso-fareast-theme-font:minor-fareast; mso-bidi-font-family:"Times New Roman";} .MsoChpDefault {mso-style-type:export-only; mso-default-props:yes; font-family:Cambria; mso-ascii-font-family:Cambria; mso-ascii-theme-font:minor-latin; mso-fareast-font-family:"MS 明朝"; mso-fareast-theme-font:minor-fareast; mso-hansi-font-family:Cambria; mso-hansi-theme-font:minor-latin; mso-bidi-font-family:"Times New Roman"; mso-bidi-theme-font:minor-bidi;} @page WordSection1 {size:8.5in 11.0in; margin:1.0in 1.25in 1.0in 1.25in; mso-header-margin:.5in; mso-footer-margin:.5in; mso-paper-source:0;} div.WordSection1 {page:WordSection1;} --> The new STORM includes modules based on the DM3730 series ti chips which have the capability to stream video at up to 720p (versus the 720x576 limit of the 3530 based chips).  However you must use the SW based on the ti's newer DVSDK 4.0 in order to enable this capability.  This page goes through how to build and setup the Gumstix for HD video streaming using the ti DVSDK 4.0.

 

Required HW: DM3730 based board (IronSTORM, FireSTORM, WaterSTORM)


I used Linux kernel version 2.6.34 because it is the latest version supported by the 5MP e-con systems sensor I am using.

 

Downloading Binaries:

If you want to just download pre-compiled versions of the SW for the 2.6.34 kernel you can find them here:

http://subversion.assembla.com/svn/evaforge/trunk/binaries_zips/

If you have a working 2.6.34 kernel with gstreamer you can simply run:

sudo tar -C (root directory of your Rootfs) -xvjf tiGstreamerModules.tar.bz2

So for example if you are doing a network boot it would look like this:

sudo tar -C /exports/overo -xvjf tiGstreamerModules.tar.bz2

 Otherwise you can expand the console image:

sudo tar -C (root directory of your Rootfs) -xvjf dsp-console-image-overo-evaforge.tar.bz2
sudo tar -C (root directory of your Rootfs) -xf modules-2.6.34-r103-overo.tgz

And then expand the DM3730 specific gstreamer files and mentioned above.


It should be noted that the dsplinkk.ko is configured to use 0x90000000 to 0xA0000000

mem=80M@0x80000000 is known to work with this setup, obviously you can expand the memory if you don't have a camera driver to make room for.

 

 Building from DVSDK 4 Source:

To build from source first we need a root file system (rootfs) with gstreamer and all its plugins.  Scott provides an excellent tutorial on how to do that here:

http://jumpnowtek.com/index.php?option=com_content&view=article&id=81:gumstix-dsp-gstreamer&catid=35:gumstix&Itemid=67

His instructions will diverge somewhat from what we need for 720p streaming, but the takeaway is download his bitbake (dsp-console-image.bb and dsp-desktop-image.bb) recipes put them in your recipes/linux folder and do the magic he specifies to get the ti_cgt_c6000_6.1.17_setup_linux_x86.bin file and get the mdasums correct.

(My recipes/linux folder is here:

~/overo-oe/org.openembedded.dev/recipes/linux/

If you followed the gumstix environment setup for open embedded (here) instructions yours should be similar.

With that done run bitbake:

Console image:

bitbake dsp-console-image

Desktop image:

bitbake dsp-console-desktop

 

After a long time you will have a rootfs.  I recommend you start the download of DVSDK 4 as well at this point as well since it is 1.5GB and will take some time.  I used dvsdk_dm3730-evm_04_03_00_06_setuplinux(Version 4.03.  You can download the file from here:

http://www.ti.com/tool/linuxdvsdk-dm37x


Once downloaded install with the default directories as you run through the install wizard.  I was running Ubuntu 12.04 LTS and did not have any issues.


 Preparing DVSDK to build for gumstix

1.  Edit Rules.make

2.  Copy some missing .h files

3.  Change the dsplinkk base address (not explicitly required, but recommended)

4.  Only build a subset of the DVSDK

5.  Modify load_modules.sh

Examples of the modified files described below can be found here:

http://subversion.assembla.com/svn/evaforge/trunk/binaries_zips/DVSDK/

1. Editing Rules.make

The most important thing to change in Rules.make is the location of the gumstix linux kernel.  At line 58 you will find the parameter LINUXKERNEL_INSTALL_DIR

Be aware that you cannot use ~ for home directory in Rules.make, it will not resolve correctly.

You want to change it from this:

 LINUXKERNEL_INSTALL_DIR=$(DVSDK_INSTALL_DIR)/psp/linux-2.6.37-psp04.02.00.07.sdk

To something like this:

LINUXKERNEL_INSTALL_DIR=/home/btgalush/overo-oe/tmp/work/overo-angstrom-linux-gnueabi/linux-omap3-2.6.34-r103/staging-pkg/sysroots/overo-angstrom-linux-gnueabi/kernel

 The full path obviously will be different depending on your username and if you build a slightly different kernel.

Next up is the compiler, although I didn't have any issues using the ARM compiler that ships with DVSDK if you want to change the compiler you can do so by changing CSTOOL_PREFIX, and CSTOO_PATH

 CSTOOL_PREFIX=/home/btgalush/overo-oe/tmp/sysroots/i686-linux/usr/armv7a/bin/arm-angstrom-linux-gnueabi-
CSTOOL_PATH=/home/btgalush/overo-oe/tmp/sysroots/i686-linux/usr/armv7a/bin/

 Once again your paths will vary, but should be similar.

2. Copy Some missing .h files

The gumstix linux kernel does not include 3 .h files that the DVSDK needs to compile, I was able to steal them from the shipped 2.6.37 kernel and get things to work without issue.  The 3 files you need are:

media.h
v4l2-mediabus.h
v4l2-subdev.h

You can find these files here:

$(DVSDK_INSTALL_DIR)/psp/linux-2.6.37-psp04.02.00.07.sdk/include/linux/

You need to copy them into the gumstix kernel root/include/linux/

So for the example above that would be:

/home/btgalush/overo-oe/tmp/work/overo-angstrom-linux-gnueabi/linux-omap3-2.6.34-r103/staging-pkg/sysroots/overo-angstrom-linux-gnueabi/kernel/include/linux/

3. Change the dsplinkk base address

The gstreamer plugin for the TI chip utilizes the DSP on the DM3720 for the video encoding/decoding.  Since both the ARM and the DSP share the same memory they must agree on who can use which memory regions and where to exchange the video data.  By default the DVSDK Makefile chooses a base address of 0x83700000 which is not ideal for the gumstix.  You can make it work, but it is much easier to just change the DSP memory region.  I recommend changing it to 0x90000000.  This allows you to use the first 128MB for kernel v4l2 drivers and the linux kernel itself, while the second 128MB is dedicated to the DSP.

Edit Makefile, and search for DSP_REGION_BASE_ADDR which should be on line 809 in the C6run section:

################################################################################
# Build C6run
################################################################################
c6run:
    $(MAKE) -C $(C6RUN_INSTALL_DIR) \
    CODEGEN_INSTALL_DIR="$(CODEGEN_INSTALL_DIR)" \
    BIOS_INSTALL_DIR="$(BIOS_INSTALL_DIR)" \
    XDC_INSTALL_DIR="$(XDC_INSTALL_DIR)" \
    LINK_INSTALL_DIR="$(LINK_INSTALL_DIR)" \
    CMEM_INSTALL_DIR="$(CMEM_INSTALL_DIR)" \
    LPM_INSTALL_DIR="$(LPM_INSTALL_DIR)" \
    ARM_TOOLCHAIN_PATH="$(CSTOOL_DIR)" \
    ARM_TOOLCHAIN_PREFIX="arm-arago-linux-gnueabi-" \
    DSP_REGION_BASE_ADDR=0x83700000 \
    DSP_REGION_CMEM_SIZE=0x02200000 \
    DSP_REGION_CODE_SIZE=0x02700000 \
    dm3730_config

 Change

DSP_REGION_BASE_ADDR=0x83700000 \

To:

DSP_REGION_BASE_ADDR=0x90000000 \

 You can change the other memory regions as well, but they are already sized to fix the max 720p resolution the ti codecs can handle.

4.  Only build a subset of the DVSDK

There are numerous components in the DVSDK, and we don't need all of them to encode/decode video.  Typing:

make help

Will give you a nice printout of all the components and make commands available in the DVSDK.  The subset we are interested in is:

cmem sdma dsplink lpm c6run c6accel codecs dmai

The easiest way to build these is to edit the Makefile and on line 24 where you see:

#==============================================================================
# Build target build necessary components to enable all other build targets.
#==============================================================================
components: linux cmem sdma dsplink lpm c6run c6accel codecs dmai  

Add just below it it to:

gumstix_components: cmem sdma dsplink lpm c6run c6accel codecs dmai gstreamer_ti

 Also change the install accordingly so add:

#==============================================================================
# Install everything
#==============================================================================
install: linux_install cmem_install sdma_install opengldemos_install ceexamples_install codecs_install lpm_install dsplink_install u-boot_install dmai_install demos_install  psp_examples_install c6accel_install xload_install c6run_install matrix_install gstreamer_ti_install loadmodule_install dvtb_install

gumstix_install: cmem_install sdma_install codecs_install lpm_install dsplink_install dmai_install c6accel_install c6run_install gstreamer_ti_install loadmodule_install

Then type:

make gumstix_components
make gumstix_install

The first command will take awhile, the second should be quick.  You may have to add some utilities to your host machine linux kernel, I had to add bison, flex, and a newer version of GLIB (sudo apt-get install libglib2.0-dev).  This will really depend on your dev machines configuration.

You should now have all the needed add on files in ~/targetfs (assumes you used default install location).  If you put the files in a tarball you can expand it in the root directory of your filesystem and all the files will go to the right place.

So run something like:

~/targetfs/tar -jcvf tiGstreamerModules.tar.bz2 .

 Copy the .tar.bz2 to the root directory of your file system and run something like:

sudo tar -C /exports/overo -xvjf tiGstreamerModules.tar.bz2

5.  Modify load_modules.sh

In Scott's gstreamer tutorial he talks about the load_modules.sh which is what installs all the dsp kernel modules and correctly sets up the CMEM for video streaming.  The DVSDK build will auto generate a nice framework for load_modules.sh which can be found here:

/usr/share/ti/c6run-apps/loadmodules.sh

However it does not correctly setup the pools for streaming 720p and below video.  So we need to change this line:

CMEMK_OPTS="phys_start=$DSP_REGION_START_ADDR phys_end=$DSP_REGION_END_ADDR allowOverlap=1"

To something like this:

CMEMK_OPTS="phys_start=$DSP_REGION_START_ADDR phys_end=$DSP_REGION_END_ADDR allowOverlap=1 pools=1x10485760,6x2097152,1x691200,1x1048576,1x1" 

The pool sizes are based on Scott's pool sizes and scaled up to 720p.  I know they work, I don't make any claims that they are the ideal sizes.
If you are using Scott's example you will also want to copy the loadmodules.sh file to:

/usr/share/ti/gst/omap3530 

Where it is called during boot.

Give it a whirl

Grab a pipeline from the ti examples which can be found in the TMS320DM3730_Software_Developers_Guide.pdf which is in the docs directory of the DVSDK install.

Here is an example pulled from there:

gst-launch videotestsrc num-buffers=100 ! \
'video/x-raw-yuv,width=1280,height=720,format=(fourcc)UYVY' ! TIVidenc1 codecName=h264enc \
engineName=codecServer ! filesink location=sample.264

If you want to stream it to another computer try something like this:

gst-launch -v videotestsrc always-copy=false ! 'video/x-raw-yuv,width=1280,height=720,format=(fourcc)UYVY',framerate=30/1 ! TIVidenc1 codecName=h264enc engineName=codecServer ! rtph264pay pt=96 ! multiudpsink clients=\"192.168.33.3:4000\"

Making sure to change the ip address to the ip address of the destination computer.