New project: gst-dsp, with beagleboard demo image

It took me a lot more than I expected, but I finally managed to get the beagleboard booting and happy with the latest linux kernel (2.6.32-rc3), DSS2, and dsp-bridge driver.

And then I could run gst-dsp: a native GStreamer plug-in to access Texas Instruments’ DSP algorithms for OMAP3 platforms. Which marks the time for making a public release.

Here’s the video :)

The video playback is running on the beagleboard with gst-dsp and gst-omapfb (no X) with TI public DSP binaries; it’s a WVGA (800×480) MPEG-4 video. It’s not running as smoothly as I wanted; it seems the public binaries are a bit buggy, and there’s some problem with the dsp-bridge driver writing directly to the framebuffer, but at least it somewhat works :)

The video recording is done with an N900, official system (which uses gst-dsp ;) ), and the resulting video is ; 848×480 MPEG-4.

You can find the demo image for the beagleboard, along with instructions, here.

The kernel code is on gitorious; the important tag is v2.6.32-felipec1.

And gst-dsp is hosted on Google Code, although the git repository is actually on github.

The code wasn’t written from scratch, TI’s projects: tiopenmax and libbridge, helped a lot.

And of course many other people made the first release possible (see the shortlog at the end).

Enjoy ;)

Andriy Shevchenko (1):
      base: fix a crash on send codec data

Felipe Contreras (180):
      Initial commit
      Register dsp node
      Add README
      Fix and update copyrights
      Add ALLOCATE_HEAP and ALLOCATE_SN to dsp_bridge
      Add handy dsp_send_message
      dummy: use dsp_send_message
      Rename gstdsp.* to plugin.*
      Makefile: cleanup
      dummy: trivial clanups
      Add log utility
      Use log utility
      dmm_buffer: size_t improvements
      dmm_buffer: always unmap when freeing
      dmm_buffer: use getpagesize()
      dmm_buffer: alignment improvements
      dmm_buffer: add user_data field
      Add MPEG-4 video decoder
      README: update
      mp4vdec: trivial cleanup
      mp4vdec: send signal to output_loop
      mp4vdec: flush output buffers too
      mp4vdec: reset output port
      mp4vdec: extra check for null buffer
      mp4vdec: use atomic operations for status
      mp4vdec: use more atomic operations for status
      mp4vdec: send stop signal before
      mp4vdec: re-use comm buffers
      dmm_buffer: reorganize a bit
      dmm_buffer: add dmm_buffer_reserve
      dmm_buffer: allow to re-reserve memory
      dmm_buffer: allow re-mapping
      mp4vdec: trivial cleanup
      dmm_buffer: unmap before unreserving
      mp4vdec: re-use mappings for output buffers
      mp4vdec: convert flush condition to semaphore
      Remove cond.h
      Rename mp4vdec to vdec
      vdec: trivial cleanup
      vdec: trivial reorganization
      vdec: prepare for multiple algos
      vdec: move create_node to dsp_start
      vdec: start dsp node after getting the caps
      vdec: initial support for H.264
      vdec: add Juha to authors list
      README: update
      vdec: cleanup
      vdec: make dsp_thread static
      vdec: reorganize a bit
      New base class
      Add new video encoder
      base: handle more commands
      base: reorganize got_message a bit
      venc: improve jpeg args
      venc: send jpeg dynamic params
      base: cleanup setup_output_buffers
      base: remove unused buffer_count
      base: reorganize a bit
      base: add use_pad_alloc option
      base: free mapped buffers on dsp_stop()
      base: be more verbose on get_slot()
      README: update
      Makefile: check for missing symbols
      New utility gstdsp_register()
      base: detect dsp errors
      base: properly handle dsp errors
      base: post error in the bus
      base: extra check for status in outout_loop()
      base: free events array
      base: reinitialize state on NULL->READY
      base: use circular buffer for timestamps
      base: increase ts_array
      base: increase mapping cache
      dummy: reorganize map_buffer
      dummy: input buffers don't need alignment
      dummy: cleanup
      dummy: don't map buffers
      venc: increase framesize limit for jpeg
      base: add gstdsp_post_error()
      venc: allocate a buffer when framesize is unaligned
      base: decrease wait for events timeout
      base: more error messages
      base: re-initialize on READY->PAUSED
      base: don't panic on wrong status
      base: destroy node at the right time
      base: catch playback completed message
      base: possible memleak fixes
      vdec: send codec data for MPEG-4
      base: make map cache optional
      plugin: set more proper ranks
      vdec: add framerate workaround
      vdec: remove gstdsp_send_buffer()
      base: add create_node() vmethod
      base: add parsing facilities
      Add h263 parser
      parse: update framesize only when unset
      Random cleanups
      base: add support for stream params
      venc: add H.263
      venc: use h263 by default
      Reorganize encoders
      base: send codec data for all the codecs
      base: keep trying if parse func fails
      base: trivial cleanup
      Trivial cleanups
      log: don't display info level
      log: decrease log level for buffer allocs
      log: add pr_test
      base: rename array to cache
      base: rename 'buffer' to 'comm'
      base: event cleanup
      base: reorganize a bit
      base: assume output buffer is always there
      base: remove out_buffer, use port buffer
      base: store input buffer
      base: trivial cleanup
      base: flush ports on stop
      base: plug some possible leaks
      base: make map_buffer() more conservative
      base: trigger semaphore after buffer modifications
      base: re-use input buffer
      base: add port index field
      Add async queue
      base: allow multiple buffers
      base: allow child elements to configure the ports
      vdec: increase the number of buffers to 2
      venc: trivial fixes
      log: add missing include
      base: re-enable queues properly
      venc: decrease input buffer size
      base: wait for eos
      base: possible fix
      Initial MPEG-4 video encoder support
      gstdspvenc.h: preemptively add H.264 to the list
      base: add send_codec_data() helper
      vdec: use send_codec_data()
      vdec: extra checks
      Add skip hack
      Revert "venc: forcing mpeg4 I frame each i_frame_interval"
      venc: reorganize stream/dynamic params
      base: trivial cleanup
      base: properly set param virt addr
      Add param argument to buffer callbacks
      Add buffer argument to buffer callbacks
      base: add buffer recv_cb
      base: add check for end addr alignment
      vdec: fix extra unref for codec-data
      base: trivial cleanups
      Rename dmm_buffer_flush() to dmm_buffer_clean()
      base: fix memory read
      dmm_buffer: clean instead of flush
      dmm_buffer: add cache 'flush' function back
      Use more proper cache functions
      base: handle bad node termination
      base: make EOS alignment an option
      jpegenc: enable eos align
      venc: improve integer framerate calculation
      venc: fix bitrate calculation
      venc: cleanup bitrate calculation
      venc: remove jpeg from bitrate calculation
      venc: tweak bitrate calculation
      venc: trivial cleanups
      venc: add 'quality' field
      venc: calculate smaller buffer sizes
      Fix some static analysis warnings
      log: avoid pr_info when gst debugging is off
      base: remove use_map_cache
      Trivial cleanups
      Cleanup type registrations
      base: improve some compiler hints
      dmm-buffer: check cache flush size
      base: properly free node resources
      Create custom dsp_node_t
      dsp-bridge: store node heap ourselves
      dsp-bridge: store node msgbuf ourselves
      dsp-bridge: cleanup node_free
      base: copy buffers when appropriate
      base: remove unnecessary cache flushing
      venc: set rate-control to variable
      base: post critical error mesages to the bus

Hoseok Chang (1):
      venc: tune mp4v parms for better performance

Juha Alanen (5):
      vdec: set profile based on the frame size
      vdec: improve H.263 args
      vdec: initial support for WMV9
      venc: set profile correctly for H.263 and MPEG4
      venc: disable single scan output for JPEG encoder

Marco Ballesio (8):
      vdec: fix srcpad setup
      venc: rename mp4venc_stream_params
      venc: add mp4venc_out_stream_params
      base: use proper buffer length
      venc: forcing mpeg4 I frame each i_frame_interval
      venc: reordered mp4venc_args initialization
      venc: added bitrate computation formula
      venc: propagate keyframes properly

Mark Nauwelaerts (2):
      base: safer buffer allocation and freeing
      base: fix element ref leak

Miguel Verdu (2):
      venc: tune MPEG-4 parameters
      venc: tune MPEG-4 parameters for quality

René Stadler (3):
      base: fix thread leak
      base: advance timestamp pointer for empty output buffers
      base: don't use DSP flushing

Tim-Philipp Müller (1):
      base: unref unused output buffer when skipping output

31 Responses to New project: gst-dsp, with beagleboard demo image

  1. kluzter says:

    video removed by user :(

    no entiendo nada sólo que se grabó con un n900 which is good y que te rolaste con el beagleboard… which is good i guess :P

  2. FelipeC says:

    @kluzter yes, I removed it because I saw the quality dropped, it looked like crap, but it turns out there’s a bug and only the account owner sees that.

    Anyway, I updated the post.

  3. ceyusa says:

    gst-dsp makes me wonder if we’ll see the bridge-driver interface in other silicon… snapdragon perhaps?

  4. FelipeC says:

    @ceyusa hopefully not; the interface is quite ugly and requires many changes in order to make it optimal, and let’s not forget that the dsp-bridge driver has a long way to go before becoming upstream.

    The good thing about gst-dsp is that it’s easy to adapt to changes in the interface :)

  5. Pingback: ceyusa's status on Tuesday, 13-Oct-09 12:58:17 UTC - Identi.ca

  6. Pete says:

    Well done! Thank you very much for you efforts.

  7. Saturn2888 says:

    Congrats man! I’ve been looking at getting one of these myself.

  8. Ricardo says:

    Hi Felipe!

    I’m new to both Beagleboard and N900, but I’m really interested in using them for projects that requires video processing – and that’s how I found your post. :)

    You said that “The video recording is done with an N900″. Do you mean that you were playing Big Buck Bunny on a monitor and using the N900 built-in camera to record it? Or where you using some other way to grab the video with the N900?

  9. FelipeC says:

    @ricardo yes, that’s exactly what I meant: I was using the beagleboard to play Big Buck Bunny through DVI output to a big display, and the N900 built-in Carl Zeiss camera, and official software to record the video.

  10. Pingback: OMAP3 public DSP binaries now work « Felipe Contreras

  11. Tiago Maluta says:

    Hi Felipe,

    I tried your demo image for the Beagle Board but the kernel hangs up after mounting EXT3-fs partition (I’m using ‘p3′). The relevant part is pasted in http://pastebin.com/mcd66bc2

    I tried other variants on kernel cmdline (i.e: changing rootdelay to rootwait) but nothing works…

    Best regards.

  12. FelipeC says:

    @Tiago It seems to be related to USB, that’s all I can say. You should ask this question in the beagleboard and linux-omap mailing lists.

  13. Pingback: Theora on TI C64x+ DSP and OMAP3 « Passing on the Left

  14. Jorge says:

    Hi Felipe,
    I’ve copied the rootfs.tar.gz, the uImage to my SD card and I’ve follow the instruccions from the readme to set the bootargs params, but I can’t get any output. I’m using the DVI-D ouput. I’ve added this arguments omapfb.debug=y omapfb.test=y omapdss.debug=y and I can see the patterns in the display, but when I play a video the display remains unchanged. The minicom console is:

    Welcome to Buildroot
    uclibc login: root
    # gst-launch playbin2 uri=file:/opt/roma.avi
    Setting pipeline to PAUSED …
    Pipeline is PREROLLING …
    Pipeline is PREROLLED …
    Setting pipeline to PLAYING …
    New clock: GstSystemClock
    Caught interrupt — handling interrupt.
    Interrupt: Stopping pipeline …
    Execution ended after 15456176745 ns.
    Setting pipeline to PAUSED …
    Setting pipeline to READY …
    Setting pipeline to NULL …
    Freeing pipeline …

    Must I change the output some way?? Any hint is welcome!

  15. FelipeC says:

    @Jorge please use the gst-dsp mailing list for such questions:
    http://groups.google.com/group/gst-dsp

  16. DeeGee says:

    Seems you know OMAP3 quite well. I wonder if we have hope of getting dsp accelerated decoding of h.264 main and high profiles (= CABAC and B-frames)? At the moment N900 only decodes baseline h.264 which basically means you need to re-encode almost all h.264 video for it.

  17. Rene Sapiens says:

    good to see beagleboard working with dsp-bridge driver.

  18. yuchih says:

    Hi Felipe,
    I could boot use your prebuild uIamge.
    Then I tried to compile your kernel source and boot from my b6 beagleboard.
    But it freezes after uncompressing linux kernel done.

    I followed the steps below:
    git clone git://gitorious.org/~felipec/linux-omap/felipec.git
    cd felipec
    git checkout -b felipec v2.6.32-felipec1
    make ARCH=arm CROSS_COMPILE=arm-none-linux-gnueabi- omap3_beagle_defconfig
    make ARCH=arm CROSS_COMPILE=arm-none-linux-gnueabi- uImage

    then i copy this uImage to my SD card.
    Is there anything i miss?

  19. FelipeC says:

    @yuchih that should work =/

  20. Nick says:

    Hi, this is great. Got BigBuckBunny playing but had to use the videoresizer plugin as the standard 854×480 movie is horribly skewed. That makes for 80% CPU! Also, can’t get any 720p movies (even low bitrate) to play, is this a limitation of the codec in the DSP (I know it’s a trial codec), or the DSP itself?

  21. FelipeC says:

    @Nick it would be much easier to get support for the display controller to render that kind of resolution than using a resizer.

    Regarding 720p, the DSP is able to handle that since some companies sell such codecs, but not the ones that TI provides.

  22. Jorge says:

    Hi,
    Is there any problem with gitorious? I’m trying to download the kernel code (v2.6.32-felipec1) with no luck

    $ git clone http://git.gitorious.org/~felipec/linux-omap/felipec.git

  23. Jorge says:

    (continue)

    Getting index for pack c0ae718f38e434799efa7bb7be53884408a33c46
    Getting index for pack 50be56768a4c78b4cd0d0209aaad9953789871d1
    Getting pack 4e59121e038ad573b95d75062cc2c333ce228f68
    which contains 374576a8b6f865022c0fd1ca62396889b23d66dd
    got 167ebea3d5bf5e6446f86ec3ce4583d6b5b3c21d
    error: Unable to get pack file http://git.gitorious.org/~felipec/linux-omap/felipec.git/objects/pack/pack-4e59121e038ad573b95d75062cc2c333ce228f68.pack
    transfer closed with 49882611 bytes remaining to read
    error: Unable to find 374576a8b6f865022c0fd1ca62396889b23d66dd under http://git.gitorious.org/~felipec/linux-omap/felipec.git
    Cannot obtain needed object 374576a8b6f865022c0fd1ca62396889b23d66dd
    fatal: Fetch failed.

  24. Roberto says:

    I try your images but I have this trying play a video:

    # uname -a
    Linux uclibc 2.6.32-felipec1 #3 Sun Oct 11 20:26:59 EEST 2009 armv7l GNU/Linux
    # gst-launch /video.ts
    gst-launch: /lib/libc.so.6: version `GLIBC_2.9′ not found (required by /usr/lib)
    #

  25. Roberto says:

    I need to change sym links to glibc and now I have:
    # gst-launch
    CMEM Error: init: Failed to open /dev/cmem: ‘No such file or directory’

    In fact, I can’t see cmem on /proc/devices and no cmem module is avaliable. Do you have sometip? :-)

  26. Toni says:

    Thanks for your nice article :)

    I am thinking about buying a beagleboard todo h.264 video encoding (from cam streaming over usb-ethernet).

    Can you say something about encoding performance?

  27. FelipeC says:

    Well, on the N900 we are able to encode MPEG-4 WVGA video without problems.

    Not sure about H.264 encoding, I don’t even know if there are publicly available codecs for that.

  28. Thanks for post from Australia

    Jozefin

  29. Pingback: OMAP3530 DSP Howto ( Really works by DSPLINK) | Robot Boby

  30. Nokia N9 says:

    you need to purchase Nokia N9 its good and lunched yet last week

  31. Dhut says:

    Nice work Felipe.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 43 other followers