New project: gst-dsp, with beagleboard demo image

It took me a lot more than I expected, but I finally managed to get the beagleboard booting and happy with the latest linux kernel (2.6.32-rc3), DSS2, and dsp-bridge driver.

And then I could run gst-dsp: a native GStreamer plug-in to access Texas Instruments’ DSP algorithms for OMAP3 platforms. Which marks the time for making a public release.

Here’s the video 🙂

The video playback is running on the beagleboard with gst-dsp and gst-omapfb (no X) with TI public DSP binaries; it’s a WVGA (800×480) MPEG-4 video. It’s not running as smoothly as I wanted; it seems the public binaries are a bit buggy, and there’s some problem with the dsp-bridge driver writing directly to the framebuffer, but at least it somewhat works 🙂

The video recording is done with an N900, official system (which uses gst-dsp ;)), and the resulting video is ; 848×480 MPEG-4.

You can find the demo image for the beagleboard, along with instructions, here.

The kernel code is on gitorious; the important tag is v2.6.32-felipec1.

And gst-dsp is hosted on Google Code, although the git repository is actually on github.

The code wasn’t written from scratch, TI’s projects: tiopenmax and libbridge, helped a lot.

And of course many other people made the first release possible (see the shortlog at the end).

Enjoy 😉

Andriy Shevchenko (1):
      base: fix a crash on send codec data

Felipe Contreras (180):
      Initial commit
      Register dsp node
      Add README
      Fix and update copyrights
      Add ALLOCATE_HEAP and ALLOCATE_SN to dsp_bridge
      Add handy dsp_send_message
      dummy: use dsp_send_message
      Rename gstdsp.* to plugin.*
      Makefile: cleanup
      dummy: trivial clanups
      Add log utility
      Use log utility
      dmm_buffer: size_t improvements
      dmm_buffer: always unmap when freeing
      dmm_buffer: use getpagesize()
      dmm_buffer: alignment improvements
      dmm_buffer: add user_data field
      Add MPEG-4 video decoder
      README: update
      mp4vdec: trivial cleanup
      mp4vdec: send signal to output_loop
      mp4vdec: flush output buffers too
      mp4vdec: reset output port
      mp4vdec: extra check for null buffer
      mp4vdec: use atomic operations for status
      mp4vdec: use more atomic operations for status
      mp4vdec: send stop signal before
      mp4vdec: re-use comm buffers
      dmm_buffer: reorganize a bit
      dmm_buffer: add dmm_buffer_reserve
      dmm_buffer: allow to re-reserve memory
      dmm_buffer: allow re-mapping
      mp4vdec: trivial cleanup
      dmm_buffer: unmap before unreserving
      mp4vdec: re-use mappings for output buffers
      mp4vdec: convert flush condition to semaphore
      Remove cond.h
      Rename mp4vdec to vdec
      vdec: trivial cleanup
      vdec: trivial reorganization
      vdec: prepare for multiple algos
      vdec: move create_node to dsp_start
      vdec: start dsp node after getting the caps
      vdec: initial support for H.264
      vdec: add Juha to authors list
      README: update
      vdec: cleanup
      vdec: make dsp_thread static
      vdec: reorganize a bit
      New base class
      Add new video encoder
      base: handle more commands
      base: reorganize got_message a bit
      venc: improve jpeg args
      venc: send jpeg dynamic params
      base: cleanup setup_output_buffers
      base: remove unused buffer_count
      base: reorganize a bit
      base: add use_pad_alloc option
      base: free mapped buffers on dsp_stop()
      base: be more verbose on get_slot()
      README: update
      Makefile: check for missing symbols
      New utility gstdsp_register()
      base: detect dsp errors
      base: properly handle dsp errors
      base: post error in the bus
      base: extra check for status in outout_loop()
      base: free events array
      base: reinitialize state on NULL->READY
      base: use circular buffer for timestamps
      base: increase ts_array
      base: increase mapping cache
      dummy: reorganize map_buffer
      dummy: input buffers don't need alignment
      dummy: cleanup
      dummy: don't map buffers
      venc: increase framesize limit for jpeg
      base: add gstdsp_post_error()
      venc: allocate a buffer when framesize is unaligned
      base: decrease wait for events timeout
      base: more error messages
      base: re-initialize on READY->PAUSED
      base: don't panic on wrong status
      base: destroy node at the right time
      base: catch playback completed message
      base: possible memleak fixes
      vdec: send codec data for MPEG-4
      base: make map cache optional
      plugin: set more proper ranks
      vdec: add framerate workaround
      vdec: remove gstdsp_send_buffer()
      base: add create_node() vmethod
      base: add parsing facilities
      Add h263 parser
      parse: update framesize only when unset
      Random cleanups
      base: add support for stream params
      venc: add H.263
      venc: use h263 by default
      Reorganize encoders
      base: send codec data for all the codecs
      base: keep trying if parse func fails
      base: trivial cleanup
      Trivial cleanups
      log: don't display info level
      log: decrease log level for buffer allocs
      log: add pr_test
      base: rename array to cache
      base: rename 'buffer' to 'comm'
      base: event cleanup
      base: reorganize a bit
      base: assume output buffer is always there
      base: remove out_buffer, use port buffer
      base: store input buffer
      base: trivial cleanup
      base: flush ports on stop
      base: plug some possible leaks
      base: make map_buffer() more conservative
      base: trigger semaphore after buffer modifications
      base: re-use input buffer
      base: add port index field
      Add async queue
      base: allow multiple buffers
      base: allow child elements to configure the ports
      vdec: increase the number of buffers to 2
      venc: trivial fixes
      log: add missing include
      base: re-enable queues properly
      venc: decrease input buffer size
      base: wait for eos
      base: possible fix
      Initial MPEG-4 video encoder support
      gstdspvenc.h: preemptively add H.264 to the list
      base: add send_codec_data() helper
      vdec: use send_codec_data()
      vdec: extra checks
      Add skip hack
      Revert "venc: forcing mpeg4 I frame each i_frame_interval"
      venc: reorganize stream/dynamic params
      base: trivial cleanup
      base: properly set param virt addr
      Add param argument to buffer callbacks
      Add buffer argument to buffer callbacks
      base: add buffer recv_cb
      base: add check for end addr alignment
      vdec: fix extra unref for codec-data
      base: trivial cleanups
      Rename dmm_buffer_flush() to dmm_buffer_clean()
      base: fix memory read
      dmm_buffer: clean instead of flush
      dmm_buffer: add cache 'flush' function back
      Use more proper cache functions
      base: handle bad node termination
      base: make EOS alignment an option
      jpegenc: enable eos align
      venc: improve integer framerate calculation
      venc: fix bitrate calculation
      venc: cleanup bitrate calculation
      venc: remove jpeg from bitrate calculation
      venc: tweak bitrate calculation
      venc: trivial cleanups
      venc: add 'quality' field
      venc: calculate smaller buffer sizes
      Fix some static analysis warnings
      log: avoid pr_info when gst debugging is off
      base: remove use_map_cache
      Trivial cleanups
      Cleanup type registrations
      base: improve some compiler hints
      dmm-buffer: check cache flush size
      base: properly free node resources
      Create custom dsp_node_t
      dsp-bridge: store node heap ourselves
      dsp-bridge: store node msgbuf ourselves
      dsp-bridge: cleanup node_free
      base: copy buffers when appropriate
      base: remove unnecessary cache flushing
      venc: set rate-control to variable
      base: post critical error mesages to the bus

Hoseok Chang (1):
      venc: tune mp4v parms for better performance

Juha Alanen (5):
      vdec: set profile based on the frame size
      vdec: improve H.263 args
      vdec: initial support for WMV9
      venc: set profile correctly for H.263 and MPEG4
      venc: disable single scan output for JPEG encoder

Marco Ballesio (8):
      vdec: fix srcpad setup
      venc: rename mp4venc_stream_params
      venc: add mp4venc_out_stream_params
      base: use proper buffer length
      venc: forcing mpeg4 I frame each i_frame_interval
      venc: reordered mp4venc_args initialization
      venc: added bitrate computation formula
      venc: propagate keyframes properly

Mark Nauwelaerts (2):
      base: safer buffer allocation and freeing
      base: fix element ref leak

Miguel Verdu (2):
      venc: tune MPEG-4 parameters
      venc: tune MPEG-4 parameters for quality

René Stadler (3):
      base: fix thread leak
      base: advance timestamp pointer for empty output buffers
      base: don't use DSP flushing

Tim-Philipp Müller (1):
      base: unref unused output buffer when skipping output

31 thoughts on “New project: gst-dsp, with beagleboard demo image

  1. video removed by user 😦

    no entiendo nada sólo que se grabó con un n900 which is good y que te rolaste con el beagleboard… which is good i guess 😛

    Like

  2. @kluzter yes, I removed it because I saw the quality dropped, it looked like crap, but it turns out there’s a bug and only the account owner sees that.

    Anyway, I updated the post.

    Like

  3. @ceyusa hopefully not; the interface is quite ugly and requires many changes in order to make it optimal, and let’s not forget that the dsp-bridge driver has a long way to go before becoming upstream.

    The good thing about gst-dsp is that it’s easy to adapt to changes in the interface 🙂

    Like

  4. Pingback: ceyusa's status on Tuesday, 13-Oct-09 12:58:17 UTC - Identi.ca

  5. Hi Felipe!

    I’m new to both Beagleboard and N900, but I’m really interested in using them for projects that requires video processing – and that’s how I found your post. 🙂

    You said that “The video recording is done with an N900”. Do you mean that you were playing Big Buck Bunny on a monitor and using the N900 built-in camera to record it? Or where you using some other way to grab the video with the N900?

    Like

  6. @ricardo yes, that’s exactly what I meant: I was using the beagleboard to play Big Buck Bunny through DVI output to a big display, and the N900 built-in Carl Zeiss camera, and official software to record the video.

    Like

  7. Pingback: OMAP3 public DSP binaries now work « Felipe Contreras

  8. Hi Felipe,

    I tried your demo image for the Beagle Board but the kernel hangs up after mounting EXT3-fs partition (I’m using ‘p3’). The relevant part is pasted in http://pastebin.com/mcd66bc2

    I tried other variants on kernel cmdline (i.e: changing rootdelay to rootwait) but nothing works…

    Best regards.

    Like

  9. Pingback: Theora on TI C64x+ DSP and OMAP3 « Passing on the Left

  10. Hi Felipe,
    I’ve copied the rootfs.tar.gz, the uImage to my SD card and I’ve follow the instruccions from the readme to set the bootargs params, but I can’t get any output. I’m using the DVI-D ouput. I’ve added this arguments omapfb.debug=y omapfb.test=y omapdss.debug=y and I can see the patterns in the display, but when I play a video the display remains unchanged. The minicom console is:

    Welcome to Buildroot
    uclibc login: root
    # gst-launch playbin2 uri=file:/opt/roma.avi
    Setting pipeline to PAUSED …
    Pipeline is PREROLLING …
    Pipeline is PREROLLED …
    Setting pipeline to PLAYING …
    New clock: GstSystemClock
    Caught interrupt — handling interrupt.
    Interrupt: Stopping pipeline …
    Execution ended after 15456176745 ns.
    Setting pipeline to PAUSED …
    Setting pipeline to READY …
    Setting pipeline to NULL …
    Freeing pipeline …

    Must I change the output some way?? Any hint is welcome!

    Like

  11. Seems you know OMAP3 quite well. I wonder if we have hope of getting dsp accelerated decoding of h.264 main and high profiles (= CABAC and B-frames)? At the moment N900 only decodes baseline h.264 which basically means you need to re-encode almost all h.264 video for it.

    Like

  12. Hi Felipe,
    I could boot use your prebuild uIamge.
    Then I tried to compile your kernel source and boot from my b6 beagleboard.
    But it freezes after uncompressing linux kernel done.

    I followed the steps below:
    git clone git://gitorious.org/~felipec/linux-omap/felipec.git
    cd felipec
    git checkout -b felipec v2.6.32-felipec1
    make ARCH=arm CROSS_COMPILE=arm-none-linux-gnueabi- omap3_beagle_defconfig
    make ARCH=arm CROSS_COMPILE=arm-none-linux-gnueabi- uImage

    then i copy this uImage to my SD card.
    Is there anything i miss?

    Like

  13. Hi, this is great. Got BigBuckBunny playing but had to use the videoresizer plugin as the standard 854×480 movie is horribly skewed. That makes for 80% CPU! Also, can’t get any 720p movies (even low bitrate) to play, is this a limitation of the codec in the DSP (I know it’s a trial codec), or the DSP itself?

    Like

  14. @Nick it would be much easier to get support for the display controller to render that kind of resolution than using a resizer.

    Regarding 720p, the DSP is able to handle that since some companies sell such codecs, but not the ones that TI provides.

    Like

  15. (continue)

    Getting index for pack c0ae718f38e434799efa7bb7be53884408a33c46
    Getting index for pack 50be56768a4c78b4cd0d0209aaad9953789871d1
    Getting pack 4e59121e038ad573b95d75062cc2c333ce228f68
    which contains 374576a8b6f865022c0fd1ca62396889b23d66dd
    got 167ebea3d5bf5e6446f86ec3ce4583d6b5b3c21d
    error: Unable to get pack file http://git.gitorious.org/~felipec/linux-omap/felipec.git/objects/pack/pack-4e59121e038ad573b95d75062cc2c333ce228f68.pack
    transfer closed with 49882611 bytes remaining to read
    error: Unable to find 374576a8b6f865022c0fd1ca62396889b23d66dd under http://git.gitorious.org/~felipec/linux-omap/felipec.git
    Cannot obtain needed object 374576a8b6f865022c0fd1ca62396889b23d66dd
    fatal: Fetch failed.

    Like

  16. I try your images but I have this trying play a video:

    # uname -a
    Linux uclibc 2.6.32-felipec1 #3 Sun Oct 11 20:26:59 EEST 2009 armv7l GNU/Linux
    # gst-launch /video.ts
    gst-launch: /lib/libc.so.6: version `GLIBC_2.9′ not found (required by /usr/lib)
    #

    Like

  17. I need to change sym links to glibc and now I have:
    # gst-launch
    CMEM Error: init: Failed to open /dev/cmem: ‘No such file or directory’

    In fact, I can’t see cmem on /proc/devices and no cmem module is avaliable. Do you have sometip? 🙂

    Like

  18. Thanks for your nice article 🙂

    I am thinking about buying a beagleboard todo h.264 video encoding (from cam streaming over usb-ethernet).

    Can you say something about encoding performance?

    Like

  19. Well, on the N900 we are able to encode MPEG-4 WVGA video without problems.

    Not sure about H.264 encoding, I don’t even know if there are publicly available codecs for that.

    Like

  20. Pingback: OMAP3530 DSP Howto ( Really works by DSPLINK) | Robot Boby

Leave a comment

This site uses Akismet to reduce spam. Learn how your comment data is processed.