Input Video to Snowmix

Updated for Snowmix 0.5.0

Snowmix takes video as input from video feeds through shared memory. Snowmix video feeds has implemented the GStreamer module shmsrc and can as such receive video from the GStreamer module shmsink.

For audio input to Snowmix, please see the detailed Snowmix Audio Guide listed on the Snowmix Guides page.

The GStreamer module shmsink is responsible for allocating a shared memory area, communicating the filename representing the shared memory area to Snowmix through a communication pipe, feeding video frames into the shared memory and signalling the receiver that frames are ready to be read. Snowmix will then use the frames and signal back to the shmsink when a frame has been used enabling the module to reuse the shared memory area for new frames.

The format of the frames the shmsink sends into Snowmix must be in the ARGB format or rather the BGRA format on little endian CPU systems such as the x86 CPU architecture. This means that each pixel is sent to Snowmix in the order Blue, Green, Red and Alpha.

The image below shows 3 different types of video input.



The name of the modules on the drawing above is from GStreamer 0.10. In GStreamer 1.x the module ffmpegcolorspace is names videoconvert and the decodebin2 module is named decodebin. Finally, the RGBA, which is actually BGRA, is not a module at all. Yeahh, we need a new more updated and correct image.

The GStreamer module needs a control channel to communicate with Snowmix. This control channel is a named pipe (a fifo file) and both GStreamer and Snowmix needs to know the name of this pipe for this to work. This page will show and example of GStreamer feeding a video stream to snowmix.

In the file $SNOWMIX/scripts/input2feed, you can find a full example of a GStreamer pipeline that can feed either just video or both audio and video to Snowmix. The pipeline is also automatically configured reading its data configuration from a running Snowmix session.

Top

GStreamer pipeline example: Input from File


Below is shown an example for inputting video to Snowmix using GStreamer version 1.0 tested with GStreamer version 1.0.0 through 1.5.2 as well as 0.10. However GStreamer version 0.10 is no longer officially supported.

#!/bin/bash # These settings are the settings defined for video feed 1 in Snowmix CONTROL_PIPE=/tmp/feed1-control-pipe width=768 height=576 framerate='24/1' which gst-launch-1.0 2>/dev/null 1>&2 if [ $? -eq 0 ] ; then gstlaunch=gst-launch-1.0 DECODEBIN=decodebin SHMSIZE='shm-size='`echo "$width * $height * 4 * 22"|bc` MIXERFORMAT='video/x-raw, format=BGRA, pixel-aspect-ratio=1/1, interlace-mode=progressive' SCALENRATE='videoconvert ! videorate ! videoscale ! videoconvert' else gstlaunch=gst-launch-0.10 DECODEBIN=decodebin2 SHMSIZE='shm-size='`echo "$width * $height * 4 * 8"|bc` MIXERFORMAT='video/x-raw-rgb,bpp=32,depth=32,endianness=4321,red_mask=65280,green_mask=16711680' MIXERFORMAT=$MIXERFORMAT',blue_mask=-16777216,pixel-aspect-ratio=1/1,interlaced=false' SCALENRATE='ffmpegcolorspace ! videorate ! videoscale ! ffmpegcolorspace' fi SRC="filesrc location=$SNOWMIX/test/LES_TDS_launch.mp4 ! $DECODEBIN " SHMOPTION="wait-for-connection=0 sync=true" SHMSINK1="shmsink socket-path=$CONTROL_PIPE $SHMSIZE $SHMOPTION" while true ; do # Remove the named pipe if it exist rm -f $CONTROL_PIPE $gstlaunch -v \ $SRC !\ $SCALENRATE !\ "$MIXERFORMAT,framerate=$framerate, width=$width, height=$height" !\ $SHMSINK1 sleep 2 done exit

In this example 'filesrc' reads from the file $SNOWMIX/test/LES_TDS.mp4, decodes it, scale it to 704x576 and converts it to 32 bit BGRA before writing it to shared memory and signaling to Snowmix via the named pipe '/tmp/feed1-control-pipe'. The example will read, decode and send it to Snowmix repeatedly with a 2 second pause between until interrupted. The frame rate is set to 24fps. The videorate element is not strictly necessary as Snowmix will insert extra or drop frames automatically upon need.

Top

Snowmix Configuration


For Snowmix to receive the video feed from GStreamer configured as shown above, the following entries need to be added to snowmix either interactively or to the ini file Snowmix reads upon starting. The entries are:

system control port 9999 system geometry 1024 576 BGRA system frame rate 24 system socket /tmp/mixer1 feed add 1 Camera 1 feed geometry 1 704 576 feed live 1 feed idle 1 100 frames/dead-704x576.bgra feed socket 1 /tmp/feed1-control-pipe

These entries first add feed number 1 and name it 'Camera 1'. Then the geometry of the feed is defined. Then the feed is defined as live. Last but not least, Snowmix is informed that the name of the named pipe used as control channel is /tmp/feed1-control-pipe. The 'feed idle' command optionally tells Snowmix to use a frame image (found in the $SNOWMIX/frames/dead-704x576.bgra) as replacement for the live feed, should the feed be missing frames for 100 frame periods. You can generate your own frame files (raw BGRA data) and place it typically in $HOME/.snowmix/frames for Linux and $HOME/Snowmix/frames for Mac OS X.

Top

GStreamer Input From a Web Camera


In this example we take live input from a web camera using V4L2 (Video for Linux).

#!/bin/bash CONTROL_PIPE=/tmp/feed1-control-pipe width=704 height=576 framerate='25/1' which gst-launch-1.0 2>/dev/null 1>&2 if [ $? -eq 0 ] ; then gstlaunch=gst-launch-1.0 SHMSIZE='shm-size='`echo "$width * $height * 4 * 22"|bc` MIXERFORMAT='video/x-raw, format=BGRA, pixel-aspect-ratio=1/1, interlace-mode=progressive' SCALE='videoconvert ! videoscale ! videoconvert' else gstlaunch=gst-launch-0.10 SHMSIZE='shm-size='`echo "$width * $height * 4 * 8"|bc` MIXERFORMAT='video/x-raw-rgb,bpp=32,depth=32,endianness=4321,red_mask=65280,green_mask=16711680' MIXERFORMAT=$MIXERFORMAT',blue_mask=-16777216,pixel-aspect-ratio=1/1,interlaced=false' SCALE='ffmpegcolorspace ! videoscale ! ffmpegcolorspace' fi SRC='v4l2src device=/dev/video0 ' SHMOPTION="wait-for-connection=0 sync=true" SHMSINK1="shmsink socket-path=$CONTROL_PIPE $SHMSIZE $SHMOPTION" while true ; do # Remove the named pipe if it exist rm -f $CONTROL_PIPE $gstlaunch -v \ $SRC !\ $SCALE !\ "$MIXERFORMAT,width=$width, height=$height, framerate=$framerate" !\ $SHMSINK1 sleep 2 done exit
Top

GStreamer Input From a MPEG-4 UDP Stream


The example below uses a MPEG-4 stream sent to a UDP port on the computer from an external IP based web camera.

#!/bin/bash # These settings are from the Snowmix ini file for the feed. CONTROL_PIPE=/tmp/feed1-control-pipe width=704 height=576 framerate='25/1' which gst-launch-1.0 2>/dev/null 1>&2 if [ $? -eq 0 ] ; then gstlaunch=gst-launch-1.0 SHMSIZE='shm-size='`echo "$width * $height * 4 * 22"|bc` MIXERFORMAT='video/x-raw, format=BGRA, pixel-aspect-ratio=1/1, interlace-mode=progressive' SCALENRATE='videoconvert ! videorate ! videoscale ! videoconvert' else gstlaunch=gst-launch-0.10 SHMSIZE='shm-size='`echo "$width * $height * 4 * 8"|bc` MIXERFORMAT='video/x-raw-rgb,bpp=32,depth=32,endianness=4321,red_mask=65280,green_mask=16711680' MIXERFORMAT=$MIXERFORMAT',blue_mask=-16777216,pixel-aspect-ratio=1/1,interlaced=false' SCALENRATE='ffmpegcolorspace ! videorate ! videoscale ! ffmpegcolorspace' fi CAPS="application/x-rtp,media=video,payload=96,clock-rate=90000,encoding-name=MP4V-ES" SRC=" udpsrc port=4444 caps=$CAPS ! rtpmp4vdepay ! mpeg4videoparse ! ffdec_mpeg4" SHMOPTION="wait-for-connection=0 sync=true " SHMSINK1="shmsink socket-path=$CONTROL_PIPE $SHMSIZE $SHMOPTION" while true ; do # Remove the named pipe if it exist rm -f $CONTROL_PIPE $gstlaunch -v \ $SRC !\ $SCALENRATE !\ "$MIXERFORMAT,framerate=$framerate,width=$width,height=$height" !\ $SHMSINK1 sleep 2 done exit
Top