Writing a GStreamer Application using Qt on the TI DM3730 EVM


Contents

Introduction

This tutorial will cover creating a platform and application that uses GStreamer to display a video and the Qt UI Framework for the control user interface. In this tutorial we will:
  • Install a Pre-built Starting Point from the TimeSys Web Factory.
  • Install the TimeStorm integrated development environment (IDE).
  • Use TimeStorm IDE to build and run a demo application
  • Use TimeStorm IDE to customize the demo application

GStreamer Overview

GStreamer is an open source multimedia framework. From a high level, it takes some audio/video sources, and modifies and routes it to the actual sound and display hardware interfaces in the kernel. More specifically, it is a collection of media encoders/decoders (codecs), filters, format handlers, such as muxes and demuxes, and other media tools, as well as a means of connecting these components together in order to properly interpret the media file. A fully defined set of components and connections is called a pipeline.

There are a number of ways to interact with GStreamer. There are off-the-shelf media players that are designed to plug into GStreamer and construct pipelines for you automatically. There is also command line utility, gst-launch, that allows you to manually build your own pipelines for testing purposes. For many applications, you will use the GStreamer APIs directly. These are C function calls and structures that are included as part of the GStreamer library. There are bindings for other languages as well.

For more information about GStreamer, please see the official documentation at: http://gstreamer.freedesktop.org/data/doc/gstreamer/head/manual/html/index.html

Qt Overview

Qt (pronounced "cute") is a programming framework that is primarily geared towards graphical applications. It forms the basis of a number of projects, including the KDE desktop and the Meego platform. It is implemented in C++, although hooks for other languages do exist. The code is ostensibly platform-independent, although it must be re-compiled for different architectures. One advantage of this property is that a Qt application compiled on a host platform should look and feel the same as one built and run on the target system. This allows you to do most of your development and testing before even powering-up the target device.

Besides basic graphical elements, Qt also provides a number of modules that may be used to interface with more general Linux systems. The Phonon multimedia framework interacts with gstreamer for use in audio/video applications. Qt also provides libraries that can use accelerated 2D/3D libraries, such as OpenGL ES and OpenVG. Qt contains an implementation of the Webkit browser engine, as well. Qt is designed to give the developer a common interface to a wide variety of systems regardless of the operating system or platform.

The Timesys Factory will automatically configure Qt for your platform based on your package selection. For instance, if you include the tslib package in your build, Qt will include support for touchscreens. Selecting 3D libraries on certain platforms will enable OpenGL in Qt (if supported). The Factory's internal logic helps to make Qt configuration relatively simple.

TI OMAP3x/AM3x/DM37xx Display Subsystem Overview

The OMAP3x/AM3x/DM37xx display subsystem displays a video frame from a memory framebuffer on an LCD or TV display. It supports multiple framebuffers which can blended via overlays into a single physical display based on priority and transparency color keys. In Linux, the display subsystem is handled by the omapdss driver. This driver exports a control structures via sysfs. This sysfs entries allow framebuffers, overlays, and other display-related features to be configured and controled by opening writing values to particular files located under /sys/.

Links to additional information about the omapdss system and driver are listed at the end of this document.

TI DVSDK Overview

Digital video systems can be highly complex, potentially requiring video, imaging, speech, audio, and other multimedia support. TI's DVSDK provide a framework to support multimedia on TI chips, including an integrated collection of codecs to run on the video acceleration hardware.

Requirements

  • PC running a recent linux distribution
  • TI DM3730 EVM board
  • LinuxLink Desktop Edition

Some additional cables will be required for your board, see the Getting Started Guide for the board for more information.

Creating the platform

The first step is to create a base platform to install on the board that contains all necessary packages for building your application.

Using a Pre-built Starting Point

The easiest way to get started is to use a Pre-built Starting Point. Timesys has already created a Starting Point that contains the necessary packages to build and run the program in this tutorial. To download the Starting Point:

  • Open https://linuxlink.timesys.com in your browser
  • Login to LinuxLink
  • Click Download BSP/SDK in the top navigation area.
  • If you have a subscription for multiple processors, choose the TI-DM3730 processor
  • Under Pre-Built Starting Points section, click DM3730EVM Qt/Multimedia Starting Point.
  • Download dm3730_evm-development-environment.sh
  • At a linux command prompt run:
    • chmod +x dm3730_evm-development-environment.sh
    • ./dm3730_evm-development-environment.sh

The console-based install program will start. After accepting the TimeSys EULA, you will be prompted for an install location. The default location is within your home directory. The installer will automatically create a sub-directory named dm3730_evm . This directory will be referred to as <install_dir> in later sections of the document. Be aware that if another Starting Point or custom build for dm3730_evm has already been installed, the installer will automatically move the old installation to prevent overlap.

Using a custom Factory build

It is also possible to use a custom platform built using either the Timesys Web Factory or Desktop Factory. For this tutorial, the platform must contain the following packages:

  • qt-embedded-linux
  • gstreamer
  • gst-plugins-good
  • gst-plugins-bad
  • ti-dvsdk-omap3
  • gst-ti-plugin
  • tslib
  • dropbear (for TimeStorm remote access)
  • busybox (for basic system commands)
Note: It is possible to use sysvinit in combination with other packages instead of busybox to provide the basic system. Use the SystemV Init template in factory to replace busybox, if desired.

This document will assume that you installed the Pre-built Starting Point. If using a custom build, substitute the appropriate toolchain, RFS, and kernel image paths. For more information on using the Timesys Web Factory or Desktop Factory, see the Additional Information section at the end of this document.

Booting the platform

Follow the getting started guide to boot the kernel and RFS on the DM3730.

https://linuxlink.timesys.com/docs/gsg/dm3730_evm

For the pre-built starting point the kernel image is <install_dir>/ uImage-2.6.32-ts-armv7l, and the RFS tar is <install_dir>/rootfs.tar.gz.

Additional bootargs

In addition to the default bootargs in the Getting Started Guide, you must pass an argument to allocate memory to the second framebuffer. Add omapfb.vram=0:1M,1:1M to the end of your normal setenv command when setting environment variables in U-Boot.

To support the TI DVSDK, you must reserve some memory for use by the DSP and also tell the video out driver to statically allocate some buffers at boot time. To do this add mem=88M and omap_vout.vid1_static_vrfb_alloc=y to your bootargs.

Example

> setenv bootargs  mem=88M console=ttyS2,115200n8 ip=10.0.0.10 nfsroot=10.0.0.1:/full/path/to/rfs root=/dev/nfs rw omapfb.vram=0:1M,1:1M omap_vout.vid1_static_vrfb_alloc=y

Set a root password on the board

Once the board is booted to a shell prompt, set the root password by typing "passwd". A password will be required for remote access from TimeStorm. This document will assume that you set the password to "password".

Installing TimeStorm with Qt integration

TimeStorm with Qt integration can be downloaded from https://linuxlink.timesys.com/ide. Be sure to get "TimeStorm for Qt". Unpack the archive into your home directory. You will need a license file for TimeStorm installed on your desktop computer. To generate and install a TimeStorm license, follow the instructions here: https://linuxlink.timesys.com/docs/manage_your_linuxlink_user_account#generate-license.

Starting TimeStorm

Before starting TimeStorm, you must export an environment variable that points to the location of the platform's qmake spec. Replace <install_dir> in the following with your actual installation directory.

export QMAKESPEC="<install_dir>/toolchain/share/mkspecs/qws/linux-timesys-g++/"

When TimeStorm starts for the first time, it will start with an empty workspace.

Adding Qt version to preferences

Once TimeStorm is started, you must add your Qt version under the preferences for TimeStorm. Select "Window->Preferences" in the TimeStorm menu and choose Qt in the Preference Dialog. Click the "Add..." button to add a new version. Name the version "Cross". Enter <install_dir>/toolchain/bin as "Bin Path" (replacing <install_dir> with your actual installation directory). "Include Path" will be filled in automatically. Click "Finish" to add this version.

Importing the demo project into TimeStorm

TimeSys provides the source for this demo in an archive that is available here: https://linuxlink.timesys.com/git?repo=timesys-omap35x-demos.git&view=refs&path=tags&h=master. Once you download the archive, unpack it using the command "tar xjf timesys-omap35x-demos-20101109.tar.bz2".

Choose "File->Import..." from the TimeStorm menu. Under the "Qt" folder, choose "Qt Project" and click "Next". The demo archive contains multiple demos. For this tutorial, browse to the file gstreamerdemo.pro within the gstreamerdemo directory that you extracted from the archive.

Building the demo project with TimeStorm

At this point you should be able to cross-build the demo source and produce an executable that can run on the target. Choose "Project->Build All" to build all projects in the workspace. A executable named " gstreamerdemo" will be created within the project.

Setting up TimeStorm to communicate with your board

Choose "Run->Hardware Targets" from the TimeStorm menu. If this menu item is not visible you may need to change the Workbench Perspective. In the upper right hand corner, click the icon to switch to the "C/C++ Perspective". The Hardware Targets menu item will be visible in this perspective. Once the target management dialog is open, click the "New" button to create a new target. Name the target "Demo". Change the download option to SCP. Enter the ip address of your target board. For the remaining fields enter Username: root, Password: password, Destination Directory: /root. Leave "Link to Execution" checked. Switch to the "Execute" tab and choose the SSH option. Now that your target is setup, click the "Check Link" button. A log will show the interaction with the board and indicate Pass or Fail for the test. Once the test has passed, close the Target Management dialog.

Copying media to your board

In addition to the executable, the gstreamer demo accesses media files containing the video that is shown. The executable will automatically be copied to the board by TimeStorm each time you launch the project. However, this is undesirable for the media as the files can be fairly large file. The media directory of the demo archive contains media files formatted for various screen sizes and codecs. In the media directory, the files have different names, but on the target the program always expects the same name (e.g. movie.ogg or movie.mp4). For the TI DM3730 EVM, the following media should be copied (type each command on your host PC):

$ scp media/bbb_720x480_mute.mp4 root@10.0.0.10:/root/movie.mp4

$ scp media/bbb_640x480_rotated_mute.ogg root@10.0.0.10:/root/movie.ogg

Note: Replace 10.0.0.10 with the IP address of your board.

This pre-copying of some files is a good practice anytime you have large files that infrequently change as part of your application.

Launching the demo on your board

Choose "Run->Run As->C/C++ Remote Application" from the TimeStorm menu. This will create a new launch configuration. Fill out the launch configuration as follows. Once all tabs are filled out, click the "Run" button to launch the program.

Main Tab
Name: gstreamerdemo
Project: gstreamerdemo
C/C++ Application: gstreamerdemo

Arguments Tab
Project Arguments: -qws

Leave all other fields as default.

Environment
Remove all environment variables to start.

Add the following variables:
QWS_DISPLAY=transformed:Rot270
QWS_MOUSE_PROTO=Tslib:/dev/event1

Debugger Tab
This tab is not required for run mode. If you want to use this launch configuration for debugging, you must choose an appropriate Toolchain to use as the debugger. See the TimeStorm manual for more information on managing TimeStorm toolchains.

Source Tab
No changes required.

Target Tab
Choose the "Demo" target in the drop-down.

Download Files Tab
The default of downloading the target program to the current directory (.) is good. No changes required.

Common Tab
No changes required.

Relaunching the demo

Once the launch configuration is setup, it can be re-used for each subsequent launch. Recently launched configurations are available under "Run->Run History". The dialog to create new configurations and edit/launch existing ones is available under "Run->Run Configurations". You will need to exit the previous launch of the demo before you can launch it again. Using the options you can quickly re-build and re-launch your project after any changes to test the out on the board. On each launch, the executable file will be re-downloaded so the latest changes can be tested.

Customizing the demo

At this point, you've been able to build the demo project and launch it on the board. The following sections will step through a simple modification to the gstreamerdemo program. We will add a new menu item that resets the movie back to beginning.

Editing the UI

We will add a new item to the existing "File" menu. Open the gstreamerdemo.ui file (double click in Project Explorer view). This will bring up the Qt Designer Editor. The designer will show the user interface for the demo. Click the "File" menu. The menu will open. Click "Type Here" under the File menu. Then type "Restart" and hit enter to create a new menu item named Restart. Save the gstreamerdemo.ui file. Next time the demo is re-built and launched, it will include this new menu item.

Add declation and implementation of new slot

Open the gstreamerdemo.h header file. Under public slots, add a new method on_actionRestart_triggered().

private slots:
    void on_actionRestart_triggered(); /* add this line */
    void on_actionTheora_hovered();
    void on_actionTheora_triggered();
    void on_actionMp4_hovered();
    void on_actionMp4_triggered();

Open the gstreamerdemo.cpp file. Implement the new method as follows:

/* This is an entirely new method. */
void gstreamerdemo::on_actionRestart_triggered() {
    if (pipeline == NULL) {
        /* Video is not open, do not seek */
        return;
    }
    /* This gstreamer call seeks the pipeline to position "0". */
    if (!gst_element_seek(pipeline, 1.0, GST_FORMAT_TIME, GST_SEEK_FLAG_FLUSH,
            GST_SEEK_TYPE_SET, 0, GST_SEEK_TYPE_NONE, GST_CLOCK_TIME_NONE)) {
        printf("Seek failed\n");
    }
}

Automatic UI signal linking

UIs created from Qt Designer will automatically establish connections between signals emitted from its widgets and any slots that follow the naming convention on_<sender>_<signal>(). Based on the name of our new slot, it will automatically be connected to the triggered signal of our new restartAction menu item.

Rebuild and relaunch

Now we've made all of our changes. Pick "Project->Build All" to rebuild the executable with the changes. Once it has built successfully, re-launch the demo using "Run->Run History" menu item. You must exit the demo on the board before re-launching. Once launched, the demo will have a new menu allowing you to restart the video at the beginning when the movie is currently playing or paused.

Additional Information