How to use Oprofile with TimeStorm

The OProfile has been around for decades, and for some time was the workhorse of performance profiling on Linux®-based systems, and can serve the same role today. However, the OProfile is not included in the latest versions of Kernels. OProfile works fine for kernel versions less than 4.14. So it may be prudent for OProfile users to start considering alternative tools such as Perf. Both OProfile and perf currently use the same basic mechanism in the Linux kernel. TimeStorm supports Perf as well, please refer to How to use Perf with TimeStorm document for more details.

TimeStorm supplements the Eclipse OProfile tool with its own Linux Profiling Suite (LPS). LPS is easy to use and well-integrated with TimeStorm hardware targets management. LPS allows embedded developers to find performance bottlenecks in their code.

LPS uses OProfile, which is an Open Source profiler for Linux systems. OProfile collects information about the kernel and all running applications. OProfile profiles your code by recording the output from hardware registers that count events such as CPU cycles. LPS provides the statistical analysis of profile results and annotated source code view, helps in identifying any performance issues easily and quickly.

For more information about OProfile, refer to the OProfile Manual found at http://oprofile.sourceforge.net/doc/index.html

Note: You must have OProfile installed on target for remote profiling or on the host machine for local profiling. For installing Oprofile on remote target, please refer to https://linuxlink.timesys.com/docs/wiki/engineering/HOWTO_Use_oprofile

This document will show How to Use OProfile with TimeStorm. To run OProfile in TimeStorm follow the below steps.

  1. Open your C/C++ Project. If you want to learn how to use oprofile, go to Learn OProfile by creating a C Project using the Oprofile sample section below
  2. Build Project
  3. Set up Application based Oprofile configuration for a Remote target
  4. Set up System based Oprofile configuration for a Remote target (For system based profiling, please skip step 1, 2, 3 and start from step4)
  5. View and analyze Profile date
  6. Importing Oprofile data
  7. Comparing Profiling Sessions

1. Open your C/C++ Project:

Open your project using project explorer (we are using a C/C++ project created using the oprofile sample) as shown below.

2. Build Project:

Right-click your project, and select Build Project to build and display the build logs in the console as shown below.

3. Set up Application based Oprofile configuration for a Remote target:

A TimeStorm Profiling Configuration is a collection of settings that are required to download and run an application. The TimeStorm Profiling configuration is persistent and can be launched multiple times to profile the application. To profile the application on the remote target, you have to create a TimeStorm Profiling configuration. To create a TimeStorm Profiling configuration, right click on the project and click on TimeStorm Profiling Tools → Oprofile as shown below.

Click on the “Oprofile” option and this will open the “TimeStorm Profiling Tools Configurations” dialog as shown below.

Note that an entry is created under TimeStorm Oprofile category with the project name appended with the Active build configuration, and other values are filled in with the default values that should work. The Timestorm Profiling Tools Configurations entries are categories in different tabs on the right-hand panel and are explained below. To profile the application, mostly you have to just select the remote target on the target tab and click the Profile button.

1. Target Tab:

The target tab is shown below. Choose “Remote” to run OProfile on a remote target or choose “Local” to run OProfile on a localhost. When the target is selected, TimeStorm checks the OProfile CPU type and fills in the details. An error message is displayed if TimeStorm cannot run OProfile commands.

2. Project Tab:

The project tab is shown below. LPS organizes profiling sessions into projects and sessions. You can create a new profiling project or choose an existing one. The session name is optional. If the session name is left blank, TimeStorm creates a session using the date and time.

3. Run Tab:

The run tab is shown below. LPS allows you to profile an application or the whole system for a specified duration or the application for a specified duration.

Application – If you want to profile an application, select it from either your workspace or file system by clicking the Browse button. If you want to skip profiling during application startup, you can delay the OProfile startup by setting the delay time.

Timed – Select this option to profile the whole system for a specific duration.

Application and Timed – If you want to profile an application for a specified duration, select both Application and Timed Profiling and set duration.

4. Options Tab:

This tab allows you to configure OProfile.

Kernel image (vmlinux) – If you want to profile the kernel, select the vmlinux kernel image file by clicking browse. If you do not have a vmlinux file or you do not want to profile the kernel, select ‘no-vmlinux’ checkbox. For a detailed explanation of other options on this tab, please refer to http://oprofile.sourceforge.net/doc/controlling-daemon.html

5. Events Tab:

This tab allows you to specify the events for each of the hardware performance counters. The OProfile CPU type and the number of performance counters on your target are displayed at the top.

Counter / Event Name – Expand the counter to view events in the counter. Select the event for profiling. You may only choose one event per counter. Also, if you choose a particular event for one counter, you may not choose the same event for another counter.

Count – The field shows the counter reset value for the given event. To edit this value, click on the counter value, type in a new value and hit Enter key.

Unit Mask – This field shows the unit mask. To edit this value, select the event and enter new mask value in ‘Unit Mask Editor’ as shown below.

For more information on the counters and events, please refer to OProfile documentation (https://oprofile.sourceforge.io/doc/eventspec.html)

After configuring OProfile, ‘Click’ the Profile button to start profiling. The profiling status and progress is displayed in the Profiling monitor view. To view and analyze Profile data, please refer to step 5 (View and analyze Profile date).

4. Set up System based Oprofile configuration for a Remote target:

To profile the whole system for a specific duration you have to set up a system-based Oprofile configuration for a remote target. For System based profiling, please skip the above steps (Step1: Open your C/C++ Project, Step2: Build Project, Step3: Set up Application based Oprofile configuration for a Remote target) and start from here.

Select the TimeStorm System Profiling Configuration option from the top menu as shown below.

It will open the “TimeStorm Profiling Tools Configurations” dialog as shown below.

Double click on the TimeStorm System Profiling option on the left menu which will create an entry under TimeStorm System Profiling category with the “New_configuration”, and other values are filled in with the default values that should work. The Timestorm System Profiling Configurations entries are categories in different tabs on the right-hand panel and are explained in the above step 3 (Set up Application based Oprofile configuration for a Remote target). To profile the system, mostly you have to just select the remote target on the target tab, enter project name in the project tab, set time interval in the run tab, and click the Profile button.

After configuring OProfile, ‘Click’ the Profile button to start profiling. The profiling status and progress is displayed in the Profiling monitor view. To view and analyse Profile data, please refer to step 5 (View and analyse Profile date).

5. View and analyse Profile date:

To view and analyze a successfully completed profiling session, right-click on the session and click on “Analyze” as shown below. Alternatively, you can also double click on the session. This will open the profile analysis view and display the profiled data.

The profile analysis view displays the session name and the profiled events in a drop down. Selecting a profile event will display the total profile count for that event and the applications, libraries, and functions that are profiled for that event. When an executable includes debugging information, the filename and the line number are also displayed. For example in the figure, CPU_CLK_HALTED event is selected, the multithread application is profiled, the thread_func has the highest-profile count at line number 26.

To analyze and drill down through large profile data, you can

  • sort by column by clicking on the column header
  • search in the filename / function by using the search facility by clicking the flashlight icon in the analysis view toolbar.
  • Focus on the functions of interest or critical bottlenecks by using the name filter, count filter, and percentage count filter. The filters can be configured, set, and unset using the down arrow in the analysis view toolbar.

For files that have debug information, you can view the profile data and the source code side by side in the annotated source editor as shown below. You can open the annotated source editor by clicking the ‘View Source’ button in the analysis view’s toolbar.

Profile Settings View: To view the OProfile settings for a profile session, right-click on the session in the profiling monitor view and select ‘Profile Settings’. The settings are displayed in Profile Settings view as shown below.

6. Importing Oprofile Data:

If you have already profiled your system/application using OProfile and you want to use LPS to analyze the results, you can import the data to create a session for analysis. To import OProfile data,

  1. Right click in the ‘Profiling Monitor’ view and select the ‘Import’ option from the context menu to open the Import wizard.
  2. Choose Other --> OProfile data into LPS Project and then click ‘Next’ to open the Import dialog as shown below.

    3. Select either the Remote or Local radio button to import the OProfile data from a remote target or local file system, click the Browse button to select the data folder. The CPU type is detected and filled in. Click next to continue to the next page as shown below.

    You can import the profile data into an existing project or create a new project. The target field is filled in with the remote target / local file system IP address. You may name or leave the session name blank. The timestamp is used if the session name is left blank.

  3. Click Finish to import OProfile data. The new session creation and import progress appear in the ‘Profiling Monitor’ view. Select this session and right-click to analyze or to view as described in previous sections (View and analyse Profile date).

7. Comparing Profiling Sessions

During development, you will profile your application and analyze the data, and fix the performance bottlenecks. You will profile the application again. You can compare profile sessions in the same target by using the profile session comparator included in TimeStorm. Select the more recent session with modified code in the ‘Profiling Monitor’ view, right-click in the view, and select ‘Compare’. The ‘Compare Profile Sessions’ window opens with other sessions. Select and double-click the other session you want to compare to. The ‘Compare Profile Sessions’ window compares and displays the profiling results side by side as shown below.

The profile events and the application/library names are displayed in the ‘Profile Events’ pane. You can double click on the name to just display the profile count for that application.

Learn OProfile by creating a C Project using the Oprofile sample:

To create an Oprofile sample project, open the ‘New Project’ wizard by choosing File → New → Project from the main menu OR click the ‘New’ button and use the drop-down menu to the right of the button to choose ‘Project’ as shown below.

Expand C/C++ , choose a C/C++ Project, and click Next. To illustrate the Sample Oprofile Project (Samples → Oprofile)

Project page

  1. Enter a Project Name.
  2. Under the ‘Project Type’ heading, expand Samples and select Oprofile project.
  3. Under the Toolchains heading select the TimeStorm Cross-Compile Toolchain.

    4. Click ‘Next’ to bring up the ‘Basic Settings’ as shown below.

    Basic settings page

    In this page, set the basic properties of the project like Author, Copyright notice, Greeting message and Source (where the project source files are to be stored within the project).

    Select build configuration page

    Build Configuration is a set of pre-defined compiler and linker settings that are used to build a project.

    By default, TimeStorm creates three Build Configurations for new projects:

    Debug — This setting performs no code optimization and attaches complete debugging information. These settings are designed to make debugging as easy as possible.

    Release — This setting has the highest compiler code optimization settings. The output will not include any debugging symbols. Code compiled with this configuration is ready for production use.

    GnuProfiler — This setting builds the software with debugging information as well as code to collect profiling information via GNU gcov.

    You can change the build configuration settings before creating the project by clicking the ‘Advanced Settings…’ button. You can also change these settings after creating the project.

    Click Next to continue.

    Select SDK page

    The Select SDK page show below allows you to choose the SDK you want to use with the project. Select the SDK Type and then choose the SDK.

    Click Finish to create and exit the wizard. The new project is created and displayed in the Project Explorer view as shown below.

    After creating the Sample Oprofile Project, follow the above steps (Step2: Build Project, Step3: Set up Application based Oprofile configuration for a Remote target, Step5:View and analyse Profile date) to run Oprofile on a remote target.