How to use Perf with TimeStorm

Perf is a performance analyzing tool in Linux, available from Linux kernel version 2.6.31. It has become the defacto profiler for Linux since kernel version 4.14 and higher. Perf is capable of statistical profiling of the entire system or user application. For more information, please refer to the https://perf.wiki.kernel.org/index.php/Tutorial

TimeStorm supplements Perf with its own Linux Profiling Suite (LPS). LPS is easy to use and well-integrated with TimeStorm hardware targets management. LPS allows embedded developers to find performance bottlenecks in their code. LPS provides the statistical analysis of profile results and annotated source code view, helps in identifying any performance issues easily and quickly.

Note: You must have Perf installed on target for remote profiling.

This document will show How to Use Perf with TimeStorm. To run Perf in TimeStorm follow the below steps.

  1. Open your C/C++ Project.
  2. Build Project
  3. Set up Application based Perf configuration for a Remote target
  4. Set up System based Perf configuration for a Remote target (For system based profiling, please skip step 1, 2, 3 and start from step4)
  5. View the Perf results in Perf editor

1. Open your C/C++ Project:

Open your project using project explorer (we are using a C/C++ project created using the Perf sample), as shown below.

2. Build Project:

Right-click your project, and select Build Project to build and display the build logs in the console as shown below.

3. Set up Application based Perf configuration for a Remote target:

A TimeStorm Profiling Configuration is a collection of settings that are required to download and run an application. The TimeStorm Profiling configuration is persistent and can be launched multiple times to profile the application. To profile the application on the remote target, you have to create a TimeStorm Profiling configuration. To create a TimeStorm Profiling configuration, right-click on the project and click on TimeStorm Profiling Tools → Perf as shown below.

Click on the “Perf” option and this will open the “TimeStorm Profiling Tools Configurations” dialog as shown below.

Note that an entry is created under TimeStorm Perf Remote category with the project name appended with the Active build configuration, and other values are filled in with the default values that should work. The Timestorm Profiling Tools Configurations entries are categories in different tabs on the right-hand panel and are explained below. To profile the application, mostly you have to just select the remote target on the target tab, events from the perf events tab, and click the Profile button.

Main Tab:

In the Main tab, you can change the C/C++application by searching the project. You can change the project too if you want to use this configuration for a different project.

Target tab:

The Target tab shown below, allows you to select the hardware target used when you Profile the application.

In the Target tab, use the drop-down list to select a hardware target that you have already created. If you have not yet created a hardware target or want to change the settings for an existing target, click ‘Manage targets’ to open the Hardware Targets management utility.

Perf Events tab:

The perf tool supports a list of measurable events. The tool and underlying kernel interface can measure events coming from different sources. For instance, some events are pure kernel counters, in this case, they are called software events.

The types of events are:

Hardware Events: CPU performance monitoring counters.

Software Events: These are low-level events based on kernel counters. For example, CPU migrations, minor faults, major faults, etc.

Kernel Tracepoint Events: These are static kernel-level instrumentation points that are hardcoded in interesting and logical places in the kernel.

User Statically-Defined Tracing (USDT): These are static tracepoints for user-level programs and applications.

Dynamic Tracing: Software can be dynamically instrumented, creating events in any location. For kernel software, this uses the kprobes framework. For user-level software, uprobes.

Timed Profiling: Snapshots can be collected at an arbitrary frequency, using perf record -FHz. This is commonly used for CPU usage profiling and works by creating custom timed interrupt events.

For more information on Perf events, please refer to https://perf.wiki.kernel.org/index.php/Tutorial#Events

For Example, Software Events is selected in the Perf Events dropdown menu, the list of Software Events supported by the remote target is displayed as shown below.

To illustrate, select cpu-clock and task-clock events in the list as shown below.

After setting the values in all the tabs, click the Profile button at the bottom of the Profile configuration dialog. This will connect your host to your target, download the application to the target, and execute the application on the target. The commands executed on the target and the program output progress are displayed in the console view.

After your program successfully runs on your target, a perf.data file will be generated in the same directory as your project binaries. By default, this location is /root/. The perf.data will automatically copy from your target to your local TimeStorm project directory. You should now see perf.data in your src/ directory as shown below.

4. Set up System based Perf configuration for a Remote target:

To profile the whole system for a specific duration you have to set up a system based Perf configuration for a remote target. For System based profiling, please skip the above steps (Step1: Open your C/C++ Project, Step2: Build Project, Step3: Set up Application based perf configuration for a Remote target) and start from here.

Select TimeStorm Perf Based System Profiling Configuration option from the top menu as shown below.

It will open the “TimeStorm Perf Based System Configurations” dialog as shown below.

Double click on the TimeStorm Perf Based System Profiling option on the left menu which will create an entry under TimeStorm System Profiling category with the “New_configuration”, and other values are filled in with the default values that should work. The Timestorm Perf Based System Profiling Configurations entries are categories in different tabs on the right-hand panel and are explained in the above step 3 (Set up Application based Perf configuration for a Remote target). To profile the system, mostly you have to just select the remote target on the target tab, enter the project name in the project tab, set time interval in the run tab, select events from perf events that are optional and click the Profile button.

After setting the values in all the tabs, click the Launch button at the bottom of the Profile configuration dialog. This will connect your host to your target and start profiling the system and a perf.data file will be generated in the same directory as your project binaries. By default, this location is /root/. The perf.data will automatically copy from your target to your local TimeStorm project directory. You should now see perf.data in your src/ directory as shown below.

5. View the Perf results in Perf editor:

A Perf view is where all the Perf profiling results can be easily read. Double click on the perf.data

file (generated while profiling) which will display the results in the tree view as shown below.

The tree structure displayed above describes one profile of one or more events in the following manner:

Events -- the name of the profiling event used by Perf

  • Session -- the name of the session the profile is stored in
    • Image -- the binary being profiled
      • Symbol -- symbols gathered from the binary's debug information
        • Sample -- individual Perf samples correlated to line numbers of source code
      • Dependent Images -- other binaries related to the run of the program; shared libraries or kernel modules
        • Image -- the dependent binary
          • Symbol -- same as above
            • Sample -- same as above

For more information on Perf results view, please refer to https://wiki.eclipse.org/Linux_Tools_Project/PERF/User_Guide#Perf_Views