MeeGo 1.2 Harmattan Developer Documentation Develop for the Nokia N9

Using functracer

The functracer tool collects backtraces, arguments and return values of functions specified in a plugin. The plugins define which set of functions are traced. Only one plugin can be used at each functracer execution. The functracer tool works with optimised code.

Packages

source: functracer

binary: functracer

Installing functracer on the Harmattan device

Install functracer through the developer mode applet.

Prerequisite: Developer mode must be enabled.

  1. Select Settings > Security > Developer mode.
  2. Install the Tracing bundle package by clicking Install.
  3. You get a notification screen that lists all the applications to be installed in the bundle package. To install, click OK.
  4. A dependency notice appears. Click Accept.

For more information on developer mode and installable tools, see Activating developer mode.

Using the tool

The functracer tool provides the following basic features:

  • It uses ptrace and breakpoints to trace the functions.
  • It uses architecture-specific execute-out-of-line instruction emulation to avoid races in handling breakpoints in threaded programs (similarly to how Linux kernel Kprobes works).
  • Backtraces are collected with libunwind (with patches for ARM that are not yet in the upstream libunwind release).
  • It provides an API for changing resource context so that allocated resources can be, for example, assigned to the correct server client.

Currently, the following plugins have been implemented:

  • audit traces any functions or C++ methods specified with the -a option. If debug symbols are available, the function does not even need to be exported.
  • file traces file descriptor operations.
  • gobject traces Glib objects creation and destruction.
  • memory traces C-library memory allocation functions (used by default, when no plugin is specified).
  • memtransfer traces C-library memory read and write functions, for example, memset.
  • shmposix traces POSIX shared memory and mmap operations.
  • shmsysv traces SYSV shared memory operations.
  • thread-resource traces thread resources.

Benefits of functracer over sp-rtrace

When comparing functracer (which uses ptrace) against sp-rtrace (which uses LD_PRELOAD), the used technology is the main difference.

The functracer has the following benefits over sp-rtrace:

  • It can attach to an already running processes, you do not need to restart it.
  • It does not affect the memory usage of the traced process, disable prelinking, or change symbol resolving (as the sp-rtrace LD_PRELOAD does).
  • It can trace arbitrary functions with the audit module.
  • It can trace function calls that do not go through PLT (are internal to a library or binary).
  • It can trace things happening in the early process start (before LD_PRELOAD has an effect).
  • It works even if the process removes the LD_PRELOAD variable (for example, when its privileges are increased).
  • It can trace resources related to forking and thread creation and destruction, and also has a module for tracing threads.
  • With functracer, it is much easier to write (simple) modules than just trace specified functions.
  • libunwind gives better backtraces than libc backtrace when unwind tables are missing.

Use functracer to trace always, or already running processes, or if you want to trace everything that happens before the process enters main.

Benefits of sp-rtrace over functracer

The sp-rtrace tool has the following benefits over functracer:

  • It can use multiple modules at the same time for tracing (for example, memory and pagemap).
  • It is portable, does not require architecture-specific emulation for instructions overwritten by the breakpoints.
    • Due to the above, sp-rtrace core parts are much easier to review and maintain.
    • ptrace changes traced process signal semantics slightly; getting tracing of new processes and threads (clone syscall handling) working properly is tricky.
  • It is safer to write modules that need to call other functions within the traced process scope (related to above).
  • The ptrace performance is worse than the performance of LD_PRELOAD (basically it is on level of peek and poke, this means that getting the backtrace information requires a lot of these syscalls), and the (larger) slowdown may affect how the process works.
  • There is a module for analysing memory fragmentation.
  • The tool provides resource reference counting support.

Use sp-rtrace to trace applications and other programs that you can start directly by yourself. Because the slowdown is relative to how often the traced functions are called and how deep the backtraces are, disable tracing during startup unless you are specifically tracing it and keep backtraces as short as you can (while still getting useful call graphs). You can also use functracer to test how deep backtraces you need to get useful call graphs.

Using the tool

Tracing

To immediately start (-s) tracing memory allocation operations and get backtraces to them, enter the following command:

 functracer -s -p $(pidof your_app_name)

To also get backtraces for deallocations (-A) and longer than default backtraces (-b), enter the following command:

 functracer -s -A -b 16 -p $(pidof your_app_name)

To trace something else, such as file accesses instead of memory, by specifying which plugin you want to use (-e), enter the following command:

 functracer -s -e file -p $(pidof your_app_name)

To trace an arbitrary function or method, use the -a option. If you do not want to give the whole method signature, use * to match a method signature pattern. Enter the following command:

 functracer -s -a "QObject::deleteLater*" -p $(pidof your_app_name)

If you want to run the program directly instead of attaching to it, just enter the program and its arguments as last.

If you want the output saved to a file instead of output to console, add the -o option and give it the output directory name. Enter the following command:

 mkdir trace;
 functracer -s -o trace -b 8 /bin/ls

Functracer uses the same PID-IDX.rtrace.txt naming convention for the ASCII trace files as sp-rtrace.

Call functracer --help for the parameter description and see its manual pages for more detailed explanations.

Post-processing

The functracer tool can optionally resolve some of the function names (ones in the symbol table) at run-time with its -r option, but as that can give misleading results and does not demangle C++ symbols, it is often better to enter the following command:

 functracer -s functracer | sp-rtrace-resolve -telf

However, there can be a slight delay due to buffering used by sp-rtrace-resolve, even with its most lightweight resolving method (-telf, the ELF symbol table).

Normally you can just save the trace file (with the -o option), install debug symbols and post-process the data to remove the freed allocations and to get correct or all symbol names. After installing debug symbols, you can use sp-rtrace-resolve to resolve the output backtrace addresses. Enter the following command:

 sp-rtrace-resolve < 14409.0.rtrace.txt > resolved.rtrace.txt

When investigating more complicated programs, you can use sp-rtrace-postproc to:

  • Merge identical backtraces with the -c option.
  • Remove freed allocations with the -l option.
  • Resolve with the -r option.

The last option asks sp-rtrace-resolve to do the resolving using the fastest or least memory method:

 sp-rtrace-postproc -clr < 14409.0.rtrace.txt

To get full resolving with the above, use sp-rtrace-resolve explicitly. Enter the following command:

 sp-rtrace-postproc -cl < 14409.0.rtrace.txt | sp-rtrace-resolve

The above mentioned sp-rtrace-postproc and sp-rtrace-resolve tools come from the sp-rtrace package which shares the same trace format.

The following example illustrates the readable text format trace output:

...
16427. [00:03:47.105] malloc(48) = 0x8fba0
16429. [00:03:47.105] malloc(128) = 0x8fc10
16433. [00:03:47.107] malloc(128) = 0x8fcd0
# allocation summary: 422 block(s) with total size 54432
        0x3b921c84 QString::realloc(int) from /usr/lib/libQtCore.so.4.7.1
        0x3b922070 QString::resize(int) from /usr/lib/libQtCore.so.4.7.1
        0x3b92bb24 operator>>(QDataStream&, QString&) from /usr/lib/libQtCore.so.4.7.1
...

For more information on post-processing and visualising the results, see Using sp-rtrace and Using sp-rtrace-visualize.

Further information

For more information on the tools, see the following links: