third(1)third(1)NAMEthird - Third Degree: heap usage and leak profiler, and memory-access
error checker for C and C++ programs.
SYNOPSISthird [-pthread] [-fork] [-invalid] [-uninit heap+stack+...] [-hide n]
[-g] [option...] program [argument...]
See the start of the OPTIONS section below for details of Third Degree
options that may be essential for the correct execution of the program.
The atom -toolthird interface is still available, for compatibility
with earlier releases. However, it is now undocumented, and it will be
retired in a future release.
DESCRIPTION
See prof_intro(1) for an introduction to the application performance
tuning tools provided with Tru64 UNIX.
Third Degree (third) creates an instrumented version of a debuggable C
or C++ program (naming it program.third) that, by default, produces a
log file of stack traces for places where heap memory was allocated and
subsequently leaked during a test run of the instrumented program.
If you specify program arguments (argument...) or -run, the instru‐
mented program is executed also. If you specify -display, more is run
afterwards to display the log file. If the program name is omitted, a
usage message is printed.
For example, to instrument, run, and display the leaks in a multi-
threaded program: cc -g -pthread -o program *.c third-pthread -display
program data/*
Specify -all or -incobj to report stack traces that lead through shared
libraries and to check memory references made by the shared libraries.
Specify -hide 0 if you need to see reports of problems in code that was
not compiled -g or -g1. Specify -g to keep debug information, so that
the instrumented program can be debugged.
To check for leaks at times before program exit, specify -before proc
or -after proc, which trigger a check when the procedure proc executes.
To check the program for access to invalid addresses within the stack
and heap areas, specify the -invalid option, and see the messages in
the log file.
To check the program for reading variables that have not been initial‐
ized, specify the -uninit option with a value of at least heap, stack,
or heap+stack. This option may cause the instrumented program to mal‐
function, because it poisons all uninitialized memory with an unusual
value -- 0xfff8a5a5. This can be used to advantage by running regres‐
sion tests on the instrumented program and comparing its results with
those that were expected. Use the -quiet option to suppress the usual
informational messages, or just manually run the program.third file for
each test after instrumenting it once (for example, without the -run
option or program arguments): third-quiet program -uninit h+s -inv
program data/* | diff benchmark -
The pixie -testcoverage command can be used to list which source lines
were not exercised (and therefore not checked) by the regression tests.
An alternative display technique for -invalid and -uninit errors is to
give the log file to emacs as if it contained compiler diagnostics.
This enables emacs to place its cursor on the source line referenced by
each error message in turn:
xemacs (run xemacs in working directory) ESC-x compile
(or click the "Compile" button in xemacs) cat pro‐
gram.3log (changed from "make -k") CTL-x ` (place cursor
on code for next error - repeat)
OPERANDS
File name of a fully linked call-shared or nonshared executable to be
profiled. This program should be compiled with the -g or -g2 option to
obtain the most complete error reports and profiles.In particular,
errors in code compiled with the default -g0 option are usually not
reported unless the -hide 0 option is specified, and line numbers,
static procedure names, and file names are unavailable. If -g1 was
used, variable names are unavailable. Inlined procedures are not dis‐
played separate from their callers, so they are best avoided.Programs
that are stripped or are optimized by spike or cc -om are not sup‐
ported. All arguments following the program name are considered to be
arguments needed by the instrumented program to execute the procedures,
lines, and instructions of interest. Multiple arguments can be speci‐
fied. They imply -run if any are specified, and they can be replaced by
-run if none are needed.
OPTIONS
Options can be abbreviated to three characters, option keyword values
to one.
Essential Options
Some or all of these options may be needed to prevent the instrumented
program from malfunctioning: Specify -pthread if the program or any of
its libraries calls pthread_create(3), for example if it was compiled
with either the -pthread option or the -threads compatibility option.
This will make the log file reporting thread-safe. Specify -fork if
the program calls any variant of fork(2). It is not usually needed if
the subprocesses also call any variant of exec(2). The -fork option
ensures that forked multi-threaded programs are profiled in a thread-
safe way, and it produces separate log files for the forked subpro‐
cesses, including the process id in their file names as if -pids was
specified. Failure to use -fork might lead to deadlock in the forked
child processes. The heap space for the program being analysed is
located at the usual address, immediately following the program's
static data. By default, any heap space needed by Third Degree's run-
time analysis code is located separately, starting at the address
38000000000. If the program needs to use this address range, use -heap‐
base to tell third to use memory at a specific hexadecimal address (for
example, 10000) or taso for the default 31-bit address (if the program
was linked with the -taso option). By default, third memory maps large
areas between address 3c000000000 and 3ff00000000. If the program
needs to use this address range, use -mapbase to specify what address
(for example, 10000) third can map. Disables third's handling of sig‐
nals SIGILL, SIGTRAP, SIGABRT, SIGEMT, SIGFPE, SIGBUS, SIGSEGV, SIGSYS,
SIGXCPU, and SIGXFSZ. Also disables reports of reentrant calls to heap
management routines (malloc, free, and so on) from signal handlers; may
avoid deadlock in applications that make significant use of signal han‐
dlers. This option can not be used with the -before or -after options
for leak detection and heap tracing.
File Generating Options
Does not print informational and progress messages on the standard
error stream. Prints the command lines used to instrument the program
and to execute the instrumented program. Names the instrumented pro‐
gram file instead of the default program.third. Specifies the direc‐
tory to which the instrumented program writes the log file(s) for each
test run. The default is the current directory. Adds the process-id
of the instrumented program's test run to the name of the log file pro‐
duced (that is, program.pid.3log). By default, the file is named pro‐
gram.3log, but the process-id is always included for the log files of
forked subprocesses.
Stack-Trace Generating Options
Searches for source files in the specified directory when printing
their names in stack traces in the log file. When distinguishing dif‐
ferent heap allocation calls, considers only the top n procedures on
the program's call stack--for example, when memory is allocated by a
common procedure that is called by many others. The default stack
depth is 200. Controls the degree to which error reports for system
libraries and other non-debuggable code are hidden (that is, not
printed). By default, n=2, so errors in code compiled with -g0 are not
reported unless one of the next two stack frames has debuggable code.
Values of n>2 request printing of errors deeper into non-debuggable
code. Use -hide 0 to request that all errors be printed, if you need to
debug code that was compiled with -g0, but note that this may yield a
large number of spurious errors for system library routines (which
should be ignored).
Heap History Options
Displays the usage of each block of memory allocated in the heap
(grouped by similar allocation-site stack trace). Each 32-bit word of
the first 1024 bytes is classified as one of the following: never writ‐
ten, always zero, a pointer, sometimes a pointer, or non-zero non-
pointer. Classifies the first n bytes of each heap object, instead of
the 1024 bytes selected by the -history option.
Leak Detection and Heap Tracing Options
Cancels the default -blocks new -after exit heap scan. Specifies that
each of the following -before or -after options will trigger a heap
scan that will print the stack trace of the allocation of: (1) each
heap leak--a heap block that is no longer referenced by any active
stack variable, static variable, or non-leak heap object; (2) each heap
object--a heap block that has not been deallocated but is still refer‐
enced.
Leak reports also distinguish leaks that are not referenced by
other leaks. They are often the root of a tree or otherwise the
cause of many other leaks.
Reports of new leaks or objects include only those that occurred
since the last scan.
The alternative options -leaks new|all and -objects new|all can
be used to select reports of just leaks or just objects, but
note that Third Degree may not distinguish between the refer‐
enced and unreferenced heap blocks reported by -blocks appropri‐
ately for some applications. Specifies that each of the follow‐
ing -before or -after options will trigger a heap scan every n
calls to the named procedure, instead of the default n = 1.
Triggers a heap scan at calls to (-before) or returns from
(-after) the procedure proc, of the type defined by the last
-blocks, -leaks, or -objects option, and of the frequency
defined by the last -every option. If the named procedure is in
a shared library, that shared library must be instrumented for
the scan to be triggered. Specify -all or -incobj to instrument
the shared library; but note that, in a multi-threaded program,
-after exit is the only trigger possible for system library pro‐
cedures, because libpthread and the libraries it uses are always
excluded from instrumentation. See the -ignore option for use
of the wildcard character (*) in the procedure name. Leaks or
objects that comprise less than the specified percentage (%) of
the total leaked or allocated heap will be reported as a single
subtotal, instead of being described individually. The default
is 1.0 During a heap scan, words of memory will be masked with
the specified hexadecimal value addr before being matched
against the existing address ranges of heap objects, to allow
for non-address information being encoded in unused bits of
pointer values. The default mask is ffffffffffffffff, meaning
the whole value is used.
Heap Management Options
The last n bytes of freed heap objects are kept in a queue instead of
being returned to the heap's free pool, as a means of trapping code
that accesses heap memory after it has been freed. By default,
1,000,000 bytes of freed objects are kept in this queue.
If the program allocated memory blocks larger than a million
bytes, the -free option should specify a size larger than the
largest block to ensure complete reporting of deallocation
errors. Adds n bytes of memory to the end of each allocated
heap object, so that the -invalid option can report overrun
errors. The default padding is 16 bytes.
Memory Access Error Debugging Options
Runs the instrumented program under the control of the dbx debugger,
instead of simply executing it. Usually, the procedure names, source
file names, and line numbers in the log file are sufficient to show the
location of a memory access error. However, if the procedure was not
instrumented, no report can be printed in the log file; and if the pro‐
cedure was not compiled -g, the relevant variable name, file name, and
line number can not be printed. In these cases, the debugger can be
used to examine further - for example, to look for the poisoned memory
values in variables and registers after a segmentation fault.
This option implies the -leaks cancel option, and if any -before
and -after options are specified, dbx will ignore SEGV, so
checking for leaks in separate non-debugging test runs is recom‐
mended.
The ladebug debugger can also be run on a program instrumented
with -g, if it is installed on the system. Reports any attempt
to access memory in a heap or stack area that is out of bounds
for application code. For example, the heap's free space data or
reserved parts of a stack frame. Addresses that are not in the
stack or heap (for example, memory mapped) are not checked.
Reports any attempt to read memory in a heap or stack area that
has not previously been initialized by application code or by
calloc. Note that this feature poisons all heap and/or stack
memory (as selected) with the value fff8a5a5, which will cause a
SEGV signal if used as a pointer or a NaN exception if used as a
floating point number, and which is likely to cause the program
to malfunction otherwise. The problem in the program must be
corrected before third's analysis can continue. The error
reports in the log file and the -g option can help locate the
cause of the problem. The cc -trapuv command offers a similar
but simpler capability for stack variables, without the need for
instrumentation.
The option value's keywords, described below, can be abbreviated
to one letter. They must be separated by a “+” sign: To indicate
which areas need to be checked. For warnings on every line of a
procedure that uses an uninitialized data value, instead of just
on the procedure's first use of it. Each execution of a given
line is still reported only once. For warnings on the first
access to the uninitialized memory, instead of on the first com‐
putation that uses it (which may follow several copy opera‐
tions). Changes the value used for poisoning from 0xfff8a5a5 to
the specified hexadecimal value. Tells third to disregard mem‐
ory errors (if error-code is represented by a three-letter
mnemonic, or an asterisk, or if none is specified) or leak and
object reports (if error-code is represented by leaks, objects,
or blocks) for a specific procedure (proc). Optionally, the
location of the error can be further qualified by specifying a
program or shared library (obj), source file (file), procedure
(proc), or source line (line). The three-letter mnemonic error-
code can be found at the start of all error reports in the log
file. The obj, file, and proc can contain a wildcard character
(*) to match any number of characters. If the procedure name
contains the character “*”, the latter must be escaped with “\”.
C++ procedures can omit the argument type list, though this will
match all overloaded procedures with that name. To select a
specific procedure, specify the full symbol name (as printed by
the nm command). Symbol names containing spaces, *, and so on
must be quoted. If a “blocks” error-code is specified, both leak
and object reports will be supressed.
Examples:
Ignore “read uninitialized heap” memory error in procedure main
of program program in source file main.c: -ignore pro‐
gram^main.c^main ruh
Ignore the same error in procedure memcpy of shared library
libc.so: -ignore libc.so^^memcpy ruh
Ignore “read invalid heap” memory error in procedure cat located
on line 6 of source file main.c: -ignore main.c^cat rih line 6
Ignore “read invalid heap” memory errors in procedure cat found
on any source line: -ignore cat rih
Do not report leaks if one of the stack trace entries points to
procedure proc1 on line 6 of source file array.cxx: -ignore
array.cxx^proc1 leaks line 6
Suppress all leak or object reports if the stack trace entry
contains references to procedure proc2: -ignore proc2 blocks
Shared-Library Analysis Options
These options are needed for the -before, -after, -invalid, and -uninit
options to be applied to shared libraries. Analyzes all the shared
libraries in addition to the program's executable. If -all was speci‐
fied, does not analyze the shared library lib. Analyzes the shared
library lib. Searches for shared libraries in the specified directory
before searching the default directories. Use the same options that
were used when linking the program with ld.
Execution Control Options
Executes the instrumented program, even if no arguments are specified.
By default, the program is just instrumented for later execution. Exe‐
cutes the instrumented program, and runs more on the resulting file(s).
Prints the tool's version number.
NOTES
If third finds any previously instrumented shared libraries in the
working directory, it will reuse them if they meet current require‐
ments, to reduce re-instrumentation costs.
Forked subprocesses are analyzed if the -fork option is specified, and
their log file contains the process-id, as if -pids had been specified.
Errors detected in signal handlers for the top-level process and all
subprocesses are reported in a single program.sig.3log file if the
process's primary log file is busy. This ensures that the primary log
files have a defined format, tbough the format of program.sig.3log is
not defined when errors are reported concurrently.
Temporary instrumentation files are created in /tmp. Set the TMPDIR
environment variable to a different directory to create the files else‐
where, for example in a disk partition with more space.
RESTRICTIONS
Heap managers other than those of C and C++ are not supported.
Third Degree has the following leak reporting limitations: Sometimes,
some leaks may not be reported because old pointers were found in mem‐
ory. Selecting checks for uninitialized heap memory may reduce this.
Any degree of optimization will skew results, because instructions that
the compiler considers non-essential may be optimized away.
Third Degree may print warnings for apparent problems that are really
harmless and do not need to be fixed. Such reports can be ignored, or
the -g option can be used to check the accuracy of the report by run‐
ning the instrumented program under the control of the dbx debugger.
If desired, the -ignore option can be used to prevent selected warnings
being printed, in much the same way as lint(1) provides options to sup‐
press certain of its static checks. For example, third cannot always
distinguish the following valid behavior from invalid behavior, result‐
ing in false reports: Initialization of variables and structure members
that are less than 32 bits in size (for example, C bit-field, char, and
short types). Use of actual data whose value matches the poison value
(0xfff8a5a5) that third uses to detect reading of uninitialized vari‐
ables. Nonstandard behavior of assembler and system procedures that
have been highly optimized for performance. Variable size stack
frames.
The procedures in system libraries that are used by the POSIX threads
package can not be instrumented in multi-threaded programs.
Approximate performance estimates are as follows but will vary accord‐
ing to the application and the machine's CPU count, type, and clock
rate. Third Degree instrumentation takes ~3s per Mb of program file on
a 500-MHz EV6 (21264) Alpha system, using ~15 Mb of memory plus another
~25 Mb per Mb of the largest program file. Large procedures can
increase both time and memory usage significantly. By default the
instrumented files are ~1.5 Mb larger than the originals and run ~3
times slower, but adding the -invalid or -uninit option can make the
files ~2.5 times larger and make them run ~10 times slower. The program
executable also includes data for each instrumented shared library --
~30% of the library size.
FILES
Instrumented version of program produced by third. Error log file pro‐
duced by program.third. Instrumented shared libraries produced by
third. Temporary file created and deleted in the current and -dirname
path directories.
SEE ALSO
Introduction: prof_intro(1)atom(1), cc(1), dbx(1), dlopen(3), dxheap(1), emacs(1), fork(2), lade‐
bug(1), ld(1), nm(1), pthread(3) (dxheap is available as an option.)
Programmer's Guide
third(1)
