# Profiling support in runtime

The AIC100 stack provides mechanism to get performance metrics of
key milestones through the inference cycle to triage low
performance and for general monitoring.

The AIC100 stack classifies profiling data into two broad categories:

1. Host metrics: An inference passes through various software layers on the
host before it makes it to the network on device. Examining the
performance on the host helps identify if tweaking the network preprocessing and postprocessing stages or the host side multi-threading knobs is required.
2. Network/device metrics: Network metrics include a wide-range of
information from whole network execution time to detailed
operator-level performance. Use this information to
make the most out of the existing network or to optimize the network itself.

    You must enable network performance collection during network compilation itself because it’s not baked into the network by
default. The amount of network performance details present depends on the parameters
passed to the AIC compiler.

## Profiling report types

You can use the AIC100 software stacks to request the following types (layouts) of profiling information:

### Latency type

Latency type is a CSV style table of key of AIC100 stack. It contains both
host and device side information.

### Trace type

Trace type is a JSON formatted Chrome trace data. It can be viewed on
any interface that consumes Chrome traces. It contains both host and device
side information.

## Profiling report collection methods

Request profiling information from the AIC100 stack in two ways:

1. Num-iter based profiling
2. Duration based profiling

The core content is the same in both mechanisms, just the delivery mechanism changes.

### Num-iter based profiling

Alias: Legacy profiling

The fundamental idea for num-iter based profiling is that you create a profiling handle where you specify the program to be profiled and the number of inferences
that need to be sampled. Only one program can be profiled by a specified profiling handle. If you wants to profile multiple
programs, you must create multiple profiling handles, one for each program.

To create a profiling handle, specify the following:

1. The program to profile.
2. Number of samples to collect.
3. Profiling callback.
4. Type of profiling output type (latency or trace).

After you start the profiling, the profiling stops and calls
the callback you specified in the previous step when either of the following occurs:

1. The number of samples requested by user has been collected.
2. You explicitly stop profiling. In this case, the number of samples
collected might be less than the number specified during profiling handle
creation.

After the profiling is stopped, you can start profiling again
using the same handle. The behavior of the profiling is the same as when the
handle is triggered for the first time.

Note in the APIs how at the creation of profiling handle, the user needs
to be aware of the program needs to be profiled. Only one program can be
profiled by a specified profiling handle. If you want to profile multiple
programs, create multiple profiling handles, one for each
program.

Last Published: Aug 25, 2026

[Previous Topic
QAicInferenceSet Example](https://docs.qualcomm.com/bundle/publicresource/80-99100-3/topics/index_qaic-inference-set-group.md) [Next Topic
Runtime](https://docs.qualcomm.com/bundle/publicresource/80-99100-3/topics/index_runtime.md)