# Code optimization The LLVM compilers provide many tools and features for improving the size or speed of the generated object code. ## Software pipelining The LLVM compilers include support for software pipelining. By default, pipelining is disabled; to enable it, use the `-O3` option (commonly used). Software pipelining attempts to increase instruction-level parallelism by dividing a loop into three distinct sections: - The *prolog* executes the initial loop iterations that are necessary to set up the kernel section. - The *kernel* executes the loop steady state (which is structured to execute more efficiently than the equivalent non-pipelined loop code). - The *epilog* executes the final loop iterations that are necessary to complete any operations left undone by the kernel section. The following example shows how software pipelining works: // Vector multiply and accumulate int foo(int *a, int *b, int n) { int i; int sum = 0; for (i = 0; i < n; i++) { sum += a[i] * b[i]; } return sum; } Copy to clipboard Without software pipelining, the compiler generates the following code for the loop: { loop0(.LBB0_2, r2) } .LBB0_2: { r2 = memw(r0++#4) r4 = memw(r1++#4) } { r3 += mpyi(r4, r2) nop }:endloop0 Copy to clipboard Each loop iteration executes two instruction packets: - The first packet loads the array elements a[i] and b[i]. - The second packet multiples the two array elements and accumulates the result. The instructions in the loop cannot be stored in a single packet, because the multiply-and- accumulate instruction depends on values that are loaded in the same loop iteration. With software pipelining enabled, the compiler generates the following code for the loop: { loop0(.LBB0_2, r2) } // prolog, iteration 0 { r2 = memw(r0++#4) r4 = memw(r1++#4) } // kernel, iteration 1 to n-1 .LBB0_2: { r3 += mpyi(r4, r2) r2 = memw(r0++#4) r4 = memw(r1++#4) }:endloop0 // epilog, iteration n-1 { r3 += mpyi(r4, r2) } Copy to clipboard In this generated code example, the prolog section (which is executed only once) performs the initial load of the array elements `a[0]` and `b[0]`. The kernel section (which executes in the loop) multiplies and accumulates the currently- loaded array element values, and then loads the next set of array element values. The epilog section (which is executed only once) performs the final multiply-and-accumulate of the values from the last iteration of the kernel (`a[n-1]` and `b[n-1]`). Software pipelining enables the main loop code in this example to be stored in a single instruction packet (rather than the two packets required in the non-pipelined version), because the multiply-and-accumulate instruction is able to operate on values that were already loaded in the previous loop iteration. ## Merge functions The LLVM compilers include support for function merging. By default, this optimization is disabled; to enable it, use the -fmerge-functions option (see [Code generation](https://docs.qualcomm.com/doc/80-N3040-23/topic/use-the-compilers.html#hccpp-compiler-options-code-gen-sub)). Function merging attempts to improve code size by merging functions that are equivalent or differ in only a few instructions. The optimization uses a number of heuristics to determine whether it is worthwhile to merge a pair of functions. For instance, very small functions or functions with significant differences are usually not merged. The following example shows how function merging works: int f1(int a, int b) { int f2(int a, int b) { int x; int x; x = a + 4; x = a + 10; return x * b; return x * b; } } Copy to clipboard Function merging determines that functions `f1` and `f2` are similar, and replaces them with the following functions: int f1__merged(int a, int b, int choice) { int x; if (choice) x = a + 10; else x = a + 4; return x * b; } int f1(int a, int b) { return f1__merged(a, b, 0); } int f2(int a, int b) { return f1__merged(a, b, 1); } Copy to clipboard This example is for illustration purposes only. In practice, the optimizer would determine that functions `f1` and `f2` are too small to be worth merging. Note Because function merging might have a negative impact on program performance, it is disabled by default, and becomes enabled only when it is specified explicitly. ## Automatic vectorization The compiler supports automatic vectorization for HVX. Auto-vectorization attempts to identify repeated operations on individual data elements (scalars) and replace them with instructions that operate on sequences of elements (vectors). A typical example is a loop that performs certain calculation on an array of input data. Auto-vectorization is disabled by default, even when compiler optimizations are enabled with `-O`. Auto-vectorization can be turned on with the compiler option, `-fvectorize`. The code generation for HVX must also be enabled via `-mhvx`. Because the default HVX vector length depends on the specific HVX version, we strongly recommend using `-mhvx-length` to specify the vector length. In addition to vectorizing integer scalars, auto-vectorization supports vectorization of floating-point scalars to their HVX equivalents. The `hvx-qfloat` option is enabled by default. The `-fvectorize` switch automatically uses qfloat instructions (if available) unless directed not to. You can tell the vectorizer to *not* use qfloat instructions by using the `-mno-hvx-qfloat` flag. Example of a compiler invocation that enables auto-vectorization: hexagon-clang -O2 -fvectorize -mcpu=v65 -mhvx=v65 -mhvx-length=128b -c file.c Copy to clipboard In summary, to enable auto-vectorization: 1. Enable compiler optimizations (`-O1`, `-O2`, and so on). 2. Enable code generation for HVX (`-mhvx`, `-mhvx-length`). 3. Turn on the auto-vectorizer (`-fvectorize`). 4. (Optional) Disable qfloat instructions (`-mno-hvx-qfloat`). **Example 1** Consider the following C code: void test(int *a, int *b, int n) { for (int i = 0; i != n; ++i) a[i] += b[i]; } Copy to clipboard Compile it with the following options: hexagon-clang -O1 -fvectorize -mv68 -mhvx -mhvx-length=128b -S test.c Copy to clipboard The following assembly code (for the loop) is produced: .LBB0_5: // %vector.body // =>This Inner Loop Header: Depth=1 { v0 = vmemu(r5+#0) } { v1 = vmemu(r4++#1) } { v2.w = vadd(v0.w,v1.w) } { nop vmemu(r5++#1) = v2 } :endloop0 Copy to clipboard The compiler provides options that allow you to get feedback about vectorization. More specifically: - `-Rpass=loop-vectorize` prints a remark when a given loop was vectorized - `-Rpass-missed=loop-vectorize` prints a remark when a loop was not vectorized - `-Rpass-analysis=loop-vectorize` displays reasons why a given loop was not vectorized **Example 2** Consider the same C code as in [Example 1](https://docs.qualcomm.com/doc/80-N3040-23/topic/code-optimization.html#hccpp-auto-vector-ex1-bookmark), with a small modification: int foo(int x); void test(int *a, int *b, int n) { for (int i = 0; i != n; ++i) a[i] = foo(b[i]); } Copy to clipboard Because the loop contains a function call, the compiler is not expected to be able to vectorize it. Using `-Rpass-missed=loop-vectorize`: hexagon-clang -O2 -fvectorize -mv68 -mhvx -mhvx-length=128b -S test.c -Rpass-missed=loop-vectorize Copy to clipboard Shows: test.c:3:36: remark: loop not vectorized [-Rpass-missed=loop-vectorize] void test(int *a, int *b, int n) { for (int i = 0; i != n; ++i) ^ Copy to clipboard And using `-Rpass-analysis=loop-vectorize`: hexagon-clang -O2 -fvectorize -mv68 -mhvx -mhvx-length=128b -S test.c -Rpass-analysis=loop-vectorize Copy to clipboard Shows why: test.c:4:12: remark: loop not vectorized: call instruction cannot be vectorized [-Rpass-analysis=loop-vectorize] a[i] = foo(b[i]); ^ test.c:3:36: remark: loop not vectorized: instruction cannot be vectorized [-Rpass-analysis=loop-vectorize] void test(int *a, int *b, int n) { for (int i = 0; i != n; ++i) ^ Copy to clipboard ### Programming considerations The data types that can be vectorized are those that correspond to the HVX data types (8-bit, 16-bit, and 32-bit signed and unsigned integers). Operations that can be vectorized are loads, stores, arithmetic operations (except division), bitwise operations, and shifts. Floating-point types cannot be vectorized. An ideal candidate for vectorization is a loop body that only contains operations that can be vectorized and no control flow. Again consider the C code from [Example 1](https://docs.qualcomm.com/doc/80-N3040-23/topic/code-optimization.html#hccpp-auto-vector-ex1-bookmark): void test(int *a, int *b, int n) { for (int i = 0; i != n; ++i) a[i] += b[i]; } Copy to clipboard This time use `-O2` and `-Rpass=loop-vectorize`: hexagon-clang -O2 -fvectorize -mcpu=v65 -mhvx=v65 -mhvx-length=128b -c test.c -Rpass=loop-vectorize test.c:2:3: remark: vectorized loop (vectorization width: 32, interleaved count: 2) [-Rpass=loop-vectorize] for (int i = 0; i != n; ++i) ^ Copy to clipboard The remark reveals the two parameters that describe how a loop is vectorized: vectorization width and interleave count. The vectorization width is the number of scalar operations that were grouped together into a single vector operation. Interleaving is conceptually similar to unrolling, and it refers to interleaving operations from consecutive iterations. In this example, the vectorization width is 32, which is expected because in 128-byte mode, an HVX vector register holds 32 four-byte integers. The interleave count is 2, meaning that for each statement in the loop that can be vectorized, two vectorized statements are generated. The result of loop vectorization is typically more than just the vectorized loop body: - Function parameters `a` and `b` can refer to overlapping memory areas, and they might not be aligned to a vector alignment. - Iteration count `n` might not be a multiple of vectorization width, or it might be too small for even a single vectorized statement. To ensure correctness, the compiler inserts checks for memory aliasing, uses unaligned memory accesses if required, and generates scalar epilog loops to finish the *leftover* loop iterations that were not executed by the vectorized code. However, this can reduce performance or increase code size. There are several ways you can mitigate these issues: - Use aligned data and inform the compiler about the alignment. For example, the previous code in `test1` can be modified as follows: void test2(int *a __attribute__((align_value(128))), int *b __attribute__((align_value(128))), int n) { for (int i = 0; i != n; ++i) a[i] += b[i]; } Copy to clipboard The `align_value` attribute indicates that the pointer parameter value is aligned to a 128-byte boundary. - Use the `restrict` keyword on pointer parameters. The code example can be further modified as follows: void test3(int *restrict a __attribute__((align_value(128))), int *restrict b __attribute__((align_value(128))), int n) { for (int i = 0; i != n; ++i) a[i] += b[i]; } Copy to clipboard This modification eliminates the need for a check of memory aliasing. Note The `restrict` keyword only exists in C. Clang supports the equivalent keyword, `__restrict`, in C++ (also in C) as an extension. - `__builtin_assume()` can be used to provide the properties of the inputs. `__builtin_assume` is used to provide the optimizer with a boolean invariant that is defined to be true. void test4(int *restrict a attribute ((align_value(128))), int *restrict b attribute ((align_value(128))), int n) { __builtin_assume(n % 32 == 0); __builtin_assume(n > 63); __builtin_assume((int)a % 128 == 0); __builtin_assume((int)b % 128 == 0); for (int i = 0; i != n; ++i) a[i] += b[i]; } Copy to clipboard - Clang allows you to specify loop optimization hints via a special `#pragma` clang loop construct. For example, to reduce the code size in `test3`, you can suggest that neither interleaving nor unrolling is to be performed: void test5(int *restrict a __attribute__((align_value(128))), int *restrict b __attribute__((align_value(128))), int n) { #pragma clang loop interleave(disable) #pragma clang loop unroll(disable) for (int i = 0; i != n; ++i) a[i] += b[i]; } Copy to clipboard The full list of such `#pragmas` is at [http://clang.llvm.org/docs/LanguageExtensions.html#extensions-for-loop-hint-optimizations](http://clang.llvm.org/docs/LanguageExtensions.html#extensions-for-loop-hint-optimizations). - The vector memory copy routine is significantly faster than the version that existed earlier. The vector memory copy routine can be used using the `-mllvm -enable- hexagon-vmemcpy=true` flag. - Enable Loop Interchange pass with the `-mllvm -enable-loopinterchange=true` flag to have the best chance of vectorizing the matrix multiplication kernels. - Use the `-ffast-math` clang flag to vectorize floating point kernels. This option lets the compiler make aggressive, potentially-lossy assumptions about floating-point math. Note Qfloat instructions have lower accuracy than IEEE instructions. For details, see the information about qfloat in the *Hexagon Programmer’s Reference Manual*. ## Link-time optimization Link-time optimization (LTO) comprises a set of powerful optimizations that are performed during the linking stage of compilation across multiple files or libraries. There are two variants of LTO: full LTO and ThinLTO. Note LTO is applicable to the 8.0 tools release and later. Support for ThinLTO is available with the 8.2 tools release and later. LTO expands the scope of optimizations from an individual source file to the entire program (or at least to all the files and libraries visible at link time and marked for LTO processing). This enables deeper compiler analysis (such as better alias analysis) and more effective code transformations (such as function inlining and dead code elimination), which can result in improved performance and code size. To mark a source file or a library for later full LTO or ThinLTO processing, it must be compiled with the `-flto` or `-flto=thin` option respectively. The compiler then produces a file containing the LLVM compiler’s intermediate representation (also known as *bitcode*) instead of the usual object code format. This file can be subsequently used in a final link step that then performs inter-module code optimizations on the file contents. Most of the optimization work is done at the link time. The linker can recognize that some of the input files or libraries contain bitcode and invoke the compiler on the combined scope for global processing. Note A mixture of object and bitcode inputs is possible, and the optimization scope will be limited to those inputs containing bitcode. You can choose an optimization level at the individual file stage, such as `-Os`, `-O1`, and `-O3`. This optimization level is used for bitcode optimization. At the LTO stage, the same optimizations are always be applied, but when necessary, `-Os` is respected. You can specify an optimization level for code generation (back end) via `-flto-options=codegen=-Ox`, where the following choices are available for `x`: - `0` = none - `1` = less - `2` = default - `3` = aggressive These values will apply regardless of the optimization level selected for the individual file. Typically, LTO results in an increase in build time due to its larger optimization scope. full LTO, in particular, does not support incremental builds. If one of the bitcode input files is changed, the linker triggers recompilation of all bitcode input files. ThinLTO, described in [ThinLTO](https://docs.qualcomm.com/doc/80-N3040-23/topic/code-optimization.html#hccpp-link-time-optimize-thinlto-sub), does not have this limitation. ### ThinLTO ThinLTO is a variant of LTO that significantly reduces build time compared to full LTO. It achieves this at the expense of performing fewer cross-module optimizations than full LTO. The main benefit of ThinLTO is the ability to inline functions across file boundaries, which can improve runtime performance in many applications. Enable ThinLTO by passing the `-flto=thin` option to the compiler instead of `-flto`. Unlike full LTO, ThinLTO also supports incremental builds. To enable support for incremental builds, use the `-flto-options=cache=` linker option to create a cache directory in the build directory. This cache directory must be unused; as part of the cleanup operation for the build system, remove the contents of this directory. ### Simple LTO usage example In `file_01.c`: int foo (int val) { return val << 2; } Copy to clipboard In `file_02.c`: int foo(int); int bar (int val) { return foo(val); } Copy to clipboard In `file_03.c`: #include int bar(int); int main(int argc) { return printf("result %d\n", bar(argc)); } Copy to clipboard To perform LTO on these files, enter the following sequence of commands. hexagon-clang -flto -c file_01.c file_02.c file_03.c hexagon-link file_01.o file_02.o file_03.o Copy to clipboard Note This example is one of several possible option combinations; architecture options are not shown for simplicity. This sequence allows the compiler to perform cross-file optimizations that result in an executable equivalent to compiling a single file with the following content: file_lto.c #include int main(int argc) { return printf("result %d\n", (argc<<2)); } Copy to clipboard ## Profile-driven section assignment Automated section assignment injects a user-specified infix (such as hot) into the section names of symbols that exceed a certain profile count threshold. For example, you can define a rule that causes function `foo` in section `.text.foo`, with a profile count of `1000`, to be placed in `.text.hot.foo`. You can add a rule to the linker script that matches `.text.hot.*` and places these input sections into a different output section. ### Configuration Two additional input files for the compiler, or, in the case of LTO, the linker, are used to configure section assignment: a placement rules file and a placement profile. #### Placement rules file The placement rules determine which symbols are tagged with an infix, what this infix should be, and above which profile count threshold the tagging should occur. 1|FUNCS:quitehot:1 2|FUNCS:prettyhot:100 3|FUNCS:veryhot:1000 4|VARS:hotvars:1 Copy to clipboard Explanation: - Line 1: Add the infix, `quitehot`, to the linker input section of all functions with a profile entry count of `[1..100)`, where 100 is the starting point of the next rule. For example, a function `bla` with entry count 5 that would have been in the `.text.bla` section is now placed in the `.text.quitehot.bla` section. In the linker script, this can then be matched by a rule in the `.text.resident` section for `.text.quitehot.*`. - Line 2: Add the infix, `prettyhot`, for functions with an entry count of `[100...1000)`. - Line 3: Add the infix, `veryhot`, for functions with an entry count of `[1000..infinity)`. - Line 4: Add the infix, `hotvars`, to any variables with access counts of `[1..infinity)`. For example, a variable that would have gone to `.bss.somevar` will now go to `.bss.hotvars.somevar`. A variable that would have gone to `.rodata.somevar` will now go to `.rodata.hotvars.somevar`. Functions and variables that are not matched by any rule or that have explicit sections set (using `__attribute__((section(.somesection))`) are not touched. #### Placement profile file This file provides profile counts for variables and functions. - For functions, the profile counts can also come from the `SamplePGO` profile that is used for other profile-guided optimizations in the compiler. If a symbol has an entry in both the sample profile and placement profile, the placement profile takes precedence for section assignment. - For variables, the placement profile is the only way of providing profile counts because variables are not represented in the sample profile format. The format is straightforward (`SymbolName:ProfileCount`): 1|myfunction:1000 2|somevariable:42 Copy to clipboard Also, a section assignment can include the profile count in the section name of every symbol being placed. This feature is required for section budgeting (see the *Qualcomm Hexagon Utilities User Guide* (80-N2040-15)). The profile count is encoded in HEX format and is inserted immediately after the infix. In the example, the bla function in the `.text.bla` section with count 5 is assigned to the `.text.quitehot._F5.bla` section, where `._Fxxx` indicates the profile count. ### Command line options The following command line options specify the configuration files and enable PGO counts in section names. - For non-LTO builds, add the following command line options to every invocation of `hexagon-clang` (for example, via `CFLAGS`): -mllvm -pgo-section-rules=FILENAME -mllvm-pgo-placement-profile= FILENAME -mllvm -enable-pgo-count-infix Copy to clipboard - For LTO builds, add the following command line options to every invocation of the linker (for example, via `LFLAGS`): -flto-options=codegen="-pgo-section-rules=FILENAME -pgo-placement- profile=FILENAME -enable-pgo-count-infix" Copy to clipboard ### Verify the section assignment To verify the correct operation of section assignment, inspect the map file of the linker. If no assignment takes place, check that the options actually reach the hexagon-link invocation. With SCons, use the `--verbose=2` build option. Last Published: Oct 06, 2026 [Previous Topic HVX and floating-point code generation](https://docs.qualcomm.com/bundle/publicresource/80-N3040-23/topics/use-the-compilers.md) [Next Topic Compiler security tools](https://docs.qualcomm.com/bundle/publicresource/80-N3040-23/topics/compiler-security-tools.md)