# Qualcomm Linux features
Source: [https://docs.qualcomm.com/doc/80-70014-27/topic/platform_software_features.html](https://docs.qualcomm.com/doc/80-70014-27/topic/platform_software_features.html)
This section provides information on the features of Qualcomm Linux including the
following topics:
- Metadata layers, which are included in the Qualcomm Linux
- Recipes and configurations in the Qualcomm Linux metadata layers
- Software features added Qualcomm Linux by Qualcomm
- Tools introduced by Qualcomm in the release
- Instructions on how to build Qualcomm Linux images and metadata layers
Note: The Qualcomm software
release is based on the Yocto Kirkstone release.
For the complete index of the Yocto Project documentation corresponding to the Kirkstone
release, see [https://docs.yoctoproject.org/4.0.18/singleindex.html](https://docs.yoctoproject.org/4.0.18/singleindex.html). If you are new to the
Yocto Project and want to know about builds, you must try the first build using the
Yocto Project documentation at [https://docs.yoctoproject.org/4.0.18/singleindex.html#document-brief-yoctoprojectqs/index](https://docs.yoctoproject.org/4.0.18/singleindex.html#document-brief-yoctoprojectqs/index).
The Qualcomm Linux environment consists of several community-maintained metadata layers
that deliver recipes, package groups, images, and configurations. The Qualcomm metadata
layers are stacked on the community layers to provide additional software components
required for Qualcomm reference devices.
## Overview of metadata layers
Source: [https://docs.qualcomm.com/doc/80-70014-27/topic/platform_software_features.html](https://docs.qualcomm.com/doc/80-70014-27/topic/platform_software_features.html)
The overview of metadata layers introduces the layers that are included in the manifest
[https://github.com/quic-yocto/qcom-manifest](https://github.com/quic-yocto/qcom-manifest). This manifest
includes all the layers that are required to reproduce the reference build. The layers
maintained by the Yocto Project are introduced in the following sections. Qualcomm
maintains the layers specific to Qualcomm reference devices, which depend on the
community layers to realize full functionality. The following figure shows the layers
included in the Qualcomm Linux release:
Figure : Qualcomm Linux metadata layers
Table : Qualcomm Linux metadata layers and descriptions
| Metadata layer | Description |
| --- | --- |
| `meta-qcom-distro` | This layer provides a reference distribution configuration for
Qualcomm products. Image recipes and package groups are defined in
this layer. |
| `meta-qcom-extras` | This layer is an optional metadata layer for registered users.
This layer enables source compilation of select components, which
are otherwise present as binary in
`meta-qcom-hwe`. |
| `meta-qcom-qim-product-sdk` | This layer provides Qualcomm's offering of Qualcomm Intelligent
Multimedia (QIM) and AI SDKs based on the GStreamer framework. This
includes a set of GStreamer plugins, sample applications for
multimedia and AI use cases. |
| `meta-qcom-realtime` | This layer contains changes specific to real-time kernel
configurations. |
| `meta-qcom-hwe` | This layer contains recipes that build software components for
Qualcomm device support and value-added software features applicable
to Qualcomm SoCs. |
| `meta-qcom` | This layer contains Qualcomm device support and upstream OSS
software components. |
| `meta-virtualization`
[https://git.yoctoproject.org/meta-virtualization](https://git.yoctoproject.org/meta-virtualization) | This layer contains packages for constructing OpenEmbedded
virtualized solutions and virtualization tool stacks such as dockerd
and Kubernetes. |
| `meta-selinux`
[https://git.yoctoproject.org/meta-selinux/](https://git.yoctoproject.org/meta-selinux/) | This layer enables SELinux. |
| `poky/meta`
[https://git.yoctoproject.org/poky/](https://git.yoctoproject.org/poky/) | This layer provides build tools and files that provide various
software components needed to build an embedded OS. |
| `meta-openembedded`
[https://git.openembedded.org/meta-openembedded/](https://git.openembedded.org/meta-openembedded/) | This layer provides a cross-compile environment. |
### Qualcomm BSP metadata layers
Source: [https://docs.qualcomm.com/doc/80-70014-27/topic/platform_software_features.html](https://docs.qualcomm.com/doc/80-70014-27/topic/platform_software_features.html)
The following layers represent Qualcomm BSP metadata:
- [meta-qcom](https://docs.qualcomm.com/doc/80-70014-27/topic/platform_software_features.html#qualcomm_bsp_metadata_layers__section_ebf_3dd_51c_vinayjk_03-18-24-1703-23-746)
- [meta-qcom-hwe](https://docs.qualcomm.com/doc/80-70014-27/topic/platform_software_features.html#qualcomm_bsp_metadata_layers__section_tg5_3dd_51c_vinayjk_03-18-24-1703-34-31)
- [meta-qcom-realtime](https://docs.qualcomm.com/doc/80-70014-27/topic/platform_software_features.html#qualcomm_bsp_metadata_layers__section_atv_3dd_51c_vinayjk_03-18-24-1703-35-26)
- [meta-qcom-distro](https://docs.qualcomm.com/doc/80-70014-27/topic/platform_software_features.html#qualcomm_bsp_metadata_layers__section_yzv_3dd_51c_vinayjk_03-18-24-1703-35-206)
- [meta-qcom-extras](https://docs.qualcomm.com/doc/80-70014-27/topic/platform_software_features.html#qualcomm_bsp_metadata_layers__section_n3w_3dd_51c_vinayjk_03-18-24-1703-35-429)
- [meta-qcom-qim-product-sdk](https://docs.qualcomm.com/doc/80-70014-27/topic/platform_software_features.html#qualcomm_bsp_metadata_layers__section_rpw_3dd_51c_vinayjk_03-18-24-1703-35-615)
### meta-qcom
The `meta-qcom` metadata layer is hosted at [git.yoctoproject.org](http://git.yoctoproject.org) and provides recipes to build the Qualcomm OSS. The
software image is built using the following recipes from `meta-qcom`:
| `recipes-devtools/qdl/qdl_git.bb` | Qualcomm downloader (QDL) flashing tool.
QDL communicates
with USB devices with ID 05c6:9008 to upload a flash loader
and uses this to flash the images. |
| --- | --- |
| `recipes-support/pd-mapper/pd-mapper_git.bb` | Qualcomm protection domain mapper application.
pd-mapper is
an implementation for the protection domain mapper
service.
The protection domain mapper service allows
user space applications to access remote processors on
Qualcomm SoCs. |
| `recipes-support/qrtr/qrtr_git.bb` | Qualcomm QRTR applications and library. |
| `recipes-support/initrdscripts/initramfs-module-copy-modules_1.0.bb` | `initramfs-framework` module for copying
kernel modules from `initramfs` to
`rootfs`. |
### meta-qcom-hwe
The `meta-qcom-hwe` metadata layer is hosted at [https://github.com/quic-yocto/meta-qcom-hwe](https://github.com/quic-yocto/meta-qcom-hwe). It provides
additional software support for enabling Qualcomm device.
- **BitBake classes**
See [https://github.com/quic-yocto/meta-qcom-hwe/tree/kirkstone/classes](https://github.com/quic-yocto/meta-qcom-hwe/tree/kirkstone/classes) for an introduction to the BitBake
classes.
| `classes/uki.bbclass` | Implements logic to pack a kernel image as a
unified kernel image (UKI).
The UKI is composed of
the following:
To understand the UKI image, see [systemd-boot](https://docs.qualcomm.com/doc/80-70014-27/topic/platform_software_features.html#systemd_boot) |
| --- | --- |
| `classes/image-efi.bbclass` | Implements logic to generate images to pack in an
extensible firmware interface (EFI) system
partition.
The generated image contains the
device tree blob (DTB), kernel image, and
`systemd-boot` manager
binary. |
| `classes/qprebuilt.bbclass` | Implements logic to use a prebuilt package
instead of fetching and compiling the source. For
any recipe inheriting **qprebuilt**, this class
allows to unpack binary packed in a
`tar.gz` archive and provides these
to be packaged using a BitBake task. |
| `classes/qmodule.bbclass` | By default, Qualcomm BSP enforces kernel module
signing by enabling the
`CONFIG_MODULE_SIG_FORCE` in the
kernel. However, some out-of-tree modules might not
be properly signed. To avoid module loading issues,
the `qmodule.bbclass` inspects all
packages providing kernel modules and signs them if
they are not already signed. |
- **Machine configurations**
Qualcomm Linux machine configuration files
are available here: [https://github.com/quic-yocto/meta-qcom-hwe/tree/kirkstone/conf/machine](https://github.com/quic-yocto/meta-qcom-hwe/tree/kirkstone/conf/machine).
| `conf/machine/qcm6490.conf` | Machine configuration for the development board,
with Qualcomm QCM6490, such as RB3 Gen 2 core kit,
RB3 Gen 2 video mezzanine kit, and RB3 Gen 2 vision
mezzanine kit. |
| --- | --- |
- **Kernel cmdline**
To update the kernel command line
parameters, update the `KERNEL_CMDLINE_EXTRA`
variable.
`KERNEL_CMDLINE_EXTRA`
`DBG_CMDLINE
=
"${@oe.utils.conditional('DEBUG_BUILD','1','qcom_scm.download_mode=1
slub_debug=FZP,zs_handle,zspage;
FZPU','',d)}"`
`KERNEL_CMDLINE_EXTRA ?=
"root=/dev/disk/by-partlabel/system rw rootwait
${CONSOLE_CMDLINE} pcie_pme=nomsi
kernel.sched_pelt_multiplier=4 rcupdate.rcu_expedited=1
rcu_nocbs=0-7 kpti=off kasan=off kasan.stacktrace=off
no-steal-acc page_owner=on ${DBG_CMDLINE}
swiotlb=128"`
- **Include DTB**
To include the corresponding DTB, update
`KERNEL_DEVICETREE`:
`KERNEL_DEVICETREE
= " \`
`qcom/qcm6490-addons-idp.dtb
\`
`qcom/qcs5430-fp1-addons-rb3gen2.dtb
\`
`qcom/qcs5430-fp1-addons-rb3gen2-vision-mezz.dtb
\`
`qcom/qcs5430-fp1p5-addons-rb3gen2.dtb
\`
`qcom/qcs5430-fp1p5-addons-rb3gen2-vision-mezz.dtb
\`
`qcom/qcs5430-fp2-addons-rb3gen2.dtb
\`
`qcom/qcs5430-fp2-addons-rb3gen2-vision-mezz.dtb
\`
`qcom/qcs6490-addons-rb3gen2.dtb
\`
`qcom/qcs6490-addons-rb3gen2-video-mezz.dtb
\`
`qcom/qcs6490-addons-rb3gen2-vision-mezz.dtb
\`
`qcom/qcs6490-addons-rb3gen2-ptz-mezz.dtb
\`
`qcom/qcs6490-addons-rb3gen2-ia-mezz.dtb
\`
`"`
- **Include additional DTBOs**
To include the additional list
of device-tree overlay (DTBO)s to be overlaid on top of the
base kernel device-tree, update
`KERNEL_TECH_DTBOS`:
`#
Additional dtbo to overylay on top of kernel devicetree
filesKERNEL_TECH_DTBOS[qcm6490-addons-idp] =
"qcm6490-graphics.dtbo qcm6490-display.dtbo
qcm6490-camera.dtbo qcm6490-bt.dtbo
qcm6490-wlan-idp.dtbo
qcm6490-video.dtbo"`
`KERNEL_TECH_DTBOS[qcs6490-addons-rb3gen2]
= "qcm6490-graphics.dtbo qcm6490-wlan-rb3.dtbo
qcm6490-display-rb3.dtbo qcm6490-bt.dtbo
qcm6490-video.dtbo"`
`KERNEL_TECH_DTBOS[qcs6490-addons-rb3gen2-video-mezz]
= "qcm6490-graphics.dtbo qcm6490-camera-rb3.dtbo
qcm6490-display-rb3.dtbo qcm6490-wlan-rb3.dtbo
qcm6490-bt.dtbo
qcm6490-video.dtbo"`
`KERNEL_TECH_DTBOS[qcs6490-addons-rb3gen2-vision-mezz]
= "qcm6490-graphics.dtbo qcm6490-camera-rb3.dtbo
qcm6490-display-rb3.dtbo qcm6490-wlan-rb3.dtbo
qcm6490-bt.dtbo
qcm6490-video.dtbo"`
`KERNEL_TECH_DTBOS[qcs6490-addons-rb3gen2-ptz-mezz]
=
""`
`KERNEL_TECH_DTBOS[qcs6490-addons-rb3gen2-ia-mezz]
=
""`
`KERNEL_TECH_DTBOS[qcs5430-fp1-addons-rb3gen2]
= "qcs5430-graphics.dtbo qcm5430-camera-rb3.dtbo
qcs5430-wlan-rb3.dtbo qcm6490-bt.dtbo
qcm6490-display-rb3.dtbo
qcm6490-video.dtbo"`
`
KERNEL_TECH_DTBOS[qcs5430-fp1-addons-rb3gen2-vision-mezz]
= "qcs5430-graphics.dtbo qcm5430-camera-rb3.dtbo
qcs5430-wlan-rb3.dtbo qcm6490-bt.dtbo
qcm6490-display-rb3.dtbo
qcm6490-video.dtbo"`
`KERNEL_TECH_DTBOS[qcs5430-fp1p5-addons-rb3gen2]
= "qcs5430-graphics.dtbo qcm5430-camera-rb3.dtbo
qcs5430-wlan-rb3.dtbo qcm6490-bt.dtbo
qcm6490-display-rb3.dtbo
qcm6490-video.dtbo"`
`KERNEL_TECH_DTBOS[qcs5430-fp1p5-addons-rb3gen2-vision-mezz]
= "qcs5430-graphics.dtbo qcm5430-camera-rb3.dtbo
qcs5430-wlan-rb3.dtbo qcm6490-bt.dtbo
qcm6490-display-rb3.dtbo
qcm6490-video.dtbo"`
`KERNEL_TECH_DTBOS[qcs5430-fp2-addons-rb3gen2]
= "qcs5430-graphics.dtbo qcm5430-camera-rb3.dtbo
qcs5430-wlan-rb3.dtbo qcm6490-bt.dtbo
qcm6490-display-rb3.dtbo
qcm6490-video.dtbo"`
`KERNEL_TECH_DTBOS[qcs5430-fp2-addons-rb3gen2-vision-mezz]
= "qcs5430-graphics.dtbo qcm5430-camera-rb3.dtbo
qcs5430-wlan-rb3.dtbo qcm6490-bt.dtbo
qcm6490-display-rb3.dtbo
qcm6490-video.dtbo"`
The package, which is
used to generate DTBO, must be added to a
`MACHINE_EXTRA_RDEPENDS` variable to set
the dependency. For example,
`qcm6490-graphics.dtbo` is added to
`KERNEL_TECH_DTBOS` list above and the
corresponding package `graphicsdevicetree`,
which generates DTBO is added to
`MACHINE_EXTRA_RDEPENDS` variable as
follows:
`MACHINE_EXTRA_RDEPENDS += "
\`
`packagegroup-firmware-qcm6490
\`
`graphicsdevicetree
\`
`displaydevicetree
\`
`cameradtb
\`
`wlan-devicetree
\`
`btdevicetree
\`
`videodtb
\`
`"`
- **Firmware recipes**
Qualcomm Linux firmware recipe files are located
here: [https://github.com/quic-yocto/meta-qcom-hwe/tree/kirkstone/recipes-firmware](https://github.com/quic-yocto/meta-qcom-hwe/tree/kirkstone/recipes-firmware).
When the Qualcomm Linux source code is synced, the firmware recipes are
available in the following directory:
`/layers/meta-
qcom-hwe/recipes-firmware/firmware`.
- **Boot critical binaries**
Critical boot images are required to
boot the kernel on the device. The following firmware recipes
provide the SOC-specific boot firmware:
| `firmware-qcom-bootbins_1.0.bb` | Handles fetch, unpack, and deploy for
critical boot firmware binaries for compatible
targets.
For the QCM6490-based targets, the
files are packed in the
`QCM6490_bootbinaries.zip`
file. |
| --- | --- |
After the build completes, the firmware binaries
in these zip files are available for flashing in the following
directory:
`/build-qcom-wayland/tmp-glibc/deploy/images///`.
`firmware-qcom-bootbins_1.0.bb`
recipe fetches the boot binaries based on SRC\_URI, unpacks the
zip. The binaries can be found in the following directory:
`tmp-glibc/deploy/images//`.
- **Subsystem firmware binaries**
Qualcomm Linux includes
firmware binaries that are loaded and run on the corresponding
subsystems. As the Qualcomm SoC boots, individual subsystems
execute the firmware as they come out of reset:
| `firmware-qcom-hlosfw_1.0.bb` | Handles fetch, unpack, and install for
subsystem firmware binaries such as aDSP, cDSP,
modem, and WLAN.
The firmware files for
QCM6490-based targets are packed in
`QCM6490_fw.zip` file. |
| --- | --- |
`firmware-qcom-hlosfw_1.0.bb` recipe
fetches the subsystem firmware binaries from the remote server
based on SRC\_URI, unpacks the zip, and installs the firmware in
`rootfs`.
- **DSP libraries**
User space utilities refer to the DSP
libraries and these libraries must be available in the
`rootfs` image. The following firmware
recipes provide the SOC-specific DSP libraries:
| `firmware-qcom-dspso_1.0.bb` | Handles fetch, unpack, and install for DSP
libraries.
The libraries for QCM6490-based
targets are packed in
`QCM6490_dspso.zip`
file. |
| --- | --- |
`firmware-qcom-dspso_1.0.bb` recipe
fetches the DSP libraries from the remote server based on
SRC\_URI, unpacks the zip, and installs the DSP libraries in
`rootfs`.
- **Installation of boot, subsystem, and dspso**
When the
Qualcomm Linux source code is built, the build system uses the
firmware recipes to deploy the prebuilt firmware based on
MACHINE\_EXTRA\_RDEPENDS configuration, which is set in the
machine configuration file. For example, in
`layers/meta-qcom-hwe/conf/
machine/qcm6490.conf`, see the inclusion of
`packagegroup-firmware-qcm6490` in the
variable
MACHINE\_EXTRA\_RDEPENDS:
`MACHINE_EXTRA_RDEPENDS
+= " \`
`packagegroup-firmware-qcm6490
\`
`graphicsdevicetree
\`
`displaydevicetree
\`
`cameradtb
\`
`wlan-devicetree
\`
`btdevicetree
\`
`videodtb
\`
`"`
Note: The
`packagegroup-firmware-qcm6490` recipe is
present in
`/layers/meta-qcom-hwe/recipes-firmware/packagegroups/`,
and groups the firmware recipes to be used for image
generation.
When Qualcomm Linux is built, based on the
configuration in `qcm6490.conf` and package group
recipe file, the respective firmware recipes from
`/layers/meta-qcom-hwe/recipes-firmware/firmware`
are built.
- **Kernel recipes**
The Linux kernel recipe used by Qualcomm Linux is at
[https://github.com/quic-yocto/meta-qcom-hwe/tree/kirkstone/recipes-kernel/linux](https://github.com/quic-yocto/meta-qcom-hwe/tree/kirkstone/recipes-kernel/linux).
Qualcomm
Linux supports the 6.6 LTS kernel and is maintained through Yocto recipe
`linux-kernel-qcom_6.6.bb` at
`recipes-kernel/linux` under the
`meta-qcom-hwe` layer. This recipe refers to Qualcomm
kernel source, which is publicly hosted at [https://git.codelinaro.org/clo/la/kernel/qcom.git](https://git.codelinaro.org/clo/la/kernel/qcom.git). The
Qualcomm Linux kernel recipe is referred to as
`linux-kernel-qcom`.
For example,
`qcm6490.conf` is a machine configuration file hosted
at `meta-qcom-hwe/conf/machine` and supports the QCM6490
machine, and that selects the kernel
`linux-kernel-qcom`.
***Kernel configuration***
The Qualcomm Linux kernel recipe uses
the following kernel configuration fragments:
| Kernel configuration fragments | Description |
| --- | --- |
| `/arch/arm64/configs/qcom_defconfig` | Base configuration that is aligned to
product/performance needs and upstream
kernel |
| `/arch/arm64/configs/qcom_debug.config` | Debug configuration fragment from upstream
kernel |
| `/arch/arm64/configs/qcom_addons.config` | Additional Qualcomm downstream additions on top
of upstream aligned base |
| `/arch/arm64/configs/qcom_addons_debug.config` | Qualcomm downstream debug enablement |
The Qualcomm Linux kernel recipe supports two build
variants, `perf` and `debug`, for QCM6490
machine configuration. The default build is `perf`.
| Build variant | Defconfig/config fragments |
| --- | --- |
| `Perf` | |
| `Debug` | |
You can modify the kernel build configurations by
updating KERNEL\_DEFCONFIG and KERNEL\_CONFIG\_FRAGMENTS variables in the
kernel recipe
`linux-kernel-qcom_6.6.bb`:
`KERNEL_DEFCONFIG
=
"${S}/arch/arm64/configs/qcom_defconfig"`
`KERNEL_CONFIG_FRAGMENTS:append
= "
${S}/arch/arm64/configs/qcom_addons.config"`
`KERNEL_CONFIG_FRAGMENTS:append
= " ${@oe.utils.vartrue('DEBUG_BUILD',
'${S}/arch/arm64/configs/qcom_debug.config', '',
d)}"`
`KERNEL_CONFIG_FRAGMENTS:append = "
${@oe.utils.vartrue('DEBUG_BUILD',
'${S}/arch/arm64/configs/qcom_addons_debug.config', '',
d)}"`
If the distro supports SELinux, then the SELinux
configuration fragments `selinux.cfg` and
`selinux_debug.cfg` are available at
`recipes-kernel/linux/linux-kernel-qcom`:
`SELINUX_CFG
= "${@oe.utils.vartrue('DEBUG_BUILD', 'selinux_debug.cfg',
'selinux.cfg',
d)}"`
`KERNEL_CONFIG_FRAGMENTS:append = "
${@bb.utils.contains('DISTRO_FEATURES', 'selinux',
'${WORKDIR}/${SELINUX_CFG}', '', d)}"`
Kernel modules
can be autoloaded for the Qualcomm platforms by updating the
KERNEL\_MODULE\_AUTOLOAD variable in the Qualcomm Linux kernel recipe. For
example, the CoreSight and STM modules are autoloaded:
`#
Coresight and STM modules for QDSS
functions`
`KERNEL_MODULE_AUTOLOAD +=
"coresight coresight-tmc
coresight-funnel"`
`KERNEL_MODULE_AUTOLOAD +=
"coresight-replicator coresight-etm4x
coresight-stm"`
`KERNEL_MODULE_AUTOLOAD +=
"coresight-cti coresight-tpdm coresight-tpda
coresight-dummy"`
`KERNEL_MODULE_AUTOLOAD +=
"coresight-remote-etm
coresight-tgu"`
`KERNEL_MODULE_AUTOLOAD +=
"stm_core stm_p_ost stm_console stm_heartbeat stm_ftrace
"`
- **Licenses**
Licenses used by recipes in `meta-qcom-hwe`
are listed at
`/meta-qcom-hwe/files/common-licenses`.
common-licenses/
├── BSD-3-Clause-Clear
├── GPLv2.0-with-linux-syscall-note
└── Qualcomm-Technologies-Inc.-ProprietaryCopy to clipboard
Yocto
provides support for automatically creating SPDX SBOM documents based on
image generation. To enable this the `create-spdx` class
is required to be inherited in `local.conf` as
follows:
INHERIT += "create-spdx"Copy to clipboard
After the class is inherited, the user can rebuild the image with
BitBake
command:
bitbake qcom-multimedia-imageCopy to clipboard
The
SPDX output is found under the following paths:
- For each recipe, the generated files are available in
`tmp/deploy/spdx/`
directory.
- The top level SPDX output file is
`tmp/deploy/images/MACHINE/-.spdx.json`.
### meta-qcom-realtime
The `meta-qcom-realtime` metadata layer is hosted at [https://github.com/quic-yocto/meta-qcom-realtime](https://github.com/quic-yocto/meta-qcom-realtime). This layer provides additional software support for building
a real-time kernel for the Qualcomm devices.
- **Kernel recipes**
Qualcomm Linux supports the 6.6 LTS kernel plus
real-time extensions and is maintained through the Yocto recipe
`linux-kernel-qcom-rt_6.6.bb` at
`recipes-kernel/linux` under the
`meta-qcom-realtime` layer. The pending preempt RT
patches can be found at [https://wiki.linuxfoundation.org/realtime/start](https://wiki.linuxfoundation.org/realtime/start). These
patches are fetched and applied on top of the Qualcomm Linux kernel
source, which is publicly hosted at [https://git.codelinaro.org/clo/la/kernel/qcom.git](https://git.codelinaro.org/clo/la/kernel/qcom.git).
The
Qualcomm Linux real-time recipe is referred to as
`linux-kernel-qcom-rt`. When compiling a real-time
kernel for the Qualcomm machines, `conf/layer.conf`
selects the kernel as
`linux-kernel-qcom-rt`.
***Kernel configuration***
KERNEL_CONFIG_FRAGMENTS:append = " ${WORKDIR}/qcom_rt.cfg"Copy to clipboard
- **Enabling `meta-qcom-realtime` in build**
To include
`meta-qcom-realtime` in the build, export
`meta-qcom-realtime` layer to EXTRALAYERS in
`bblayers.conf`as shown in the following steps:
1. Source the environment, following is an example to source the
environment for QCM6490 and `qcom-wayland`
distro:
MACHINE=qcm6490 DISTRO=qcom-wayland source setup-environmentCopy to clipboard
2. Open the `build-qcom-wayland/conf/bblayers.conf`
file and update the EXTRALAYERS variable as
follows:
EXTRALAYERS ?= " \
${WORKSPACE}/layers/meta-qcom-realtime \
"Copy to clipboard
3. Run the build command to rebuild with
`meta-qcom-realtime` as
follows:
bitbake qcom-multimedia-imageCopy to clipboard
### meta-qcom-distro
This layer provides a reference distribution configuration for Qualcomm products.
Image recipes and package groups are defined in this layer.
- **BitBake classes**
Following table provides an introduction to the
BitBake classes, which are available at [https://github.com/quic-yocto/meta-qcom-distro/tree/kirkstone/classes](https://github.com/quic-yocto/meta-qcom-distro/tree/kirkstone/classes).
| `image-adbd.bbclass` | Qualcomm Linux provides support for SSH and UART
serial shell. Use SSH or UART for accessing the
device.
Some users find it easier to use ADB for
debugging issues when IP interfaces are down or
large files are to be transferred. For them ADB is
provided as an option.
This class installs
adbd into the image.
The adbd daemon is
disabled unless IMAGE\_FEATURES contains the
enable-adbd.
You can disable adbd by
removing
`/var/usb-debugging-enabled` from
`rootfs` manually. |
| --- | --- |
| `image-qcom-deploy.bbclass` | Deploys the image files under the path
`tmp-glibc/deploy/images//`.
The
generated images are deployed in the
`` subdirectory.
|
- **Distro configuration**
Following table provides an introduction to
the distro configurations, which are defined at [https://github.com/quic-yocto/meta-qcom-distro/tree/kirkstone/conf/distro](https://github.com/quic-yocto/meta-qcom-distro/tree/kirkstone/conf/distro).
| `conf/distro/qcom-wayland.conf` | This distro configuration file defines the
`qcom-wayland` distro that users use
in the following example command.
MACHINE=qcm6490 DISTRO=qcom-wayland source setup-environmentCopy to clipboard
This
configuration file includes
`meta-qcom-distro/conf/distro/
include/qcom-base.inc`file, which defines
common DISTRO\_FEATURES and adds following
DISTRO\_FEATURES:
These distro features are defined by Yocto
Project documentation and they are available at:
[https://docs.yoctoproject.org/4.0.18/singleindex.html#distro-features](https://docs.yoctoproject.org/4.0.18/singleindex.html#distro-features) |
| --- | --- |
| `conf/distro/include/qcom-base.inc` | The INIT\_MANAGER is set to `systemd`.
For Yocto documentation regarding INIT\_MANAGER, see
[https://docs.yoctoproject.org/4.0.18/singleindex.html#term-INIT_MANAGER](https://docs.yoctoproject.org/4.0.18/singleindex.html#term-INIT_MANAGER).
Other
DISTRO\_FEATURES enabled
are:
DISTRO_FEATURES:append = " pam overlayfs acl xattr selinux ptest security virtualization"Copy to clipboard
To
understand the purpose of these DISTRO\_FEATURES, see
[https://docs.yoctoproject.org/4.0.18/singleindex.html#distro-features](https://docs.yoctoproject.org/4.0.18/singleindex.html#distro-features).
This
file selects `systemd` as
INIT\_MANAGER and `udev` as the dev
manager. |
| `conf/distro/include/qcom-security_flags.inc` | This file includes the security flags as defined in
[https://git.yoctoproject.org/poky/tree/meta/conf/distro/include/security_flags.inc?h=kirkstone](https://git.yoctoproject.org/poky/tree/meta/conf/distro/include/security_flags.inc?h=kirkstone). |
- **Package groups**
Package groups are defined in
`meta-qcom-hwe` and
`meta-qcom-distro`. These package groups help you to
understand the features defined by the Qualcomm BSP.
| `packagegroup-qcom.bb` | Package group that contains all the basic
packages |
| --- | --- |
| `packagegroup-qcom-multimedia.bb` | Package group that contains packages to enable
multimedia support: |
| `packagegroup-qcom-test-pkgs.bb` | – |
- **Image recipes**
Qualcomm Linux metadata layer
`meta-qcom-distro` defines image recipes, which are
available at [https://github.com/quic-yocto/meta-qcom-distro/tree/kirkstone/recipes-products/images](https://github.com/quic-yocto/meta-qcom-distro/tree/).
The following table lists various images, their IMAGE\_FEATURES, and the
functions that the images serve:
| Image recipe | Description of the image |
| --- | --- |
| `qcom-minimal-image.bb` | Defines a small `rootfs` to boot
to shell.
The IMAGE\_FEATURES enabled are as
follows:
IMAGE_FEATURES += "splash tools-debug debug-tweaks enable-adbd read-only-rootfs"Copy to clipboard
For
more information on IMAGE\_FEATURES, see [https://docs.yoctoproject.org/4.0.18/singleindex.html#image-features](https://docs.yoctoproject.org/4.0.18/singleindex.html#image-features). |
| `qcom-console-image.bb` | Extends `qcom-minimal-image` by
adding more images and enables a few more
IMAGE\_FEATURES:
IMAGE_FEATURES += "package-management ssh-server-openssh" Copy to clipboard |
| `qcom-multimedia-image.bb` | Requires DISTRO\_FEATURE Wayland and it includes
all the multimedia packages in
`rootfs`. |
| `qcom-multimedia-full-image.bb` | Provides an extension to
`qcom-multimedia-image.bb` with
GStreamer support. |
| `qcom-multimedia-test-image.bb` | Includes test packages in `rootfs`
to test
`qcom-multimedia-image`. |
| `qcom-multimedia-crossesdk-image.bb` | Generates eSDK for
`qcom-multimedia-image`. |
- **QDL flashing tool**
Qualcomm download tool
(QDL) is a flashing tool that communicates with USB devices of ID
05c6:9008 to upload a flash loader to the device. The flash loader
flashes the images to UFS or eMMC built into the device.
There are
two ways to set up QDL on a host PC:
- QDL tool is built by default in your Qualcomm Linux
workspace.
The QDL tool is integrated in the build
workflow for ease of use. The QDL recipe is found at the
path
`/meta-qcom/recipes-devtools/qdl/qdl_git.bb`.
This recipe fetches and compiles the QDL tool, after which
you can BitBake the recipe. The QDL tool is integrated by
default due to the
`image-qcom-deploy.bbclass`, which adds
it is a dependency for the image being
built:
DEPENDS:append = " \
python3-native \
qdl-native \
"Copy to clipboard
When the image build completes, the
QDL binary is generated at the following directory:
`tmp-glibc/deploy/image//`.
This QDL binary is for host architecture.
- **QDL is built as a standalone project**
Clone the source project, install dev
dependencies, and compile QDL.
On the host
PC:
git https://github.com/linux-msm/qdl.git
sudo apt-get install libxml2-dev libudev-dev
cd qdl
makeCopy to clipboard
The QDL compiled in this way can be copied
and moved around on your host PC as it links to the dynamic
linker at `/lib64/ld-linux-x86-64.so.2`,
which is native to the host PC you are using.
**Prerequisites to run QDL**
- A one-time prerequisite on the host machine is required to use
QDL flasher.
To enable the host PC to enumerate the Qualcomm
device for the host correctly so that QDL can flash, follow
these steps:
1. Change the directory to
`/etc/udev/rules.d`.
2. Create a file `51-qcom-usb.rules` at
this path.
3. Add the following line in the file
`51-qcom-usb.rules`:
SUBSYSTEMS=="usb", ATTRS{idVendor}=="05c6", ATTRS{idProduct}=="9008", MODE="0666", GROUP="plugdev"Copy to clipboard
- Examine if the host PC is running
`ModemManager`.
Note: Check
this every time that you use QDL.
Some Linux
distributions come with `ModemManager`, a
tool for configuring mobile broadband. When the Qualcomm
SoC-based device is connected in USB mode, it is identified
as a Qualcomm modem and `ModemManager` tries
to configure the device. The `ModemManager`
interferes with QDL flashing and hence, if you have
`ModemManager` running, you must disable
it before connecting your test device. If you are using a
Linux distribution with `systemd`,
`ModemManager` can be stopped by using
`systemctl`. To determine how to manage
`ModemManager` with QDL flashing, see
[QDL and ModemManager](https://docs.qualcomm.com/doc/80-70014-27/topic/debug.html#qdl_and_modemmanager).
**Flashing using QDL**
The following is a generic
command to run
QDL:
qdl [ ]Copy to clipboard
This
sample generic input depicts that the QDL tool expects three types of
input files. All these input files are generated under the directory
`tmp-glibc/deploy/image//`.
When the image recipe completes successfully, you must perform the steps
to flash the device. If you want to use the QDL generated as part of the
build, change the directory to
`tmp-glibc/deploy/image//`, as
the Qualcomm Linux workflow generates and deploys QDL at this path. For
example:
cd /build-qcom-wayland/tmp-glibc/deploy/images/qcm6490/qcom-multimedia-image/Copy to clipboard
Run the flash command as
follows:
./qdl prog_firehose_ddr.elf rawprogram*.xml patch*.xmlCopy to clipboard
**Copying QDL to other locations**
The QDL binary at
`/build-qcom-wayland/tmp-glibc/deploy/images/qcm6490/qcom-multimedia-image/`,
which is built as part of the Qualcomm Linux build workflow may not be
copied or moved to other paths but is used as-is because of the
dependencies. To understand these dependencies, run the following
command:
file qdlCopy to clipboard
The
output is as follows:
qdl: ELF 64-bit LSB pie
executable, x86-64, version 1 (SYSV), dynamically linked,
interpreter
<workspace>/build-qcom-wayland/tmp-glibc/sysroots-uninative/x86_64-linux/lib/ld\u0002linux-x86-64.so.2,
BuildID[sha1]=430c23d51190ef6c225586c5d63d2a7c9cd50430, for
GNU/Linux 3.2.0, stripped
This output shows
that there is a dependency on
`/build-test-distro/tmp-glibc/sysroots-uninative/x86_64-linux/lib/ld-linux-x86-64.so.2`,
which is due to the way Qualcomm Linux builds QDL. If you are interested
to run QDL independently, see [standalone
QDL](https://docs.qualcomm.com/doc/80-70014-27/topic/platform_software_features.html#qualcomm_bsp_metadata_layers__qdl_standalone).
**QDL advanced options**
QDL supports
more advanced options that are optional flags passed to QDL from the
command line. These advanced options are described in the following
table:
| Option | Description |
| --- | --- |
| `--debug` | Prints debug logs while flashing the
image |
| `--storage
` | Use this to define the storage type. If this
option is not provided, QDL detects the storage
type. |
| `--finalize-provisioning` | Irreversible one-time provisioning (OTP) for UFS
storage. Perform the following safety mechanisms to
prevent unintentional device locking:
Note: Provisioning is not performed if there is a
mismatch between command-line parameter and
XML. |
| `--include` | Points to the path where the images are present.
By default it is NULL, that is, images are present
at the same location. This is useful when QDL is
built as a standalone, `--include` is
used to point to the path where images are
present. |
Example usage with the
flags:
/qdl --storage ufs --include /qcm6490_images/qcom-multimedia-image /qcm6490_images/qcom-multimedia-image/prog_firehose_ddr.elf /qcm6490_images/qcom-multimedia-image/rawprogram*.xml /qcm6490_images/qcom-multimedia-image/patch*.xml
Copy to clipboard
### meta-qcom-extras
This layer is an optional metadata layer for registered users. This layer enables
source compilation of select components, which are otherwise present as binary in
`meta-qcom-hwe`. If you are entitled to receive this metadata
layer, you can use the clone steps shared in the [Qualcomm Linux Build Guide](https://docs.qualcomm.com/bundle/publicresource/topics/80-70014-254/introduction.html).
- **Firmware recipes**
Firmware recipes in
`meta-qcom-extras` integrate the firmware ZIP that is
generated on compiling other subsystems.
| ├── recipes-firmware
├── firmware
│ ├── firmware-qcm6490-boot_2.0.bb
│ ├── firmware-qcm6490-dspso_2.0.bb
│ └── firmware-qcm6490-msl_2.0.bb
└── packagegroups
└── packagegroup-firmware-qcm6490.bbCopy to clipboard | The recipes in `meta-qcom-extras`
override the recipes in
`meta-qcom-hwe`.
`meta-qcom-hwe` uses pre-built
firmware binaries by default, while
`meta-qcom-extras` builds the
firmware from Qualcomm proprietary
repositories.
The `meta-qcom-extras` layer
ignores the pre-built binaries found in the
following .zip files. These recipes instead search
the Zip files set in the path FWZIP\_PATH as
follows:
|
| --- | --- |
### meta-qcom-qim-product-sdk
- **BitBake classes**
| BitBake class | Description |
| --- | --- |
| `qim-prod-sdk-pkg.bbclass` | |
| `qimsdk-pkg.bbclass` | |
| `tflitesdk-pkg.bbclass` | |
- **Distro configuration**
| `layer.conf` | Configures the project layers with the following
information:
|
| --- | --- |
- **Image recipes**
| Recipe | Description |
| --- | --- |
| `recipes-gst` | Consists of the upstream GStreamer recipe changes
(`.bbapend`) along with Qualcomm recipes: |
| `recipes-qcom-ml` | Consists of two recipes: |
| `recipes-qim-product-sdk` | Recipe to install QIM product SDK that has QIM, Qualcomm
Neural Processing, QNN, and TensorFlow Lite SDKs |
| `recipes-tensorflow-lite` | TensorFlow Lite recipes build and install TensorFlow Lite
for the following versions: |
- **Package groups**
| Package group | Description |
| --- | --- |
| `packagegroup-qcom-gst` | Package group for upstream basic GStreamer enablement
along with Qualcomm plugins:
It also packs the upstream GStreamer packages: |
| `packagegroup-qcom-gst-basic` | Package group for upstream GStreamer enablement: |
| `packagegroup-qcom-qim-product` | Package group to pack the following packages along with
an install script: |
| `packagegroup-qcom-ml` | Package group to pack Qualcomm ML framework: |
## Qualcomm Linux software components
### Filesystem overlays
Source: [https://docs.qualcomm.com/doc/80-70014-27/topic/platform_software_features.html](https://docs.qualcomm.com/doc/80-70014-27/topic/platform_software_features.html)
Qualcomm Linux uses an overlay filesystem to create overlays on certain mount-points. It
is recommended to have a read-only rootfs, but many applications might need read-write
access to some parts of the filesystem. This is achieved by `overlayfs`
bbclass keeping the base rootfs read-only.
The `overlayfs-qcom-paths` recipe creates `/var`,
`/etc`, and `/opt` as overlay mount points and it
inherits the `overlayfs` bbclass. In the recipe, the variable
`overlayfs_mount_point` is defined as `mnt-overlay`,
which implies that the `mnt-overlay.mount` systemd unit is defined in the
BSP and it is installed into the image. At boot, the systemd unit
`mnt-overlay.mount` mounts the
`/dev/disk/by-partlabel/overlay` disk partition to all the mount
points.
The variable `overlayfs_writable_paths` specify the writable paths used at
the runtime, that is, `/opt`, `/etc`, and
`/var`.
To include `overlayfs`support, it must be added to the
DISTRO\_FEATURES.
- Overlay for `/var`
To see the `/var` overlayfs
mount point, run the following mount
command:
sh-5.1# mount | grep var
overlay on /var type overlay (rw,relatime,seclabel,lowerdir=/var,upperdir=/mnt/overlay/upper/var,workdir=/mnt/overlay/workdir/var)Copy to clipboard
- Overlay for
`/etc`
sh-5.1# mount | grep etc
/dev/sda11 on /mnt/overlay type ext4 (rw,relatime,rootcontext=system_u:object_r:etc_t:s0,seclabel,stripe=128,inlinecrypt)
overlay on /etc type overlay (rw,relatime,seclabel,lowerdir=/etc,upperdir=/mnt/overlay/upper/etc,workdir=/mnt/overlay/workdir/etc)Copy to clipboard
- Overlay for
`/opt`
sh-5.1# mount | grep opt
overlay on /opt type overlay (rw,relatime,seclabel,lowerdir=/opt,upperdir=/mnt/overlay/upper/opt,workdir=/mnt/overlay/workdir/opt)Copy to clipboard
- Overlay for
`/home`
sh-5.1# mount | grep home
overlay on /home type overlay (rw,relatime,seclabel,lowerdir=/home,upperdir=/mnt/overlay/upper/home,workdir=/mnt/overlay/workdir/home)Copy to clipboard
- Resizing the overlay partition: This is performed using the recipe
`resize-partitions.bb` and
`resize-partition.service.in`file. The resize-partition
service runs at device bootup to create and set the filesystem according to the
size of the partition. In resize-partition service, running the command to
resize the overlay partition performs the following:
- `/sbin/e2fsck -n /dev/disk/by-partlabel/%i` checks the
specified filesystem based on its partition label for errors, without
making any changes.
- `/sbin/mkfs.ext4 /dev/disk/by-partlabel/%i` invokes the
`mkfs.ext4` utility and formats a partition with the
ext4 filesystem.
- `/sbin/resize2fs /dev/disk/by-partlabel/%i` attempts to
increase the size of the partition.
- `/sbin/tune2fs -O encrypt,stable_inodes
/dev/disk/by-partlabel/overlay` modifies various parameters.
For example, `encrypt` allows you to encrypt data stored
on the filesystem and the stable\_inodes option ensures that inode
numbers remain stable across filesystem operations.
### System initscripts
Source: [https://docs.qualcomm.com/doc/80-70014-27/topic/platform_software_features.html](https://docs.qualcomm.com/doc/80-70014-27/topic/platform_software_features.html)
System initscripts added to the image by `meta-qcom-hwe` are as
follows:
| System initscripts | Description |
| --- | --- |
| `mnt-overlay.mount` | Mounts `/dev/disk/by-partlabel/overlay` disk
partitions to all the mount points. |
| `var-persist.mount` | Mounts `/dev/disk/by-partlabel/persist` disk
partition to `/var/persist`. |
| `android-tools-adbd.service` | Provides adbd daemon on the device. |
| `logrotate.service` | Archives old logs. For more information on this service, see
[Logging shim for syslog()](https://docs.qualcomm.com/doc/80-70014-27/topic/platform_software_features.html#logging_shim_for_syslog). The
`logrotate.service` is an upstream systemd
unit.
The `rsyslog.logrotate` configuration
file is modified in Qualcomm Linux to manage on-device logs. The
modified `rsyslog.logrotate` is at
`meta-qcom-hwe/dynamic-layers/openembedded-layer/recipes-devtools/rsyslog/rsyslog/rsyslog.logrotate`
directory. This overrides the default configuration file
provided by the upstream at
`meta-openembedded/meta-oe/recipes-extended/rsyslog/rsyslog/rsyslog.logrotate`
directory. |
| `pd-mapper.service` | Configures and manages protection domains. The
`pd-mapper.service.in` file is updated to run the
service as a system user instead of as the root user by the
`pd-mapper_git.bbappend` in the Qualcomm Linux
BSP layer
`meta-qcom-hwe`.
do_install:prepend() {
# convert the service from root user to system user
sed -i "/ExecStart=/i\User=system\nGroup=system" pd-mapper.service.in
}Copy to clipboard |
| `property-vault.service` | Provides `property_get` and
`property_set` functionality. For more
information on this service, see [Properties](https://docs.qualcomm.com/doc/80-70014-27/topic/platform_software_features.html#properties). |
| `persist-property-vault.service` | Runs `set-persist-prop.sh`, which sets the flag
`le.persistprop.enable` to true. To use persist
properties, set this flag. |
| `resize-partition@.service` | Resizes the filesystem at boot time according to the size of the
partition.
ExecStart=/bin/sh -c "/sbin/e2fsck -n /dev/disk/by-partlabel/%i; if [ $? -gt 1 ]; then /sbin/mkfs.ext4 /dev/disk/by-partlabel/%i; /sbin/resize2fs /dev/disk/by-partlabel/%i; #COMMAND# fi;"Copy to clipboard |
| `rsyslog.service` | Redirects logs according to a specified configuration. For more
information on this service, see [Logging shim for syslog()](https://docs.qualcomm.com/doc/80-70014-27/topic/platform_software_features.html#logging_shim_for_syslog). |
| `sys-kernel-debug.mount` | Masks the `sys-kernel-debug.mount` point when
`perf` is built. |
### Debug tools
Source: [https://docs.qualcomm.com/doc/80-70014-27/topic/platform_software_features.html](https://docs.qualcomm.com/doc/80-70014-27/topic/platform_software_features.html)
Debug tools are added as part of `rootfs` by
`packagegroup-core-tool-debug` defined in the
`/layers/poky/meta` directory. This recipe is appended
in `/layers/meta-qcom-hwe/recipes-devtools/` as
`packagegroup-core-tools-debug.bbappend`. This append file adds
`valgrind` and `ltrace`. For more information, see
[Debug Linux user space](https://docs.qualcomm.com/bundle/publicresource/topics/80-70014-12/using_open_source_debug_tools.html).
### systemd-boot
Source: [https://docs.qualcomm.com/doc/80-70014-27/topic/platform_software_features.html](https://docs.qualcomm.com/doc/80-70014-27/topic/platform_software_features.html)
systemd-boot is a unified extensible firmware interface (UEFI) boot manager. It presents
a menu of boot options to control the boot flow and loads the user-selected boot loader.
systemd-boot can support different boot loaders from different operating systems. Pack
the kernel and other related information in an EFI partition.
The kernel must be built as EFI using `EFI_CONFIG_STUB`. systemd-boot
supports Type1 and Type2 configurations. Type1 configuration is not tested. Type2
configuration is enabled where the kernel and other details are packed in the UKI. Type2
is enabled for better security. UKI contains all the information for the device to boot.
Signing a UKI image ensures that all the entities included in it are secure. If UEFI
secure boot is enabled, only signed images are loaded; therefore, signing is required.
For more details on systemd-boot, see [systemd-boot](https://wiki.archlinux.org/title/systemd-boot).
Note: Signing
is optional during the development phase.
- **UKI**
A UKI is a combination of a UEFI boot stub program, a Linux kernel
image, an initrd, and other resources in a single UEFI portable executable
(PE) file. The UEFI boot stub looks for various resources for the kernel
invocation inside the UEFI PE binary. This allows combining various
resources inside a single PE binary image (UKI), which may then be signed
using OpenSSL utilities.
For more details on UKI, see [unified_kernel_image](https://uapi-group.org/specifications/specs/unified_kernel_image/). The content
of `uki.efi`is shown in the following figure. It contains:
- Ramdisk image
- Uncompressed kernel image
- Architecture information
- systemd-boot stub
- Command-line parameters passed to the kernel
- OS-release information
Table : uki.efi file
| Components of uki.efi file | Contents |
| --- | --- |
| Initrd = Init ramdisk | `initramfs-qcom-image-qcm6490.cpio.gz` |
| Linux = Kernel Image | Image |
| Efi-arch = Architecture | `aa64` |
| Stub = System-boot efi stub | `linuxx64.efi.stub` |
| Cmdline = Command line parameters | `root=/dev/disk/by-partlabel/system rw
rootwait console=ttyMSM0,1150200n8 pcie_pme=nomsi
earlycon
kernel.sched_pelt_multiplier=4` |
| OS-release = OS-release |
ID = qcom-wayland
Name = “QCOM Reference Distro with
Wayland”
VERSION = “1.0”
VERSION_ID = 1.0
PRETTY_NAME = “QCOM Reference Distro
with Wayland 1.0”
|
**Image classes**
`meta-qcom-hwe/classes` contains two classes,
`uki.bbclass` and `image-efi.bbclass`:
- `uki.bbclass` generates
`uki.efi`.
- `image-efi.bbclass` generates
`efi.bin` using `uki.efi`.
- `qcom-minimal-image` inherits the
`image-efi.bbclass`.
- **EFI image**
EFI system partition (ESP) contains
`efi.bin`, a vfat file. This file contains all the
details necessary for UEFI to enable systemd-boot. UEFI mounts the ESP
partition and executes systemd-boot manager (`bootaa64.efi`).
systemd-boot manager parses the UKI image, loads the kernel image, and then
transfers the control to it.
For more information on the structure of
ESP, see [EFI system partition](https://wiki.archlinux.org/title/EFI_system_partition). A sample
structure of the ESP partition is shown in the following figure. It contains
systemd-boot (`bootaa64.efi`) and UKI
(`uki.efi`).
Figure : efi.bin file
- **Signing**
Secure boot is a security feature found in the UEFI standard
and it is not enabled in development devices. It adds a layer of protection
to the preboot process by maintaining a cryptographically signed list of
binaries that are authorized or forbidden to run at boot. It helps ensure
confidence that the machine boot firmware and HLOS boot components (boot
manager, kernel, and initramfs) have not been tampered with.
UEFI
secure boot uses a digital signature to validate the authenticity and
integrity of the code that it loads. All the keys are stored in UEFI secure
variables. UEFI secure boot is achieved by using PK, KEK, db, and dbx
keys.
To use a secure boot, the keys PK, KEK, and db are required.
While multiple KEK, db, and dbx certificates can be added, only one platform
key is allowed.
UEFI secure boot is enabled only when PK is registered
in the system. It is advised to provision PK key in the last step of the
secure boot enabling process. For more information on Qualcomm's
implementation of UEFI secure boot, see [Secure boot](https://docs.qualcomm.com/bundle/publicresource/topics/80-70014-11/secure-boot.html).
**Host tool for signing HLOS images**
When UEFI secure boot is enabled, the
EFI and DTB images must be signed. To streamline this process, a host
signing tool is used. This command-line Python script runs on a Linux host
machine (preferably Ubuntu 18.04 or later versions). It automates the
signing of EFI and DTB images in two separate operations, invoked after the
build process completes.
**Overview of host signing tool**
The host signing tool runs on a Linux machine with Python3
installed. It can sign either the EFI image or the DTB image in a single
operation. To sign both the EFI and DTB images, the user must invoke the
tool twice with different inputs.
The host tool expects unsigned EFI or DTB files, along with
certificates and keys, as input. After invoking, the tool unpacks the
unsigned image, signs the available items using the provided key and
certificate, and then repacks the images, replacing the unsigned version
with the signed one.
**Working of host signing tool**
- **Prerequisites to run the tool**
To run this tool, the host
Linux machine must install the following:
- OpenSSL and sbsign utilities
- Python3
- pip, subprocess, shlex, socket, and shutil python
modules
- **Configuration of the host signing tool**
The user must
configure the host signing tool before starting the
operation.
- **`config.in` file**
The host tool
expects users to provide necessary information in a
`config.ini` configuration file.
Upon boot up, the script reads this file and signs
the image accordingly. Following are the variables
listed in the configuration file:
Figure : config.ini file
Table : Variables in config.ini file
| Variable in config.ini | Values | Description |
| --- | --- | --- |
| `image_type` | `efi/dtb` | Use this configuration to select
`efi` or `dtb` to
sign separately. |
| `file_path` | `local/remote` | `local` - Keys and
`efi.bin`/`dtb.bin`
are present on the same path as the
script.
`remote` - Copy
`efi.bin`/`dtb.bin`
and keys from a remote Linux machine to the
current path. |
| `local_machine_private_key_path` | ` machine>` | This file establishes an SSH connection
with a remote machine if `file_path =
remote`. |
| `loader_conf_timeout` | ` seconds>` | systemd-boot wait time to let you choose to
authenticate the binaries. This option is required
to sign `efi.bin`. |
| `efi/keys/dtb_remote_hostname` | ` Linux machine>` | If `file_path = remote`,
then the host tool selects the host name of the
remote machine to copy
`efi/keys/dtb` file from the remote
machine using SCP. |
| `efi/keys/dtb_remote_username
` | `` | If `file_path = remote`,
then the host tool selects the user name of the
remote machine to copy
`efi/keys/dtb` file from the remote
machine using SCP, provided the user name is
created on the local machine. |
| `efi/keys/dtb_remote_filepath` | `` | If `file_path = remote`,
then the host tool selects the path of a
`efi/key/dtb` file on the remote
machine to copy that file from the remote machine
using SCP. |
- **Configure using config.ini file**
1. Image selection: The user must specify which
image to sign by setting the
`image_type` variable. The options
are either `efi` or
`dtb`.
2. File location: The user must indicate the
location of the unsigned EFI/DTB image, keys, and
certificates using the `file_path`
variable.
If the user selects
`local`in the configuration file,
they must manually copy the EFI/DTB image, keys,
and certificate files to the local working
directory.
1. Create `unsigned_binaries`
directory in the same path as the script, and then
copy the
`efi.bin`/`dtb.bin`
image into that directory.
2. Create `keys` directory in the
same path as the script, and then copy
`db.auth`, `db.crt`,
`db.key`,
`KEK.auth`, and
`PK.auth` files into that
directory.
If the user wants the script to
automatically copy the required files from a
remote Linux machine on the same network, they
must select `remote` in the
configuration file.
In the configuration
file, the user must provide information for the
following variables:
- `local_machine_private_key_path`
(mandatory)
- `[efi_config]` section (if
`image_type` is
`efi`)
- `[keys_config]` section
(mandatory)
- `[dtb_config]` section (if
`image_type` is
`dtb`)
Note: The script supports copying from another Linux
machine over SCP within the same
network.
3. Loader configuration timeout: If the
`image_type` is set to
`efi` in the configuration file,
the user must update the
`loader_conf_timeout`
variable.
4. Handling missing configuration: If the user
misses any configuration information, the script
runs and prompt the user for the missing details
through the command line.
- **Run the host signing tool**
1. Invoke the host tool: After completing the
code build process and obtaining the unsigned
`efi.bin` and
`dtb.bin` images, invoke the host
signing tool.
2. Prepare a host machine: Store the host signing
tool files (`signing_tool.py` and
`config.ini`) on a Linux machine.
Ensure that both files are in the same working
directory.
3. Configure the tool: Set up the host signing
tool according to the configuration
instructions.
4. Run the tool: Execute the host tool from the
command line using the command: `$python3
signing_tool.py`
5. Interactive process: The host signing tool
displays user selections and operational commands
on the screen. It displays errors on the command
line.
6. Signed images: After the tool completes its
process, it creates a directory called
`signed_binaries` in the same
working directory. The signed
`efi.bin` or
`dtb.bin` image is stored there.
The tool deletes other user-created directories
after signing.
7. Repeat for both images: Follow this process
twice, once for `efi.bin` and once
for `dtb.bin`. After each signing
operation, delete the
`signed_binaries` directory before
starting a new operation.
- **Host signing tool workflow**
Following figure shows the
workflow of host signing tool:
Figure : Host signing tool workflow
- The host tool requires the `efi.bin` and
`dtb.bin`paths (absolute path or network
path).
- `efi.bin` contains
`uki.efi` (Linux kernel image) and
`bootaa64.efi` (boot loader
image).
- `dtb.bin` contains
`combined-dtb.dtb`.
- The host tool requires the path of
`certificate` and `key`
(absolute path or network path) to sign the images.
- The host tool mounts
`efi.bin`/`dtb.bin` on FAT
partition, which provides the following directory structure
and follows its separate signing process:
- After signing the images, the host tool copies the
*Auth* files under
`/loader/keys/authkeys` directory for
both `efi.bin` and `dtb.bin`.
The *Auth* files must be available with keys and
certificates on Target or Security team hosted path.
- The Host tool must configure the wait-time in
`systemdboot` loader configuration. This
wait-time stops the Kernel loading and allows the user to
review and select `systemdboot` menu options.
The `loader.conf` file must be available in
an updated `efi.bin` file.
Note: This
process is not followed for `dtb.bin`
file.
- The host tool configures
`/loader/loader.conf`.
- The syntax for `loader.conf` is as
follows: `timeout x`, where x =
timeout in seconds.
- After completion of the image signing, the host tool must
unmount the `efi.bin/dtb.bin` from FAT
partition. Store the signed `efi.bin/dtb.bin`
on the host machine on the similar path as the host tool
under the `signed_binaries` directory.
- Following is the directory structure for signed
`efi.bin` and
`dtb.bin`.
**`efi.bin` signing process**
- The host tool uses the `sbsign` utility
to sign `uki.efi` and
`bootaa64.efi` images
separately.
- `sbsign` requires
`certificate` and
`key` for the signing process. Check
the following syntax where `dsk1.key` is
key, `dsk1.crt` is certificate, and the
output filename is the same as the input file:
sbsign --key --cert Copy to clipboard
Examples:
- sbsign --key dsk1.key --cert dsk1.crt bootaa64.efi bootaa64.efiCopy to clipboard
- sbsign --key dsk1.key --cert dsk1.crt uki.efi uki.efiCopy to clipboard
**`dtb.bin` signing process**
- The host tool requires the path of
`dtb.bin` file.
- The host tool requires the path of `key`
and `certificate` (absolute path or
network path) to sign the images.
- UEFI secure boot requires PE format files for
verification. Non-PE files, like `dtb`,
cannot be signed using `sbsign` as this
signing tool requires PE format files as input.
- The host tool uses the `openssl` utility
to sign the `dtb` file. Check the
following syntax where `dsk1.key` is key
and `dsk1.crt` is certificate:
openssl cms -sign -inkey <.key file> -signer <.crt file> -binary -in --out -outform DERCopy to clipboard
Example:
openssl cms -sign -inkey dsk1.key -signer dsk1.crt -binary -in --out -outform DERCopy to clipboard
This
command adds the signature for dtb file in a separate
file (`foo.dtb.sig`) and does not alter
the original file (`foo.dtb`). Hence, the
host tool must keep both the files where the
`*.dtb.sig` file is used during the
UEFI secure boot verification.
- **Multi-DTB support**
Qualcomm supports multiple board variants based on
the same SoC. For instance, the QCS6490 board variants include the RB3 core
kit and RB3 Gen2 vision kit. Each board variant has its own DTB in the
kernel. During bootup, UEFI selects the appropriate DTB based on the
specific board variant. To facilitate this, all DTBs for boards sharing the
same SoC are combined and stored in the DTB partition.
**Combined DTB generation**
To enable multi-DTB support, all supported DTBs are appended one after the other and a combined
DTB is generated.
For RB3 Gen 2, the following DTBs are combined to generate a
combined DTB,
`combined-dtb.dtb`.
qcom/qcm6490-addons-idp.dtb \
qcom/qcs5430-fp1-addons-rb3gen2.dtb \
qcom/qcs5430-fp1-addons-rb3gen2-vision-mezz.dtb \
qcom/qcs5430-fp1p5-addons-rb3gen2.dtb \
qcom/qcs5430-fp1p5-addons-rb3gen2-vision-mezz.dtb \
qcom/qcs5430-fp2-addons-rb3gen2.dtb \
qcom/qcs5430-fp2-addons-rb3gen2-vision-mezz.dtb \
qcom/qcs6490-addons-rb3gen2.dtb \
qcom/qcs6490-addons-rb3gen2-video-mezz.dtb \
qcom/qcs6490-addons-rb3gen2-vision-mezz.dtb \
qcom/qcs6490-addons-rb3gen2-ptz-mezz.dtb \
qcom/qcs6490-addons-rb3gen2-ia-mezz.dtb \Copy to clipboard
**DTB partition**
- `dtb.bin`, a `vfat` image is
generated, which contains the combined DTB image. A dedicated
partition named `dtb` is created and
`dtb.bin` is flashed on this partition.
- The UEFI parses the combined DTB present in the `dtb`
partition and selects a matching DTB for the hardware.
Figure : Create/sign DTB
### Partitioning
Source: [https://docs.qualcomm.com/doc/80-70014-27/topic/platform_software_features.html](https://docs.qualcomm.com/doc/80-70014-27/topic/platform_software_features.html)
Partitioning in the Yocto workflow involves the following entities:
| Entity | Description |
| --- | --- |
| Partition layout file | Defines all partitions for storage in JSON format. |
| gen\_partition tool | Reads the partition layout file and generates an internal XML
format handled by the Qualcomm Ptool. |
| Partition XML | Defines the Qualcomm internal XML format. |
| Ptool | Provisions Qualcomm partition tool that converts information in
partition XML to GPT binaries. |
The following are details on partition definitions, existing partitions, and the tool
that generates the GUID partition table for Qualcomm boards:
Note: The following partition layout and
examples are specific to UFS.
- **Partition layout file**
The UFS on the device is partitioned to store
various images of bootchain and high-level OS. Complete information
regarding the partitions defined for a specific board are defined in the
configuration file. For the RB3 Gen 2 board, the UFS flash partitions are
defined in the following configuration file in the workspace:
`meta-qcom-hwe/recipes-devtools/partition-utils/partition-confs/generic-ufs-partitions.conf`.
- **Partition layout file syntax**
To understand the syntax of the partition
layout file, a few examples from the
`generic-ufs-partitions.conf` file are discussed as
follows. One of the first entries in the configuration file defines the disk
type, disk size, LBA size (sector size), and whether the last partition
should grow till the last usable
LBA.
# select disk type emmc | nand | ufs mandatory
# disk size in bytes is mandatory
--disk --type=ufs --size=137438953472 --write-protect-boundary=0 --sector-size-in-bytes=4096 --grow-last-partitionCopy to clipboard
The
following are a few examples of partitions defined in the file
`generic-ufs-partitions.conf`. A couple of partitions are
defined in LUN0 and each line defines an individual partition:
Note: In the following
examples, GUIDs are taken from the `partition.xml`
file.
- Example 1: `ssd` partition of size 8 kB defined for
LUN0. This partition does not need a file
flashed.
--partition --lun=0 --name=ssd --size=8KB --type-guid=2C86E742-745E-4FDD-BFD8-B6A7AC638772Copy to clipboard
- Example 2: `misc` partition of size 1024 kB defined
for LUN0. This partition does not need a file flashed.
--partition --lun=0 --name=misc --size=1024KB --type-guid=82ACC91F-357C-4A68-9C8F-689E1B1A23A1Copy to clipboard
A couple of partitions are defined in LUN1. Each line defines an
individual partition:
- Example 3: `xbl_a` partition of size 3604 kB defined
for LUN1. File `xbl.elf` is flashed to this
partition.
#This is LUN 1 - Boot LUN A
--partition --lun=1 --name=xbl_a --size=3604KB --type-guid=DEA0BA2C-CBDD-4805-B4F9-F428251C3E98 --filename=xbl.elfCopy to clipboard
- Example 4: `xbl_config_a` partition of size 300 kB
defined for LUN1. File `xbl_config.elf` is flashed to
this
partition.
#This is LUN 1 - Boot LUN A
--partition --lun=1 --name=xbl_config_a --size=300KB --type-guid=5A325AE4-4276-B66D-0ADD-3494DF27706A --filename=xbl_config.elfCopy to clipboard
A partition defined in LUN4:
- Example 5: `aop_a` partition of size 512 kB defined
for LUN4. File `x` is flashed to this
partition.
--partition --lun=4 --name=aop_a --size=512KB --type-guid=D69E90A5-4CAB-0071-F6DF-AB977F141A7F --filename=aop.mbnCopy to clipboard
The options used for each partition entry are as follows:
- Mandatory
options:
--lun (mandatory for UFS, optional for emmc. Expressed as number)
--name (name for the partition, a string)
--size (size of the partition, generally expressed in KB)
--type-guid (GUID for the partition)Copy to clipboard
- Optional
options:
--attributes (Optional 64 bit attribute, e.g., 1000000000000004)
--filename (Name of the file that is flashed to the partition)
--readonly (whether the partition should be read-only, values true or false)
--sparse (whether the partition is for a sprased image, values true or false)Copy to clipboard
- **Linux HLOS partitions**
| Partitions | Description |
| --- | --- |
| EFI partition | The EFI system partition (ESP) contains
`Esp.bin` and a `vfat`
file. It contains all the details necessary for UEFI to
enable systemd-boot. For more information on this image, see
[EFI image](https://docs.qualcomm.com/doc/80-70014-27/topic/platform_software_features.html#systemd_boot__efi_image). |
| Rootfs partition | This partition is used to flash the
`system.img` image file. This image
consists of all the user space libraries and
binaries. |
| Overlay partition | This partition is left empty. On boot, a filesystem is
seeded into this partition that is used as an overlay on a
few of the `/var` mount-points. The image in
this filesystem works as a writable storage filesystem.
Associated IMAGE\_FEATURES with this image is
`overlay-etc`. |
- **Partition tool (Ptool)**
To partition the storage on the device, use the
`gen_partition` command to generate a
`partitions.xml` file, and then process the
`partitions.xml` file with Ptool. Ptool generates a GUID
partition table binary that the QDL tool uses to partition the flash. The
Ptool workflow is shown in the following figure:
Figure : Ptool workflow
The `generic-ufs-partitions.conf` files present
in the location
`/layers/meta-qcom-hwe/recipes-devtools/partition-utils/partition-confs`
define all the partitions. The tool `gen_partition.py`
processes and generates `partition.xml`, which is a mandatory
input for `ptool.py`. As a final step, Ptool generates the
files `rawprogram.xml`, `patch.xml`,
`gpt_main*.bin`, and `gpt_backup*.bin`,
which are crucial for the QDL tool to flash the device.
To add a
partition, make a partition entry in the configuration file, for example
`generic-ufs-partitions.conf` with a universal unique
identifier (UUID). After adding an entry, run the BitBake command to
generate all necessary files:
bitbake Copy to clipboard
After
the image build is complete, flash the
image:
cd /build-qcom-wayland/tmp-glibc/deploy/images/qcm6490/qcom-console-image
./qdl prog_firehose_ddr.elf rawprogram*.xml patch*.xmlCopy to clipboard
### Containerization
Source: [https://docs.qualcomm.com/doc/80-70014-27/topic/platform_software_features.html](https://docs.qualcomm.com/doc/80-70014-27/topic/platform_software_features.html)
Docker containers are enabled in Qualcomm Linux. To use Docker, ensure that
containerization features are enabled in Qualcomm BSP, which are integrated and tested.
To use containerization on Qualcomm LE devices, ensure that
`meta-virtualization` layer ([https://git.yoctoproject.org/meta-virtualization](https://git.yoctoproject.org/meta-virtualization)) is added
as part of the workspace.
- **Docker**
Docker is enabled in the
`qcom-multimedia-image`:
1. For the image recipe `qcom-multimedia-image`,
`packagegroup-container` is included using
`meta-qcom-distro/recipes-products/packagegroups/packagegroup-qcom-multimedia.bb`.
2. After the image is built and flashed, Docker is available on the
device.
3. To check kernel compatibility, run the script
`check-config.sh` on the device. To check
configuration, the required kernel configurations must be enabled
and the image is rebuilt. For more information, see [Kernel compatibility](https://docs.docker.com/engine/install/troubleshoot/#:~:text=Kernel%20compatibility,check%2Dconfig.sh%20script.&text=The%20script%20only%20works%20on%20Linux).
4. | `systemctl status docker`
output after any docker command is run on the
device. | `systemctl status docker`
output if no docker commands have been run on the
device. |
| --- | --- |
| Inactive (dead) | Active (running) |
The following use cases are verified with Docker enabled:
1. Images pulled from the Docker repository for the
`linux/arm64`platform and tested to run
successfully on the device are as follows:
- `busybox`
- `python`
- `alpine`
- `ubuntu`
- `postgres`
- `mongo`
- `nginx`
- `redis`
2. Load Docker images from tar files using the Docker load
utility.
3. Build a Docker image from the Docker file.
4. Save Docker images that were pulled/built into a
`.tar` file.
5. Load Docker images, which are saved as `.tar`
files.
6. Run multiple container instances of multiple images
pulled/built.
7. Check for the list of Docker images loaded on the device.
8. Check for the list of active and total containers on the
device.
9. Inspect Docker images using the Docker inspect utility.
10. Obtain logs from a container using the Docker logs utility.
11. Stop and kill container using Docker stop and Docker kill
utilities.
12. Remove containers and Docker images from the device.
- **Hardware node access from Docker**
Applications running inside a Docker container might need
access to device nodes and files from the device (target-host). Applications
running inside containers can access device nodes. This is performed by
passing the respective device nodes as an option with the Docker run
command:
docker run -it --rm --device= --device= Copy to clipboard
The
device nodes to be passed can be found under the `/dev/`
directory on the device. Depending on the use case, for example, in a
graphics scenario, `/dev/kgsl-3d0` is one of the nodes to be
passed to the Docker using
`--device=/dev/kgsl-3d0`.
Storage partition access from
Docker files/directories from the device (target-host) can also be exposed
to the inside of a container. Exposing files/directories as bind mounts is
achieved using the following
command:
docker run -it --rm --mount type=bind,source=,target= Copy to clipboard
Run
a docker container exposing multiple directories, files, and device nodes to
the inside of the container using the following
command:
docker run -it --rm --device= --device= --mount type=bind,source=,target= --mount type=bind,source=,target=
Copy to clipboard
- **Docker-compose**
Docker-compose is a powerful tool for defining and
running multicontainer applications. It simplifies the management of the
entire application stack by allowing you to define services, networks, and
volumes in a single, comprehensible YAML configuration file. By default,
Qualcomm Linux release includes Docker-compose to the image:
- The `python3-docker-compose` package is added to the
`packagegroup-qcom-multimedia` package group in
`meta-qcom-distro` metadata layer.
**To run docker-compose on device:**
1. Create or copy Docker-compose YAML files under writable path on a
device such as `/var` (for example
`/var/docker-compose-yaml`) to be used with
Docker-compose on the device.
2. Ensure that the Docker images listed in your Docker-compose YAML
files are available on the device. The containers started by the
Docker-compose utility need these images.
3. To run Docker-compose for multiple YAML files, use the following
command:
docker-compose -f .yml -f .yml upCopy to clipboard
Docker-compose runs and initiates Docker containers as
configured from the Docker-compose YAML file. After the Docker-compose
commands returns, run `docker ps -a` and docker logs
`` to check if the docker containers are
running as expected.
To know more about Docker-compose, see [https://docs.docker.com/compose](https://docs.docker.com/compose).
### Kubernetes
Source: [https://docs.qualcomm.com/doc/80-70014-27/topic/platform_software_features.html](https://docs.qualcomm.com/doc/80-70014-27/topic/platform_software_features.html)
Kubernetes is an open-source platform used to automate the deployment, scaling, and
management of applications running in a container. It helps orchestrate containers
across multiple devices, ensuring efficient resource utilization, high availability, and
scalability of applications. For more information on Kubernetes, see [https://kubernetes.io/](https://kubernetes.io/).
- **Enable Kubernetes**
- Qualcomm Linux release enables Kubernetes by default in
`qcom-multimedia-image`, via the following
changes:
- `packagegroup-qcom-k8s` is defined in
`meta-qcom-hwe`.
- `packagegroup-qcom-k8s` is included in
`recipes-products/packagegroups/packagegroup-qcom-multimedia.bb`.
- Qualcomm Linux kernel has the following kernel configurations
enabled in ` arch/arm64/configs/qcom_defconfig`to
support the Kubernetes runtime:
- CONFIG_CGROUP_FAVOR_DYNMODS=yCopy to clipboard
- CONFIG_CFS_BANDWIDTH=yCopy to clipboard
- CONFIG_CGROUP_HUGETLB=yCopy to clipboard
- CONFIG_NETFILTER_XT_MATCH_COMMENT=mCopy to clipboard
- CONFIG_IP_NF_TARGET_REDIRECT=mCopy to clipboard
- CONFIG_HUGETLBFS=yCopy to clipboard
- | `systemctl status kubelet`
output if the device is not setup as a Kubernetes
node using `kubeadm`. | `systemctl status kubelt`
output if the device is setup as a Kubernetes node
using `kubeadm`. |
| --- | --- |
| Inactive (dead) | Active (running) |
Kubernetes operates with containers that run as pods, which allow users
to package an application along with its dependencies (libraries and
configuration files) into a single unit. This simplifies deployment and
ensures consistency across different environments and nodes of a cluster. A
node in the context of Kubernetes refers to a device or a machine. The
architecture of Kubernetes has two types of resources:
- **Master node/control plane:** The master node is the device that
acts as the control plane for Kubernetes. It makes global decisions
about the cluster (like scheduling), and it detects and responds to
cluster events (like starting up a new container when a replica
field of deployment is unsatisfied).
- **Worker node:** These nodes are devices that contain the
necessary services to run containers, managed by the master node.
Each worker node runs `Kubelet`, a process
responsible for communication between the Kubernetes master node and
the worker node; it manages the pods running on a machine.
A pod is the smallest unit of execution in Kubernetes. It
represents a single instance of a running process in a cluster acting as an
abstraction to the containers run by Kubernetes.
- **Use of Kubernetes**
The following is an example of two devices connected
as nodes to a common access point:
- **Master node**: One of the devices is designated as the master
node.
- **Worker node**: The other device serves as the worker node.
Deployments and pods created from the master node runs on the worker
node.
Deployments and pods are created using YAML files. These files
define the preferred state of deployments and pods at the time of creation.
Users can enter the shell of these pods/containers for debugging,
monitoring, and other tasks from both the master node and the worker
node.
- **Set up a device as a master node**
Following are the steps to
set up a device as a master node:
1. Connect the device to Wi-Fi.
2. Run the following command:
kubeadm init --ignore-preflight-errors=SystemVerification
--apiserver-advertise-address=Copy to clipboard
Note: `` is the IP
address of `wlan0` interface, which
is the output of `ifconfig` command
after connecting the device to Wi-Fi.
3. The previous command initializes the Kubernetes
control plane. To start using the cluster, run the
following commands as root
user:
mount -o remount, rw /
mkdir -p $HOME/.kube
cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
chown $(id -u):$(id -g) $HOME/.kube/configCopy to clipboard
After successful initialization of the device as
master node/control plane using `kubeadm init`,
`kubeadm` prints the command that must be
used for any device to join the cluster as a worker
node.
- **Set up a device as a worker node**
To set up the device as a
worker node and join the cluster, run the `kubeadm
join` command, where `kubeadm` prints
after initializing the master node.
Note: To suppress
an error that Kubernetes reports when running Docker with
the vfs graph driver, run
`--ignore-preflight-errors=SystemVerification
flag` with the `kubeadm join`
command.
Note: Each node
in a cluster is supposed to have a different hostname. If
when trying to add more than one device with the same
hostname to a single cluster fails, change the hostname of
the device using the `hostname `
command.
- **Create deployment and pods from a master node**
To create
deployments/pods on the worker node, run the following command:
kubectl apply -f Copy to clipboard
Create
your own YAML file at a location such as `/var`.
Example for YAML files:
- YAML file to create a deployment with the Ubuntu
image:
**ubuntu-deployment.yaml**
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: ubuntu-deployment
name: ubuntu-deployment
spec:
replicas: 2
selector:
matchLabels:
app: ubuntu-deployment
template:
metadata:
labels:
app: ubuntu-deployment
spec:
containers:
- name: ubuntu-depl-pod
image: ubuntu:latest
command: ["sleep", "3650d"]Copy to clipboard
- YAML file to create a pod with the Ubuntu
image:
**ubuntu-pod.yaml**
apiVersion: v1
kind: Pod
metadata:
name: ubuntu-pod
labels:
app: ubuntu-pod
spec:
containers:
- name: ubuntu
image: ubuntu:latest
command: ["sleep", "604800"]
imagePullPolicy: IfNotPresent
restartPolicy: AlwaysCopy to clipboard
- **Using Kubernetes pods with `kubectl`**
- **Using `kubectl` get nodes in the cluster**
To list the nodes in the cluster, run the
following command on the master
node:
kubectl get nodesCopy to clipboard
- **Using `kubectl` get deployments present in the cluster**
To list the deployments present in the
cluster, run the following command on the master
node:
kubectl get deploymentCopy to clipboard
- **Using `kubectl` get pods present in the cluster**
To list the pods present in the cluster,
run the following command on the master
node:
kubectl get podsCopy to clipboard
- **Access the shell of a pod from the master node**
To
access the shell instance of a pod from the master node,
run the following command on the master
node:
kubectl exec -it -- Copy to clipboard
- **Access shell of a pod from the worker node**
To
access the shell instance of a pod (container) using
Docker, run the following command on the worker
node:
docker exec -it Copy to clipboard
Note: `` is obtained
using listing all the containers using `docker ps
-a` command.
**Changes made in Docker and Kubernetes in contrast to the upstream solution**
- Recipe
`meta-qcom-hwe`/`recipes-containers`/`docker`/`docker-ce_git.bbappend`
changes `/lib/systemd/system/docker.service`. During this change,
`--exec-opt native.cgroupdriver=systemd` is passed to a
`dockerd` command that runs as part of
`ExecStart` of the service. This is performed to keep parity
with `cgroupdriver` between Docker and Kubernetes.
- Recipe
`meta-qcom-hwe`/`recipes-containers`/`kubernetes`/`kubernetes_git.bbappend`
changes `/lib/systemd/system/kubelet.service.d/10-kubeadm.conf`.
During this change, `--fail-swap-on=false` is passed as part of
`KUBELET_EXTRA_ARGS`. This is performed because Kubernetes
expected the swap to be **OFF**. As the platform software of Qualcomm has
zram enabled, swap is set to **ON**.
- The upstream `kubelet.service` attempts to start automatically
upon device bootup, even if the device is not configured as a Kubernetes node.
However, this behavior consumes resources and impacts power efficiency. To
address this, `kubelet.service` is modified by removing
`WantedBy=multi-user.target` in the
`meta-qcom-hwe/recipes-containers/kubernetes/kubernetes_git.bbappend`
file. Now, it starts when the device is set up as a Kubernetes node using
`kubeadm`, rather than automatically in
`multi-user.target`.
### Properties
Source: [https://docs.qualcomm.com/doc/80-70014-27/topic/platform_software_features.html](https://docs.qualcomm.com/doc/80-70014-27/topic/platform_software_features.html)
Properties (`property-vault`) provide functionality to store and share
key-value pairs stored as strings across the system. Any software component can share
any value for a particular key and any other process can access the value by using the
same key. `property-vault` gives an option to define persistent
properties across reboot. Components can use `property-vault`to share
specific information, which might be relevant to any other module on the device. These
key-value pairs can be accessed using the command-line interface (CLI).
- **Properties using CLI**
To set a property using the CLI, use the following
command:
setprop "my-key" "my-value"Copy to clipboard
To
access a property using the CLI, use the following
command:
getprop "my-key"Copy to clipboard
#
Result of the previous command: my-value
To set a property
using the CLI and persist it across reboots, use the following
command:
setprop "persist.my-key" "my-value"Copy to clipboard
- **Properties in C/C++ source files**
Setting up to use properties in C/C++
source files:
1. If you must use properties such as `property-vault`
in a recipe, for example, `example-recipe.bb`, add a
dependency on
`property-vault`:
DEPENDS = "glib-2.0 property-vault"
RDEPENDS:${PN} = "property-vault"Copy to clipboard
2. Change `make`files:
1. If you are using
`automake`:
`configure.ac`
...
PKG_CHECK_MODULES(GLIB, glib-2.0 >= 2.16, dummy=yes, AC_MSG_ERROR(GLib >= 2.16 is required))
GLIB_CFLAGS="$GLIB_CFLAGS"
GLIB_LIBS="$GLIB_LIBS"
AC_SUBST(GLIB_CFLAGS)
AC_SUBST(GLIB_LIBS)
AC_CONFIG_FILES([Makefile])
...Copy to clipboard
`Makefile.am`
root_sbindir = "/sbin"
root_sbin_PROGRAMS = property-test
property_test_SOURCES = source.c
property_test_CFLAGS = @GLIB_CFLAGS@
property_test_LDFLAGS = @GLIB_LIBS@ -lpropertyvaultCopy to clipboard
2. If you are using
`cmake`:
`CMakeList.txt`
set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -std=c99 -lpthread -lrt -lm -lglib-2.0 -ldl -latomic")
target_link_libraries (techteamlocalibname propertyvault)Copy to clipboard
3. Modify the source files to `get` and
`set`
properties:
`source.c`
#include "properties.h"
int main() {
// setting a property
property_set("my-key", "my-value");
// getting a property
char paramstr[PROPERTY_VALUE_MAX];
property_get("my-key", paramstr, ""); // value gets stored in paramstr
// setting a property and persisting it across reboot
property_set("persist.my-key", "my-value");
return 0;
}Copy to clipboard
### Logging shim for syslog()
Source: [https://docs.qualcomm.com/doc/80-70014-27/topic/platform_software_features.html](https://docs.qualcomm.com/doc/80-70014-27/topic/platform_software_features.html)
Logs from Qualcomm proprietary and open-source projects are logged into
`syslog`.
The `syslog()` is used to write logs on the device. Use
`rsyslog` as the logging system to process logs according to
specified rules. `rsyslog` can accept inputs not only from system and
kernel messages, but also accepts logs from applications that use `syslog
protocol` for logging.
When an application makes a `syslog` call, `rsyslog`
redirects these logs based on its configuration. `rsyslog` configuration
is found at `/etc/rsyslog.conf` or `/etc/ryslog.d`
directory.
Use the `logrotate` service for archiving old logs.
### Persist partition
Source: [https://docs.qualcomm.com/doc/80-70014-27/topic/platform_software_features.html](https://docs.qualcomm.com/doc/80-70014-27/topic/platform_software_features.html)
The persist partition defined for the UFS is used to store persistent data that BSP
software components require across reboots. User-requested operations must not erase
this data during the device lifecycle.
- **Persist mount point**
The `/var/persist` directory serves as a
mount point for a filesystem, which is created on the partition named persist.
The recipe `overlayfs-qcom-paths_1.0.bb` is responsible for
creating the directory on the rootfs and installs the systemd unit
`var-persist.mount` to
`local-fs.target`.
At boot, the systemd unit
`var-persist.mount` mounts
`/dev/disk/by-partlabel/persist` to
`/var/persist`. The persist mount point is displayed by
running the mount command as
follows:
sh-5.1# mount | grep persist
/dev/sda10 on /var/persist type ext4 (rw,relatime,rootcontext=system_u:object_r:qcom_persist_t:s0,seclabel,stripe=128)Copy to clipboard
- **Resize persist partition**
Resize the overlay partition using the recipe
`resize-partitions.bb` and
`resize-partition.service.in` files. The resize-partition
service runs at the device bootup to create and set the filesystem according to
the size of the partition.
### Secondary virtual machine
Source: [https://docs.qualcomm.com/doc/80-70014-27/topic/platform_software_features.html](https://docs.qualcomm.com/doc/80-70014-27/topic/platform_software_features.html)
### About this task
This section provides the steps to build a Qualcomm Linux image with guest virtual
machine (VM) support, including VM management tools (`crosvm`,
`qemu`), the kernel image, and a minimal rootfs for the guest VM.
Once built, it generates a Guest VM kernel (image) and rootfs
(`initrd.img`), which are then packaged into the system root
filesystem. The following subsections provide a step-by-step guide on how users can
enable VM support for the image.
### Procedure
1. **Setup the host**
Install clang on the host machine using the following
command:
sudo apt install clang-11Copy to clipboard
2. **Enable meta-rust layer**
The meta-rust layer provides rust compiler (rustc) and package manager
(cargo) to compile crosvm, which is implemented in rust language.
Add the meta-rust layer to EXTRALAYERS in the
`meta-qcom-distro/conf/bblayers.conf` directory.
`vi
conf/bblayers.conf`
EXTRALAYERS ?= " \
${WORKSPACE}/layers/meta-rust \
"Copy to clipboard
3. **Upgrade the rust version**
Create a file
`meta-qcom-distro/conf/distro/include/rust_version.inc`
with the following content:
`vi
meta-qcom-distro/conf/distro/include/rust_version.inc`
# include this in your distribution to easily switch between versions
# just by changing RUST_VERSION variable
RUST_VERSION ?= "1.73.0"
PREFERRED_VERSION_cargo ?= "${RUST_VERSION}"
PREFERRED_VERSION_cargo-native ?= "${RUST_VERSION}"
PREFERRED_VERSION_libstd-rs ?= "${RUST_VERSION}"
PREFERRED_VERSION_rust ?= "${RUST_VERSION}"
PREFERRED_VERSION_rust-cross-${TARGET_ARCH} ?= "${RUST_VERSION}"
PREFERRED_VERSION_rust-llvm ?= "${RUST_VERSION}"
PREFERRED_VERSION_rust-llvm-native ?= "${RUST_VERSION}"
PREFERRED_VERSION_rust-native ?= "${RUST_VERSION}"Copy to clipboard
Include the
`rust_version.inc` file in the
`meta-qcom-distro/conf/distro/qcom-wayland.conf`
path.
`conf/distro/qcom-wayland.conf`
require conf/distro/include/rust_version.incCopy to clipboard
4. **Add VM package into image**
Create a packagegroup as `packagegroup-qcom-vm` and add it to
the image via `packagegroup-qcom`. Update the packages,
`RDEPENDS:${PN}` variables and add
`RDEPENDS:packagegroup-qcom-vm` as follows:
`meta-qcom-distro/recipes-products/packagegroups/packagegroup-qcom.bb`
PACKAGES += "packagegroup-qcom-vm"
RDEPENDS:${PN} += "packagegroup-qcom-vm"
RDEPENDS:packagegroup-qcom-vm = "\
crosvm \
pack-svm \
kvmtool \
qemu \
"Copy to clipboard
5. **Build an image**
To build an image, use the following
command:
bitbake qcom-console-imageCopy to clipboard
6. **Launch VM**
To see the expected output as mentioned in [Guest VM build support](https://docs.qualcomm.com/bundle/publicresource/topics/80-70014-3/features.html#guest-vm-build-support), launch the
VM.
### Handle out-of-tree kernel modules and device tree using recipes
Source: [https://docs.qualcomm.com/doc/80-70014-27/topic/platform_software_features.html](https://docs.qualcomm.com/doc/80-70014-27/topic/platform_software_features.html)
For building out-of-tree kernel module, see [Add kernel module](https://docs.qualcomm.com/bundle/publicresource/topics/80-70014-3/customize.html#add-kernel-module).
For device-tree/device-tree blob management, see [Platform support](https://docs.qualcomm.com/bundle/publicresource/topics/80-70014-3/customize.html#platform-support_0).
Last Published: Jul 11, 2024
[Previous Topic
Overview](https://docs.qualcomm.com/bundle/publicresource/80-70014-27/topics/intro_yocto_linux_qualcomm.md) [Next Topic
User customizations](https://docs.qualcomm.com/bundle/publicresource/80-70014-27/topics/user_customizations.md)