# Enable virtualization
Virtualization uses software to create an abstraction layer over
computer hardware.
The abstraction layer divides the hardware elements of a single computer,
such as processor, memory, and storage into different
virtual entities, called as virtual machines (VMs). Each VM runs its
own OS and behaves like an independent computer,
though it’s running on the same underlying computer hardware.
**Figure: Virtualization**
## KVM overview
A kernel-based virtual machine (KVM) is an open-source virtualization module integrated into the Linux kernel.
This integration allows the KVM to act as a hypervisor.
KVM facilitates hardware-assisted virtualization for guest operating systems.
Note
KVM is supported on Dragonwing™ IQ-8275 and Dragonwing IQ-9075 SoCs only.
**High-level overview of KVM/QEMU virtualization**
The following are the prerequisites to enable KVM Hypervisor on Arm CPU-based systems:
- The Boot loader starts the Linux kernel in the exception level 2 (EL2)
- `CONFIG_KVM` is enabled in the kernel configuration
To verify if KVM is available on the device, run the following command to verify the presence of `/dev/kvm`:
ls -l /dev/kvm
Copy to clipboard
For instructions to boot the Linux kernel in EL2 to enable the KVM hypervisor,
see [UEFI](https://docs.qualcomm.com/bundle/publicresource/topics/80-80020-4/boot-developer-touchpoints.html#uefi).
Consider the following when enabling KVM:
- Virtual host extensions (VHE) are enabled by default on all the Qualcomm Linux SoCs that support Arm (v8.1) or
later instruction sets. This configuration allows the entire Linux Kernel to run at EL2 with slight modifications.
The guest OS kernel and user space run at EL1 and EL0.
With VHE, transitions between the guest and host incur less penalty because the guest and host kernels
operate at different exception levels.
- The Linux kernel manages all accessible non-secure memory. It doesn’t support protected
use cases that require specific isolation from the Linux kernel.
- The power state coordination interface (PSCI) for KVM is supported on the Linux kernel. PSCI ensures that
CPU hotplug and low-power modes (LPM) function correctly with KVM.
### Virtual machine manager
Quick emulator (QEMU) is used as the virtual machine manager (VMM) for virtualization.
Libvirt acts as a management layer that interacts with QEMU to start, stop, and manage virtual machines.
#### QEMU
Use QEMU as a VMM with the KVM to run VMs at near-native performance. QEMU supports emulated devices, for example,
virtio devices, that provide high-performance I/O operations and allows access to USB and peripheral component
interconnect (PCI) devices from the VM.
The QEMU process and its threads manage a single VM. Invoke QEMU directly to create the VMs. For more information,
see [QEMU’s](https://qemu-project.gitlab.io/qemu/index.html).
#### Libvirt
Libvirt is a suite of tools, including an API library, a daemon (libvirtd), and a command-line utility (virsh),
for managing VMs. The virsh utility is useful for managing multiple VMs.
For more information, see [virsh](https://www.libvirt.org/manpages/virsh.html).
### Launch guest VM
To launch an ARM64-based VM, use QEMU or virsh commands that internally work with libvirt. The commands allow you to configure the VM with CPU, memory, and storage.
Specify the number of virtual CPUs and the amount of memory allocated to the VM. Additionally, the VM can boot with a ramdisk (initrd) and a `.ext4` root file system.
Note
Verify that the guest kernel image (`Image`), root filesystem CPIO (`rootfs.cpio.gz`), and
root filesystem image (`rootfs.ext4`) are present in the `/mnt/overlay/guest` directory
in the host filesystem before launching the guest. These file paths and formats are examples for reference.
#### QEMU
To bring up the VMs with various configurations, run the following commands:
##### Boot with ramdisk
To boot with ramdisk, run the following command:
qemu-system-aarch64 \
-M virt -m 2G \
-kernel /mnt/overlay/guest/Image \
-initrd /mnt/overlay/guest/rootfs.cpio.gz \
-cpu host --enable-kvm -smp 4 -nographic
Copy to clipboard
##### Boot with rootfs image
To boot with rootfs image, run the following command:
qemu-system-aarch64 \
-M virt -m 2G \
-kernel /mnt/overlay/guest/Image \
-drive file=/mnt/overlay/guest/rootfs.ext4,if=virtio,format=raw \
-append "root=/dev/vda" \
-cpu host --enable-kvm -smp 4 -nographic
Copy to clipboard
#### Libvirt
Libvirt enables you to use QEMU and KVM to create, interact, and manage VMs.
##### VM management with virsh
virsh provides a range of commands to create, control, and monitor VMs.
Run the following virsh commands for VM management:
# Define a new VM domain called initrd_simple
virsh define /mnt/overlay/libvirt_initrd_simple.xml
Copy to clipboard
# List all domains and check the state of initrd_simple, it is not started yet
virsh list --all
Copy to clipboard
# start the initrd_simple VM
virsh start initrd_simple
Copy to clipboard
# connect to console
virsh console initrd_simple
Copy to clipboard
# Disconnect from console by pressing (Ctrl + ]) key combination
Copy to clipboard
# shutdown the VM
Copy to clipboard
virsh shutdown initrd_simple
Copy to clipboard
# Undefine the VM if there is no intention to restart/resue it
virsh undefine initrd_simple
Copy to clipboard
##### Boot with ramdisk
Copy the following XML content to `/mnt/overlay/guest/libvirt_initrd_simple.xml` on the host:
##### Boot with rootfs image
Copy the following XML file to the host file system in the `/mnt/overlay/guest/libvirt_rootfs_simple.xml` file.
Use the VM management commands with `rootfs_simple` as VM domain/name instead of `initrd_simple`.
>
>
>
### Virtio framework
Virtio abstracts devices in a paravirtualized hypervisor environment.
It provides an I/O paravirtualization framework to interact with the paravirtualized (paravirt) devices.
The virtual machine monitor (VMM) or hypervisor (HYP) emulates most of the devices exposed to virtual machines (VMs) using virtio.
### Key features
The following are the key features of virtual network:
- Paravirtualization and full virtualization
>
>
> - Full virtualization: In a fully virtual environment, the guest VM does not recognize the hypervisor, it runs without modifications. However, full virtualization results in higher overhead due to the device emulation.
> - Paravirtualization: In a paravirtualize environment, the VM recognizes the hypervisor and requires modifications to the OS. Paravirtualization allows efficient communication between the guest and the host.
- Virtio architecture
>
>
>
>
>
>
>
>
> **Figure : A high-level overview of the virtio architecture**
>
> - Front-end drivers: Implements in the guest OS. The front-end drivers interact with the back-end drivers
> in the hypervisor
> - Back-end drivers: Implements in the VMM/HYP. The back-end drivers handle the actual device emulation,
> and interacts with the front-end drivers through virtual queues
> - Virtual queues: Virtio uses virtual queues (virtqueues) to facilitate communication between front-end and back-end drivers. These queues are implemented as rings to manage guest-to-hypervisor transitions efficiently.
- Vhost
>
>
> Vhost is a protocol that offloads the virtio data plane implementation to another element
> (user process or kernel module) to enhance performance. This offloading reduces the overhead
> of context switching between the guest VM and the hypervisor.
- Vhost-net: A kernel-level implementation that allows the data plane to bypass the QEMU process,
to reduce latency and improves performance.
For more information about v-host-net, see [Introduction to
virtio-networking and vhost-net](https://www.redhat.com/en/blog/introduction-virtio-networking-and-vhost-net).
- Vhost-user: A user-space implementation that handles the data plane with a separate process, to provide flexibility and
better performance for workloads. For more information about v-host-user,
see [Vhost-user Protocol](https://www.qemu.org/docs/master/interop/vhost-user.html).
### Benefits
The following are the advantages of using virtio framework:
- Standardization: Virtio provides a common interface for device emulation. It promotes code reuse and efficiency across different
virtualization platforms.
- Flexibility: Virtio supports block devices and network devices.
### Virtio interfaces
The following are the virtio-supported interfaces:
#### 9P transport overview
`virtio-9p` enables you to share files between the host and the VMs under the 9P
(Plan 9 file system) protocol to enhance the performance within the virtio framework.
Note
Ensure that the following configurations are enabled in the host and guest kernels:
Host side: CONFIG_NET_9P, CONFIG_NET_9P_VIRTIO
Copy to clipboard
Guest side: CONFIG_NET_9P, CONFIG_NET_9P_VIRTIO, CONFIG_9P_FS
Copy to clipboard
To create a `virtio-9p` shared directory, define it on the host, and configure the guest VM to mount it.
See the following example:
root@qcs9100-ride-sx:~# ls -la /mnt/overlay/test_dir
total 12
drwxr-xr-x. 2 root root 4096 Apr 29 21:36 .
drwxr-xr-x. 7 root root 4096 Apr 29 21:36 ..
-rw-r--r--. 1 root root 8 Apr 29 21:36 file.txt
root@qcs9100-ride-sx:~# cat /mnt/overlay/test_dir/file.txt
testing
root@qcs9100-ride-sx:~#
Copy to clipboard
Create a `/mnt/overlay/test_dir` directory on the host to share it with the guest VM. Pass this information either in the libvirt XML configuration or as QEMU arguments.
To configure the 9P transport, run the following command:
qemu-system-aarch64 \
-M virt -m 2G \
-kernel /mnt/overlay/guest/Image \
-drive file=/mnt/overlay/guest/rootfs.ext4,if=virtio,format=raw \
-append "root=/dev/vda" \
-cpu host --enable-kvm -smp 4 -nographic \
-fsdev local,id=fsdev0,path=/mnt/overlay/test_dir,security_model=passthrough \
-device virtio-9p-pci,fsdev=fsdev0,mount_tag=hostshare
Copy to clipboard
To identify and mount the shared directory on the guest VM, use the `fsdev0` and `hostshare` filesystem devices.
To launch the Libvirt guest VM from the libvirt interface with the following sample XML,
see Libvirt.
simple_9p20971524/machinehvm/mnt/overlay/guest/Imageroot=/dev/vda
Copy to clipboard
Note
Copy the XML content to `/mnt/overlay/guest/libvirt_virtio_9p.xml` file in the host.
After the guest VM boots up, verify if the virtio device is probed successfully.
root@v8a-arm64:~# lspci
00:00.0 Host bridge: Red Hat, Inc. QEMU PCIe Host bridge
00:01.0 PCI bridge: Red Hat, Inc. QEMU PCIe Root port
00:01.1 PCI bridge: Red Hat, Inc. QEMU PCIe Root port
00:01.2 PCI bridge: Red Hat, Inc. QEMU PCIe Root port
01:00.0 Unclassified device [0002]: Red Hat, Inc. Virtio 1.0 filesystem (rev 01)
02:00.0 SCSI storage controller: Red Hat, Inc. Virtio 1.0 block device (rev 01)
root@v8a-arm64:~#root@v8a-arm64:~# cat /sys/bus/pci/devices/0000\:01\:00.0/virtio0/uevent
DRIVER=9pnet_virtio
MODALIAS=virtio:d00000009v00001AF4
root@v8a-arm64:~#
Copy to clipboard
To mount the shared directory in the guest VM, run the following command:
mount -t 9p -o trans=virtio hostshare mountpoint
Copy to clipboard
#### VSOCK overview
VSOCK is a virtual socket interface that allows communication between VMs and the host OS.
It’s used in KVM and QEMU. VSOCK is supported through a virtio interface. virtio-vsock is a vhost-based virtio device where the host kernel manages all the data transfer
while the KVM hypervisor controls the information.
To create a VSOCK connection in a KVM and QEMU environment, specify the context identifier (CID) and port the
number. The CID is a unique identifier assigned to each VM in a VSOCK environment, which is used to route
communication between the host and the VMs. The host has a CID of 2 while VMs are assigned CIDs starting from 3
and above.
The following table lists the CID values:
Table: CID values
| CID | Description |
| --- | --- |
| -1 | Any address for binding |
| 0 | Hypervisor |
| 1 | Loopback |
| 2 | Host |
Note
Ensure that the following configurations are enabled in the host and guest kernels.
Host side: CONFIG_VSOCKETS, CONFIG_VHOST_VSOCK
Guest side: CONFIG_VSOCKETS, CONFIG_VIRTIO_VSOCKETS
Copy to clipboard
- Enable VSOCK device: To enable the VSOCK device, specify the device in the VM configuration file (libvirt XML), or pass an argument to QEMU.
qemu-system-aarch64 \
-machine virt -m 2G \
-kernel /mnt/overlay/guest/Image \
-drive file=/mnt/overlay/guest/rootfs.ext4,if=virtio,format=raw \
-append "root=/dev/vda" \
-cpu host --enable-kvm -smp 4 -nographic \
-device vhost-vsock-pci,guest-cid=73
Copy to clipboard
In this command, `guest-cid=73` specifies the CID for the VM.
- Launch the guest VM using the libvirt interface: To use the VSOCK device with libvirt, define the VSOCK device in the VM XML configuration file.
To launch the guest VM using the libvirt interface, see Libvirt.
Copy the XML content to `/mnt/overlay/guest/libvirt_virtio_vsock.xml` on the host.
After the Guest VM boots up, verify that the device is probed successfully.
>
>
> root@v8a-arm64:~# lspci
> 00:00.0 Host bridge: Red Hat, Inc. QEMU PCIe Host bridge
> 00:01.0 PCI bridge: Red Hat, Inc. QEMU PCIe Root port
> 00:01.1 PCI bridge: Red Hat, Inc. QEMU PCIe Root port
> 00:01.2 PCI bridge: Red Hat, Inc. QEMU PCIe Root port
> 01:00.0 SCSI storage controller: Red Hat, Inc. Virtio 1.0 block device (rev 01)
> 02:00.0 Communication controller: Red Hat, Inc. Virtio 1.0 socket (rev 01)
> root@v8a-arm64:~#
> root@v8a-arm64:~# cat /sys/bus/pci/devices/0000\:02\:00.0/virtio1/uevent
> DRIVER=vmw_vsock_virtio_transport
> MODALIAS=virtio:d00000013v00001AF4
> root@v8a-arm64:~#
> Copy to clipboard
To verify the host-to-guest and guest-to-host interaction, use the `socat` utility.
Ensure that the `socat` utility is supported on host and guest.
- On the host, verify the following port:
>
>
> socat STDIN VSOCK-LISTEN:1234
> Copy to clipboard
- On the guest, connect to the host using the host CID (2):
socat STDOUT VSOCK-CONNECT:2:1234
Copy to clipboard
- In this setup, the guest and host show the same values.
#### Virtio block overview
The virtio block is a standardized way to present block devices to the VM. Each virtio block device appears as a disk
inside the guest VM. The virtio block allows the VMs to read and write operations.
The following are the key features of the virtio-block device:
- Simplicity: Easy to implement and use
- Performance: Designed to minimize overhead and maximize throughput
- Flexibility: Designed to use with various types of storage back ends
Note
Ensure that `CONFIG_VIRTIO_BLK` is enabled in the guest kernel.
To use a virtio-block device, do the following:
1. To configure the VM for including a virtio-block device, specify the device in the
VM configuration file (libvirt XML), or pass an argument to QEMU.
1. To enable the virtio-block device in the VM, use the following QEMU command:
qemu-system-aarch64 \
-M virt -m 2G \
-kernel /mnt/overlay/guest/Image \
-drive file=/mnt/overlay/guest/rootfs.ext4,if=virtio,format=raw \
-append "root=/dev/vda" \
-cpu host --enable-kvm -smp 4 -nographic \
-drive file=/mnt/overlay/guest/disk.img,if=virtio,format=raw
Copy to clipboard
In this command, `rootfs.ext4` and `disk.img` are the disks inside the guest VM.
2. Launch the guest VM using the libvirt interface: To launch the guest VM with the following sample XML,
see Libvirt:
simple_block_device20971524/machinehvm/mnt/overlay/guest/Imageroot=/dev/vda
Copy to clipboard
Note
Copy the XML content to `/mnt/overlay/guest/libvirt_virtio_blk.xml` on the host.
2. The guest OS detects the virtio-block device and initializes it. In this process, the guest driver
communicates with the device to set up the necessary data structures and queues.
>
>
> root@v8a-arm64:~# dmesg | grep virtio
> [ 0.225425] virtio-pci 0000:01:00.0: enabling device (0000 -> 0002)
> [ 0.227073] virtio-pci 0000:02:00.0: enabling device (0000 -> 0002)
> [ 0.234012] virtio_blk virtio0: 4/0/0 default/read/poll queues
> [ 0.235405] virtio_blk virtio0: [vda] 2220734 512-byte logical blocks (1.14 GB/1.06 GiB)
> [ 0.241213] virtio_blk virtio1: 4/0/0 default/read/poll queues
> [ 0.242418] virtio_blk virtio1: [vdb] 8192000 512-byte logical blocks (4.19 GB/3.91 GiB)
> root@v8a-arm64:~#
> root@v8a-arm64:~# lspci
> 00:00.0 Host bridge: Red Hat, Inc. QEMU PCIe Host bridge
> 00:01.0 PCI bridge: Red Hat, Inc. QEMU PCIe Root port
> 00:01.1 PCI bridge: Red Hat, Inc. QEMU PCIe Root port
> 00:01.2 PCI bridge: Red Hat, Inc. QEMU PCIe Root port
> 01:00.0 SCSI storage controller: Red Hat, Inc. Virtio 1.0 block device (rev 01)
> 02:00.0 SCSI storage controller: Red Hat, Inc. Virtio 1.0 block device (rev 01)
> root@v8a-arm64:~#
> Copy to clipboard
3. After initializing, you can use the virtio-block and other block devices.
The guest OS performs read and write operations on the virtio driver and passes the operations
to the host for processing.
4. To manage the virtio-block device, use the standard tools and commands available in the guest OS.
For example, `lsblk` and `df`.
#### Virtio-IOMMU overview
Virtio-IOMMU is a paravirtualized input/output memory management unit (IOMMU) that provides
direct memory access (DMA) management in the virtual environments.
Virtio-IOMMU integrates with the existing software APIs such as virtual function I/O (VFIO) and removes the
need for page table emulation, making it a lightweight and high-performance solution.
Virtio-IOMMU acts as a proxy for physical IOMMUs and manages the devices assigned to the guest.
As virtual IOMMU, it manages, emulates, and paravirtualizes the devices.
The following are the key features of Virtio-IOMMU:
- Paravirtualization: Leverages existing transport mechanisms and reduces overhead.
- Flexibility: Supports PCI passthrough and shares virtual memory.
- Integration: Integrates with software APIs and enhances compatibility. For example, VFIO.
Note
Ensure that `CONFIG_VIRTIO_IOMMU` is enabled in the guest kernel.
To use a Virtio-IOMMU device, do the following:
1. To configure the VM for including a Virtio-IOMMU device, specify the device in the
VM configuration file (XML), or pass an argument to QEMU.
>
>
> 1. Enable a Virtio-IOMMU device: To enable the Virtio-IOMMU device in the VM, use the following QEMU command:
>
>
>
>
> >
> >
> > qemu-system-aarch64 \
> > -M virt -m 2G \
> > -kernel /mnt/overlay/guest/Image \
> > -drive file=/mnt/overlay/guest/rootfs.ext4,if=virtio,format=raw \
> > -append "root=/dev/vda" \
> > -cpu host --enable-kvm -smp 4 -nographic \
> > -device virtio-iommu-pci
> > Copy to clipboard
>
>
> 2. Launch guest VM using the libvirt interface: To launch the guest VM with the following sample XML, see Libvirt.
>
>
>
>
> >
> >
> >
> > simple_iommu
> > 2097152
> > 4
> >
> > /machine
> >
> >
> > hvm
> > /mnt/overlay/guest/Image
> > root=/dev/vda
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > Copy to clipboard
>
>
>
> Note
>
>
> Copy the XML content to `/mnt/overlay/guest/libvirt_virtio_iommu.xml` on the host.
2. The guest OS detects the Virtio-IOMMU device and initializes it.
In this process, the guest driver communicates with the device to set up the necessary data structures and mappings.
>
>
> root@v8a-arm64:~# dmesg | grep virtio
> [ 0.184166] virtio-pci 0000:00:03.0: enabling device (0000 -> 0002)
> [ 0.189122] virtio_iommu virtio0: input address: 64 bits
> [ 0.189516] virtio_iommu virtio0: page mask: 0xfffffffffffff000
> [ 0.220135] virtio-pci 0000:01:00.0: Adding to iommu group 0
> [ 0.220676] virtio-pci 0000:01:00.0: enabling device (0000 -> 0002)
> [ 0.223065] virtio_blk virtio1: 4/0/0 default/read/poll queues
> [ 0.224713] virtio_blk virtio1: [vda] 2220734 512-byte logical blocks (1.14 GB/1.06 GiB)
> root@v8a-arm64:~#
> Copy to clipboard
>
>
> The DMESG logs show that the virtio-block device is attached to the IOMMU domain,
> which is a part of the IOMMU group `0`.
3. After initializing, the Virtio-IOMMU device manages DMA operations for attached devices.
The guest OS performs mapping and unmapping operations with the virtio driver
and passes the operations to the host for processing.
>
>
> The following example shows Virtio-IOMMU device operations:
>
>
> root@v8a-arm64:~# cat /sys/kernel/iommu_groups/0/devices/0000\:01\:00.0/virtio1/uevent
> DRIVER=virtio_blk
> MODALIAS=virtio:d00000002v00001AF4
> root@v8a-arm64:~#
> Copy to clipboard
#### Virtio-net overview
The virtio-net device is a virtual network device that provides an interface for network
operations in virtualized environments. It offers high performance and low overhead.
The following are the key features of the virtio-net device:
- Efficiency: Minimizes overhead and maximizes throughput.
- Simplicity: Easy to implement and use.
- Flexibility: Supports various network configurations and back ends.
Note
Ensure that `CONFIG_VIRTIO_NET` is enabled in the guest kernel.
To use a virtio-net device, do the following:
1. To configure the VM for including a virtio-net device, specify the device in
the VM configuration file (libvirt XML), or pass an argument to QEMU.
>
>
> 1. To enable the virtio-net device in the VM, use the following QEMU command:
>
>
>
>
> >
> >
> > qemu-system-aarch64 \
> > -M virt -m 2G \
> > -kernel /mnt/overlay/guest/Image \
> > -drive file=/mnt/overlay/guest/rootfs.ext4,if=virtio,format=raw \
> > -append "root=/dev/vda" \
> > -cpu host --enable-kvm -smp 4 -nographic \
> > -netdev tap,id=net0,ifname=tap0,script=no,downscript=no \
> > -device virtio-net-pci,netdev=net0
> > Copy to clipboard
> >
> >
> > Note
> >
> >
> > This command creates a Tap interface on the host. To create the Tap interface manually,
> > run the following command:
> >
> >
> > ip tuntap add dev tap0 mode tap
> > Copy to clipboard
>
>
> 2. To launch the guest VM with the following sample XML, see Libvirt.
>
>
>
>
> >
> >
> >
> > simple_net
> > 2097152
> > 4
> >
> > /machine
> >
> >
> > hvm
> > /mnt/overlay/guest/Image
> > root=/dev/vda
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > Copy to clipboard
> >
> >
> > To create a tap interface for direct communication between the host VM and the guest VM, copy the XML content to `/mnt/overlay/guest/libvirt_virtio_net.xml` on the host.
2. The guest operating system detects the virtio-net device and initializes it.
In this process, the guest driver communicates with the device to set up the necessary data structures and queues.
>
>
> root@v8a-arm64:~# dmesg | grep virtio
> [ 0.394365] virtio-pci 0000:01:00.0: enabling device (0000 -> 0002)
> [ 0.396364] virtio-pci 0000:02:00.0: enabling device (0000 -> 0002)
> [ 0.404496] virtio_blk virtio1: 4/0/0 default/read/poll queues
> [ 0.406048] virtio_blk virtio1: [vda] 2220734 512-byte logical blocks (1.14 GB/1.06 GiB)
> [ 0.980919] virtio_net virtio0 enp1s0: renamed from eth0
> root@v8a-arm64:~# lspci
> 00:00.0 Host bridge: Red Hat, Inc. QEMU PCIe Host bridge
> 00:01.0 PCI bridge: Red Hat, Inc. QEMU PCIe Root port
> 00:01.1 PCI bridge: Red Hat, Inc. QEMU PCIe Root port
> 00:01.2 PCI bridge: Red Hat, Inc. QEMU PCIe Root port
> 01:00.0 Ethernet controller: Red Hat, Inc. Virtio 1.0 network device (rev 01)
> 02:00.0 SCSI storage controller: Red Hat, Inc. Virtio 1.0 block device (rev 01)
> root@v8a-arm64:~#
> Copy to clipboard
3. Operate: After you initialize and configure the virtio-net device, use it as other network interface.
The guest OS performs standard network operations with the virtio driver
and passes the operations to the host for processing.
To use the Tap interface, run the following network configurations on the host VM and the guest VM:
- Host configuration:
root@qcs9100-ride-sx:~# ip addr add 192.168.100.1/24 dev tap0
root@qcs9100-ride-sx:~# ip link set dev tap0 up
Copy to clipboard
- Guest configuration:
root@v8a-arm64:~# ip addr add 192.168.100.2/24 dev enp1s0
root@v8a-arm64:~# ip link set dev enp1s0 up
Copy to clipboard
To run the network traffic on this interface, use the following example:
>
>
> root@v8a-arm64:~# ping 192.168.100.1
> PING 192.168.100.1 (192.168.100.1): 56 data bytes
> 64 bytes from 192.168.100.1: seq=0 ttl=64 time=0.364 ms
> 64 bytes from 192.168.100.1: seq=1 ttl=64 time=0.156 ms
> Copy to clipboard
#### Virtio-serial and virtio-console overview
The virtio-serial device provides an interface for serial communication between the host VM and the guest VM in virtual environments.
The following are the key features of the virtio-serial device:
- Multiple ports: Supports multiple serial ports, allows various communication channels.
- Efficiency: Minimizes overhead and maximizes throughput.
- Flexibility: Supports different types of data exchange. For example, console access and file transfer.
Note
Ensure that `CONFIG_VIRTIO_CONSOLE` is enabled in the guest kernel.
To use a virtio-serial device, do the following:
1. To configure the VM for including a virtio-serial device, specify the device in
the VM configuration file (libvirt XML), or pass an argument to QEMU.
1. To enable the virtio-net device in the VM, use the following QEMU command:
>
>
> qemu-system-aarch64 \
> -M virt -m 2G \
> -kernel /mnt/overlay/guest/Image \
> -drive file=/mnt/overlay/guest/rootfs.ext4,if=virtio,format=raw \
> -append "root=/dev/vda" \
> -cpu host --enable-kvm -smp 4 -nographic \
> -device virtio-serial-pci,id=virtio-serial0 \
> -chardev pty,id=charconsole0 \
> -device virtconsole,chardev=charconsole0,id=console0 \
> -chardev socket,path=/tmp/qemu.sock,server=on,wait=off,id=charchannel0 \
> -device virtserialport,chardev=charchannel0,name=org.qemu.guest_agent.0
> Copy to clipboard
2. To launch the guest VM with the following sample XML, see Libvirt.
simple_serial20971524/machinehvm/mnt/overlay/guest/Imageroot=/dev/vda
Copy to clipboard
Note
Copy the XML content to `/mnt/overlay/guest/virtio_serial.xml` on the host.
2. The guest operating system detects the virtio-serial device and initializes it.
In this process, the guest driver communicates with the device to set up the necessary data structures and ports.
>
>
> root@v8a-arm64:~# lspci
> 00:00.0 Host bridge: Red Hat, Inc. QEMU PCIe Host bridge
> 00:01.0 PCI bridge: Red Hat, Inc. QEMU PCIe Root port
> 00:01.1 PCI bridge: Red Hat, Inc. QEMU PCIe Root port
> 00:01.2 PCI bridge: Red Hat, Inc. QEMU PCIe Root port
> 01:00.0 Communication controller: Red Hat, Inc. Virtio 1.0 console (rev 01)
> 02:00.0 SCSI storage controller: Red Hat, Inc. Virtio 1.0 block device (rev 01)
> root@v8a-arm64:~#
> root@v8a-arm64:~# cat /sys/bus/virtio/devices/virtio0/uevent
> DRIVER=virtio_console
> MODALIAS=virtio:d00000003v00001AF4
> root@v8a-arm64:~#
> Copy to clipboard
3. After initialization, you can use the virtio-serial device as a serial device. The guest OS performs read and write operations with the virtio driver and passes the operations to the host for processing.
4. To manage the virtio-serial device, do the following:
- Monitor the performance
- Verify the device status
- Perform maintenance tasks
- Use the standard tools and commands available in the guest OS
##### Memory balloon device support
The traditional virtio memory balloon device manages the guest memory.
It allows the host system to reclaim memory from VMs. The memory balloon instructs VMs to return a
portion of their memory to the host. This process involves inflating the memory balloon inside the Guest VM, which reduces the memory available for other tasks within the VM. The Guest OS decides which memory pages to give back to the host, indicating which pages it does not need or access. The host then un-maps these pages from the guest VM and marks them as unavailable for the guest VM and allows the host system to use them. If the guest VM requires more memory later, the host deflates the balloon to return the pages. This feature allows each guest VM to continue running while its available memory is managed.
Use the following for the virtio-balloon device to relocate physical memory between a guest VM and the host:
1. Balloon inflation: The guest driver allocates memory and informs the host. The host then reuses the inflated memory for other VMs.
2. Balloon deflation: After informing the host, the guest driver frees the previously allocated memory
and enables the guest VM to use the deflated memory.
`Target balloon size` controls the balloon inflation or deflation through a request to change the guest
VM memory size. The resize request is sent through QEMU monitor mode or through virsh commands.
The following are the key features of the virtio-balloon device:
- Memory management: Adjusts memory allocation in real time based on the requirements.
- Efficiency: Reclaims unused memory from idle VMs and redistributes it to the active ones.
- Flexibility: Enables various virtualization setups to optimize resource usage.
Note
Ensure that `CONFIG_VIRTIO_BALLOON` is enabled in the guest kernel.
To use a virtio-balloon device, do the following:
1. Configure the VM for including a virtio-balloon device, specify the device in the VM configuration file (libvirt XML), or pass an argument to QEMU.
1. To enable the virtio-balloon device in the VM, use the following QEMU command:
qemu-system-aarch64 \
-M virt -m 2G \
-kernel /mnt/overlay/guest/Image \
-drive file=/mnt/overlay/guest/rootfs.ext4,if=virtio,format=raw \
-append "root=/dev/vda" \
-cpu host --enable-kvm -smp 4 -nographic \
-device virtio-balloon-pci,id=balloon0
Copy to clipboard
2. To launch the guest VM with the following sample XML, see Libvirt:
simple_balloon20971524/machinehvm/mnt/overlay/guest/Imageroot=/dev/vda
Copy to clipboard
Note
Copy the XML content to `/mnt/overlay/guest/libvirt_virtio_balloon.xml` file on the host.
2. The guest OS detects the virtio-balloon device and initializes it. In this process, the guest driver
communicates with the device to set up the necessary data structures and memory management mechanisms.
root@v8a-arm64:~# lspci
00:00.0 Host bridge: Red Hat, Inc. QEMU PCIe Host bridge
00:01.0 PCI bridge: Red Hat, Inc. QEMU PCIe Root port
00:01.1 PCI bridge: Red Hat, Inc. QEMU PCIe Root port
00:01.2 PCI bridge: Red Hat, Inc. QEMU PCIe Root port
01:00.0 SCSI storage controller: Red Hat, Inc. Virtio 1.0 block device (rev 01)
02:00.0 Unclassified device [00ff]: Red Hat, Inc. Virtio 1.0 memory balloon (rev 01)
root@v8a-arm64:~#
Copy to clipboard
3. After initializing, the virtio-balloon device adjusts the memory allocated to the VM. The guest OS performs memory
inflation and deflation operations, which are handled by the virtio driver, and passes it to the host for processing.
To adjust the guest memory, run the following virsh commands on the host:
- To verify the guest VM initial memory (2 GB), run the following command:
>
>
> root@v8a-arm64:~# cat /proc/meminfo | grep Mem
> MemTotal: 1966180 kB
> MemFree: 1807432 kB
> MemAvailable: 1817552 kB
> root@v8a-arm64:~#
> Copy to clipboard
- To inflate the balloon and reduce the guest VM memory to 512 MB, run the following command:
root@qcs9100-ride-sx:~# virsh setmem simple_balloon 512M --live
Copy to clipboard
- To verify the guest VM memory after inflation, run the following command:
root@v8a-arm64:~# cat /proc/meminfo | grep Mem
MemTotal: 393316 kB
MemFree: 235708 kB
MemAvailable: 245836 kB
root@v8a-arm64:~#
Copy to clipboard
- To deflate the balloon (Guest VM memory back to 2 GB):
root@qcs9100-ride-sx:~# virsh setmem simple_balloon 2G --live
Copy to clipboard
- To verify the guest VM memory after deflation:
root@v8a-arm64:~# cat /proc/meminfo | grep Mem
MemTotal: 1966180 kB
MemFree: 1811844 kB
MemAvailable: 1821988 kB
root@v8a-arm64:~#
Copy to clipboard
##### Access host devices
You can use the enumerated devices on the host system from the guest.
The device assignment interface to the guest varies based on the device type.
Some devices act as a back end for a device emulated by QEMU, for example,
UART. Configure universal asynchronous
receiver transmitter (UART) as a character device (chardev) back end on the host.
For more information about QEMU chardev, see [Character device options](https://www.qemu.org/docs/master/system/invocation.html#hxtool-6).
PCI and USB devices work on the kernel VFIO framework.
- PCI passthrough allows a guest VM to directly access a physical PCI device on
the host. The VFIO framework in the Linux kernel is used for the device assignment.
For more information about the VFIO, see [VFIO - Virtual Function I/O](https://docs.kernel.org/driver-api/vfio.html).
- A USB device operates in Passthrough mode, allowing the guest VM to use USB peripherals
connected to the host. QEMU emulates the USB controller and works
with libusb on the host to present USB peripherals to the guest.
For more information, see [USB emulation](https://qemu-project.gitlab.io/qemu/system/devices/usb.html).
**USB passthrough overview**
Identify the USB device that is enumerated on the host computer to assign it to the guest.
To see all the USB devices that are enumerated on the host, use the `lusb` command.
Find the device by its `vendor id` and `product id`.
In the following example, `vendor id` is 0x0781 and `product id` is 0x5567 for the USB memory stick.
sh-5.1# lsusb
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 004: ID 0781:5567 SanDisk Corp. Cruzer Blade
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 003 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Copy to clipboard
Note
Copy the following XML file on the host file system in `/mnt/overlay/guest/libvirt_usb.xml`
file and modify the XML file per `vendor id` and `product id`.
Launch a guest VM with the following sample XML and do the following:
1. After the guest VM is launched, connect to the console.
2. Verify the `lsusb` output in the guest VM.
3. Ensure that the output lists the USB device assigned to the guest.
simple_usb20971524/machinehvm/mnt/overlay/guest/Imageroot=/dev/vda
Copy to clipboard
**PCI overview**
Identify the PCI device that is enumerated on the host computer to assign it to the guest.
To see all the PCI devices that are enumerated on the host, use the `lspci` command.
Find the device by its `[domain:]bus:device.function`.
For more information about `lspci` command, see [lspci(8)—Linux manual page](https://man7.org/linux/man-pages/man8/lspci.8.html).
The following example shows supported values for domain, bus, device, and function for `Ethernet controller: Aquantia Corp. AQC107 NBase-T/IEEE 802.3bz` device:
- Domain is 0001
- Bus is 01
- Device is 00
- Function is 0 for
sh-5.1# lspci
0000:00:00.0 PCI bridge: Qualcomm Device 0115
0000:01:00.0 Network controller: Qualcomm QCNFA765 Wireless Network Adapter (rev 01)
0001:00:00.0 PCI bridge: Qualcomm Device 0115
0001:01:00.0 Ethernet controller: Aquantia Corp. AQC107 NBase-T/IEEE 802.3bz
Copy to clipboard
Note
Copy the XML content to `/mnt/overlay/guest/libvirt_pci.xml` file and change the `hostdev` section in the XML file according to your device requirements.
To launch the guest VM, see Libvirt. After you launch the guest VM, connect it to the console
and verify the `lspci` output in the guest. Ensure that the output lists the PCI device assigned to the guest.
Note
The output shows different slots based on the guest PCI topology selected by QEMU and Libvirt.
When `managed` is marked as *yes* in `hostdev` in the following XML, the PCI device gets detached from the
host before it passes on to the guest, and re-attached to the host after the guest exits.
For more information, see [USB / PCI / SCSI devices](https://libvirt.org/formatdomain.html#usb-pci-scsi-devices).
simple_pci20971524/machinehvm/mnt/overlay/guest/Imageroot=/dev/vda
Copy to clipboard
**UART passthrough overview**
Identify the `/dev/ttyX` teletype (TTY) device file corresponding to the UART to use it from the guest.
Ensure that no application on the host is using this interface.
Note
Copy the XML content to `/mnt/overlay/guest/libvirt_uart.xml` file on the host and modify the TTY device file as per your device requirements.
Launch a guest VM and connect to the console. The serial interface at `/dev/virtio-ports/hostserial`
acts as a front end to the real UART on the host. Use the following XML file:
simple_uart20971524/machinehvm/mnt/overlay/guest/Imageroot=/dev/vda
Copy to clipboard
### KVM customizations
The following features allow you to modify the KVM hypervisor to meet specific requirements:
### Remote command execution
To enable the QEMU guest agent in the guest OS user space, run the remote commands on the guest VMs from the host.
For more information about `qemu-ga`, see [QEMU Guest Agent](https://qemu-project.gitlab.io/qemu/interop/qemu-ga.html).
`qemu-agent-command` is the virsh command that includes `guest-exec` and `guest-exec-status`
as the subcommands to execute and verify the status or output of the guest VMs.
For more information about how to configure `qemu-ga` through virtio-serial interface, see
[QEMU Guest Agent](https://wiki.libvirt.org/Qemu_guest_agent.html).
For more information about how `virt-exec` wraps virsh commands and provides an interface,
see [kvm-qemu/virt-exec](https://github.com/ildar-shaimordanov/my-scripts/blob/master/kvm-qemu/virt-exec).
The following example shows how to run `/proc/meminfo` on the guest VM from the host computer using a `virt-exec` utility:
sh-5.2# ./virt-exec testvm cat /proc/meminfo | grep -i total
MemTotal: 1968056 kB
SwapTotal: 0 kB
VmallocTotal: 133141626880 kB
CmaTotal: 32768 kB
HugePages_Total: 0
Copy to clipboard
#### Watchdog configuration
A QEMU-emulated I6300 ESB watchdog device is supported in the guest VM.
The Linux kernel has a driver for this watchdog and exposes the standard watchdog character device.
To enable the watchdog, set `CONFIG_I6300ESB_WDT` in the guest kernel configuration.
A guest user space daemon must pet the watchdog.
The guest Libvirt XML has an option to select `poweroff` or `reset` on watchdog
timeout. For more information, see [Watchdog devices](https://libvirt.org/formatdomain.html#watchdog-devices).
Add the following snippet to the guest XML to enable I6300 ESB watchdog emulation in QEMU. The
default action is `reset`, which restarts the guest.
Copy to clipboard
#### KVM traces
To enable KVM traces, run the following command:
echo 1 > /sys/kernel/tracing/events/kvm/enable
Copy to clipboard
The KVM guest interactions are recorded in kernel function trace (ftrace).
To inspect interactions between KVM and guest VMs, read the kernel trace buffer.
QEMU also supports trace events, which can be redirected to kernel ftrace. For more information about how to redirect QEMU trace back-end events to the ftrace buffer, see [Trace backends](https://qemu-project.gitlab.io/qemu/devel/tracing.html#trace-backends). This option is selected by default in the Qualcomm Linux release.
To launch a guest VM with virtio traces enabled in the QEMU code, run the following command:
qemu-system-aarch64 \
-M virt -m 2G \
-kernel /mnt/overlay/guest/Image \
-drive file=/mnt/overlay/guest/rootfs.ext4,if=virtio,format=raw \
-append "root=/dev/vda" \
-cpu host --enable-kvm -smp 4 -nographic \
-trace "virtio*"
Copy to clipboard
Last Published: Mar 04, 2026
Previous Topic
Learn Real-time (RT) kernel Next Topic
Capture the kernel logs