# Deploy the model with Kubernetes

You can use [Kubernetes](https://kubernetes.io/docs/home/) to deploy Docker-containerized AI applications built for the Cloud AI 100 inference accelerator. The following
figure shows a sample Kubernetes deployment.

The following figure shows a sample Kubernetes deployment.

![../../../../../_images/Kubernetes_cluster.png](data:image/png;base64,UklGRrYdAABXRUJQVlA4TKodAAAvpEJnEBVherZtlhs35ydk6aWWXnJZSy+19JJLLblkZi4rcy1rySUza8kll1p6qeyYWcssiay8E/r73vc7p6pYdU59PbCAPgzheRbYMSpzIlSGhgi0CvRs8/OgSqI0ITQ9m8HxJDQBAj6Z5LIVeFSJFU9fhgoNCM0QVRLEbrS4YOoQZR94YMsTocp4IEAWAXqqzNWJ0KgqBc22JePa9kySLgy+s99YtoXBwrYtDDY2zk9qHCwMZqywsDBjwaE9hYMytm3PI+3EwcVi7Q0Gg2tvsVgcnJ80uFgMZqxYDD5rwcXF7z1gUNtmN8re77uDwWAwWAwGg8XBwWAxWAwWi8ViMRgsDgaLxbWHFmTbYdvmvAQtaYo6yAAwSJHU+07w0RjynyH/GfKfIf+Jqewk9jEa0ehlF+TdaBKkGuB/AEY03rST6Kdop/CMJQPc5c/CeIb8832dvHfwbnv8k50kP8HPu2vzT3pInYQugut53JMehlbLLXNQuigzFheldYMXpJTRXhcpJZdASmaLlAMGXzn11mcyLKqaG7WUkt2+Q6sLS1JNpvdW+Y7VwEByCOSO8iJQF1JeVhS720yGExlYqx1vK6nF5fiUlLunRlkYllAhHMhlBQUJ6sBLyDeG0QAMgA4gtiQKtgxFbqm4bMm8shRjpJOJJcJygekL66KhDhQfFFcDO5KgUgUK1OAODDgoD4ZpAwYN85rZW0kJMp4TESHmp+W48igCgpgf2AcVmaophChgMbkqkFJByuUdHWW5oqoMbWA9Tbm8OabiIgmBmphLoKMxqpBviM0XHZ9CGjwB0EYLkglpxavz03J5SdL5woBDijhQGSqCZVaXJMMhf2haSgnnHuOf3t2GFOkHwJRazQ/sB2otCeQvNh0YB+ACeiUZ9QlYcpr2r/HU8SlUI8kXIAMwm2FJx2AyawBvMJ4aZklhGQp2AFMUMImdn877rKbIZsDq0LREfWWLEOgoSzEGTLELnt8s046GJcfIG9Dq7dagGnEGCqvz06qkhvMNQUJRqIFkANiBChQkgRiuLhG7gTEASThyvxUhNscYpmzGUwXqiof5eQisEXfgmrmkpTg0LWmogGQYYzSNLegGKrA8TJR7ezg0raEYXj0+hU3HmZBtAyotBawRP1jNHEjpRDXfwKvpGxTUYcnQpo4dJNUCXPYdHJomDj4sgf4A570Y2P8KGrhmCroaz7sLtO6fMgt8ArqrAYeUmO9VEXyF32oI+GtfCHF8SgUsQ2cW2OGyEPR8F1g8PiXlqhiWQAJYY+R4Cpt+lcmWzDeWJJIEE5H53aN5yxSqkeQPn3SpBVVhO+Dt2H4qrqACTEemNnRQs2ZUmFUlqaX6nPTCliH/GYoWqZcR5yF3ow/6GBsyzuPr4EvoY9wV7yH2xAd9DBH/8RM/fuhhGFlKPdx3QkP0kK9D4WEQ8hO//uhfGFoCVJXsD0A1shXPTnyML+KnIvwgVILglu0EB6VkeR4HpeWWYyzpdDoux3IHNWkaWEDdJzJNPd1sNtNa3bXXJJ2OD7EHMekmjTRDQNUHNvVwt8o9EdruXc9eNNPa3NNaa0JN0lwCPJCm082e0ggh/K1oEgF/qYRQZLuOCR57GRrN9vuOWjGG4zTaEx6eGXtinBhFpNMa3KnRRJPYtJ2LPdeHZppdi+ZL27dFM80lQMNps7knDsoj23VMmrtyoyyZ1QRNtsn4izRI32k7k7bcERjDcRPvCQ/PjD0xgkINVdzVUakFpZnmEqRBl7hnoeczKMtmz/mB/oiuZ/5FGrQ59+yTkwCqIIRiQyX9KdA9Ee+GEAebaE9IB0jcerJnJ0VGbhV3dVAtwBi/6/hwEv4S4RTFS5+55wonJ5oHzuw5A3ld856FttQJ8bJmGoI06Nh6K6NhU40z51+VOnOeNvlXteXFR2kzR3tqAp1XpSaIT+MkWXPgZWx3Jk0KrsWuq3n5eVbBKXhVqi1Bx9IDgr6upzSkQd+jLOAjLaxxz8vPY4gCHaetV6UONtNwmbqgPREOEvkz5/HC/c/8KMudhaBxHbgWv98gihxjNewB4iBo1C+9rhWDdk3aMAJp+Nvs5EE46qaBCWj3BPQLQOCJgjiDobNcBuruzIJRi3saYqLJNTah09cz5+Hv/Ak6M2aCNPztQwqPuKoF7D7UhexJe6HuzlzNqEWT9k6+wYGXkRkpaNwNMMjSuQWCodFkjr4vA4MvmwNnAAdQgVcj1N2ZMGoBuhI9z8AtXkZeSwfqA/CQHp4dwE0UaaAGjQb8FBiHGQtE+L+Vx+Y83BMAyisyCxfq7mxQLe4X4JAO7CbNM87Pqp1uZXwUADSQHvtjq6a6h4oOox7AW4M7klM5n8uQSHONT7rUPrhiNFikA/TYFwCk1T1UdJj1oErq7iw5liFDgm9cM6B2q8erUgLERDMdqyNOJd3Um0jHpaSbgaAzoS/RFTTTcSjp5q7gLj/0JbZhZ3xopuNSru6FvsQjMDQ2ez0orzd732E7MAmuu+C24Krs9cp3p7/9wwXp+1cpbva+P5yrU4H8uaCdEKbT/7dpCGESoEFwS3z3kxAdzxUHpH80ynJTzyOUV4Lbwskrr0V/G87V6ahNyAOD/gYXLI7O1R0EWOgod8FOQ5Doq/8RqsCgr8744MKLO54rVXw2ez9Aq6uvr3HE1TMmmFXMgYXqs9duXkHkb17hiJA+kBSi45n47lk978ubjvlOiBw4TJur403m6rYY5aQfIgz1wsOwC/oSbYhT6Y8P2xO+hL7EIxKPCuJMcBPSn0jfF4dyX1p/4j5Dj5H6Jj5eWnDS1Vz0ZXz0/Cwn/aRr6BmSpkaC6y4XAX8+weSEEMKkbzDqVIN9gah2nCtnZrTaEAK1bargOzY1DMOPJWUSNGlN+u5ZHSqH2B+wWP5izZnsmJMalyBpOuDG/WRwS47UqQO5uZRim9TIzk1OKIoV0Fkc26ROk0fqjt0B6AccgVR1PUn/h3rPCEG0ksGNo7M1pOTAB1VAV2DUr2mMUonZGtCzQb+yxR9OiLm601G6FS5BcN3l4L35wys2bN9wUKatfkaYYJ4Aqa6D3mLnJo/0qaAf8ADYJoZofbVvqghzdWiGlMFgDrSIkT1bG16hmqNAL0/6EVDuq5DCsXM8g/zFWvX1fVPF6gwzOaf6+r0fryZy+aOjDGwhhFldr66j/QEPsGA6mOQo0QcGasyIuTr1BHo50Mmc/EUJ9arPBk4nOAZJOqt4fS3ZcdLwHZkvExRY5M/V6Yp12sAhMzngNHnkxTkAasdQikFwDf87W0syYSiTAngyIb0M4gyvUKc8vwBMXsUomIImfTCd/FExW6MtOjn7goZQFCszAqxuCAGcRI4+0CcJ/fZ+PFAPBsTkcceo4dyHGVTXkXJSkAJ4Y4jE5gpgRohRUKe+OpsAq/nFjpKzNWewREehzh6D6eFdeANswMM+RKg2xKjKOYvwn5/NObqP31vYNo+5vaeeT37S1Tb34/aehkSifY5eMd87+1MTXrbPMbSd6cjl6IxolMl9LcE/6TJKJVvXYKe4bdtIs/9TMG9p76In6Hhu3iKEIq9dQ5k4rOR52e5tFK1tRlZxoa8TI1hs+xyDIEo8jO39abxlJMGENPXtcwyk0V8ZSySyD5w9rGTrY09TF+qBXWiBJTr1VpnycpqP3bZyhQzOLMAbHP7yHZ0RjTLN+Q7wBuzRHbsAkET79R73r+kAu4P/9iV0Jh7SpwUyYKtBekRnRLNfCDKBUAG5QLAEeYMRHnYr9ZUR85YxLQWaPrDBLgAggdU42EMproyQ5j1Cp7Zf3o9X0zcYUYXhAl9AJFindOOOx/voFHRXh5VsFY9tF5GwbXb6t/30ZAE4aSCu24QNdqGbjSCJ9j+xFUKMcdJnUF1PRiyc14b9B/wtaa50ZP5l0nEo6WZXIGJX3PWKZpr+y+yMDyK80R000/EoD+5zYlZcfQgSCHYgrLLWC27qh9seOMu7MG8Ra6CBiwfOhuVq6itiBCjxN7axgKZ9pY1DuHjtY0/zOT6v5Hm5g15H1S4i5cv7hVgDF/h3nlr2Jf/0loDHnkZXZY8B0M0A1LR19YGzSINjsXZlhDb32wiHt0U6nGVc4M++5N9/RU1uCcBXwJALrCkmeJfH+vt7fQu9buuBs/hGAWMByOVoYrZmELmNDtF3wLkCHcs7P6Pbv0ZgX/Lf2d86xlWqpCNdeeBsdoxewgpuBsicVrlRwHCQv7mvh9cbRcgVgKR5U+h+xGNPgwv82Zf8q3DYGyHcBi8hfNljT7NvFNB39nq4GJx3+IFoIuckheh4QnM7qhpGSDcgbZ5AWzwdmsHl1exL/g+zIXbbxaSlyJzW/01g5mYr+ZtXaAJnN9P0jowlX97/2NMMrsCheQ1f8q8OlRJ0NZolj6CbCEaiHD9lkBhkDNAHeY92YFJcuSR4VOqTRqmIoQQ2dLBwRYjt3aTkfGkXEVzgz77k/+fZUKMx2mHwjQGP/eVt0olxo0DUEiBVKtlKD6ufjZ8i8bGYBDf4ODbIUMlmi9lsxU9jR/1C9dl1ulB9NsnfMFYuyIiGwSJARSSLl4rF7H+W7cTHornocCdEzqH7VxRkIyHIU/3zR4UQuZiGfGcXIhh7II0Jfno/XYBWYyz38J8tArPFLiq3nSK6F2LxhwjyTt/XRed1EbF4uItuGBEuFRe7mWn0MGuIU//ZgyLqPn5a1zNe/F//NYlYPNLFLxiUopPQcsu2pgjQIcnihU3L9aL649hYTf//SYTCuNAqWa6nKUr/SSDlYjeXTMmy+SB+njWEV34ascwWF3vIn2lZLv8j03I929MQtu36aXrI2Uo2yLmLupVxyx7/o2R5XkJDeN5n3VLmaZubmUzJ9Xggmg/HvLJbKmUyrRL9L/dbHcpM1/MsEGU7wTNJeJ7nlsuel+CGhPA/9P8GAf14Z0SiF0GA+I13DqaQYSUZ3BZRi3G/3/+qD/dGIq5ebQOIewdRXN0XGjBzhLieg04kYgcuOIMu9oUEK+kEuqktRpPm5NIKyiAhxxdBX2xOfkaLmRzOIEEfiz0sTphjNydjD8gT4W3gosRRB2eQoI8McPuNbFjjnXu4GnlmYJ4INsVEGSjoKcUwBsHt5t2Iw0w6QVGs5I+K2b6pIqtPHhUip6tcCltQ3ve+nRQGCtR0k2aYiNT52TidasN0HJp9THUdMDkxqhGGJRbMX6yFRvUYE+DBd+g4E5LxRvKV65jk86jHiaMoRyh1XnmuXl0H+UhNiLn68MrkkTp9VsVsjbo+iZ6hcpC1CfZUbQgTnLBKmtTWoWlNcRnWaWI7/aPBgHSkHJoe1CjYTp31pIkyPKKFnas2yBI9QwVczZmc4yBrE6UiNXmkTk9YJU1qCw67+AygxIB0pGpJE+YIlb9YU2dyqa8HmI8UgfaV/z9mX8tRQXoyVYDOhKyJCGFGgN0RoO3RUU6DM3xthQFIR2qFUF0H22konOTj1sEwTJk88qS97w++nCWDdjngYzqogNYQmn8UJocmzJyG5GwN5gulKFZgOlImuKMf3NavBWdmFOQTNUmThiIJxAGNzRUAXA2sEUnBKIDtjAhxNWvPLMCvimPxN2E7h5U0o/b8bHUd5noGj/neMQG6FzPvw1EuCPxdQkDHfLRImsy8D+0cX/gQIYqLySN1iOkgWHkfCiFMTgjZNgRyZDUvJH9ujaaDCA8CJ+bqLEASiXEwd8UMHomI7oijcS/nIGbcW8U5+N1sJO9Szb5zEMUO3Mk3uHrQiQ2xb9+Nn0maDGYUxQrnY3JCCGHC21eqxxDVhhAmIn+xlr9Y09Z5hAC3AAhwNdfMKNciaYKvjoC3r5j0YifWBVHVhjAnJ4Si8DmJ2ZotwJ0tOcZlUY5toosIbbO6TiRiEQsL1AFuBwKqYA20p3gIcG3T5NIKvH0FjNoQ20Q3ocAiaTrgzhYHXxZFbyOcrQGq69XGaExaWGhfdqK7bRbI69q3FdmuLzolI0Tbuek/JwsLXfkgutpm4Xao8eQFtKf4iZPo9hU7hxfwTSgIOwfvbHHwh/lA79l1xPpMLgax0H91Sinav4ZnQRa680lR9LWC/Ow1U0qhq2Vtt46wMLX8GqqC9rQQDy3QOfDr+6YK3L5CwYWDbkJhUV1PquIMr8BnnlTX8+fqMYjf+lrf1Sn5qr9NFihTy09Gk4r+6wPgt+bvJwZPRnuKh3BmhBCj8PYVcK/JbI1BUpBicmJu7WIN3KvCBgiPwntO0M0wsQPG8mtIVwBcJu8CXBAnvnaWGqA9LcRJO6ItW4fOzy5cVnJxjfYCym/Na4H0HbSnBY73IQLoAfRHkJAKrgdp9YLGa9irb9e4mucBTyQsnJi/n77sBHzZAmABdBayBu2J4wFPy95OuwE4P3tC0POzU8tgUAcnZhXFNbyn23neapWPtG5fYIfaJ103nLj6qg0iVkQQ/IIYEze5RLFY5KAUs9lsUYfoCkTsirteAdkZH0R4ozvQeYrZR2GjD9mifsD4E9MiDf5lIiCr5xSzPwP5FXlrtqgbMP7EtIjQv8x9LEIJvaKY3fjKxsbGo0H8dIcYGBFkX2gPu+r4utApZD82fgTkR7pDuqk/kWYw14dQYgf0qnA2nvzkjYMH9YZ0szd4MDzRkfAlYHQED0Y62hM7Eg8O+niomdb1ha8cJIW8VX94ZDCyL5xxVyf2RToelhjZN9jjyAt0nnfKjY1+yAhvEqCOTVwI7ynTg5F5+g8zHo4LzmCPHWDShztDiXt162W3Phryeb8RSd7Xw5KtZLs5+wmxCE1P8cGP8dGCBrvqMQR4dk+M4RdC4x26dX4224dsJM/PPqaH2wWadOF/vs9PEXvQ9BQfh/kYHzU02BEHakexoQL4RugTR2ESMVGIDi+oNtNitli8FDl6yDh9Tc+LOWh6io/DfIyPGhrskiZq/+DZPThfGPA9ZEDCcHPpElGLHAE2lSzsWIt/XPZiIHnGY3zU0WDXN1W0B8CFmRwjXxiwBJKHiWo2QtuRMbn66omo1RcXe7ieInFIa5w6cKF4KfoWtDzFpwa/DVobGuyCAov8uTpQgCDrwUrSDAt3hvghgiHhbkYRIPnp7g7QhhBBtotR/7IZIcSoo/IYH5WXabDzwXSAHXx2z0eOCpED4C9+DhvVdUfty55DPww0CAtFzRExSgTyE5rgZsste9ojRuwoOVsLYUcRedZPGIFf9lyDPQvk5gEPA/FRYC6JDwNDRUcvbClmNUcxUnhBUoU4Zy9U/PQZ0tZDINaenx2MzNWTJqT6+reZ6DCQkb0Z6zBQx1G/jkVj/Ea2GCnKAVKlkq1U/PSbrZLltVNoD90HrwH9YbaGDgPxgR/zMDCcSHboD5qbQQSvvvLcPw4Qg+z9xC7qzU5Cifx7etrljAT5cx3ucugw8MX0wI8FtQgjcvfUNIQI1MWAwVdOQsxP53f0Ff2RvvDODY0RuYVEO0XZbWVIlFzLTiQ+pjUSMYFqQ8y9OEcfC02ecR4CGrLUrB6LNA2hKP4XAA4AH3dsJpc/V0dppTTggR/zMDCsvJos9d6qHW8rSYIQWCuxJPVwNXOT8P9xLavU+ia+Q2zoMYJ1+BB+H3LO8EqoML4pAmYddaTu2DmcoiaLwfohQlhZDdTFam99JoHC5YIEClt697JLEcTzypZrWeXQ2nnMKGY/crHWUegfzTr5qp0wUV2HB1r0cegfxgdXMYSGID2iAFkVQhT0kEgWiQQ4TvMSH4uFhdNRupMXP3KxX3PMCRtgegC+MYJxOBVTCinxAlkrpc7uKIIRE0m+8pjTUboOXS5czOQgk0deDFPUjDXQ7VfhJgMOKSGuW9DHl+kj1UaOng0NA+ibIiBOcN3N1cHhVExhC47YKm9l/NCPHUUDhueTrjgQfdwk3rm3yhhwSX946MHwxF/4EiDufTDi8XeHHxz08QpDsECKOOh+/1DRyY9j9QIdjFCIo16mixEGIhWx8RJBveB5+hghEoCIYGSN8o7e2Qd9jHeGxJaMaGwZ4x1tSb2MrcEbhvZZ/TocQ/5zA3/CF3yPcAbHox8bYY2DsZg3SSMabxrfzEQXvcRdjB/bxRTW2EmMvSR6GfoYn9zHvpe94Yxexs9kMi4hquJ94Y3vpIy9JGyrlGlljGa0Wi3Li2W8Lwbjla1Sq9QymFFquZadiG0Re/HKlutaRjNc1zOYJDzP9gxo2F7CWBC/xJD/DPnP9a7se7IRjbcHeY+BcIEbAHcSPduLdXCBW6n3WWXb45/sJJVc1+af9JAyLdfzuCfjrZZb5qBsZiwuSmvIf65XxV1b45+4QabFSmUxSPJiDFLKaK+LlDK+4NRilkTlb20vypGS2SLlgMFXTsMyLKqaG7WUkt2+Q6sLS1JN5vLxgYGMJwiQKlnwjVYVPxvPi2rkjvIiUBdSXlYUu9tMhhO5uiS1uByfkiSJzWEJFcKBXBYFCepg/OkklCztHLhQhHJBDhbJtTPKyTeG0QAMwA5SSrQnKdkyFLml4rIl88pSjJFOJpYIywWmL6yLhjpQfJiflgSVKgCklFIaWdT/hBZF5NbN1XXtKC8GHFLDtAGDhnnN7K2kBBnPiYgQ89Ny/NMvEvMD+6AiUzWFEAUsJlcFUipIufxfynJFVRnawHqacnlzTMVFEgI1MZdAR2NUId8Qmy86PoU0jHTRcstan7Nv95CzqAhyalleFAPaaEEyIa14dX5aLi9JOl8YcEgRBypDRbDM6pJkOOQPTUsp4dxj/NO725Ai/QCYUqv5n6PWkkD+YtOBcQAuoFeSUZ+AJadp/xpPHZ9CNZIGenXJcr3PagsvyG4RrfazbVmeF9WQAZjNsKRjMJk1gDcYTw2zpLAMBTuAKQqYxM5P531WU2QzYHVoWqK+skUIdJSlGAOm2AXPb5ZpR8OSY+QNaPV2a1CNjDZaw+0hg5ctBjlnop7C6vy0KqnhfEOQUBRqIBkAdqACBUkghqtLxG5gDEASjtxvRYjNMYYpm/FUgbriYX4egmtksBe0Rvl93ZwrJIIcOykzLddLRDfXzCUtxaFpSUMFJMMYo2lsQTdQgeVhotzbw6FpDcXw6vEpbDrOhGwbUGkpUI0M9ctCGJ1ty3pagPj3fjabrUwn79ELcyClE9V8A6+mb1BQhyVDmzp2kFQLcPlxh6aJgw9LoD/AeS8G9r+CBq6Zgq7G8+4CrfunzIIBx058TFskPM8F31XaKrmfjXrorgYcUmL+fwi+Qm8PDQF/7Qshjk+pgGXozAI7XBaCnu8Ci8enpFwVwxJIAGuMHE9h068y2ZL5xpJEkmAiMr97NG+ZQjWS8cEnXZ5nWVap5FqeF/2fdGlK/43twNiObaYhWzimI1MbOqhZMyrMqpLUUv244cIWu+xabtn2uOaFLQmylW3bHie9jnXIf4ZchhdkG2kscd9uSLON/D0/dablGkoSCc8tGc9kIzOtTMvyjCV22SqVWgY020jXThjLwnYt1zKg2UZ+1jOWJDzb9spGMzzb9hLGgkESnuEMTpqxxJD/3BQ8f319fo58NhfNke9rNyzNkc+68U2OfL8cJAUZ7+afT05ZrixmKz3kx3BOupkWs9liNvvP77N5Jp8nnsVLxWL2VNnmffjp/XR3r2mObg49KEK7Ravs8TpQuz8FoodZS5w69c9ZQDG72EXlep4hKeKB3E9sO0hazBYpF8hcpGxQiANyP/G8P/7nCzD3kzdnLJdf4gbYVmi+R4vdLJ8h8MsFz/LT/vNiZfEPPtOKLQtSymivi5TSgFOytF+J5pWt1muetrnZapUsOxHNoDQjmKnChUVVc6OWUrLbd2h1YUluxQnpx31MGx8Dz+RtZTKtklv2jEaOfPHFx7Fe2QLheYmEEcmRL24g4XllEraXSBiTHPniBhKMx8ga9Bz5DNh1rIYkR754BCOSI188gkHJkS/+wHDkyBevYFBy5Is7MCo58sUhGJILW4b85yblUQteiXrwSN63WNEQ/8kf8fw0frqtr6iGyFiuxxdz5Et4biuTabEjk/lty/Pilhz5Mq4+Y7uW1VILyxoc+YN5biljPJPka7VKlucldPmPp54knFu2vcRgoGyVSiWjmiOfLq+y1cMbLL2ubLmGM0k+13JtL2Esw/Nsz3iGTf7y0RjynyH/GfKfIf+50TAJ)

Tip

Use the [cloud_ai_inference_ubuntu24 Docker image](https://github.com/quic/cloud-ai-containers/pkgs/container/cloud_ai_inference_ubuntu24)
to quickly containerize your Cloud AI workload.

## Prerequisites

- Install Platform SDK on the Kubernetes worker node (required for QAic Linux kernel drivers and firmware images).
- Cloud AI devices available on the worker node.
- QAic K8s Device Plugin Docker image available through a local Docker registry or preloaded on the Kubernetes worker node.
- Cloud AI workload Docker image available through a local Docker registry or preloaded on the Kubernetes worker node.

Note

To serve models as a Kubernetes-native InferenceService
using KServe, see [vLLM deployment using KServe](https://docs.qualcomm.com/doc/80-99100-3/topic/Kserve.html).
KServe requires a functioning Kubernetes environment
as described on this page.

## QAic Kubernetes device plugin

The QAic K8s device plugin exposes Cloud AI device resources to containerized AI applications. The plugin supports:

- Reporting the number of QAic devices on each cluster node
- Monitoring QAic device health status
- QAic device allocation and cleanup

Download the plugin from the [Cloud AI Containers](https://github.com/quic/cloud-ai-containers/pkgs/container/cloud_ai_k8s_device_plugin) repository:

docker pull ghcr.io/quic/cloud_ai_k8s_device_plugin:1.21.2.0
    Copy to clipboard

## Package contents

The plugin source is available in the Cloud AI Apps SDK installer: `qaic-apps-1.x.y.z/common/tools/k8s-device-plugin`.

- Plugin source
- Docker image build script
- Deployment scripts (YAML)

    - Device Plugin Deployment Script (deploys Qaic K8s Device Plugin as
DaemonSet)
    - Sample Cloud AI Workload Deployment Script

## Device plugin configuration

See the `qaic-device-plugin.yml` in the Cloud AI Apps SDK installer: `qaic-apps-1.x.y.z/common/tools/k8s-device-plugin` for a sample QAic K8s device plugin configuration.
The following plugin options are available:

- Allocate by Card Type (QAIC\_SKU\_BASED\_RESOURCE\_ENABLED)
- Fractional allocation (QAIC\_FRACTIONAL\_DEVICE\_STRATEGY)

### Allocate by card type

You can do the allocation using either `qaic` or `qaic-<sku>` (std | pro | ultra).

- The `qaic` setting doesn’t look for what type of card SKU is present, it
just allocates the available resources.
- The `qaic-<sku>` setting helps to allocate resources based on card SKU.

You can use the `qaic-device-plugin.yml` file to set or unset the `QAIC_SKU_BASED_RESOURCE_ENABLED` flag to enable `qaic-<sku>` or `qaic` resources as shown in the following example.

env:
       - name: QAIC_SKU_BASED_RESOURCE_ENABLED
       value: "1"
    securityContext:
       privileged: true
    volumeMounts:
       - name: device-plugin
    Copy to clipboard

You can use the `deploy-qaic-single.yaml` file, to specify the supported device resource, for example:
`qaic`, `qaic-std`, `qaic-pro`, or `qaic-ultra`. The following example shows specifying the `qaic-ultra` card SKU as the resource.

spec:
    containers:
    - name: qaic
       image: ghcr.io/quic/cloud_ai_inference_ubuntu24:1.21.2.0
       imagePullPolicy: Never
       command: [ "/bin/bash", "-ce", "tail -f /dev/null" ]
       securityContext:
          runAsUser: 1000
          runAsGroup: 995   # Local machine group qaic
       resources:
          limits:
          qualcomm.com/qaic-ultra: 1
    Copy to clipboard

### Fractional allocation

Fractional allocation of the device resources is supported with the `QAIC_FRACTIONAL_DEVICE_STRATEGY` flag.

The following example shows how to allocate half of the device resources.

env:
       - name: QAIC_FRACTIONAL_DEVICE_STRATEGY
       value: "Half"
    Copy to clipboard

This example shows how to allocate a quarter of the device resources.

env:
       - name: QAIC_FRACTIONAL_DEVICE_STRATEGY
       value: "Quarter"
    Copy to clipboard

## Deployment configuration

A Kubernetes deployment is defined using a YAML configuration file. The following example defines
a deployment for a single Cloud AI device.

deploy-qaic-single.yaml:

apiVersion: apps/v1 # for versions before 1.9.0 use apps/v1beta2
    kind: Deployment
    metadata:
    name: qaic-deployment
    namespace: kube-system
    spec:
    selector:
       matchLabels:
          app: qaic
    replicas: 1 # tells deployment to run 1 pods matching the template
    template:
       metadata:
          labels:
          app: qaic
       spec:
          containers:
          - name: qaic
          image: ghcr.io/quic/cloud_ai_inference_ubuntu24:1.21.2.0
          imagePullPolicy: Never
          command: [ "/bin/bash", "-ce", "tail -f /dev/null" ]
          securityContext:
             runAsUser: 1000
             runAsGroup: 995   # Local machine group qaic
          resources:
             limits:
                qualcomm.com/qaic: 1
    Copy to clipboard

## Examples

- [Installation and Deployment Steps for Kubernetes with Cloud AI 100](https://github.com/quic/cloud-ai-sdk/tree/1.21/tutorials/Kubernetes)

## Next steps

- See [vLLM deployment using KServe](https://docs.qualcomm.com/doc/80-99100-3/topic/Kserve.html) to serve models as a Kubernetes-native InferenceService using KServe.

Last Published: Aug 25, 2026

[Previous Topic
Test container](https://docs.qualcomm.com/bundle/publicresource/80-99100-3/topics/test-container.md) [Next Topic
Serve the model](https://docs.qualcomm.com/bundle/publicresource/80-99100-3/topics/index_Model-Serving.md)