# Run a LiteRT model on supported runtimes

## LiteRT overview

Lite Runtime (LiteRT) is an open-source framework for efficient
on-device deep learning inference. TensorFlow provides tools to convert
pretrained models (SavedModel or Keras) into the LiteRT format and
optimize them for edge deployment. On Qualcomm® Linux®, LiteRT models
can be run using the native LiteRT application or through the
GStreamer-based Qualcomm Intelligent Multimedia SDK. The framework
supports execution on multiple hardware accelerators—CPU, GPU, and
Qualcomm Hexagon™ Tensor Processor—via delegates, enables running and
benchmarking sample applications, and is optimized for low latency,
small model size, and low power consumption on mobile, embedded, and
edge devices.

The following figure shows the delegates the LiteRT framework uses to
run models:

![../_images/litert-arch.png](data:image/png;base64,UklGRgokAABXRUJQVlA4TP4jAAAvSwNmABWL47aNJEl2/llXbV092HdETACfKgEC9CD/pbtX2s6PiRd1twau85GKVxRrTjxfkURdKW3bzKqPimyXLkMBSBzZlkD8FgddT65WDDjio5LkMjdG2R3iDd9QzICk3TlL1f5/R27zCoaGhoF5hDyGYR5BjyAYKGgoaCgoGGho1nuddLH0vkX6j/4z879X0Zzd9B53p/eIWUjwh9zLRek9Cy3mIyShXObuhe5G8rnpdVI3bJnOIg3TMr1AemDcuG0jSPz/K4OtpkzpwyRzTmvbqmNhh/99qCeRyMjIJykgyUz/YUGSpFjJxmGRvt0n6PLrRf4/z5Ek/7vmHGfgwIULBRsWHNiwYcOFDRcubNZm4W8PKTuVaZOp/ReOAnrsX2n/h6WZUFPBRmvh7LX/ykygjhBUzJmztCBJHSUWc/Q2E1CxMLFAejMFywts3ABp20b6/ad8GGjYYywEuBEsKraLwWKwGBw8XLz+04Ik227b9C2hDASPxQEvSVEXsvPrRds+L/7c/4vMMnoZWcLILCMyMksYGRkZGTlLyBJGjhyZdTQP6H4C1a8wfxKJxCGRSFzJWz0cEolDIpE4JLI2ULKPowQSicQhkUg2UK4lEf2nBdt23bbZSNkCo2Pdbgh6kJTD75txW27Ela8umjFnbMbMGXfqnMy6Nv/K2371y0lm2mgmLaQ2XAPNE+vRUE+GtrAwssB10DwB4EULwzVGkthzgNNBQXruJuwYsnB6KOi5cuuDPXC6iH3zvbI24jBvNOcanD6ybPtZncTb9p9OsuK12H/doUOY5PkPZmZNOaex/iGR3/U+afj1uQ+mZxnTWE9IBEjPf7aatPF7QiIi+E5+fd6DhfHbb0PjBjM896mHi1n8lf/uaEYkP4OCJ7J4l0sz82KaIxTLD5KfQbkTQHIOMKebCmgTaO1n6JTy2/4ZHERP/VXTOSZV7KT7XmZQkQXfxAcEc17PvrXfm2n6bqkc1h9a++OA3LXzGf4Gebz49GzVTvC7//j3lqp+YCBFQzeXbDs0f7Y7TX9ifpyCWU/TZItpmgjJbfobUPpOlNByP+fTZtBygggAQDIjABfUdigmbUaQ21RXzc1DCYmt+O98qhO+3UaOHih3n+MAQgR1gIgiQvL9vvXX2Bxc+6WpDQR/9L9D88OzL9Ua9Z8+j3ul8/Gz7xS/0xnrHfs13KWpvPWgE8RGR4N+/HTaDKfb5zAM4QK4cHnzNgJlKAYfMt2meBvOk1H+yAy8dcBgu4xzxnM1V+iAHbfAlQ12KmTLVwxsgDZTsQGuXNUEmjjD8ThwQsNZs2+QlzkuyDbiTjD7cItmIj38gDFCyOKxyGJf2WkC4WZVa9F8McazuYOUOXsljtjUmg2N/Ya6J5JrU0M1urgL413kTC5T7iTS2MqeankO1mUSIai4hdzFFEogcYXh1tPF38rNAEiq3viatYowXH9T0YdLtY9Fp+3osrmbPjTVW9h0KMOlegsZV4S39ft5q4L3Il0krUY1QuCLETAfRTP1wOSVGWKCoB50w5RZx4yJCTB1cdmVEJgxxGNsAXcoLrUxZnyfKrfgyFkhIUS0ICQJjZJtp58XybdISJp/fmz1/bMed7mjg4YgSbQDaLnIdiSnlmCBkFr+ubH10s/zxv6Ly9OV/G972gNpGiL2KT4O+4xJQ2MzBEWr+jj03WfCIs0dBajAveO8+njnTwufTI8ziAWc+uMcGKQVF4DJLoxkJPrNjtuwmB2x/2L/xf6L/ddNKHxb2+3CNpEa8OO//vlNbQ04DyPhmtEyIjSo7//yj/6b2hpxPsK15ZHqhZgGRGh/71TE7cxEaSEQlWFsFCLuTdCCanSWdzBRFw+l6Kzfoi/lyYj9F/svnonHdxB/DQJyHLc/jeOo8GsQ2R9Iu/0qhEFFyGF+FeLfDhN+JYphZv7j8asRfb9vqsCpOQijrJuI8VD6BcsCArz6eAhlZnwKPR5GsRAPHCsaMeXASggHPLrPNNYjp5iVEI6p5zbpi3JKal6FbBSFpVc4muoIy0Dsv+65iOgHkPmJMvNimiPagwCiHQCqu+Ynok2Q02TFQVAfVOkEqUiQ3G4qGJqm2byvkRqtDZBVh89fqP7ggB+nATnvJfGd/PhOVQ0j+/7t7W1zztZrVyNy3klyRnsvqQDAZEu1XaAzq5JJW5ih4HPRoZzOHA0UEVRsIIpaZWViBk594l0CNpdjigUBYMcloQq8UkFmsOYEGN5mjsftjk+QFY6dHbL202nAjq21rD4FWeGobVStIjPm5lcuRHJ5C9XoKtjZfsUyoxp180DLVaYqKWenlYrwrpojmrArMXHtYzDajjwnrE4MO9JKc+qB9L28l4J1PXoHbMk+ytM6s/erEC+C3CGzQsZUiB6C1FS0KLIaudcj/1YOOqYiJwQJpkVZLd2D+Ff+i+PUhe5ve640aSqn2/Q+KxyFqjFy2gqB6YMTDQe7cquYtujxVR1OOBx8y2AJQZb9O7yy9mMV4IUYScESnsvI94hqe+bF8Q7uURD7L/Zf7D+1hdG/5aF68F4xZIB6bSWP9FkoJzjGZcX8VAARVJo7DJu4za+KCVYaNrmvDiI1Rr9BCOaZCQY4rZitBdC4L+X0Ww5HqCVFkzJaMT+RQMGwslM7mJ/7HA0gJ4VrZey/LglEnmmJyCNEcNd0Is+TRACI/CydWkQeDoFLkLt28edIAph1W0LmRIEsmBDygHiGiOT9w+poAD0F+n0pz48EbmRmW0HmdBDSgeb2G7d3kIcFyPsHP9IA+nrYcw9ZRlAWQGUTZujKHe+R10G2bEgHmttvWk4PDEyLtkoQpFTuK0MCQEgcz59qgiwfFICagI0AAcGetxAgFsjvIFWCSeWa9ptfIOhnQBpUHHlQ/PkHPgf2QH0JheVPFRhjxlvEwGFaSrfv9gFC5st8JFAH8nL3qbJBvckMTDpIjUNRBMsD8NFk3rgbVZyHhvy3n04d6luLMp7LHNrMMuLfG1AWMn/mY8fFSLS8zUFmcOAtutG6HA9xxlsI9mxQf4Yg8fp+0ObOpk07n5Exf+ZjxzUjYXg/XwEVp3r4thrpeLgPv5l/Jw6T9rmEDfszH0uEdxYisP6x962UiXO7MwF7ZXzHn/lI+eRHDxJ5mnkAMyUPUBSPlMZBNEzUZllnXWyGBhgeOjsDyhg6eGtgisyW2yVdZRpTFcM+HwOIvq6F0qsPuoefTgnmkmm5BQ68V6uUeOOEvsn8wtuHK4Rws1vAFRzsJuOCfYcsoSFCH9fyZT5ScD8wW7ZvEJQXDjYypAw/nUYcfpwuwd4+VUjZcLhF7qQLtwh5MPoEtVxQpi6b6dq1gwi6dt07FcetE+Q5bbwF5fcBzlBmu8jvINuEQ9Glda/y5MZpIQ9pfLpaTgKoxQBEGwCsxYE8ouMAAQe8qoUCOLwsK4ADgNd5fjrpD3M7CA6FnmdOv3lI903ljUCkm4mWR/Q4L48nFPv+t2nfd7yo15t9/77a913hVUnJbvu+/wcv9HtT8jK/g2SbLdhvlhDNSlxXi2V9TGVM2p9fFD8wadeLkjIwaTf1KvBt9s+wYL95mF4Ei349pjOVg/3pVb19+45QvCopMyF/NR1/1s1jgbzJfOwNDn/Fy9qpYJm5eVlSftbMPOHV/54JkT8of2cLZubyfFUYJMxavi4p6Z0Dvyj8/qNfd///n//26/HRMn5607oAK0XdeIGJqsCv/MP/PZC/nhQ+Ze8klIOslF6fQUkREFwTFrb3+pPvNt9s7inuaEG8SI71+pSWXn6VUmBSSmAz5hEgHag31THc0toXZXM4VjTcspK0lMf5lTL13BaygccPGH5bZb+4N1118Og+hqXUCEuxWxmW8kbUNK13p9PckaM0Rt3dVwGZlRxLMQWQqbEs5VWEHYcDrDzf6VRZzsTH6XzFUp3OKyz9gbD7O7kOKxldyDDVSig4Rue3YNPcwdJAp7MvgP5+u+03bFI9BqeUOLxh2OS+M3AkHNC7ksuuQSycYuJQP7qSR/YVEET7MOIibxx/XgSSJjeDiCwwKkxEcpOIPDnwXjFBtD8wLiTv/jIVIf8WyrwMAES0THrPJVlIEm27P6QF33IIbYI+dUL2VCIZ0AiAS53qml1yQMra1xIkkzF6P0/7toNImUqIwCTIKkDiHUS6S1Hi8tPpK0RwCMAbn1SEr7wHWLVbXUJQ2gt9+WsLDrcwA9DyEYIt+1WAF8GBE+CzrogLb/Jwgu7knPtU7ZipLCfy8mfHHmVtDYFk62xmBeBFyuIzjJ0guAQeoAtNp/MA4NkBm2kzkW5uHQSOPWFC7ORlNQAz8BDPTnmfrIFmyx45RFCxgeH2zZaZUJPv2qq10hdNiRWBF0HNvIcoZYg6t+OUPxlth12Au10h4BM9BjLXJvSQl5WBxAHyomDLla4rmo6Kp9MucgUED4wR0vErOQeRzJlvaZUgWn6DEKhtr0HLA1XEOO6S9jvi92wA9nmTc2oKh5XB4oqvo+1BuTDqcLjt82YgGJk0DxS5i/cwdlwRELhzRG9HSnDloCtDyFlFyMeFKWt1QLZZmbsZ9jxAVgNqa4A33oMQjKxRnyzqVMCBHQD66bsdHdy77bWfER+4hTz/U/30FSKIZ6fprU5Q52tGepiiBtqCzxxaUNI1wUFWAAwdB+iS08cBmgS4RPptmRyQeiLnACZBgHadkDcO0KVnBZHk+9PnZsg/wELj0yHPmps9KvdNqYT5B1jsvil5bv+51rI/Ev8iiQXeFMrP/1AVEVkmdywAQWVL7l7OMN8BFbC5H1YI7vq8L4ulhiwhESDnb5FMs9+DrGV7TyUTWR4iwJwaiORFyLNsaQ+oDAnclCB6CF/dtUfK/CDte0yQpSNAabR78XPv+i/fshWkyeWV7Hu52/s/AWRZCOD6/BoIIHkRcpc34HK1cnRfhw+oBImg5ytEe1ZHU0P0p3V6AMl7OZHS0OCS89A0tvEGoidRAZaBIEVm/mIgKJj5VvhOtJu3Wm6hlwJrNnkl40HLkFMoWiOayvV6ye6boG2Y7akTQQj0x+gF8MpXFeEXzgso4Ne8qoyRWPiCdqbTqvKUIwBF38xRoj+tkwOoh6Ge6so9REuhj8btoadWOQSQ+ycwdtP27SWUQBFnOB5f7Yj5BKBoUSitJNqLlFYjSGYXcgp4vwTtbTx+HWyNDB/ynB002p+Q0auiEVn4grb8S84FfbIRlKknAJMACnCpJA98lxwcRMUzAMqKTSkCfDWAkNieXQlBl1QS6RxMD1kCtXUAvvKQqQFg4DSXoE+lopWBAvrUkZKZ1HWO1rCHwB2C60QVyEHuWXMBgIpbIFhuobO1Tr+gGb0SlErjIFoqWgd6QY98LMnFN4A80Qgqy1zzFWgDc5GgoByZ7QCBZ+YDt8Dpxtz0CMw8goSN9LHYMPOGJrkkwNuap6Vw/iCvEusEFDXtudt5BAdmO7LJKYMC0zCft5BOle6DO5TvqlZdz1l7BXbMqjD3SGC4UjXp7C5Tx6LUvfEaOsGaXNBXIF2YQ6vduDkx8w70ch/4CgyWXO7IzA2wDsyNgTzJCAx/9P2Y6e2Hc7GhxiKRF6CWvTMxU/EWKdR0ghl46EyYIDmTLfvSxLOD57Ht75/A0RcSALLwRHDloUwNG1qGYiK6IvbdZEsMXJVtyEwhqWR0Am+3aG0FuU9b7bybmJnMbQdN9yU4jcDZsXdTpgyjK8fQKdidW1x5nXe5j1x1/SVqk3Q7lf1lfLLZnTvAZPy5BFpOGZIDKA6oLwDajGkBTAXIYsQIYLD0eX9gAF82APrbAH/r1OL713MFmOz7MzWJ+/3ez18s9Oz3bLQy3DrlygZwXNGeP2RSAnliIothdwDqzf26siHGjQLPrc7YjxwD93RxxT3Q81opbwOAONLL3fOVXlBvQRaT/NhzD3maGb9A0GdimKap5r1y5XqapjCi+YAgqRmzG2PQ2WaappENhBIE0ruaCd4i22wpDMBgLdeZm7WhyTab6+whZBV7oWVQdjxlzfqOKwgq7gTtYYyWguE4TVMT7teek/YcQqC5dBoMfNQdLhBRJjtljQel5XGapssFY4TA8RUwfowFay7FNE3v3D7VxFFXnFQjk4oPKkOF4qCZgev9cdTdIsno8lhiUyvZ1xJs1tkDBEBRQ5uBzGdz0DL0ysGSDNtSh+Hm1zNcNInfVSx/vxKviaKmEnuNIIajpm409ZlU4kjVKsMr4qhJdrweJ13RkIzmqWYqNFMB0jKkl5D2JWpiBHDQNbRX5cxoH+B31i8PjKETER0ZbyEIk/bLAYmmpnLKoAxcgrTzSRNOADbaTMkDrdi9btadyYcwiStCcOJGZ+xG421HedtBNaXnK0ibCs2hADDoxgitLE8zM/AeGDJrfgP6Q6l0IZaAb+nnzQfl0sGEzJoN4G+/AMnnGm4J8JyWSMtTB/RnjfaQtglwTUNP2KsUdcZzq8pAGD4A5a7HFHp0MWPfgVatd7IAYuiB632uQiu9B1wMpfas0l1YI3hlTeIB2GcSn4Dy0JMO2lwc8LbGkew983F2cE0m8RGoeA2YXSdPtR298+bzha/AjjejHTvy6tLaMF5CQhdvsQlkJkR7sYC5FSeU8Ra/2FzSjTY2PGAJ5IS92jBubo3JV0aOX2xhqLKx8XJhQ6ozWgoVX8bQOPTBxkJtNnDx+VxEYMtNBVPYMdr9vdG7FodobYL+Ct1qyM8m9B3tuInBvgIDN2PYlEQK5/GzPQIjbz4HMnOOtuEepQ0fQM1xtBOeaF4Ee1+VlYEgvb5mDz+sWwhc5atS9c3KX1tOgHkdvpqqBJLfQ7D1Q4LQb1sq8vx/JEmQqiXxQk79dQsI9m3OYxqt9/sup0tXfu+qEjknXan1zOD3UKV780fyWt36qm8rUqwW6K6+Mvc6Q2s9+B2PJXBtSdHXV+j1N1WvXZCjf3NXVYmvg18DaLcgtVLByOU2vAb64TX1VQ989RUp0tBCnnRjlwTk4Qe1p7wM6msNwNsSAmhRkJeR5iYB8lMvzfh0ep5zTkWEZslPsWDJRNC3AGKDeelEtJLdL3pux/MJeUWX3PpLXuFyKzHngmIPoGKz8AV9snlRHyeQbjH3f2dIRJBs9CMPkLwbLmi4uX/TZ37q5blvSvRzkzlh5qdYsGTkvWl9iup7pjnp9Oj3S8uOHpJX9Pz637UqNHPYnGo+LHxBn1J+3pvgjB+n7MH7GN/PK0i78ZAgS3w/r9xrkQTudaz3Cyd/otLfCr08Hu6jZEt9b7wsRVWewua/wXtE+FDZ0v2eifBZETQRWeTmdC3R6uB3ttBe/Fns5nSypxULgZmKsKkgGJqsjUMpONVk0zhAVioE5txU6x174MTDMOxsU+K9Iexu5eJLKAGcbo4OoMMvPKxqCHoeIIJUzKCICM7T6sYM2t1JgDYp+bS6seekIqr9nm7GmDTe0uoHMqxaMwNWN1o6HhWuo5MSkNWN8nbSfh0eXUyGVU9BHaBjv3LZUX02AOpzSTcjWa6cbYSK25UL15wPPto1CP0Rf6v9O+9WMGOXurdx3BkI1gc9rGBfj1OLFcyYWskdSTv/RqqVi5eXvDs85t9IpQ10jpkY+y+kfBU9/CEcX7FUCLZl8jidUZ4nMMKxz6TsYvrhyPzBsRRDWEqdZSkV2XB4xbBLsQDCwkYNu/QbylJqhl22HoOTDYfeN/pZZYZ9FqSN1Ud5lbJ93IrS3FG77cRPOhz4pT2VRahSxECGif0X+6+7Ij9k0RcPpeisaKMvG/dEZgbJgIjbyg34OEL7qel+HKNsa0VpBC4yM1Bbu0TZdhHG4CI0g8jbXHBTkon9F/tPYXp1p6Z926RETwC1p0sTWEvS/gTCvUaGNUgp7fZzuBdKwFKaNhuBiAlUWrItb8E1yzxLhFjKGryU7w6E+L+6D3OQUjTzVCBSgvHN86AQZDtSrXuEVYq8ON6DlTKupc1CK+WcuM6DlSK7G6+IkECW5YB9iafAWt1sSk4c0j84b3rHEKyU7Dxj7kIqpVzZHkcbpJS5O1XDKSIkVznKiXUVHFPJ5Y7ajggltT7Omm2wUnSPmQuEkpId1ByslBvrDAvniIog41zedIZA21DZ/CzjMIKcM1XpVSHIVky8qc8VSikDr2qyacDvymvuUH2iI9WEWB8I/E1WqVaWWELTG4kdgIClTFpuofwX5lKlnActBbE9cs4RDYHkbAzBx24U3EnOwgeUTU+7BC9ltKmyaRjRCfQcvJRTJyiTqEjORcbjHQA+58yDXBXjloSQgvOcB/9EfCFXX46XMD77bQX3XxNBNK75oJ6iIhvLlOGsl/ItgeP2HoHx8zkA+S4L4KzKATgvvQNq0jmgar3BINYZDGbd4U58oZpDyM5FxicMDpcIkqMiqtIRcemc6OhhUHuJQKNTGCy2xZ0knMsEB+c5HzCe85O3iKzj4xKxwiCxjo7WLjBIbYM7qUXzjpJY8kBnzoCWeQVW5hTomBdgY26AnHkGduYaKJgvYGSuAMl8I1ThfMnJ+egIqVtES0QkQO8WEQMD8wQcXiJwMpd3mG2YKXjvebyTURF0fnIXkX183i4RUMz6jmW+UzIrRFUGfZNAocsDOMqyAMZSD4D8OPnzbXNAVXYD1GUzGEzcwSDmVAiOsjyIiMJDxMcRcVa2pzo8RNScEh3tnYYXIZAfMj9UxHCOIlD+3yVipfPjFoGEmzstz1F149/PVB41GCj3p43Q87yCZ0U8riH0+CDFfE1KNMWfCr+QE4CnRfiUE4AAdRiYqIov+cfP8+6cwBtfdCjBNs+K+GIuGg/rLw4zHV5hUAs07C3iiY9FGaCeG06EGB6/Ojzx6iFi9hxeOP5tZKwoqyeOa308j9eU7uNaBt7HtRrdiYAjYnx4XOsqvQ+09Y4IomP2PIy06FYIZPXF41qzzjxFpHdaTY5r6STqx+CU+CgnwHBQOoSJBSQC0TYWbPkixAItX4RYwOVH4I0/JQu2CVaKGAQoJbKCM64lIOvYt9cfrFcPg/bKhRhW8zSCqPFNSnZloGqE6DRTAUfN4ZuU/lrhqOkgVitzSmKdT+BkvtH7EYKNOaEx37Bsye66Uoib7MwxjflGyXzCUSNXg+UdyNkOvj0SK54BdeNECA7NG1BYzn1j4hqOmlgIpOYOGDVvvjFzRQ7ATWJlICVcOb9QB0hOkOFzDE6KEJ3TnfOv5EOB5OTKwJXzsWR3tS6uEMeClRL+mwQhxbNQhTHs+lUhNscnIONZwa+KuymHwToNEACoJpagOd90ThsMsimHEJ2lPgCV1L51kU8dDPppg1h1JJYGGbv+i4p9SExH2djOnAM5s3+xN8ck1gjBYTkjp6r2vnHxBYO3OLEUjprON2J+w2DiS6wMRrLL7Ripa8DjZDbvElwlOSjys89BkGF5gKj5ohR4S1GgaoToyILmHs0NXFa8S3CV7JISmWNQXTO3BwyG9t7STcIgb8l3pLo2g+AxnI6UAQaqdVpXAJBtO1JNrqUkaAxy3+FM275L7O52oLjnhkMCxqvEyBxDUbLTUiBj0uwKLKwdLq69lwnE4IGUQRMVCXAybbXCwLzfWZkVBIx5qcmYvBmWzBvY3WLUTFqZkxmLvEAlzQmcTULe99y47PqZuXdo17Rizh+BLd5h0MUFhOgsswRRQz8XlV0+M28O8brOzM0jkMc5qBohOmk8wqCdJR5xVdWHuaqq5Y4mRVintj3eiJpexMNIEQeAiycHCaiSa2/cMSlKrCevfhlzeif1kOLQAWjYqkdo1iT2ESiWMycPubeNzBrIiUpw9K0gMRVxYzlzTx5l+52arsy+WQi0koebG9dkbc9CUJC1ujNvcO0bNDj6nEAXzviIiiuyWCYhGK075gYUYyjG46Q/ywuZww/RV4mVwUHeJBYHXZ+x66OEa3OWHD/kzBUM5A4jROcY4KGm5AZECdyb+YmNyhWIGjE6405yAwyhTdO0fQogpxuVn0RNRI6h9Dje6VDHtWYenrmJkDHXiwqaaZpihyqOb8yLuDd5ICVDZtpBT1dJhw0idOyJDWmf/FXEXE8weDNz6Y7pFi8lVnr00UdL+blmb4/KsWVRuOda5XUuwJBtDv2+H4DxIqYf4goYbM0BITppcgKqXU7yGewC1NlQln0vFIwnTp174xSRN4MYnbWRcNTIjj256vPhoMt2aHo4anIhD0vpbq+LL9cwjFMaLUGTHWXK8ps8Vi9RYotHrKWxhNK5XiWle1diCeOOWUFiA/P8RKydTwat+MOWxLRc5WTv8cac00NH7y1PmDNXzDhIy1efz8yDQ8kTcGqOhWBgXskyyaiUWzZsHzeuIi/W2ZBaThzeosUWoGBOO8qj4olL8rL+FiuDgewF2wdSxO4+pYHiLtKSalpSmuwVDMZeidEpchA13lJs9oDxRqq5lMPZk5XUn2J0jtythorqJYgaEQ8jKzSN62SAwdA0Cu4qjzaul8FVGkBzwsWcFbrG09ydgGqawUOK6ua62ZwKhY4N7tyZYefOQWS9Gw6x9piVKB1zYR6f9/2gGjFx5Tz/9/EFKSLHXO6jle8NmAgNVJoqODn4t8CSAwZ9IkhsbU8YpK2CXxUP5LV8TwoxOtkiYdC1J8je42QHVSNWmftv1sj8kS1cLHf/VR+pP4gbW8i8SayMsrHeyZKjWP7gGatFiaVkmXS+cSNfmS35EoKDeQFRQ+bi8s6bP2JlkOegOd8qHrMTBscmSGzvSc5HKTKTd8ZNkNjguMfQ+yflzMhX6jcp4GG+lCxczGcpot4kMCmSBWPwwnDlXhtQB9CtTmzt4FvFQ0u+gNdKMTpZqkDV+LbA2gKOmkOMTp+eMNhS/6Qc7QCiRrTDOhLb4FPBynIFSMuTKLGWxFbfKLkElOa3EDcpmBtg/0UhtpFY52dM8wWcVpDYTmK9v7EPoRIllpBY6xsV6zsfLsXKoN/Ij/NxL9jRSRgU3SlGJ88UqBq/GLsRRI0Ynb1TIGp8Q3YHiBp1N4zmzC+kNzHwJWeeywnwg74m4us54c6pNcbfnFxBKTybO71yeJwzOBF+AhIhXTUIAZT6mohHOh5IESvkVo/AYPUBFNoOwKHtDoylzQFZ8gacN84A9eYVUBX5zt3ECwQAhdYFETF4iug9RXQeIi5uiY4EBjE3YnwHUVMRtgfkxzU/HZ0fLxGu3yLWwGDmGAYJ1xCrJfhrx7lex7XjvD2vHUcLQca8/NJw7TjHezqBsbokIK9qBM6rOoBzeheAqt8DoOLbDoP5k8Og+fRw1kcmBLKqvHRM78NDRP2UiA0Gy4f8uoxyFYJzqhwRrvn5DDDwmB9PERkM2rK7s37SO92nhWh/l5hfOfMoJ8CwHyK+khNgOAARUTkWVE6gmN85gWJ+i3iUkyvIE1456eROz5wan8+Ff+Mtwn8dhwAb/0V456JsjPUB7NYWzv3YHTgs565rz4DUnAFnSfaL3Tgle8cWGFTcCDE8WOeOCmtz4NDcA5LqICJOKuLDLRGRkJ12M9lpF4sw7BIxeosoeaUiyPwkdH7cIlBzTe7nkvSw7InY4I5NL+YaEIsnYuoFxGp3TLpjH8+YjbAxdhinWAFyqk9A1pMEznoaARVfB6Dmq4BBUw0wSKodBkuVwyCteoc65NcY7yVCEhEjEXEQEQURMXiIaKue6NjudFXm0IT9GuNV7DE/Bs/MT1vlHiJWRwSyqruzVevtpew5je/FaOdi49G/O/pC7jajXMK5THYevibCp9w1hfMJRnJ2cP9YRAC5e4fx+edSdPIqbYLAqx84G3lDCK3YVuk4eCkjpyevYZRSNrhNwUvp+72C0xUhMpgqJDYHAi0d5/umUpZhJP4g5S1oKeoqz46PMJJo1fEauJRay573yEjOm6psKxFgU9lNF9I2MEto4BUTJ2OgUvq33c+yDqWUwy6YuTkClZJX3KtPBRMRMahtoRpmfQuslczVoSotEUopjd3VEqiUD/N7UJMdwyml5Q2pDVjKZ0dsiwjJ+dYbxm5pAmtJOuB42wEmlKjLdpBZoFJ2yItzhPXpj9OApbQ5ZM0ZTGTE4Jz4ykaF4JrMZ1s6/4T0T8xVdwQp5dwbq/fwSmn4HbCUYdG6h4mQGGB7c8CtbE+EWEpeccBNLzLMUvYpcCmJhImUGAOMfbcG17YCQLilyEClZIMKvZQ8eClRE2MQcHtxUvDypHxLUV/5v/90J23WTJ1k+fb5Iwt9ZNHqkm0eaSQzZ14tzyRtBB4t3Z686gVools7dN0/JcnCKAX00K1NkuTmHz09ZeifpEM2yUjbq/PG5lxbMmPZdr2ThTMe+PM6KUkS)

**LiteRT architecture**

## LiteRT on-device inference

The LiteRT on-device inference loads the model into an interpreter,
which parses the model and uses a delegate to run it.

The process includes the following:

1. The inference loads the LiteRT model into a LiteRT interpreter
interface, which parses the model to identify the neural network
operators present in it.
2. The interpreter interface is further configured to run the model
using a delegate.
3. The interpreter invokes a model inference on the provided inputs and
saves the corresponding outputs into the buffers provided to the
interpreter interface.

Qualcomm supports running LiteRT models on the following accelerators
using delegates:

- CPU
- Adreno GPU
- NPU

The following table lists the delegates and their accelerators:

Supported delegates and accelerators

| **Delegate** | **Acceleration** |
| --- | --- |
| XNNPACK delegate | CPU |
| GPU delegate | GPU |
| Qualcomm^®^ AI Engine direct delegate (QNN delegate) | CPU, GPU, and NPU |

## Next steps

- [Run a LiteRT model on CPU](https://docs.qualcomm.com/doc/80-80022-15B/topic/run-litert-model-on-cpu.html)
- [Run LiteRT Model on GPU](https://docs.qualcomm.com/doc/80-80022-15B/topic/run-litert-model-on-gpu.html)
- [Run LiteRT Model on NPU](https://docs.qualcomm.com/doc/80-80022-15B/topic/run-litert-model-on-npu.html)

Last Published: May 14, 2026

[Previous Topic
Use available frameworks and runtimes](https://docs.qualcomm.com/bundle/publicresource/80-80022-15B/topics/use-available-frameworks-and-runtimes.md) [Next Topic
Supported LiteRT runtimes](https://docs.qualcomm.com/bundle/publicresource/80-80022-15B/topics/run-a-litert-model.md)