Yallo AI academy
Sign up for free
Menu
CoursesBusinessInsightsPricing
Explore
Start with a learning pathClaude for DevelopersAI for Product ManagersAI for LeadersAI for Data AnalystsView allFree coursesRedeem a code
Log in
Technical AIintermediateVideo + quiz

Run Small Language Models on Devices: Optimise, Deploy and Benchmark

For AI and ML engineers, software and embedded developers and edge architects.

33 lessons · 1.5 hours
Watch the first lesson freeWelcome · 3 min

Sign in to watch one lesson from this course free.

What you will be able to do

Decide when a language model belongs on the deviceSize a small model to a memory budget with memory mathsShrink a model with quantization and prove it still does the jobMatch llama.cpp, Ollama, ONNX Runtime GenAI, LiteRT and ExecuTorch to laptops, phones and boardsBenchmark speed, memory and quality on a device honestlyBuild an offline assistant and set the security, privacy, update and fleet rules it needs

Skills you will gain

  • On-device AI
  • Quantization
  • Memory budgeting
  • Edge runtimes
  • Model benchmarking
  • Edge deployment security

About this course

For AI and ML engineers, software and embedded developers and edge architects. Learn when a language model belongs on the device, size a small model to a memory budget, and shrink it with quantization while proving it still does the job. Match llama.cpp, Ollama, ONNX Runtime GenAI, LiteRT and ExecuTorch to laptops, phones and boards; build an offline assistant that validates every reply; benchmark speed, memory and quality honestly; and set the security, privacy, update and fleet rules a real deployment needs. Finish with a free lab where you build and benchmark an offline field assistant on your own laptop.

Technical AI

Curriculum

10 modules · 33 lessons · 1.5h
WelcomeFree preview3 min

Requirements

Basic Python, enough to run a short script
Called a model through an API at least once
Comfort running commands in a terminal

What you walk away with

A Yallo AI academy certificate of completion, issued when every lesson is complete and the final assessment is passed at 70% or aboveA public verification page, so a stranger can check the certificate without asking you for a filePDF download, and a one-click share to your LinkedIn profile

Yallo AI academy certifies completion of

Run Small Language Models on Devices: Optimise, Deploy and Benchmark