AI LiteracyfoundationalVideo + quiz
LLM Guardrails, Evals and Agentic Memory
Learn to take LLM applications from prototype to production.
What you will be able to do
Explain the role of AI guardrails in securing LLM-based applications and identify common attack vectors such as prompt injection and jailbreakingImplement NemoGuardrails using Colang to enforce topic, jailbreak, sensitive-topic, dialogue, and custom rails on an LLM applicationApply observability tooling (Pydantic LogFire) to trace and monitor guardrail events and LLM interactions in real timeConstruct a golden dataset and run a RAG evaluation pipeline using RAGAS metrics including faithfulness, answer relevancy, context precision, context recall, and answer correctnessDescribe the agent memory lineage from conversational buffer memory through sliding window, summary, and summary buffer memory, articulating the trade-offs of each approachDifferentiate episodic, semantic, and procedural memory types and explain when each is appropriate in a production agentic AI systemDesign a production-grade agentic memory architecture that balances token cost, recall accuracy, and long-term persistence
Skills you will gain
- AI Guardrails
- NemoGuardrails
- Colang
- Prompt Injection Defense
- Jailbreak Detection
- LLM Observability
- Pydantic LogFire
- LLM Evaluation (Evals)
- RAGAS Framework
- Golden Dataset Construction
- RAG Pipeline
- Agentic Memory
- Conversational Buffer Memory
- Sliding Window Memory
- Summary Memory
- AgentOps
About this course
Learn to take LLM applications from prototype to production. You will add guardrails that block prompt injection, jailbreaking and PII leakage, build automated evaluation pipelines using RAGAS metrics instead of eyeballing outputs, and design agent memory architectures that balance token cost against recall. The course ends with a production readiness checklist you can apply to your own system.
AI Literacy
Curriculum
Why LLM Security Is Non-Negotiable8 min
Guardrail Frameworks Landscape: NemoGuardrails, Guardrails AI, LLaMA Firewall, and AWS Bedrock8 min
Building Your First Secured LLM App with NemoGuardrails14 min
Output Rails, Custom Rails, and PII Detection8 min
Observability with Pydantic LogFire10 min
Module 1 Knowledge Check
Requirements
Working knowledge of Python (functions, lists, dictionaries)
Basic understanding of Large Language Models (LLMs) and how to make API calls to them
Familiarity with the concept of Retrieval-Augmented Generation (RAG)
A free Groq API key (obtained at console.groq.com) and a free Pydantic LogFire account
Common questions
Who is this course for?
This course is designed for AI engineers, ML engineers, backend engineers building LLM applications, and data scientists transitioning to AI engineering. It is also useful for AI product managers and founders who need to make informed architecture decisions about security, evaluation, memory, and scaling for their AI products. The content is intermediate-to-advanced; learners who are completely new to LLMs should complete a foundational LLM course first.
What do I need to know before starting this course?
You need working knowledge of Python (functions, lists, dictionaries), a basic understanding of how LLMs work and how to make API calls to them, and familiarity with the concept of Retrieval-Augmented Generation (RAG). Before the labs, you will also need a free Groq API key (from console.groq.com) and a free Pydantic LogFire account. No prior experience with guardrails, evaluation frameworks, or agentic memory is required — the course builds those from scratch.
How long does this course take to complete?
The course contains approximately 9–10 hours of content across five modules: roughly 2 hours for Module 1 (guardrails), 1.5 hours for Module 2 (evals), 1 hour for Module 3 (short-term memory), 1.5 hours for Module 4 (long-term memory), and 1.5 hours for Module 5 (AgentOps). Each module includes lectures, hands-on labs or assignments, and a knowledge-check quiz. Completing the optional labs and the capstone assignment may add 2–4 additional hours depending on your pace.
Does this course include hands-on projects or just theory?
The course is heavily practical. It includes three hands-on labs: building a secured LLM app with NemoGuardrails and Pydantic LogFire observability (Module 1), running an end-to-end RAG evaluation pipeline with RAGAS (Module 2), and implementing cross-session episodic memory with LangMem (Module 4). Module 5 concludes with a capstone assignment where you assess a realistic production system against a readiness checklist and write an Architecture Decision Record. Each module also includes a knowledge-check quiz. The course does NOT cover model training, fine-tuning, or frontend development.
What you walk away with
A Yallo AI academy certificate of completion, issued when every lesson is complete and the final assessment is passed at 70% or aboveA public verification page, so a stranger can check the certificate without asking you for a filePDF download, and a one-click share to your LinkedIn profile
Yallo AI academy certifies completion of
LLM Guardrails, Evals and Agentic Memory
Reviews
Loading reviews…