← Library
Course · by Yash Tiwari

AI Engineering: Build, evaluate and design AI systems

Revision notes for software engineers who have heard the AI terms and want to build with them: call an LLM, open the box (embeddings, neural networks, transformers, training), add knowledge with RAG, let it act with agents, ship it to production, and design AI systems with a delivery framework and worked breakdowns. Each lesson is short and links to the best external resources to go deeper.

  1. 01

    Orientation

    The terms you have heard, decoded, and the map of what follows.

     2 / 6
    1. 1.1What AI actually is, and the terms you have heardLocked
    2. 1.2Kinds of learning, and discriminative vs generative modelsLocked
    3. 1.3Deterministic vs probabilistic softwareSoon
    4. 1.4Training vs inference; models, parameters and datasetsSoon
    5. 1.5The AI engineering landscape and rolesSoon
    6. 1.6The delivery framework and the spine projectSoon
  2. 02

    Your first LLM app

    One API call, grown into a small app with a first eval.

     0 / 10
    1. 2.1Anatomy of an LLM API call: system, developer and user messagesSoon
    2. 2.2Tokens, context windows and costSoon
    3. 2.3Prompting fundamentals: zero-shot and few-shotSoon
    4. 2.4Structured outputsSoon
    5. 2.5Tool callingSoon
    6. 2.6Streaming responsesSoon
    7. 2.7Hallucinations and why they happenSoon
    8. 2.8Your first evalSoon
    9. 2.9Errors, retries and fallbacksSoon
    10. 2.10Hosted vs open-weight models; model selectionSoon
  3. 03

    Representing things

    Text becomes numbers, and numbers can be compared.

     0 / 5
    1. 3.1Tokenisation and vocabulariesSoon
    2. 3.2Vectors and embeddingsSoon
    3. 3.3Matrices as transformationsSoon
    4. 3.4Similarity and distance: cosine and EuclideanSoon
    5. 3.5Before transformers: bag of words, TF-IDF and Word2VecSoon
  4. 04

    How machines learn

    Loss, gradients and the habits of mind from classical ML that everything later reuses.

     0 / 7
    1. 4.1What learning means: loss functionsSoon
    2. 4.2Gradient descentSoon
    3. 4.3Features, labels and train, validation and test setsSoon
    4. 4.4Overfitting and bias-varianceSoon
    5. 4.5Accuracy, precision, recall, F1 and the confusion matrixSoon
    6. 4.6Linear and logistic regression from scratchSoon
    7. 4.7Probability, entropy and cross-entropySoon
  5. 05

    Neural networks

    Layers of simple functions, trained by backpropagation.

     0 / 6
    1. 5.1Neurons, layers and activationsSoon
    2. 5.2The forward passSoon
    3. 5.3BackpropagationSoon
    4. 5.4Train a network from scratch with NumPySoon
    5. 5.5PyTorch: tensors, autograd and the training loopSoon
    6. 5.6CNNs and RNNs: what they solved and where they hit a wallSoon
  6. 06

    Transformers

    Attention, the idea behind every modern language model.

     0 / 6
    1. 6.1Attention intuition: query, key and valueSoon
    2. 6.2Self-attention and multi-head attentionSoon
    3. 6.3Positional encoding, residuals, layer norm and feed-forward blocksSoon
    4. 6.4The full transformer: encoder, decoder and encoder-decoderSoon
    5. 6.5What building a mini GPT teaches youSoon
    6. 6.6Next-token prediction and sampling: temperature, top-k and top-pSoon
  7. 07

    Training and serving LLMs

    How a base model becomes an assistant, and what it costs to run.

     0 / 6
    1. 7.1Pre-trainingSoon
    2. 7.2Fine-tuning and instruction tuningSoon
    3. 7.3Alignment: RLHF and preference optimisationSoon
    4. 7.4Parameter-efficient fine-tuning: LoRA and QLoRASoon
    5. 7.5Inference: KV cache, batching and quantisationSoon
    6. 7.6Multimodal and speech modelsSoon
  8. 08

    Embeddings, search and RAG

    Let the model answer from your own documents, and measure whether it did.

     0 / 14
    1. 8.1Why LLMs need external knowledgeSoon
    2. 8.2Embedding models in practice: dimensions, dense and sparseSoon
    3. 8.3Keyword search and BM25Soon
    4. 8.4Vector search and indexes: HNSW and IVFSoon
    5. 8.5Metadata filtering and hybrid searchSoon
    6. 8.6Reranking and cross-encodersSoon
    7. 8.7Document ingestion and parsingSoon
    8. 8.8Chunking strategiesSoon
    9. 8.9RAG architecture and the indexing pipelineSoon
    10. 8.10Query rewriting and context constructionSoon
    11. 8.11Advanced RAG: multi-hop and agentic retrievalSoon
    12. 8.12Evaluating RAGSoon
    13. 8.13Debugging failed retrievalSoon
    14. 8.14Data quality and deduplication for AI pipelinesSoon
  9. 09

    AI agents

    From answering questions to taking actions, safely.

     0 / 14
    1. 9.1LLM application vs agentSoon
    2. 9.2The agent loop and ReActSoon
    3. 9.3Tool design and executionSoon
    4. 9.4Model Context Protocol (MCP)Soon
    5. 9.5Planning and task decompositionSoon
    6. 9.6Memory: short-term and long-termSoon
    7. 9.7Context managementSoon
    8. 9.8Workflows and state machinesSoon
    9. 9.9Multi-agent systemsSoon
    10. 9.10Human-in-the-loop and approvalsSoon
    11. 9.11Failure recovery and resumabilitySoon
    12. 9.12Prompt injection and untrusted inputSoon
    13. 9.13Tool permissions and sandboxingSoon
    14. 9.14Evaluating agentsSoon
  10. 10

    Full-stack AI applications

    The product around the model: APIs, streaming, state and the interface.

     0 / 10
    1. 10.1Anatomy of an AI app and its service boundariesSoon
    2. 10.2Building AI APIs with FastAPISoon
    3. 10.3Auth and multi-tenant data isolationSoon
    4. 10.4Streaming end to end with SSESoon
    5. 10.5Conversations and message persistenceSoon
    6. 10.6Background jobs and async workflowsSoon
    7. 10.7Storing embeddings and agent stateSoon
    8. 10.8Frontend: chat UI, streaming and tool statusSoon
    9. 10.9Multimodal inputs: files, images and audioSoon
    10. 10.10Integrating AI with existing servicesSoon
  11. 11

    Production AI engineering

    Evaluate, observe, secure, pay for and release it.

     0 / 12
    1. 11.1Why AI testing differs; building eval datasetsSoon
    2. 11.2LLM-as-judge and regression evals in CISoon
    3. 11.3Tracing multi-step workflowsSoon
    4. 11.4Logging prompts and tool calls safely; privacySoon
    5. 11.5Monitoring tokens, latency and time to first tokenSoon
    6. 11.6Cost control: routing, caching and batchingSoon
    7. 11.7Reliability: timeouts, rate limits and fallback modelsSoon
    8. 11.8Security: data exfiltration and guardrailsSoon
    9. 11.9Deploying on AWSSoon
    10. 11.10CI/CD for prompts, models and releasesSoon
    11. 11.11Product metrics and feedback loopsSoon
    12. 11.12Operating self-hosted models: GPU basicsSoon
  12. 12

    AI system design framework

    The routine to follow for any AI design problem, aligned with the System Design delivery framework.

     0 / 3
    1. 12.1How to approach an AI system design problemSoon
    2. 12.2Estimating latency, throughput and costSoon
    3. 12.3Choosing the approach: prompt vs RAG vs fine-tune vs agentSoon
  13. 13

    Breakdowns: easy

    Single-purpose AI applications.

     0 / 5
    1. 13.1Design a text summarisation serviceSoon
    2. 13.2Design a PDF question-answering applicationSoon
    3. 13.3Design a semantic search engineSoon
    4. 13.4Design an AI chatbotSoon
    5. 13.5Design an AI email assistantSoon
  14. 14

    Breakdowns: medium

    Several AI components working together.

     0 / 5
    1. 14.1Design an enterprise knowledge assistantSoon
    2. 14.2Design an AI customer support systemSoon
    3. 14.3Design a natural-language-to-SQL platformSoon
    4. 14.4Design an AI coding assistantSoon
    5. 14.5Design a document extraction pipelineSoon
  15. 15

    Breakdowns: hard

    Scale, quality, reliability and orchestration.

     0 / 5
    1. 15.1Design a ChatGPT-like platformSoon
    2. 15.2Design an AI agent platformSoon
    3. 15.3Design a coding agentSoon
    4. 15.4Design an enterprise-scale RAG platformSoon
    5. 15.5Design a real-time voice assistantSoon