Skip to content
AI-grafen
Sign in
☰ Menu

The goal F AI engineering

Fine-tune and run your own models

PEFT and QLoRA, dataset design, distributed training, catastrophic forgetting, export, and a capstone where you fine-tune and evaluate for real.

Knowledge nodes
89
From zero
about 78 h
Labs
6
See what you already know — no account

The diagnostic removes what you already know, so your path is usually much shorter.

What you can do afterwards

Labs along the way

You write the code. Tests you cannot see decide whether it holds up.

Project 4: Fine-tune a model with LoRAFa sandbox · about 180 minLab: LoRA on a small network — train only the adaptersFa sandbox · about 75 minLab: build an eval harnessEa sandbox · about 60 minLab: a neural network in pure NumPy — forward, backprop, gradient checkDa sandbox · about 75 minLab: dot product, norm and cosine similarityDin the browser · about 40 minLab: gradient descent from scratchDin the browser · about 50 min

The whole path

Everything the goal builds on, grouped by level and in the order it builds on itself. Show on the map

AExplorer4 knowledge nodes
  1. Patterns and categories
  2. Sequences and precise instructions
  3. Integer arithmetic and order of operations
  4. Comparing and sorting values
BInvestigator11 knowledge nodes
  1. Data in everyday life
  2. Algorithmic thinking
  3. Rule-based systems and machine learning
  4. Source criticism and responsibility in AI use
  5. Fractions, decimals, and percentages
  6. Coordinates: positions on a grid
  7. Negative numbers in everyday life and AI
  8. Training data, features and labels
  9. Classification: how a model sorts information
  10. Generative AI: how it creates content
  11. Language models and probabilities
CBuilder19 knowledge nodes
  1. Functions and coordinate systems
  2. Programming logic — variables, conditions, loops
  3. Python — the basics
  4. Python — lists, loops and dictionaries
  5. Python — strings and text processing
  6. Python — files, CSV and JSON
  7. Statistics — mean, median and spread
  8. Probability — the basics
  9. Manual data labelling
  10. Linear regression: fitting a straight line to data
  11. Neural networks — the intuition
  12. Prompting — steering a language model
  13. Experiment: biased training data and its consequences
  14. Project: train an image classifier in the browser
  15. Retraining the model with your own examples
  16. Attention: how words influence each other
  17. Words as points: similar words close together
  18. Variables and algebraic expressions
  19. Powers and roots
DAI developer26 knowledge nodes
  1. Derivatives and optimisation
  2. Python — functions, scope and exceptions
  3. APIs and HTTP
  4. Regular expressions
  5. Licences and open data
  6. Loss functions
  7. Testing with pytest
  8. Tokenisation
  9. Gradient descent
  10. The transformer — an overview without formulas
  11. How a language model is trained — at upper-secondary level
  12. Fine-tuning — an overview for upper secondary
  13. Logarithms
  14. Vectors
  15. Matrices and matrix multiplication
  16. Linear regression with several features
  17. Neural networks — the forward pass with matrices
  18. Backpropagation
  19. Embeddings — words as vectors
  20. Attention
  21. Overfitting and generalisation
  22. PyTorch — tensors and autograd
  23. Retrieval — finding the right text
  24. Training, validation and test
  25. Train a neural network in PyTorch
  26. Transformers — the architecture
EUniversity10 knowledge nodes
  1. Information theory: entropy and KL divergence
  2. Matrix factorisation and low-rank approximation
  3. Optimisers: momentum, Adam, scheduling
  4. Language models — training and generation
  5. Model evaluation
  6. Build an eval harness
  7. Fine-tuning language models
  8. Instruction fine-tuning (SFT)
  9. RAG — retrieval-augmented generation
  10. Web scraping — the technique and the rules
FAI engineering18 knowledge nodes
  1. Quantisation
  2. Distributed training
  3. Dataset design for fine-tuning
  4. Knowledge distillation
  5. Catastrophic forgetting
  6. LoRA — Low-Rank Adaptation
  7. Merging adapters and exporting models
  8. Parameter-efficient fine-tuning: adapters, prefix, LoRA
  9. QLoRA
  10. Evals for language models and agents
  11. Regression tests for models
  12. Pre-training in practice
  13. Continued pre-training on domain data
  14. Scaling laws
  15. Synthetic data
  16. Target modules and rank in LoRA
  17. Project F: fine-tune and evaluate a language model
  18. Training data for language models: filtering and dedup
GFrontier Lab1 knowledge nodes
  1. FSDP, ZeRO and DeepSpeed