Follow-up Curriculum for Consumer-Hardware LLM Work
Plan
This follow-up sequence assumes a single consumer workstation: roughly 8-24 GB of VRAM, enough system RAM to memory-map larger quantized files, and a local stack built around llama.cpp, GGUF files, Hugging Face tools, and small PyTorch fine-tuning jobs.
Chapter 13: The Local Model Stack
Inventory the machine, the runtime, and the local model zoo before writing application code. The chapter teaches what model families, quantization formats, context windows, and memory budgets mean in practice.
Deliverable: a hardware/model inventory table and a practical fit estimate for the GGUF files already on disk.
Chapter 14: Quantization and Local Inference
Compare full-precision and quantized GGUF files, inspect metadata, and run small inference/latency tests. The goal is to connect quantization labels such as BF16, Q8_0, and Q4_K_M to file size, memory pressure, and throughput.
Deliverable: a local quantization comparison and a reproducible inference timing harness.
Chapter 15: Serving Models Locally
Start llama-server, query it through OpenAI-compatible HTTP endpoints, measure latency, and shut it down cleanly. This makes local models usable from normal application code.
Deliverable: a working local API server pattern using the GGUF stack in ~/tmp/llama-cpp.
Chapter 16: Modern Prompt Interfaces
Replace ad hoc prompting with typed interfaces: JSON schema, constrained decoding, retries, and validation.
Deliverable: a small extraction API with schema checks and regression tests.
Chapter 17: Retrieval That Holds Up
Build retrieval-augmented generation from separate pieces: chunking, embeddings, hybrid retrieval, reranking, prompt assembly, citations, and evals.
Deliverable: a local document QA system with retrieval metrics separated from generation metrics.
Chapter 18: LoRA and QLoRA Fine-Tuning
Train small adapters instead of full models. Cover target modules, rank, alpha, dropout, 4-bit loading, adapter merging, and export formats.
Deliverable: a QLoRA adapter for a 1B-8B model that fits consumer hardware.
Chapter 19: LLMs as Predictive Models
Use language-model infrastructure for structured prediction rather than chat. Compare classification heads, sequence classification, pairwise ranking, and listwise discrete choice against classical baselines.
Deliverable: a benchmark on the retail delivery choice data with conditional logit and neural-choice baselines.
Chapter 20: Preference Optimization on a Budget
Introduce DPO-style preference tuning and show when it is a better fit than supervised fine-tuning.
Deliverable: a small preference dataset, a tuned adapter, and a pairwise evaluation.
Chapter 21: Evaluation Harnesses
Make evaluation reproducible: golden sets, task metrics, judge prompts, pairwise comparisons, latency, memory, and cost.
Deliverable: a single evaluation runner that compares prompted, RAG, and fine-tuned systems.
Chapter 22: Tools and Agents
Treat agents as bounded workflow systems: typed tools, state, retries, logging, permissions, and human approval.
Deliverable: a local model driving a narrow, auditable tool workflow.
Chapter 23: Multimodal Local Models
Run small vision-language and embedding models locally. Cover OCR-like extraction, image search, captioning, and document understanding.
Deliverable: a local image/text retrieval and captioning notebook.
Chapter 24: Deployment for One Person
Version models, adapters, prompts, datasets, evals, and server settings. Track licenses and quant provenance.
Deliverable: a lightweight model-ops checklist and reproducible local deployment folder.