Generative AI
Generative AI encompasses models that can create new content, including text, images, code, and more. At the core of modern generative AI are Large Language Models (LLMs) built on the Transformer architecture.1
Topics in This Section
| Topic | Description |
|---|---|
| Attention Mechanisms | Self-attention, cross-attention, multi-head attention |
| Transformers | The architecture behind GPT, BERT, and modern LLMs |
| Prompt Engineering | Techniques for effective LLM interactions |
| RAG Systems | Retrieval-Augmented Generation for grounded responses |
| Chunk Engineering | Optimal text segmentation for retrieval |
| Fine-Tuning | Adapting models to specific tasks (LoRA, QLoRA) |
| LLM Evaluation | Measuring quality, hallucination, and safety |
Learning Path
Attention Mechanisms → Transformers → Prompt Engineering → RAG Systems → Fine-Tuning
Related Domains
- Need NLP foundations? See 04 - NLP
- Interested in RLHF? See 06 - Reinforcement Learning
- Deploying LLMs? See 07 - MLOps & DevOps
References
Vaswani, A., et al. (2017). Attention Is All You Need. Advances in Neural Information Processing Systems. https://arxiv.org/abs/1706.03762 ↩︎