first commit

.gitignore

@@ -172,3 +172,21 @@ cython_debug/
 
 # PyPI configuration file
 .pypirc
+
+# block media files
+*.mp4
+*.mov
+*.avi
+*.mkv
+*.flv
+*.wmv
+*.wav
+
+# block images and pdf
+*.png
+*.jpg
+*.jpeg
+*.gif
+*.bmp
+*.tiff
+*.pdf

chapter1/material/1.md

@@ -0,0 +1,121 @@
+# Introduction[[introduction]]
+
+<CourseFloatingBanner
+    chapter={1}
+    classNames="absolute z-10 right-0 top-0"
+/>
+
+## Welcome to the 🤗 Course![[welcome-to-the-course]]
+
+<Youtube id="00GKzGyWFEs" />
+
+This course will teach you about large language models (LLMs) and natural language processing (NLP) using libraries from the [Hugging Face](https://huggingface.co/) ecosystem — [🤗 Transformers](https://github.com/huggingface/transformers), [🤗 Datasets](https://github.com/huggingface/datasets), [🤗 Tokenizers](https://github.com/huggingface/tokenizers), and [🤗 Accelerate](https://github.com/huggingface/accelerate) — as well as the [Hugging Face Hub](https://huggingface.co/models). It's completely free and without ads.
+
+## Understanding NLP and LLMs[[understanding-nlp-and-llms]]
+
+While this course was originally focused on NLP (Natural Language Processing), it has evolved to emphasize Large Language Models (LLMs), which represent the latest advancement in the field. 
+
+**What's the difference?**
+- **NLP (Natural Language Processing)** is the broader field focused on enabling computers to understand, interpret, and generate human language. NLP encompasses many techniques and tasks such as sentiment analysis, named entity recognition, and machine translation.
+- **LLMs (Large Language Models)** are a powerful subset of NLP models characterized by their massive size, extensive training data, and ability to perform a wide range of language tasks with minimal task-specific training. Models like the Llama, GPT, or Claude series are examples of LLMs that have revolutionized what's possible in NLP.
+
+Throughout this course, you'll learn about both traditional NLP concepts and cutting-edge LLM techniques, as understanding the foundations of NLP is crucial for working effectively with LLMs.
+
+## What to expect?[[what-to-expect]]
+
+Here is a brief overview of the course:
+
+<div class="flex justify-center">
+<img class="block dark:hidden" src="https://huggingface.co/datasets/huggingface-course/documentation-images/resolve/main/en/chapter1/summary.svg" alt="Brief overview of the chapters of the course.">
+<img class="hidden dark:block" src="https://huggingface.co/datasets/huggingface-course/documentation-images/resolve/main/en/chapter1/summary-dark.svg" alt="Brief overview of the chapters of the course.">
+</div>
+
+- Chapters 1 to 4 provide an introduction to the main concepts of the 🤗 Transformers library. By the end of this part of the course, you will be familiar with how Transformer models work and will know how to use a model from the [Hugging Face Hub](https://huggingface.co/models), fine-tune it on a dataset, and share your results on the Hub!
+- Chapters 5 to 8 teach the basics of 🤗 Datasets and 🤗 Tokenizers before diving into classic NLP tasks and LLM techniques. By the end of this part, you will be able to tackle the most common language processing challenges by yourself.
+- Chapter 9 goes beyond NLP to cover how to build and share demos of your models on the 🤗 Hub. By the end of this part, you will be ready to showcase your 🤗 Transformers application to the world!
+- Chapters 10 to 12 dive into advanced LLM topics like fine-tuning, curating high-quality datasets, and building reasoning models.
+
+This course:
+
+* Requires a good knowledge of Python
+* Is better taken after an introductory deep learning course, such as [fast.ai's](https://www.fast.ai/) [Practical Deep Learning for Coders](https://course.fast.ai/) or one of the programs developed by [DeepLearning.AI](https://www.deeplearning.ai/)
+* Does not expect prior [PyTorch](https://pytorch.org/) or [TensorFlow](https://www.tensorflow.org/) knowledge, though some familiarity with either of those will help
+
+After you've completed this course, we recommend checking out DeepLearning.AI's [Natural Language Processing Specialization](https://www.coursera.org/specializations/natural-language-processing?utm_source=deeplearning-ai&utm_medium=institutions&utm_campaign=20211011-nlp-2-hugging_face-page-nlp-refresh), which covers a wide range of traditional NLP models like naive Bayes and LSTMs that are well worth knowing about!
+
+## Who are we?[[who-are-we]]
+
+About the authors:
+
+[**Abubakar Abid**](https://huggingface.co/abidlabs) completed his PhD at Stanford in applied machine learning. During his PhD, he founded [Gradio](https://github.com/gradio-app/gradio), an open-source Python library that has been used to build over 600,000 machine learning demos. Gradio was acquired by Hugging Face, which is where Abubakar now serves as a machine learning team lead.
+
+[**Ben Burtenshaw**](https://huggingface.co/burtenshaw) is a Machine Learning Engineer at Hugging Face. He completed his PhD in Natural Language Processing at the University of Antwerp, where he applied Transformer models to generate children stories for the purpose of improving literacy skills. Since then, he has focused on educational materials and tools for the wider community.
+
+[**Matthew Carrigan**](https://huggingface.co/Rocketknight1) is a Machine Learning Engineer at Hugging Face. He lives in Dublin, Ireland and previously worked as an ML engineer at Parse.ly and before that as a post-doctoral researcher at Trinity College Dublin. He does not believe we're going to get to AGI by scaling existing architectures, but has high hopes for robot immortality regardless.
+
+[**Lysandre Debut**](https://huggingface.co/lysandre) is a Machine Learning Engineer at Hugging Face and has been working on the 🤗 Transformers library since the very early development stages. His aim is to make NLP accessible for everyone by developing tools with a very simple API.
+
+[**Sylvain Gugger**](https://huggingface.co/sgugger) is a Research Engineer at Hugging Face and one of the core maintainers of the 🤗 Transformers library. Previously he was a Research Scientist at fast.ai, and he co-wrote _[Deep Learning for Coders with fastai and PyTorch](https://learning.oreilly.com/library/view/deep-learning-for/9781492045519/)_ with Jeremy Howard. The main focus of his research is on making deep learning more accessible, by designing and improving techniques that allow models to train fast on limited resources.
+
+[**Dawood Khan**](https://huggingface.co/dawoodkhan82) is a Machine Learning Engineer at Hugging Face. He's from NYC and graduated from New York University studying Computer Science. After working as an iOS Engineer for a few years, Dawood quit to start Gradio with his fellow co-founders. Gradio was eventually acquired by Hugging Face.
+
+[**Merve Noyan**](https://huggingface.co/merve) is a developer advocate at Hugging Face, working on developing tools and building content around them to democratize machine learning for everyone.
+
+[**Lucile Saulnier**](https://huggingface.co/SaulLu) is a machine learning engineer at Hugging Face, developing and supporting the use of open source tools. She is also actively involved in many research projects in the field of Natural Language Processing such as collaborative training and BigScience.
+
+[**Lewis Tunstall**](https://huggingface.co/lewtun) is a machine learning engineer at Hugging Face, focused on developing open-source tools and making them accessible to the wider community. He is also a co-author of the O'Reilly book [Natural Language Processing with Transformers](https://www.oreilly.com/library/view/natural-language-processing/9781098136789/).
+
+[**Leandro von Werra**](https://huggingface.co/lvwerra) is a machine learning engineer in the open-source team at Hugging Face and also a co-author of the O'Reilly book [Natural Language Processing with Transformers](https://www.oreilly.com/library/view/natural-language-processing/9781098136789/). He has several years of industry experience bringing NLP projects to production by working across the whole machine learning stack..
+
+## FAQ[[faq]]
+
+Here are some answers to frequently asked questions:
+
+- **Does taking this course lead to a certification?**
+Currently we do not have any certification for this course. However, we are working on a certification program for the Hugging Face ecosystem -- stay tuned!
+
+- **How much time should I spend on this course?**
+Each chapter in this course is designed to be completed in 1 week, with approximately 6-8 hours of work per week. However, you can take as much time as you need to complete the course.
+
+- **Where can I ask a question if I have one?**
+If you have a question about any section of the course, just click on the "*Ask a question*" banner at the top of the page to be automatically redirected to the right section of the [Hugging Face forums](https://discuss.huggingface.co/):
+
+<img src="https://huggingface.co/datasets/huggingface-course/documentation-images/resolve/main/en/chapter1/forum-button.png" alt="Link to the Hugging Face forums" width="75%">
+
+Note that a list of [project ideas](https://discuss.huggingface.co/c/course/course-event/25) is also available on the forums if you wish to practice more once you have completed the course.
+
+- **Where can I get the code for the course?**
+For each section, click on the banner at the top of the page to run the code in either Google Colab or Amazon SageMaker Studio Lab:
+
+<img src="https://huggingface.co/datasets/huggingface-course/documentation-images/resolve/main/en/chapter1/notebook-buttons.png" alt="Link to the Hugging Face course notebooks" width="75%">
+
+The Jupyter notebooks containing all the code from the course are hosted on the [`huggingface/notebooks`](https://github.com/huggingface/notebooks) repo. If you wish to generate them locally, check out the instructions in the [`course`](https://github.com/huggingface/course#-jupyter-notebooks) repo on GitHub.
+
+
+- **How can I contribute to the course?**
+There are many ways to contribute to the course! If you find a typo or a bug, please open an issue on the [`course`](https://github.com/huggingface/course) repo. If you would like to help translate the course into your native language, check out the instructions [here](https://github.com/huggingface/course#translating-the-course-into-your-language).
+
+- ** What were the choices made for each translation?**
+Each translation has a glossary and `TRANSLATING.txt` file that details the choices that were made for machine learning jargon etc. You can find an example for German [here](https://github.com/huggingface/course/blob/main/chapters/de/TRANSLATING.txt).
+
+
+- **Can I reuse this course?**
+Of course! The course is released under the permissive [Apache 2 license](https://www.apache.org/licenses/LICENSE-2.0.html). This means that you must give appropriate credit, provide a link to the license, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use. If you would like to cite the course, please use the following BibTeX:
+
+```
+@misc{huggingfacecourse,
+  author = {Hugging Face},
+  title = {The Hugging Face Course, 2022},
+  howpublished = "\url{https://huggingface.co/course}",
+  year = {2022},
+  note = "[Online; accessed <today>]"
+}
+```
+
+## Let's Go
+Are you ready to roll? In this chapter, you will learn:
+
+* How to use the `pipeline()` function to solve NLP tasks such as text generation and classification
+* About the Transformer architecture
+* How to distinguish between encoder, decoder, and encoder-decoder architectures and use cases
+

chapter1/material/10.md

@@ -0,0 +1,258 @@
+<!-- DISABLE-FRONTMATTER-SECTIONS -->
+
+# End-of-chapter quiz[[end-of-chapter-quiz]]
+
+<CourseFloatingBanner
+    chapter={1}
+    classNames="absolute z-10 right-0 top-0"
+/>
+
+This chapter covered a lot of ground! Don't worry if you didn't grasp all the details; the next chapters will help you understand how things work under the hood.
+
+First, though, let's test what you learned in this chapter!
+
+
+### 1. Explore the Hub and look for the `roberta-large-mnli` checkpoint. What task does it perform?
+
+
+<Question
+	choices={[
+		{
+			text: "Summarization",
+			explain: "Look again on the <a href=\"https://huggingface.co/roberta-large-mnli\">roberta-large-mnli page</a>."
+		},
+		{
+			text: "Text classification",
+			explain: "More precisely, it classifies if two sentences are logically linked across three labels (contradiction, neutral, entailment) — a task also called <em>natural language inference</em>.",
+			correct: true
+		},
+		{
+			text: "Text generation",
+			explain: "Look again on the <a href=\"https://huggingface.co/roberta-large-mnli\">roberta-large-mnli page</a>."
+		}
+	]}
+/>
+
+### 2. What will the following code return?
+
+```py
+from transformers import pipeline
+
+ner = pipeline("ner", grouped_entities=True)
+ner("My name is Sylvain and I work at Hugging Face in Brooklyn.")
+```
+
+<Question
+	choices={[
+		{
+			text: "It will return classification scores for this sentence, with labels \"positive\" or \"negative\".",
+			explain: "This is incorrect — this would be a <code>sentiment-analysis</code> pipeline."
+		},
+		{
+			text: "It will return a generated text completing this sentence.",
+			explain: "This is incorrect — it would be a <code>text-generation</code> pipeline.",
+		},
+		{
+			text: "It will return the words representing persons, organizations or locations.",
+			explain: "Furthermore, with <code>grouped_entities=True</code>, it will group together the words belonging to the same entity, like \"Hugging Face\".",
+			correct: true
+		}
+	]}
+/>
+
+### 3. What should replace ... in this code sample?
+
+```py
+from transformers import pipeline
+
+filler = pipeline("fill-mask", model="bert-base-cased")
+result = filler("...")
+```
+
+<Question
+	choices={[
+		{
+			text: "This &#60;mask> has been waiting for you.",
+			explain: "This is incorrect. Check out the <code>bert-base-cased</code> model card and try to spot your mistake."
+		},
+		{
+			text: "This [MASK] has been waiting for you.",
+			explain: "Correct! This model's mask token is [MASK].",
+			correct: true
+		},
+		{
+			text: "This man has been waiting for you.",
+			explain: "This is incorrect. This pipeline fills in masked words, so it needs a mask token somewhere."
+		}
+	]}
+/>
+
+### 4. Why will this code fail?
+
+```py
+from transformers import pipeline
+
+classifier = pipeline("zero-shot-classification")
+result = classifier("This is a course about the Transformers library")
+```
+
+<Question
+	choices={[
+		{
+			text: "This pipeline requires that labels be given to classify this text.",
+			explain: "Right — the correct code needs to include <code>candidate_labels=[...]</code>.",
+			correct: true
+		},
+		{
+			text: "This pipeline requires several sentences, not just one.",
+			explain: "This is incorrect, though when properly used, this pipeline can take a list of sentences to process (like all other pipelines)."
+		},
+		{
+			text: "The 🤗 Transformers library is broken, as usual.",
+			explain: "We won't dignify this answer with a comment!"
+		},
+		{
+			text: "This pipeline requires longer inputs; this one is too short.",
+			explain: "This is incorrect. Note that a very long text will be truncated when processed by this pipeline."
+		}
+	]}
+/>
+
+### 5. What does "transfer learning" mean?
+
+<Question
+	choices={[
+		{
+			text: "Transferring the knowledge of a pretrained model to a new model by training it on the same dataset.",
+			explain: "No, that would be two versions of the same model."
+		},
+		{
+			text: "Transferring the knowledge of a pretrained model to a new model by initializing the second model with the first model's weights.",
+			explain: "Correct: when the second model is trained on a new task, it *transfers* the knowledge of the first model.",
+			correct: true
+		},
+		{
+			text: "Transferring the knowledge of a pretrained model to a new model by building the second model with the same architecture as the first model.",
+			explain: "The architecture is just the way the model is built; there is no knowledge shared or transferred in this case."
+		}
+	]}
+/>
+
+### 6. True or false? A language model usually does not need labels for its pretraining.
+
+<Question
+	choices={[
+		{
+			text: "True",
+			explain: "The pretraining is usually <em>self-supervised</em>, which means the labels are created automatically from the inputs (like predicting the next word or filling in some masked words).",
+			correct: true
+		},
+		{
+			text: "False",
+			explain: "This is not the correct answer."
+		}
+	]}
+/>
+
+### 7. Select the sentence that best describes the terms "model", "architecture", and "weights".
+
+<Question
+	choices={[
+		{
+			text: "If a model is a building, its architecture is the blueprint and the weights are the people living inside.",
+			explain: "Following this metaphor, the weights would be the bricks and other materials used to construct the building."
+		},
+		{
+			text: "An architecture is a map to build a model and its weights are the cities represented on the map.",
+			explain: "The problem with this metaphor is that a map usually represents one existing reality (there is only one city in France named Paris). For a given architecture, multiple weights are possible."
+		},
+		{
+			text: "An architecture is a succession of mathematical functions to build a model and its weights are those functions parameters.",
+			explain: "The same set of mathematical functions (architecture) can be used to build different models by using different parameters (weights).",
+			correct: true
+		}
+	]}
+/>
+
+
+### 8. Which of these types of models would you use for completing prompts with generated text?
+
+<Question
+	choices={[
+		{
+			text: "An encoder model",
+			explain: "An encoder model generates a representation of the whole sentence that is better suited for tasks like classification."
+		},
+		{
+			text: "A decoder model",
+			explain: "Decoder models are perfectly suited for text generation from a prompt.",
+			correct: true
+		},
+		{
+			text: "A sequence-to-sequence model",
+			explain: "Sequence-to-sequence models are better suited for tasks where you want to generate sentences in relation to the input sentences, not a given prompt."
+		}
+	]}
+/>
+
+### 9. Which of those types of models would you use for summarizing texts?
+
+<Question
+	choices={[
+		{
+			text: "An encoder model",
+			explain: "An encoder model generates a representation of the whole sentence that is better suited for tasks like classification."
+		},
+		{
+			text: "A decoder model",
+			explain: "Decoder models are good for generating output text (like summaries), but they don't have the ability to exploit a context like the whole text to summarize."
+		},
+		{
+			text: "A sequence-to-sequence model",
+			explain: "Sequence-to-sequence models are perfectly suited for a summarization task.",
+			correct: true
+		}
+	]}
+/>
+
+### 10. Which of these types of models would you use for classifying text inputs according to certain labels?
+
+<Question
+	choices={[
+		{
+			text: "An encoder model",
+			explain: "An encoder model generates a representation of the whole sentence which is perfectly suited for a task like classification.",
+			correct: true
+		},
+		{
+			text: "A decoder model",
+			explain: "Decoder models are good for generating output texts, not extracting a label out of a sentence."
+		},
+		{
+			text: "A sequence-to-sequence model",
+			explain: "Sequence-to-sequence models are better suited for tasks where you want to generate text based on an input sentence, not a label.",
+		}
+	]}
+/>
+
+### 11. What possible source can the bias observed in a model have?
+
+<Question
+	choices={[
+		{
+			text: "The model is a fine-tuned version of a pretrained model and it picked up its bias from it.",
+			explain: "When applying Transfer Learning, the bias in the pretrained model used persists in the fine-tuned model.",
+			correct: true
+		},
+		{
+			text: "The data the model was trained on is biased.",
+			explain: "This is the most obvious source of bias, but not the only one.",
+			correct: true
+		},
+		{
+			text: "The metric the model was optimizing for is biased.",
+			explain: "A less obvious source of bias is the way the model is trained. Your model will blindly optimize for whatever metric you chose, without any second thoughts.",
+			correct: true
+		}
+	]}
+/>

chapter1/material/2.md

@@ -0,0 +1,67 @@
+# Natural Language Processing and Large Language Models[[natural-language-processing-and-large-language-models]]
+
+<CourseFloatingBanner
+    chapter={1}
+    classNames="absolute z-10 right-0 top-0"
+/>
+
+Before jumping into Transformer models, let's do a quick overview of what natural language processing is, how large language models have transformed the field, and why we care about it.
+
+## What is NLP?[[what-is-nlp]]
+
+NLP is a field of linguistics and machine learning focused on understanding everything related to human language. The aim of NLP tasks is not only to understand single words individually, but to be able to understand the context of those words.
+
+The following is a list of common NLP tasks, with some examples of each:
+
+- **Classifying whole sentences**: Getting the sentiment of a review, detecting if an email is spam, determining if a sentence is grammatically correct or whether two sentences are logically related or not
+- **Classifying each word in a sentence**: Identifying the grammatical components of a sentence (noun, verb, adjective), or the named entities (person, location, organization)
+- **Generating text content**: Completing a prompt with auto-generated text, filling in the blanks in a text with masked words
+- **Extracting an answer from a text**: Given a question and a context, extracting the answer to the question based on the information provided in the context
+- **Generating a new sentence from an input text**: Translating a text into another language, summarizing a text
+
+NLP isn't limited to written text though. It also tackles complex challenges in speech recognition and computer vision, such as generating a transcript of an audio sample or a description of an image.
+
+## What are Large Language Models (LLMs)?[[what-are-llms]]
+
+Large Language Models (LLMs) are a type of artificial intelligence model designed to understand, generate, and manipulate human language. They represent a significant advancement in the field of natural language processing (NLP).
+
+<Tip>
+
+A large language model is an AI system trained on massive amounts of text data that can understand and generate human-like text, recognize patterns in language, and perform a wide variety of language tasks without task-specific training.
+
+</Tip>
+
+LLMs are generalist models that can perform an impressive range of tasks within a single model:
+
+- Generate human-like text for creative writing, emails, or reports
+- Answer questions based on their training data
+- Summarize long documents
+- Translate between languages
+- Write and debug computer code
+- Reason through complex problems
+
+However, they also have important limitations:
+
+- They can generate incorrect information confidently (hallucinations)
+- They lack true understanding of the world and operate purely on statistical patterns
+- They may reproduce biases present in their training data or inputs.
+- They have limited context windows (though this is improving)
+- They require significant computational resources
+
+## The Rise of Large Language Models (LLMs)[[rise-of-llms]]
+
+In recent years, the field of NLP has been revolutionized by Large Language Models (LLMs). These models, which include architectures like GPT (Generative Pre-trained Transformer) and [Llama](https://huggingface.co/meta-llama), have transformed what's possible in language processing.
+
+LLMs are characterized by:
+- **Scale**: They contain millions, billions, or even hundreds of billions of parameters
+- **General capabilities**: They can perform multiple tasks without task-specific training
+- **In-context learning**: They can learn from examples provided in the prompt
+- **Emergent abilities**: As these models grow in size, they demonstrate capabilities that weren't explicitly programmed or anticipated
+
+The advent of LLMs has shifted the paradigm from building specialized models for specific NLP tasks to using a single, large model that can be prompted or fine-tuned to address a wide range of language tasks. This has made sophisticated language processing more accessible while also introducing new challenges in areas like efficiency, ethics, and deployment.
+
+## Why is language processing challenging?[[why-is-it-challenging]]
+
+Computers don't process information in the same way as humans. For example, when we read the sentence "I am hungry," we can easily understand its meaning. Similarly, given two sentences such as "I am hungry" and "I am sad," we're able to easily determine how similar they are. For machine learning (ML) models, such tasks are more difficult. The text needs to be processed in a way that enables the model to learn from it. And because language is complex, we need to think carefully about how this processing must be done. There has been a lot of research done on how to represent text, and we will look at some methods in the next chapter.
+
+Even with the advances in LLMs, many fundamental challenges remain. These include understanding ambiguity, cultural context, sarcasm, and humor. LLMs address these challenges through massive training on diverse datasets, but still often fall short of human-level understanding in many complex scenarios.

chapter1/material/3.md

@@ -0,0 +1,410 @@
+# Transformers, what can they do?[[transformers-what-can-they-do]]
+
+<CourseFloatingBanner chapter={1}
+  classNames="absolute z-10 right-0 top-0"
+  notebooks={[
+    {label: "Google Colab", value: "https://colab.research.google.com/github/huggingface/notebooks/blob/master/course/en/chapter1/section3.ipynb"},
+    {label: "Aws Studio", value: "https://studiolab.sagemaker.aws/import/github/huggingface/notebooks/blob/master/course/en/chapter1/section3.ipynb"},
+]} />
+
+In this section, we will look at what Transformer models can do and use our first tool from the 🤗 Transformers library: the `pipeline()` function.
+
+<Tip>
+👀 See that <em>Open in Colab</em> button on the top right? Click on it to open a Google Colab notebook with all the code samples of this section. This button will be present in any section containing code examples. 
+
+If you want to run the examples locally, we recommend taking a look at the <a href="/course/chapter0">setup</a>.
+</Tip>
+
+## Transformers are everywhere![[transformers-are-everywhere]]
+
+Transformer models are used to solve all kinds of tasks across different modalities, including natural language processing (NLP), computer vision, audio processing, and more. Here are some of the companies and organizations using Hugging Face and Transformer models, who also contribute back to the community by sharing their models:
+
+<img src="https://huggingface.co/datasets/huggingface-course/documentation-images/resolve/main/en/chapter1/companies.PNG" alt="Companies using Hugging Face" width="100%">
+
+The [🤗 Transformers library](https://github.com/huggingface/transformers) provides the functionality to create and use those shared models. The [Model Hub](https://huggingface.co/models) contains thousands of pretrained models that anyone can download and use. You can also upload your own models to the Hub!
+
+<Tip>
+⚠️ The Hugging Face Hub is not limited to Transformer models. Anyone can share any kind of models or datasets they want! <a href="https://huggingface.co/join">Create a huggingface.co</a> account to benefit from all available features!
+</Tip>
+
+Before diving into how Transformer models work under the hood, let's look at a few examples of how they can be used to solve some interesting NLP problems.
+
+## Working with pipelines[[working-with-pipelines]]
+
+<Youtube id="tiZFewofSLM" />
+
+The most basic object in the 🤗 Transformers library is the `pipeline()` function. It connects a model with its necessary preprocessing and postprocessing steps, allowing us to directly input any text and get an intelligible answer:
+
+```python
+from transformers import pipeline
+
+classifier = pipeline("sentiment-analysis")
+classifier("I've been waiting for a HuggingFace course my whole life.")
+```
+
+```python out
+[{'label': 'POSITIVE', 'score': 0.9598047137260437}]
+```
+
+We can even pass several sentences!
+
+```python
+classifier(
+    ["I've been waiting for a HuggingFace course my whole life.", "I hate this so much!"]
+)
+```
+
+```python out
+[{'label': 'POSITIVE', 'score': 0.9598047137260437},
+ {'label': 'NEGATIVE', 'score': 0.9994558095932007}]
+```
+
+By default, this pipeline selects a particular pretrained model that has been fine-tuned for sentiment analysis in English. The model is downloaded and cached when you create the `classifier` object. If you rerun the command, the cached model will be used instead and there is no need to download the model again.
+
+There are three main steps involved when you pass some text to a pipeline:
+
+1. The text is preprocessed into a format the model can understand.
+2. The preprocessed inputs are passed to the model.
+3. The predictions of the model are post-processed, so you can make sense of them.
+
+## Available pipelines for different modalities
+
+The `pipeline()` function supports multiple modalities, allowing you to work with text, images, audio, and even multimodal tasks. In this course we'll focus on text tasks, but it's useful to understand the transformer's architectures potential, so we'll briefly outline it. 
+
+Here's an overview of what's available:
+
+### Text pipelines
+
+- `sentiment-analysis`: Analyze whether text is positive or negative
+- `text-classification`: Classify text into predefined categories
+- `text-generation`: Generate text from a prompt
+- `fill-mask`: Fill in missing words in text
+- `summarization`: Create a shorter version of a text while preserving key information
+- `translation`: Translate text from one language to another
+- `question-answering`: Answer questions based on context
+- `zero-shot-classification`: Classify text without prior training on specific labels
+- `ner` (named entity recognition): Identify entities like people, locations, organizations
+- `feature-extraction`: Extract vector representations of text
+
+### Image pipelines
+
+- `image-classification`: Identify what's in an image
+- `object-detection`: Locate and identify objects in images
+- `image-segmentation`: Identify which pixels belong to which object
+- `depth-estimation`: Estimate the depth of objects in an image
+- `image-to-text`: Generate text descriptions of images
+
+### Audio pipelines
+
+- `automatic-speech-recognition`: Convert speech to text
+- `audio-classification`: Classify audio into categories
+- `text-to-speech`: Convert text to spoken audio
+
+### Multimodal pipelines
+
+- `document-question-answering`: Answer questions about documents with text and images
+- `visual-question-answering`: Answer questions about images
+
+Let's explore some of these pipelines in more detail!
+
+## Zero-shot classification[[zero-shot-classification]]
+
+We'll start by tackling a more challenging task where we need to classify texts that haven't been labelled. This is a common scenario in real-world projects because annotating text is usually time-consuming and requires domain expertise. For this use case, the `zero-shot-classification` pipeline is very powerful: it allows you to specify which labels to use for the classification, so you don't have to rely on the labels of the pretrained model. You've already seen how the model can classify a sentence as positive or negative using those two labels — but it can also classify the text using any other set of labels you like.
+
+```python
+from transformers import pipeline
+
+classifier = pipeline("zero-shot-classification")
+classifier(
+    "This is a course about the Transformers library",
+    candidate_labels=["education", "politics", "business"],
+)
+```
+
+```python out
+{'sequence': 'This is a course about the Transformers library',
+ 'labels': ['education', 'business', 'politics'],
+ 'scores': [0.8445963859558105, 0.111976258456707, 0.043427448719739914]}
+```
+
+This pipeline is called _zero-shot_ because you don't need to fine-tune the model on your data to use it. It can directly return probability scores for any list of labels you want!
+
+<Tip>
+
+✏️ **Try it out!** Play around with your own sequences and labels and see how the model behaves.
+
+</Tip>
+
+
+## Text generation[[text-generation]]
+
+Now let's see how to use a pipeline to generate some text. The main idea here is that you provide a prompt and the model will auto-complete it by generating the remaining text. This is similar to the predictive text feature that is found on many phones. Text generation involves randomness, so it's normal if you don't get the same results as shown below.
+
+```python
+from transformers import pipeline
+
+generator = pipeline("text-generation")
+generator("In this course, we will teach you how to")
+```
+
+```python out
+[{'generated_text': 'In this course, we will teach you how to understand and use '
+                    'data flow and data interchange when handling user data. We '
+                    'will be working with one or more of the most commonly used '
+                    'data flows — data flows of various types, as seen by the '
+                    'HTTP'}]
+```
+
+You can control how many different sequences are generated with the argument `num_return_sequences` and the total length of the output text with the argument `max_length`.
+
+<Tip>
+
+✏️ **Try it out!** Use the `num_return_sequences` and `max_length` arguments to generate two sentences of 15 words each.
+
+</Tip>
+
+
+## Using any model from the Hub in a pipeline[[using-any-model-from-the-hub-in-a-pipeline]]
+
+The previous examples used the default model for the task at hand, but you can also choose a particular model from the Hub to use in a pipeline for a specific task — say, text generation. Go to the [Model Hub](https://huggingface.co/models) and click on the corresponding tag on the left to display only the supported models for that task. You should get to a page like [this one](https://huggingface.co/models?pipeline_tag=text-generation).
+
+Let's try the [`distilgpt2`](https://huggingface.co/distilgpt2) model! Here's how to load it in the same pipeline as before:
+
+```python
+from transformers import pipeline
+
+generator = pipeline("text-generation", model="distilgpt2")
+generator(
+    "In this course, we will teach you how to",
+    max_length=30,
+    num_return_sequences=2,
+)
+```
+
+```python out
+[{'generated_text': 'In this course, we will teach you how to manipulate the world and '
+                    'move your mental and physical capabilities to your advantage.'},
+ {'generated_text': 'In this course, we will teach you how to become an expert and '
+                    'practice realtime, and with a hands on experience on both real '
+                    'time and real'}]
+```
+
+You can refine your search for a model by clicking on the language tags, and pick a model that will generate text in another language. The Model Hub even contains checkpoints for multilingual models that support several languages.
+
+Once you select a model by clicking on it, you'll see that there is a widget enabling you to try it directly online. This way you can quickly test the model's capabilities before downloading it.
+
+<Tip>
+
+✏️ **Try it out!** Use the filters to find a text generation model for another language. Feel free to play with the widget and use it in a pipeline!
+
+</Tip>
+
+### The Inference API[[the-inference-api]]
+
+All the models can be tested directly through your browser using the Inference API, which is available on the Hugging Face [website](https://huggingface.co/). You can play with the model directly on this page by inputting custom text and watching the model process the input data.
+
+The Inference API that powers the widget is also available as a paid product, which comes in handy if you need it for your workflows. See the [pricing page](https://huggingface.co/pricing) for more details.
+
+## Mask filling[[mask-filling]]
+
+The next pipeline you'll try is `fill-mask`. The idea of this task is to fill in the blanks in a given text:
+
+```python
+from transformers import pipeline
+
+unmasker = pipeline("fill-mask")
+unmasker("This course will teach you all about <mask> models.", top_k=2)
+```
+
+```python out
+[{'sequence': 'This course will teach you all about mathematical models.',
+  'score': 0.19619831442832947,
+  'token': 30412,
+  'token_str': ' mathematical'},
+ {'sequence': 'This course will teach you all about computational models.',
+  'score': 0.04052725434303284,
+  'token': 38163,
+  'token_str': ' computational'}]
+```
+
+The `top_k` argument controls how many possibilities you want to be displayed. Note that here the model fills in the special `<mask>` word, which is often referred to as a *mask token*. Other mask-filling models might have different mask tokens, so it's always good to verify the proper mask word when exploring other models. One way to check it is by looking at the mask word used in the widget.
+
+<Tip>
+
+✏️ **Try it out!** Search for the `bert-base-cased` model on the Hub and identify its mask word in the Inference API widget. What does this model predict for the sentence in our `pipeline` example above?
+
+</Tip>
+
+## Named entity recognition[[named-entity-recognition]]
+
+Named entity recognition (NER) is a task where the model has to find which parts of the input text correspond to entities such as persons, locations, or organizations. Let's look at an example:
+
+```python
+from transformers import pipeline
+
+ner = pipeline("ner", grouped_entities=True)
+ner("My name is Sylvain and I work at Hugging Face in Brooklyn.")
+```
+
+```python out
+[{'entity_group': 'PER', 'score': 0.99816, 'word': 'Sylvain', 'start': 11, 'end': 18}, 
+ {'entity_group': 'ORG', 'score': 0.97960, 'word': 'Hugging Face', 'start': 33, 'end': 45}, 
+ {'entity_group': 'LOC', 'score': 0.99321, 'word': 'Brooklyn', 'start': 49, 'end': 57}
+]
+```
+
+Here the model correctly identified that Sylvain is a person (PER), Hugging Face an organization (ORG), and Brooklyn a location (LOC).
+
+We pass the option `grouped_entities=True` in the pipeline creation function to tell the pipeline to regroup together the parts of the sentence that correspond to the same entity: here the model correctly grouped "Hugging" and "Face" as a single organization, even though the name consists of multiple words. In fact, as we will see in the next chapter, the preprocessing even splits some words into smaller parts. For instance, `Sylvain` is split into four pieces: `S`, `##yl`, `##va`, and `##in`. In the post-processing step, the pipeline successfully regrouped those pieces.
+
+<Tip>
+
+✏️ **Try it out!** Search the Model Hub for a model able to do part-of-speech tagging (usually abbreviated as POS) in English. What does this model predict for the sentence in the example above?
+
+</Tip>
+
+## Question answering[[question-answering]]
+
+The `question-answering` pipeline answers questions using information from a given context:
+
+```python
+from transformers import pipeline
+
+question_answerer = pipeline("question-answering")
+question_answerer(
+    question="Where do I work?",
+    context="My name is Sylvain and I work at Hugging Face in Brooklyn",
+)
+```
+
+```python out
+{'score': 0.6385916471481323, 'start': 33, 'end': 45, 'answer': 'Hugging Face'}
+```
+
+Note that this pipeline works by extracting information from the provided context; it does not generate the answer.
+
+## Summarization[[summarization]]
+
+Summarization is the task of reducing a text into a shorter text while keeping all (or most) of the important aspects referenced in the text. Here's an example:
+
+```python
+from transformers import pipeline
+
+summarizer = pipeline("summarization")
+summarizer(
+    """
+    America has changed dramatically during recent years. Not only has the number of 
+    graduates in traditional engineering disciplines such as mechanical, civil, 
+    electrical, chemical, and aeronautical engineering declined, but in most of 
+    the premier American universities engineering curricula now concentrate on 
+    and encourage largely the study of engineering science. As a result, there 
+    are declining offerings in engineering subjects dealing with infrastructure, 
+    the environment, and related issues, and greater concentration on high 
+    technology subjects, largely supporting increasingly complex scientific 
+    developments. While the latter is important, it should not be at the expense 
+    of more traditional engineering.
+
+    Rapidly developing economies such as China and India, as well as other 
+    industrial countries in Europe and Asia, continue to encourage and advance 
+    the teaching of engineering. Both China and India, respectively, graduate 
+    six and eight times as many traditional engineers as does the United States. 
+    Other industrial countries at minimum maintain their output, while America 
+    suffers an increasingly serious decline in the number of engineering graduates 
+    and a lack of well-educated engineers.
+"""
+)
+```
+
+```python out
+[{'summary_text': ' America has changed dramatically during recent years . The '
+                  'number of engineering graduates in the U.S. has declined in '
+                  'traditional engineering disciplines such as mechanical, civil '
+                  ', electrical, chemical, and aeronautical engineering . Rapidly '
+                  'developing economies such as China and India, as well as other '
+                  'industrial countries in Europe and Asia, continue to encourage '
+                  'and advance engineering .'}]
+```
+
+Like with text generation, you can specify a `max_length` or a `min_length` for the result.
+
+
+## Translation[[translation]]
+
+For translation, you can use a default model if you provide a language pair in the task name (such as `"translation_en_to_fr"`), but the easiest way is to pick the model you want to use on the [Model Hub](https://huggingface.co/models). Here we'll try translating from French to English:
+
+```python
+from transformers import pipeline
+
+translator = pipeline("translation", model="Helsinki-NLP/opus-mt-fr-en")
+translator("Ce cours est produit par Hugging Face.")
+```
+
+```python out
+[{'translation_text': 'This course is produced by Hugging Face.'}]
+```
+
+Like with text generation and summarization, you can specify a `max_length` or a `min_length` for the result.
+
+<Tip>
+
+✏️ **Try it out!** Search for translation models in other languages and try to translate the previous sentence into a few different languages.
+
+</Tip>
+
+## Image and audio pipelines
+
+Beyond text, Transformer models can also work with images and audio. Here are a few examples:
+
+### Image classification
+
+```python
+from transformers import pipeline
+
+image_classifier = pipeline("image-classification")
+result = image_classifier(
+    "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/pipeline-cat-chonk.jpeg"
+)
+print(result)
+```
+
+```python out
+[{'score': 0.4585989117622375, 'label': 'lynx, catamount'}, 
+ {'score': 0.10480582714080811, 'label': 'Egyptian cat'}, 
+ {'score': 0.08250780403614044, 'label': 'tabby, tabby cat'}, 
+ {'score': 0.0732262134552002, 'label': 'tiger cat'}, 
+ {'score': 0.04910798370838165, 'label': 'cougar, puma, catamount, mountain lion, painter, panther, Felis concolor'}]
+```
+
+### Automatic speech recognition
+
+```python
+from transformers import pipeline
+
+transcriber = pipeline("automatic-speech-recognition")
+result = transcriber(
+    "https://huggingface.co/datasets/Narsil/asr_dummy/resolve/main/mlk.flac"
+)
+print(result)
+```
+
+```python out
+{'text': 'I HAVE A DREAM BUT ONE DAY THIS NATION WILL RISE UP LIVE UP THE TRUE MEANING OF ITS CREED'}
+```
+
+## Combining data from multiple sources
+
+One powerful application of Transformer models is their ability to combine and process data from multiple sources. This is especially useful when you need to:
+
+1. Search across multiple databases or repositories
+2. Consolidate information from different formats (text, images, audio)
+3. Create a unified view of related information
+
+For example, you could build a system that:
+- Searches for information across databases in multiple modalities like text and image.
+- Combines results from different sources into a single coherent response. For example, from an audio file and text description.
+- Presents the most relevant information from a database of documents and metadata.
+
+
+## Conclusion
+
+The pipelines shown in this chapter are mostly for demonstrative purposes. They were programmed for specific tasks and cannot perform variations of them. In the next chapter, you'll learn what's inside a `pipeline()` function and how to customize its behavior.

chapter1/material/4.md

@@ -0,0 +1,178 @@
+# How do Transformers work?[[how-do-transformers-work]]
+
+<CourseFloatingBanner
+    chapter={1}
+    classNames="absolute z-10 right-0 top-0"
+/>
+
+In this section, we will take a high-level look at the architecture of Transformer models.
+
+## A bit of Transformer history[[a-bit-of-transformer-history]]
+
+Here are some reference points in the (short) history of Transformer models:
+
+<div class="flex justify-center">
+<img class="block dark:hidden" src="https://huggingface.co/datasets/huggingface-course/documentation-images/resolve/main/en/chapter1/transformers_chrono.svg" alt="A brief chronology of Transformers models.">
+<img class="hidden dark:block" src="https://huggingface.co/datasets/huggingface-course/documentation-images/resolve/main/en/chapter1/transformers_chrono-dark.svg" alt="A brief chronology of Transformers models.">
+</div>
+
+The [Transformer architecture](https://arxiv.org/abs/1706.03762) was introduced in June 2017. The focus of the original research was on translation tasks. This was followed by the introduction of several influential models, including:
+
+- **June 2018**: [GPT](https://cdn.openai.com/research-covers/language-unsupervised/language_understanding_paper.pdf), the first pretrained Transformer model, used for fine-tuning on various NLP tasks and obtained state-of-the-art results
+
+- **October 2018**: [BERT](https://arxiv.org/abs/1810.04805), another large pretrained model, this one designed to produce better summaries of sentences (more on this in the next chapter!)
+
+- **February 2019**: [GPT-2](https://cdn.openai.com/better-language-models/language_models_are_unsupervised_multitask_learners.pdf), an improved (and bigger) version of GPT that was not immediately publicly released due to ethical concerns
+
+- **October 2019**: [DistilBERT](https://arxiv.org/abs/1910.01108), a distilled version of BERT that is 60% faster, 40% lighter in memory, and still retains 97% of BERT's performance
+
+- **October 2019**: [BART](https://arxiv.org/abs/1910.13461) and [T5](https://arxiv.org/abs/1910.10683), two large pretrained models using the same architecture as the original Transformer model (the first to do so)
+
+- **May 2020**, [GPT-3](https://arxiv.org/abs/2005.14165), an even bigger version of GPT-2 that is able to perform well on a variety of tasks without the need for fine-tuning (called _zero-shot learning_)
+
+This list is far from comprehensive, and is just meant to highlight a few of the different kinds of Transformer models. Broadly, they can be grouped into three categories:
+
+- GPT-like (also called _auto-regressive_ Transformer models)
+- BERT-like (also called _auto-encoding_ Transformer models) 
+- BART/T5-like (also called _sequence-to-sequence_ Transformer models)
+
+We will dive into these families in more depth later on.
+
+## Transformers are language models[[transformers-are-language-models]]
+
+All the Transformer models mentioned above (GPT, BERT, BART, T5, etc.) have been trained as *language models*. This means they have been trained on large amounts of raw text in a self-supervised fashion. Self-supervised learning is a type of training in which the objective is automatically computed from the inputs of the model. That means that humans are not needed to label the data!
+
+This type of model develops a statistical understanding of the language it has been trained on, but it's not very useful for specific practical tasks. Because of this, the general pretrained model then goes through a process called *transfer learning*. During this process, the model is fine-tuned in a supervised way -- that is, using human-annotated labels -- on a given task.
+
+An example of a task is predicting the next word in a sentence having read the *n* previous words. This is called *causal language modeling* because the output depends on the past and present inputs, but not the future ones.
+
+<div class="flex justify-center">
+<img class="block dark:hidden" src="https://huggingface.co/datasets/huggingface-course/documentation-images/resolve/main/en/chapter1/causal_modeling.svg" alt="Example of causal language modeling in which the next word from a sentence is predicted.">
+<img class="hidden dark:block" src="https://huggingface.co/datasets/huggingface-course/documentation-images/resolve/main/en/chapter1/causal_modeling-dark.svg" alt="Example of causal language modeling in which the next word from a sentence is predicted.">
+</div>
+
+Another example is *masked language modeling*, in which the model predicts a masked word in the sentence.
+
+<div class="flex justify-center">
+<img class="block dark:hidden" src="https://huggingface.co/datasets/huggingface-course/documentation-images/resolve/main/en/chapter1/masked_modeling.svg" alt="Example of masked language modeling in which a masked word from a sentence is predicted.">
+<img class="hidden dark:block" src="https://huggingface.co/datasets/huggingface-course/documentation-images/resolve/main/en/chapter1/masked_modeling-dark.svg" alt="Example of masked language modeling in which a masked word from a sentence is predicted.">
+</div>
+
+## Transformers are big models[[transformers-are-big-models]]
+
+Apart from a few outliers (like DistilBERT), the general strategy to achieve better performance is by increasing the models' sizes as well as the amount of data they are pretrained on.
+
+<div class="flex justify-center">
+<img src="https://huggingface.co/datasets/huggingface-course/documentation-images/resolve/main/en/chapter1/model_parameters.png" alt="Number of parameters of recent Transformers models" width="90%">
+</div>
+
+Unfortunately, training a model, especially a large one, requires a large amount of data. This becomes very costly in terms of time and compute resources. It even translates to environmental impact, as can be seen in the following graph.
+
+<div class="flex justify-center">
+<img class="block dark:hidden" src="https://huggingface.co/datasets/huggingface-course/documentation-images/resolve/main/en/chapter1/carbon_footprint.svg" alt="The carbon footprint of a large language model.">
+<img class="hidden dark:block" src="https://huggingface.co/datasets/huggingface-course/documentation-images/resolve/main/en/chapter1/carbon_footprint-dark.svg" alt="The carbon footprint of a large language model.">
+</div>
+
+<Youtube id="ftWlj4FBHTg"/>
+
+And this is showing a project for a (very big) model led by a team consciously trying to reduce the environmental impact of pretraining. The footprint of running lots of trials to get the best hyperparameters would be even higher.
+
+Imagine if each time a research team, a student organization, or a company wanted to train a model, it did so from scratch. This would lead to huge, unnecessary global costs!
+
+This is why sharing language models is paramount: sharing the trained weights and building on top of already trained weights reduces the overall compute cost and carbon footprint of the community.
+
+By the way, you can evaluate the carbon footprint of your models' training through several tools. For example [ML CO2 Impact](https://mlco2.github.io/impact/) or [Code Carbon]( https://codecarbon.io/) which is integrated in 🤗 Transformers. To learn more about this, you can read this [blog post](https://huggingface.co/blog/carbon-emissions-on-the-hub) which will show you how to generate an `emissions.csv` file with an estimate of the footprint of your training, as well as the [documentation](https://huggingface.co/docs/hub/model-cards-co2) of 🤗 Transformers addressing this topic.
+
+
+## Transfer Learning[[transfer-learning]]
+
+<Youtube id="BqqfQnyjmgg" />
+
+*Pretraining* is the act of training a model from scratch: the weights are randomly initialized, and the training starts without any prior knowledge.
+
+<div class="flex justify-center">
+<img class="block dark:hidden" src="https://huggingface.co/datasets/huggingface-course/documentation-images/resolve/main/en/chapter1/pretraining.svg" alt="The pretraining of a language model is costly in both time and money.">
+<img class="hidden dark:block" src="https://huggingface.co/datasets/huggingface-course/documentation-images/resolve/main/en/chapter1/pretraining-dark.svg" alt="The pretraining of a language model is costly in both time and money.">
+</div>
+
+This pretraining is usually done on very large amounts of data. Therefore, it requires a very large corpus of data, and training can take up to several weeks.
+
+*Fine-tuning*, on the other hand, is the training done **after** a model has been pretrained. To perform fine-tuning, you first acquire a pretrained language model, then perform additional training with a dataset specific to your task. Wait -- why not simply train the model for your final use case from the start (**scratch**)? There are a couple of reasons:
+
+*  The pretrained model was already trained on a dataset that has some similarities with the fine-tuning dataset. The fine-tuning process is thus able to take advantage of knowledge acquired by the initial model during pretraining (for instance, with NLP problems, the pretrained model will have some kind of statistical understanding of the language you are using for your task). 
+*  Since the pretrained model was already trained on lots of data, the fine-tuning requires way less data to get decent results.
+*  For the same reason, the amount of time and resources needed to get good results are much lower.
+
+For example, one could leverage a pretrained model trained on the English language and then fine-tune it on an arXiv corpus, resulting in a science/research-based model. The fine-tuning will only require a limited amount of data: the knowledge the pretrained model has acquired is "transferred," hence the term *transfer learning*.
+
+<div class="flex justify-center">
+<img class="block dark:hidden" src="https://huggingface.co/datasets/huggingface-course/documentation-images/resolve/main/en/chapter1/finetuning.svg" alt="The fine-tuning of a language model is cheaper than pretraining in both time and money.">
+<img class="hidden dark:block" src="https://huggingface.co/datasets/huggingface-course/documentation-images/resolve/main/en/chapter1/finetuning-dark.svg" alt="The fine-tuning of a language model is cheaper than pretraining in both time and money.">
+</div>
+
+Fine-tuning a model therefore has lower time, data, financial, and environmental costs. It is also quicker and easier to iterate over different fine-tuning schemes, as the training is less constraining than a full pretraining.
+
+This process will also achieve better results than training from scratch (unless you have lots of data), which is why you should always try to leverage a pretrained model -- one as close as possible to the task you have at hand -- and fine-tune it.
+
+## General architecture[[general-architecture]]
+
+In this section, we'll go over the general architecture of the Transformer model. Don't worry if you don't understand some of the concepts; there are detailed sections later covering each of the components.
+
+<Youtube id="H39Z_720T5s" />
+
+## Introduction[[introduction]]
+
+The model is primarily composed of two blocks:
+
+* **Encoder (left)**: The encoder receives an input and builds a representation of it (its features). This means that the model is optimized to acquire understanding from the input.
+* **Decoder (right)**: The decoder uses the encoder's representation (features) along with other inputs to generate a target sequence. This means that the model is optimized for generating outputs.
+
+<div class="flex justify-center">
+<img class="block dark:hidden" src="https://huggingface.co/datasets/huggingface-course/documentation-images/resolve/main/en/chapter1/transformers_blocks.svg" alt="Architecture of a Transformers models">
+<img class="hidden dark:block" src="https://huggingface.co/datasets/huggingface-course/documentation-images/resolve/main/en/chapter1/transformers_blocks-dark.svg" alt="Architecture of a Transformers models">
+</div>
+
+Each of these parts can be used independently, depending on the task: 
+
+* **Encoder-only models**: Good for tasks that require understanding of the input, such as sentence classification and named entity recognition.
+* **Decoder-only models**: Good for generative tasks such as text generation.
+* **Encoder-decoder models** or **sequence-to-sequence models**: Good for generative tasks that require an input, such as translation or summarization.
+
+We will dive into those architectures independently in later sections.
+
+## Attention layers[[attention-layers]]
+
+A key feature of Transformer models is that they are built with special layers called *attention layers*. In fact, the title of the paper introducing the Transformer architecture was ["Attention Is All You Need"](https://arxiv.org/abs/1706.03762)! We will explore the details of attention layers later in the course; for now, all you need to know is that this layer will tell the model to pay specific attention to certain words in the sentence you passed it (and more or less ignore the others) when dealing with the representation of each word.
+
+To put this into context, consider the task of translating text from English to French. Given the input "You like this course", a translation model will need to also attend to the adjacent word "You" to get the proper translation for the word "like", because in French the verb "like" is conjugated differently depending on the subject. The rest of the sentence, however, is not useful for the translation of that word. In the same vein, when translating "this" the model will also need to pay attention to the word "course", because "this" translates differently depending on whether the associated noun is masculine or feminine. Again, the other words in the sentence will not matter for the translation of "course". With more complex sentences (and more complex grammar rules), the model would need to pay special attention to words that might appear farther away in the sentence to properly translate each word.
+
+The same concept applies to any task associated with natural language: a word by itself has a meaning, but that meaning is deeply affected by the context, which can be any other word (or words) before or after the word being studied.
+
+Now that you have an idea of what attention layers are all about, let's take a closer look at the Transformer architecture.
+
+## The original architecture[[the-original-architecture]]
+
+The Transformer architecture was originally designed for translation. During training, the encoder receives inputs (sentences) in a certain language, while the decoder receives the same sentences in the desired target language. In the encoder, the attention layers can use all the words in a sentence (since, as we just saw, the translation of a given word can be dependent on what is after as well as before it in the sentence). The decoder, however, works sequentially and can only pay attention to the words in the sentence that it has already translated (so, only the words before the word currently being generated). For example, when we have predicted the first three words of the translated target, we give them to the decoder  which then uses all the inputs of the encoder to try to predict the fourth word.
+
+To speed things up during training (when the model has access to target sentences), the decoder is fed the whole target, but it is not allowed to use future words (if it had access to the word at position 2 when trying to predict the word at position 2, the problem would not be very hard!). For instance, when trying to predict the fourth word, the attention layer will only have access to the words in positions 1 to 3.
+
+The original Transformer architecture looked like this, with the encoder on the left and the decoder on the right:
+
+<div class="flex justify-center">
+<img class="block dark:hidden" src="https://huggingface.co/datasets/huggingface-course/documentation-images/resolve/main/en/chapter1/transformers.svg" alt="Architecture of a Transformers models">
+<img class="hidden dark:block" src="https://huggingface.co/datasets/huggingface-course/documentation-images/resolve/main/en/chapter1/transformers-dark.svg" alt="Architecture of a Transformers models">
+</div>
+
+Note that the first attention layer in a decoder block pays attention to all (past) inputs to the decoder, but the second attention layer uses the output of the encoder. It can thus access the whole input sentence to best predict the current word. This is very useful as different languages can have grammatical rules that put the words in different orders, or some context provided later in the sentence may be helpful to determine the best translation of a given word.
+
+The *attention mask* can also be used in the encoder/decoder to prevent the model from paying attention to some special words -- for instance, the special padding word used to make all the inputs the same length when batching together sentences.
+
+##  Architectures vs. checkpoints[[architecture-vs-checkpoints]]
+
+As we dive into Transformer models in this course, you'll see mentions of *architectures* and *checkpoints* as well as *models*. These terms all have slightly different meanings: 
+
+* **Architecture**: This is the skeleton of the model -- the definition of each layer and each operation that happens within the model. 
+* **Checkpoints**: These are the weights that will be loaded in a given architecture.
+* **Model**: This is an umbrella term that isn't as precise as "architecture" or "checkpoint": it can mean both. This course will specify *architecture* or *checkpoint* when it matters to reduce ambiguity.
+
+For example, BERT is an architecture while `bert-base-cased`, a set of weights trained by the Google team for the first release of BERT, is a checkpoint. However, one can say "the BERT model" and "the `bert-base-cased` model."

chapter1/material/5.md

@@ -0,0 +1,250 @@
+# How 🤗 Transformers solve tasks
+
+In [🤗 Transformers can do](/course/chapter1/2), you learned about natural language processing (NLP), speech and audio, computer vision tasks, and some important applications of them. This page will look closely at how models solve these tasks and explain what's happening under the hood. There are many ways to solve a given task, some models may implement certain techniques or even approach the task from a new angle, but for Transformer models, the general idea is the same. Owing to its flexible architecture, most models are a variant of an encoder, a decoder, or an encoder-decoder structure. 
+
+<Tip>
+
+Before diving into specific architectural variants, it's helpful to understand that most tasks follow a similar pattern: input data is processed through a model, and the output is interpreted for a specific task. The differences lie in how the data is prepared, what model architecture variant is used, and how the output is processed.
+
+</Tip>
+
+To explain how tasks are solved, we'll walk through what goes on inside the model to output useful predictions. We'll cover the following models and their corresponding tasks:
+
+- [Wav2Vec2](model_doc/wav2vec2) for audio classification and automatic speech recognition (ASR)
+- [Vision Transformer (ViT)](model_doc/vit) and [ConvNeXT](model_doc/convnext) for image classification
+- [DETR](model_doc/detr) for object detection
+- [Mask2Former](model_doc/mask2former) for image segmentation
+- [GLPN](model_doc/glpn) for depth estimation
+- [BERT](model_doc/bert) for NLP tasks like text classification, token classification and question answering that use an encoder
+- [GPT2](model_doc/gpt2) for NLP tasks like text generation that use a decoder
+- [BART](model_doc/bart) for NLP tasks like summarization and translation that use an encoder-decoder
+
+<Tip>
+
+Before you go further, it is good to have some basic knowledge of the original Transformer architecture. Knowing how encoders, decoders, and attention work will aid you in understanding how different Transformer models work. Be sure to check out our [the previous section](https://huggingface.co/course/chapter1/4?fw=pt) for more information! 
+
+</Tip>
+
+## Transformer models for Language 
+
+Language models are at the heart of modern NLP. They're designed to understand and generate human language by learning the statistical patterns and relationships between words or tokens in text.
+
+The Transformer was initially designed for machine translation, and since then, it has become the default architecture for solving all AI tasks. Some tasks lend themselves to the Transformer's encoder structure, while others are better suited for the decoder. Still, other tasks make use of both the Transformer's encoder-decoder structure.
+
+<Tip>
+A language model is a statistical model that predicts the probability of a sequence of words. Modern language models can both understand and generate text, making them versatile tools for many NLP tasks.
+</Tip>
+
+### How language models work
+
+Language models work by being trained to predict the probability of a word given the context of surrounding words. This gives them a foundational understanding of language that can generalize to other tasks.
+
+There are two main approaches for training a transformer model:
+
+1. **Masked language modeling (MLM)**: Used by encoder models like BERT, this approach randomly masks some tokens in the input and trains the model to predict the original tokens based on the surrounding context. This allows the model to learn bidirectional context (looking at words both before and after the masked word).
+
+2. **Causal language modeling (CLM)**: Used by decoder models like GPT, this approach predicts the next token based on all previous tokens in the sequence. The model can only use context from the left (previous tokens) to predict the next token.
+
+### Types of language models
+
+In the Transformers library, language models generally fall into three architectural categories:
+
+1. **Encoder-only models** (like BERT): These models use a bidirectional approach to understand context from both directions. They're best suited for tasks that require deep understanding of text, such as classification, named entity recognition, and question answering.
+
+2. **Decoder-only models** (like GPT): These models process text from left to right and are particularly good at text generation tasks. They can complete sentences, write essays, or even generate code based on a prompt.
+
+3. **Encoder-decoder models** (like T5, BART): These models combine both approaches, using an encoder to understand the input and a decoder to generate output. They excel at sequence-to-sequence tasks like translation, summarization, and question answering.
+
+Language models are typically pretrained on large amounts of text data in a self-supervised manner (without human annotations), then fine-tuned on specific tasks. This approach, known as transfer learning, allows these models to adapt to many different NLP tasks with relatively small amounts of task-specific data.
+
+In the following sections, we'll explore specific model architectures and how they're applied to various tasks across speech, vision, and text domains.
+
+<Tip>
+Understanding which part of the Transformer architecture (encoder, decoder, or both) is best suited for a particular NLP task is key to choosing the right model. Generally, tasks requiring bidirectional context use encoders, tasks generating text use decoders, and tasks converting one sequence to another use encoder-decoders.
+</Tip>
+
+### Text classification
+
+Text classification involves assigning predefined categories to text documents, such as sentiment analysis, topic classification, or spam detection.
+
+[BERT](model_doc/bert) is an encoder-only model and is the first model to effectively implement deep bidirectionality to learn richer representations of the text by attending to words on both sides.
+
+1. BERT uses [WordPiece](tokenizer_summary#wordpiece) tokenization to generate a token embedding of the text. To tell the difference between a single sentence and a pair of sentences, a special `[SEP]` token is added to differentiate them. A special `[CLS]` token is added to the beginning of every sequence of text. The final output with the `[CLS]` token is used as the input to the classification head for classification tasks. BERT also adds a segment embedding to denote whether a token belongs to the first or second sentence in a pair of sentences.
+
+2. BERT is pretrained with two objectives: masked language modeling and next-sentence prediction. In masked language modeling, some percentage of the input tokens are randomly masked, and the model needs to predict these. This solves the issue of bidirectionality, where the model could cheat and see all the words and "predict" the next word. The final hidden states of the predicted mask tokens are passed to a feedforward network with a softmax over the vocabulary to predict the masked word.
+
+    The second pretraining object is next-sentence prediction. The model must predict whether sentence B follows sentence A. Half of the time sentence B is the next sentence, and the other half of the time, sentence B is a random sentence. The prediction, whether it is the next sentence or not, is passed to a feedforward network with a softmax over the two classes (`IsNext` and `NotNext`).
+
+3. The input embeddings are passed through multiple encoder layers to output some final hidden states.
+
+To use the pretrained model for text classification, add a sequence classification head on top of the base BERT model. The sequence classification head is a linear layer that accepts the final hidden states and performs a linear transformation to convert them into logits. The cross-entropy loss is calculated between the logits and target to find the most likely label.
+
+Ready to try your hand at text classification? Check out our complete [text classification guide](tasks/sequence_classification) to learn how to finetune DistilBERT and use it for inference!
+
+### Token classification
+
+Token classification involves assigning a label to each token in a sequence, such as in named entity recognition or part-of-speech tagging.
+
+To use BERT for token classification tasks like named entity recognition (NER), add a token classification head on top of the base BERT model. The token classification head is a linear layer that accepts the final hidden states and performs a linear transformation to convert them into logits. The cross-entropy loss is calculated between the logits and each token to find the most likely label.
+
+Ready to try your hand at token classification? Check out our complete [token classification guide](tasks/token_classification) to learn how to finetune DistilBERT and use it for inference!
+
+### Question answering
+
+Question answering involves finding the answer to a question within a given context or passage.
+
+To use BERT for question answering, add a span classification head on top of the base BERT model. This linear layer accepts the final hidden states and performs a linear transformation to compute the `span` start and end logits corresponding to the answer. The cross-entropy loss is calculated between the logits and the label position to find the most likely span of text corresponding to the answer.
+
+Ready to try your hand at question answering? Check out our complete [question answering guide](tasks/question_answering) to learn how to finetune DistilBERT and use it for inference!
+
+<Tip>
+💡 Notice how easy it is to use BERT for different tasks once it's been pretrained. You only need to add a specific head to the pretrained model to manipulate the hidden states into your desired output!
+</Tip>
+
+### Text generation
+
+Text generation involves creating coherent and contextually relevant text based on a prompt or input.
+
+[GPT-2](model_doc/gpt2) is a decoder-only model pretrained on a large amount of text. It can generate convincing (though not always true!) text given a prompt and complete other NLP tasks like question answering despite not being explicitly trained to.
+
+<div class="flex justify-center">
+    <img src="https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/gpt2_architecture.png"/>
+</div>
+
+1. GPT-2 uses [byte pair encoding (BPE)](tokenizer_summary#bytepair-encoding-bpe) to tokenize words and generate a token embedding. Positional encodings are added to the token embeddings to indicate the position of each token in the sequence. The input embeddings are passed through multiple decoder blocks to output some final hidden state. Within each decoder block, GPT-2 uses a *masked self-attention* layer which means GPT-2 can't attend to future tokens. It is only allowed to attend to tokens on the left. This is different from BERT's [`mask`] token because, in masked self-attention, an attention mask is used to set the score to `0` for future tokens.
+
+2. The output from the decoder is passed to a language modeling head, which performs a linear transformation to convert the hidden states into logits. The label is the next token in the sequence, which are created by shifting the logits to the right by one. The cross-entropy loss is calculated between the shifted logits and the labels to output the next most likely token.
+
+GPT-2's pretraining objective is based entirely on [causal language modeling](glossary#causal-language-modeling), predicting the next word in a sequence. This makes GPT-2 especially good at tasks that involve generating text.
+
+Ready to try your hand at text generation? Check out our complete [causal language modeling guide](tasks/language_modeling#causal-language-modeling) to learn how to finetune DistilGPT-2 and use it for inference!
+
+<Tip>
+For more information about text generation, check out the [text generation strategies](generation_strategies) guide!
+</Tip>
+
+### Summarization
+
+Summarization involves condensing a longer text into a shorter version while preserving its key information and meaning.
+
+Encoder-decoder models like [BART](model_doc/bart) and [T5](model_doc/t5) are designed for the sequence-to-sequence pattern of a summarization task. We'll explain how BART works in this section, and then you can try finetuning T5 at the end.
+
+<div class="flex justify-center">
+    <img src="https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/bart_architecture.png"/>
+</div>
+
+1. BART's encoder architecture is very similar to BERT and accepts a token and positional embedding of the text. BART is pretrained by corrupting the input and then reconstructing it with the decoder. Unlike other encoders with specific corruption strategies, BART can apply any type of corruption. The *text infilling* corruption strategy works the best though. In text infilling, a number of text spans are replaced with a **single** [`mask`] token. This is important because the model has to predict the masked tokens, and it teaches the model to predict the number of missing tokens. The input embeddings and masked spans are passed through the encoder to output some final hidden states, but unlike BERT, BART doesn't add a final feedforward network at the end to predict a word.
+
+2. The encoder's output is passed to the decoder, which must predict the masked tokens and any uncorrupted tokens from the encoder's output. This gives additional context to help the decoder restore the original text. The output from the decoder is passed to a language modeling head, which performs a linear transformation to convert the hidden states into logits. The cross-entropy loss is calculated between the logits and the label, which is just the token shifted to the right.
+
+Ready to try your hand at summarization? Check out our complete [summarization guide](tasks/summarization) to learn how to finetune T5 and use it for inference!
+
+<Tip>
+For more information about text generation, check out the [text generation strategies](generation_strategies) guide!
+</Tip>
+
+### Translation
+
+Translation involves converting text from one language to another while preserving its meaning.
+
+Translation is another example of a sequence-to-sequence task, which means you can use an encoder-decoder model like [BART](model_doc/bart) or [T5](model_doc/t5) to do it. We'll explain how BART works in this section, and then you can try finetuning T5 at the end.
+
+BART adapts to translation by adding a separate randomly initialized encoder to map a source language to an input that can be decoded into the target language. This new encoder's embeddings are passed to the pretrained encoder instead of the original word embeddings. The source encoder is trained by updating the source encoder, positional embeddings, and input embeddings with the cross-entropy loss from the model output. The model parameters are frozen in this first step, and all the model parameters are trained together in the second step.
+
+BART has since been followed up by a multilingual version, mBART, intended for translation and pretrained on many different languages.
+
+Ready to try your hand at translation? Check out our complete [translation guide](tasks/translation) to learn how to finetune T5 and use it for inference!
+
+<Tip>
+As you've seen throughout this guide, many models follow similar patterns despite addressing different tasks. Understanding these common patterns can help you quickly grasp how new models work and how to adapt existing models to your specific needs.
+</Tip>
+
+## Modalities beyond text
+
+Transformers are not limited to text. They can also be applied to other modalities like speech and audio, images, and video. Of course, on this course we will focus on text, but we can briefly introduce the other modalities.
+
+### Speech and audio
+
+Let's start by exploring how Transformer models handle speech and audio data, which presents unique challenges compared to text or images.
+
+[Wav2Vec2](model_doc/wav2vec2) is a self-supervised model pretrained on unlabeled speech data and finetuned on labeled data for audio classification and automatic speech recognition. 
+
+<div class="flex justify-center">
+    <img src="https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/wav2vec2_architecture.png"/>
+</div>
+
+This model has four main components:
+
+1. A *feature encoder* takes the raw audio waveform, normalizes it to zero mean and unit variance, and converts it into a sequence of feature vectors that are each 20ms long.
+
+2. Waveforms are continuous by nature, so they can't be divided into separate units like a sequence of text can be split into words. That's why the feature vectors are passed to a *quantization module*, which aims to learn discrete speech units. The speech unit is chosen from a collection of codewords, known as a *codebook* (you can think of this as the vocabulary). From the codebook, the vector or speech unit, that best represents the continuous audio input is chosen and forwarded through the model.
+
+3. About half of the feature vectors are randomly masked, and the masked feature vector is fed to a *context network*, which is a Transformer encoder that also adds relative positional embeddings.
+
+4. The pretraining objective of the context network is a *contrastive task*. The model has to predict the true quantized speech representation of the masked prediction from a set of false ones, encouraging the model to find the most similar context vector and quantized speech unit (the target label).
+
+Now that wav2vec2 is pretrained, you can finetune it on your data for audio classification or automatic speech recognition!
+
+<Tip>
+The key innovation in Wav2Vec2 is how it converts continuous audio signals into discrete units that can be processed like text tokens. This bridging between continuous and discrete representations is what makes it so powerful for speech tasks.
+</Tip>
+
+### Audio classification
+
+To use the pretrained model for audio classification, add a sequence classification head on top of the base Wav2Vec2 model. The classification head is a linear layer that accepts the encoder's hidden states. The hidden states represent the learned features from each audio frame which can have varying lengths. To create one vector of fixed-length, the hidden states are pooled first and then transformed into logits over the class labels. The cross-entropy loss is calculated between the logits and target to find the most likely class.
+
+Ready to try your hand at audio classification? Check out our complete [audio classification guide](tasks/audio_classification) to learn how to finetune Wav2Vec2 and use it for inference!
+
+### Automatic speech recognition
+
+To use the pretrained model for automatic speech recognition, add a language modeling head on top of the base Wav2Vec2 model for [connectionist temporal classification (CTC)](glossary#connectionist-temporal-classification-ctc). The language modeling head is a linear layer that accepts the encoder's hidden states and transforms them into logits. Each logit represents a token class (the number of tokens comes from the task vocabulary). The CTC loss is calculated between the logits and targets to find the most likely sequence of tokens, which are then decoded into a transcription.
+
+Ready to try your hand at automatic speech recognition? Check out our complete [automatic speech recognition guide](tasks/asr) to learn how to finetune Wav2Vec2 and use it for inference!
+
+### Computer vision
+
+Now let's move on to computer vision tasks, which deal with understanding and interpreting visual information from images or videos.
+
+There are two ways to approach computer vision tasks:
+
+1. Split an image into a sequence of patches and process them in parallel with a Transformer.
+2. Use a modern CNN, like [ConvNeXT](model_doc/convnext), which relies on convolutional layers but adopts modern network designs.
+
+<Tip>
+
+A third approach mixes Transformers with convolutions (for example, [Convolutional Vision Transformer](model_doc/cvt) or [LeViT](model_doc/levit)). We won't discuss those because they just combine the two approaches we examine here.
+
+</Tip>
+
+ViT and ConvNeXT are commonly used for image classification, but for other vision tasks like object detection, segmentation, and depth estimation, we'll look at DETR, Mask2Former and GLPN, respectively; these models are better suited for those tasks.
+
+### Image classification
+
+Image classification is one of the fundamental computer vision tasks. Let's see how different model architectures approach this problem.
+
+ViT and ConvNeXT can both be used for image classification; the main difference is that ViT uses an attention mechanism while ConvNeXT uses convolutions.
+
+
+[ViT](model_doc/vit) replaces convolutions entirely with a pure Transformer architecture. If you're familiar with the original Transformer, then you're already most of the way toward understanding ViT.
+
+<div class="flex justify-center">
+    <img src="https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/model_doc/vit_architecture.jpg"/>
+</div>
+
+The main change ViT introduced was in how images are fed to a Transformer:
+
+1. An image is split into square non-overlapping patches, each of which gets turned into a vector or *patch embedding*. The patch embeddings are generated from a convolutional 2D layer which creates the proper input dimensions (which for a base Transformer is 768 values for each patch embedding). If you had a 224x224 pixel image, you could split it into 196 16x16 image patches. Just like how text is tokenized into words, an image is "tokenized" into a sequence of patches.
+
+2. A *learnable embedding* - a special `[CLS]` token - is added to the beginning of the patch embeddings just like BERT. The final hidden state of the `[CLS]` token is used as the input to the attached classification head; other outputs are ignored. This token helps the model learn how to encode a representation of the image.
+
+3. The last thing to add to the patch and learnable embeddings are the *position embeddings* because the model doesn't know how the image patches are ordered. The position embeddings are also learnable and have the same size as the patch embeddings. Finally, all of the embeddings are passed to the Transformer encoder.
+
+4. The output, specifically only the output with the `[CLS]` token, is passed to a multilayer perceptron head (MLP). ViT's pretraining objective is simply classification. Like other classification heads, the MLP head converts the output into logits over the class labels and calculates the cross-entropy loss to find the most likely class.
+
+Ready to try your hand at image classification? Check out our complete [image classification guide](tasks/image_classification) to learn how to finetune ViT and use it for inference!
+
+<Tip>
+Notice the parallel between ViT and BERT: both use a special token (`[CLS]`) to capture the overall representation, both add position information to their embeddings, and both use a Transformer encoder to process the sequence of tokens/patches.
+</Tip>
+
+

chapter1/material/6.md

@@ -0,0 +1,219 @@
+<CourseFloatingBanner
+    chapter={1}
+    classNames="absolute z-10 right-0 top-0"
+/>
+
+# Transformer Architectures[[transformer-architectures]]
+
+In the previous sections, we introduced the general Transformer architecture and explored how these models can solve various tasks. Now, let's take a closer look at the three main architectural variants of Transformer models and understand when to use each one.
+
+<Tip>
+
+Remember that most Transformer models use one of three architectures: encoder-only, decoder-only, or encoder-decoder (sequence-to-sequence). Understanding these differences will help you choose the right model for your specific task.
+
+</Tip>
+
+## Encoder models[[encoder-models]]
+
+<Youtube id="MUqNwgPjJvQ" />
+
+Encoder models use only the encoder of a Transformer model. At each stage, the attention layers can access all the words in the initial sentence. These models are often characterized as having "bi-directional" attention, and are often called *auto-encoding models*.
+
+The pretraining of these models usually revolves around somehow corrupting a given sentence (for instance, by masking random words in it) and tasking the model with finding or reconstructing the initial sentence.
+
+Encoder models are best suited for tasks requiring an understanding of the full sentence, such as sentence classification, named entity recognition (and more generally word classification), and extractive question answering.
+
+<Tip>
+As we saw in [How 🤗 Transformers solve tasks](chapter1/4a), encoder models like BERT excel at understanding text because they can look at the entire context in both directions. This makes them perfect for tasks where comprehension of the whole input is important.
+</Tip>
+
+Representatives of this family of models include:
+
+- [BERT](https://huggingface.co/docs/transformers/model_doc/bert)
+- [DistilBERT](https://huggingface.co/docs/transformers/model_doc/distilbert)
+- [ModernBERT](https://huggingface.co/docs/transformers/en/model_doc/modernbert)
+
+## Decoder models[[decoder-models]]
+
+<Youtube id="d_ixlCubqQw" />
+
+Decoder models use only the decoder of a Transformer model. At each stage, for a given word the attention layers can only access the words positioned before it in the sentence. These models are often called *auto-regressive models*.
+
+The pretraining of decoder models usually revolves around predicting the next word in the sentence.
+
+These models are best suited for tasks involving text generation.
+
+<Tip>
+Decoder models like GPT are designed to generate text by predicting one token at a time. As we explored in [How 🤗 Transformers solve tasks](chapter1/4a), they can only see previous tokens, which makes them excellent for creative text generation but less ideal for tasks requiring bidirectional understanding.
+</Tip>
+
+Representatives of this family of models include:
+
+- [Hugging Face SmolLM Series](https://huggingface.co/HuggingFaceTB/SmolLM2-1.7B-Instruct)
+- [Meta's Llama Series](https://huggingface.co/docs/transformers/en/model_doc/llama4)
+- [Google's Gemma Series](https://huggingface.co/docs/transformers/main/en/model_doc/gemma3)
+- [DeepSeek's V3](https://huggingface.co/deepseek-ai/DeepSeek-V3)
+
+### Modern Large Language Models (LLMs)
+
+Most modern Large Language Models (LLMs) use the decoder-only architecture. These models have grown dramatically in size and capabilities over the past few years, with some of the largest models containing hundreds of billions of parameters.
+
+Modern LLMs are typically trained in two phases:
+1. **Pretraining**: The model learns to predict the next token on vast amounts of text data
+2. **Instruction tuning**: The model is fine-tuned to follow instructions and generate helpful responses
+
+This approach has led to models that can understand and generate human-like text across a wide range of topics and tasks.
+
+#### Key capabilities of modern LLMs
+
+Modern decoder-based LLMs have demonstrated impressive capabilities:
+
+| Capability | Description | Example |
+|------------|-------------|---------|
+| Text generation | Creating coherent and contextually relevant text | Writing essays, stories, or emails |
+| Summarization | Condensing long documents into shorter versions | Creating executive summaries of reports |
+| Translation | Converting text between languages | Translating English to Spanish |
+| Question answering | Providing answers to factual questions | "What is the capital of France?" |
+| Code generation | Writing or completing code snippets | Creating a function based on a description |
+| Reasoning | Working through problems step by step | Solving math problems or logical puzzles |
+| Few-shot learning | Learning from a few examples in the prompt | Classifying text after seeing just 2-3 examples |
+
+You can experiment with decoder-based LLMs directly in your browser. Here's an interactive demo of a large language model:
+
+<iframe
+	src="https://huggingface.co/spaces/course-demos/llm-demo"
+	frameborder="0"
+	width="100%"
+	height="450"
+></iframe>
+
+## Sequence-to-sequence models[[sequence-to-sequence-models]]
+
+<Youtube id="0_4KEb08xrE" />
+
+Encoder-decoder models (also called *sequence-to-sequence models*) use both parts of the Transformer architecture. At each stage, the attention layers of the encoder can access all the words in the initial sentence, whereas the attention layers of the decoder can only access the words positioned before a given word in the input.
+
+The pretraining of these models can take different forms, but it often involves reconstructing a sentence for which the input has been somehow corrupted (for instance by masking random words). The pretraining of the T5 model consists of replacing random spans of text (that can contain several words) with a single mask special token, and the task is then to predict the text that this mask token replaces.
+
+Sequence-to-sequence models are best suited for tasks revolving around generating new sentences depending on a given input, such as summarization, translation, or generative question answering.
+
+<Tip>
+As we saw in [How 🤗 Transformers solve tasks](chapter1/4a), encoder-decoder models like BART and T5 combine the strengths of both architectures. The encoder provides deep bidirectional understanding of the input, while the decoder generates appropriate output text. This makes them perfect for tasks that transform one sequence into another, like translation or summarization.
+</Tip>
+
+### Practical applications
+
+Sequence-to-sequence models excel at tasks that require transforming one form of text into another while preserving meaning. Some practical applications include:
+
+| Application | Description | Example Model |
+|-------------|-------------|---------------|
+| Machine translation | Converting text between languages | Marian, T5 |
+| Text summarization | Creating concise summaries of longer texts | BART, T5 |
+| Data-to-text generation | Converting structured data into natural language | T5 |
+| Grammar correction | Fixing grammatical errors in text | T5 |
+| Question answering | Generating answers based on context | BART, T5 |
+
+Here's an interactive demo of a sequence-to-sequence model for translation:
+
+<iframe
+	src="https://huggingface.co/spaces/course-demos/translation-demo"
+	frameborder="0"
+	width="100%"
+	height="450"
+></iframe>
+
+Representatives of this family of models include:
+
+- [BART](https://huggingface.co/docs/transformers/model_doc/bart)
+- [mBART](https://huggingface.co/docs/transformers/model_doc/mbart)
+- [Marian](https://huggingface.co/docs/transformers/model_doc/marian)
+- [T5](https://huggingface.co/docs/transformers/model_doc/t5)
+
+## Choosing the right architecture[[choosing-the-right-architecture]]
+
+When working on a specific NLP task, how do you decide which architecture to use? Here's a quick guide:
+
+| Task | Suggested Architecture | Examples |
+|------|------------------------|----------|
+| Text classification (sentiment, topic) | Encoder | BERT, RoBERTa |
+| Text generation (creative writing) | Decoder | GPT, LLaMA |
+| Translation | Encoder-Decoder | T5, BART |
+| Summarization | Encoder-Decoder | BART, T5 |
+| Named entity recognition | Encoder | BERT, RoBERTa |
+| Question answering (extractive) | Encoder | BERT, RoBERTa |
+| Question answering (generative) | Encoder-Decoder or Decoder | T5, GPT |
+| Conversational AI | Decoder | GPT, LLaMA |
+
+<Tip>
+When in doubt about which model to use, consider:
+1. What kind of understanding does your task need? (Bidirectional or unidirectional)
+2. Are you generating new text or analyzing existing text?
+3. Do you need to transform one sequence into another?
+
+The answers to these questions will guide you toward the right architecture.
+</Tip>
+
+## The evolution of LLMs
+
+Large Language Models have evolved rapidly in recent years, with each generation bringing significant improvements in capabilities. Here's an interactive timeline showing this evolution:
+
+<iframe
+	src="https://huggingface.co/spaces/course-demos/llm-timeline"
+	frameborder="0"
+	width="100%"
+	height="450"
+></iframe>
+
+## Attention mechanisms[[attention-mechanisms]]
+
+Most transformer models use full attention in the sense that the attention matrix is square. It can be a big
+computational bottleneck when you have long texts. Longformer and reformer are models that try to be more efficient and
+use a sparse version of the attention matrix to speed up training.
+
+<Tip>
+Standard attention mechanisms have a computational complexity of O(n²), where n is the sequence length. This becomes problematic for very long sequences. The specialized attention mechanisms below help address this limitation.
+</Tip>
+
+### LSH attention
+
+[Reformer](model_doc/reformer) uses LSH attention. In the softmax(QK^t), only the biggest elements (in the softmax
+dimension) of the matrix QK^t are going to give useful contributions. So for each query q in Q, we can consider only
+the keys k in K that are close to q. A hash function is used to determine if q and k are close. The attention mask is
+modified to mask the current token (except at the first position), because it will give a query and a key equal (so
+very similar to each other). Since the hash can be a bit random, several hash functions are used in practice
+(determined by a n_rounds parameter) and then are averaged together.
+
+### Local attention
+
+[Longformer](model_doc/longformer) uses local attention: often, the local context (e.g., what are the two tokens to the
+left and right?) is enough to take action for a given token. Also, by stacking attention layers that have a small
+window, the last layer will have a receptive field of more than just the tokens in the window, allowing them to build a
+representation of the whole sentence.
+
+Some preselected input tokens are also given global attention: for those few tokens, the attention matrix can access
+all tokens and this process is symmetric: all other tokens have access to those specific tokens (on top of the ones in
+their local window). This is shown in Figure 2d of the paper, see below for a sample attention mask:
+
+<div class="flex justify-center">
+    <img scale="50 %" align="center" src="https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/local_attention_mask.png"/>
+</div>
+
+Using those attention matrices with less parameters then allows the model to have inputs having a bigger sequence
+length.
+
+### Axial positional encodings
+
+[Reformer](model_doc/reformer) uses axial positional encodings: in traditional transformer models, the positional encoding
+E is a matrix of size \\(l\\) by \\(d\\), \\(l\\) being the sequence length and \\(d\\) the dimension of the
+hidden state. If you have very long texts, this matrix can be huge and take way too much space on the GPU. To alleviate
+that, axial positional encodings consist of factorizing that big matrix E in two smaller matrices E1 and E2, with
+dimensions \\(l_{1} \times d_{1}\\) and \\(l_{2} \times d_{2}\\), such that \\(l_{1} \times l_{2} = l\\) and
+\\(d_{1} + d_{2} = d\\) (with the product for the lengths, this ends up being way smaller). The embedding for time
+step \\(j\\) in E is obtained by concatenating the embeddings for timestep \\(j \% l1\\) in E1 and \\(j // l1\\)
+in E2.
+
+## Conclusion[[conclusion]]
+
+In this section, we've explored the three main Transformer architectures and some specialized attention mechanisms. Understanding these architectural differences is crucial for selecting the right model for your specific NLP task.
+
+As we move forward in the course, you'll get hands-on experience with these different architectures and learn how to fine-tune them for your specific needs. In the next section, we'll look at some of the limitations and biases present in these models that you should be aware of when deploying them.

chapter1/material/7.md

@@ -0,0 +1,241 @@
+# Inference with LLMs[[inference-with-llms]]
+
+<CourseFloatingBanner
+    chapter={1}
+    classNames="absolute z-10 right-0 top-0"
+/>
+
+In this page, we'll explore the core concepts behind LLM inference, providing a comprehensive understanding of how these models generate text and the key components involved in the inference process.
+
+## Understanding the Basics
+
+Let's start with the fundamentals. Inference is the process of using a trained LLM to generate human-like text from a given input prompt. Language models use their knowledge from training to formulate responses one word at a time. The model leverages learned probabilities from billions of parameters to predict and generate the next token in a sequence. This sequential generation is what allows LLMs to produce coherent and contextually relevant text. 
+
+## The Role of Attention
+
+The attention mechanism is what gives LLMs their ability to understand context and generate coherent responses. When predicting the next word, not every word in a sentence carries equal weight - for example, in the sentence *"The capital of France is ..."*, the words "France" and "capital" are crucial for determining that "Paris" should come next. This ability to focus on relevant information is what we call attention.
+
+<img src="https://huggingface.co/datasets/agents-course/course-images/resolve/main/en/unit1/AttentionSceneFinal.gif" alt="Visual Gif of Attention" width="60%">
+
+This process of identifying the most relevant words to predict the next token has proven to be incredibly effective. Although the basic principle of training LLMs—predicting the next token—has remained generally consistent since BERT and GPT-2, there have been significant advancements in scaling neural networks and making the attention mechanism work for longer and longer sequences, at lower and lower costs.
+
+<Tip>
+
+In short, the attention mechanism is the key to LLMs being able to generate text that is both coherent and context-aware. It sets modern LLMs apart from previous generations of language models.
+
+</Tip>
+
+### Context Length and Attention Span
+
+Now that we understand attention, let's explore how much context an LLM can actually handle. This brings us to context length, or the model's 'attention span'.
+
+The context length refers to the maximum number of tokens (words or parts of words) that the LLM can process at once. Think of it as the size of the model's working memory.
+
+These capabilities are limited by several practical factors:
+- The model's architecture and size
+- Available computational resources
+- The complexity of the input and desired output
+
+In an ideal world, we could feed unlimited context to the model, but hardware constraints and computational costs make this impractical. This is why different models are designed with different context lengths to balance capability with efficiency.
+
+<Tip>
+
+The context length is the maximum number of tokens the model can consider at once when generating a response.
+
+</Tip>
+
+### The Art of Prompting
+
+When we pass information to LLMs, we structure our input in a way that guides the generation of the LLM toward the desired output. This is called _prompting_.
+
+Understanding how LLMs process information helps us craft better prompts. Since the model's primary task is to predict the next token by analyzing the importance of each input token, the wording of your input sequence becomes crucial.
+
+<Tip>
+
+Careful design of the prompt makes it easier **to guide the generation of the LLM toward the desired output**.
+
+</Tip>
+
+## The Two-Phase Inference Process
+
+Now that we understand the basic components, let's dive into how LLMs actually generate text. The process can be broken down into two main phases: prefill and decode. These phases work together like an assembly line, each playing a crucial role in producing coherent text.
+
+### The Prefill Phase
+
+The prefill phase is like the preparation stage in cooking - it's where all the initial ingredients are processed and made ready. This phase involves three key steps:
+
+1. **Tokenization**: Converting the input text into tokens (think of these as the basic building blocks the model understands)
+2. **Embedding Conversion**: Transforming these tokens into numerical representations that capture their meaning
+3. **Initial Processing**: Running these embeddings through the model's neural networks to create a rich understanding of the context
+
+This phase is computationally intensive because it needs to process all input tokens at once. Think of it as reading and understanding an entire paragraph before starting to write a response.
+
+### The Decode Phase
+
+After the prefill phase has processed the input, we move to the decode phase - this is where the actual text generation happens. The model generates one token at a time in what we call an autoregressive process (where each new token depends on all previous tokens).
+
+The decode phase involves several key steps that happen for each new token:
+1. **Attention Computation**: Looking back at all previous tokens to understand context
+2. **Probability Calculation**: Determining the likelihood of each possible next token
+3. **Token Selection**: Choosing the next token based on these probabilities
+4. **Continuation Check**: Deciding whether to continue or stop generation
+
+This phase is memory-intensive because the model needs to keep track of all previously generated tokens and their relationships. 
+
+## Sampling Strategies
+
+Now that we understand how the model generates text, let's explore the various ways we can control this generation process. Just like a writer might choose between being more creative or more precise, we can adjust how the model makes its token selections.
+
+### Understanding Token Selection: From Probabilities to Token Choices
+
+When the model needs to choose the next token, it starts with raw probabilities (called logits) for every word in its vocabulary. But how do we turn these probabilities into actual choices? Let's break down the process:
+
+<img src="https://huggingface.co/reasoning-course/images/blob/main/inference/1.png" alt="Token Selection Process" />
+
+1. **Raw Logits**: Think of these as the model's initial gut feelings about each possible next word
+2. **Temperature Control**: Like a creativity dial - higher settings (>1.0) make choices more random and creative, lower settings (<1.0) make them more focused and deterministic
+3. **Top-p (Nucleus) Sampling**: Instead of considering all possible words, we only look at the most likely ones that add up to our chosen probability threshold (e.g., top 90%)
+4. **Top-k Filtering**: An alternative approach where we only consider the k most likely next words
+
+### Managing Repetition: Keeping Output Fresh
+
+One common challenge with LLMs is their tendency to repeat themselves - much like a speaker who keeps returning to the same points. To address this, we use two types of penalties:
+
+1. **Presence Penalty**: A fixed penalty applied to any token that has appeared before, regardless of how often. This helps prevent the model from reusing the same words.
+2. **Frequency Penalty**: A scaling penalty that increases based on how often a token has been used. The more a word appears, the less likely it is to be chosen again.
+
+<img src="https://huggingface.co/reasoning-course/images/blob/main/inference/2.png" alt="Token Selection Process" />
+
+These penalties are applied early in the token selection process, adjusting the raw probabilities before other sampling strategies are applied. Think of them as gentle nudges encouraging the model to explore new vocabulary.
+
+### Controlling Generation Length: Setting Boundaries
+
+Just as a good story needs proper pacing and length, we need ways to control how much text our LLM generates. This is crucial for practical applications - whether we're generating a tweet-length response or a full blog post.
+
+We can control generation length in several ways:
+1. **Token Limits**: Setting minimum and maximum token counts
+2. **Stop Sequences**: Defining specific patterns that signal the end of generation
+3. **End-of-Sequence Detection**: Letting the model naturally conclude its response
+
+For example, if we want to generate a single paragraph, we might set a maximum of 100 tokens and use "\n\n" as a stop sequence. This ensures our output stays focused and appropriately sized for its purpose.
+
+<img src="https://huggingface.co/reasoning-course/images/blob/main/inference/3.png" alt="Token Selection Process" />
+
+### Beam Search: Looking Ahead for Better Coherence
+
+While the strategies we've discussed so far make decisions one token at a time, beam search takes a more holistic approach. Instead of committing to a single choice at each step, it explores multiple possible paths simultaneously - like a chess player thinking several moves ahead.
+
+<img src="https://huggingface.co/reasoning-course/images/blob/main/inference/4.png" alt="Beam Search" />
+
+Here's how it works:
+1. At each step, maintain multiple candidate sequences (typically 5-10)
+2. For each candidate, compute probabilities for the next token
+3. Keep only the most promising combinations of sequences and next tokens
+4. Continue this process until reaching the desired length or stop condition
+5. Select the sequence with the highest overall probability
+
+This approach often produces more coherent and grammatically correct text, though it requires more computational resources than simpler methods.
+
+## Practical Challenges and Optimization
+
+As we wrap up our exploration of LLM inference, let's look at the practical challenges you'll face when deploying these models, and how to measure and optimize their performance.
+
+### Key Performance Metrics
+
+When working with LLMs, four critical metrics will shape your implementation decisions:
+
+1. **Time to First Token (TTFT)**: How quickly can you get the first response? This is crucial for user experience and is primarily affected by the prefill phase.
+2. **Time Per Output Token (TPOT)**: How fast can you generate subsequent tokens? This determines the overall generation speed.
+3. **Throughput**: How many requests can you handle simultaneously? This affects scaling and cost efficiency.
+4. **VRAM Usage**: How much GPU memory do you need? This often becomes the primary constraint in real-world applications.
+
+### The Context Length Challenge
+
+One of the most significant challenges in LLM inference is managing context length effectively. Longer contexts provide more information but come with substantial costs:
+
+- **Memory Usage**: Grows quadratically with context length
+- **Processing Speed**: Decreases linearly with longer contexts
+- **Resource Allocation**: Requires careful balancing of VRAM usage
+
+Recent models like Qwen2.5-1M offer impressive 1M token context windows, but this comes at the cost of significantly slower inference times. The key is finding the right balance for your specific use case.
+
+
+<div style="max-width: 800px; margin: 20px auto; padding: 20px; 
+font-family: system-ui;">
+    <div style="border: 2px solid #ddd; border-radius: 8px; 
+    padding: 20px; margin-bottom: 20px;">
+        <div style="display: flex; align-items: center; 
+        margin-bottom: 15px;">
+            <div style="flex: 1; text-align: center; padding: 
+            10px; background: #f0f0f0; border-radius: 4px;">
+                Input Text (Raw)
+            </div>
+            <div style="margin: 0 10px;">→</div>
+            <div style="flex: 1; text-align: center; padding: 
+            10px; background: #e1f5fe; border-radius: 4px;">
+                Tokenized Input
+            </div>
+        </div>
+        <div style="display: flex; margin-bottom: 15px;">
+            <div style="flex: 1; border: 1px solid #ccc; 
+            padding: 10px; margin: 5px; background: #e8f5e9; 
+            border-radius: 4px; text-align: center;">
+                Context Window<br/>(e.g., 4K tokens)
+                <div style="display: flex; margin-top: 10px;">
+                    <div style="flex: 1; background: #81c784; 
+                    margin: 2px; height: 20px; border-radius: 
+                    2px;"></div>
+                    <div style="flex: 1; background: #81c784; 
+                    margin: 2px; height: 20px; border-radius: 
+                    2px;"></div>
+                    <div style="flex: 1; background: #81c784; 
+                    margin: 2px; height: 20px; border-radius: 
+                    2px;"></div>
+                    <div style="flex: 1; background: #81c784; 
+                    margin: 2px; height: 20px; border-radius: 
+                    2px;"></div>
+                </div>
+            </div>
+        </div>
+        <div style="display: flex; justify-content: 
+        space-between; text-align: center; font-size: 0.9em; 
+        color: #666;">
+            <div style="flex: 1;">
+                <div style="border: 1px solid #ffcc80; padding: 
+                8px; margin: 5px; background: #fff3e0; 
+                border-radius: 4px;">
+                    Memory Usage<br/>∝ Length²
+                </div>
+            </div>
+            <div style="flex: 1;">
+                <div style="border: 1px solid #90caf9; padding: 
+                8px; margin: 5px; background: #e3f2fd; 
+                border-radius: 4px;">
+                    Processing Time<br/>∝ Length
+                </div>
+            </div>
+        </div>
+    </div>
+</div>
+
+### The KV Cache Optimization
+
+To address these challenges, one of the most powerful optimizations is KV (Key-Value) caching. This technique significantly improves inference speed by storing and reusing intermediate calculations. This optimization:
+- Reduces repeated calculations
+- Improves generation speed
+- Makes long-context generation practical
+
+The trade-off is additional memory usage, but the performance benefits usually far outweigh this cost.
+
+## Conclusion
+
+Understanding LLM inference is crucial for effectively deploying and optimizing these powerful models. We've covered the key components:
+
+- The fundamental role of attention and context
+- The two-phase inference process
+- Various sampling strategies for controlling generation
+- Practical challenges and optimizations
+
+By mastering these concepts, you'll be better equipped to build applications that leverage LLMs effectively and efficiently.
+
+Remember that the field of LLM inference is rapidly evolving, with new techniques and optimizations emerging regularly. Stay curious and keep experimenting with different approaches to find what works best for your specific use cases.

chapter1/material/8.md

@@ -0,0 +1,32 @@
+# Bias and limitations[[bias-and-limitations]]
+
+<CourseFloatingBanner chapter={1}
+  classNames="absolute z-10 right-0 top-0"
+  notebooks={[
+    {label: "Google Colab", value: "https://colab.research.google.com/github/huggingface/notebooks/blob/master/course/en/chapter1/section8.ipynb"},
+    {label: "Aws Studio", value: "https://studiolab.sagemaker.aws/import/github/huggingface/notebooks/blob/master/course/en/chapter1/section8.ipynb"},
+]} />
+
+If your intent is to use a pretrained model or a fine-tuned version in production, please be aware that, while these models are powerful tools, they come with limitations. The biggest of these is that, to enable pretraining on large amounts of data, researchers often scrape all the content they can find, taking the best as well as the worst of what is available on the internet. 
+
+To give a quick illustration, let's go back the example of a `fill-mask` pipeline with the BERT model:
+
+```python
+from transformers import pipeline
+
+unmasker = pipeline("fill-mask", model="bert-base-uncased")
+result = unmasker("This man works as a [MASK].")
+print([r["token_str"] for r in result])
+
+result = unmasker("This woman works as a [MASK].")
+print([r["token_str"] for r in result])
+```
+
+```python out
+['lawyer', 'carpenter', 'doctor', 'waiter', 'mechanic']
+['nurse', 'waitress', 'teacher', 'maid', 'prostitute']
+```
+
+When asked to fill in the missing word in these two sentences, the model gives only one gender-free answer (waiter/waitress). The others are work occupations usually associated with one specific gender -- and yes, prostitute ended up in the top 5 possibilities the model associates with "woman" and "work." This happens even though BERT is one of the rare Transformer models not built by scraping data from all over the internet, but rather using apparently neutral data (it's trained on the [English Wikipedia](https://huggingface.co/datasets/wikipedia) and [BookCorpus](https://huggingface.co/datasets/bookcorpus) datasets). 
+
+When you use these tools, you therefore need to keep in the back of your mind that the original model you are using could very easily generate sexist, racist, or homophobic content. Fine-tuning the model on your data won't make this intrinsic bias disappear.

chapter1/material/9.md

@@ -0,0 +1,66 @@
+# Summary[[summary]]
+
+<CourseFloatingBanner
+    chapter={1}
+    classNames="absolute z-10 right-0 top-0"
+/>
+
+In this chapter, you've been introduced to the fundamentals of Transformer models, Large Language Models (LLMs), and how they're revolutionizing AI and beyond.
+
+## Key concepts covered
+
+### Natural Language Processing and LLMs
+
+We explored what NLP is and how Large Language Models have transformed the field. You learned that:
+- NLP encompasses a wide range of tasks from classification to generation
+- LLMs are powerful models trained on massive amounts of text data
+- These models can perform multiple tasks within a single architecture
+- Despite their capabilities, LLMs have limitations including hallucinations and bias
+
+### Transformer capabilities
+
+You saw how the `pipeline()` function from 🤗 Transformers makes it easy to use pre-trained models for various tasks:
+- Text classification, token classification, and question answering
+- Text generation and summarization
+- Translation and other sequence-to-sequence tasks
+- Speech recognition and image classification
+
+### Transformer architecture
+
+We discussed how Transformer models work at a high level, including:
+- The importance of the attention mechanism
+- How transfer learning enables models to adapt to specific tasks
+- The three main architectural variants: encoder-only, decoder-only, and encoder-decoder
+
+### Model architectures and their applications
+A key aspect of this chapter was understanding which architecture to use for different tasks:
+
+| Model           | Examples                                   | Tasks                                                                            |
+|-----------------|--------------------------------------------|----------------------------------------------------------------------------------|
+| Encoder-only    | BERT, DistilBERT, ModernBERT               | Sentence classification, named entity recognition, extractive question answering |
+| Decoder-only    | GPT, LLaMA, Gemma, SmolLM                  | Text generation, conversational AI, creative writing                             |
+| Encoder-decoder | BART, T5, Marian, mBART                    | Summarization, translation, generative question answering                        |
+
+### Modern LLM developments
+You also learned about recent developments in the field:
+- How LLMs have grown in size and capability over time
+- The concept of scaling laws and how they guide model development
+- Specialized attention mechanisms that help models process longer sequences
+- The two-phase training approach of pretraining and instruction tuning
+
+### Practical applications
+Throughout the chapter, you've seen how these models can be applied to real-world problems:
+- Using the Hugging Face Hub to find and use pre-trained models
+- Leveraging the Inference API to test models directly in your browser
+- Understanding which models are best suited for specific tasks
+
+## Looking ahead
+
+Now that you have a solid understanding of what Transformer models are and how they work at a high level, you're ready to dive deeper into how to use them effectively. In the next chapters, you'll learn how to:
+
+- Use the Transformers library to load and fine-tune models
+- Process different types of data for model input
+- Adapt pre-trained models to your specific tasks
+- Deploy models for practical applications
+
+The foundation you've built in this chapter will serve you well as you explore more advanced topics and techniques in the coming sections.

chapter1/presentation.md

@@ -0,0 +1,518 @@
+class: impact
+
+# The LLM Course
+## Chapter 1 
+## Introduction to NLP, LLMs & Transformers
+
+.center[![Hugging Face Logo](https://huggingface.co/front/assets/huggingface_logo.svg)]
+
+???
+Welcome to the first chapter of the LLM Course! In this session, we'll dive into the foundational concepts of Natural Language Processing (NLP), explore the capabilities of Large Language Models (LLMs), and understand the revolutionary Transformer architecture that underpins many of these powerful tools.
+
+---
+
+# Introduction to NLP, LLMs & Transformers
+
+- Understanding the Basics
+- Exploring Capabilities
+- How They Work
+
+???
+Today, we'll cover three main points: understanding the basics of these technologies, exploring their impressive capabilities, and getting an initial look at their inner workings.
+
+---
+
+# What is Natural Language Processing (NLP)?
+
+**Definition**: Field blending linguistics and machine learning to enable computers to understand/process human language.
+
+**Common Tasks**:
+- Classifying sentences (sentiment, spam)
+- Classifying words (POS tagging, NER)
+- Generating text (autocomplete, translation, summarization)
+
+???
+So, what exactly is Natural Language Processing? It's an interdisciplinary field combining linguistics and machine learning, aiming to equip computers with the ability to understand, interpret, and process human language effectively. Common NLP tasks include classifying sentences (like sentiment analysis or spam detection), classifying individual words (such as Part-of-Speech (POS) tagging or Named Entity Recognition (NER) to identify entities like people, locations, and organizations), and generating text (for applications like autocomplete, machine translation, or document summarization).
+
+---
+
+# The Rise of Large Language Models (LLMs)
+
+**Shift**: From task-specific models to versatile, large-scale models.
+
+**Definition**: Massive models trained on vast text data.
+
+**Key Characteristics**:
+- Versatility: Handle multiple NLP tasks.
+- Power: Learn intricate language patterns.
+- Emergent Abilities: Show unexpected capabilities at scale.
+
+**Examples**: GPT, Llama, BERT
+
+???
+Historically, NLP often involved creating specialized models for each specific task. However, the landscape shifted dramatically with the emergence of LLMs. These are exceptionally large neural networks trained on enormous datasets of text and code. Key characteristics distinguish LLMs: they exhibit remarkable *versatility*, capable of handling diverse NLP tasks within one framework; they possess immense *power*, learning complex language patterns that lead to state-of-the-art performance; and they often display *emergent abilities* – unexpected capabilities that arise as model size increases. Prominent examples include models like the Generative Pre-trained Transformer (GPT) series, Llama, and Bidirectional Encoder Representations from Transformers (BERT). These represent a paradigm shift towards leveraging single, powerful, adaptable models for a multitude of language challenges.
+
+---
+
+# The Growth of Transformer Models
+
+.center[![Model Parameters Growth](https://huggingface.co/datasets/huggingface-course/documentation-images/resolve/main/en/chapter1/model_parameters.png)]
+
+???
+This chart illustrates the exponential growth in the size of Transformer models, measured by their number of parameters, over recent years. While there are exceptions like DistilBERT (intentionally designed to be smaller and faster), the dominant trend has been towards larger models to achieve higher performance. This scaling has unlocked significant capabilities but also brings challenges related to computational demands and environmental footprint. The clear trajectory from millions to billions, and even trillions, of parameters highlights how model scale has been a primary driver of recent advancements in NLP.
+
+---
+
+# Environmental Impact of LLMs
+
+.center[![Carbon Footprint](https://huggingface.co/datasets/huggingface-course/documentation-images/resolve/main/en/chapter1/carbon_footprint.svg)]
+
+???
+Training these massive LLMs comes with a substantial environmental cost, as depicted in this chart showing the carbon footprint associated with training a large model. A single training run can have a carbon footprint comparable to the lifetime emissions of several cars. This underscores the critical importance of sharing pretrained models. Platforms like the Hugging Face Hub facilitate this sharing, allowing the community to build upon existing models instead of repeatedly training from scratch. This collaborative approach significantly reduces the collective computational burden and environmental impact, distributing the cost across numerous users and applications.
+
+---
+
+# Why is Language Processing Hard?
+
+**Human Understanding**: Context, nuance, sarcasm are easy for humans, hard for machines.
+
+**Machine Needs**: Text requires processing into a machine-understandable format.
+
+**LLM Challenges**: Despite advances, LLMs struggle with full complexity and can inherit biases from training data.
+
+???
+Why does processing human language pose such a challenge for computers? Humans effortlessly grasp context, nuance, implied meanings, and even sarcasm. Machines, however, require language to be converted into a structured, numerical format they can process. While LLMs represent a huge leap forward, they still grapple with the full richness and complexity of human communication. We rely on shared knowledge, cultural context, and subtle cues that are difficult to encode explicitly. Even today's most advanced models can struggle with these aspects and may also inherit biases from the vast datasets they are trained on.
+
+---
+
+# The Hugging Face Ecosystem
+
+.center[![Hugging Face Hub](https://huggingface.co/datasets/huggingface-course/documentation-images/resolve/main/en/chapter1/companies.PNG)]
+
+- Thousands of pretrained models
+- Datasets for various tasks
+- Spaces for demos
+- Community collaboration
+- Used by major companies and organizations
+  
+???
+The Hugging Face ecosystem plays a pivotal role in the modern NLP and AI landscape, serving as a central hub for collaboration and resource sharing. Many leading companies and research organizations utilize and contribute to this platform. Key components include the Model Hub, hosting thousands of freely downloadable pretrained models; extensive Datasets for training and evaluation; Spaces for hosting interactive demos; and libraries like 🤗 Transformers. This open and collaborative environment has significantly accelerated progress by making state-of-the-art AI accessible to a global community.
+
+---
+
+# Transformers: What Can They Do?
+
+**Architecture**: Introduced in 2017, powers most modern LLMs.
+
+**Accessibility**: Libraries like Hugging Face 🤗 Transformers make them easy to use.
+
+**Tool Highlight**: The pipeline() function.
+
+.center[![Hugging Face logo](https://huggingface.co/front/assets/huggingface_logo.svg)]
+
+???
+Let's focus on the Transformer architecture. Introduced in the seminal 2017 paper 'Attention Is All You Need', this architecture forms the backbone of most contemporary LLMs. Libraries like Hugging Face 🤗 Transformers simplify the process of using these powerful models. Before Transformers, architectures like Long Short-Term Memory networks (LSTMs) were common for sequence tasks, but they struggled with long-range dependencies in text and were less efficient to train due to their sequential nature. Transformers addressed these limitations, particularly through the innovative 'attention mechanism', enabling better handling of long sequences and more parallelizable, efficient training on massive datasets.
+
+---
+
+# The pipeline() Function
+
+**Simplicity**: Perform complex tasks with minimal code.
+
+**Example**: Sentiment Analysis
+
+```python
+from transformers import pipeline
+classifier = pipeline('sentiment-analysis')
+classifier('This is a great course!')
+```
+
+**Output**:
+```
+[{'label': 'POSITIVE', 'score': 0.9998}]
+```
+
+???
+A great entry point into using 🤗 Transformers is the `pipeline()` function. It provides a high-level abstraction, allowing you to perform complex NLP tasks with minimal code. As shown in the example, sentiment analysis takes just three lines. The `pipeline()` handles the necessary steps behind the scenes: preprocessing the input text (tokenization), feeding it to the model for inference, and post-processing the model's output into a human-readable format. This ease of use significantly lowers the barrier to entry and has been instrumental in the broad adoption of Transformer models.
+
+---
+
+# pipeline() Task Examples (Text)
+
+- **Sentiment Analysis**: Determine if text is positive/negative
+- **Named Entity Recognition**: Identify people, organizations, locations
+- **Question Answering**: Extract answers from context
+- **Text Generation**: Complete prompts with new text
+- **Summarization**: Condense long documents
+- **Translation**: Convert text between languages
+
+???
+The `pipeline()` function is incredibly versatile, supporting numerous NLP tasks out-of-the-box. For text-based applications, this includes: Sentiment Analysis (determining positive/negative tone), Named Entity Recognition (identifying entities like people, places, organizations), Question Answering (extracting answers from context), Text Generation (completing prompts or creating new text), Summarization (condensing documents), and Translation (converting between languages). Each task typically leverages a model fine-tuned specifically for that purpose, all accessible through the consistent `pipeline()` interface.
+
+---
+
+# pipeline() Task Examples (Beyond Text)
+
+**Image Classification**:
+```python
+from transformers import pipeline
+
+img_classifier = pipeline('image-classification')
+img_classifier('path/to/your/image.jpg')
+```
+
+**Automatic Speech Recognition**:
+
+```python
+from transformers import pipeline
+
+transcriber = pipeline('automatic-speech-recognition')
+transcriber('path/to/your/audio.flac')
+```
+
+???
+The power of Transformers extends beyond text. The `pipeline()` function also supports tasks in other modalities, demonstrating the architecture's flexibility. Examples include Image Classification (identifying objects in images) and Automatic Speech Recognition (ASR - transcribing spoken audio to text). This ability to handle multimodal data stems from adapting the core Transformer concepts, like the attention mechanism, to process sequences derived from images (e.g., patches) or audio (e.g., frames).
+
+---
+
+# How Do Transformers Work?
+
+.center[![Transformer Architecture](https://huggingface.co/datasets/huggingface-course/documentation-images/resolve/main/en/chapter1/transformers.svg)]
+
+???
+At its core, the original Transformer architecture, as shown in this diagram from the 'Attention Is All You Need' paper, comprises two main blocks: an Encoder (left side) and a Decoder (right side). The Encoder's role is to process the input sequence and build a rich representation of it. The Decoder then uses this representation, along with the sequence generated so far, to produce the output sequence. The groundbreaking element connecting these is the *attention mechanism*, which enables the model to dynamically weigh the importance of different parts of the input sequence when generating each part of the output.
+
+---
+
+# The Attention Mechanism
+
+**Core Idea**: Allow the model to focus on relevant parts of the input when processing each word.
+
+**Example**: In translating "You like this course" to French:
+- When translating "like", attention focuses on "You" (for conjugation)
+- When translating "this", attention focuses on "course" (for gender agreement)
+
+**Key Innovation**: Enables modeling of long-range dependencies in text.
+
+???
+The *attention mechanism* is arguably the most crucial innovation of the Transformer architecture. It empowers the model to selectively focus on the most relevant parts of the input sequence when processing or generating each element (like a word). Consider translating 'You like this course' to French: To correctly conjugate 'like' (aimer), the model must attend to 'You'. To choose the correct form of 'this' (ce/cet/cette), it needs to attend to 'course' to determine its gender. This dynamic weighting of input elements allows Transformers to effectively model long-range dependencies and contextual relationships in data, a significant advantage over earlier sequential models.
+
+---
+
+# Transformer Architectures: Overview
+
+Most models fall into three main categories:
+
+1. **Encoder-only**: Best for understanding tasks (BERT)
+2. **Decoder-only**: Best for generation tasks (GPT)
+3. **Encoder-Decoder**: Best for transformation tasks (T5)
+
+???
+While based on the original design, most modern Transformer models utilize one of three main architectural patterns: 1. **Encoder-only**: Uses just the encoder stack. Excels at tasks requiring a deep understanding of the entire input sequence (e.g., BERT). 2. **Decoder-only**: Uses just the decoder stack. Ideal for generative tasks where the output is produced sequentially based on previous context (e.g., GPT). 3. **Encoder-Decoder**: Uses both stacks. Suited for sequence-to-sequence tasks that transform an input sequence into a new output sequence (e.g., T5, BART). Each architecture is optimized for different kinds of problems, so understanding their characteristics is key to selecting the appropriate model.
+
+---
+
+# Architecture 1: Encoder-only
+
+**Examples**: BERT, RoBERTa
+
+**How it works**: Uses only the encoder part. Attention layers access the entire input sentence (bi-directional).
+
+**Best for**: Tasks requiring full sentence understanding.
+- Sentence Classification
+- Named Entity Recognition (NER)
+- Extractive Question Answering
+
+.center[![BERT Architecture](https://huggingface.co/datasets/huggingface-course/documentation-images/resolve/main/en/chapter1/encoder_decoder/encoder.svg)]
+
+???
+Encoder-only models, exemplified by BERT and RoBERTa, leverage solely the encoder component of the Transformer. A key characteristic is their use of *bi-directional attention*, meaning that when processing any given word, the attention layers can access information from *all* other words in the input sentence, both preceding and succeeding. This allows the model to build a deep contextual understanding of the entire input. Consequently, these models excel at tasks demanding comprehensive input analysis, such as sentence classification, Named Entity Recognition (NER), and extractive question answering (where the answer is a span within the input text). They are generally not used for free-form text generation.
+
+---
+
+# Architecture 2: Decoder-only
+
+**Examples**: GPT, Llama
+
+**How it works**: Uses only the decoder part. Predicts the next word based on preceding words (causal/auto-regressive attention).
+
+**Best for**: Text generation tasks.
+- Auto-completion
+- Creative Writing
+- Chatbots
+
+.center[![GPT Architecture](https://huggingface.co/datasets/huggingface-course/documentation-images/resolve/main/en/chapter1/encoder_decoder/decoder.svg)]
+
+???
+Decoder-only models, such as those in the GPT family and Llama, utilize only the decoder stack. They operate differently from encoders: attention is *causal* or *auto-regressive*. This means that when predicting the next word (or token), the model can only attend to the words that came *before* it in the sequence, plus the current position. This inherent structure makes them exceptionally well-suited for text generation tasks, where the goal is to predict subsequent tokens based on preceding context. Applications include auto-completion, creative writing assistants, and chatbots. Indeed, many of the most prominent modern LLMs fall into this category.
+
+---
+
+# Architecture 3: Encoder-Decoder
+
+**Examples**: BART, T5, MarianMT
+
+**How it works**: Uses both encoder (processes input) and decoder (generates output). Attention in the decoder can access encoder outputs.
+
+**Best for**: Sequence-to-sequence tasks (transforming input to output).
+- Translation
+- Summarization
+- Generative Question Answering
+
+.center[![Encoder-Decoder Architecture](https://huggingface.co/datasets/huggingface-course/documentation-images/resolve/main/en/chapter1/encoder_decoder/encoder_decoder.svg)]
+
+???
+Encoder-Decoder models, also known as sequence-to-sequence models (e.g., BART, Text-to-Text Transfer Transformer (T5), MarianMT for translation), employ the complete original Transformer structure. The encoder first processes the entire input sequence to create a comprehensive representation. The decoder then generates the output sequence token by token, utilizing both its own previously generated tokens (causal attention) and the encoder's output representation (cross-attention). This architecture excels at tasks that involve transforming an input sequence into a different output sequence, such as machine translation, text summarization, or generative question answering (where the answer is generated, not just extracted). They effectively combine the input understanding strengths of encoders with the text generation capabilities of decoders.
+
+---
+
+# Model Architectures and Tasks
+
+| Architecture | Examples | Best For |
+|--------------|----------|----------|
+| **Encoder-only** | BERT, RoBERTa | Classification, NER, Q&A |
+| **Decoder-only** | GPT, LLaMA | Text generation, Chatbots |
+| **Encoder-Decoder** | BART, T5 | Translation, Summarization |
+
+???
+This table provides a concise summary of the three primary Transformer architectures and their typical applications. To reiterate: Encoder-only models (like BERT) are strong for tasks requiring deep input understanding (classification, NER). Decoder-only models (like GPT, Llama) excel at generating text sequentially (chatbots, creative writing). Encoder-Decoder models (like BART, T5) are ideal for transforming input sequences into output sequences (translation, summarization). Selecting the most appropriate architecture is a critical initial decision in designing an effective NLP solution, as each is optimized for different strengths.
+
+---
+
+# Causal Language Modeling
+
+.col-6[
+  **Key Characteristics:**
+  - Used by decoder models (GPT family)
+  - Predicts next word based on previous words
+  - Unidirectional attention (left-to-right)
+  - Well-suited for text generation
+  - Examples: GPT, LLaMA, Claude
+]
+.col-6[
+  .center[![Causal Language Modeling](https://huggingface.co/datasets/huggingface-course/documentation-images/resolve/main/en/chapter1/causal_modeling.svg)]
+]
+
+???
+Causal Language Modeling (CLM) is the training objective typically associated with decoder-only models (like the GPT family, Llama, Claude). The core task is to predict the *next* token in a sequence, given all the preceding tokens. This inherently relies on *unidirectional* or *causal* attention – the model can only 'look back' at previous tokens and the current position. The term 'causal' highlights that the prediction at any point depends only on past information, mirroring the natural flow of language generation. This auto-regressive process is fundamental to how these models generate coherent text, one token after another.
+
+---
+
+# Masked Language Modeling
+
+.col-6[
+  **Key Characteristics:**
+  - Used by encoder models (BERT family)
+  - Masks random words and predicts them
+  - Bidirectional attention (full context)
+  - Well-suited for understanding tasks
+  - Examples: BERT, RoBERTa, DeBERTa
+]
+.col-6[
+  .center[![Masked Language Modeling](https://huggingface.co/datasets/huggingface-course/documentation-images/resolve/main/en/chapter1/masked_modeling.svg)]
+]
+
+???
+Masked Language Modeling (MLM) is the pretraining strategy characteristic of encoder-only models (like BERT, RoBERTa, DeBERTa). During training, a certain percentage of input tokens are randomly replaced with a special `[MASK]` token. The model's objective is then to predict the *original* identity of these masked tokens, using the surrounding *unmasked* context. This task necessitates *bidirectional* attention, as the model needs to consider context from both the left and the right of the mask to make an accurate prediction. MLM forces the model to develop a deep understanding of word meanings in context, making it highly effective for downstream tasks requiring rich text representations, such as classification or NER.
+
+---
+
+# Transfer Learning in NLP
+
+**The key to efficient NLP:**
+- Train once on general data
+- Adapt to specific tasks
+- Reduces computational costs
+- Democratizes access to advanced AI
+
+???
+Transfer learning is a cornerstone technique in modern NLP. The core idea is to leverage the knowledge captured by a model trained on a massive, general dataset (pretraining) and adapt it to a new, specific task (fine-tuning). This approach significantly reduces the need for task-specific data and computational resources compared to training from scratch. It has democratized access to powerful AI, enabling smaller teams and individuals to build sophisticated applications by standing on the shoulders of large pretrained models. Let's examine the two key phases: pretraining and fine-tuning.
+
+---
+
+# Pretraining Phase
+
+.col-6[
+  **Characteristics:**
+  - Train on massive datasets (billions of words)
+  - Learn general language patterns and representations
+  - Computationally expensive (can cost millions)
+  - Usually done once by research labs or companies
+  - Foundation for many downstream tasks
+]
+.col-6[
+  .center[![Transfer Learning](https://huggingface.co/datasets/huggingface-course/documentation-images/resolve/main/en/chapter1/pretraining.svg)]
+]
+
+???
+The *pretraining* phase involves training a model from scratch on an enormous corpus of text (often billions or trillions of words). The goal is to learn general statistical patterns, grammar, and world knowledge embedded in the language. This is achieved using self-supervised objectives like Masked Language Modeling (MLM) or Causal Language Modeling (CLM), which don't require manual labels. Pretraining is extremely computationally intensive, often costing millions of dollars and requiring specialized hardware. It's typically performed once by large research labs or companies, creating foundational models that others can then adapt.
+
+---
+
+# Fine-tuning Phase
+
+.col-6[
+  **Characteristics:**
+  - Adapt pretrained model to specific tasks
+  - Use smaller, task-specific datasets
+  - Much less expensive than pretraining
+  - Can be done on consumer hardware
+  - Preserves general knowledge while adding specialized capabilities
+]
+.col-6[
+  .center[![Fine-tuning](https://huggingface.co/datasets/huggingface-course/documentation-images/resolve/main/en/chapter1/finetuning.svg)]
+]
+
+???
+The *fine-tuning* phase takes a pretrained model and further trains it on a smaller, labeled dataset specific to a target task (e.g., sentiment analysis, medical text summarization). This adaptation process is significantly less computationally expensive than pretraining and often feasible on standard hardware. Fine-tuning allows the model to specialize its general language understanding, acquired during pretraining, for the nuances of the specific task. It effectively transfers the broad knowledge base and refines it for a particular application, requiring far less task-specific data than training from scratch.
+
+---
+
+# Understanding LLM Inference
+
+**Inference**: The process of using a trained model to generate predictions or outputs.
+
+**Key Components**:
+- Input processing (tokenization)
+- Model computation
+- Output generation
+- Sampling strategies
+
+???
+Once a model is trained (either pretrained or fine-tuned), we use it for *inference* – the process of generating predictions or outputs for new, unseen inputs. For LLMs, inference typically means taking an input prompt and generating a textual response. Understanding the mechanics of inference is vital for deploying these models effectively, as it involves trade-offs between speed, cost, memory usage, and output quality. Key components include how the input is processed, the model's computations, how the output is generated step-by-step, and the strategies used to select the next token.
+
+---
+
+# The Two-Phase Inference Process
+
+**1. Prefill Phase**:
+- Process the entire input prompt
+- Compute initial hidden states
+- Computationally intensive for long prompts
+- Only happens once per generation
+
+**2. Decode Phase**:
+- Generate one token at a time
+- Use previous tokens as context
+- Repeat until completion criteria met
+- Most of the generation time is spent here
+
+???
+LLM inference typically proceeds in two distinct phases: 1. **Prefill Phase**: The model processes the entire input prompt simultaneously. This involves calculating initial internal states (often called hidden states or activations) based on the prompt. This phase is compute-bound, especially for long prompts, as it involves parallel processing across the input length, but it only occurs once per generation request. Think of it as the model 'reading and understanding' the prompt. 2. **Decode Phase**: The model generates the output token by token, auto-regressively. In each step, it uses the context of the prompt *and* all previously generated tokens to predict the next token. This phase is memory-bandwidth-bound, as it involves sequential generation and accessing previously computed states. This is the 'writing' part, repeated until a stopping condition is met.
+
+---
+
+# Sampling Strategies
+
+**Temperature**: Controls randomness (higher = more creative, lower = more deterministic)
+
+**Top-k Sampling**: Only consider the k most likely next tokens
+
+**Top-p (Nucleus) Sampling**: Consider tokens covering p probability mass
+
+**Beam Search**: Explore multiple possible continuations in parallel
+
+???
+During the decode phase, selecting the *next* token isn't always about picking the single most probable one. Various *sampling strategies* allow control over the generation process: **Temperature**: Adjusts the randomness of predictions. Lower values (<1.0) make generation more focused and deterministic; higher values (>1.0) increase diversity and creativity, potentially at the cost of coherence. **Top-k Sampling**: Limits the selection pool to the 'k' most probable next tokens. **Top-p (Nucleus) Sampling**: Selects from the smallest set of tokens whose cumulative probability exceeds a threshold 'p'. **Beam Search**: Explores multiple potential sequences (beams) in parallel, choosing the overall most probable sequence at the end. These strategies enable tuning the output for different needs, balancing predictability and creativity.
+
+---
+
+# Optimizing Inference
+
+**KV Cache**: Store key-value pairs from previous tokens to avoid recomputation
+
+**Quantization**: Reduce precision of model weights (e.g., FP16, INT8)
+
+**Batching**: Process multiple requests together for better hardware utilization
+
+**Model Pruning**: Remove less important weights to reduce model size
+
+???
+Making LLM inference efficient is critical for practical deployment. Several optimization techniques are commonly used: **KV Cache**: Stores intermediate results (Key and Value projections from attention layers) for previously processed tokens, avoiding redundant computations during the decode phase. This significantly speeds up generation but increases memory usage. **Quantization**: Reduces the numerical precision of model weights (e.g., from 32-bit floats to 8-bit integers). This shrinks model size and speeds up computation, often with minimal impact on accuracy. **Batching**: Processing multiple input prompts simultaneously to maximize hardware (GPU) utilization. **Model Pruning/Sparsification**: Removing less important weights or structures from the model to reduce its size and computational cost. These techniques are essential for reducing latency, cost, and memory footprint in production.
+
+---
+
+# Bias and Limitations
+
+**Source**: Models learn from vast internet data, including harmful stereotypes (sexist, racist, etc.).
+
+**Problem**: LLMs can replicate and amplify these biases.
+
+**Caution**: Fine-tuning does not automatically remove underlying bias. Awareness is crucial.
+
+???
+Despite their impressive capabilities, LLMs have significant limitations and potential harms. A major concern is *bias*. Trained on vast datasets scraped from the internet, models inevitably learn and can replicate harmful societal biases present in that data – including sexist, racist, and other prejudiced stereotypes. They learn statistical patterns, not true understanding or ethical reasoning. Consequently, their outputs can reflect and even amplify these biases. It's crucial to remember that fine-tuning on specific data doesn't automatically eliminate these underlying biases learned during pretraining.
+
+---
+
+# Bias Example: BERT Fill-Mask
+
+**Code**:
+```python
+from transformers import pipeline
+
+unmasker = pipeline("fill-mask", model="bert-base-uncased")
+print(unmasker("This man works as a [MASK]."))
+print(unmasker("This woman works as a [MASK]."))
+```
+
+**Results**:
+- Man: lawyer, carpenter, doctor, waiter, mechanic
+- Woman: nurse, waitress, teacher, maid, prostitute
+
+???
+This code snippet provides a concrete example of gender bias using the BERT model in a `fill-mask` task. When prompted to predict occupations for 'This man works as a...' versus 'This woman works as a...', the model suggests stereotypical roles (e.g., 'lawyer', 'carpenter' for men; 'nurse', 'waitress' for women). This occurs even though BERT was pretrained on seemingly neutral sources like Wikipedia and BookCorpus. It starkly illustrates how models absorb and reflect societal biases present in their training data, highlighting the need for careful evaluation and mitigation when using them in real-world scenarios.
+
+---
+
+# Sources of Bias in Models
+
+**Three main sources**:
+1. **Training Data**: Biased data leads to biased models
+2. **Pretrained Models**: Bias persists through transfer learning
+3. **Optimization Metrics**: What we optimize for shapes model behavior
+
+**Mitigation approaches**:
+- Careful dataset curation
+- Bias evaluation and monitoring
+- Fine-tuning with debiased data
+- Post-processing techniques
+
+???
+Bias in AI models can originate from several sources throughout the development lifecycle: 1. **Training Data**: The most direct source; biased data leads to biased models. If the data reflects societal stereotypes, the model will learn them. 2. **Pretrained Models**: Bias embedded in foundational models persists even after fine-tuning on new data (negative transfer). 3. **Optimization Metrics & Objectives**: The very definition of 'good performance' can inadvertently favor biased outcomes if not carefully designed. Addressing bias requires a multi-faceted approach, including careful dataset curation and filtering, rigorous evaluation and monitoring for bias, specialized fine-tuning techniques using debiased data or objectives, and potentially post-processing model outputs.
+
+---
+
+# Summary & Recap
+
+- NLP & LLMs: Defined the field and the impact of large models.
+- Transformer Capabilities: Explored tasks solvable via pipeline().
+- Attention: Introduced the core mechanism.
+- Architectures: Covered Encoder-only, Decoder-only, Encoder-Decoder.
+- Transfer Learning: Pretraining and fine-tuning approach.
+- Inference: Understanding how models generate text.
+- Bias: Highlighted the importance of awareness.
+
+???
+Let's quickly recap the key topics we've covered in this chapter: We defined Natural Language Processing (NLP) and saw how Large Language Models (LLMs) represent a major advancement. We explored the diverse capabilities of Transformers using the simple `pipeline()` function for tasks ranging from classification to generation. We introduced the core concept of the *attention mechanism*. We differentiated between the three main Transformer architectures (Encoder-only, Decoder-only, Encoder-Decoder) and their typical use cases. We explained the crucial *transfer learning* paradigm involving pretraining and fine-tuning. We looked into the mechanics of LLM *inference*, including sampling and optimization. Finally, we highlighted the critical issues of *bias and limitations* in these models. This provides a solid foundation for the rest of the course.
+
+---
+
+# Next Steps
+
+- Deeper dive into the 🤗 Transformers library.
+- Data processing techniques.
+- Fine-tuning models for specific needs.
+- Exploring advanced concepts.
+
+???
+Building on this foundation, the upcoming chapters will delve into more practical aspects. We will explore the 🤗 Transformers library in greater detail, learn essential data processing techniques for preparing text for models, walk through the process of fine-tuning pretrained models for specific tasks, and touch upon more advanced concepts and deployment strategies. Get ready to gain hands-on skills for effectively working with these powerful models.
+
+---
+
+class: center, middle
+
+# Thank You!
+
+???
+Thank you for joining this first chapter. We hope this introduction has sparked your interest, and we look forward to seeing you in the next lesson!

chapter1/template/index.html

@@ -0,0 +1,22 @@
+<!DOCTYPE html>
+<html>
+<head>
+  <meta charset="UTF-8">
+  <title>{{title}}</title>
+  {{{style}}}
+  <script src="remark.min.js"></script>  
+  <script>
+    function create() {
+      return remark.create({
+        {{{source}}},
+        ratio: '16:9',
+        highlightLines: true,
+        countIncrementalSlides: false,
+        highlightStyle: 'github'
+      });
+    }
+  </script>
+</head>
+<body onload="slideshow = create()">
+</body>
+</html>

chapter1/template/style.scss

@@ -0,0 +1,259 @@
+@use "sass:math";
+
+// Theme
+// ---------------------------------------------------------------------------
+
+// Hugging Face colors
+$primary          : #FFD21E; // HF Yellow
+$secondary        : #FF9D00; // HF Orange
+$tertiary         : #6B7280; // HF Gray
+$light            : #FFF;
+$dark             : #333;
+$text-dark        : #212121;
+$text-light       : $light;
+$code-background  : #F8F8F8;
+$overlay          : transparentize(#000, .5);
+$font-size        : 28px;
+$font-size-impact : 128px;
+$font             : Arial, Helvetica, sans-serif;
+$font-title       : Arial, Helvetica, sans-serif;
+$font-fixed       : 'Lucida Console', Monaco, monospace;
+$margin           : 20px;
+$iframe-scale     : 1.5;
+
+
+// CSS Base
+// ---------------------------------------------------------------------------
+
+* { box-sizing: border-box; }
+body { font-family: $font; }
+h1, h2, h3, h4, h5, h6 {
+  margin: 0 0 $margin 0;
+  font-family: $font-title;
+}
+h1 { color: $primary; }
+h2 { color: $secondary; }
+h3 { color: $tertiary; }
+li { margin-bottom: .25em; };
+pre, code {
+  text-align: left;
+  font-family: $font-fixed;
+  color: $secondary;
+  background: $code-background;
+}
+a, a:visited, a:hover, a:active { color: $text-dark; }
+img { 
+  vertical-align: inherit; 
+  max-width: 100%;
+  height: auto;
+}
+blockquote {
+  border-left: 8px solid;
+  padding-left: .5em;
+  color: $tertiary;
+  text-align: left;
+  margin: 1em 0;
+  & > p { margin: 0; }
+}
+
+
+// Remark base
+// ---------------------------------------------------------------------------
+
+.remark-container { background: $dark; }
+.remark-slide-scaler { box-shadow: none; }
+.remark-notes { font-size: 1.5em; }
+
+.remark-slide-content {
+  font-size: $font-size;
+  padding: 1em 2em;
+  color: $text-dark;
+  background-size: cover;
+}
+
+.remark-slide-number {
+  color: $text-light;
+  right: 1em;
+  opacity: .6;
+  font-size: 0.8em;
+  z-index: 2;
+  .no-counter & { display: none; }
+}
+
+// Additions
+.impact {
+  background-color: #f0f0f0;
+  vertical-align: middle;
+  text-align: center;
+  &, h1, h2 { color: $dark; } // Dark text on yellow background for better readability
+  h1 { font-size: $font-size-impact; }
+}
+
+.full {
+  &, h1, h2 { color: $text-light; }
+  &iframe {
+    height: calc(math.div(100%, $iframe-scale) - 1.2em);
+    width: math.div(100%, $iframe-scale);
+    transform: scale($iframe-scale);
+    transform-origin: 0 0;
+    border: 0;
+  }
+}
+
+.bottom-bar {
+  background-color: $primary;
+  color: $dark; // Dark text on yellow background for better readability
+  position: absolute;
+  bottom: 0;
+  left: 0;
+  right: 0;
+  font-size: 20px;
+  padding: .8em;
+  text-align: left;
+  z-index: 1;
+  p { margin: 0;}
+  .impact &, .full & { display: none; }
+}
+
+
+// Utilities
+// ---------------------------------------------------------------------------
+
+// Positioning
+.side-layer {
+  position: absolute;
+  left: 0;
+  width: 100%;
+  padding: 0 2em;
+}
+.middle { &, & img, & span { vertical-align: middle; } };
+.top { vertical-align: top; };
+.bottom { vertical-align: bottom; };
+.inline-block {
+  p, ul, ol, blockquote {
+    display: inline-block;
+    text-align: left;
+  }
+}
+.no-margin { &, & > p, & > pre, & > ul, & > ol { margin: 0; } }
+.no-padding { padding: 0; }
+.space-left { padding-left: 1em; }
+.space-right { padding-right: 1em; }
+
+// Images
+.responsive > img { width: 100%; height: auto; };
+.contain { background-size: contain; };
+.overlay { box-shadow: inset 0 0 0 9999px $overlay; }
+
+// Center images
+.center {
+  text-align: center;
+  img {
+    display: block;
+    margin: 0 auto;
+    max-width: 100%;
+    max-height: 400px; // Limit height to fit on slides
+  }
+}
+
+// Text
+.left { text-align: left; }
+.right { text-align: right; }
+.justify { text-align: justify; }
+.primary { color: $primary; }
+.primary-bg { background-color: $primary; }
+.secondary { color: $secondary; }
+.secondary-bg { background-color: $secondary; }
+.tertiary { color: $tertiary; }
+.tertiary-bg { background-color: $tertiary; }
+.alt { color: $secondary; };
+.em { color: $tertiary; };
+.thin { font-weight: 200; }
+.huge { font-size: 2em; }
+.big { font-size: 1.5em; }
+.small { font-size: .8em; }
+.strike { text-decoration: line-through; }
+.dark { color: $dark; }
+.dark-bg { background-color: $dark; }
+.light { color: $light; }
+.light-bg { background-color: $light; }
+.alt-bg { background-color: $secondary; };
+
+// Simple 12-columns grid system
+.row {
+  width: 100%;
+  &::after {
+    content: '';
+    display: table;
+    clear: both;
+  }
+  &.table { display: table; };
+  &.table [class^="col-"] {
+    float: none;
+    display: table-cell;
+    vertical-align: inherit;
+  }
+}
+
+[class^="col-"] {
+  float: left;
+  &.inline-block {
+    float: none;
+    display: inline-block;
+  }
+}
+
+@for $i from 1 through 12 {
+  .col-#{$i} { width: math.div(100%, 12) * $i; }
+}
+
+// Animations
+@keyframes fadeIn {
+  from { opacity: 0; }
+  to { opacity: 1; }
+}
+
+.animation-fade {
+  animation-duration: 300ms;
+  animation-fill-mode: both;
+  animation-timing-function: ease-out;
+  .remark-visible & { animation-name: fadeIn; }
+}
+
+// Hugging Face specific styles
+.hf-logo {
+  max-height: 80px;
+  margin: 20px 0;
+}
+
+// Fix PDF print with chrome
+// ---------------------------------------------------------------------------
+
+@page {
+  // 908px 681px for 4/3 slides
+  size: 1210px 681px;
+  margin: 0;
+}
+
+@media print {
+  .remark-slide-scaler {
+    width: 100% !important;
+    height: 100% !important;
+    transform: scale(1) !important;
+    top: 0 !important;
+    left: 0 !important;
+  }
+}
+
+// Code
+.remark-code { 
+  font-size: .7em; 
+  line-height: 1.2; 
+  background: $dark !important; 
+  color: $light !important;
+
+} 
+
+.remark-code .hljs-keyword { color: $primary; }
+
+.remark-code .hljs-string { color: $secondary; }

pyproject.toml

@@ -0,0 +1,7 @@
+[project]
+name = "video-course"
+version = "0.1.0"
+description = "Add your description here"
+readme = "README.md"
+requires-python = ">=3.11"
+dependencies = []

scripts/create_video.py

@@ -0,0 +1,218 @@
+import os
+import glob
+import argparse
+import sys
+from typing import List, Tuple
+
+import natsort
+from moviepy import *
+from pdf2image import convert_from_path
+
+
+def parse_arguments():
+    """Parse command line arguments."""
+    parser = argparse.ArgumentParser(
+        description="Create a video from PDF slides and audio files."
+    )
+    parser.add_argument(
+        "--pdf", required=True, help="Path to the PDF file containing slides"
+    )
+    parser.add_argument(
+        "--audio-dir", required=True, help="Directory containing audio files"
+    )
+    parser.add_argument(
+        "--audio-pattern",
+        default="*.wav",
+        help="Pattern to match audio files (default: *.wav)",
+    )
+    parser.add_argument(
+        "--buffer",
+        type=float,
+        default=1.5,
+        help="Buffer time in seconds after each audio clip (default: 1.5)",
+    )
+    parser.add_argument(
+        "--output",
+        default="final_presentation.mp4",
+        help="Output video filename (default: final_presentation.mp4)",
+    )
+    parser.add_argument(
+        "--fps", type=int, default=5, help="Frame rate of output video (default: 5)"
+    )
+    parser.add_argument(
+        "--dpi",
+        type=int,
+        default=120,
+        help="DPI for PDF to image conversion (default: 120)",
+    )
+    return parser.parse_args()
+
+
+def find_audio_files(audio_dir: str, pattern: str) -> List[str]:
+    """Find and sort audio files in the specified directory."""
+    search_pattern = os.path.join(audio_dir, pattern)
+    audio_files = natsort.natsorted(glob.glob(search_pattern))
+    return audio_files
+
+
+def convert_pdf_to_images(pdf_path: str, dpi: int) -> List:
+    """Convert PDF pages to images."""
+    print(f"Converting PDF '{pdf_path}' to images...")
+    try:
+        pdf_images = convert_from_path(pdf_path, dpi=dpi)
+        print(f"Successfully converted {len(pdf_images)} pages from PDF.")
+        return pdf_images
+    except Exception as e:
+        print(f"Error converting PDF to images: {e}")
+        sys.exit(1)
+
+
+def create_video_clips(
+    pdf_images: List, audio_files: List[str], buffer_seconds: float, output_fps: int
+) -> List:
+    """Create video clips from images and audio files."""
+    video_clips_list = []
+
+    print("\nCreating individual video clips...")
+    for i, (img, aud_file) in enumerate(zip(pdf_images, audio_files)):
+        print(
+            f"Processing pair {i + 1}/{len(pdf_images)}: "
+            f"Page {i + 1} + {os.path.basename(aud_file)}"
+        )
+        try:
+            # Load audio to get duration
+            audio_clip = AudioFileClip(aud_file)
+            audio_duration = audio_clip.duration
+
+            # Calculate target duration for the image clip
+            target_duration = audio_duration + buffer_seconds
+
+            # Create a temporary file for the image
+            temp_img_path = f"temp_slide_{i + 1}.png"
+            img.save(temp_img_path, "PNG")
+
+            # Create video clip from image with the correct duration
+            # In MoviePy v2.0+, we use ImageSequenceClip with a single image
+            img_clip = ImageSequenceClip([temp_img_path], durations=[target_duration])
+
+            # Set FPS for the individual clip
+            img_clip = img_clip.with_fps(output_fps)
+
+            # Set the audio for the image clip
+            video_clip_with_audio = img_clip.with_audio(audio_clip)
+
+            video_clips_list.append(video_clip_with_audio)
+            print(
+                f"  -> Clip created (Audio: {audio_duration:.2f}s + "
+                f"Buffer: {buffer_seconds:.2f}s = "
+                f"Total: {target_duration:.2f}s)"
+            )
+
+        except Exception as e:
+            print(f"  Error processing pair {i + 1}: {e}")
+            print("  Skipping this pair.")
+            # Close clips if they were opened, to release file handles
+            if "audio_clip" in locals() and audio_clip:
+                audio_clip.close()
+            if "img_clip" in locals() and img_clip:
+                img_clip.close()
+            if "video_clip_with_audio" in locals() and video_clip_with_audio:
+                video_clip_with_audio.close()
+
+    return video_clips_list
+
+
+def concatenate_clips(
+    video_clips_list: List, output_file: str, output_fps: int
+) -> None:
+    """Concatenate video clips and write to output file."""
+    if not video_clips_list:
+        print("\nNo video clips were successfully created. Exiting.")
+        sys.exit(1)
+
+    print(f"\nConcatenating {len(video_clips_list)} clips...")
+    final_clip = None
+    try:
+        final_clip = concatenate_videoclips(video_clips_list, method="compose")
+
+        print(f"Writing final video file: {output_file}...")
+        # Write the final video file
+        final_clip.write_videofile(
+            output_file,
+            fps=output_fps,
+            codec="libx264",
+            audio_codec="aac",
+            # logger=None,  # Suppress verbose output
+        )
+        print("Final video file written successfully.")
+
+    except Exception as e:
+        print(f"\nError during concatenation or writing video file: {e}")
+        print("Ensure you have enough free disk space and RAM.")
+
+    finally:
+        # Close clips to release resources
+        if final_clip:
+            final_clip.close()
+        for clip in video_clips_list:
+            clip.close()
+
+
+def cleanup_temp_files(pdf_images: List) -> None:
+    """Clean up temporary image files."""
+    print("\nCleaning up temporary files...")
+    for i in range(len(pdf_images)):
+        temp_img_path = f"temp_slide_{i + 1}.png"
+        if os.path.exists(temp_img_path):
+            os.remove(temp_img_path)
+
+
+def main():
+    """Main function to run the script."""
+    args = parse_arguments()
+
+    # Validate inputs
+    if not os.path.exists(args.pdf):
+        print(f"Error: PDF file '{args.pdf}' not found.")
+        sys.exit(1)
+
+    if not os.path.exists(args.audio_dir):
+        print(f"Error: Audio directory '{args.audio_dir}' not found.")
+        sys.exit(1)
+
+    # Find audio files
+    audio_files = find_audio_files(args.audio_dir, args.audio_pattern)
+    if not audio_files:
+        print(
+            f"Error: No audio files found matching pattern '{args.audio_pattern}' "
+            f"in directory '{args.audio_dir}'."
+        )
+        sys.exit(1)
+
+    # Convert PDF to images
+    pdf_images = convert_pdf_to_images(args.pdf, args.dpi)
+
+    # Check if number of PDF pages matches number of audio files
+    if len(pdf_images) != len(audio_files):
+        print("Error: Mismatched number of files found.")
+        print(f"  PDF pages ({len(pdf_images)})")
+        print(f"  Audio files ({len(audio_files)}): {audio_files}")
+        print("Please ensure you have one corresponding audio file for each PDF page.")
+        sys.exit(1)
+
+    print(f"Found {len(pdf_images)} PDF pages with {len(audio_files)} audio files.")
+
+    # Create video clips
+    video_clips = create_video_clips(pdf_images, audio_files, args.buffer, args.fps)
+
+    # Concatenate clips and create final video
+    concatenate_clips(video_clips, args.output, args.fps)
+
+    # Clean up
+    cleanup_temp_files(pdf_images)
+
+    print("\nScript finished.")
+
+
+if __name__ == "__main__":
+    main()

scripts/transcription_to_audio.py

@@ -0,0 +1,318 @@
+#!/usr/bin/env python3
+"""
+Script to extract speaker notes from presentation.md files and convert them to
+audio files.
+
+Usage:
+    python transcription_to_audio.py path/to/chapter/directory
+
+This will:
+1. Parse the presentation.md file in the specified directory
+2. Extract speaker notes (text between ??? markers)
+3. Generate audio files using FAL AI with optional voice customization
+4. Save audio files in {dir}/audio/{n}.wav format
+"""
+
+import re
+import sys
+import os
+import argparse
+import json
+import hashlib
+from pathlib import Path
+import logging
+import time
+import requests
+
+from dotenv import load_dotenv
+import fal_client
+
+load_dotenv()
+
+# Configure logging
+logging.basicConfig(
+    level=logging.INFO, format="%(asctime)s - %(levelname)s - %(message)s"
+)
+
+logger = logging.getLogger(__name__)
+
+VOICE_ID = os.getenv("VOICE_ID")
+
+
+def extract_speaker_notes(markdown_content):
+    """Extract speaker notes from markdown content."""
+    # Pattern to match content between ??? markers
+    pattern = r"\?\?\?(.*?)(?=\n---|\n$|\Z)"
+
+    # Find all matches using regex
+    matches = re.findall(pattern, markdown_content, re.DOTALL)
+
+    # Clean up the extracted notes
+    notes = [note.strip() for note in matches]
+
+    return notes
+
+
+def get_cache_key(text, voice, speed, emotion, language):
+    """Generate a unique cache key for the given parameters."""
+    # Create a string with all parameters
+    params_str = f"{text}|{voice}|{speed}|{emotion}|{language}"
+
+    # Generate a hash of the parameters
+    return hashlib.md5(params_str.encode()).hexdigest()
+
+
+def load_cache(cache_file):
+    """Load the cache from a file."""
+    if not cache_file.exists():
+        return {}
+
+    try:
+        with open(cache_file, "r") as f:
+            return json.load(f)
+    except (json.JSONDecodeError, IOError) as e:
+        logger.warning(f"Error loading cache: {e}")
+        return {}
+
+
+def save_cache(cache_data, cache_file):
+    """Save the cache to a file."""
+    try:
+        with open(cache_file, "w") as f:
+            json.dump(cache_data, f)
+    except IOError as e:
+        logger.warning(f"Error saving cache: {e}")
+
+
+def text_to_speech(
+    text,
+    output_file,
+    voice=None,
+    speed=1.0,
+    emotion="happy",
+    language="English",
+    cache_dir=None,
+):
+    """Convert text to speech using FAL AI and save as audio file.
+
+    Args:
+        text: The text to convert to speech
+        output_file: Path to save the output audio file
+        voice: The voice ID to use
+        speed: Speech speed (0.5-2.0)
+        emotion: Emotion to apply (neutral, happy, sad, etc.)
+        language: Language for language boost
+        cache_dir: Directory to store cache files
+    """
+    try:
+        start_time = time.monotonic()
+
+        # Create the output directory if it doesn't exist
+        output_file.parent.mkdir(exist_ok=True)
+
+        # Set up caching
+        cache_file = None
+        cache_data = {}
+        cache_key = get_cache_key(text, voice, speed, emotion, language)
+
+        if cache_dir:
+            cache_dir_path = Path(cache_dir)
+            cache_dir_path.mkdir(exist_ok=True)
+            cache_file = cache_dir_path / "audio_cache.json"
+            cache_data = load_cache(cache_file)
+
+            # Check if we have a cached URL for this request
+            if cache_key in cache_data:
+                audio_url = cache_data[cache_key]
+                logger.info(f"Using cached audio URL: {audio_url}")
+
+                # Download the audio from the cached URL
+                response = requests.get(audio_url)
+                if response.status_code == 200:
+                    with open(output_file, "wb") as f:
+                        f.write(response.content)
+
+                    logger.info(f"Downloaded cached audio to {output_file}")
+                    return True
+                else:
+                    logger.warning(f"Cached URL failed, status: {response.status_code}")
+                    # Continue with generation as the cached URL failed
+
+        # Set up voice settings
+        voice_setting = {"speed": speed, "emotion": emotion}
+
+        # Add custom voice ID if provided
+        if voice:
+            voice_setting["custom_voice_id"] = voice
+
+        def on_queue_update(update):
+            if isinstance(update, fal_client.InProgress):
+                for log in update.logs:
+                    logger.debug(log["message"])
+
+        # Generate speech with FAL AI
+        logger.info(f"Generating speech with voice ID: {voice}")
+
+        result = fal_client.subscribe(
+            "fal-ai/minimax-tts/text-to-speech/turbo",
+            arguments={
+                "text": text,
+                "voice_setting": voice_setting,
+                "language_boost": language,
+            },
+            with_logs=True,
+            on_queue_update=on_queue_update,
+        )
+
+        # Download the audio file from the URL
+        if "audio" in result and "url" in result["audio"]:
+            audio_url = result["audio"]["url"]
+            logger.info(f"Downloading audio from {audio_url}")
+
+            # Cache the URL if caching is enabled
+            if cache_file:
+                cache_data[cache_key] = audio_url
+                save_cache(cache_data, cache_file)
+                logger.info(f"Cached audio URL for future use")
+
+            response = requests.get(audio_url)
+            if response.status_code == 200:
+                # Save the audio file
+                with open(output_file, "wb") as f:
+                    f.write(response.content)
+            else:
+                logger.error(f"Failed to download audio: {response.status_code}")
+                return False
+        else:
+            logger.error(f"Unexpected response format: {result}")
+            return False
+
+        end_time = time.monotonic()
+        logger.info(
+            f"Generated audio in {end_time - start_time:.2f} seconds, "
+            f"saved to {output_file}"
+        )
+
+        return True
+    except Exception as e:
+        logger.error(f"Error generating audio: {e}")
+        return False
+
+
+def process_presentation(
+    chapter_dir,
+    voice=None,
+    speed=1.0,
+    emotion="happy",
+    language="English",
+    cache_dir=None,
+):
+    """Process the presentation.md file in the given directory."""
+    # Construct paths
+    chapter_path = Path(chapter_dir)
+    presentation_file = chapter_path / "presentation.md"
+    audio_dir = chapter_path / "audio"
+
+    # Check if presentation file exists
+    if not presentation_file.exists():
+        logger.error(f"Presentation file not found: {presentation_file}")
+        return False
+
+    # Create audio directory if it doesn't exist
+    audio_dir.mkdir(exist_ok=True)
+
+    # Read the presentation file
+    with open(presentation_file, "r", encoding="utf-8") as file:
+        content = file.read()
+
+    # Extract speaker notes
+    notes = extract_speaker_notes(content)
+
+    if not notes:
+        logger.warning("No speaker notes found in the presentation file.")
+        return False
+
+    logger.info(f"Found {len(notes)} slides with speaker notes.")
+
+    # Generate audio files for each note
+    for i, note in enumerate(notes, 1):
+        if not note.strip():
+            logger.warning(f"Skipping empty note for slide {i}")
+            continue
+
+        output_file = audio_dir / f"{i}.wav"
+        logger.info(f"Generating audio for slide {i}")
+
+        success = text_to_speech(
+            note,
+            output_file,
+            voice,
+            speed,
+            emotion,
+            language,
+            cache_dir,
+        )
+        if success:
+            logger.info(f"Saved audio to {output_file}")
+        else:
+            logger.error(f"Failed to generate audio for slide {i}")
+
+    return True
+
+
+def main():
+    parser = argparse.ArgumentParser(
+        description="Extract speaker notes from presentation.md and convert to"
+        " audio files."
+    )
+    parser.add_argument(
+        "chapter_dir", help="Path to the chapter directory containing presentation.md"
+    )
+    parser.add_argument(
+        "--voice",
+        default=VOICE_ID,
+        help="Voice ID to use (defaults to VOICE_ID from .env)",
+    )
+    parser.add_argument(
+        "--speed", type=float, default=1.0, help="Speech speed (0.5-2.0, default: 1.0)"
+    )
+    parser.add_argument(
+        "--emotion",
+        default="happy",
+        help="Emotion to apply (neutral, happy, sad, etc.)",
+    )
+    parser.add_argument(
+        "--language", default="English", help="Language for language boost"
+    )
+    parser.add_argument(
+        "--cache-dir",
+        default=".cache",
+        help="Directory to store cache files (default: .cache)",
+    )
+    parser.add_argument(
+        "--no-cache", action="store_true", help="Disable caching of audio URLs"
+    )
+    args = parser.parse_args()
+
+    # Determine cache directory
+    cache_dir = None if args.no_cache else args.cache_dir
+
+    logger.info(f"Processing presentation in {args.chapter_dir}")
+    success = process_presentation(
+        args.chapter_dir,
+        args.voice,
+        args.speed,
+        args.emotion,
+        args.language,
+        cache_dir,
+    )
+
+    if success:
+        logger.info("Audio generation completed successfully.")
+    else:
+        logger.error("Audio generation failed.")
+        sys.exit(1)
+
+
+if __name__ == "__main__":
+    main()