The visualization heavily uses Cohere’s generation and embedding models and a workflow described below to explore such large text archives. The data is ultimately plotted with datamapplot with some customization.

Exploring the frontier is vital—despite information overload

NeurIPS is known as one of the leading machine learning conferences at which some of the top research is presented. Having attended several, the experience can be challenging in several ways:

1. In a field that moves as fast as ML, the field often evolves between the May submission deadline and the December conference. Exploring papers earlier helps.

2. It is overwhelming in size and magnitude. We need better tools to reduce the information overload. Intentional use of AI and visualization can help here.

3. Outside of your own domain, work in other domains can often be hard to decipher. An LLM can explain in lay terms.

Over the last few years I’d often build simple interactive visualizations that help explore these papers. And since the accepted papers list was just announced, here’s that visualization so you can explore it for yourself.

A guided tour of the visualization

Zooming in reveals the names of smaller, more fine-grained clusters. We can also expand the top-level categories to reveal the major clusters inside of them. Clicking on a cluster name from the topic tree focuses the plot on that cluster. An LLM proposes cluster names, which I then revise.

Hovering over a paper reveals information containing the paper’s title and abstract. But I’ve always wanted the model to do some work on this text as well, so you’ll see an LLM extracted summary, problem statement, and methodology, plus an explanation for a five-year-old which turned out to be my personal favorite section especially as I switched to explore domains outside of my immediate focus.

The tooltip presents the abstract as well as other sections that help break it down.

Conference observations

Largest Themes: LLMs, Multimodality, and Reinforcement Learning
These three stand out to me as the biggest groups. Not only as major clusters, but they also tend to constitute parts of other clusters (I also run a multi-label classification step, so a paper isn’t limited to a single cluster). By my calculation, about 28% of the papers include multimodality as a primary focus, and 13% include reinforcement learning as a primary focus (and those can overlap). Also at 13%, I’m seeing Evaluation papers and Reasoning papers.

A spike in LLM reasoning research. Reasoning was the major topic of discussion in the corridors of NeurIPS 2024 due to the recent release (at the time) of the O1 models. This interest is echoed in the accepted research as expected. I’m seeing about 766 papers with reasoning as a core focus.

Diffusion joins LLMs and Reinforcement Learning as one of the major themes of the conference.

The top part of the space is predominantly computer vision and multimodality, with its western region exploring lots of different aspects of diffusion models.

The topic tree echoes this with multiple major clusters on diffusion

Over in science land, I’m really grateful for ELI5. I do see myself spending a few days just reading these. I find that my process is to read the summary, and if I don’t understand it, I’d revisit it after reading the ELI5, and it’s often helpful.

Here are a couple of examples where it feels the AI is hugging your brain and helping it absorb information it may otherwise find to be obscure. I absolutely adore this use case and its potential to extend the human mind.

Another example:

AI Should Extend The Human Mind

In this approach, we’re intentionally pointing the AI at many sub-problems to make reading this collection of text easier.

Individual text analysis

Some of these steps apply at individual item level, and some of them are applied to groups (clusters) to help us navigate our way through the collection.

• Text extraction

• Classification

• Question answering

• Summarization

The great thing in text-to-text models is that we can do all of these in a single step. We simply prepare a prompt template, and inject each text in that prompt, ending up with 5,787 prompts (one per paper accepted in the conference).

Running thousands of prompts is one of the superpowers you do not get to use if all of your LLM usage is through a playground. It’s probably not even wise yet to delegate to many agents given the possible costs involved. So these are often run as individual scripts, or as workflows and in both cases triggered intentionally by a human.

Clustering into many small groups

Three years ago, I wrote Combing For Insight in 10,000 Hacker News Posts With Text Clustering, which described a process very close to the one I used here. We can see that process in this figure:

The abstracts are embedded, then reduced to two dimensions with UMAP, then clustered with K-Means into many clusters. These clusters are later presented to a model to assign cluster names.

Now, the UMAP step dramatically reduces the size of the embeddings and a lot of the information is lost. We’re okay with that in this scenario because the coherence of the plot is a high priority (e.g., the clusters are actually grouped together). In other scenarios, we might choose to either cluster the embeddings directly, or reduce them to an interim embedding size.

You can read a lot more about these flows in Chapter 5 of our book, Hands-On Large Language Models.

Clustering (the small clusters) into a few larger clusters

Let’s switch to plot these vectors so we can see the effect of the clustering more clearly.

The small circles in the second grid are cluster centroids. They are produced as a part of the K-Means clustering algorithm. We can then cluster those centroids to produce a higher level clustering for, say, 10 top-level categories.

This already adds a lot of information to the plot. But we can reveal a lot more information if we assign meaningful names to the clusters.

Cluster name assignment

Now we can start to see embeddings and generation models working together. Given the embeddings clusters, and the summaries of the abstracts, we can present each cluster to the generative model to assign a name to it. We can either use the entire abstracts or the summaries we generated in the previous step.

There are a few naming techniques beyond simply sampling examples from the cluster.

From here, all that remains is plugging this data into the incredible datamapplot, and customizing some of its parameters leading to the final figure.

Context Handoffs and Naming Larger Clusters

One pattern that this workflow highlights is the focus we need to pay to context across pipeline steps. Prompt engineering and context engineering are key areas for working with LLMs, and that extends to LLM processing pipelines.

The cluster names resulting from the previous step could be redundant. So two adjacent clusters could both be called “LLM Reasoning” if the model is fed the papers from that cluster alone.

There are a couple of ways around this: one is to pack papers from other clusters and perhaps name all clusters in one go.

This approach can work if your data can fit in the model context. Another approach would be to do it over two steps, 1) name by looking only at the cluster, and 2) allow the model another look at the clusters after the first pass of naming.

In this pipeline, I leaned towards this approach because it fits neatly with the step of assigning a high-level cluster name (I will also refer to this as “category name”). This way, The first type of context is freed up for the model to focus on an individual cluster and not be distracted by other clusters. But this only works because in addition to the cluster_name, we’re also generating a cluster_description which contains enough details to make this context hand-off possible without overpacking the renaming step with too much information.

The shape of the prompt can look like this:

Because we are only doing a few of these calls, we can assign this to a more advanced model. We can use Command-A Reasoning here for example and allow it to reason about the names. It’s always fascinating to look at these traces. Here’s an example reasoning trace for assigning a category name:

For the high-level category name, it should encapsulate this theme. Terms like “LLM Reasoning” and “Evaluation” come to mind. Since the category is part of a larger collection of ML research, it needs to be specific. “LLM Reasoning and Evaluation” seems appropriate as it covers both the development and assessment aspects.

About The Models

Command A, Command A Reasoning, and Embed 4 were released earlier in 2025. You can learn more about Command A by reading the Command A Technical Report which contains 55 pages of insights into how it was built.

Future work and limitations

This work is simply one possible pipeline and can certainly be improved. My aim is more people innovate around methods and user interfaces that empower individuals to process vast amounts of information in super-human ways.

Some areas I find need further developments include:

• Better ways to automate review of the many small clusters, I was able to scan them (usually in excel), and look more closely at the large cluster names. But as things grow in the hundreds, we’ll need the aid of more tools.

• Clustering workflows that better address noise. I know HDBSCAN is useful here. I often still lean towards a first K-Means step, but then disregard clusters with a few number of papers as noise, because they might not actually coherently be a part of a coherent grouping despite their semantic similarity.

• UI that allows switching between different topologies, or different assignments. Datamapplot allows some of this, and I’m planning to dig deeper there.

Acknowledgements

Thanks to Adrien Morisot, Ahmet Ustun, Case Ploeg, Eugene Cho, Irem Ergun, Keith Hall, Komal Kumar Teru, Madeline Smith, Nick Frosst, Patrick Lewis, Rafid Al-Humaimidi, Sarra Habchi, Sophia Althammer, Suhas Pai, Thomas Euyang, Trent Fowler, and Varun Kumethi for feedback, thoughts, and hallway chatter about this exploration.

Have you explored such methods in the past? Please share and link in the comments!

Difference From Previous Open Source GPT Models

GPT-OSS is similar to previous models in that it’s an autoregressive Transformer generating one token at a time.

The major area of difference in a mid-2025 LLM is that the tokens they generate can solve far more difficult problems by:

• Using tools

• Reasoning

• Being better at problem solving and coding

In the following figure, we see that main architectural features, which are not a major departure from the current crop of capable open source models. The major architectural difference from GPT2 is that GPT-OSS is a mixture-of-experts model.

If you want to understand more about the architecture, we go over it in detail and lots of visuals (and exclusive animations!) in our free course How Transformer LLMs Work.

Using the the visual language we introduce in the course for attention, the GPT-OSS Transformer Block looks like this the following figure.

Note that little of these architectural details is particularly novel. They’re generally similar in the latest SoTA open source MoE models.

Message Formatting

For many more users, the details of the behavior and formatting of the model’s reasoning and tool calls are more important than the architecture.

In the following figure, we can see the shapes of the input and output to the model.

Messages and Output Channels

Let’s break this down by looking at the three main types of users of an open source LLM:

• End-users of an LLM app

  • Example: Users of the ChatGPT app

  • These users mainly interact with the user message they send and the final answer they see. In some apps, they may see some of the interim reasoning traces.

• Builders of LLM apps

  • Example: Cursor or Manus

  • Input messages: These builders get to set their own system and developer messages — defining general model expected behaviors and instructions, safety choices, reasoning level, and tool definitions for the model to use. They also have to do a lot of prompt engineering and context management in the user message.

  • Output messages: builders can choose whether to show the reasoning traces to their users. They’ll also define the tools, set how much reasoning

• Post-trainers of LLMs

  • Power users who fine-tune models will have interact with all message types and format data in the right shape including for reasoning and tool calls and responses.

The latter two categories, builders of LLM apps and post-trainers of LLMs benefit from understanding the channels concepts of assistant messages. This is implemented in the OpenAI Harmony repo.

(If you find this type of explanation helpful, be sure to check out best-selling book with over 300 figures explaining LLMs at this level of depth, and the Github repo currently at 14K stars)

Message Channels

Model outputs are all assistant messages. The model assigns them to a ‘channel’ category to indicate the type of message.

• Analysis for reasoning (and some tool calls)

• Commentary for functional calling (and most tool calls)

• Final for the message including the final response

So assuming we give the model a prompt where it needs to reason and use a couple of tool calls, the next figure shows a conversation where all three message types are used.

These are indicated as turns 1, 3, and 5 because turns 2 and 4 would be the tool responses to those calls. The final answer is what the end user would see.

Reasoning

Reasoning has trade-offs that advanced users have to make choices about. On the one hand, more reasoning allows the model more time and compute to reason about a problem which helps it tackle more difficult problems. On the other hand, that comes at a cost of latency and compute. This choice makes itself apparent in how there are both strong reasoning LLMs and non-reasoning LLMs which are each best at tackling different kinds of problems.

One middle ground option is to have a reasoning model that responds to a specific reasoning budget. This is the category that GPT-OSS belongs to. It allows the reasoning mode (low, medium, or high) in the system message. Figure 3 from the model card shows how that effects scores on benchmarks and how many tokens are in the the reasoning traces (a.k.a., chain-of-thought or CoT).

We can contrast this with Qwen3’s reasoning modes, which are a binary thinking / non-thinking modes. For thinking mode, they do show a method to stop thinking beyond a certain token threshold and report how that effects the scores on various reasoning benchmarks.

Reasoning Modes (low, medium, and high)

A good way to show the difference between the reasoning modes is to ask a difficult reasoning question, so I picked one from the AIME25 dataset and asked the 120B model in the three reasoning mode,

The correct answer to this question is 104. So both the the medium and high reasoning modes get it right. The high reasoning mode takes double the compute/generation time to arrive at that answer, however.

This underscores the point we mentioned earlier about picking the right reasoning mode for your use case:

• Doing agentic tasks? High or even medium reasoning might take too long if your trajectory can span lots of steps.

• Real time vs. offline - consider what tasks might be conducted offline where a user isn’t actively waiting to achieve their goal

  • An example to consider here is a search engine — you can get very fast results on query time because lots of processing and design already happened to prepare the system for that experience.

Tokenizer

The tokenizer is pretty similar to GPT-4’s, but strikes me as slightly more efficient — especially with non-English tokens. Notice how the emoji and the Chinese character are each tokenized in two tokens instead of three, and how more segments of the Arabic text are grouped as an individual token instead of letters.

But while the tokenizer might be better on this regard, the model is mostly trained on English data.

Code (and tabs, used in python code for indentation) looks to behave mainly the same. Number tokenization also seems to work in the same way, assigning numbers up to three digits an individual token, and breaking up bigger tokens.

Further Readings

Here are a couple of further readings I’ve found compelling:

• From GPT-2 to gpt-oss: Analyzing the Architectural Advances by

• gpt-oss: OpenAI validates the open ecosystem (finally) by

• gpt-oss-120B (high): API Provider Benchmarking & Analysis

(If you find this type of explanation helpful, be sure to check out best-selling book with over 300 figures explaining LLMs at this level of depth, and the Github repo currently at 14K stars)

Enroll for free now: https://bit.ly/4aRnn7Z
Github Repo: https://github.com/HandsOnLLM/Hands-On-Large-Language-Models

We're ecstatic to bring you "How Transformer LLMs Work" -- a free course with ~90 minutes of video, code, and crisp visuals and animations that explain the modern Transformer architecture, tokenizers, embeddings, and mixture-of-expert models.

and I have developed a lot of the visual language over the last several years (tens of thousands of iterations for hundreds of figures) for the book. This was informed by many incredible colleagues at Cohere, C4AI, and the open source and open science ML community. But to have an opportunity to collaborate with the legendary Andrew Ng and the team at DeepLearning.ai we took them to the next level with animations and a concise narrative meant to enable technical learners to pick up an ML paper and understand the architecture description.

In this course, you'll learn how a transformer network architecture that powers LLMs works. You'll build the intuition of how LLMs process text and work with code examples that illustrate the key components of the transformer architecture.

Key topics covered in this course include:

• The evolution of how language has been represented numerically, from the Bag-of-Words model through Word2Vec embeddings to the transformer architecture that captures word meanings in full context.

• How LLM inputs are broken down into tokens, which represent words or pieces before they are sent to the language model.

• The details of a transformer and the three main stages, consisting of tokenization and embedding, the stack of transformer blocks, and the language model head.

• The details of the transformer block, including attention, which calculates relevance scores followed by the feedforward layer, which incorporates stored information learned in training.

• How cached calculations make transformers faster, how the transformer block has evolved over the years since the original paper was released, and how they continue to be widely used.

• Explore an implementation of recent models in the Hugging Face transformer library.

By the end of this course, you’ll have a deep understanding of how LLMs process language and you'll be able to read through papers describing models and understand the details that are used to describe these architectures. This intuition will help improve your approach to building LLM applications.

We hope you enjoy it!

DeepSeek-R1 is the latest resounding beat in the steady drumroll of AI progress. For the ML R&D community, it is a major release for reasons including:

1. It is an open weights model with smaller, distilled versions and

2. It shares and reflects upon a training method to reproduce a reasoning model like OpenAI O1.

In this post, we’ll see how it was built.

Translations: Chinese, Korean, Turkish (Feel free to translate the post to your language and send me the link to add here)

Contents:

• Recap: How LLMs are trained

• DeepSeek-R1 Training Recipe

• 1- Long chains of reasoning SFT Data

• 2- An interim high-quality reasoning LLM (but worse at non-reasoning tasks).

• 3- Creating reasoning models with large-scale reinforcement learning (RL)

  • 3.1- Large-Scale Reasoning-Oriented Reinforcement Learning (R1-Zero)

  • 3.2- Creating SFT reasoning data with the interim reasoning model

  • 3.3- General RL training phase

• Architecture

Most of the foundational knowledge you need to understand how such a model works is available in our book, Hands-On Large Language Models.

Recap: How LLMs are trained

Just like most existing LLMs, DeepSeek-R1 generates one token at a time, except it excels at solving math and reasoning problems because it is able to spend more time processing a problem through the process of generating thinking tokens that explain its chain of thought.

The following figure, from Chapter 12 of our book shows the general recipe of creating a high-quality LLM over three steps:

1) The language modeling step where we train the model to predict the next word using a massive amount of web data. This step results in a base model.

2) a supervised fine-tuning step that makes the model more useful in following instructions and answering questions. This step results in an instruction tuned model or a supervised fine -tuning / SFT model.

3) and finally a preference tuning step which further polishes its behaviors and aligns to human preferences, resulting in the final preference-tuned LLM which you interact with on playgrounds and apps.

DeepSeek-R1 Training Recipe

DeepSeek-R1 follows this general recipe. The details of that first step come from a previous paper for the DeepSeek-V3 model. R1 uses the base model (not the final DeepSeek-v3 model) from that previous paper, and still goes through an SFT and preference tuning steps, but the details of how it does them are what's different.

There are three special things to highlight in the R1 creation process.

1- Long chains of reasoning SFT Data

This is a large number of long chain-of-thought reasoning examples (600,000 of them). These are very hard to come by and very expensive to label with humans at this scale. Which is why the process to create them is the second special thing to highlight

2- An interim high-quality reasoning LLM (but worse at non-reasoning tasks).

This data is created by a precursor to R1, an unnamed sibling which specializes in reasoning. This sibling is inspired by a third model called R1-Zero (that we’ll discuss shortly). It is significant not because it’s a great LLM to use, but because creating it required so little labeled data alongside large-scale reinforcement learning resulting in a model that excels at solving reasoning problems.

The outputs of this unnamed specialist reasoning model can then be used to train a more general model that can also do other, non-reasoning tasks, to the level users expect from an LLM.

3- Creating reasoning models with large-scale reinforcement learning (RL)

This happens in two steps:

3.1 Large-Scale Reasoning-Oriented Reinforcement Learning (R1-Zero)

Here, RL is used to create the interim reasoning model. The model is then used to generate the SFT reasoning examples. But what makes creating this model possible is an earlier experiment creating an earlier model called DeepSeek-R1-Zero.

R1-Zero is special because it is able to excel at reasoning tasks without having a labeled SFT training set. Its training goes directly from a pre-trained base model through a RL training process (no SFT step). It does this so well that it’s competitive with o1.

This is significant because data has always been the fuel for ML model capability. How can this model depart from that history? This points to two things:

1- Modern base models have crossed a certain threshold of quality and capability (this base model was trained on 14.8 trillion high-quality tokens).

2- Reasoning problems, in contrast to general chat or writing requests, can be automatically verified or labeled. Let’s show this with an example.

Example: Automatic Verification of a Reasoning Problem

This can be a prompt/question that is a part of this RL training step:

Write python code that takes a list of numbers, returns them in a sorted order, but also adds 42 at the start.

A question like this lends itself to many ways of automatic verification. Say we present this this to the model being trained, and it generates a completion:

• A software linter can check if the completion is proper python code or not

• We can execute the python code to see if it even runs

• Other modern coding LLMs can create unit tests to verify the desired behavior (without being reasoning experts themselves).

• We can go even one step further and measure execution time and make the training process prefer more performant solutions over other solutions — even if they’re correct python programs that solve the issue.

We can present a question like this to the model in a training step, and generate multiple possible solutions.

We can automatically check (with no human intervention) and see that the first completion is not even code. The second one is code, but is not python code. The third is a possible solution, but fails the unit tests, and the forth is a correct solution.

These are all signals that can be directly used to improve the model. This is of course done over many examples (in mini-batches) and over successive training steps.

These reward signals and model updates are how the model continues improving on tasks over the RL training process as seen in Figure 2 from the paper.

Corresponding with the improvement of this capability is the length of the generated response, where the model generates more thinking tokens to process the problem.

This process is useful, but the R1-Zero model, despite scoring high on these reasoning problems, confronts other issues that make it less usable than desired.

Although DeepSeek-R1-Zero exhibits strong reasoning capabilities and autonomously develops unexpected and powerful reasoning behaviors, it faces several issues. For instance, DeepSeek-R1-Zero struggles with challenges like poor readability, and language mixing.

R1 is meant to be a more usable model. So instead of relying completely on the RL process, it is used in two places as we’ve mentioned earlier in this section:

1- creating an interim reasoning model to generate SFT data points

2- Training the R1 model to improve on reasoning and non-reasoning problems (using other types of verifiers)

3.2 Creating SFT reasoning data with the interim reasoning model

To make the interim reasoning model more useful, it goes through an supervised fine-tuning (SFT) training step on a few thousand examples of reasoning problems (some of which are generated and filtered from R1-Zero). The paper refers to this as cold start data”

2.3.1. Cold Start
Unlike DeepSeek-R1-Zero, to prevent the early unstable cold start phase of RL training from the base model, for DeepSeek-R1 we construct and collect a small amount of long CoT data to fine-tune the model as the initial RL actor. To collect such data, we have explored several approaches: using few-shot prompting with a long CoT as an example, directly prompting models to generate detailed answers with reflection and verification, gathering DeepSeek-R1- Zero outputs in a readable format, and refining the results through post-processing by human annotators.

But wait, if we have this data, then why are we relying on the RL process? It’s because of the scale of the data. This dataset might be 5,000 examples (which is possible to source), but to train R1, 600,000 examples were needed. This interim model bridges that gap and allows to synthetically generate that extremely valuable data.

If you’re new to the concept of Supervised Fine-Tuning (SFT), that is the process that presents the model with training examples in the form of prompt and correct completion. This figure from chapter 12 shows a couple of SFT training examples:

3.3 General RL training phase

This enables R1 to excel at reasoning as well as other non-reasoning tasks. The process is similar to the the RL process we’ve seen before. But since it extends to non-reasoning applications, it utilizes a helpfulnes and a safety reward model (not unlike the Llama models) for prompts that belong to these applications.

Architecture

Just like previous models from the dawn of GPT2 and GPT 3, DeepSeek-R1 is a stack of Transformer decoder blocks. It’s made up 61 of them. The first three are dense, but the rest are mixture-of-experts layers (See my co-author Maarten’s incredible intro guide here: A Visual Guide to Mixture of Experts (MoE)).

In terms of model dimension size and other hyperparameters, they look like this:

More details about the model architecture are presented in their two earlier papers:

• DeepSeek-V3 Technical Report

• DeepSeekMoE: Towards Ultimate Expert Specialization in

  Mixture-of-Experts Language Models

Conclusion

With this, you should now have the main intuitions to wrap your head around the DeepSeek-R1 model.

If you felt needed a little more foundational information to understand this post, I’d suggest you pick up a copy of Hands-On Large Language Models or read it online on O’Reilly and check it out on Github.

Other suggested resources are:

• A Visual Guide to Reasoning LLMs by

• DeepSeek R1's recipe to replicate o1 and the future of reasoning LMs by

• A Visual Guide to Mixture of Experts (MoE) by

• Sasha Rush’s YouTube video Speculations on Test-Time Scaling (o1)

• Yannis Kilcher’s DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models (Paper Explained)

• Open R1 is the HuggingFace project to openly reproduce DeepSeek-R1

• Putting RL back in RLHF

• While reading this paper, the Galactica paper from 2022 came to mind. It had a lot of great ideas including a dedicated thinking token.

The SWE-bench task measures AI agents on software engineering tasks at the level of a github issue. It was one of the most important tasks measuring the progress of agents tackling software engineering tasks in 2024. We caught up with two of its creators, Ofir Press and Carlos E. Jimenez, to share their ideas on the state of LLM-backed agents.

All the code examples are available on Github here (1.7K stars so far! Wild!). Maarten and I chose a small model so you can run all the examples on the free Colab instances.

We are floored by the initial reception, Andrew Ng called it “a valuable resource for anyone looking to understand the main techniques behind how large language models are built". Josh Starmer, maker of StatQuest, said "I can't think of another book that is more important to read right now. On every single page, I learned something that is critical to success in this era of language models".

Content Overview

The book is broken down into three parts. Part 1 explains how large language models work. This includes an updated, expanded, and modernized version of The Illustrated Transformer explaining 2024-era transformers. Part 2 is application focused and each chapter addresses a type of use case. Part 3 is for the more advanced users who want to fine-tune models (representation or generation).

Chapter 1 Overview

Chapter 1 paves the way for understanding LLMs by providing a history and overview of the concepts involved. A central concept the general public should know is that language models are not merely text generators, but that they can form other systems (embedding, classification) that are useful for problem solving.

Embeddings are numeric representations that capture the meaning of text -- at the level of a document, sentence, word, or token.

Prior to Transformers, Encoder-Decoder RNNs led the way in text generation and translation

Throughout the book, we color-code language models as either representation models (green with a vector icon at the top right) or generative models (pink with a speech bubble icon). These figures start establishing that distinction. It becomes more important later on.

Chapter 2 Overview

Chapter 2: Tokens and Embeddings lays the foundation to understanding LLMs by breaking down two of their fundamental concepts.

For more advanced readers, we show nuances of tokenization by comparing how different LLMs tokenize a certain string, showing their sensitivity to unicode, multi-linguality, code, numbers...etc -- all things that impact their performance.

The code for visualizing these tokenizations is in the chapter 2 notebook here. You can use it to visualize other models, as long their tokenizers are on the HuggingFace hub. These two concepts open the door to plenty of clever solutions that go beyond text LLMs. The chapter then goes into one of these examples of using them for music recommendation systems (treating songs as tokens and playlists as sentences).

Stay Tuned!

More chapter overviews to come in future posts. I hope you are able to get your hands on it and that you let us know what your experience was like going through it! (Oh and once you’re done, we’d really appreciate if you’re able to review it on the platform you purchased it from or on Goodreads).

Happy reading!
Jay

I’ve recently put together a bunch of videos and collaborated with some of my ML heros to create a course about semantic search with LLMs on Deeplearning AI. Here’s a run down of them, and and update about our upcoming book.

Video: ChatGPT has Never Seen a SINGLE Word (Despite Reading Most of The Internet). Meet LLM Tokenizers

Despite processing internet-scale text data, large language models never see words as we do. Yes, they consume text, but another piece of software called a tokenizer is what actually takes in the text and translates it into a different format that the language model actually operates on. In this video, Jay goes examines a language model tokenizer to give you a sense of how they work.

Video: What makes LLM tokenizers different from each other? GPT4 vs. FlanT5 Vs. Starcoder Vs. BERT and more

One of the best ways to understand what tokenizers do is to compare the behavior of different tokenizers. In this video, Jay takes a carefully crafted piece of text (that contains English, code, indentation, numbers, emoji, and other languages) and passes it through different trained tokenizers to reveal what they succeed and fail at encoding, and the different design choices for different tokenizers and what they say about their respective models.

Course: New course with Cohere: Large Language Models with Semantic Search

It was incredible to collaborate with my heros, Luis Serrano, Meor Amer, and Andrew Ng on this short course. Enrol here: https://bit.ly/3OLOEzo

Here's what you can expect:

- Understand LLM Fundamentals: Deepen your understanding of how large language models work, equipping you to become a more proficient AI builder.

- Enhance Keyword Search: Learn to integrate ReRank, a tool that boosts the quality of keyword or vector search systems without overhauling existing frameworks.

- Leverage Dense Retrieval: Discover how to use embeddings and large language models to enhance Q&A capabilities in your search applications.

- Evaluate and Implement: Gain insights into evaluating your search models and efficiently implementing these techniques in your projects.

- Real-World Application: Work with the Wikipedia dataset to understand how to optimize processes like retrieval and nearest neighbors, providing practical experience with large data sets.

By the end of the course, you will gain a deeper understanding of the fundamentals of how Large Language Models (LLMs) work, enhancing your skills as an AI developer.

Book update

We are working hard on writing Hands-On Large Language Models. It is currently on Early Release on the O’Reilly platform with five chapters (150 pages) available now:

1. Categorizing Text
2. Semantic Search
3. Text Clustering And Topic Modeling
4. Multimodal Large Language Models
5. Tokens & Token Embeddings

Access the Early Release version of the book with a 30-day free trial: https://learning.oreilly.com/get-learning/?code=HOLLM23

Coming up next to the book: The Illustrated Transformer Revisited

Maarten (my co-author) and I have a bunch of chapters in review that are not yet in the Early Release but should be coming in the next few weeks. The one I just finished is titled “A Look Inside Transformer LLMs”. It’s basically revisiting The Illustrated Transformer with the major updates to The Transformer Architecture in the last five years. But it’s focused on text generation LLMs (autoregressive models generating one token at a time).

That chapter contains 39 new figures and I believe explains self-attention in the clearest formulation I know of. Here are some teaser visuals:

The two major steps for self-attention

The queries, keys, values of multi-head self-attention

The more efficient multi-query attention — heads have their individual queries but share keys and values. (Paper: Fast Transformer Decoding: One Write-Head is All You Need)

Transformer adapters are one approach to efficient fine-tuning

Low-Rank adaptation, or LoRA is another method of efficient fine-tuning that relies on reducing large weight matrices to smaller, lower-rank matrices which are often able to compress the size and required storage/memory/compute while maintaining close performance. It works because language models “have a very low intrinsic dimension”. So an efficient version of a 175B model can do a lot with rank = 8, for example. That vastly reduces the size of the matrix and the time needed to fine-tune those parameters

Video: Introducing KeyLLM - Keyword Extraction with Mistral 7B and KeyBERT

Maarten, my co-author, has been creating some excellent LLM software and videos that explain them.

In this video, I'm proud to introduce KeyLLM, an extension to KeyBERT for extracting keywords with Large Language Models! We will use the incredible Mistral 7B LLM and go through several use cases.

…

That’s it for this update. Still a bunch more exciting things coming up I can’t tell you about just yet, so stay tuned!

We aim to distill the main intuitions for using language models for text understanding and generation with LOTs of visuals.

We’ll be posting updates about the writing process on social media and on this newsletter.

I’d love to hear your thoughts and suggestions for what you’d like us to cover! Our outline covers practical use cases that developers need (classification, semantic search, topic modeling, text generation, and prompt engineering). That will be followed by several chapters explaining the underlying models and training methods themselves.

As we collectively wrap our heads around the new age of AI and how it changes our world, I’d love to share with you:

• LLM University

• Eight key observations from what I’m learning about the space.

LLM University

I’m lucky to get to team up with some of my favorite ML educators to bring you Cohere’s LLM University — a highly accessible visual guide to language models and the important concepts involved in understanding them and using them properly. Check it out now! We’re on standby in the Cohere Discord to answer any questions you may have on the space.

Eight Key Observations on Generative AI

Here are the articles and videos where I walk you through those ideas in a series I call Generative AI is…Not Enough?:

Part 1:

Article: What is Generative AI? 4 Important Things to Know (about ChatGPT, MidJourney, Cohere & future AIs)

Video:

Part 2:

Article: AI is Eating The World - This is Where YOU Can Use it to Compete (AI Product Moats)

Video:

Another project I’m super excited about is that we’ve openly released the embeddings of all of Wikipedia in ten languages. It’s an incredible resource for injecting factual information into anything you build with LLMs.

The Embedding Archives: Millions of Wikipedia Article Embeddings in Many Languages

Hope you enjoy these. Let me know what you think!

Language Models are All The Rage Now

It’s truly fascinating how quickly language models have been developing. Their commercial potential is now bleeding into the mainstream tech industry, and lots of people are starting to pay attention. Hundreds of millions of people are now asking themselves “Now what? What does this mean for my job, my company, my economy?”.

If you’re on this mailing list, you’ve likely been pondering these questions before everybody else. But with each surprising new capability that these models develop, those questions arise again and again.

My latest articles highlight a few of my observations on these developments:

What's the big deal with Generative AI? Is it the future or the present?

1- Recent AI developments are awe-inspiring and promise to change the world. But when?

2- Make a distinction between impressive 🍒 cherry-picked demos, and reliable use cases that are ready for the marketplace
3- Think of models as components of intelligent systems, not minds

4- Generative AI alone is only the tip of the iceberg

AI is Eating The World (Part 2)

5) Maps and Landscapes of AI Technology and Value Stacks

Don’t forget about the business moats:

6) Enterprises: Plan Not for One, but Thousands of AI Touchpoints in Your Systems

7) Account for the Many Descendants and Iterations of a Foundation Model

The data development loop is one of the most valuable areas in this new regime:

8) Model Usage Datasets Allow Collective Exploration of a Model’s Generative Space

Pockets of Moats for Application Layer Players:

The Illustrated Stable Diffusion

Image generation models also went through surprising and rapid developments in 2022. I made this article and video to be the most accessible intro to how diffusion models work and how they’re trained. If you’ve been curious about what goes on under the hood, this shall be a great place to start building the intuitions about their major components.

Talking Language AI YouTube Series

In the last few months, I got to learn from some of the leading creators of NLP tools in a YouTube series called Talking Language AI. Five episodes are now out. They cover tools like BERTopic, Sentence Transformer, TalkToModel, Human Learn, and many more. In episode 4 I also share my experience at attending the EMNLP conference and some of my major takeaways.

More 2022 Videos

I’m also glad to have been able to make a few more videos in 2022 on my YouTube channel. They cover some of the main topics I’ve found fascinating. From Retrieval-augmented Transformers to multimodal models, and to the future data sources of language models after web text.

What are you thinking of?

I’m curious about what you are thinking of with all these recent developments. What questions are you tackling at the moment? Let me know on Twitter.

Subscribe now