By the end of 2022, ChatGPT had made clear that language models were no longer a research curiosity. By 2023, the trajectory accelerated. GPT-4 passed the bar exam in the 90th percentile. Gemini matched it. LLaMA-2 ran on a laptop. What had once required a cluster of specialized hardware could now fit in a consumer device. The field had moved, with extraordinary speed, from demonstrating that scale produced capabilities to grappling with what those capabilities implied — and what they still lacked.
The gains were real but uneven. Fluency had been solved, to a first approximation. Factual recall was impressive for well-represented domains and unreliable for the rest. Reasoning — the ability to follow a chain of deductions, check intermediate work, and recognize when an approach was failing — remained inconsistent in ways that improved with scale but never quite resolved. The models could write code that worked and code that confidently failed. They could explain the chain rule but miscalculate a specific derivative. They could describe their own limitations without reliably exhibiting them less.
This chapter covers what came next: the emergence of reasoning models that spent compute thinking rather than just generating, the integration of vision and other modalities, the democratization of frontier capability through open weights, the augmentation of language models with retrieval and tools, and the questions that remain stubbornly open after a decade of extraordinary progress.
Frontier Models and the Scale Plateau
GPT-4 was released in March 2023. Unlike its predecessors, OpenAI published almost no technical details about its architecture, training data, or parameter count — a deliberate choice reflecting the commercial value of the frontier. What was published was a technical report focused on capabilities: GPT-4 scored in the 90th percentile on the Uniform Bar Examination, matched or exceeded average human expert performance on most subtasks of MMLU (Massive Multitask Language Understanding), and set new state of the art on a wide range of professional certification exams. On MMLU itself, GPT-4 scored 86.4% against GPT-3.5’s 70%, a gain roughly equivalent to all the progress in the preceding two years.
The capability jump was real, but its source was opaque, and a pattern of reduced technical transparency at the frontier was established. Anthropic’s Claude, Google DeepMind’s Gemini, and the models that followed were introduced through capability demonstrations and safety evaluations rather than the detailed architectural ablations that had characterized the GPT, BERT, and T5 papers. The academic and commercial incentives had diverged.
Beneath the frontier model announcements, a subtler question was accumulating: was the return on pre-training compute decelerating? The Chinchilla scaling laws had predicted smooth power-law improvement over many orders of magnitude. By 2024, reports — mostly informal, rarely published in the manner of the original scaling work — suggested that gains per unit of pre-training compute were shrinking at the frontier. Models grew from 70B to hundreds of billions of parameters, training runs consumed unprecedented quantities of electricity, and benchmark improvements, while real, appeared smaller per order of magnitude of compute than the scaling laws had predicted at smaller scales. Whether this was a true bend in the curve or a temporary plateau before the next data or architecture innovation was contested.
The practical response was not to wait for theoretical clarity but to look for new levers. One was found in a place that, in retrospect, should have been obvious: inference.
Test-Time Compute and Reasoning Models
A language model, in its default operation, produces its answer in a single forward pass through the network, regardless of the difficulty of the question. Solving a differential equation and completing the phrase “the cat sat on the” are treated identically by the generation process: run the model, sample or take the argmax, return the result. This is not how hard problems are solved by humans, or by algorithms. Hard problems warrant more exploration, more verification, more willingness to discard a wrong approach and start again. The question was whether a language model could do the same.
Chain-of-thought prompting (Chapter 7) had shown that generating intermediate reasoning steps before the final answer improved accuracy substantially on arithmetic and multi-step reasoning. The next insight was that this behavior could be trained in rather than elicited by prompting, and that the model could be rewarded not for the correctness of its reasoning trace but for the correctness of its final answer — letting it discover, through reinforcement, which reasoning strategies reliably worked.
OpenAI’s o1 (2024) was the clearest public instantiation of this paradigm. Rather than generating an immediate answer, o1 produced a chain of internal reasoning tokens — sometimes dozens of tokens, sometimes hundreds, depending on problem difficulty — before committing to a response. The amount of computation was not fixed but adaptive: the model spent more time on harder problems. On the AIME (American Invitational Mathematics Examination), a competition benchmark where GPT-4 solved approximately 12% of problems, o1 solved over 83%. The improvement was not from a larger model. It was from more deliberate reasoning.
DeepSeek-R1 (DeepSeek AI, 2025) demonstrated that this approach could be achieved independently, with open weights, and at substantially lower cost. The training procedure used GRPO (Group Relative Policy Optimization), a reinforcement learning algorithm adapted for language models. The reward signal was purely outcome-based: scalar rewards for correct final answers, with no labeled chain-of-thought data and no human feedback on reasoning quality. The model learned effective reasoning strategies through exploration — discovering when to verify calculations, when to backtrack from a failing approach, and when to express uncertainty. On public reasoning benchmarks, DeepSeek-R1 matched or exceeded o1. The weights were released openly.
The scaling question, which had been asked about training compute since Kaplan et al., was now asked about inference compute. How much better does a model become when given more tokens to reason before answering? Early empirical work (Snell et al., 2024) found approximately power-law returns: more inference compute continued to improve performance, with diminishing but nonzero returns across the range studied. This opened a new axis of the allocation problem: given a fixed total compute budget, how much should be spent training the model and how much should be reserved for inference-time reasoning? The optimal answer depended on the task. For problems with verifiable correct answers — mathematics, code execution, formal logic — inference-time reasoning provided outsized returns. For open-ended generation tasks, the gains were less clear.
Multimodal Integration
Language models, despite their name, are not bounded by language. The Transformer’s self-attention mechanism operates on any sequence of tokens, and tokens need not represent words. The extension of language models to images, audio, and video proceeded along two complementary tracks: contrastive vision-language pretraining to align visual and textual representations, and visual instruction tuning to connect those representations to conversational language models.
CLIP (Contrastive Language-Image Pre-training, Radford et al., 2021) established the foundation. Trained on 400 million image-text pairs scraped from the internet, CLIP learned to represent images and texts in a shared embedding space where semantically related pairs were close. The training objective was contrastive: for a batch of image-text pairs, maximize similarity for the matching pairs and minimize it for the non-matching pairs,
where is cosine similarity and is a learned temperature parameter. The resulting visual representations transferred remarkably to downstream tasks zero-shot: classify an image by computing its similarity to text embeddings of candidate class names, with no task-specific training.
LLaVA (Visual Instruction Tuning, Liu et al., 2023) demonstrated that connecting a CLIP vision encoder to a language model required surprisingly little additional machinery. A linear projection mapped CLIP’s visual embeddings into the language model’s embedding space; the combined system was fine-tuned on visual instruction-following data generated by GPT-4 from image captions. Despite the simplicity of the connection, LLaVA produced capable visual question answering and image description — suggesting that the hard work of aligning visual representations to language had been done by CLIP’s pretraining, and the language model’s conversational capabilities transferred to vision almost for free.
Gemini (Google DeepMind, 2023) and GPT-4V (OpenAI, 2023) showed what frontier compute and data could achieve. Gemini’s key architectural claim was native multimodality: trained simultaneously on text, images, audio, and video rather than grafting a vision encoder onto a text-only backbone. Its Ultra variant matched GPT-4 on text benchmarks while adding competitive visual and audio capabilities. The detailed architecture was not disclosed, but the result established that handling multiple modalities in a single model was not a distraction from language capability — in many settings it complemented it.
The practical reach of multimodal models extended well beyond image description. Models could reason about scientific figures, interpret diagrams, solve visual mathematics problems, read handwritten text, and understand spatial relationships in photographs — tasks that any text-only model was simply unable to approach. What remained debated was whether these capabilities reflected genuine visual understanding or sophisticated pattern-matching over a vast visual training distribution. The question had no clean empirical resolution: the test cases that would distinguish genuine from apparent understanding were precisely the out-of-distribution ones where both interpretations predicted failure.
The Open-Source Ecosystem
For most of the history described in this series, frontier language models were accessible only through research publications, API calls, or not at all. The release of LLaMA (Touvron et al., 2023) broke this pattern.
LLaMA was trained by Meta AI and distributed as a research artifact with open weights. At 65 billion parameters, LLaMA-65B was competitive with GPT-3 175B on many benchmarks despite being substantially smaller — a direct consequence of Chinchilla-optimal training: the same compute budget, allocated toward a smaller model trained on more data for longer, produced better performance than a larger model trained for fewer steps. Within days of its restricted research release, the weights had spread to the public internet; within weeks, the open-source community had fine-tuned, quantized, and run LLaMA on consumer hardware.
Mistral 7B (Jiang et al., 2023) demonstrated that careful architecture choices — grouped-query attention to reduce the KV cache, sliding window attention for efficient long-context handling — could make a 7B model match much larger models trained with less attention to data quality and training efficiency. The implication was that parameter count was not the only axis on which capability was defined: a well-engineered smaller model could consistently outperform a carelessly trained larger one.
LLaMA 2 followed with a commercial license; LLaMA 3 (Meta AI, 2024) completed the transition to genuine frontier competition. Trained on over 15 trillion tokens with a 128K-token context window, LLaMA 3 405B was competitive with proprietary frontier models on public benchmarks — a qualitative shift from earlier open models, which had been competitive with proprietary models from one or two generations prior. For the first time, frontier-competitive capability was available as open weights.
The consequences were multiple. For research, open weights meant that mechanistic interpretability, fine-tuning methodology, and alignment work could be conducted on models that actually mattered. For deployment, open weights enabled privacy-preserving local inference, model customization, and elimination of API dependency. For the policy debate over AI governance, open weights made capability diffusion effectively irreversible: a released model cannot be recalled. The latter consequence made the open weights question one of the sharpest points of disagreement in the field, with researchers and policymakers holding genuinely different views about whether the benefits of openness outweighed the risks of proliferation.
Retrieval-Augmented Generation
A recurring limitation of parametric language models — models that store all knowledge in their weights — was the problem of factual currency and specificity. A model trained through a cutoff date could not know about subsequent events. A model trained on the public internet could not know about private, proprietary, or poorly-represented documents. And even for well-represented knowledge, the compression from web-scale text into a finite parameter vector was lossy in ways that produced hallucination.
Retrieval-Augmented Generation (RAG, Lewis et al., 2020) addressed this by giving models access to an external document store at inference time. The architecture had two components: a retriever that mapped a query to a set of relevant passages, and a language model that generated a response conditioned on both the query and the retrieved context. In the original formulation, the retriever used dense passage representations and maximum inner product search,
where is a passage embedding and is a query embedding, with retrieval and generation trained jointly end-to-end.
The appeal of RAG was structural: it separated knowledge storage from language processing. The retrieval index could be updated — with new documents, corrected facts, proprietary data — without retraining the language model. Retrieval also provided an interpretable information pathway: you could inspect which documents the model had accessed and evaluate the quality of its grounding directly. On questions about specific facts, RAG consistently reduced hallucination by anchoring generation to retrieved text rather than reconstructed parametric memory.
The limitations were complementary. Retrieval quality was the upstream bottleneck: if the relevant document was absent from the index or returned below the top results, the model generated without support. RAG could not fix hallucination on questions outside the retrieval index, and it could not force the model to faithfully use retrieved context rather than ignoring it. For tasks requiring genuine synthesis — integrating information from dozens of documents into a coherent analysis — RAG produced less coherent results than a model that had internalized the relevant knowledge deeply. By 2024, RAG had become a standard production component, often combined with long-context prompting: retrieve relevant passages, concatenate them into a 64K or 128K-token prompt, and rely on in-context reasoning for synthesis.
Tool Use and the Agentic Turn
The models described in preceding chapters were primarily reactive: given a prompt, generate a response. The period 2023–2025 saw sustained effort toward agentic behavior — models that could take sequences of actions, observe outcomes, and pursue goals over extended horizons.
The foundation was tool use. Toolformer (Schick et al., 2023) showed that a language model could learn to call external APIs — a calculator, a search engine, a calendar, a translation service — by self-supervised training on examples where useful tool calls were automatically identified and labeled. The model learned when to invoke a tool, what arguments to pass, and how to integrate the result into its generation. Function calling, formalized in the GPT-4 API, operationalized this: the model could emit structured call syntax, the application could execute the function and return results, and generation could continue with the result in context. Combined with sandboxed code execution, this gave models access to arbitrary computation — a model asked to analyze data could write code, run it, observe the output, debug failures, and iterate.
The ReAct framework (Yao et al., 2023) described the general pattern: alternate between reasoning (generating a chain-of-thought trace) and acting (executing a tool call or environmental action), with each step informed by the previous. This interleaving of thought and action produced more reliable performance on tasks requiring information gathering than either pure generation or pure tool use alone. A model told to answer a question about a current event could search for relevant articles, extract the key facts, reason about their implications, and synthesize an answer — with each step correcting the errors of the previous one.
Agents built on this foundation attempted more extended task completion: a coding agent given a bug report that reads the relevant code, generates hypotheses, runs tests, implements a fix, verifies the result, and commits — without human intervention at each step. The gap between these idealized descriptions and reliable practice remained substantial. Agents accumulated errors over long horizons, got stuck in reasoning loops, misinterpreted intermediate tool outputs, and required careful scaffolding for non-trivial tasks. But the trajectory was clear. The question was no longer whether language models could use tools, but how far tool use could extend before the agent’s reliability fell below the threshold of usefulness — and that threshold was rising with each generation.
The Evaluation Crisis
A problem that had been building since BERT became acute by 2023: the benchmarks used to track progress were saturating and had lost validity as measures of genuine capability.
GLUE was surpassed by fine-tuned BERT within months of its release. SuperGLUE lasted two years. MMLU, designed to test expert-level knowledge across 57 diverse domains, was approached by frontier models in 2023 and effectively saturated in 2024. New benchmarks were released, performed well at the moment of release, and surpassed within months. The half-life of a meaningful language model benchmark had shortened from years to quarters.
The deeper problem was contamination: training data for large models included substantial quantities of internet text, and internet text included solutions to benchmark problems. A model that had seen MMLU questions during pretraining was not demonstrating expert-level knowledge of medicine when it answered correctly — it was demonstrating pattern recall. Contamination was difficult to detect definitively and impossible to fully prevent. The practical effect was that benchmark scores became unreliable upper bounds on actual capability: a 90% MMLU score might reflect genuine cross-domain reasoning, or a high-coverage memorization of the test corpus, or, most likely, some mixture of both in proportions that were impossible to separate.
Arena (Zheng et al., 2023) replaced static benchmarks with dynamic human preference judgments: pairwise comparisons between anonymized model responses, aggregated into an Elo-style ranking. This proved more durable than fixed benchmarks — new tasks could be included continuously, and memorization was less useful against questions generated in real time. Its limitations were different: human preference correlates with fluency and agreeableness rather than truth or correctness, which meant Elo-ranked models were, in a precise sense, optimized to seem good rather than to be good. The same sycophancy pressure that RLHF had introduced at training time was reproduced by Arena evaluation.
The evaluation crisis was not merely technical. It reflected the absence of a clear theoretical account of what language model capability was. Predicting text well was necessary but obviously insufficient. Scoring highly on benchmarks was evidence but compromised evidence. Human preference was meaningful but systematically biased. The field was measuring a thing it had not formally defined, with instruments it knew were imperfect, in a competitive environment that created incentives to report the most favorable numbers. Progress was real; the degree to which it was captured by any particular measurement was not.
The Open Questions
After a decade of progress from n-gram models to systems capable of advanced mathematical reasoning, a set of problems remains open in ways that the history of the field gives no reliable basis for predicting.
Hallucination has not been solved. It has been reduced. RAG, better calibration training, and alignment work have lowered the rate of confident false claims. What has not changed is the root cause: language models are trained to produce fluent, contextually plausible text, and truth is not the same as plausibility. A model that outputs “the Battle of Waterloo was fought in 1815” and one that outputs “1820” both satisfy the same fluency criteria; the training signal to distinguish them exists in the data but is diluted across billions of examples of text that never discuss Waterloo at all. Grounding generation in verified truth — rather than in the statistical patterns of a training corpus — remains an unsolved problem.
Long-horizon reliability is a different kind of limitation. Over short horizons — a few hundred tokens, a single mathematical proof, a function implementation — frontier models are often extraordinarily reliable. Over long horizons — planning a multi-week research agenda, managing a complex software project, maintaining coherent reasoning across a book-length document — reliability degrades in ways that are difficult to anticipate. Each individual step may be locally coherent; error compounds over many steps. The agent frameworks described above are attempts to address this through external scaffolding, but the scaffolding does not change the underlying model’s tendency to lose track of earlier commitments as context grows.
The scaling question itself is genuinely open. The test-time compute paradigm offers one new axis: reasoning models spend more compute at inference time and improve, at least on verifiable tasks. Whether inference-time scaling eventually hits the same diminishing returns that pretraining scaling appears to be approaching, and what architectural or training intervention might restore the power-law regime, is unknown. The history of the field does not offer a reliable predictor: every time progress appeared to plateau, a new lever was found. Whether that record is evidence of continued fertility or a finite sequence approaching exhaustion is not a question the data can answer in advance.
What Has Been Built
Ten chapters from Shannon’s entropy to o1’s reasoning traces. The history is a sequence of solved problems: each era compressed a previous limitation into engineering infrastructure, revealing the next one underneath. The n-gram era established the problem and the metric. Distributed representations solved the curse of dimensionality. RNNs introduced sequence memory and revealed the vanishing gradient. Attention eliminated the sequential bottleneck. Pre-training showed what was latent in unlabeled text. Scaling revealed what emerged above certain parameter thresholds. Alignment addressed what the training objective had left unconstrained. Efficiency made the resulting systems deployable at scale.
Each solved problem was real. Each solution exposed the next. Hallucination was invisible when models produced incoherent text; it became a critical problem when they produced fluent text. Long-horizon reasoning was irrelevant with 512-token context windows; it became central with 128K-token ones. The evaluation problem was academic when models scored 50% on MMLU; it became acute when they scored 86%.
The system that has emerged from this sequence is genuinely remarkable — not in the way that progress reports tend to say things are remarkable, but in the more careful sense that it represents a qualitative change in what machines can do with language. Frontier models can engage with arbitrary text across virtually any domain, reason through multi-step problems, synthesize knowledge from thousands of sources, follow complex instructions, generate functional code, and explain their reasoning. These capabilities were not present in any system a decade ago. They were not predicted by any theory that preceded them. They emerged from the combination of an architectural insight (attention), a training paradigm (self-supervised pretraining), a computational resource (internet-scale text), and a persistent willingness to scale.
What has not emerged is reliability in the deeper sense: the ability to be trusted on questions where verification is difficult, to maintain factual accuracy without external grounding, to reason correctly on problems sufficiently removed from the training distribution. Whether these are shallow limitations — engineering problems that will yield to continued progress — or deep ones that point to a representational gap the current paradigm cannot close, is the central open question in the field.
The honest position is that no one knows. The history of the field argues against confident predictions that progress will stall: every apparent ceiling has, so far, had a way through it. But this history also shows that each solution introduced new constraints; there is no reason to assume the sequence converges smoothly rather than encountering a harder obstacle. What the history does suggest, clearly, is that the way to find out is to continue asking precise questions, building careful evaluations, and following the answers wherever they lead — as researchers in this field have done, with remarkable results, for the better part of a century.
The GPT-4 Technical Report is OpenAI (2023). Gemini is described in Google DeepMind (2023), “Gemini: A Family of Highly Capable Multimodal Models.” The o1 system is described in OpenAI (2024), “OpenAI o1 System Card.” DeepSeek-R1 is in DeepSeek-AI et al. (2025), “DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning.” Test-time compute scaling is studied in Snell et al. (2024). CLIP is in Radford et al. (2021), “Learning Transferable Visual Models from Natural Language Supervision.” LLaVA is in Liu et al. (2023), “Visual Instruction Tuning.” LLaMA is in Touvron et al. (2023); Mistral 7B in Jiang et al. (2023). Retrieval-Augmented Generation is in Lewis et al. (2020). Toolformer is in Schick et al. (2023); ReAct in Yao et al. (2023). Chatbot Arena is in Zheng et al. (2023), “Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena.” Contamination and evaluation methodology are discussed in Jacovi et al. (2023) and Sainz et al. (2023).