12.1.1 — AI, ML, Deep Learning and Generative AI

Cartwise is an online store. Like every online store, it loses money to fraud: somebody steals a credit card number, buys three phones, ships them to an empty flat, and two weeks later the real card owner disputes the charge. The bank takes the money back from Cartwise, the phones are gone, and Cartwise also pays a penalty fee for the dispute.

The first fraud check anybody at Cartwise wrote looked like this:

def is_suspicious(order) -> bool:
    if order.amount > 50_000 and order.account_age_days < 7:      # (1)
        return True
    if order.shipping_country != order.card_country:               # (2)
        return True
    return False                                                    # (3)

Line (1) says that a big order from a brand-new account is suspicious. Line (2) says that a card issued in one country being used to ship to another country is suspicious. Line (3) says everything else is fine.

Somebody sat down, thought about what fraud looks like, and wrote those rules by hand. This works on the first day. Then three things happen, and they happen to every team that writes rules like this.

First, the fraudsters adapt. Within a month they learn that orders under ₹50,000 pass, so they place orders of ₹49,000. Second, the rules start blocking real customers. A student studying abroad uses an Indian card to ship a gift to her parents in India, and line (2) blocks her. Third, the rules multiply. Six months later there are two hundred of them, they contradict each other, nobody remembers why rule 117 exists, and nobody dares delete it.

The real problem is underneath all three. A person can only write rules for patterns they have already noticed.

Now think about what Cartwise already has. It has two million old orders. It also knows which of those orders were fraud, because for each one, the card owner either complained or did not.

Those two million orders hide patterns that no person will ever spot by reading them. Here is a made-up example. Suppose fraud is common when three things happen together. The order is placed in the middle of the night. The account was opened yesterday. The delivery address has received orders under four different names this month. Each fact on its own means very little, because plenty of honest people shop at night. Put the three together, and the order is almost always fraud. A person scrolling through two million rows will never notice that. A computer can count it in seconds.

So the question becomes: instead of a person writing the rules, can the computer find the rules by looking at the examples? That question is the whole of machine learning.

1. Rules written by hand versus rules learned from examples

Here is the difference stated as plainly as it can be. In ordinary programming, a human writes the rules, the computer receives data, and the computer applies the rules to produce answers. In machine learning, a human supplies the data together with the right answers, and the computer works out the rules.

Top row: you write the rules and the computer applies them. Bottom row: you give the computer examples with the right answers, and it produces the rules itself. The thing it produces is called a model, and once it exists it is used exactly like an ordinary program, which is what the green arrow shows.

Look at the top row first. The amber box, the rules, is the part a human wrote. Everything Cartwise knows about fraud has to pass through somebody's head and then out through their fingers into an if statement.

Now look at the bottom row. The amber box has moved to the output side. The human no longer writes it. What the human supplies instead is the green box, the answers: for each old order, whether it turned out to be fraud. Training is the step that reads all those examples and produces the rules. And the green arrow on the right shows the part that surprises people: once the model exists, it sits in the checkout and answers new orders exactly like the hand-written function did. From the outside, a model is just a function. You call it with an order and it gives you back an answer.

What a model actually is

That word "model" will appear on every page of this Part, so it is worth pinning down now in the plainest terms.

A model is a function with adjustable numbers inside it. Here is a tiny but completely real one for Cartwise:

def fraud_score(order, w) -> float:
    return (w[0] * order.amount_in_thousands      # (1) how much the size of the order matters
          + w[1] * order.is_new_account            # (2) how much a new account matters (1 or 0)
          + w[2] * order.countries_differ          # (3) how much a country mismatch matters (1 or 0)
          + w[3])                                  # (4) a starting offset

Lines (1) to (3) each take one fact about the order and multiply it by a number from the list w. Line (4) adds one more number on its own. The function adds all of that up, and a bigger result means more suspicious.

The numbers in w are the adjustable part. They are called the model's parameters, and you will also hear them called weights, because each one says how much weight to put on one fact. If w[1] comes out as 3.0 and w[2] comes out as 0.2, the model is saying that a new account is a much stronger warning sign than a country mismatch.

The structure of the function, meaning which facts it looks at and the fact that it adds them up, is chosen by a person. The values of the parameters are not chosen by a person. They are found by training: a procedure that starts with random values, measures how badly the function scores the two million known orders, nudges every parameter in the direction that makes it slightly less wrong, and repeats that until it stops improving. Chapter 12.2.4 shows that nudging procedure step by step with real numbers. For now, the idea is enough: training is a loop that adjusts numbers until the function's guesses match the known answers as closely as possible.

Using a trained model on a new order is called inference. Training happens once in a while, can take minutes or weeks, and needs the full set of examples. Inference happens on every single checkout, has to finish in milliseconds, and needs only the one order in front of it. That split between an expensive, occasional training job and a cheap, constant inference call shapes almost every engineering decision in this Part.

One more word, because it explains why any of this works. A model is useful only if the patterns it found in old orders still hold for new ones. The ability to do well on examples it has never seen is called generalisation. A model that memorised the two million old orders perfectly but fails on tomorrow's orders has learned nothing useful. Chapter 12.3.3 is entirely about this, because it is the single most common way machine learning goes wrong.

2. The nested sets: AI, ML, deep learning, generative AI, LLMs

These five terms get used as if they mean the same thing. They do not. Each one is a smaller circle inside the one before it, and knowing exactly where the boundaries are saves you from a lot of confusion in meetings.

Each term is a subset of the one around it. Every LLM is generative AI, every generative model here is deep learning, every deep learning system is machine learning, and every machine learning system counts as AI. The reverse is never true: plenty of AI involves no learning at all.

We will walk this figure from the outside in, and for each circle we will answer two questions: what is inside it, and where the name came from.

Artificial intelligence — the outer circle

Artificial intelligence is the broadest term. It covers any machine that does something we would call intelligent if a person did it. That includes things that learn nothing at all. The chess program that beat world champion Garry Kasparov in 1997, IBM's Deep Blue, did not learn chess from examples. Engineers wrote rules for judging a position, and the machine searched through about 200 million positions per second to pick the best move. That is AI. The route planner in a maps app, which searches a graph of roads for the shortest path using the algorithms of Chapter 4.19.3, is also AI in this broad sense. Cartwise's hand-written fraud rules at the top of this page are AI too, in the old-fashioned meaning of the word.

The name comes from a specific document. In August 1955, four researchers, John McCarthy, Marvin Minsky, Nathaniel Rochester and Claude Shannon, wrote a proposal for a summer research workshop at Dartmouth College to be held in 1956. McCarthy needed a title for the proposal, and he chose "artificial intelligence" partly to set the new field apart from cybernetics, an older field about feedback and control that he did not want the project to be confused with. The proposal claimed that "every aspect of learning or any other feature of intelligence can in principle be so precisely described that a machine can be made to simulate it." That sentence is still the field's ambition, nearly seventy years later.

Machine learning — the rules are learned

Machine learning is the part of AI where the rules are found from examples rather than written by hand. Everything in the bottom row of the first figure is machine learning. Cartwise's fraud score, once its parameters are trained from old orders, is machine learning. So is your email's spam filter, a bank's credit score, and the "estimated delivery: Thursday" on a shopping site.

The phrase comes from Arthur Samuel, an engineer at IBM who wrote a program in the 1950s that played the board game checkers. Samuel was not a strong checkers player himself, so he could not write down good rules. Instead he made the program play thousands of games against itself and adjust the numbers in its position-scoring function according to which positions led to wins. It eventually played better than Samuel did. His 1959 paper describing it was titled "Some Studies in Machine Learning Using the Game of Checkers", and the name stuck. Notice how close Samuel's checkers program is to Cartwise's fraud score: a function with adjustable numbers, and a loop that adjusts them according to outcomes. The idea has not changed. The scale has.

Deep learning — many layers, and the features are learned too

To understand what makes deep learning different, you need one more idea: the feature. A feature is one fact about the example that the model is allowed to look at. In the fraud_score function above, the features are amount_in_thousands, is_new_account and countries_differ. Somebody had to decide that those three facts were worth computing and handing to the model. Choosing and computing good features used to be the most time-consuming part of machine learning, and it is called feature engineering. Chapter 12.4.2 is about doing it well.

For an order, picking features is easy because an order is already a neat row of facts. But now think about a photograph of a product that a seller uploaded, and the question "is this photo actually a shoe?" The raw data is a grid of a few million pixel colour values. No single pixel tells you anything. What features would you even hand-write? "Has a curved edge near the bottom"? People spent decades trying, and the results were poor.

Deep learning is machine learning using neural networks with many layers stacked on top of each other. A neural network, which Chapter 12.5.1 builds from a single unit upwards, is a model made of many small, simple functions wired together in layers. The first layer takes the raw pixels. Each later layer takes the previous layer's outputs. The important part is what training does to those layers: the early ones end up detecting simple things like edges, the middle ones combine edges into shapes like curves and corners, and the late ones combine shapes into parts like laces and soles. Nobody programmed those detectors. They emerged during training, because they turned out to be useful for getting the answer right. That is the real meaning of deep learning: the model learns its own features, layer by layer, straight from raw data.

The word "deep" simply means "many layers". A network with two layers is shallow, and one with fifty or a hundred is deep. The term was popularised in the mid-2000s, mostly by the researcher Geoffrey Hinton and his colleagues, partly as a fresh label for neural networks at a time when the old name had a bad reputation from earlier failures. That reputation is the subject of section 3.

Generative AI — producing new content

Most machine learning produces a judgement about an input: fraud or not, a price, a category. Generative AI produces new content instead: a paragraph, an image, a voice, a piece of code. It is not a different kind of mathematics. It is deep learning pointed at a different job. Instead of being trained to output one label, a generative model is trained to output the next piece of something, such as the next word of a sentence or a slightly less noisy version of an image, and then it is run over and over to build up a whole new item one piece at a time.

Large language models — the inner circle

A large language model, or LLM, is a generative model trained on an enormous amount of text to do one thing: given some text, predict what comes next. Claude, GPT, Gemini and Llama are all LLMs. "Large" refers to the number of parameters, which for current models runs from a few billion to well over a trillion, and to the amount of training text, which is measured in trillions of words. "Language model" is an old term from the 1980s speech-recognition world, where it meant any system that assigns a probability to the next word.

It sounds too simple to be useful. How can "predict the next word" produce something that writes code and answers support emails? The answer is that predicting the next word well, across trillions of words of real human writing, forces the model to learn a great deal about the world. To predict the next word of "The capital of France is", it must know geography. To predict the next line of a half-written Python function, it must model what the code does. Chapter 12.7.5 takes this apart properly, and Chapter 12.7.9 explains how a raw next-word predictor is turned into an assistant that follows instructions.

For Cartwise, this inner circle is where the support assistant lives. "Ask Cartwise", which answers customers' questions about returns and deliveries, will be an LLM fed with Cartwise's own policy documents. That is the main thread of Chapters 12.8 and 12.9.

3. Why it took seventy years: the booms and the winters

The ideas above are old. A neural network unit was described in 1943 and a learning machine called the perceptron was built in 1958. So a fair question is why all of this only started working in the 2010s. The answer is a story of two crashes, and the story is worth knowing for a practical reason: it explains why the field now trusts data and measurement over clever theories.

Enthusiasm and funding for AI over seventy years (the height of the curve is illustrative, not a measurement). Two long slumps, the AI winters, followed periods when the field promised more than it could deliver. The modern rise begins in 2012.

The first boom and the first winter. In 1958 the psychologist Frank Rosenblatt built the perceptron, a machine that learned to sort simple images into two groups by adjusting its weights. Newspapers reported it as the embryo of a computer that would one day walk, talk and be conscious of its own existence. Then in 1969 Marvin Minsky and Seymour Papert published a book called Perceptrons that proved, mathematically, that a single-layer perceptron cannot learn some very simple patterns. The most famous example is "exclusive or": output yes when exactly one of two inputs is on. The book is often blamed for killing neural network research for a decade, which oversimplifies it, but the broader mood shifted. In 1973 a report to the British government by the mathematician James Lighthill concluded that AI had failed to deliver on its promises, funding in Britain and the United States dried up, and the period from roughly 1974 to 1980 is now called the first AI winter. "Winter" is the field's own word for a long stretch when money and interest freeze.

The second boom and the second winter. In the 1980s the excitement came back in a different form, called expert systems. These were large collections of hand-written if-then rules, gathered by interviewing human experts, for jobs like diagnosing infections or configuring computer orders. Companies spent heavily on them. But they had exactly the problem Cartwise's fraud rules had: they were brittle, they could not handle any case their rule-writers had not thought of, and keeping thousands of rules consistent became impossible. The market for the specialised computers that ran them collapsed in 1987, and the second winter lasted into the early 1990s.

The lesson the field took from both winters is the same lesson Cartwise learned from its rules. Systems built from human-written knowledge hit a ceiling, because people cannot write down everything they know. Meanwhile, one quiet but important thing happened in the second boom: in 1986, David Rumelhart, Geoffrey Hinton and Ronald Williams published a clear account of backpropagation, a method for training neural networks with many layers. That answered Minsky and Papert's objection, because a network with more than one layer can learn exclusive-or. But there was not yet enough data or computing power to show that it mattered.

What finally changed. Three things arrived together in the 2000s, and the modern field rests on all three. The first was data: the internet produced billions of labelled examples almost for free, and in 2009 a team led by Fei-Fei Li released ImageNet, a collection of over fourteen million photographs labelled by hand into thousands of categories. The second was compute: graphics cards built for video games turned out to be excellent at the kind of arithmetic neural networks need, which Chapter 12.5.7 explains. The third was better recipes: small changes to how networks are built and trained, which you will meet in 12.5, made deep networks stable to train.

The moment the three came together has a date. In 2012, a deep network called AlexNet, built by Alex Krizhevsky, Ilya Sutskever and Geoffrey Hinton and trained on two gaming graphics cards, entered the yearly ImageNet competition. Its top-5 error rate, meaning how often the right label was not among its five best guesses, was about 15 percent. The best entry that did not use deep learning scored about 26 percent. In a competition where teams fought for a single percentage point, that gap ended the argument, and within three years nearly every serious image system used deep learning.

The last two steps lead straight to this Part's later chapters. In 2017 a team at Google published a paper titled "Attention Is All You Need", which introduced the transformer, the network design behind every modern LLM and the subject of Chapter 12.7. Transformers turned out to keep getting better as they were made bigger and fed more text, and in 2020 OpenAI's GPT-3, with 175 billion parameters, showed that a large enough language model could do new tasks from just a few examples in the prompt. On 30 November 2022 OpenAI released ChatGPT, and the rest of the world noticed.

4. Where an AI engineer fits

You will hear three job titles around this work, and they map onto the circles above more neatly than people expect.

A data scientist asks questions of data and builds models to answer them. Is the fraud rate rising? Which customers are about to leave? Their output is often an analysis or a prototype model.

A machine learning engineer takes models into production and keeps them healthy there: the training pipelines, the serving infrastructure, the monitoring that notices when a model has gone stale. Chapters 12.4 and 12.12 are their territory.

An AI engineer is the newest of the three, and the title became common only after 2022. An AI engineer mostly does not train models from scratch. They build products on top of large pre-trained models: they design the prompts, connect the model to the company's data with retrieval, give it tools so it can take actions, measure whether it is actually right, and keep it safe and affordable. "Ask Cartwise" and the refund agent are AI engineering projects.

In practice the boundaries blur, and a strong AI engineer needs the foundations of the other two. When an LLM-based classifier costs too much, you need to know that a small model trained on a few thousand examples (Chapter 12.3) might do the same job for a thousandth of the price. When retrieval returns the wrong documents, you need to understand what an embedding is (Chapter 12.6). When the model says something odd, you need to know how it produces text (Chapter 12.7) to tell whether the cause is the prompt, the data or the sampling settings. That is why this Part starts with the foundations rather than with prompts.

Recall

• In ordinary programming a person writes the rules. In machine learning a person supplies examples with the right answers, and training finds the rules.
• A model is a function with adjustable numbers called parameters (or weights). Training sets those numbers. Using the trained model on new input is inference, and doing well on unseen input is generalisation.
• The circles nest: artificial intelligence ⊃ machine learning ⊃ deep learning ⊃ generative AI ⊃ large language models. Deep learning means many-layer neural networks that learn their own features from raw data.
• The two AI winters (about 1974–1980 and 1987–early 1990s) followed systems built on hand-written knowledge hitting a ceiling. The modern rise came from data, compute and better training recipes arriving together, made visible by AlexNet in 2012, and the transformer of 2017 led to today's LLMs.
• An AI engineer builds products on pre-trained models: prompts, retrieval, tools, evaluation, safety and cost. The foundations of classical ML and deep learning are what let them debug those products.

Self-test: Why does a hand-written fraud rule set get worse over time even if nobody touches it? What exactly is a parameter, and who chooses its value? Name one AI system that involves no machine learning. What does the word "deep" in deep learning count?

Next: 12.1.2 The Kinds of Problems ML Solves — "find the rules from examples" covers very different jobs, and the job you pick decides the model, the data and the way you measure success.