Deborah Carver

A late 2025 series aiming to break down the technologies that underpin generative AI, Large Language Models, Natural Language Understanding, and Information Retrieval. Featuring absolutely no math, minimal technical jargon, and lots of pop culture.

Transformers take static vector embeddings, which assign single values to every token, and expand their context, nearly simultaneously as they process the context of every other word in the sentence. But who cares, let's listen to a pop song!

Transformers take static vector embeddings, which assign single values to every token, and expand their context, nearly simultaneously as they process the context of every other word in the sentence. But who cares, let's listen to a pop song!

The first step in natural language processing is creating word-numbers, represented as points in space. If this confuses you, you're not alone. Keep reading.

How to understand tokens and vector embeddings, for word people.

Even in the face of "black box" algorithms, the history of artificial intelligence—natural language processing, more specifically—has left plenty of clues.