https://storage.ghost.io/c/ac/fd/acfd0c66-490d-4a75-84b3-c8090897fb78/content/images/2024/05/DC-headshot-2023-1.jpg
Minneapolis, MN

Deborah Carver

Deborah Carver is the publisher of The Content Technologist. She is an independent consultant on all things digital publishing, specializing in large content-driven websites.

364 posts

Tokens and vector embeddings: The first steps in calculating semantics for LLMs

Tokens and vector embeddings: The first steps in calculating semantics for LLMs

The first step in natural language processing is creating word-numbers, represented as points in space. If this confuses you, you're not alone. Keep reading.

CT No.242: What's your vector, Victor?

How to understand tokens and vector embeddings, for word people.

When words and math collide: Old-school tried and true language processing algorithms

When words and math collide: Old-school tried and true language processing algorithms

Even in the face of "black box" algorithms, the history of artificial intelligence—natural language processing, more specifically—has left plenty of clues.

CT No.241: TF? ID(GA)F: The old faithfuls of language processing

Even in the face of "black box" algorithms, the history of artificial intelligence—natural language processing, more specifically—has left plenty of clues. While we can't understand the full equation, we can see how building blocks create common patterns in how current algorithms process language.

CT No.240: Mad inconvenient infrastructure

Infrastructure, no matter how solid or planned or continuously improved, gets messy. Observing humanity in July feels like the ultimate UX research experiment.

CT No.239: Please automate the tedious part for me

July is prime chart-making season. July 2025 is one more chartmaking season that will not be powered by artificial intelligence, and here's why.

Your link has expired. Please request a new one.
Your link has expired. Please request a new one.
Your link has expired. Please request a new one.
Welcome to The Content Technologist! You've successfully subscribed.
Welcome to The Content Technologist! You've successfully subscribed.
Welcome back to The Content Technologist
Success! You now have access to additional content.