Short answer: An AI chat assistant is a large language model wrapped in a chat interface. The model splits your message into tokens, then predicts the next token again and again until the reply ends. Training leaves behind statistical patterns, not a stored library of facts. Everything the model can see sits inside one context window.
Type a question, wait a second, and a finished paragraph appears. It reads as though a well-briefed colleague wrote it. The machine underneath is stranger than that, and far simpler. A large language model has one job: guess the next small piece of text. Fluency, structure and apparent judgement all grow out of that single job.
That idea is worth holding on to, because it explains both sides of the technology. It explains why an assistant can draft a speech, translate a menu and outline a project plan. It also explains the confident wrong date and the citation that does not exist. This guide follows the whole path, from your keystrokes to the words on screen.
The one job a language model does
A large language model is not a database, and your question is not a lookup. Training exposed the model to an enormous quantity of text. It did not file that text away for later retrieval. What survives training is a set of parameters: billions of learned numbers that encode how language behaves.
When you ask something, the model builds its reply one token at a time. Each token is chosen because it is probable, given everything before it. The nearest household relative is the autocomplete on a phone keyboard. The difference is scale, and scale changes the character of the thing.
Parameters are the model's only memory of training. Each one is a weight applied somewhere in the calculation that turns an input into a prediction. A model with more parameters can capture finer distinctions, up to a point. Capability also depends on data quality, training length and the alignment work that follows.
Predict the next word well enough, across trillions of words about the world, and you absorb a great deal about the world. Grammar comes first. Then geography, the shape of a legal clause, the rhythm of an apology, the structure of a recipe. Facts arrive as a by-product of prediction. They were never the target, which is exactly why they are sometimes wrong.
A language model does not look answers up. It writes them, one probable token at a time.
From text to tokens to vectors
Tokens: the model's alphabet
Text must become numbers before anything else can happen. Tokenisation splits your message into small pieces called tokens. Common words usually survive whole. Rarer words break apart, so “watch” is one token while “watchmaking” is likely two. Spaces, punctuation and emoji count as tokens as well.
A useful rule of thumb: 1,000 tokens hold roughly 750 words of English. That ratio has commercial consequences. Context limits, message allowances and API prices are all counted in tokens rather than words. Languages written in other scripts often need more tokens for the same meaning, which makes them more expensive to process.
Tip: To estimate tokens before pasting a long document, take the word count and add about a third. A 6,000-word report lands near 8,000 tokens, which tells you at a glance whether it will fit.
Embeddings: meaning stored as coordinates
Each token is then mapped to an embedding, a long list of numbers. Treat it as an address in a space with thousands of dimensions. Training sets those addresses so that tokens used in similar ways end up close together. “Minute” sits near “second” and “hour”. “Geneva” sits near “Zurich”.
Relationships become directions in the same space. The step from “king” to “queen” points the same way as the step from “man” to “woman”. For a language model, meaning is geometry. Reasoning about words means moving through that space in small, learned steps.
Attention: the idea that made this possible
What self-attention actually does
The architecture behind the current boom arrived in 2017. Researchers at Google published a paper titled “Attention Is All You Need”. It introduced the transformer. Almost every well-known model since descends from it, including the GPT series, Claude, Gemini and Llama.
Self-attention is the transformer's central device. As the model reads a passage, every token weighs every other token for relevance. Take the sentence “The watch stopped because its mainspring was broken”. Attention is what ties “its” back to “the watch”. Stack dozens of attention layers and the model can hold a thread across many pages.
Why parallel training let models grow
Attention has a second virtue that matters just as much. Most of it can be computed in parallel, which suits modern graphics processors. Earlier designs read text strictly word by word and trained slowly. The transformer removed that bottleneck.
The trick carries a cost. Comparing every token with every other means the work grows sharply as the passage gets longer. Doubling the input more than doubles the computation. That is why very long context windows stayed rare for years, and why they still cost more to run.
Model size then climbed from millions of parameters to hundreds of billions in under a decade. The same architecture now drives image generation and modern speech recognition. We follow that second branch in our guide to how AI speech-to-text works.
How a base model becomes an assistant
Stage one: pretraining on raw text
Pretraining is a brute-force apprenticeship. The model reads text drawn from the public web, books, reference works and code. It is scored, trillions of times, on predicting the next token. No human labels the data, because the text supplies its own answers. Researchers call this self-supervised learning.
Weeks of computation across thousands of processors produce a base model. It is superb with language and close to useless as a product. Ask a base model a question and it may answer with three more questions. On a page of questions, that is a perfectly plausible continuation.
Stage two: alignment with human feedback
The second stage teaches manners. Instruction tuning trains the model on curated pairs of instruction and good response. That teaches the shape of helpful dialogue. Next comes reinforcement learning from human feedback, usually shortened to RLHF.
Under RLHF, reviewers compare two candidate answers to the same prompt and rank them. The model is then tuned towards the answers people prefer. This stage produces the behaviour you recognise: direct replies, readable formatting, refusal of harmful requests and the occasional admission of ignorance. Products expose different amounts of this work at different prices, a trade-off we weigh in what free and paid AI tools actually give you.
What happens when you press send
The chat you see is an assembly job. The application joins several pieces into one long token sequence and hands it to the model. The model reads all of it and predicts a continuation. Each new token is fed back in as input for the next one, until a stop signal ends the turn.
- The system prompt: hidden standing instructions about role, tone and safety, placed ahead of your message.
- The conversation so far: every earlier turn, replayed in full on every single request.
- Your new message: the only part of the sequence you write yourself.
- Tool results: search snippets, file text or calculator output, pasted in as ordinary text.
Two habits of the machine follow from that assembly. The first is sampling. The model does not always take the single most likely token. A controlled dose of randomness, usually governed by a temperature setting, keeps the prose varied. That is why one question can produce two different answers.
Streaming is the visible side of the same process. The reply arrives word by word because it is genuinely being written that way. Nothing sits finished behind the screen, waiting to be revealed. The model has not planned the last sentence when it writes the first.
The second is the context window, the maximum number of tokens the model can hold at once. In a long chat, the earliest messages eventually drop out of view. The assistant has not forgotten them in any human sense. It simply cannot see them any more.
The parameters themselves are frozen. They stopped changing on the day training finished, which is why every model has a knowledge cutoff. Tools patch that gap by placing fresh text inside the window. Your message is the part of the window you control, which makes prompt craft the main lever you hold. Pulling that lever deliberately is the whole subject of writing prompts that get better answers.
Tip: When a long conversation starts to drift, open a fresh one and paste in only the facts that matter. A clean context window almost always beats a crowded one.
Where the answer actually comes from
Several kinds of system can answer a typed question, and they work in completely different ways. Knowing which one is in front of you tells you what to check. The table below sets the main options side by side.
| System | Where the answer comes from | Strongest at | Typical failure |
|---|---|---|---|
| Large language model | Patterns learned in training, plus the text in the context window | Drafting, rewriting, summarising, translating, explaining | Confident detail that was never true |
| Web search engine | An index of pages, ranked for relevance | Finding sources and very recent events | You still have to read and judge the results |
| Database or spreadsheet | Exact records stored on purpose | Precise recall of known values | Returns nothing when the query does not match |
| Assistant with retrieval | Your documents, fetched at question time and read by the model | Answers grounded in your own files | Wrong documents lead to a wrong answer |
The table explains a common complaint. People ask a model for an exact quotation or a journal reference, then discover it was invented. Precise recall is the weakest corner of a prediction engine, because plausible and true are different targets. We take that failure apart in why chatbots make things up.
Reading the machine well
Seen clearly, a language model is neither a mind nor a conjuring trick. It is an instrument with a known mechanism, real range and legible limits. Instruments reward the people who understand them. A keyboard is no different from a watchmaker's bench in that respect. We follow that parallel in why craftsmanship still matters in the age of AI.
Three habits follow directly from the mechanism.
- Verify anything countable. Names, dates, prices, statutes and citations sit exactly where prediction is weakest.
- Supply context instead of assuming it. The model knows nothing about your project until you paste it in, a discipline that also underpins using AI assistants for research and study.
- Watch what you paste. Your text usually leaves your device, so treat a prompt like an email. The reasoning behind that rule sits in data, privacy and good AI habits.
Key takeaways
- One mechanism underneath. A large language model generates text by predicting the next token, over and over.
- Words become geometry. Tokenisation splits text into pieces, and embeddings place those pieces in a space where similar meanings sit close together.
- Attention unlocked the scale. The 2017 transformer weighs every token against every other and trains in parallel.
- Assistants are made, not born. Pretraining builds raw capability; instruction tuning and RLHF shape it into helpful behaviour.
- The context window is the model's world. It sees your prompt, the chat so far and any tool output, and nothing else.
The machinery will keep changing. Models now read images, listen to audio, call tools mid-answer and hold far longer contexts. The foundation is unlikely to move, because learned prediction over human language keeps proving useful. Neighbouring mechanisms, from speech recognition to retrieval, are gathered on our how AI works topic page. For the wider history of the field, Britannica's overview of artificial intelligence is a sound starting point.
The best way to test any of this is on a real task. Ask an assistant such as ASKAI.FREE the same question twice and watch the wording shift. Then paste in a paragraph of your own and watch the answer sharpen. From there the AI Assistants hub branches into research technique and everyday automation.
