Plain-language answers to the AI questions that keep coming up. Updated as the field moves.
Start from the constraints rather than the leaderboard. Write down your latency budget, your cost ceiling per request, your context requirement, whether you need tool calling or structured output, and where the data is allowed to go. Those five usually eliminate most candidates. Test the remaining two or three on fifty real cases from your own traffic, compare per-case rather than on average, and pick the cheapest one that passes.
Diagnose the failure first. If the model does not know something, that is a knowledge gap and retrieval fixes it. If the model knows but answers in the wrong shape, tone or format, that is a behaviour gap and prompting fixes it first, fine-tuning second. If the model fails at a specialised skill after both, fine-tuning is the remaining option. Prompting is cheapest and reversible, retrieval is the right default for facts, and fine-tuning is the most expensive and least reversible of the three.
AI accelerators draw far more power per rack than traditional servers, and that power has to be delivered, cooled and paid for continuously. Training a large model is a one-off spike; serving it to millions of users is a permanent load, and inference is what dominates energy use over a deployed model's life.
Retrieval-augmented generation is a pattern where the system searches your own documents for passages relevant to a question, puts those passages into the model's prompt, and asks the model to answer using them. The model's weights never change; the knowledge arrives as context at request time.
An AI agent is a language model that has been given tools it can call, a goal to pursue, and permission to take several steps without asking a human between each one. A chatbot answers a question and stops; an agent keeps acting until it decides the goal is met or it runs out of budget.
Open weights means the trained model file can be downloaded and run yourself, under whatever licence the publisher chose. Open source is a stricter legal standard requiring freedom to use, study, modify and redistribute without restrictions on field of use. Many widely used models are open weights but not open source.
Almost every AI API bills per million tokens, with separate prices for input and output. Output usually costs several times more than input. Cached input, batch processing and smaller models can each cut the bill substantially, and the total for a conversation grows with history because most APIs re-send the whole thread every turn.
A context window is the maximum amount of text, measured in tokens, that a model can consider in a single request. It holds the system instructions, the conversation so far, any documents you paste in, and the answer being generated. When the total exceeds the limit, something has to be dropped or summarised.