Large language models are the technology behind chat assistants, writing aids, and many AI features in office software. Since the end of 2022, they have reached a broad public and are now among the best-known forms of generative AI. They are of interest to companies because they work with natural language: tasks are described as text, not programmed.
How does a large language model work?
An LLM breaks text down into tokens, that is, words or parts of words. At its core, it is trained to predict, for a given sequence of tokens, which token is most likely to follow next. When this step is repeated, entire sentences and texts are created.
Development takes place in several stages. In pretraining, the model learns grammar, relationships, and factual knowledge from very large text collections. It is then tuned with selected examples and human feedback to follow instructions and give helpful answers. Technically, LLMs are based on the transformer architecture and often have billions of parameters.
Important for understanding: an LLM does not remember anything between individual requests. What it can take into account in a request is limited by the context window. It only knows about events after the end of its training data if current information is provided to it.
Typical examples
- Creating drafts for emails, quotes, or job postings
- Summarizing meeting minutes and long documents
- Translating texts or converting them into plain language
- Pre-sorting service desk requests and providing suggested replies
- Answering questions about internal documents, combined with a search in your own knowledge base
What companies should keep in mind
- Plan for errors: LLMs can generate convincing-sounding but false statements, so-called hallucinations. Important results need expert review.
- Clarify data processing: Before use, check where inputs are processed, whether they are used for training, and which contracts apply.
- Supply your own knowledge deliberately: For questions about internal documents, retrieval-augmented generation is better suited than relying on the model's knowledge.
- Security: If an LLM processes third-party content such as emails or web pages, it may contain hidden instructions (prompt injection).
- Keep an eye on costs: Many services charge by processed tokens. Long inputs and outputs increase costs.
How it differs
An LLM is the model itself. Chat assistants and AI features in applications are products that use one or more LLMs and add a user interface, search functions, rules, and data connections. Two products with the same model can therefore behave quite differently.
View AI assistance