4. Language and AI

This chapter works at the interface layer of the five layers. Chapter 2 described the machinery; this chapter is about the surface you touch, which for a language model is a text box and, on most days, nothing else. That surface is thinner than it looks. Everything one of these systems will do for you, it does because of something you wrote into it.

Of all the tools in this book, large language models are the ones you are most likely to have used already. They draft emails, summarise readings, fix code, explain concepts, brainstorm, translate, and write assignments that students then hand in as their own. That last use is a genuine problem, and this chapter ends by taking apart the technology most often proposed as the solution to it.

The aim here is narrower than “learn to prompt”. It is to give you an accurate mental model of what happens between your keystroke and the answer, so that you can predict where the system will be reliable and where it will quietly fail you.

What is a language model?¶

A language model is a system trained to predict the next word, or more precisely the next token, given the words before it. That is the whole of the training objective.

Given the input “The capital of Norway is”, the model assigns a probability to every possible next token. The probability of “Oslo” should be high; the probability of “purple” should be low.

To generate text, the model samples one token, appends it to the input, and repeats:

The capital of Norway is ☐ → Oslo The capital of Norway is Oslo ☐ → . The capital of Norway is Oslo. ☐ → It

This is the same loop whether the model has a million parameters or a trillion. The systems in daily use in 2026 are scaled-up descendants of the models that established the recipe between 2018 and 2020 Brown et al., 2020, built on the transformer architecture introduced in 2017 Vaswani et al., 2017. Chapter 2 covers the sampling knobs, temperature and top-p among them, that decide how adventurous each draw is.

Figure: The prompt is split into tokens, and the model gives every token in its vocabulary a probability of coming next, so that Oslo takes most of the distribution and purple almost none. One token is drawn, appended to the prompt, and the loop runs again.

Two consequences follow immediately, and in practice they cause more confusion than anything else in this chapter.

Tokens, not words¶

The model does not see words. It sees tokens, and a token is usually a fragment of a word. Frequent words become a single token, rare words are broken into several, and the split follows the statistics of the training text rather than any linguistic rule. “Oslo” is likely to be one token. A Norwegian surname may well be three or four.

This sounds like an implementation detail. It is not, for three reasons.

• Cost. Commercial models bill per token, in both directions. A long prompt and a long answer both cost you.
• Context length. Every model has a maximum number of tokens it can attend to at once, its context window. Anything outside that window is not merely forgotten; it was never there.
• Language. Tokenisers are fitted to their training text, which is overwhelmingly English. Any other language is chopped into smaller, less meaningful pieces, so it costs more and fits into less. What that does to Norwegian is taken up below.

Tokenisation is also the reason for a family of small, comic failures. A model asked to count the letters in a word, or to reverse one, is working with fragments rather than letters, and often gets it wrong. You would struggle in the same way if you could only see a word three syllables at a time.

Context, not memory¶

A language model has no persistent memory between conversations unless a system has been built around it to provide one. What it has is the context window: a buffer holding the conversation so far, made up of the system instructions, your messages, and its own previous replies. Every reply is generated by reading that whole buffer from the start.

This is why “remember that we are writing a fantasy novel” works inside a chat and does not survive into a new one. The buffer is gone. Assistants that appear to remember you across sessions achieve it by storing notes somewhere else and quietly pasting the relevant ones back into the context before the model runs.

Figure: The context window holds the system instructions and the recent turns and is read from the start for every reply, while earlier turns fall outside it and were never there at all.

It also explains a failure you will meet in the lab. In a long conversation, early instructions compete with hundreds of intervening messages for the model’s attention, and they lose. If an instruction matters, restate it near the end of the prompt rather than trusting that it was established an hour ago.

In-context learning, or “prompting”¶

The striking discovery reported in 2020 was that you can teach a model new behaviour simply by showing it examples in the prompt, with no retraining at all Brown et al., 2020. This is called in-context learning, and it is the mechanism underneath everything marketed as prompt engineering.

Translate to Norwegian.

EN: The library is open.
NO: Biblioteket er åpent.

EN: Where is the train station?
NO: ☐

The model continues the pattern it has been shown. Nothing has been learned in the training sense; the parameters have not moved. The examples simply make the desired continuation the most probable one.

Three prompt patterns¶

Three patterns cover most of what you will need.

• Zero-shot. Just ask. This works well for common tasks the model has seen a great deal of.
• Few-shot. Give two to six worked examples of input and output before the real request. Use this whenever the format of the answer matters, because examples specify format far more reliably than adjectives do.
• Chain-of-thought. Ask the model to work through the problem step by step before answering. This usually improves multi-step reasoning, at the cost of length, and it is worth nothing at all on tasks that have no steps.

A practical prompt template¶

For anything non-trivial, this skeleton holds up across disciplines.

ROLE: You are a [role with relevant expertise].

TASK: [What you want done, in one sentence.]

CONSTRAINTS:
- [Length, style, format]
- [What to avoid]
- [Audience]

CONTEXT:
[Any background the model needs.]

OUTPUT:
[The exact shape you want, with placeholders or an example.]

There is nothing magic about it: it is the brief you would write for a competent freelancer who has never met you. Most prompts fail because they under-specify the output, not because they used the wrong incantation. The tips and tricks page collects the moves that transfer between tools.

The failure modes you need to recognise¶

Language models fail in characteristic, recognisable ways. You will meet all six of these during the semester, and at least two of them in this week’s lab.

Hallucination¶

The model produces a confident, fluent statement that is false. It invents a paper title, a court case, a quotation, a statistic, a function that does not exist in the library you are using. This is not a bug awaiting a patch. It is a direct consequence of a training objective that rewards plausibility, and plausibility is not truth Bender et al., 2021.

Three mitigations, in order of usefulness:

• Provide grounding. Paste the source and ask the model to answer from it rather than from its parameters. This is the single largest reduction in error you can buy.
• Restrict the task to what the model is structurally good at: rewriting, summarising, reformatting, and critiquing text you supplied.
• Ask for sources and check them. Note the second half. The model can hallucinate a citation as easily as a fact, and a fabricated reference with real authors and a real journal is the hardest kind to catch.

Sycophancy¶

The model agrees with you, including when you are wrong. Insist confidently that a correct answer is mistaken and it will often capitulate and apologise. This is a side effect of training procedures that reward answers people rate as helpful, and helpfulness and agreement are hard to separate in ratings data.

Do not lead the witness. Ask “is the following correct?” rather than “I think this is right, isn’t it?”, and ask explicitly for the strongest counterargument.

Verbosity¶

You get three paragraphs where one sentence would do, padded with restatements of your own question. Ask for fewer words, specify the exact format, and give a word count. Models follow length instructions reasonably well in 2026, and much better when the instruction is a number.

Style drift¶

Over a long generation, tone and register wander. The model starts writing like a research paper and ends writing like a press release. Regenerate from a fresh prompt every few hundred words, and keep a short style example in the prompt rather than a style description.

Arithmetic and counting¶

Language models are not calculators. They will confidently get 17 × 23 wrong, miscount items in a list, and sort things almost but not quite correctly. Chat products in 2026 work around this by giving the model a code interpreter, which is a genuine fix because the arithmetic then happens in Python rather than in the sampler. If anything numeric matters to you, check that the model is actually running code and not narrating it. Chapter 9 goes into what that changes.

Out-of-date knowledge¶

Every model has a training cutoff, after which it knows nothing except what you or a search tool put in its context. It will rarely volunteer this. If recency matters to your question, either check the cutoff or supply the recent material yourself.

Open vs closed models¶

You will work with two kinds of model this semester, and the distinction is about access to the weights rather than about price.

Closed models are reached through a website or a web API. The parameters stay on the provider’s servers, the training data is usually undisclosed, and the model can be changed or withdrawn under you without notice. These have been the most capable systems on most benchmarks throughout the period covered by this book.

Open-weight models are published as files you can download and run on your own machine or your institution’s servers. “Open weights” is not the same as “open source”: you generally get the parameters and a licence, not the training data or the training code. As of 2026 the strongest open-weight models trail the strongest closed ones by something like six to eighteen months on general tasks, and the gap has been narrowing rather than widening.

Pragmatic guidance:

• For fast, high-quality drafting, a frontier closed model is usually still the best tool.
• For research, reproducibility, teaching, or sensitive data, prefer an open-weight model you can run yourself with a local runner. A model that lives on your laptop cannot send a colleague’s unpublished manuscript anywhere, and it will still behave the same way in five years, which is a requirement for any result you want to be able to repeat.
• For anything you will be judged on, record which model and which version you used. “I used a chat assistant in March” is not a method section.

Chapter 3 takes up the ethical and political side of this, including what it means that a handful of organisations can afford to train the models everyone else builds on. The section below is about a narrower question: what the choice looks like from inside a small language.

Norwegian and other small languages¶

Almost everything written about language models is written from inside English, and the defaults follow. It is worth spelling out what changes when you work in Norwegian.

You pay more for the same text. Because tokenisers are fitted to predominantly English training data, Norwegian is split into more and smaller pieces. A 2023 study of widely used tokenisers found that the same text can cost substantially more tokens in other languages than in English, with the gap widening the further a language sits from English in script and structure Petrov et al., 2023. Norwegian is close enough to English to escape the worst of it and still pays a real surcharge; Sámi languages fare considerably worse. That surcharge lands twice: on your bill, and on your context window, which holds proportionally less of your document.

You get weaker output. Norwegian is a small fraction of a per cent of the text these models are trained on. That shows up as fluent prose with the wrong idiom, bokmål quietly leaking into nynorsk, dialect handled poorly, and institutional vocabulary invented rather than recalled. Bokmål and nynorsk are the two written standards of Norwegian, and mixing them is the kind of error a Norwegian reader notices at once. The failure is dangerous precisely because it is not obvious. The grammar is usually fine. It is the register and the terminology that are off, and only a competent speaker will notice.

Nobody else will fix this for you. Five million speakers is not a market that justifies a frontier lab’s attention. A language whose digital capability depends on the roadmap of a foreign company has a governance problem as well as a technical one.

This is the argument for building locally, and Norway has been doing so. The NB AI Lab at the National Library of Norway trains Norwegian language and speech models on the library’s digitised collections and publishes the weights openly Norwegian National Library, 2024Norwegian National Library, 2024. The NorwAI centre at NTNU works on Norwegian language models with industrial and public-sector partners NorwAI, 2026. Between them they have made Norwegian one of the better-served small languages.

Open weights are what make that work compound. A published model can be evaluated independently, and fine-tuned by a hospital or a municipality on data that must never leave the building. It can also be archived so that a result from 2026 can still be reproduced in 2036, and improved by someone other than its authors. A hosted service offers none of those, and for a language this size the alternative to open models is not a better closed one. It is nothing.

from openai import OpenAI

client = OpenAI()  # reads OPENAI_API_KEY from the environment

resp = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[
        {"role": "system", "content": "You answer in one sentence."},
        {"role": "user", "content": "What is the etymology of the word 'fjord'?"},
    ],
    temperature=0.2,
)
print(resp.choices[0].message.content)

How to write with an LLM¶

A working pattern that holds up across disciplines, and the one this course expects you to be able to describe:

1. Think first. Make a bullet outline yourself. Do not ask the model to brainstorm from a blank page; that route leads to the average of everything ever written on the topic, which is exactly the prose people have learned to recognise and discount.
2. Use the model to argue with the outline. “What is missing? What is wrong? Which reader would object, and on what grounds?” This is where these systems are genuinely strong, and it costs you nothing in authorship.
3. Draft it yourself. Write a rough version of each section in your own words, however ugly.
4. Use the model to edit. “Halve this paragraph. Make this sentence clearer. Give me three alternative openings.” Narrow, local, verifiable instructions.
5. Verify every factual claim against an original source, including the ones you agreed with.
6. Keep your prompts. In a file, beside the draft. You will need them for the process memo, and you will want them again in six months.

Treat the model as a fast, slightly drunk colleague: quick, well read, opinionated, occasionally brilliant, and never to be trusted on anything that matters without a check. That framing will get you further than either of the two slogans, just autocomplete and it understands, that chapter 2 weighed against each other.

This week’s lab: Explore, Reflect, Create¶

This lab has two halves: finding out how the model fails, and building something that survives the discovery.

Explore (about 30 min)¶

1. Hallucination hunt. Ask a chat assistant for five academic references on a topic in your field that is narrow enough to be obscure. Then try to find each one. How many exist exactly as given? How many are partly real: right authors and wrong title, right title and wrong year, real journal and invented volume? Record the pattern, not just the score, because the pattern is what tells you which parts of a citation the model is reconstructing and which it is fabricating.
2. Two-model comparison. Take one short writing task from your own discipline, a single paragraph. Run the identical prompt through two different models, ideally one commercial chat assistant and one open-weight model run locally. Compare the outputs on four axes: factuality, tone, length, and confidence. Save both outputs with the model name and date beside them.
3. Optional. Run your prompt and one paragraph of Norwegian through the tokeniser explorer and see what the token count difference costs you.

Reflect (about 15 min)¶

Work in pairs, then in plenary. This is a discussion, not a writing block.

1. Show your partner one place in the lab where the model clearly helped and one where it clearly got in the way. Try to say why in terms of this chapter: was it grounding, context length, sycophancy, a training cutoff?
2. Compare hallucination hunts. Did the two of you see the same failure pattern, and would either of you have caught it if you had been in a hurry?
3. Close the round by stating aloud the three tasks your prompt library will cover. Say them as tasks you actually do, not as categories.

Create (about 45 min)¶

Build a personal prompt library for your discipline. This is one of the most useful artefacts you can leave this course with, and you will keep using it after week 12.

1. Take the three tasks you named in the Reflect round, for example a paragraph for a project report, an explanation of a technical concept for a non-specialist, a critique of a paper, a translation, or a summary of a meeting.
2. Write each one as a reusable prompt on the ROLE / TASK / CONSTRAINTS / CONTEXT / OUTPUT template above, with {{placeholders}} for the parts you will swap in next time.
3. Test each template on one concrete instance and paste the output underneath it, so the file records what the prompt actually produces rather than what you hope it produces.
4. Add one line per template saying which failure mode it is most exposed to and what you will check.
5. Commit prompt-library.md to your portfolio.

A2 starts this week. AI-assisted text in your discipline is set now and due in week 5: 800–1 200 words in a genre of your choosing, plus a one to two page reflection documenting your prompts and your edits. The prompt library is what you take into A2, not a substitute for it. A2 asks for a finished piece of writing; the library is part of the toolkit you write it with, and the reflection is where you say which templates you used and where you overruled them.

At home, write this week’s entry in your practice log using the practice log template.

A critical look: do AI-text detectors work?¶

The claim. Software can reliably tell whether a piece of text was written by a person or by a language model, and an institution can therefore use a detector score to decide whether a student cheated.

The evidence. The strongest evidence runs against the claim. A 2023 study ran essays by non-native English writers through seven widely used detectors and found that a majority were classified as AI-generated, while essays by native-speaking writers were classified almost perfectly Liang et al., 2023. The mechanism is not mysterious: detectors key on low perplexity and low variation in sentence length, which is also what writing in a second language looks like. The same study showed that asking a model to rewrite AI-generated text in more literary language dropped detection to near chance, so the defence is defeated by one extra prompt. Meanwhile one of the largest providers of these models was widely reported to have withdrawn its own detector in 2023, citing low accuracy.

The method. Look at how the reported accuracies are produced. A vendor evaluates its detector on a test set it assembled: a fixed set of generators, a fixed set of prompts, no adversarial paraphrasing, and human text drawn from a narrow and usually well-edited pool. Under those conditions high accuracy is easy and close to meaningless, because none of the conditions hold in a real course. Real student text comes from writers of very different fluency, and real AI-assisted text has usually been edited by a human before anyone sees it. Very few published figures come from an evaluation the vendor did not design.

The limits. Even a genuinely good detector runs into base rates. A one per cent false-positive rate sounds excellent until you apply it across a cohort of four hundred students submitting weekly. At that scale the system manufactures dozens of false accusations over a twelve-week semester, and it will concentrate them on the students least able to contest them. So a detector score is not evidence of misconduct. It is a probability estimate from a classifier of unknown calibration on text unlike its test set, and it cannot distinguish “written by a model” from “written by someone whose prose is plain”.

The honest alternative is not a better detector but a different question. Instead of asking whether AI was used, ask that it be declared, and ask for the process, the prompt log, the drafts, and the record of what was changed and why. That is why this course requires declaration and a prompt log rather than running your work through a detector, and it is why the process memo asks where you exerted your own will over the output. Chapter 3 sets out the wider policy argument.