Context Windows, Bias and Privacy

Three more limits shape how you should work with an LLM: how much it can read at once, the biases it absorbed, and what happens to the data you give it.

The context window

The context window is the maximum number of tokens a model can consider at one time. It includes everything: instructions, the whole conversation so far, any documents you pasted, and the answer being written.

• If a model's window is 8,000 tokens and your prompt already uses 6,500, at most 1,500 tokens are left for the reply.
• In a long chat, the oldest messages may be dropped or summarised to make room — the model can "forget" what you said at the start.
• A bigger window lets the model take in more, but it does not guarantee it uses every detail well; important points buried in the middle of very long inputs can be missed.

Tip: put the most important instruction clearly at the start (and repeat key constraints at the end of long prompts), and start a fresh chat when the topic changes.

Bias

A model learns from human-written data, so it can learn human biases too — stereotypes, one-sided views, or the patterns of unfair past decisions. If historical hiring data favoured one group, a model trained on it can score other groups lower.

• Do not let a model make decisions about people on its own.
• Check outputs for fairness, and ask for other perspectives.

Privacy and confidentiality

What you type into an AI service is sent to that provider. Depending on its policy and your settings, it may be stored, reviewed or used to improve models.

• Never paste passwords, API keys or other secrets.
• Do not share personal data about customers or colleagues, or confidential company documents, unless your organisation has approved that tool for that data.
• Remove names and identifying details when you only need help with the structure of a problem.

A checklist for responsible use

1. Is this a task where a plausible-but-wrong answer is acceptable? If not, verify.
2. Did I give the model the source, or am I relying on its memory?
3. Does anything here change over time (the knowledge cutoff)?
4. Could the answer affect people unfairly?
5. Am I allowed to share this data with this tool?

Key takeaways

• The context window counts prompt, conversation, documents and reply together.
• Models can repeat biases from their training data; keep humans in charge of decisions about people.
• Treat anything you type into an AI tool as shared with its provider.