AI Privacy and Security: What to Know Before You Paste Data

What is it?

AI privacy and security covers two related questions:

1. What happens to my data when I use an AI tool — where does it go, who can see it, how long is it kept, is it used to improve models?
2. What can go wrong when AI is part of an application — leaked secrets, manipulated behaviour, over-powerful automation, unreliable output used as truth.

It is mostly ordinary data protection and application security applied to a new kind of component, plus a few risks specific to language models.

Why does it matter?

People routinely paste emails, contracts, source code, customer records, logs and credentials into chat boxes. Once data leaves your control it may be stored, reviewed by staff for abuse monitoring, or — depending on the product and plan — used for model improvement. Meanwhile AI-enabled applications introduce new attack paths: text can now carry instructions that change behaviour (What is Prompt Injection?).

How does it work?

Data you send

• Policies differ between providers and between consumer and business/API plans (retention periods, training use, human review, regional processing). Read the current terms for the exact product and plan you use; do not assume.
• Sensitive categories need extra care: personal data, health and financial data, client-confidential material, unreleased IP, credentials. Check organisational policy and applicable law (for example GDPR or sector rules).
• Minimise and redact. Remove names, identifiers and secrets before sending when they are not needed for the task.
• Never paste credentials. API keys, tokens, passwords and private keys in prompts, code or logs can be exposed or logged.
• Consider running locally when data must not leave your environment.

Applications you build

• Least privilege. Give the model and its tools only the permissions and data the task needs (Agent Tools: How AI Agents Take Actions). Enforce authorisation in code, outside the prompt.
• Treat model output as untrusted input. Validate it before running commands, rendering HTML, or executing SQL.
• Treat retrieved and external text as untrusted — documents, web pages and emails can contain hidden instructions (What is Prompt Injection?).
• Confirm high-impact actions (payments, deletions, sending messages) with a human.
• Protect indexes. Embeddings and vector stores contain your data in another form (What is a Vector Database?).
• Log carefully. Logs of prompts may themselves contain sensitive data.
• Vet third parties: plugins, MCP servers and model downloads are supply-chain risks.
• Use established references, such as the OWASP Top 10 for LLM Applications, as a checklist.

Example

An engineer wants help debugging a failing API call. Safe workflow:

1. Copy the failing request or log.
2. Strip tokens, cookies and personal data (for requests copied as cURL, remove the Authorization header and similar credentials).
3. Paste the sanitised version and describe the problem.
4. If a real key was ever pasted, rotate it — assume it is exposed.

Unsafe workflow: pasting the raw request with a live bearer token "because it's just a quick question".

When should I use it?

Apply this thinking every time data leaves your device or an AI component gains access to systems. The stakes rise with data sensitivity and with how much autonomy the AI has: a read-only summariser is low risk, while an agent that can send email and query customer data is high risk and needs layered controls.

Common mistakes

• Assuming a chatbot is a private notebook.
• Believing a system prompt can hide secrets or enforce permissions.
• Letting an agent act with the user's full privileges.
• Forgetting indirect exposure via file uploads, browser extensions, connectors and logs.
• Redacting by eye. Use tooling and review; patterns are easy to miss.
• Assuming "anonymised" text cannot be re-identified. Context can identify people even without names.

Related concepts