The uncomfortable fact behind most enterprise AI programmes is that the useful prompts are the ones full of customer data. “Summarise this ticket”, “draft a reply to this email”, “explain this invoice” all carry names, account numbers and sometimes a great deal more. Each one is a disclosure to whichever vendor hosts the model.
What counts as PII and PHI in a prompt?
More than teams expect. Under the GDPR, personal data is any information relating to an identified or identifiable person, and special categories (health, biometrics, religion, sexual orientation, political opinions and others) carry extra restrictions. Health data counts as a special category, so a diagnosis next to a name is among the most sensitive things a prompt can carry.
| Category | Examples that show up in prompts | Typical placeholder |
|---|---|---|
| Direct identifiers | Names, email addresses, phone numbers, usernames | [NAME_1], [EMAIL_1], [PHONE_1] |
| Government and financial IDs | National ID, passport, tax number, card number, IBAN | [ID_1], [CARD_1], [IBAN_1] |
| Location and dates | Street addresses, postcodes, dates of birth, admission dates | [ADDRESS_1], [DOB_1] |
| Health (PHI) | Diagnoses, medications, record numbers, insurer IDs | [DIAGNOSIS_1], [MRN_1] |
| Credentials and secrets | API keys, passwords, tokens, connection strings | [API_KEY_1], [SECRET_1] |
| Your own entities | Account IDs, order numbers, internal project codes | Custom patterns you define |
Secrets deserve a special mention. Developers paste configuration into assistants constantly. An sk-live- key in a debugging prompt is a credential leak to a third party, whatever the vendor’s retention policy says.
Why is “the vendor doesn’t train on our data” not enough?
- Training is not the only processing. Data sent to a model is processed, logged for abuse monitoring, sometimes retained for a period, and is subject to the vendor’s subprocessors and jurisdictions. A no-training clause addresses one use out of several.
- Data minimisation is a legal principle, not a preference. GDPR Article 5(1)(c) requires that personal data be adequate, relevant and limited to what is necessary. The same reasoning applies to health records. Sending a full ticket when the model only needs the question is hard to defend.
- Prompts leak into outputs. A model that saw a card number can repeat it, in a log, a summary, a tool call, or an answer to a different user with a cleverly phrased question.
- You cannot audit what you did not control. If the application decides what to send, every application is a separate compliance review. If the edge decides, there is one.
How does adaptive masking work, step by step?
- Detect. The outbound prompt is scanned for entities: built-in detectors for names, contact details, IDs, financial and health data, and secrets; plus custom regex entities for anything specific to your business.
- Substitute. Each value is replaced with a typed, numbered placeholder. The same value gets the same placeholder throughout the conversation, so the model can still reason about “[NAME_1]’s order [ORDER_1]” coherently.
- Transmit. The masked prompt goes to the model. The model never receives the real values and cannot echo, store or leak them.
- Restore. When the response comes back, placeholders are swapped for the original values before the application sees it. The application’s code and the user’s experience are unchanged.
- Log. The event records that masking happened, which entity types were found, and the call’s attribution. It does not need to record the values.
Outbound (what your app sent)
"Refund Jane Okafor, card 4111 1111 1111 1111, key sk-live-8a39…"
What the model received
"Refund [NAME_1], card [CARD_1], key [API_KEY_1]"
What your app got back
"Refund issued to Jane Okafor on card ending 1111."The word adaptive matters. A naive redactor blanks everything and ruins the task. Adaptive masking keeps the structure the model needs, including consistent references across turns, while removing the values that matter. Konsole’s masking page shows the before-and-after.
Where should masking happen: in the app, in the SDK, or at the edge?
| Place | Coverage | Consistency | Audit story |
|---|---|---|---|
| In each application | Only apps that implemented it | Varies by team | One review per app |
| In a shared SDK | Apps that upgraded | Good, until a team forks it | Per version |
| At the network edge (harness) | Every call, every app, every model | One policy | One control, one log |
Edge masking wins on the dimension compliance cares about: it is impossible to forget. A new internal tool that calls a model is covered the moment it points at the gateway, before anyone has reviewed it.
How does edge masking map to GDPR requirements?
- Data minimisation (GDPR Art. 5(1)(c)). The model receives only what the task needs. Identifiers are removed by default.
- Security of processing (GDPR Art. 32). Pseudonymisation is named explicitly as an appropriate measure. Placeholder substitution is pseudonymisation applied per call.
- International transfers (GDPR Chapter V). If identifiers never leave your boundary, the transfer analysis for the model vendor becomes dramatically simpler.
- Accountability (GDPR Art. 5(2)). A single edge log that shows masking on every call is evidence you can hand over, rather than a policy document you hope was followed.
Implementation checklist for PII masking on LLM traffic
- Route every model call through one gateway. Confirm with network policy that nothing can reach a model provider directly.
- Turn on masking for the standard entity types, then add custom patterns for your own identifiers.
- Decide per app whether secrets should be masked or blocked outright. A pasted production key is usually a block.
- Keep placeholders consistent within a conversation so multi-turn tasks still work.
- Verify restoration end to end: the user should never see a placeholder.
- Log entity types and counts, not values. Export the log to your SIEM.
- Run the sandbox with real-looking data before go-live, and keep the payloads as regression tests.
- For regulated data, run the harness inside your own cloud or data centre so identifiers never cross a boundary you do not own.