How to de-identify health chatbot logs
A step-by-step guide to removing names, phone numbers and IDs from health chatbot transcripts before they reach logs, analytics or model training.
- Global
Key takeaways
- Redact before storage. Cleaning logs after the fact means the identifiers already reached disks, backups and vendors.
- Use rules for structured identifiers (phone numbers, record numbers, emails) and a model for names and addresses.
- Replace identifiers with typed labels such as [NAME] and [PHONE] so transcripts stay useful.
- Measure recall for each identifier type on real, labelled samples in every language you support.
- HIPAA's Safe Harbor method lists 18 identifiers to remove; use it as a checklist even outside the US.
Where should redaction happen?
At the point of capture: in the service that first receives the user’s message, before anything is written to logs, queues, analytics, error trackers or a model provider. Once raw text reaches backups and third-party tools, cleaning it later is slow and incomplete.
Which identifiers should you remove?
Start from HIPAA’s Safe Harbor list of 18 identifiers, which includes names, small geographic areas, dates more specific than a year, phone numbers, emails, record and account numbers, and device identifiers. Add local formats: Indian mobile numbers, Aadhaar-style IDs and local address patterns.
How do you detect identifiers reliably?
| Identifier | Best method |
|---|---|
| Phone numbers, emails, record numbers | Pattern rules with format checks |
| Dates | Rules plus context (“DOB”, “born”) |
| Names, places, organisations | A trained entity recognition model |
| Mixed-language text | Models tested on that language and script |
Combine both: rules are precise on structured data, models catch names in free text. Score with context, for example raising confidence when “MRN” or “DOB” appears nearby.
What should replace an identifier?
A typed label such as [NAME], [PHONE] or [DATE], so analysts and models still understand the sentence. For training data, realistic fake values keep text natural. For dates, a consistent shift preserves intervals between events.
How do you know it works?
Label 200 to 500 real messages per language, run redaction, and measure recall per identifier type. A missed phone number matters more than an over-redacted word, so set a high recall target for direct identifiers and block releases that fall below it.
Frequently asked questions
Is redacted chat data anonymous?
Not automatically. Free text can re-identify people through rare details, places or events. Treat redacted transcripts as personal data unless an expert assessment says otherwise, and restrict who can see them.
Should we redact before sending text to an LLM?
Wherever the task allows it. Many tasks, such as intent detection or summarising symptoms, work fine on redacted text. Where the model genuinely needs an identifier, keep it inside your own environment.
Sources
- Guidance regarding methods for de-identification of PHI under the HIPAA Privacy RuleUS Department of Health and Human Services
- Presidio: data protection and de-identification SDKMicrosoft
- Digital Personal Data Protection Rules, 2025Ministry of Electronics and IT, India · 2025-11-13
Found something out of date? Write to contact@brandlabs.app and we will fix it.