How to de-identify workplace data without destroying what makes it useful

If you are thinking about licensing your company's records for AI training, sooner or later you will need to de-identify that workplace data, and that is where a real tension shows up. The data is valuable because it shows real people doing real work. It is sensitive for exactly the same reason.

Removing personal information is not optional. It protects your employees and customers, it protects you legally, and it is the only fair way to do this at all. Recent coverage of startups selling their old chats and emails, including this piece in Fast Company, has rightly pointed out that the people who wrote those messages never expected them to end up in a training set.

The trouble is that careless de-identification can ruin the data. Strip out every name and you lose track of who asked whom for help. Delete every date and you lose any sense of how long things took. This guide walks through a way of doing it that protects people and still leaves something worth learning from.

Start with consent and rights, then clean

De-identification is one layer of protection. It works alongside consent rather than standing in for it.

Before any cleaning starts, be clear about who needs to be told and who needs to agree. That usually includes current and former employees whose messages are in the archive, and it can include customers whose details sit in your tickets or CRM. Contracts with clients sometimes limit what you can do with their data no matter how much you remove from it.

In Europe and the UK, the GDPR treats truly anonymous data differently from data that can still be linked back to a person. The UK regulator's guidance on anonymisation is a readable explanation of where that line sits, and it is worth an hour of your time before you decide how far the cleaning needs to go. If you are unsure, talk to a lawyer who knows data protection.

Know what you are looking for

Personal information in workplace data comes in a few forms, and each one needs a slightly different approach.

The obvious ones are direct identifiers. Names, email addresses, phone numbers, employee IDs, customer account numbers and home addresses all fall into this group. They are relatively easy to find because they follow recognizable patterns.

Then there are indirect identifiers, details that do not name anyone on their own but can point to a person when combined. A job title, an office location and a start date together might describe exactly one employee. A customer's company size, city and industry can do the same.

The hardest category hides in free text, in email signatures, in a casual line like "say hi to Priya when you see her", in pasted screenshots and in attachments such as CVs. Automated tools miss these more often than you would hope.

Technical secrets deserve attention too. Code repositories and engineering chats often contain passwords, API keys and internal addresses that should never leave the company, whatever else happens to the data.

Replace things rather than deleting them

The most useful idea in this whole process is to swap personal details for consistent stand-ins instead of deleting them outright.

Take a line from a support ticket.

Priya asked Tom to refund order 4417 for Acme Ltd after the second failed delivery.

If you delete every name and number, you end up with something like this.

asked to refund order for after the second failed delivery.

That sentence has lost almost everything a model could learn from it. Compare it with a version that uses consistent stand-ins.

Person 12 asked Person 3 to refund order [ORDER 881] for [CUSTOMER 57] after the second failed delivery.

Nobody can tell who Priya or Tom are, but the shape of the work survives. If Person 12 shows up in another thread, it is still the same person, so a model can follow their work across the archive. It can still learn that a support agent asked a colleague to approve a refund for a repeat customer after a failed delivery.

A few related habits help a great deal.

Keep roles wherever you can. Calling someone the support lead or a finance manager tells a model far more than a random label does, and it usually reveals less than a name.

Shift dates instead of removing them. Moving every date in a thread by the same random amount keeps the order of events and the time between them, which often matters more than the dates themselves.

Generalize places and numbers that could single someone out. A city can become a region, and an exact headcount can become a range.

There is one important caveat here. Replacing names with consistent stand-ins is what the GDPR calls pseudonymisation, and pseudonymised data can still count as personal data because someone holding the key could reverse it. If you need the data to be anonymous in the legal sense, the key has to be destroyed and the remaining details have to be general enough that nobody can be picked out another way.

Use tools, then check their work

Automated tools do most of the heavy lifting. Pattern matching reliably catches structured things like email addresses, phone numbers and account IDs. Named entity recognition tools, such as the open source Presidio framework, find names, places and organizations in free text.

They are not enough on their own. They miss unusual names, nicknames, names written in other scripts and anything inside images or attachments. A sensible approach is to run the tools first, then have people review a meaningful sample of the output, paying extra attention to the places where tools tend to struggle. Images and scanned documents either need their text extracted first or need to be left out entirely.

Test whether people can be re-identified

Once the data looks clean, try to break it. Pick a sample and ask whether someone who knew the company could work out who is who. Look for rare job titles, unusual events and very small teams, because those are the details that give people away even when every name is gone.

The US National Institute of Standards and Technology has a useful overview of de-identification techniques and re-identification risk if you want to go deeper.

Keep a record of everything you did

Finally, write down every step. Note which tools ran and with which settings, what was removed, what was replaced, and who reviewed the results and when. That record lets a buyer understand exactly what they are getting, and it lets you show employees, customers or regulators how their information was handled.

We keep this kind of record for every dataset we work with, and our provenance page explains how it fits together.

Doing it properly is worth the effort

Careful de-identification takes time, and it is tempting to treat it as a box to tick before the real work begins. In practice it is part of the real work. A dataset that protects people properly is one a lab can actually use, and one you can stand behind if anyone ever asks how it was made.

If you are considering licensing your company's records and want to understand what preparation would involve, our guide to selling company data to AI labs covers the wider process, and you can tell us about your data whenever you are ready.