8.7 — Governance, Privacy and the Obligations on Engineers

An email arrives: "Under GDPR, please delete all personal data you hold about me." You have 30 days.

The user row is easy. Then the questions start. Their name is in the search index, their email is in the support ticketing system, their address is in six months of order records the finance team must keep, their identifier is in every access log, their profile is embedded in a dozen documents, and their data is in last night's backup and in eleven months of backups before that. A third-party analytics service also holds an event history keyed on their id.

Nobody can list where the data is, and that is the real problem. Every obligation in this chapter reduces to one prerequisite: knowing what you hold, where it is, and why.

1. Data classification

Classification is the input to every other control. Without it, teams either over-protect everything (expensive, and people work around it) or under-protect the thing that mattered.

Class	Examples	Handling
Public	Marketing pages, docs	No restriction
Internal	Runbooks, roadmaps	Employees only
Confidential	Customer personal data, financials	Encrypted, access-controlled, logged
Restricted	Payment credentials, health data, secrets	All the above, plus need-to-know, strict retention, extra audit

Make the rules concrete per class, or the labels do nothing: who may access, where it may be stored, whether it may leave the production environment, whether it may go to a third party, how long it is kept, and what is logged.

The rule that saves the most trouble: production personal data must not reach development or test environments. The usual failure is a database restored into staging "to reproduce a bug", which places customer data in a system with weaker access control and no monitoring. Use generated or masked data, and make it easy enough that nobody needs the shortcut.

2. Knowing what you hold

A data inventory records, per data set: what it contains, which classification, where it is stored, why you have it (the lawful basis), how long you keep it, who can access it, and which third parties receive it.

It is tedious and it is the prerequisite for everything else — deletion requests, breach notification, vendor review, retention. Teams that skip it discover the gap during an incident, which is the worst moment.

Data flows matter as much as stores. Personal data reaching an analytics service, an error reporter, a support tool or a language model API is a transfer with obligations, and it usually happens without a decision — an error reporter capturing a request body, an analytics tag capturing a URL containing an email address.

3. Personal data, precisely

Personal data is anything relating to an identifiable person — much broader than the American term "PII". It includes IP addresses, device identifiers, cookie ids and location data, even without a name attached.

Special category data — health, biometrics, race, religion, political opinion, sexual orientation, trade union membership — carries stricter rules and usually requires explicit consent.

Two words that are used interchangeably and must not be:

Pseudonymisation replaces identifiers with a key you still hold. The data is still personal data, because you can reverse it. It reduces risk; it removes no obligations.

Anonymisation makes re-identification impossible. It is much harder than it sounds, and hashing does not achieve it: hashing an email address is reversible by brute force over an enumerable input space (Chapter 8.2.2), and so is hashing a phone number or a postcode. Aggregation can also fail — famously, "anonymised" datasets have been re-identified by combining them with public data, because a small number of attributes is often unique to one person.

The engineering consequence: assume you are pseudonymising, not anonymising, unless a specialist has confirmed otherwise.

4. The rights, and what each one means in code

GDPR is the reference framework; similar regimes exist elsewhere (CCPA/CPRA, LGPD, India's DPDP Act). Every right below is a feature someone has to build.

A lawful basis is required to process at all. The three that matter in practice: consent (freely given, specific, withdrawable), contract (needed to deliver what the user asked for), and legitimate interest (a documented balancing test). Consent is the weakest basis — it can be withdrawn — so using it where "contract" applies creates unnecessary fragility.

Access and portability — provide the data, in a machine-readable format. Build an export endpoint before you are asked; assembling it by hand under a deadline is how mistakes and over-disclosure happen.

Rectification — the user can correct it. Usually already exists.

Erasure — the hard one, covered next.

Objection and restriction — stop processing for certain purposes, particularly marketing and profiling. This needs a flag consulted by every relevant pipeline, which is a design decision, not a checkbox in an admin panel.

Breach notification within 72 hours of becoming aware, where there is risk to individuals. That deadline is why detection matters (Chapter 8.1): you cannot notify what you have not noticed, and the clock runs from awareness.

Controller versus processor. A controller decides why and how; a processor acts on instructions. If you run a SaaS product, you are usually a processor for your customers' data and a controller for your own users, and your contract sets out the difference. Every sub-processor you use — a cloud provider, an analytics service — must be disclosed.

Cross-border transfers are restricted, which is why data residency options exist. Design for it before a customer asks, because retrofitting per-region storage is a large piece of work.

5. Deletion, honestly

The user row is the easy part. The hard parts, each with a real answer:

Derived data. Search indexes, caches, analytics warehouses, message queues, third-party systems. The only reliable approach is a deletion pipeline — a single event that every system subscribes to — rather than a checklist someone follows.

Data you must keep. Financial records have statutory retention periods, and a deletion request does not override a legal obligation. The answer is anonymisation rather than deletion: keep the transaction, replace the personal fields, retain referential integrity (Chapter 7.2.4). Be able to explain which data was kept and why.

Logs. Access logs contain IP addresses and user ids. Set a retention period and enforce it automatically — this is the most common gap, because logs are voluminous and nobody owns them.

Backups. You cannot practically delete one person from a year of backups. The accepted answer is documented: backups have a defined retention, deleted data expires with them, and any restore re-applies outstanding deletions. Write that down as a procedure, because it is what a regulator asks about.

Immutable and append-only stores. An event log or ledger that must not be edited (Chapter 9.7.29) conflicts directly with erasure. Crypto-shredding is the technique that resolves it: encrypt each user's personal fields with a per-user key, keep the keys in a key management service, and delete the key on erasure. The ciphertext remains and is permanently unreadable. It is the only clean answer for immutable storage, and it must be designed in from the start — you cannot retrofit per-user keys to existing plaintext.

And the cheapest control of all: retention limits by default. Data you deleted last year cannot leak this year, cannot appear in a subject access request, and does not need protecting. "Keep everything forever" is a liability disguised as an asset, and automated deletion jobs per data class are the single most effective privacy engineering measure.

6. Cookie consent, as an architecture

The banner is the visible part of a system with real engineering requirements.

Categories:

• Strictly necessary — session, security, load balancing. No consent required.
• Functional — language, theme.
• Analytics — measurement.
• Marketing — advertising and cross-site tracking.

The rules that constrain the implementation:

Nothing non-essential may run before consent. Not the cookie, not the script. A tag that loads and fires before the user chooses is the most common failure, because the tag was added to the page directly rather than gated.

Rejecting must be as easy as accepting. Regulators have ruled repeatedly against banners where "accept all" is one click and rejection takes three, and against pre-ticked boxes.

Consent must be granular and withdrawable, with the withdrawal mechanism as reachable as the original prompt.

Consent must be recorded — what was consented to, when, and which banner version — because the obligation is to demonstrate consent, not to have obtained it.

Architecturally, a consent management platform stores the choice (usually in a first-party cookie), exposes it to the page, and gates the tag manager so scripts load only in permitted categories. Server-side, the same signal must gate anything you do on the back end: a server-set analytics cookie or a server-to-server event send bypasses the banner entirely, which is a real and frequently overlooked leak.

And the honest framing for an engineer: the reliable way to reduce consent complexity is to collect less. Every third-party tag is a consent obligation, a performance cost (Chapter 6.7) and a security dependency.

7. Audit logs

What to record: who (both identities under impersonation, per Chapter 8.4.10), what action, on what object, when (with a timezone), from where, the outcome, and before-and-after values for changes.

What must never be logged: passwords (even failed attempts — people mistype their password into the username field), tokens and API keys, full card numbers, and personal data beyond what the log needs. Redact by allow-list, not by trying to detect secrets, and Chapter 9.7.31 covers the mechanism.

Log denials, not just successes. A burst of authorisation failures is one of the most reliable intrusion signals available.

Store audit logs separately from application logs, append-only, with different access control. An attacker's first act after gaining administrative access is to clean the logs, so logs that the compromised system can edit have limited value in an investigation.

Access to audit logs is itself sensitive — they contain a detailed record of user behaviour — so reading them should require authorisation and should itself be logged.

Set a retention period that satisfies both investigation needs and data minimisation, and enforce it automatically.

8. Intellectual property for engineers

Rarely taught, routinely encountered.

Open-source licences fall into three groups, and the difference decides what you may ship:

• Permissive (MIT, Apache 2.0, BSD) — use freely, keep the notice. Apache 2.0 also grants patent rights, which is why it is often preferred for commercial use.
• Weak copyleft (LGPL, MPL) — modifications to that component must be shared; your own code need not be.
• Strong copyleft (GPL, AGPL) — distributing software that includes it requires releasing your source under the same terms. AGPL extends this to network use, so running a modified AGPL service publicly triggers the obligation even without distribution. This is why many companies prohibit AGPL dependencies, and it is a legitimate engineering constraint rather than hostility to open source.

"Source available" is not open source — Business Source License and similar terms restrict commercial use. Read them; several widely used projects have relicensed this way.

Practical rules: keep an inventory of dependency licences (your SBOM from Chapter 8.6.2 gives it), preserve attribution notices, and get advice before shipping anything strongly copyleft in a proprietary product.

AI coding assistants raise a live question: generated code may reproduce training data. Review generated code as you would a contribution, prefer tools that offer indemnity and filtering if your organisation requires it, and follow your employer's policy rather than inventing one.

Employment terms usually assign work created in the course of employment to the employer, sometimes broadly. Read your contract, and if you maintain personal projects, know where the line sits and get written clarity before it matters.

Trade secrets are protected by being kept secret — which is a security control, not a legal filing. Customer lists, algorithms and internal metrics are typically covered, and NDAs are enforceable.

What the interviewer will push on

"A user requests deletion. What actually has to happen?" Name the systems: primary store, derived stores (search index, cache, warehouse), third parties, logs, and backups. Then the answers — a deletion pipeline rather than a checklist, anonymisation where a legal retention obligation applies, automatic log retention, and documented backup expiry with re-application on restore.

"How do you delete from an immutable event log?" Crypto-shredding: encrypt each user's personal fields with a per-user key and delete the key. The ciphertext remains and is unreadable. Add that it must be designed in from the start, because you cannot retrofit per-user keys to existing plaintext.

"Is hashing an email address enough to anonymise it?" No — the input space is enumerable, so it is reversible by brute force, which makes it pseudonymisation. It reduces risk and removes no obligations. This is the same point as Chapter 8.2.2 and it is one of the most common privacy mistakes in analytics pipelines.

"What does a cookie banner actually require technically?" Nothing non-essential loading before consent — including the script, not just the cookie — granular categories, rejection as easy as acceptance, an easy withdrawal path, and a record of what was consented to and when. Then volunteer the server-side gap: a server-set cookie or server-to-server event bypasses the banner entirely.

"What should and should not be in an audit log?" Who, what, when, where, outcome and before/after values; never passwords, tokens or full card numbers. Then the two design points: log denials, because a burst of them is a strong intrusion signal, and store audit logs separately and append-only, because cleaning logs is an attacker's first move.

"Why do companies avoid AGPL dependencies?" Because the network clause extends the source-sharing obligation to running a modified version as a service, without distribution. It is a legitimate engineering constraint, and knowing the mechanism rather than repeating "GPL is dangerous" is what the question checks.

One thing to volunteer: point out that automated retention limits are the cheapest privacy control available. Data deleted last year cannot leak this year, cannot appear in a subject access request, and does not need protecting — and most organisations keep everything forever because deleting requires a decision nobody wants to make.

Recall

• Classification is the input to every other control, and the rule that saves most trouble is that production personal data must never reach development or test.
• A data inventory — what, where, why, how long, who, and which third parties — is the prerequisite for deletion requests, breach notification and vendor review. Flows matter as much as stores.
• Pseudonymisation is not anonymisation. Hashing an email is reversible by brute force, so it stays personal data. Assume you are pseudonymising.
• Rights are features: an export endpoint, correction, erasure, and an objection flag every pipeline consults. 72 hours to notify a breach from awareness — which is why detection matters.
• Deletion in practice: a deletion pipeline, not a checklist; anonymise where retention is legally required; automatic log retention; documented backup expiry with re-application on restore; and crypto-shredding for immutable stores — delete the per-user key, designed in from the start.
• Automated retention limits are the cheapest privacy control. Data you deleted cannot leak, cannot be requested, and needs no protecting.
• Consent: nothing non-essential loads before consent (the script, not just the cookie), granular, rejection as easy as acceptance, easy withdrawal, and a record of what and when. Server-set cookies and server-to-server events bypass the banner — gate them too.
• Audit logs: who/what/when/where/outcome/before-after; never passwords or tokens; log denials; store separately and append-only, because cleaning logs is the attacker's first move. Licences: permissive · weak copyleft · strong copyleft, with AGPL's network clause being why many companies prohibit it.

Self-test: What is the prerequisite for every obligation in this chapter? · Why is a hashed email still personal data? · What is the documented answer for backups? · What does crypto-shredding solve, and when must it be designed? · Which consent failure happens on the server rather than in the browser? · What triggers the AGPL obligation that the GPL does not?

Next: 8.8 closes the Part with the incidents themselves — what actually went wrong in the breaches everyone names, and which of them your current design would still permit.