8.3.1 — Certificates, PKI and the File Formats

An operations engineer is given three files — server.crt, server.key, ca-bundle.crt — and a load balancer that wants a .pfx. A colleague sends a .pem that turns out to contain both a certificate and a private key. Java refuses all of it and wants a keystore.

None of this is conceptually hard. It is an accumulation of forty years of container formats around two objects: a public key with an identity attached, and a private key that must never leave the machine. This page makes the formats boring, and then covers the operational parts that cause real outages.

1. What a certificate actually is

Chapter 5.7 introduced it from the browser's side. Structurally, an X.509 certificate is a signed statement:

Certificate:
  Version: 3
  Serial Number: 04:a1:… 
  Signature Algorithm: ecdsa-with-SHA256
  Issuer:   C=US, O=Let's Encrypt, CN=R11              ← (1)
  Validity: Not Before 2026-06-01  Not After 2026-08-30 ← (2)
  Subject:  CN=shop.example.com                         ← (3)
  Subject Public Key Info:
     Public Key Algorithm: id-ecPublicKey
     EC Public Key: (256 bit)                           ← (4)
  X509v3 extensions:
     Subject Alternative Name: DNS:shop.example.com, DNS:www.shop.example.com  ← (5)
     Key Usage: Digital Signature                       ← (6)
     Extended Key Usage: TLS Web Server Authentication  ← (7)
     Basic Constraints: CA:FALSE                        ← (8)
  Signature: 30:45:02:20:…                              ← (9)

(1) Who vouched for this. (2) The two dates that cause outages. (3) The historical name field. (4) The subject's public key — the actual payload. (5) The SAN list is what browsers check; CN has been ignored for hostname matching since around 2017, so a certificate with only a CN and no SAN fails everywhere. (6) and (7) restrict what the key may be used for, which is why a certificate issued for client authentication does not work as a server certificate. (8) CA:FALSE means this certificate may not sign others — the check whose absence caused the 2002 Internet Explorer flaw where any valid certificate could sign for any domain. (9) The issuer's signature over everything above.

A certificate is public. It contains no secret. The private key is a separate file and is the thing that matters.

2. Getting one: the CSR

A certificate signing request is what you send to a certificate authority.

# Generate a private key, then a CSR for it
openssl ecparam -genkey -name prime256v1 -out server.key                # (1)
openssl req -new -key server.key -out server.csr \
  -subj "/CN=shop.example.com" \
  -addext "subjectAltName=DNS:shop.example.com,DNS:www.shop.example.com" # (2)

(1) The private key is generated on the machine that will use it, and never sent anywhere. (2) The CSR contains the public key, the requested names, and a self-signature proving you hold the matching private key.

The CA then discards most of what you asked for. It verifies you control the domain, and issues a certificate with its own serial number, its own validity dates, and its own extensions. Fields like organisation and country in your CSR are only used if the CA validated them — which is why a DV certificate ignores them entirely.

On the passphrase question, which comes up with Windows and IIS: OpenSSL will happily encrypt a private key with a passphrase. A server that starts unattended cannot use one, because something would have to type it at every boot. So the passphrase is either absent, or stored next to the key, which provides nothing. IIS's certificate request flow does not prompt for one because the key is generated into the Windows certificate store and protected by the operating system's access control instead — the protection moved from a passphrase to file permissions and key isolation, which is the more useful mechanism anyway. The same logic applies on Linux: protect server.key with chmod 600 and a dedicated user, not with a passphrase you must automate away.

3. Encodings and file extensions

There are only two encodings, and everything else is a container.

DER — the binary form. Compact, not human-readable.

PEM — DER, base64-encoded, wrapped in header lines. Text, pasteable, and the common form on Unix.

-----BEGIN CERTIFICATE-----
MIIDdzCCAl+gAwIBAgIEAgAAuTANBgkqhki…
-----END CERTIFICATE-----

Read the header line, not the extension. The extension is convention; the header says what it actually is.

Header line	Contents
BEGIN CERTIFICATE	A certificate (public)
BEGIN CERTIFICATE REQUEST	A CSR
BEGIN PRIVATE KEY	PKCS#8 private key, unencrypted
BEGIN ENCRYPTED PRIVATE KEY	PKCS#8, passphrase-protected
BEGIN RSA PRIVATE KEY	PKCS#1, RSA-specific, older
BEGIN EC PRIVATE KEY	SEC1, elliptic-curve, older
BEGIN PUBLIC KEY	A bare public key, no identity

Then the extensions:

Extension	Usually	Notes
.pem	Anything PEM-encoded	May hold several objects concatenated
.crt, .cer	A certificate	Either DER or PEM
.der	Binary
.key	A private key	Never share this
.csr	A signing request
.p7b, .p7c	Certificates only, no key	Windows chain export
.pfx, .p12	PKCS#12: key + certificate + chain, in one encrypted file	Windows and Java
.jks	Java KeyStore, proprietary	Deprecated in favour of PKCS#12
.ppk	PuTTY's private key format	Not an X.509 thing — an SSH key

Two clarifications that resolve most confusion.

A .pem can contain several things. Many servers want the leaf certificate followed by the intermediates in one file — a "bundle" or "full chain". Order matters: leaf first, then each issuer upward. The root is usually omitted because the client already has it.

.pfx/.p12 is the one that carries the private key. It is password-protected because it must be. Never email a .pfx with the password in the same message, which is exactly how they usually travel.

# PEM to PKCS#12, for Windows or Java
openssl pkcs12 -export -out server.pfx \
  -inkey server.key -in server.crt -certfile ca-bundle.crt

# PKCS#12 back to PEM
openssl pkcs12 -in server.pfx -nocerts -nodes -out server.key
openssl pkcs12 -in server.pfx -clcerts -nokeys -out server.crt

A .ppk is a different world. PuTTY uses its own format for SSH keys, which are not certificates at all — bare key pairs with no issuer and no identity binding. puttygen converts between .ppk and OpenSSH format. Chapter 8.3.2 covers SSH properly.

4. The chain, and the failure it causes

Browsers ship a root store of a few hundred trusted CA certificates. Those roots sign intermediates; intermediates sign your certificate.

Root CA           offline, ~20 years, in every trust store
  └── Intermediate  online, ~5 years, signs customer certificates
        └── Leaf     your server, ~90 days

The root is offline because it cannot be revoked. A root certificate is baked into billions of devices; if its key leaks, there is no mechanism to withdraw it. So roots live in hardware security modules, are used a few times a year under ceremony, and sign only intermediates — which can be revoked.

Your server must send the leaf plus the intermediates. It must not send the root (the client has it, and sending it wastes bytes and proves nothing).

A missing intermediate is the most common TLS misconfiguration, and its symptom is deceptive: it often works in browsers — because some fetch missing intermediates using the Authority Information Access field, or have cached the intermediate from another site — and fails in curl, in mobile applications, in Java clients and in server-to-server calls. "It works in Chrome but our API client rejects it" is nearly always this.

openssl s_client -connect shop.example.com:443 -servername shop.example.com -showcerts

That prints exactly what the server sent, which settles the question in seconds.

5. Trust stores, and why Java disagrees with your browser

There is no single list of trusted CAs on a machine.

• The operating system has one (/etc/ssl/certs, the Windows certificate store, the macOS Keychain).
• Firefox ships its own and ignores the OS store on Linux.
• Java has cacerts, a separate keystore in the JDK, which is why a Java service can reject a certificate every browser accepts — usually because the JDK is old and lacks a newer root.
• Node.js bundles Mozilla's list, and NODE_EXTRA_CA_CERTS adds to it.
• Containers frequently have no store at all unless ca-certificates was installed, producing the classic "certificate verify failed" inside a minimal image.

To trust an internal CA you must add it to every one of these that applies. Doing it in one place and assuming coverage is a recurring cause of "it works on my machine".

And the rule that matters more: never disable verification to make an error go away. curl -k, rejectUnauthorized: false, verify=False — each turns a TLS connection into an encrypted conversation with anyone. They appear as a temporary debugging measure and they survive into production. Fix the trust store instead.

6. Validation, issuance and automation

Domain validation is the only level that matters now, per Chapter 5.7: browsers removed the extended-validation indicator because users did not notice it.

ACME is the protocol that automated issuance, and it is the reason free certificates are universal. The CA gives you a challenge; you prove control:

• HTTP-01 — serve a token at /.well-known/acme-challenge/…. Simple, needs port 80 reachable, cannot issue wildcards.
• DNS-01 — publish a TXT record. Works for wildcards and for servers with no public HTTP, and needs API access to your DNS.
• TLS-ALPN-01 — prove control over port 443 using a special TLS handshake. Useful where port 80 is blocked.

Wildcards (*.example.com) are less useful than they look. They cover exactly one label — a.example.com yes, a.b.example.com no — they require DNS-01, and one private key then protects every subdomain, so a compromise anywhere is a compromise everywhere. Per-host certificates with automated issuance are usually better; wildcards earn their place when hostnames are dynamic.

CAA records name which CAs may issue for your domain, and every CA is required to check them:

example.com.  IN CAA 0 issue "letsencrypt.org"
example.com.  IN CAA 0 iodef "mailto:security@example.com"

Two lines of DNS that make mis-issuance by another CA a policy violation rather than a possibility.

Automate renewal, without exception. With 90-day certificates — and the industry moving toward 47 days by 2029 — manual renewal is a scheduled outage. And renewal is not the whole job: the new certificate must be deployed and the service reloaded. A cron job that renews into a directory nobody reads is a very common way to have an expired certificate despite "having automation".

Monitor expiry externally, from outside your network, against the certificate the server actually presents. Alert at 30, 14 and 7 days. Every large company has had this outage, and it is entirely avoidable.

7. Revocation, honestly

If a private key leaks before expiry, the certificate must stop being trusted. This is the weakest part of the system.

CRLs — a downloaded list of revoked serial numbers. They grew to megabytes and do not scale.

OCSP — ask the CA in real time. It leaks browsing behaviour to the CA and adds latency, so browsers made it soft-fail: a failed check is ignored, which means an attacker who can block the check defeats it entirely.

OCSP stapling — the server fetches its own signed status periodically and attaches it to the handshake. Better on privacy and latency, and worth enabling. Must-Staple in the certificate makes a missing staple fatal, which is stronger and riskier.

What the industry actually converged on is short lifetimes. A 90-day certificate that cannot be revoked reliably is still only 90 days of exposure, and the move to 47 days is the same reasoning continued. Revocation is the backstop; short validity is the mechanism.

Browsers additionally ship curated lists of high-impact revocations (CRLSets, CRLite), which covers the cases that matter most without a per-connection check.

8. Running a private CA

Internal services, mTLS between microservices, and device identity all need certificates that no public CA will issue.

What you must run: a root key kept offline (or in a hardware security module or cloud key management service), an intermediate that does the signing, an issuance mechanism, and distribution of the root certificate to every trust store that must accept it.

The two failures that dominate. Expiry — an internal certificate expiring takes a service down completely, and internal certificates are exactly the ones nobody monitors. And distribution — a new service, a new container image or a new laptop that never received the root certificate fails in a way that looks like a network problem.

So automate it or do not do it. Managed services (AWS Private CA, a Kubernetes certificate controller, HashiCorp Vault's PKI engine) issue short-lived certificates automatically and rotate them long before expiry, which turns the expiry problem into a non-event. A hand-run internal CA with one-year certificates is an outage with a date on it.

9. Diagnosis

openssl x509 -in server.crt -noout -text            # read a certificate
openssl x509 -in server.crt -noout -dates -subject  # just the essentials
openssl req  -in server.csr -noout -text            # read a CSR

# Do this key and this certificate match? Both hashes must be identical.
openssl x509 -noout -modulus -in server.crt | openssl md5
openssl rsa  -noout -modulus -in server.key | openssl md5

openssl s_client -connect host:443 -servername host -showcerts   # what the server sends
openssl verify -CAfile ca-bundle.crt server.crt                  # does the chain validate

The modulus comparison is the one to remember: "key values mismatch" on a server start almost always means the key and certificate are from different generations, and this is how you prove it in two commands.

What the interviewer will push on

"What is inside a certificate?" Subject, issuer, validity dates, public key, extensions — with SAN as the field browsers actually check for hostnames, since CN has been ignored since 2017. Then mention Basic Constraints: CA:FALSE, and why a missing check on that field was a real Internet Explorer vulnerability.

"Why does a certificate chain exist?" A root cannot be revoked once it is in a billion trust stores, so it stays offline and signs revocable intermediates. Then give the operational consequence: your server must send the leaf plus intermediates, and a missing intermediate works in Chrome and fails in curl, which is the most common TLS misconfiguration.

"What is the difference between a .pem, a .crt, a .key and a .pfx?" PEM is an encoding; .crt is usually a certificate, .key is a private key that must never be shared, and .pfx/.p12 is a password-protected bundle of key plus certificate plus chain for Windows and Java. Add that you read the BEGIN header rather than trusting the extension.

"Why does the Java service reject a certificate my browser accepts?" Different trust stores. Java uses cacerts in the JDK, Firefox ships its own, containers often have none. The tell is finishing with the rule: never disable verification to clear the error — fix the trust store, because verify=False survives into production.

"How would you issue certificates for internal service-to-service TLS?" A private CA with an offline root, an intermediate that signs, short-lived certificates, and automated issuance and rotation. Then name both failure modes: internal expiry is an outage nobody monitors, and a machine that never received the root fails in a way that looks like a network problem.

"How does revocation work, and does it?" CRLs do not scale; OCSP leaks browsing and is soft-fail, so blocking it defeats it; stapling improves both. The industry's real answer is short lifetimes — 90 days moving to 47 — with revocation as a backstop and browser-curated lists for the high-impact cases.

One thing to volunteer: mention CAA records. Two lines of DNS declare which CAs may issue for your domain, every CA must check them, and it converts mis-issuance from a possibility into a policy violation. It costs nothing and almost nobody sets it.

Recall

• A certificate is a public statement: subject, issuer, dates, public key, extensions, signed by the issuer. SAN is what browsers check — CN has been ignored since 2017 — and CA:FALSE is what stops a leaf signing other certificates.
• A CSR proves you hold the private key and requests names; the CA discards most other fields unless it validated them. The private key is generated locally and never sent. Unattended servers cannot use a key passphrase, so the real protection is file permissions or an OS key store.
• Two encodings: DER (binary) and PEM (base64 with headers). Read the BEGIN line, not the extension. .key is secret; .pfx/.p12 carries the key and must be password-protected; .ppk is PuTTY's SSH format, not X.509.
• The chain exists because a root cannot be revoked once shipped. Send leaf + intermediates, never the root. A missing intermediate works in Chrome and fails in curl — the most common TLS misconfiguration.
• Trust stores are plural: OS, Firefox, Java cacerts, Node's bundle, and containers that have none. Add an internal root to all of them, and never disable verification to clear an error.
• ACME challenges: HTTP-01 (simple, no wildcards), DNS-01 (wildcards, needs DNS API), TLS-ALPN-01. Wildcards cover one label and put every subdomain behind one key. CAA records restrict which CAs may issue.
• Automate renewal and deployment and reload — a renewed file nobody loads is still an outage — and monitor expiry from outside.
• Revocation is weak: CRLs do not scale, OCSP is soft-fail, stapling helps. Short lifetimes are the actual mechanism — 90 days, heading to 47.

Self-test: Which field do browsers use for hostname matching? · Why is the root kept offline and used only for intermediates? · Which file must never be emailed, and which format contains it? · Why can a Java client reject what Chrome accepts? · When must you use DNS-01 instead of HTTP-01? · Which two openssl commands prove a key and certificate match?

Next: 8.3.2 covers the protocol you use every day and rarely read about properly — SSH, from the host key warning to agent forwarding, tunnels and certificate-based access.