13.9 — The Enterprise Estate

You join a large company and find: a Windows domain, file shares mapped to drive letters, an application server running a Java stack from 2011, an integration platform routing XML between forty systems, a nightly batch window, a mainframe nobody is allowed to change, and a change advisory board that meets on Wednesdays.

The instinct is that this is broken. It is not broken — it earns the company's money, and it was built by people solving real problems with the tools available. Understanding why it looks like this is what makes you useful in it, because you cannot replace what you do not understand, and most of it will still be there in ten years.

1. The stack, layer by layer

The classic on-premises stack, from the top:

Layer	What it is	Who owns it
Application	Your code	Development
Data	Databases, files	DBAs, data team
Runtime	JVM, .NET, Node	Development / middleware
Middleware	App servers, message brokers, integration	Middleware team
OS	Windows Server, RHEL	Systems administration
Virtualisation	VMware, Hyper-V	Infrastructure
Servers	Physical machines	Data centre
Storage	SAN, NAS	Storage team
Network	Switches, routers, firewalls	Network team

Two things to take from this table.

The cloud service models are exactly this stack with a line drawn across it. IaaS hands you everything from the OS up; PaaS from the runtime up; SaaS none of it. Chapter 13.1's shared responsibility chart is this table, which is why the model clicks once you have seen the stack it describes.

Each layer has an owner, and they are different people. A performance problem that spans layers becomes a meeting, and "the network team says it is not the network" is a genuine coordination cost rather than obstruction. The single largest practical difference from a cloud team is that you cannot change most of the stack yourself.

2. Storage and virtualisation

A SAN presents block storage over a dedicated network — a machine sees what looks like a local disk. Fibre Channel is the traditional transport; iSCSI carries the same protocol over ordinary Ethernet, more cheaply. A LUN is a slice of that array presented to a host.

A NAS presents a filesystem over SMB or NFS — shared, and accessed as files rather than blocks. Same distinction as Chapter 13.4's block versus file storage, twenty years earlier.

RAID levels are worth knowing because the vocabulary persists: RAID 1 mirrors, RAID 5 stripes with one parity disk (survives one failure), RAID 6 two, RAID 10 mirrors then stripes — the usual choice for databases because it survives failures without the write penalty of parity.

Virtualisation is where enterprises met the cloud's core idea first (Chapter 2.9). A VMware or Hyper-V cluster runs many virtual machines across pooled hosts, with:

• Live migration — move a running virtual machine between hosts with no downtime, which is how hardware is maintained without an outage.
• Automatic placement — balance load across hosts.
• High availability — restart virtual machines elsewhere when a host dies.
• Snapshots — a point-in-time state to roll back to. And the standard warning: a snapshot left in place grows without limit and degrades performance, so they are for a change window, not for backup.
• Templates — the golden image idea of Chapter 13.3.

Understanding this makes cloud concepts land faster, not slower. An availability zone is a failure domain like a cluster; an instance type is a virtual machine specification; a managed disk is a LUN with an API.

3. Middleware and application servers

An application server hosts your application and provides services it does not implement: connection pooling, transactions, security, clustering, deployment. WebSphere, WebLogic, JBoss/WildFly, Tomcat; IIS on Windows.

They exist because in 2005 you did not want every application implementing its own thread pool and transaction manager. Modern frameworks embed those, so the application is a self-contained process — which is exactly the container model, arrived at from the other direction.

What you will actually deal with: JVM heap sizing and garbage collector selection (Chapter 3.4), deploying a .war or .ear to a running server, a shared server hosting several applications so one can starve the others, and configuration living in the server rather than the artefact — which breaks the build-once-promote rule of Chapter 13.7 and is a real reason these estates are hard to make reproducible.

4. Identity and file shares

Active Directory is the centre of a Windows estate (Chapter 8.4.8): authentication by Kerberos, authorisation by group membership, and Group Policy pushing configuration to every machine and user.

File shares over SMB are still how a great deal of business data moves. \\fileserver\finance\budgets, mapped to a drive letter by a login script. DFS puts a namespace over several servers so \\corp\finance resolves to whichever server currently holds it, with replication between sites.

Why this matters to an engineer: many integrations are still "write a file to a share and a job picks it up". It is unfashionable, it works, it has been running for fifteen years, and replacing it with an API is a project, not a refactor. The failure modes are worth knowing: permissions inherited from a group nobody can name, a file locked by a user who went home, and a batch job that reads a file while it is still being written — which is why the convention is to write to a temporary name and rename on completion, since rename is atomic within a filesystem.

5. Integration middleware

The problem it solves is real: N systems integrating point-to-point need N(N−1)/2 connections, each with its own format, schedule and error handling. Forty systems is 780 potential integrations, and nobody can change anything.

An enterprise service bus centralises it. Every system connects once, to the bus; the bus routes, transforms and orchestrates.

Software AG webMethods is a common example, and its parts are representative of the category:

• Integration Server — runs the integrations, hosts adapters and services.
• Universal Messaging / Broker — the asynchronous messaging layer.
• Trading Networks — business-to-business document exchange, especially EDI.
• Adapters — pre-built connectors to SAP, JDBC, mainframe systems, SOAP and file systems.

MuleSoft, TIBCO, IBM Integration Bus and BizTalk occupy the same space, and cloud iPaaS products are the modern equivalent.

The honest assessment. The bus solves a genuine problem and creates two of its own: it becomes a bottleneck — every change queues behind one team — and it becomes a single point of failure with an enormous blast radius. That reaction is what produced the microservices argument for decentralised integration (Chapter 10.8.3), which then rediscovered that you still need routing, transformation and reliable delivery, and reinvented them as service meshes, API gateways and event brokers.

The distinction worth being able to state:

	Does
ESB	Routes, transforms, orchestrates between many systems
API gateway	Fronts APIs: auth, rate limits, routing (Chapter 13.2)
Event broker	Publish-subscribe delivery, no transformation (Chapter 10.8.1)
iPaaS	ESB as a managed cloud service

And the file-transfer half. Managed file transfer products move files between organisations with scheduling, encryption, retry and audit — because "SFTP plus a cron job" has no audit trail and no reliable retry. EDI — X12 in North America, EDIFACT elsewhere — is the fixed-format standard for purchase orders and invoices, decades old, and still how an enormous volume of trade is transacted. Fixed-width and delimited files with an agreed layout are everywhere.

Batch windows exist for a reason worth understanding: the source systems are transactional and cannot take an analytical load during the day (Chapter 7.8.1), so extraction runs overnight. The overnight window is a capacity decision, not laziness, and it is why "why can't this be real time" has a real answer.

6. Mainframes, and why they are still there

A significant share of large banks, insurers, airlines and governments still run core processing on IBM mainframes: COBOL, CICS for transactions, DB2 for data, JCL for batch, and 3270 terminal screens.

Why they persist, honestly:

• They work, with reliability and transaction throughput that is genuinely excellent for their workload.
• The business logic is decades of accumulated rules, much of it undocumented and all of it still in use.
• Rewrites have failed repeatedly and expensively, and organisations have learned from watching each other.
• The migration risk is asymmetric: the upside is lower costs, the downside is not being able to process payments.

Integration patterns, in increasing order of quality:

• Screen scraping — drive the 3270 interface programmatically. Fragile, and it is what you do when nothing else is available.
• File exchange — batch extracts, the most common pattern.
• IBM MQ — reliable messaging, and mainframes speak it natively. MQ is very often the actual integration point.
• APIs over CICS — modern IBM stacks expose transactions as REST or SOAP, which is the right answer where it is available.

The modernisation pattern that works is the strangler fig (section 8), not a rewrite.

7. The Microsoft data stack

An enormous amount of the world's business reporting runs on this, and it is worth knowing what each piece is.

SQL Server — the relational engine (Chapter 7.1).

SSIS (Integration Services) — the ETL tool. A package has two distinct designers, and the distinction is the thing to understand:

• Control flow — the sequence of tasks: run this, then that, branching on success or failure, with loops and precedence constraints.
• Data flow — inside a data-flow task, a pipeline of source → transformations → destination, moving rows in buffers.

Packages are deployed to an Integration Services catalogue with environments, parameters and execution logging, and scheduled by SQL Server Agent.

Why SSIS persists: thousands of working packages, staff who know it, and a strong story for on-premises sources — flat files, Excel, mainframe extracts, and every ODBC source. It is also visual, which makes it maintainable by people who are not developers, and hard to review in a pull request, because the package is XML that does not diff meaningfully. That last point is the real modernisation argument.

SSRS — paginated operational reports, the kind that must print correctly.

SSAS — analytical models: the older multidimensional cubes with MDX, and the newer tabular model with DAX, which is the same engine underneath Power BI.

Power BI — the current reporting layer, with a semantic model, DAX measures, and refresh schedules. The failure everyone hits is a model importing a hundred million rows and hitting refresh limits — the answer is a proper star schema (Chapter 7.8.1) and aggregation, not a bigger capacity.

The modern path: Azure Data Factory for orchestration, Synapse or Fabric for the warehouse, and dbt-style transformation in the warehouse (Chapter 7.8.2). Migrating SSIS is usually a rewrite rather than a lift, which is why estates run both for years.

8. Change management, and working within it

Enterprises run ITIL-shaped processes: incidents, problems, changes and a configuration database.

A change advisory board reviews and approves changes, with a change window — often a weekend — and a documented back-out plan. To a team used to deploying twenty times a day this looks absurd, and the reasoning is not. When a failed change stops payments, the cost of a bad deploy is measured in millions and in regulatory attention.

The productive response is not to fight it but to make changes boringly safe, which is what earns exemption:

• Standard changes — pre-approved, low-risk, repeatable — bypass the board entirely. Getting your deployment classified as a standard change is the single highest-value conversation available, and it is won with evidence: automated tests, automated rollback, a track record.
• Feature flags separate deploying from releasing (Chapter 13.7), so the code ships in a quiet window and the feature is enabled independently.
• Automated rollback turns "what is your back-out plan" from a paragraph into a command.

And engage with the configuration database. It is frequently out of date and it is what an incident responder reads at 3am. Keeping your service's entry accurate is a small favour that pays back during your own incident.

9. Modernising, when you must

Strangler fig — put a facade in front of the old system, route one capability at a time to a new implementation, and shrink the old system until nothing routes to it. Incremental, reversible per capability, and it is the pattern that actually works.

Anti-corruption layer — a translation boundary so the legacy system's model does not leak into the new one. Without it, the new system inherits the old data model and you have rewritten the legacy system in a newer language.

Read-replica offload — point reporting at a replica or an extracted store, which removes load and risk from the core system without changing it. Often the highest value for the lowest risk, and a good first move.

Event interception — capture changes from the legacy database (change data capture, Chapter 7.3.3) and publish them as events, so new systems consume without the old one being modified. This is frequently the best available integration point with a system nobody may change.

And when not to modernise. A stable system with low change rate, no scaling pressure and no security problem is not a problem to solve. Modernisation should follow a business need — a new capability the old system cannot support, a compliance requirement, an unsupported platform, an unmanageable cost. "It is old" is not a business case, and projects justified that way are the ones that fail.

What the interviewer will push on

"Why do large organisations still run this?" Because it works and it carries the revenue, the business logic is decades of accumulated undocumented rules, rewrites have failed publicly and expensively, and the risk is asymmetric — the upside is cost, the downside is not processing payments. Answering with respect rather than contempt is what the question is testing.

"What is an ESB and would you build one now?" Central routing, transformation and orchestration, solving the N(N−1)/2 integration problem. Today you would decompose it into an API gateway, an event broker and per-service integration — while acknowledging that the problems it solved are real and get rediscovered, which is why service meshes exist.

"How would you integrate with a mainframe you cannot change?" In order: an API over CICS if available, IBM MQ if not, file exchange, and screen scraping only as a last resort. Then add change data capture as event interception, and an anti-corruption layer so its model does not leak into the new system.

"How do you deploy frequently in an ITIL environment?" Get your deployment classified as a standard change by making it evidently safe — automated tests, automated rollback, a track record — and use feature flags so the risky moment is a toggle rather than a deploy. Fighting the process loses; making the change boring wins.

"How would you modernise a legacy system?" Strangler fig with a facade, one capability at a time, behind an anti-corruption layer, starting with read-only offload because it is the lowest risk. And name when not to: "it is old" is not a business case, and modernisation without a business driver is the category of project that fails.

"What is SSIS and why is it still around?" The Microsoft ETL tool, with control flow for task sequencing and data flow for row pipelines, deployed to a catalogue and scheduled by SQL Server Agent. It persists because thousands of packages work and staff know it — and its real weakness is that a visual package is XML that does not diff, so it cannot be reviewed like code.

One thing to volunteer: point out that the on-premises stack table is the cloud shared-responsibility chart with a line drawn across it. It shows you understand the cloud service models as a business arrangement about who operates which layer, rather than as three product names.

Recall

• The stack — application, data, runtime, middleware, OS, virtualisation, servers, storage, network — is exactly the cloud shared-responsibility chart with a line drawn across it. Each layer has a different owner, so cross-layer problems are coordination problems.
• SAN is block over a dedicated network; NAS is a filesystem over SMB or NFS. RAID 10 for databases. Virtualisation gave enterprises live migration, HA and templates first — and a snapshot left in place grows without limit.
• Application servers exist because 2005 applications should not each implement a thread pool and transaction manager. Modern self-contained processes are the container model reached from the other side, and server-held configuration is why these estates resist build-once-promote.
• File shares are still a primary integration channel. Write to a temporary name and rename on completion, because rename is atomic and a reader must not see a partial file.
• An ESB solves N(N−1)/2 integrations and becomes a bottleneck and a single point of failure — which produced decentralised integration, which rediscovered the same needs as gateways, meshes and brokers. EDI and managed file transfer still carry enormous trade volume, and batch windows are a capacity decision.
• Mainframes persist because they work, the logic is undocumented and still in use, and the risk is asymmetric. Integrate via API over CICS → IBM MQ → files → screen scraping, plus change data capture as event interception.
• SSIS: control flow sequences tasks, data flow moves rows. It persists on installed base and non-developer maintainability, and its packages do not diff, which is the real modernisation argument.
• In an ITIL estate, get classified as a standard change by making deployment evidently safe, and use feature flags. Modernise with strangler fig behind an anti-corruption layer, starting with read-only offload — and "it is old" is not a business case.

Self-test: How does the on-premises stack explain IaaS, PaaS and SaaS? · Why write a file under a temporary name first? · What two problems does an ESB create while solving one? · Which mainframe integration point is best, and which is last resort? · What is the difference between SSIS control flow and data flow? · What earns you frequent deploys inside a change advisory process?

Next: 13.10 closes the Part with the certification map — which ones are worth the time, what they actually test, and what they cannot tell an employer.