The integration that decides your timeline predates the web

The system holding the data you need is often twenty years old, has no API in the sense you mean, and is owned by nobody. Four integration shapes cover most of them, and choosing the right one is a decision about the customer's organisation as much as their technology.

14 MIN

TL;DR: Old enterprise systems expose themselves in four shapes: a file drop, a request-response interface from an older era, direct database access, and a change feed. Each has a different failure profile and a different political cost, and the choice is usually organisational rather than technical.

Where you are. Module two. Foundations taught you to integrate an undocumented system by getting one real response and probing the edges. This is what to do when the system is old enough that "an API" is not on the menu.

Four shapes

Our freight customer's transport system was written before the customer portal existed and has outlived three IT directors. On the solution architect's slide it appears as one tidy box with an arrow pointing at your product. The box has been accumulating exceptions since before anyone in the meeting joined the company, it does not have an API, and it has several ways of getting data out of it, none of them equivalent.

There is a reason the shapes are always this messy, and it is not incompetence. Conway's law, from a 1968 paper by Melvin Conway, holds that an organisation designing a system will produce a design that copies its own communication structure. Enterprise data is fragmented because the org chart was fragmented when each piece was built. That is worth knowing on day one, because it tells you the integration you are looking at is a fossil of a reporting line, and that changing it means changing who talks to whom. Which is why the choice below is usually organisational rather than technical.

The file drop. A scheduled export onto a shared location. Universally available, requires almost no permission, and is where most engagements start. Its failure mode is silence: the job that produces the file breaks, no file appears, and your pipeline has nothing to react to unless you built an expectation. Always alert on absence, not just on malformed content.

Request-response in an older style. A verbose envelope, a rigid contract, no pagination worth the name, and a habit of returning a success wrapper around an internal error. Treat the wrapper as untrusted: parse the payload for an actual error string rather than believing the status.

Direct database read. Fast and immediate, and the most dangerous. You become coupled to a schema nobody promised you, owned by a team that does not know you exist. If you take this route, insist on a replica rather than the primary, and write a contract test that runs daily against the columns you depend on, so a schema change reaches you as a test failure rather than as wrong output.

Hyrum's law is the thing to have in mind here: with enough users of an interface, every observable behaviour of it will be depended on by somebody, regardless of what was promised. Read a table directly and you have quietly become that somebody. The other team owes you nothing, has no idea you exist, and will rename a column in a routine release. The contract test is how you find out that morning rather than at the end of the quarter.

A change feed. The best of the four when it exists, and it usually does not. If it does, ordering and replay become your problem, which is a good trade.

A note from the field before you choose: veterans of this work tell stories of weeks spent writing production code in five languages to move one record across three decades of systems. Nobody plans that. It is simply what the seams between eras are made of, and the seam module below is how you keep it from spreading.

The choice is organisational

Here is the thing engineers get wrong. All four are technically viable. What differs is who has to say yes.

A nightly file export is often within one manager's authority. Direct database access touches a team that guards it and may involve a formal review. A new event stream is a project on somebody else's roadmap, and roadmaps are where pilot timelines go to die.

So the real question is not which is technically best. It is: which of these can I get approved within the pilot, and what does that cost me in freshness? For our freight customer that usually means starting on the nightly export, being explicit in the proposal that latency is therefore measured in hours, and treating anything better as phase two.

That is not settling. It is picking the integration that lets you answer the pilot's question this quarter, while being honest that the production design may differ.

Build the seam early

Whichever shape you take, put a boundary between it and everything else on day one.

One module knows how the freight system exposes data. Everything downstream consumes your own shape: a normalised shipment record with the fields your logic needs. Nothing outside that module knows whether the data arrived as a file or a query.

Two payoffs make this worth the small cost. When the customer eventually grants the better access, you swap one module rather than rewriting the pipeline. And you can run the entire system against recorded fixtures, which means you can develop without their environment, which matters enormously when access is intermittent and approvals are slow.

Do this before moving on

For the freight engagement, pick a shape and write three sentences: which one, who has to approve it, and what freshness the customer is therefore getting. Then write the sentence you would put in the proposal's risks section about what happens if that approval does not arrive in time. That sentence is the difference between a slipped pilot and a planned fallback.

Go deeper

Change data capture is the fourth shape in depth, including the ordering problems it hands you.
Data quality is what you need at the seam when the source is a nightly file nobody validates.
Idempotent pipelines matter more here, because file drops get redelivered and feeds get replayed.
Testability and dependency injection is how the seam lets you develop against fixtures.
Integrating an undocumented system is the Foundations method this builds on.

Key takeaways

Four shapes cover most legacy integrations: file drop, older request-response, direct database read, and a change feed.
Alert on the absence of an expected file. Silence is the file drop's failure mode and it is invisible by default.
The choice is usually organisational: pick the shape you can get approved inside the pilot and be explicit about the freshness that buys.
Put a seam between the source and your logic on day one, so better access later is a swap rather than a rewrite.

Check yourself

Answer before you look. Recalling it is what makes it stick; recognising it does not.

1Your integration reads a nightly export. What failure mode is invisible by default, and what do you do about it?
2Direct database access to the legacy system is technically available. What is the strongest argument for choosing the nightly file export instead?
3Why build a seam between the source system and the rest of your code on day one?