Choose the integration path that meets the workflow and can be operated

This is lesson 4 of The FDE Engagement. Choose a source path that can meet the agreed workflow, then verify permission, freshness and replay behavior on that path. An adapter isolates parsing and transport details. It doesn't make stale snapshots equivalent to a live feed.

Module two. The freight pilot has a decision and a dependency list, and you now need an approved, testable route from the transport system to the exception workflow.

Four shapes to investigate. The fictional freight transport system predates its customer portal, and its owner offers several ways to export information, with different guarantees.

Don't infer those guarantees from the system's age, or from a box labeled integration on the architecture slide. A 1968 paper connects a system's design to the communication structure of its designers. Use that as a reason to ask who owns each boundary, though it doesn't prove that every fragmented dataset comes from an old reporting line, or that technical changes always require reorganizing teams.

The four paths organize the questions to ask: a file drop, a request-response interface, a direct database read, and a change feed. None is automatically best, universally available, or exempt from security review.

In a file drop, a producer exports records to an agreed location, so confirm the schedule, whether the file is a snapshot or a delta, how completion is signalled, and who can read it. Publish through an atomic rename or a completion manifest, where the storage supports that pattern, so the consumer doesn't mistake a partially written file for a complete export. And check both expected arrival and the source watermark, because a newly downloaded old file is still old data.

A request-response interface, an API or RPC endpoint, may provide current records, cached records, or an asynchronous job result, so confirm the response semantics, pagination, rate limits and timeout behavior. Some interfaces carry an application error inside a successful transport response, so parse the documented error structure, rather than searching arbitrary payload text for the word error. A customer's business record may legitimately contain that word.

For a direct database read, agree a supported view or schema contract with the owner, and use a read-only account. A replica, a copy kept in step with the primary database, can fall seconds or hours behind it when replication is delayed. It can reduce primary load, but it may lag. It isn't automatically current, or harmless to overload. Establish query budgets and timeouts, and test schema changes before relying on the result.

A bounded, approved primary read can be appropriate, where its owner has assessed the load and consistency requirement, and a replica isn't a substitute for that decision. Clients can come to depend on observable behavior beyond an interface's stated promises, and direct table reads create exactly that risk if you rely on undocumented status values or join semantics. Write those dependencies down and get an owner to review them. A contract test detects some changes, but it can't guarantee that a field's business meaning stayed constant.

A change feed can support incremental freshness, but you need its ordering scope, retention, duplicate behavior and recovery procedure. A resume token, the bookmark a feed hands you so you can continue from where you stopped, is an operating constraint if it expires during an outage. Ask how to combine an initial snapshot with the feed without losing changes between them, and how deletions are represented. A low-latency feed you can't restart is incomplete.

Now choose against the actual pilot question, using a synthetic option packet. The pilot tests historical shipment identification, with a separate possible phase for live fifteen-minute handling.

The nightly export has a complete manifest and a midnight source snapshot, approved for the pilot dataset, so it supports historical lookup and daily reconciliation. The portal API has recent exception messages, but no transport shipment status. The transport replica matches a sample query, but its replication lag is unmeasured and the owner's review is pending. And the change feed exists as a design document only, with seven-day retention proposed.

Choose the export plus portal messages for a historical lookup pilot, after confirming the dataset and identity mapping, and keep live freshness out of its claim. The portal API can't make transport data fresher, and a replica query returning quickly measures response time, not the age of its values.

If the buyer requires live fifteen-minute handling as the pilot's sole decision, none of the current evidence establishes a compliant end-to-end path. Escalate the replica and feed checks, or renegotiate the question. Calling an available export good enough for phase one doesn't solve a requirement the funded pilot must test now.

Test a file's age and completeness separately. Suppose the exception demo uses a historical snapshot, with a 26-hour maximum age agreed for this exercise.

At 02:00 UTC on April 11, you receive this manifest. The download completed at 01:55. The source snapshot watermark is April 9, at midnight. It declares 1,000 rows, and 998 parsed. Two were rejected at parse, and the completion marker is present.

The file arrived five minutes ago, but the data is 50 hours old, because from the April 9 midnight watermark to the April 11 check is two full days, plus two hours. Its age exceeds the 26-hour requirement by 24 hours. A fresh timestamp can't fix that.

The row ledger balances: 998 parsed plus 2 rejected equals 1,000 declared. That establishes accounting for this manifest, not complete usable data. Retain the rejected records in an access-controlled quarantine with reasons, and decide whether the pilot can proceed with those exclusions. Don't publish "1,000 records loaded" when two couldn't be parsed.

The count in the manifest itself also needs a source-side guarantee, because a balanced consumer ledger can't detect rows the exporter never included. If the completion marker were missing, wait or investigate according to the producer contract, before processing. An empty file may be a legitimate zero-record export, but a missing file is different. Give each a distinct status, so the operator knows whether to retry a transfer, ask the producer, or inspect rejected records.

Build an adapter, without erasing the source contract. Put transport and parsing behind a small adapter, so downstream code consumes a normalized record, plus the source version, extraction time and relevant watermark.

The file drop, the older request-response interface, and a contract-tested read from the replica all feed one source adapter, which produces a normalized shipment record for workflow logic that uses both values and freshness. Preserve missing, deleted and unknown states, rather than flattening them into ordinary values.

Recorded fixtures let you develop without continuous customer access, subject to permission to retain them and appropriate redaction, and you maintain at least a happy response, a missing-field response, an application-error response and a replayed input. Prefer synthetic fixtures where real customer records are unnecessary, and never put secrets or unapproved personal data into the repository.

The adapter makes a future switch more localized, but the workflow still needs a migration review, because a full snapshot has different deletion and completeness semantics from a stream of changes. Run both paths on comparable records, reconcile their watermarks and counts, and test restart before changing the source of truth. Reusing the same normalized fields doesn't establish semantic equivalence.

Finally, write the operating agreement. For the chosen path, name the producer, the consumer, the approval owner, and the failure contact.

Record the expected arrival, freshness limit, retention, credentials, source schema version, and the response to a missed interval, and include the recovery method and the largest replay the source owner permits during business hours.

Use a measurable statement: the export watermark must be no older than 26 hours when the historical demo starts, otherwise the demo reports stale input and pauses. Avoid "the export is daily", which leaves both its cutoff and the consequence of a missed run undefined. And if a stale dataset can safely support a narrower demonstration, label its age visibly, and get agreement on that narrower claim.

Your turn. Pause the video here. Choose a path from the option packet, and write its approval dependency, freshness claim and recovery question. Diagnose the supplied manifest with separate freshness and row-accounting verdicts. Then move its source watermark to April 10 at midnight, keeping the 02:00 check time on April 11, and the two rejected rows.

The original snapshot is 50 hours old, and fails the 26-hour age limit. The changed snapshot is exactly 26 hours old, and passes an inclusive maximum-age policy. Both ledgers still contain two rejected rows, so neither proves 1,000 usable records. For the historical pilot, the approved export is a feasible path, if those data-quality limits are handled. The live fifteen-minute question remains unproven by every available path in this packet.

To recap. Check the source contract, permissions and recovery path for each integration option, because age alone doesn't determine them. Distinguish transfer time, query latency and the source watermark. An export can balance its row ledger while containing unusable or stale data. An adapter localizes transport details, but it must preserve freshness, deletion and missing-value semantics, and keep the funded question visible when you choose a fallback, because historical lookup isn't proof of live handling.

That's lesson 4. Next in the series, carry the caller's permissions through every data and action boundary. The full lesson, with its exercise and drill, is on fdeinterviews.com.