The situation
Salary Finance's products live and die on employment data: loans repaid through payroll only work if you know who is employed and at what salary. That data arrived from employers as periodic files which was workable, but often slightly behind the real data. The Sainsburys integration was the company's first direct-to-employer API - live validation, live salary data, leavers known the day it happens. This same approach was later agreed for NHS trust foundations across the country, what follows are the two decisions that shaped the build.
Don't touch the core - translate to it
Salary Finance already had a validation service at its centre which every product depended on. Rewriting it for real-time inputs would have put all of them at risk but we also believed employer integrations were the future, not a one-off. So the design was a middle layer: it takes external validation data in whatever shape an employer's systems produce, and translates it into the data points the core service already understands. The core stayed untouched and the next employer becomes a translation problem, not a rebuild.
When real-time breaks the batch assumptions
The database was built on the assumption that employer data arrives weekly or monthly and real-time broke it everywhere. We had to define scenarios and statuses that had never existed, decide what a customer should see the moment they became a leaver, and work out which documentation had to trigger immediately rather than at the next cycle. This meant that I ended up leading a system redesign for the future of integrated employer data internally while coordinating externally with Sainsburys' outsourced Oracle delivery team.