Screen-Scraping to APIs: Modernising Legacy Access

Before banks offered a clean way to share data, fintechs still needed it — to aggregate accounts, verify balances, or initiate payments. Their solution was audacious and fragile: screen-scraping. The user handed over their online banking username and password, the fintech logged in as if it were the customer, and it read the resulting web pages to extract balances and transactions. It worked well enough to build an industry, and it was a security and reliability nightmare. The move from scraping to APIs is one of the defining modernisations in payments.

How screen-scraping worked

The mechanics were exactly what the name suggests. A user granted a third-party app their bank credentials. The app stored those credentials and used them to log into the bank's website automatically, then parsed the HTML of account pages to pull out the data it wanted. To the bank, this traffic was indistinguishable from the real customer logging in — same credentials, same login page. Some providers ran this at scale across thousands of banks, maintaining brittle parsers for each institution's page layout.

Why it was a bad idea

Screen-scraping had structural problems that no amount of engineering could fully fix:

• Credential sharing. The third party held the customer's full banking password, granting total access with no way to limit scope. If the aggregator was breached, those credentials were exposed.
• No granular consent. Because the app logged in as the user, it could see and do anything the user could. There was no way to grant read-only access to just balances.
• Fragility. Any change to the bank's website — a new field, a redesign — broke the scraper until someone rewrote the parser.
• MFA friction. Multi-factor authentication, designed to stop unauthorised logins, tripped up automated scraping and pushed providers toward storing second factors too.
• Indistinguishable from fraud. Banks could not tell legitimate aggregation from account takeover, undermining their own fraud controls.

The API alternative

Open banking APIs solve these problems by design. Instead of impersonating the user, a third party requests access through a dedicated, documented interface. The customer authenticates directly with their own bank, approves a specific scope of access, and the bank issues the third party a limited-lifetime access token. The third party never sees the password. Access can be read-only, time-bound, and revoked by the customer at any time. Data comes back as clean structured JSON rather than scraped HTML, so it does not break when a web page changes.

The regulatory push

The transition was accelerated by regulation, most prominently the EU's second Payment Services Directive, PSD2, and the UK's Open Banking initiative. These frameworks required banks to expose secure APIs for account information and payment initiation to licensed third parties, and introduced strong customer authentication (SCA) to protect logins. The regulatory intent was explicit: enable competition and innovation while ending the credential-sharing model. In Europe, the rules pushed firmly toward dedicated APIs and away from scraping, though the two coexisted during transition.

Standardised versus proprietary APIs

Not all APIs are equal. In some markets, regulators or industry bodies defined common standards — the UK's Open Banking specifications, for example — so a third party could integrate once and reach many banks. In others, each bank published its own proprietary API, leaving aggregators to build and maintain dozens of different integrations, an echo of the old per-bank scraper problem. Standardisation is what turns open banking from a compliance checkbox into a genuinely scalable ecosystem, which is why the degree of standardisation varies so much in how useful it is.

Lessons for legacy modernisation

The scraping-to-API story generalises beyond banking. Whenever integration is done by impersonating a human through an interface built for humans, it is fragile, insecure and unscalable. The durable fix is a purpose-built machine interface with scoped authorisation, delegated authentication and structured data. Screen-scraping was a rational response to the absence of that interface — and its decline shows what happens once the proper interface finally exists.

Key takeaways

• Screen-scraping accessed bank data by logging in as the customer with their shared credentials.
• It exposed full credentials, offered no granular consent, and broke whenever a web page changed.
• Open banking APIs let customers authenticate with their own bank and grant scoped, revocable, tokenised access.
• PSD2 and Open Banking mandated secure APIs and strong customer authentication to end credential sharing.
• Standardised APIs scale far better than proprietary ones, which recreate the per-bank integration burden.