How PsyFi Stops FY From Inventing Numbers

The number that shouldn't be there

Here is a failure mode that should worry anyone building AI into personal finance.

You ask an assistant what you're worth. It pulls your accounts, adds them up, and answers:

Your net worth is about $252,000, though your $450,000 mortgage offsets some of it.

The first number is real. The second one is invented. There is no mortgage. No tool returned $450,000. The model produced a figure because a sentence about net worth felt like it wanted one, and the number is plausible, specific, and completely fabricated.

That example is not hypothetical. It is a test case that runs in PsyFi's continuous integration pipeline every time an engineer changes how FY, our financial coaching agent, is built. If FY ever starts producing that answer again, the build fails and the change does not ship.

This post is about why we built that trust test cases, what else runs alongside it, and why we think this layer, not the model is what separates a finance app you can trust from a chatgpt or meta muse demo.

What the industry is learning the hard way

Plaid recently published a whitepaper from its Effects 2026 conference, Powering, building, and governing AI in financial services, collecting lessons from engineering and product leaders at T-Mobile, Brex, and Databricks on moving AI from pilots into production.

Its central finding matches what we have seen building PsyFi: the AI models are not the differentiator. Frontier models improve every quarter and every company can call the same APIs. What separates a product that works from one that only demos well is everything wrapped around the model, the data that gives it context, the architecture that makes it dependable, and the discipline that catches it when it is wrong.

Three lessons in that paper describe decisions we have already made. We want to show our work on each one.

Lesson 1: one agent that does everything makes a great demo

Brex's AI team described a familiar arc. They started with narrow tools that worked. Then, as models improved, they built a single agent with the full handbook and every tool, reasoning across the whole workflow, much like onboarding a new hire.

It demoed beautifully. Production was a different story. User conversations were unpredictable, context varied case by case, every new data source created new ways to misbehave, and a prompt change that fixed one problem quietly broke another. As their Director of AI Products put it, one model that can do it all makes for great demos, but in production the back-and-forth with the user isn't predictable and neither is the context.

Their conclusion was that the approach itself was wrong. Reliability came from breaking work into small, individually testable pieces and recombining them only once each piece was trustworthy on its own.

What we built. FY is not one agent. It is a set of specialists behind a router:

a coaching specialist for investments, tax, and long-term decisions
a money coach for spending, budgets, and day-to-day cash flow
a behavioral specialist for the emotional side of money
a research desk for analyzing holdings and portfolio's in depth

Underneath them sits a library of narrow, single-purpose tools — fetch this specific metric, summarize these transactions, calculate this return — plus a separate family of focused agents that generate insights on your dashboard, each one responsible for a single job like cash flow, recurring expenses, or asset allocation.

None of them is "analyze my finances." Every one is small enough to test in isolation. That is deliberately less impressive to describe and considerably more reliable to use.

Lesson 2: as code gets cheap, the test suite becomes the product

The sharpest observation in the Plaid paper is about where durable advantage now lives. When software was expensive to write, the codebase was the asset. As AI drives the cost of writing code down, the more valuable asset becomes the test suite that defines what "correct" means — because that suite encodes policy nuance, edge cases, and hard-won judgment. A competitor can call the same model API. It cannot easily copy your definition of a good answer.

Brex also found that grading answers on a 1–5 scale didn't work, because neither people nor models could reliably tell a 2 from a 3. Breaking each judgment into binary pass/fail criteria did work.

What we built. A grounding check, and a growing corpus of pass/fail cases around it.

The check does one thing: it takes every dollar figure in FY's reply and tries to reconcile it against the numbers the tools actually returned. Exact matches pass. So do honest derivations — summing the rows FY just showed you, or annualizing a monthly figure. Anything left over is a figure FY produced that no underlying data supports, and it gets flagged and counted.

The corpus captures the judgment calls that make this hard:

The invented mortgage. The example at the top of this post. Must be caught.
The total that doesn't match its rows. Dining $1,200 and Shopping $3,000, described as "about $8,000 total." Must be caught.
The total that does. The same two rows described as "$4,200 total." Must pass — adding up numbers you were shown is not a fabrication.
The annualization. "$500 a month in dividends, roughly $6,000 a year." Must pass.
Crisis resources. If someone is in distress, FY responds to the humans emotions, and surface a support line. Those digits are never treated as suspect financial claims. Must pass, always.

That last case matters more than it looks. A naive number-checker would flag a crisis helpline as an unverified figure and create pressure to suppress it. Deciding, in advance and in code, that support resources are never subject to that check is exactly the kind of judgment the Plaid paper means when it says evals capture institutional knowledge.

These cases run as a gate. A change that makes FY worse at distinguishing a real number from an invented one does not reach you.

Lesson 3: if it fails, but how fast you find out

Brex pairs every real-world failure with both a fix and a new test, so the same problem cannot quietly return. They run fast, predictable checks on every code change and slower, judgment-based ones on a schedule.

What we built, and where we went further. The grounding check does not only run in CI against saved examples. It runs on every real conversation, in production. Every time FY answers you, the figures in that answer are reconciled against the data it retrieved, and anything unmatched is recorded.

We want to be precise about what that does and does not do today, because the whole point of this post is to be checkable rather than impressive. The check measures; it does not censor. FY streams its answer word by word, so by the time a figure can be verified it has already appeared on your screen — text cannot be retracted mid-sentence. So the check's job right now is to make the failure rate visible and to hold it down over time, not to silently block answers. We would rather tell you that honestly than imply a guarantee the architecture doesn't yet provide.

Alongside it, a second, independent model reviews every completed conversation and scores it: did FY route the question to the right specialist, did it reach for the right data, were any numbers unsupported, and was the answer actually good. Those verdicts are written into the record of the conversation and aggregated anonymized on an internal quality dashboard.

The effect is a loop. A failure becomes a permanent test case. The test case prevents the regression. The production monitor finds the next failure. As the paper puts it, organizations stuck in pilots treat AI as a deliverable, while organizations in production treat it as a system that learns.

Why you should care about any of this

This could all read as engineering housekeeping. The consumer research cited in the same whitepaper suggests it is closer to the whole point.

75% of consumers consider it important to know when AI is being used in financial decisions.
78% say AI should be held to the same accountability standard as a human advisor.
80% believe companies should make them whole for AI-driven mistakes.
Around six in ten say they would trust AI more if they understood the reasoning behind it.

The most telling pattern is that the people who use AI most heavily are the ones demanding the most oversight. Even among the most enthusiastic users, roughly nine in ten still want the option to review high-stakes decisions before they happen. Wanting to check the work is not skepticism about AI. It is what using it seriously looks like.

An AI that invents a plausible number in a financial conversation is worse than one that says "I don't have that." A made-up figure is indistinguishable from a real one at a glance, and you may act on it. That is why we would rather spend our engineering time on the machinery that catches invented numbers than on making FY sound more confident.

What this does not mean

Some honest limits, because a trust post that only lists strengths is marketing:

Grounded is not the same as right. These checks verify that FY's figures trace back to your actual data. They do not certify that its judgment is good. FY can cite entirely correct numbers and still give you advice that doesn't fit your situation.
The check measures rather than blocks. As above: today it records unverified figures so we can drive the rate down, and it does not prevent them from reaching you.
Your data still has to be right. If an account is disconnected or a transaction is miscategorized by user, FY reasons faithfully over incomplete information. This is why reconnecting a stale bank link matters more than it seems.
FY is not a licensed financial advisor, and nothing it says is personalized investment advice. It is a coach that helps you see your own money clearly and act on it.

The layer underneath

The Plaid paper closes on the observation that every one of these practices rests on the same foundation: trustworthy, consented, real-time financial data with clear lineage. We agree, and it is the reason PsyFi connects to your accounts through regulated data providers with your explicit permission rather than asking you to type your numbers in by hand.

But the data layer is only where it starts. Between your bank feed and a sentence you are willing to act on sits a lot of unglamorous work: small agents instead of one big one, binary tests instead of vague scores, a permanent test for every failure, and a monitor watching real conversations rather than a curated demo.

None of it demos well. All of it compounds. That is the part we would rather be judged on.

Source: Plaid, Powering, building, and governing AI in financial services (Effects 2026 whitepaper), featuring T-Mobile, Brex, Databricks, and Plaid. Consumer statistics are as cited in that paper. PsyFi is not affiliated with Plaid's conference programming, and the architecture described here is our own.