Tom Xiao

Stage 4 · AI data engineer · AI agent case

Agents on your data

Let an agent find the right table and read its numbers correctly, without ever seeing more than the person asking is allowed to see. I designed the semantic layer and the access path. It runs in production inside a financial platform.

RoleDesigned the approach and pushed it into the platform
StatusIn production at a financial platform
PrincipleThe agent never chooses who it is

One question, end to end

Definitions first, then a draft, then a person, then the query. The caller's identity is added by the gateway at the last step. Everything here is sample data.

data-agent · one questionsample data
You
What was the settled balance per day last week?
  1. find_tablesThe semantic layer says what the table and its values mean.definitions
  2. draft_sqlThe model drafts SQL from those definitions.model
  3. confirmNothing runs until the person agrees.you
  4. run_querySubmit, poll, fetch a page. Cancel is available.as you
Draft SQL, using the definition "2 = settled"
SELECT day, SUM(amount)
FROM balances_daily
WHERE status = 2  -- 2 = settled
  AND day >= DATE '2026-11-09'
GROUP BY day
Runs as you, with your permissionsConfirmed
What reaches the warehouse
model sendsrun_query(sql=…)
gateway addsuser = session.user
runs asyour account, your permissions
submit→poll→fetch pagecancel
2026-11-09118,420.00
2026-11-10121,775.50
2026-11-11119,008.25
done · 4 steps

How it stays correct and inside its permissions

Definitions come first

  • For the core tables: what each one means, what each value means, and which business questions it serves
  • Dozens of core tables and a couple of hundred value meanings, written down once and read by the agent every time

Identity is injected

  • The gateway attaches who is asking. The model cannot pass a user as an argument
  • The query runs on that person's own account, so the permissions already in place apply

Long queries are four calls

  • Submit, poll, fetch a page and cancel, so a slow query never blocks the conversation
  • The user can stop it at any point

A person confirms

  • Generated SQL is shown first and runs only after the user agrees
  • The confirmation says whose permissions it will use

The decision: judge the result, not the SQL

Two correct queries rarely look alike, and one wrong value meaning looks almost identical to the right query. So evaluation compares what the SQL returns.

Scoring an agent's SQL sample

Reference SQL
SELECT day, SUM(amount) FROM balances_daily WHERE status = 2 GROUP BY day

Case 1. Same meaning, different wording

SELECT b.day, SUM(b.amount) AS total FROM balances_daily b WHERE b.status IN (2) GROUP BY 1
Compare the SQL textdiffersa text match would fail it
Compare the resultssame rowspass

Case 2. One wrong value meaning

SELECT day, SUM(amount) FROM balances_daily WHERE status = 1 GROUP BY day
Compare the SQL textone character aparta text match would nearly pass it
Compare the resultsdifferent rowsfail: 1 is pending, not settled

What I took from it

  • When an agent gets the number wrong, look at the definitions before the model.
  • Authority belongs to the gateway. If the model can name the user, it can name someone else.
  • A confirmation step costs one click and removes a whole class of silent mistakes.

Text-to-SQL fails on definitions, not on the model.

Next

Work with me

Need an agent that gets the numbers right?