Lawrence Berkeley National Laboratory

Background

I worked on the financial data infrastructure behind research programs at Berkeley Lab, the DOE national lab in the hills above campus that runs five national user facilities on a budget north of a billion dollars a year. Money there doesn't arrive as one pot. It arrives as federal program awards, each with its own rules, its own reporting cadence, and its own definition of what counts as spent.

That structure is what makes the reporting hard. General ledger entries, invoices, and budget allocations live in different internal systems and disagree with each other in ordinary ways, so before anyone can ask whether a program is on track, the numbers have to be reconciled. Most of that was happening by hand, in spreadsheets, once a month.

Building the pipeline

I engineered Python and SQL ETL that ingests, normalizes, and reconciles multi-source financial data into a central PostgreSQL warehouse, so producing a report became a query instead of a week of assembly. The interesting part was never the extraction, it was the normalization: the same vendor written four different ways, entries landing in a period after it closed, allocations that only balance once you know which correction supersedes which.

Flagging spend that matters

On top of that warehouse I trained a TensorFlow LSTM on time-series budget data to learn each portfolio's own spending shape and flag deviations from it, then auto-generated variance reports across $40M+ in portfolios. Research spending is lumpy by nature: a quiet quarter followed by a large equipment purchase isn't an anomaly, it's a procurement cycle. A model that treats every spike as a problem produces reports nobody opens, so the goal was less about finding outliers and more about producing a number a program manager can defend.

Getting it into production

I architected and deployed a FastAPI microservice with asynchronous endpoints and PostgreSQL connection pooling to serve the model and report generation at scale, handling 500+ report-generation requests a day. Report generation is bursty and I/O-bound rather than fast and uniform, so most of the design went into not holding a connection open while waiting on the database, and not letting one large portfolio's report block everyone else's.