Lab notes

From Google Sheets to FinSimLab

When I started my investment journey, I didn’t want to hand over my finances to an insurance company whose advisor would push their high-fee products to me; I wanted to learn to manage my own finances. But knowing how to do that is not trivial, let alone in Ireland, where I live.

So I hired a flat-fee financial advisor who gave me sound advice and a view of my financial future with nice charts and data. When I saw the report I wanted to make all kinds of changes, experiment with a thousand different scenarios and see how they impacted the outcome, to answer questions like what if we delay the house purchase by a year, what if we lower expenses by 5%, what if we invest a bit more, what if there's higher inflation, what if..., what if..., what if...

Spreadsheet

The first thing I tried was Google Sheets. But spreadsheet formulas didn’t cut it, and since I had experience as a software developer, I used Google Sheets’ scripting language (javascript), to write a finance simulator engine. The spreadsheet provided the user-facing side.

I was able to run a lot of experiments with it. Learning what was possible made my meetings with my advisor more productive. I went in with a better understanding and some insights that I wanted to run by him.

I realised it was a valuable tool, so I shared it in the Irish personal finance subreddit and it was well received, but every time I made a change it forced users to make a new copy and re-enter their data, so I wanted to turn it into a website. I just had a stressful job and a family, and it never seemed the right time.

FinSim.ie

But then I lost my job, and by that time AI coding tools had arrived. So I switched from working for someone else to working for myself, at my own pace, and having a ton of fun trying the new coding tools. I used AI to help me create a website closely mimicking the spreadsheet, plugged in the code and the tax model that ran the spreadsheet, and published it as finsim.ie. Again I got a good response and valuable feedback.

Then I started considering the possibility of modelling other countries, which would open the door to experimenting with relocation scenarios and answer questions like what if I retire earlier and move to a country with lower taxes and cost of living? That would be valuable and open up the whole world. So I bought the finsimlab.com domain and got to work.

FinSimLab.com

It took quite a lot more time to develop than I would have predicted, and in the meantime the tools evolved a huge amount. I got to try a lot of different AI models and techniques, I learned what works and what doesn't, that lot of what I used as a profesional software engineer also applies here, plus there's a lot of new techniques that are necessary as well.

But I didn’t use AI just because I’m lazy, which I am, or easily drawn to the latest shiny toy, which I also am. I also used AI because it allowed me to scale in two dimensions: adding tax models for different countries, and translating the site into languages I don’t master.

What's next

And now that I got it to where I think it’s ready for prime time, my focus turns to marketing, which is definitely not my forte. I will not use AI in my communications with you, like this blog or answering to feedback. Genuine human contact is much more important to me than numbers.

Give it a try. I look forward to talking to you, hearing your feedback and helping you get value from this, my passion project.

How FinSimLab Authors Country Tax Models

You may wonder how I added all these countries' tax models. I have a reasonable understanding of the Irish and the Agentinian tax systems, having lived in both. But if other countries are as different from each other as these two, there’s no way for me to hand-model more than a couple of countries. It simply doesn’t scale.

As a solo developer, AI was my only option. I know, LLMs “hallucinate” every word they say, and sometimes that happens to match reality. Asking an AI to “model the tax rules for country X for the simulator” would not work. But the same way there's techniques to get an AI to write complex software, there's also ways to achieve complex goals like this one. So let me explain how I add a new country to the simulator:

  1. Independent research packet

    The first step is to get an AI agent to research the country’s tax system and write an official-source citation packet following a template that has a section for each aspect of the tax system that we care about, with exact source quotes. This is done without any access or knowledge of the simulator, to avoid any bias. The source quotes are critical because if they match the text found in the source, it proves that the AI actually had to read the document. It’s well known that LLMs can make up sources, with likely sounding URLs and everything.

  2. Mechanical quote verification

    The next step is a mechanical verification of the actual presence of the quoted text in the linked source material. A simple script goes through the list of citations, fetches the cited document, and verifies that the quote is present. Scripts can’t hallucinate. If this gate is passed, the research agent did actually read a real document.

  3. Semantic review

    A valid quote doesn’t guarantee that a tax rule claimed to come from the quoted document is actually in the document. So now another AI agent goes through the tax rules and the cited documents, one by one, and tries to find cases where the tax rule doesn't match what the document describes. LLMs like to please the user. This user is pleased if it finds an issue, a mistake. This kind of adversarial review, while not perfect, works quite well.

  4. Targeted addendum

    If any of the last two steps produced findings (invalid quotes or unsupported claims), a new research brief is created to correct the mistake. That goes back to step 1, which produces a research addendum, which then goes through steps 2 and 3 again. And if there’s still issues, another round is made. This keeps going until there’s no more issues. In most cases this takes one or two passes.

  5. Modeling spec

    Now we have a set of tax rules for each aspect of the simulator that has been vetted. Now it’s time to bring in the knowledge about the simulator: what it can model and what it can’t. An AI agent takes the research packet and decides what can be represented directly, what should be approximated, and what is not currently representable by the tax engine. That last part becomes a “gap report” and includes a list of recommendations to expand the tax engine.

  6. Ruleset implementation

    Another AI agent takes the modeling spec and encodes the country’s tax rules as a configuration file. Part of the research packet was to find worked examples and calculators. Those are then turned into tests and used to validate the implementation.

  7. Economic data

    A script then fetches economic figures like inflation, exchange rates, cost of living and rental yield, from sources like the World Bank and Numbeo, and adds that to the country configuration file.

  8. Audit

    The final gate audits the chain from research packet to implemented tax model, looking for dropped rules, unsupported modelled claims, undocumented approximations, label problems, missing tests and remaining issues, and any findings produce a new review request that repeats all the necessary steps.

Do I blindly trust the output of this process? Of course not. I need real people checking the result. Since I can’t hire experts, I let people try it for free if they promise to give me feedback. If they do, they get a promo code.