Email setup

View as Markdown

This guide walks you through wiring up the self-hosted finance dashboard to a Gmail inbox so the IMAP poller daemon can ingest bank alert emails and turn them into transactions.

For self-hosters with the project cloned and Docker installed. No prior Gmail IMAP or email-forwarding experience required. By the end, your IMAP poller will fetch bank emails from a dedicated Gmail inbox, parse them, and display new transactions on the dashboard. Follow the steps in order — later steps assume earlier ones are complete.

How it works at a glance.

Your bank sends transaction alerts to your personal Gmail.
A Gmail filter on your personal account forwards those alerts to a dedicated finance-only Gmail account.
The imap-poller Docker service logs into that dedicated account over IMAP, fetches new bank emails, and runs them through the same parser pipeline as the AWS Lambda stack.
Parsed transactions land in data/finance.db and show up on the dashboard.

The supported banks for direct purchase / transaction alerts are RBC, CIBC, MBNA, Simplii, and PC Financial. Interac e-Transfer notifications are handled separately — any Canadian bank that uses the standard payments.interac.ca sender will be parsed.

0. Set your timezone (non-Pacific users)

The app defaults to America/Los_Angeles. Before you start ingesting transactions, change it from the dashboard: Settings → Timezone. The selector lists every IANA zone your browser knows about, and the Detect from browser button picks your local one in a click. The setting is instance-wide and controls how transactions are bucketed into days and months, and how “latest transaction age” is computed.

If you’d rather configure it before launching the dashboard, edit the timezone key in data/config.json directly:

{ "timezone": "Europe/Berlin" }

A note on switching mid-stream: the sort key is local-time and is written once at ingest, so changing timezone does not rewrite history. New transactions bucket in the new zone; older rows keep their original prefix. For most users this is invisible. For users who move continents, midnight-adjacent transactions ingested before the switch may group on the wrong calendar day in the Journal until they age out.

1. Create a dedicated Gmail account

Why a dedicated account, instead of pointing the poller at your main inbox?

Isolation. The poller marks messages as \Seen and stores a last-seen UID cursor. Running it against your personal inbox would mutate state you care about and race with the way you read mail.
Blast radius. If the App Password is ever leaked, you can burn the whole account and forwarding rule without touching your primary email.
Easier revocation. To disconnect the dashboard later, delete the App Password — or the whole Gmail account — with no risk of clobbering unrelated settings on your main account.

Pick a strong, unique password and store it in a password manager. You will also create an App Password later (step 3), which is what the poller actually uses. Signed in as the dedicated account, move on to step 2.

2. Enable 2-Step Verification on the dedicated account

Google will not let you create App Passwords until 2-Step Verification (2SV) is enabled on the account.

While signed in to the dedicated account, go to myaccount.google.com/security.
Under How you sign in to Google, click 2-Step Verification.
Follow Google’s walkthrough — detailed instructions live at support.google.com/accounts/answer/185839 (the canonical “Turn on 2-Step Verification” article). Google’s walkthrough now recommends Google prompts on a signed-in phone or a passkey as the primary method; SMS/voice codes and authenticator apps are still supported fallbacks.
Finish the setup until the page reports “2-Step Verification is On”.

Double-check you are configuring 2SV on the dedicated finance account, not your personal Gmail.

3. Generate a Gmail app password

The App passwords option only appears after 2-Step Verification has been enabled. If you don’t see it, go back and finish step 2.

Visit myaccount.google.com/apppasswords (or navigate to Google Account → Security → 2-Step Verification → App passwords).
Re-enter your password if prompted.
In the App name field, enter something recognizable like finance-dashboard-imap. This is a label for your own reference; Google does not use it for anything other than display.
Click Create. Google will display a 16-character password (the UI shows it as four groups of four with spaces — the spaces are cosmetic and may be omitted when pasting).
Copy it now and store it in your password manager. Google will not show it again.

Notes:

The poller strips spaces from the App Password automatically, so pasting it with or without spaces both work.
This App Password is what goes into IMAP_PASSWORD in step 6 — not your regular Google account password.

4. Confirm IMAP is enabled

For personal Gmail accounts, IMAP is always on as of January 2025 — Google removed the Enable IMAP / Disable IMAP toggle, and the option no longer appears under Forwarding and POP/IMAP. You can skip this step if you signed up at gmail.com. The poller will connect to imap.gmail.com:993 over SSL without any further configuration.

For Google Workspace accounts (custom domains administered through admin.google.com), IMAP access is still controlled by an admin. If your dedicated account is on a Workspace domain, an admin needs to enable POP and IMAP under Apps → Google Workspace → Gmail → End User Access. Reference (Workspace admin docs): support.google.com/a/answer/105694. For end-user Gmail forwarding setup (separate from IMAP), see support.google.com/mail/answer/10957.

5. Forward bank alerts from your personal Gmail

Switch accounts: sign in to your personal Gmail (the one that actually receives bank alerts today).

5a. Add the forwarding address

Gear icon → See all settings → Forwarding and POP/IMAP.
Click Add a forwarding address and enter your dedicated account (e.g. [email protected]).
Google sends a confirmation email to the dedicated account. Sign in there, open the confirmation, and click the verification link.
Back on the personal account, refresh the settings page and confirm the dedicated address is listed as verified.

Do not change the default (“Forward a copy … and keep Gmail’s copy in the inbox”) globally — you only want to forward bank alerts, not all email. Use a filter for that, below.

5b. Create a filter that matches bank senders

In Gmail, click the filters icon in the search bar (or use Settings → Filters and Blocked Addresses → Create a new filter).
In the From field, paste one of the sender expressions below depending on which banks you use. To cover multiple banks with a single filter, join them with OR.
Click Create filter.
Check Forward it to and choose your dedicated address.
Optionally also check Skip the Inbox (Archive it) and/or Apply the label → Forwarded/Finance so the clutter stays out of your personal inbox.
Click Create filter to save.

Sender patterns, matching the senders the pipeline parses — by sender-domain match, or body-text detection for Interac-routed alerts (src/finance/email_pipeline.py):

Bank	Suggested Gmail `From:` filter
RBC	`from:(@alerts.rbc.com)`
CIBC	`from:(@cibc.com)`
MBNA	`from:(@mbna.ca)`
PC Financial	`from:(@pcfinancial.ca)`
Simplii (e-transfers only)	`from:(@payments.interac.ca)`
Interac	`from:([email protected] OR @payments.interac.ca)`

Simplii note. The Simplii parser only handles Interac e-Transfer notifications, which arrive from payments.interac.ca rather than simplii.com. Filter on the Interac sender domain for Simplii. (Direct purchase alerts on @simplii.com are not currently supported by the parser.)

One catch-all filter covering every supported institution looks like:

from:(@alerts.rbc.com OR @cibc.com OR @mbna.ca OR @pcfinancial.ca OR @simplii.com OR @payments.interac.ca)

Interac e-Transfer notifications from RBC, Simplii, or any other Canadian bank all come from payments.interac.ca — keep that line even if you don’t think you receive them today.

@simplii.com is in the filter on purpose, even though those direct alerts aren’t parsed yet: forwarded copies land in the Needs review queue instead of being dropped, AI extraction (if enabled) can recover the transactions, and the captured emails are exactly the samples a future Simplii parser needs.

If you see bank alerts in your inbox from a slightly different domain (e.g. a regional RBC address), add it to the filter’s from: list — the pipeline matches on substrings of the domain, so anything ending in one of the domains above is picked up automatically.

5c. Sanity check

After saving the filter, send yourself a small real transaction, or wait for the next bank alert. Within a minute or two, the email should appear in the dedicated account’s inbox. If it lands in Spam, open it and click Not spam so future alerts skip the spam folder.

6. Configure `.env`

.env is a local file at the repo root that stores configuration as environment variables. It’s gitignored — never commit it. The Docker Compose stack reads it automatically when you run docker compose up.

At the repo root you’ll find .env.example. Copy it to .env (if you haven’t already) and fill in the IMAP block. The variable names below are the exact ones read by src/finance/imap_poller.py::main() and passed through by docker-compose.yml — don’t rename them.

# IMAP Polling (for self-hosted email ingestion)
IMAP_SERVER=imap.gmail.com                          # Gmail's IMAP host; rarely needs changing
IMAP_PORT=993                                       # IMAP-over-SSL; default is correct for Gmail
IMAP_USER=<[email protected]>                  # the DEDICATED account from step 1
IMAP_PASSWORD=<YOUR_16_CHARACTER_APP_PASSWORD>      # the APP PASSWORD from step 3 (NOT your Google password)
IMAP_FOLDER=INBOX                                   # which mailbox to poll; INBOX is correct for the filter above
IMAP_POLL_INTERVAL=60                               # seconds between polls; 60 is a reasonable default

Optional but recommended:

OPENAI_API_KEY=<sk-...>                             # enables AI category suggestions; without it, categories default to "Miscellaneous"

Defaults baked in: if you omit IMAP_SERVER, IMAP_PORT, IMAP_FOLDER, or IMAP_POLL_INTERVAL, the poller falls back to imap.gmail.com, 993, INBOX, and 60 respectively. IMAP_USER and IMAP_PASSWORD are required — the process exits immediately if either is missing.

Never commit .env. It’s already in .gitignore; keep it that way.

7. Start the stack

From the repo root:

docker compose up -d

This brings up two services defined in docker-compose.yml:

finance — the FastAPI dashboard on port 8000 (open http://localhost:8000)
imap-poller — the long-lived IMAP polling daemon

Both containers share the finance_data volume, so the poller writes transactions directly into the same SQLite DB the dashboard reads.

To stop the stack:

docker compose down

8. Verify it’s working

Stream the poller logs:

docker compose logs -f imap-poller

(You can also use docker logs -f <container_name> directly, where the container name is usually <project>_imap-poller_1 or <project>-imap-poller-1 — docker ps shows the exact name.)

On a healthy startup you should see, in order:

Starting IMAP poller: server=imap.gmail.com, [email protected], folder=INBOX, interval=60s
Connected to imap.gmail.com:993 as [email protected]

After that, one of two things happens on each poll cycle:

Idle tick — nothing is logged until new mail arrives. That’s normal; the daemon only emits a line per fetched message.
New mail — you’ll see Fetching uid=<N> followed by uid=<N>: new transaction stored as … and, once per batch, Processed <N> message(s).

The first real transaction usually appears on the dashboard within one to two poll intervals (so ~1–2 minutes on the default 60-second interval) after the bank email actually arrives in the dedicated account. Refresh the dashboard in your browser to see it.

If you want to force a fast end-to-end test, send yourself a small e-Transfer from one of your supported banks — the alert should flow through forwarding → dedicated inbox → poller → dashboard within a couple of minutes.

Troubleshooting

“Invalid credentials” / authentication failure on connect. Gmail rejects your regular account password for IMAP — you must use an App Password. Re-check that IMAP_PASSWORD is the 16-character value from step 3 and not the password you log in to Google with. If in doubt, generate a new App Password and paste it in fresh. Spaces in the App Password are fine; the poller strips them.

Server rejects LOGIN with an “IMAP disabled” message. On personal Gmail this is rare — IMAP is always on since January 2025. The likely cause is a Google Workspace account where an admin has disabled IMAP for the org. Ask your admin to enable IMAP under Apps → Google Workspace → Gmail → End User Access, then restart the poller: docker compose restart imap-poller.

Not receiving emails / the dashboard stays empty. Three common causes, in order of likelihood:

The bank alert landed in Spam on the dedicated account. Open the message and click Not spam so future alerts go to INBOX.
Your Gmail forwarding filter on the personal account doesn’t match the sender. Check step 5b and confirm each bank you use is in the from: list. Don’t forget payments.interac.ca for e-Transfers.
The forwarding address was added but never verified. Go back to Forwarding and POP/IMAP on the personal account and make sure the dedicated address shows as verified, not pending.

“Connection refused” or timeout errors in the poller log. Something between the container and Gmail is blocking port 993. Confirm:

IMAP_SERVER=imap.gmail.com and IMAP_PORT=993 (no typos, no imaps://, no trailing spaces).
Your host firewall or corporate network allows outbound TCP 993.
If you run behind a VPN, try disabling it briefly to isolate the problem.

The poller automatically retries with exponential backoff (5s → 10s → … up to 5 minutes), so transient network blips self-heal — persistent errors every cycle indicate a config or firewall issue.

App Password option missing, even though 2-Step Verification is on. Some Google Workspace administrators disable App Passwords organization-wide via Admin Console → Security policy. This is common on work or school accounts and can only be re-enabled by the admin. The simplest workaround is a dedicated personal @gmail.com account (step 1). App Passwords are available on personal Gmail accounts with 2SV enabled, except for accounts enrolled in Advanced Protection or whose 2SV is set up with security keys only.

AI categorization (optional)

The dashboard can use OpenAI to auto-assign a category to each new transaction. AI categorization is enabled by default when OPENAI_API_KEY is set; toggle in Settings → Intelligence to disable it. Manual category overrides and merchant aliases run before OpenAI, so local rules always win — OpenAI is only consulted when no override matches.

What gets sent to OpenAI

When a built-in parser recognizes the bank, it extracts the fields locally and OpenAI sees only two things, for the category lookup:

the transaction amount (e.g. 42.50)
the merchant name as it appears in the bank email (e.g. STARBUCKS #1234)

That is the whole payload. Your account number, card number, balance, email address, and every other transaction stay on your machine.

When the full email is sent

If an email can’t be read by a built-in parser — an unrecognized sender, or a known bank that changed its format — and AI categorization is on, Tidings falls back to OpenAI to recover it. The email’s subject and body are sent so the model can decide whether the message is a transaction and pull out the amount, merchant, and type. Whatever the bank put in that email goes with it.

This fallback honors the same opt-out as categorization, so turning AI off under Settings → Intelligence stops it. The surest way to keep everything local is to run without an OpenAI key at all: the pipeline then never calls out, unparsed emails are simply held for review, and recognized transactions fall back to the Miscellaneous category.

How to turn it off

UI: Settings → Intelligence → AI categorization toggle (live; takes effect on the next new transaction)
Config file: set "ai_categorization_enabled": false in data/config.json
No API key: if you never set OPENAI_API_KEY, the flag defaults off and the feature never runs

When disabled, transactions receive the Miscellaneous category. You can re-categorize them by hand in the transaction list, or add a Category Rule under Categorize → Rules so future transactions from the same merchant get the right category automatically — all without any data leaving your machine.

Next step

Your inbox is wired. The last step is notifications: pick a provider so you get alerted when transactions arrive.