gizmoduck/docs/local-workflows.md

Local owner workflows

This guide covers the local MVP from setup through import, categorization, reporting, export, and verified backup. Commands assume PowerShell in the repository root. Financial source files, review JSON, exports, backups, and command output that contains financial details must remain local.

Start the application

Follow the secret setup in local development, then run:

docker compose config --quiet
docker compose build
docker compose up -d db
docker compose run --rm manage migrate
docker compose up -d web
Invoke-RestMethod http://127.0.0.1:8000/health/

Open http://127.0.0.1:8000/. The application and PostgreSQL listen only on loopback. The web UI uses port 8000; PostgreSQL uses 127.0.0.1:5432 for local database tools such as DBeaver. The database name is gizmoduck. Use gizmoduck_app with the RUNTIME_PASSWORD value from the ignored .env for routine inspection. The migration and bootstrap roles have broader privileges and are for maintenance only.

The navigation provides Accounts, Transactions, Categories, Payees, Reconciliation, Reports, Imports, and Exports. Create accounts, categories, and payees before assigning them. References may be archived, but historical transactions continue to show them. Transaction editing is limited to description, payee, and category. Amount, dates, account, kind, import provenance, transfers, refunds, statements, and installment identity are immutable. Corrections use a new transaction or the domain workflows described in the financial domain guide.

Import local files

1. Create the destination accounts through Accounts. Bank statements need a bank account; Ourocard sections need credit-card accounts. Only active BRL accounts can be mapped.
2. Copy up to 10 source files into the ignored repository-root imports/ directory. Use bank-account CSV or Banco do Brasil/Ourocard TXT. Keep real documents out of tests/fixtures/. The web and management services share this local directory through a Compose bind mount.
3. Open http://127.0.0.1:8000/imports/ and choose Scan files. The page shows filenames, detected profiles, section numbers, and safe status codes.
4. Choose an account for every section and select Prepare imports. Check the per-file status and planned counts. Preparation writes no transactions.
5. Select Import ready files to create clean new transactions and finish exact reimport skips. Each file commits independently; one file needing attention does not stop another ready file. The result shows created, matched, and skipped counts without displaying source rows or amounts.
6. Resolve the listed exceptions and scan again. After checking successful results, move processed documents to another ignored local directory such as documents/ before copying the next group into imports/.

More than 10 files returns SCAN_TOO_LARGE and leaves the entire scan untouched. Reduce the group and scan again. Scans and prepared selections expire after 15 minutes; refresh by scanning again if a selection expires or a file changes. The signed selections stay in the forms, while the browser session retains only compact safe result summaries. Import actions use CSRF protection and record the local human actor HUMAN/local-owner.

INVALID_SECTION_MAPPING requires a current active BRL account for every section. FILE_CHANGED, STALE_REVISION, or STALE_PREVIEW requires a fresh scan and preparation. MISSING_CREATION_DATE or INVALID_CREATION_DATE means the partial Ourocard export lacks valid creation-date context; supply a valid export or use the manual workflow with explicit date context. REVIEW_REQUIRED needs manual review of the file's issues or duplicate suggestions. Unsupported or unrecognized files need a supported export. IMPORT_FAILED means that file did not finish; investigate locally and use the advanced workflow to inspect or cancel any unfinished preview before retrying. Raw exceptions and review details are never displayed on Imports.

Advanced manual import review

Create the destination account at http://127.0.0.1:8000/accounts/new/. In DBeaver, retrieve its UUID without exposing transaction data:

SELECT id, name, kind, currency, archived
FROM finance_account
ORDER BY name, id;

Copy the source document into the ignored imports/ directory. These commands use the tracked synthetic CSV; replace source.csv and the UUID for a real local import. Never copy a real document into tests/fixtures/.

New-Item -ItemType Directory -Force imports | Out-Null
Copy-Item tests/fixtures/synthetic/ingestion/bank.csv imports/source.csv
git check-ignore imports/source.csv imports/mapping.json imports/review-1.json
$account = '11111111-1111-4111-8111-111111111111' # Replace with the account UUID.
@{"1"=$account} | ConvertTo-Json | Set-Content -Encoding utf8 imports/mapping.json
$actor = @('--actor-kind', 'HUMAN', '--actor-identifier', 'local-owner')
docker compose run --rm manage import_review preview `
  --input source.csv --profile BANK_CSV_V1 --mapping mapping.json @actor

Copy the printed batch UUID, export the review, and inspect every candidate, issue, and duplicate suggestion:

$batch = '22222222-2222-4222-8222-222222222222' # Replace with the printed batch UUID.
docker compose run --rm manage import_review export `
  --batch $batch --output review-1.json @actor
notepad imports/review-1.json

Edit the top-level decisions array. Every row needs CREATE, MATCH, or SKIP; use the actual locators and current revision from the review. Apply the file, export a fresh review under a new name, and commit only when all issues are resolved:

docker compose run --rm manage import_review apply `
  --batch $batch --input review-1.json @actor
docker compose run --rm manage import_review export `
  --batch $batch --output review-2.json @actor
notepad imports/review-2.json
$revision = 2 # Replace with the revision printed by apply/export.
docker compose run --rm manage import_review commit `
  --batch $batch --revision $revision @actor

Only commit writes transactions and provenance. Preview, export, and apply are inert. See reviewed imports for all profiles, decision fields, duplicate handling, refresh, cancellation, and Ourocard date contexts.

Categorize and report

Open http://127.0.0.1:8000/transactions/, select an imported transaction, and choose Edit annotations. Assign an active category or payee and save. These POST forms use CSRF protection and record the fixed local human actor HUMAN/local-owner in the audit trail.

Open http://127.0.0.1:8000/reports/monthly/, select one calendar month and one currency, then choose Show report. Reports use posting dates. Income is inflow; expenses increase spending; refunds reduce spending in their posting month. Transfers, including card-payment transfers, and opening balances are excluded from cash flow. A card purchase is counted once as an expense. Spending is grouped by expense category with a visible Uncategorized row, and account balances remain separate by account and currency.

Reconcile an account balance

Open Reconciliation from the navigation. Select the account, enter the checkpoint date and the institution balance, then choose Preview to review the ledger balance and difference. Choose Confirm checkpoint only after the preview is correct. Checkpoint dates are inclusive, so the ledger includes every transaction posted on that date.

A checkpoint is immutable. To correct an earlier checkpoint, record a new one for the appropriate account and date. Newly imported activity or a transaction added later with a backdated posting date can change a previously reconciled status; review the reconciliation status and record a new checkpoint when the institution balance has been verified.

Export portable data

Use http://127.0.0.1:8000/exports/ to download a ZIP, or create one under the ignored exports/ directory:

New-Item -ItemType Directory -Force exports | Out-Null
git check-ignore exports/local-export.zip
docker compose run --rm manage export_finances --output local-export.zip

Choose a new plain .zip name for each run; exports never overwrite an existing file. The ZIP contains UTF-8 CSV files for accounts, payees, categories, card statements, installment plans, transactions, transfers, and source records. Its manifest.json records schema version, creation time, row counts, and SHA-256 checksums. Audit events and unfinished import reviews are excluded. Command output contains only the package name, format version, and row counts.

Back up and verify restoration

Create a PostgreSQL 17 custom-format dump and checksum manifest under the ignored backups/ directory:

New-Item -ItemType Directory -Force backups | Out-Null
git check-ignore backups/local-backup.dump backups/local-backup.dump.json
docker compose run --rm manage backup_database --output local-backup.dump

Choose a new plain .dump name for each backup. The command never overwrites an existing dump or manifest. Its output contains only format and table counts. The manifest records the source database, PostgreSQL client version, migrations, row counts, byte size, and SHA-256 digest.

The supported verification command creates synthetic data in disposable test-db, backs it up, restores it into disposable restore-db, checks migrations and table counts, then clears both databases. It deliberately runs two complete cycles to prove cleanup and repeatability:

docker compose --profile backup run --rm verify-backup python manage.py test_backup_restore

This acceptance check never restores the live db/gizmoduck database. Verify a named backup against the empty disposable restore database with:

$backup = 'local-backup.dump' # Replace with the exact backup name.
docker compose --profile backup run --rm verify-backup python manage.py verify_backup `
  --input $backup --target-name gizmoduck_restore `
  --target-host restore-db --target-user gizmoduck_restore
docker compose --profile backup rm -sf restore-db

The named verification checks its manifest and digest, restores it, and compares migrations and table counts. Removing restore-db afterward clears only the disposable tmpfs target so the next verification starts empty. The command rejects the configured live host and database pair.

Deliberate live recovery

Live recovery replaces financial state. Perform it only after identifying the exact dump and its matching .json manifest. Keep the web service stopped until all restore checks pass.

First stop writes and take a separate safety backup of the current live database:

docker compose stop web
$safety = 'pre-restore-safety-20260908T120000Z.dump' # Use a new timestamped name.
docker compose run --rm manage backup_database --output $safety

Verify the intended named backup with the disposable procedure above before touching the live database. Then name both the backup and live target explicitly. The following PostgreSQL 17 command restores into db/gizmoduck; --clean replaces objects represented by the backup, and the single transaction rolls back on an error:

$backup = 'local-backup.dump' # Confirm this exact file and matching manifest first.
$targetHost = 'db'
$targetDatabase = 'gizmoduck'
docker compose run --rm -e BACKUP_FILE=$backup -e TARGET_HOST=$targetHost `
  -e TARGET_DATABASE=$targetDatabase --entrypoint sh manage -c `
  'PGPASSWORD="$DB_PASSWORD" pg_restore --exit-on-error --single-transaction --clean --if-exists --no-owner --no-acl --host "$TARGET_HOST" --port 5432 --username gizmoduck_migrate --dbname "$TARGET_DATABASE" "/app/backups/$BACKUP_FILE"'

Do not restart the web process yet. Bring schema state forward if necessary, then validate Django and the live database connection:

docker compose run --rm manage migrate
docker compose run --rm manage check
docker compose run --rm manage migrate --check
docker compose run --rm manage shell -c "from django.db import connection; c=connection.cursor(); c.execute('SELECT 1'); print(c.fetchone()[0])"

Only after those checks succeed, restart the local application and verify its readiness endpoint:

docker compose up -d web
Invoke-RestMethod http://127.0.0.1:8000/health/

Keep the safety backup until the recovered data has been reviewed. If validation fails, leave web stopped and investigate locally; logs and database output may contain financial data.