gizmoduck/docs/imports.md

Reviewed local imports

Imports require a saved preview and explicit row decisions before financial records are written. The import_review management command supports preview, export, apply, refresh, commit, and cancel. Every operation requires --actor-kind HUMAN|APPLICATION|AGENT and --actor-identifier.

Keep source documents, mappings and review JSON in the ignored repository-root imports/ directory. The management container mounts it at /app/imports. Command paths may be absolute under that root or relative to it; traversal and symlink escapes are rejected. Output files must not already exist: choose a new filename for every export. Input documents and JSON files are limited to 10 MiB. Review files contain financial descriptions and must never enter Git, logs or external services. See data policy.

PowerShell workflow

Run the setup and migrations in development first. Create the destination bank account or card through the existing financial services; use its UUID below. These names and UUIDs are illustrative, synthetic examples.

New-Item -ItemType Directory -Force imports | Out-Null
git check-ignore imports/source.csv imports/mapping.json imports/review-1.json
@'
{"1":"11111111-1111-4111-8111-111111111111"}
'@ | Set-Content -Encoding utf8 imports/mapping.json
$actor = @('--actor-kind', 'HUMAN', '--actor-identifier', 'local-owner')
docker compose run --rm manage import_review preview `
  --input source.csv --profile BANK_CSV_V1 --mapping mapping.json @actor

Place the source CSV in imports/source.csv before previewing it. Substitute the batch UUID printed by preview, then export its review:

$batch = '22222222-2222-4222-8222-222222222222'
docker compose run --rm manage import_review export `
  --batch $batch --output review-1.json @actor
notepad imports/review-1.json

Read items, including candidate amounts/dates, validation issues and duplicate suggestions. Edit only the top-level decisions array. Preserve every other field, including each item's stored decision. Alternatively save a minimal object with exactly batch_id, integer revision, and decisions. Unknown keys, repeated JSON keys, repeated locators, foreign batch items, stale revisions and changes to source evidence are rejected.

{
  "batch_id": "22222222-2222-4222-8222-222222222222",
  "revision": 1,
  "decisions": [
    {"locator": "line:2", "action": "CREATE"},
    {"locator": "line:3", "action": "SKIP", "reason": "USER_EXCLUDED"}
  ]
}

Use the actual locators in the exported file. Apply the complete decision set, then export again to inspect the resulting issues and latest revision:

docker compose run --rm manage import_review apply `
  --batch $batch --input review-1.json @actor
docker compose run --rm manage import_review export `
  --batch $batch --output review-2.json @actor
notepad imports/review-2.json
$revision = 2 # Replace with the revision printed by apply/export.
docker compose run --rm manage import_review commit `
  --batch $batch --revision $revision @actor

Only commit writes financial transactions, source provenance and new installment plans. It posts all accepted rows atomically. Repeating a successful commit with its latest revision returns the recorded result. If live account, reference or duplicate information has materially changed, refresh and review again. Refresh increments the revision and clears all decisions:

docker compose run --rm manage import_review refresh `
  --batch $batch --revision $revision @actor
docker compose run --rm manage import_review export `
  --batch $batch --output review-refreshed.json @actor

To abandon a preview, use its current revision:

docker compose run --rm manage import_review cancel `
  --batch $batch --revision $revision @actor

Cancellation creates no financial entries and prevents later commitment. Terminal output contains batch identifiers, state, revision and counts; inspect the local review file for financial details. Errors expose only safe codes.

Decisions and duplicate handling

Every candidate needs one explicit decision. Applying a file replaces the complete decision set. The supported actions are:

• CREATE: create a transaction. Optional annotations are kind, category_id, payee_id, statement_id, refund_of_id, installment_plan_id, and new_plan. Possible duplicates require "acknowledge_duplicate": true after review.
• MATCH: attach the source to the existing transaction_id selected from compatible duplicate suggestions. The immutable financial fields and signature must still agree. Existing annotations are preserved.
• SKIP: omit the item with a reason code: USER_EXCLUDED, MALFORMED, UNRECOGNIZED, ALREADY_IMPORTED, or SECTION_EXCLUDED. Malformed rows and unknown lines require explicit skipping; file-level corruption cannot be overridden. Exact already-imported source items cannot be created or matched again; acknowledge them with SKIP / ALREADY_IMPORTED.

For example, an installment decision can include "new_plan": "synthetic-plan-a". This is a batch-scoped plan name, not an existing UUID; the account and installment count come from the candidate. Reuse the same name only for rows belonging to the same plan, or select an existing installment_plan_id. Plan identity is never inferred from merchant names, and missing installments are never generated. Refunds can remain unlinked or use an explicit compatible refund_of_id. No missing statements or transfer counterparts are created automatically.

Section mapping JSON maps ordinal string keys such as "1" and "2" to account UUIDs, or null to exclude a section. Map every section explicitly; several sections may share one account. Owner names are never database lookup keys. Correct mappings or immutable dates/amounts by creating a new preview from a corrected input. They cannot be rewritten in decisions.

Supported compatibility profiles

All profiles decode UTF-8 (including BOM), then Latin-1 without replacement characters. NUL bytes, files over 10 MiB and more than 20,000 candidates are rejected. Initial profiles require BRL destination accounts. Source dates become both occurrence and posting dates. OFX and PDF are unsupported.

Profile	Accepted layout
BANK_CSV_V1	Bank account CSV, comma-delimited with CSV quoting, at least six header columns; date at column 0 as DD/MM/YYYY, description at 2, signed decimal-point amount at 5. Each record must match the header width.
OUROCARD_FULL_V1	Card TXT, DD.MM.YYYY, with either a 37-character description/location field plus a two-character country field, or a 33-character description/location field plus a seven-character country field; debit and credit use Brazilian decimal notation. A separate PGTO DEBITO CONTA layout is recognized.
OUROCARD_PARTIAL_V1	Card TXT, DD/MM, 43-character description/location field, country, debit and credit; requires an explicit date interval.

CSV header aliases are case/accent insensitive: date accepts Data, Data movimento, Data do movimento, or Data lancamento; description accepts Historico, Descricao, or Descricao do lancamento; amount accepts Valor, Valor lancamento, or Valor do lancamento. Other columns are retained only as layout positions. Recognized balance rows are excluded and counted.

Card amounts use credit minus debit. Nonzero amounts in both columns block creation pending explicit review; known account-debit payment lines propose an unmatched positive transfer. Card credits propose refunds, bank credits income, and debits expenses. PARC n/total supplies installment metadata. Section headings establish ordinal mappings; unknown nonblank text remains visible for acknowledgement, and transaction-like malformed lines are never silently lost.

Yearless dates require both --context-start and --context-end as ISO dates in an inclusive interval of at most 366 days. A date must resolve uniquely; the importer never guesses the current year. For example:

docker compose run --rm manage import_review preview `
  --input card.txt --profile OUROCARD_PARTIAL_V1 --mapping card-mapping.json `
  --context-start 2025-12-01 --context-end 2026-01-31 @actor