gizmoduck/docs/superpowers/specs/2026-09-09-guided-local-import-design.md

Guided Local Import Design

Purpose

Let the local owner place supported financial documents in the ignored imports/ directory and load clean, new transactions through the web app with minimal interaction. The owner scans the folder, selects destination accounts where needed, imports all ready files, and receives safe status summaries.

The workflow preserves Gizmoduck's existing review boundaries: parsing, normalization, validation, duplicate detection, preview, decisions, and persistence remain distinct. The UI orchestrates those boundaries; it does not replace them or expose source contents in browser messages.

User workflow

1. The owner copies supported files to the ignored imports/ directory.
2. The owner opens Imports and selects Scan imports.
3. Gizmoduck lists each safe candidate filename, detected profile, account mapping requirements, row counts, and a safe status.
4. The owner selects an account for each bank CSV and every detected card statement section that needs mapping.
5. The owner selects Import ready files.
6. Each ready file completes its own preview, automatic-decision, and commit lifecycle. The page reports created, matched, and skipped counts.
7. Files needing attention remain uncommitted. The owner corrects the account choice or required context, rescans, and imports when the file becomes ready.

Scope and file handling

The scanner examines only direct files under IMPORT_ROOT, never follows symlinks, rejects traversal, and applies the existing 10 MiB input limit. It supports only the existing Banco do Brasil bank CSV and Ourocard TXT formats. OFX, PDFs, unknown extensions, and unrecognized layouts are reported as safe exceptions and never parsed or committed.

The scanner retains no source rows, descriptions, amounts, or review exports in templates, messages, or logs. Those remain local source material governed by the data policy. The web UI displays filenames, profile labels, account mapping state, safe reason codes, and counts only.

Readiness and automatic decisions

Each scanned file has one of these statuses:

• Ready: every candidate can be created without a validation issue or unresolved duplicate.
• Already imported: every source candidate is already attached through existing provenance; the file is skipped without financial writes.
• Needs attention: the file has an unsupported/unrecognized profile, malformed or unresolved candidate, missing account mapping, ambiguous potential duplicate, or missing/invalid statement context.

The import action creates a normal existing preview per ready file. It applies CREATE to clean candidates. Exact source reimports receive SKIP with ALREADY_IMPORTED; they produce no transaction and are not an exception. Any potential duplicate that is not an exact existing source record requires review and prevents that file from committing automatically.

An exception in one file does not prevent other ready files from committing. Every file retains the import service's atomic commit boundary; a failed file creates no partial transactions or provenance. All import lifecycle actions use the local human actor and retain existing audits.

Account mapping

The scan presents an account choice for every source section needed by the detected profile. Bank CSV has one section. Ourocard documents may have multiple sections and require a choice for each section. The UI uses active BRL accounts and cards only. It never maps an owner name or filename to an account automatically.

The server validates that every submitted mapping corresponds to the current scan and to an active account. Missing, archived, stale, or altered mappings are exceptions; they cannot be committed.

Ourocard partial-statement dates

OUROCARD_PARTIAL_V1 gains a bounded context extractor. It reads and validates the statement creation date from line 21 of the TXT document. That date supplies the statement year and an explicit import interval. Candidate month/day values later than the creation month belong to the preceding calendar year; remaining values belong to the creation year. This handles statements crossing New Year without using the computer clock or filesystem metadata.

If line 21 is absent, malformed, ambiguous, or does not yield an interval that resolves every candidate date uniquely, the file is Needs attention and is not committed. The existing explicit parser context remains available for the management command; this UI-only extractor supplies equivalent validated context before invoking the existing parser contract.

Components

• ingestion receives a folder scanner, safe profile detector, and a UI-orchestration service that constructs existing previews and applies only the approved automatic decisions.
• core receives Import navigation, scan/mapping/import forms and views, and server-rendered status templates.
• Existing parser, normalization, validation, duplicate, persistence, audit, and import-batch models remain their own boundaries.
• The management command remains supported for advanced/manual reviewed import workflows.

Error handling and security

All POST actions use CSRF protection and revalidate scan tokens, file identity, profile, mappings, and source metadata before creating a preview or committing. The UI returns field errors and safe status codes. It does not reveal source text, paths outside the allowed folder, raw parser exceptions, or database errors. Stale scans require a new scan; the server does not reuse a changed file under an old readiness result.

Verification

Tests use only synthetic fixtures and cover:

• direct-folder scanning and supported-profile detection;
• rejected path/symlink/oversize/unsupported input without source disclosure;
• bank and multi-section card account mapping with active BRL references;
• clean automatic creation through the existing audit and import lifecycle;
• exact reimport skipping through source provenance;
• an exceptional file not blocking an unrelated ready file;
• ambiguous duplicates, malformed candidates, stale scans, mapping tampering, and CSRF rejection;
• line-21 partial-statement date extraction, invalid context, and New Year rollover; and
• focused web tests plus full Django, migration, permission, and integration checks.