Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 1 addition & 10 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,6 @@ minance/
- Conversational assistant endpoint with explainable output and drill-down filters
- Database backup and reload feature
- Help center for self-host UX
- Legacy Minance API loader script for dev database seeding
- Responsive web UI covering dashboard, explorer, imports, transactions, analytics, assistant, help, and settings
- Saved views/bookmarks

Expand Down Expand Up @@ -81,14 +80,6 @@ Dry-run summary:
pnpm seed:fixture -- --dry-run
```

## Legacy API seed (dev)

Load accounts + transactions from legacy Minance API into the current dev database (mapped category as tier-2, inferred tier-1 group):

```bash
pnpm seed:legacy-api -- --base-url http://10.0.0.20:18080 --start 2024-01-01 --end 2026-12-31
```

Fixture source of truth:
- `services/api/test/fixtures/deterministic-financial-fixture.js`
- `services/api/test/fixtures/deterministic-financial-store.json`
Expand Down Expand Up @@ -159,4 +150,4 @@ docker compose -f docker-compose.selfhost.yml --env-file .env.selfhost up -d
- AI key encryption uses `AI_CREDENTIAL_SECRET` (set in environment for non-local use).
- Account provider abstraction is exposed via `GET /v1/accounts/providers` and `GET /v1/accounts/providers/:providerId` (self-host default provider is `manual_csv`; direct-link actions return explicit unsupported-action errors).
- CrewAI analysis agent script lives at `services/agents/crewai_analysis_agent.py` (enable/disable with `AI_CREW_ANALYSIS_ENABLED`; install Python deps from `services/agents/requirements.txt`).
- SQLite migration requires `sqlite3` CLI installed on the host machine.
- SQLite storage requires the `sqlite3` CLI on the host machine.
6 changes: 2 additions & 4 deletions docs/json-to-sqlite-migration-runbook.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,8 +13,7 @@ This runbook defines a deterministic, idempotent path for loading a JSON fixture
- Runtime store: `services/api/data/{env}-minance.sqlite` (or the environment-specific override).
- Default JSON fixture input: `services/api/test/fixtures/deterministic-financial-store.json` (or `MINANCE_DATA_FILE` override).
- SQLite foundation bootstrap exists in API startup (`MINANCE_SQLITE_FILE`, `MINANCE_SQLITE_SCHEMA_FILE`, `MINANCE_SQLITE_AUTO_INIT`) and status is observable via `GET /v1/system/storage`.
- Active write paths: auth, imports, transactions, categories/rules, AI settings, assistant queries, saved views, migration runs, audit events.
- Legacy Minance SQLite import endpoint already exists (`/v1/migrations/minance/sqlite`) and is separate from this JSON-to-SQLite cutover.
- Active write paths: auth, imports, transactions, categories/rules, AI settings, assistant queries, saved views, and audit events.

## Target SQLite Schema (Canonical Tables)

Expand All @@ -35,7 +34,6 @@ Use one table per top-level JSON collection (from `services/api/src/store.js`):
- `ai_provider_preferences`
- `assistant_queries`
- `saved_views`
- `migration_runs`
- `audit_events`

Recommended constraints/indexes:
Expand Down Expand Up @@ -85,7 +83,7 @@ Insert in dependency order:
5. `transactions`
6. `ai_provider_credentials`, `ai_provider_preferences`
7. `assistant_queries`, `saved_views`
8. `migration_runs`, `audit_events`
8. `audit_events`

Use idempotent writes:

Expand Down
5 changes: 2 additions & 3 deletions docs/self-host-feature-matrix.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,6 @@ This document defines how Minance Next maps Copilot-style product expectations t
| Authentication and sessions | Email/password auth, session refresh, user profile | Supported | `POST /v1/auth/signup`, `POST /v1/auth/login`, `POST /v1/auth/refresh`, `GET/DELETE /v1/users/me` | Local session/token storage in SQLite runtime data. |
| Canonical data store | Durable relational storage | Supported | Production SQLite runtime store (`services/api/data/production-minance.sqlite`) is active, with foundation/bootstrap status exposed at `GET /v1/system/storage` | JSON fixtures remain only for explicit test or fixture-import flows; see [JSON-to-SQLite runbook](./json-to-sqlite-migration-runbook.md). |
| CSV import and mapping | Bank CSV ingestion with review, mapping, diagnostics | Supported | Implemented import workflow (`/v1/imports*`) with processed-row editor and dedupe | Deterministic parser + manual mapping/editing when heuristics/AI confidence is low. |
| Legacy Minance migration | Import from legacy Minance SQLite DB | Supported | `POST /v1/migrations/minance/sqlite` and migration report endpoint are implemented | Requires host `sqlite3` CLI. If unavailable, operator uses CSV import path. |
| Transactions lifecycle | Create/edit/delete and filter transactions | Partially supported | Manual CRUD and query filters are implemented (`/v1/transactions*`) with canonical day-boundary semantics documented in [`transaction-date-day-boundary-semantics.md`](./transaction-date-day-boundary-semantics.md) | Bulk operations, review workflows, and parity details tracked by open parity tasks. |
| Categories and rules | Category CRUD, strategy tuning, mapping rules | Partially supported | Category list/create/update/delete, rules create, strategy get/update implemented | Group/type/budget parity and full Categories-tab parity are tracked separately. |
| Accounts workflows | Dedicated accounts onboarding/settings flows | Partially supported | Accounts API create/update/list, supported-type, balance-history, and manual-adjustment endpoints are implemented (`/v1/accounts*`), while the Accounts tab UI is still placeholder | Manual/CSV provider fallback remains default; deeper account UX/settings/archive flows are tracked by open tasks. |
Expand All @@ -34,7 +33,7 @@ This document defines how Minance Next maps Copilot-style product expectations t
- Required runtime dependencies:
- Node.js runtime for web/API.
- Optional dependencies:
- `sqlite3` CLI for legacy migration flow and SQLite foundation bootstrap/validation.
- `sqlite3` CLI for SQLite foundation bootstrap/validation.
- AI provider APIs (OpenAI/OpenRouter/Anthropic/Google) only when the operator/user configures BYOK keys.
- Explicitly non-required for baseline operation:
- Proprietary bank-link providers.
Expand All @@ -55,7 +54,7 @@ This document defines how Minance Next maps Copilot-style product expectations t
- [`self-host-operations-runbook.md`](./self-host-operations-runbook.md)
- [`self-host-breaking-migration-guide.md`](./self-host-breaking-migration-guide.md)
- App remains usable without any AI keys:
- import, manual transaction CRUD, analytics, categories, and migration UI continue to function.
- import, manual transaction CRUD, analytics, and categories continue to function.
- Development defaults:
- optional dev/test account seeding (can be disabled via env).

Expand Down
8 changes: 0 additions & 8 deletions justfile
Original file line number Diff line number Diff line change
Expand Up @@ -156,11 +156,3 @@ seed-fixture:
pnpm seed:fixture -- --target services/api/test/fixtures/deterministic-financial-store.json
just _prepare-dev-data
just _prepare-test-data

# Load data from the legacy API into the dev database
seed-legacy-api:
just --dotenv-filename .env.development _seed-legacy-api

[private]
_seed-legacy-api:
pnpm seed:legacy-api -- --base-url http://10.0.0.20:18080 --start 2024-01-01 --end 2026-12-31
1 change: 0 additions & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,6 @@
"migrate:sqlite": "tsx scripts/migrate-json-to-sqlite.ts",
"validate:sqlite": "tsx scripts/validate-json-vs-sqlite.ts",
"seed:fixture": "tsx scripts/seed-deterministic-fixture.ts",
"seed:legacy-api": "tsx scripts/load-legacy-api.ts",
"docs:api": "tsx scripts/generate-api-docs.ts",
"guardrails": "tsx scripts/run-guardrails.ts",
"test:test-first": "node --test scripts/check-root-script-binaries.test.mjs && env NODE_ENV=test tsx --test scripts/check-frontend-test-first.test.ts && env NODE_ENV=test tsx --test scripts/run-with-open-ports.test.ts && env NODE_ENV=test tsx scripts/check-frontend-test-first.ts",
Expand Down
4 changes: 1 addition & 3 deletions packages/domain/src/constants.ts
Original file line number Diff line number Diff line change
Expand Up @@ -8,9 +8,7 @@ export const IMPORT_STATUSES = [
"failed"
];

export const SOURCE_TYPES = ["imported", "manual", "migrated"];

export const DIRECTIONS = ["debit", "credit"];
export const SOURCE_TYPES = ["imported", "manual"];

export const AI_PROVIDERS = {
openai: {
Expand Down
107 changes: 0 additions & 107 deletions scripts/load-legacy-api.ts

This file was deleted.

56 changes: 19 additions & 37 deletions scripts/sqlite-cutover-lib.ts
Original file line number Diff line number Diff line change
Expand Up @@ -64,32 +64,30 @@ export const STORE_TABLE_SPECS = [
sampleKey: { source: "id", table: "id" },
mapRow: (row) => ({
id: row.id ?? null,
user_id: row.user_id ?? row.userId ?? null,
account_id: row.account_id ?? row.accountId ?? null,
account_key: row.account_key ?? row.accountKey ?? null,
source_type: row.source_type ?? row.sourceType ?? null,
source_file_id: row.source_file_id ?? row.sourceFileId ?? null,
transaction_date: row.transaction_date ?? row.transactionDate ?? null,
post_date: row.post_date ?? row.postDate ?? null,
merchant_raw: row.merchant_raw ?? row.merchantRaw ?? null,
merchant_normalized: row.merchant_normalized ?? row.merchantNormalized ?? null,
user_id: row.user_id ?? null,
account_id: row.account_id ?? null,
account_key: row.account_key ?? null,
source_type: row.source_type ?? null,
source_file_id: row.source_file_id ?? null,
transaction_date: row.transaction_date ?? null,
post_date: row.post_date ?? null,
merchant_raw: row.merchant_raw ?? null,
merchant_normalized: row.merchant_normalized ?? null,
description: row.description ?? null,
amount: toFiniteNumber(row.amount),
currency: row.currency ?? null,
direction: row.direction ?? null,
category_raw: row.category_raw ?? row.categoryRaw ?? null,
category_final: row.category_final ?? row.categoryFinal ?? null,
category_coarse: row.category_coarse ?? row.categoryCoarse ?? null,
category_emoji: row.category_emoji ?? row.categoryEmoji ?? null,
category_confidence: toFiniteNumber(row.category_confidence ?? row.categoryConfidence),
category_strategy: row.category_strategy ?? row.categoryStrategy ?? null,
needs_category_review: toNullableBooleanInt(
row.needs_category_review ?? row.needsCategoryReview
),
category_raw: row.category_raw ?? null,
category_final: row.category_final ?? null,
category_coarse: row.category_coarse ?? null,
category_emoji: row.category_emoji ?? null,
category_confidence: toFiniteNumber(row.category_confidence),
category_strategy: row.category_strategy ?? null,
needs_category_review: toNullableBooleanInt(row.needs_category_review),
memo: row.memo ?? null,
dedupe_fingerprint: row.dedupe_fingerprint ?? row.dedupeFingerprint ?? null,
created_at: row.created_at ?? row.createdAt ?? null,
updated_at: row.updated_at ?? row.updatedAt ?? null,
dedupe_fingerprint: row.dedupe_fingerprint ?? null,
created_at: row.created_at ?? null,
updated_at: row.updated_at ?? null,
payload_json: JSON.stringify(row ?? {})
})
},
Expand Down Expand Up @@ -346,22 +344,6 @@ export const STORE_TABLE_SPECS = [
payload_json: JSON.stringify(row ?? {})
})
},
{
storeKey: "migrationRuns",
tableName: "migration_runs",
keyColumns: ["id"],
sampleKey: { source: "id", table: "id" },
mapRow: (row) => ({
id: row.id ?? null,
user_id: row.userId ?? row.user_id ?? null,
status: row.status ?? null,
sqlite_path: row.sqlitePath ?? row.sqlite_path ?? null,
created_at: row.createdAt ?? row.created_at ?? null,
updated_at: row.updatedAt ?? row.updated_at ?? null,
report_json: toJsonOrNull(row.report),
payload_json: JSON.stringify(row ?? {})
})
},
{
storeKey: "auditEvents",
tableName: "audit_events",
Expand Down
Binary file modified services/api/data/development-minance.sqlite
Binary file not shown.
Binary file modified services/api/data/test-minance.sqlite
Binary file not shown.
11 changes: 0 additions & 11 deletions services/api/sql/schema.sql
Original file line number Diff line number Diff line change
Expand Up @@ -247,17 +247,6 @@ CREATE TABLE IF NOT EXISTS investment_snapshots (
payload_json TEXT NOT NULL
);

CREATE TABLE IF NOT EXISTS migration_runs (
id TEXT PRIMARY KEY,
user_id TEXT NOT NULL,
status TEXT,
sqlite_path TEXT,
created_at TEXT,
updated_at TEXT,
report_json TEXT,
payload_json TEXT NOT NULL
);

CREATE TABLE IF NOT EXISTS audit_events (
id TEXT PRIMARY KEY,
user_id TEXT NOT NULL,
Expand Down
14 changes: 2 additions & 12 deletions services/api/src/analytics.ts
Original file line number Diff line number Diff line change
Expand Up @@ -55,19 +55,9 @@ function resolveTxnCategory(resolveCategory, txn) {

function normalizeTransactionForAnalytics(txn) {
const rawAmount = Number(txn?.amount ?? 0);
const amount = Number.isFinite(rawAmount) ? Math.abs(rawAmount) : 0;
const amount = Number.isFinite(rawAmount) ? rawAmount : 0;
const rawDirection = String(txn?.direction || "").trim().toLowerCase();

let direction = "outflow";
if (rawDirection === "inflow" || rawDirection === "outflow") {
direction = rawDirection;
} else if (rawDirection === "credit") {
direction = "inflow";
} else if (rawDirection === "debit") {
direction = "outflow";
} else if (rawAmount > 0) {
direction = "inflow";
}
const direction = rawDirection === "inflow" ? "inflow" : "outflow";

return {
...txn,
Expand Down
1 change: 0 additions & 1 deletion services/api/src/auth.ts
Original file line number Diff line number Diff line change
Expand Up @@ -275,7 +275,6 @@ export function deleteUser(userId) {
store.aiProviderPreferences = store.aiProviderPreferences.filter((entry) => entry.userId !== userId);
store.assistantQueries = store.assistantQueries.filter((entry) => entry.userId !== userId);
store.savedViews = store.savedViews.filter((entry) => entry.userId !== userId);
store.migrationRuns = store.migrationRuns.filter((entry) => entry.userId !== userId);
store.auditEvents = store.auditEvents.filter((entry) => entry.userId !== userId);

saveStore(store);
Expand Down
Loading
Loading