Initial import: LifeTrack v1 (santé, vape, finances)

Tracker de vie auto-hébergé : suivi poids/calories/sport avec planning de
pesées, sevrage tabac (vape) avec modèle de coût DIY et économies, et
finances personnelles avec import de relevés bancaires.

Architecture : FastAPI + SQLAlchemy 2.0 + PostgreSQL 16, React 18 + TS +
Vite + Tailwind + ECharts, déploiement Docker Compose. Modules
auto-découverts des deux côtés (pkgutil / import.meta.glob) et framework
de connecteurs à deux voies (importeurs de fichiers + ingestion JSON)
pour brancher de nouvelles sources sans toucher au noyau.

Validé : 292 tests pytest, tsc + vite build, contrat API/web vérifié
contre le schéma OpenAPI, et déploiement Docker réel sur PostgreSQL 16
(28 tables, SPA servie par nginx, wizard de premier démarrage).

Documentation : README.md, docs/GUIDE.md, CONVENTIONS.md, et les
documents de conception et de recherche dans docs/.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-08-14 10:48:57 +02:00
co-authored by Claude Opus 5
commit 93f0689c1e
273 changed files with 80746 additions and 0 deletions
+47
View File
@@ -0,0 +1,47 @@
# Docker build-context exclusions.
# Both images build from the repository root (docker-compose.yml: `context: .`),
# so without this file the daemon would receive .git, node_modules and the
# Python venv on every build — hundreds of MB over the wire, and
# `COPY apps/web/ ./` in web.Dockerfile would overwrite the node_modules that
# `npm ci` just installed inside the image.
# VCS
.git
.gitignore
.gitattributes
# Python
**/__pycache__
**/*.py[cod]
**/.pytest_cache
**/.mypy_cache
**/.ruff_cache
**/*.egg-info
apps/api/.venv
apps/api/venv
# Node
apps/web/node_modules
apps/web/dist
apps/web/.vite
# Secrets & local data (never bake them into an image)
.env
.env.local
*.local.env
*.sqlite3
pgdata
uploads
# Documentation (not needed at runtime)
docs
README.md
DESIGN.md
CONVENTIONS.md
# IDE / OS
.idea
.vscode
*.swp
Thumbs.db
.DS_Store
+37
View File
@@ -0,0 +1,37 @@
# LifeTrack — copy to .env next to docker-compose.yml, then edit.
# cp .env.example .env
# .env is git-ignored (C7): it is the only place secrets live.
# --- PostgreSQL ---------------------------------------------------------
POSTGRES_USER=lifetrack
# Ends up inside LIFETRACK_DATABASE_URL: keep it URL-safe (letters/digits/-/_),
# a '@', ':' or '/' would break the connection string.
# openssl rand -hex 16
POSTGRES_PASSWORD=change-me
POSTGRES_DB=lifetrack
# --- API (every name is LIFETRACK_ + a field of app/core/config.py) ------
# MUST be overridden: it signs the JWTs. openssl rand -hex 32
LIFETRACK_JWT_SECRET=generate-a-long-random-string
# Default timezone of the daily aggregations (?tz= overrides it per request).
LIFETRACK_TIMEZONE=Europe/Paris
# Max size of an uploaded import file, in bytes (20 MiB). Keep it <= the
# client_max_body_size of docker/nginx.conf (25m).
# LIFETRACK_MAX_UPLOAD_BYTES=20971520
# Dev only (Vite dev server without the nginx proxy). JSON array, not a bare
# list: pydantic-settings parses complex fields as JSON.
# LIFETRACK_CORS_ORIGINS=["http://localhost:5173"]
# Dev/tests only: comma-separated whitelist of backend modules (empty = all).
# Read straight from the environment by app/core/module_loader.py.
# LIFETRACK_MODULES=auth,imports,health
# --- Published ports ----------------------------------------------------
# The SPA (nginx). This is the URL you open in a browser.
WEB_PORT=80
# The API, exposed for /api/docs and the device ingestion endpoints.
API_PORT=8000
# --- Build ---------------------------------------------------------------
# Empty by default. On a host with less than ~2 GB of RAM, cap the V8 heap so
# `vite build` cannot be OOM-killed:
# NODE_OPTIONS=--max-old-space-size=1024
+31
View File
@@ -0,0 +1,31 @@
# Python
__pycache__/
*.py[cod]
.venv/
venv/
.pytest_cache/
.mypy_cache/
.ruff_cache/
*.egg-info/
# Node
node_modules/
dist/
.vite/
# Env & secrets
.env
.env.local
*.local.env
# Data
*.sqlite3
pgdata/
uploads/
# IDE / OS
.idea/
.vscode/
*.swp
Thumbs.db
.DS_Store
+133
View File
@@ -0,0 +1,133 @@
# LifeTrack — CONVENTIONS
> Extrait de docs/design/architecture.md §10 — règles obligatoires pour tout développeur (humain ou agent).
### C1. Langues
- **Code** (identifiants, fichiers, tables, colonnes, routes API, commentaires, messages de
commit) : **anglais**.
- **UI et messages d'erreur destinés à l'utilisateur** (`AppError(message=...)`, labels,
`strings.ts`, docs utilisateur) : **français**, ponctuation française correcte, pas d'anglicismes
gratuits. Formats d'affichage `fr-FR` (virgule décimale, espace insécable des milliers, `€`
après le montant).
### C2. Créer un module backend
1. Créer `apps/api/app/modules/<name>/` avec `__init__.py` (vide) et **obligatoirement**
`router.py` exposant `router = APIRouter(prefix="/<name>", tags=["<name>"])`.
Fichiers standard : `models.py`, `schemas.py`, `service.py` ; optionnels : `importers.py`,
`ingest.py`, `calculations.py`.
2. **Ne jamais modifier** `main.py`, `module_loader.py`, ni le module d'un autre chantier.
L'enregistrement est automatique (pkgutil) : routers montés sous `/api`, `models.py`/
`importers.py`/`ingest.py` importés au démarrage pour `create_all` et les registres.
3. `models.py` : style SQLAlchemy 2.0 typé (`Mapped`/`mapped_column`), héritage
`class X(TimestampMixin, Base)` ; noms de tables `snake_case` **pluriel** ; toute table métier
porte `user_id = mapped_column(ForeignKey("users.id"), index=True)` ; datetimes en
`DateTime(timezone=True)` et valeurs **UTC** ; colonnes « jour local » en `Date` ; poids kg,
longueurs cm ou m (distances), volumes ml, énergie kcal, durées secondes, argent en
**centimes** (`int`) ou `Numeric` pour les prix unitaires.
4. Table alimentée par import/ingestion → ajouter `SourceMixin` et les deux contraintes uniques :
`(user_id, source, external_id)` et `(user_id, content_hash)` (noms `uq_<table>_external`,
`uq_<table>_hash`). Agrégats journaliers → clé `(user_id, day, source)` + upsert
`ON CONFLICT DO UPDATE`.
5. `schemas.py` : Pydantic v2 ; suffixes `XxxCreate`, `XxxUpdate` (champs optionnels),
`XxxRead` (avec `model_config = ConfigDict(from_attributes=True)`) ; jamais de modèle ORM
retourné sans schéma `Read`.
6. `service.py` : logique métier ; signature `def fn(db: Session, user_id: int, ...)`. Les
routeurs restent minces (dépendances + appel service + schéma de réponse). Le service lève
les sous-classes d'`AppError` (`NotFoundError`, `ConflictError`, `DomainValidationError`…)
avec message **français** ; **interdit** de lever `HTTPException` dans un module.
7. Sécurité : tout endpoint (hors `auth` public et `healthz`) dépend de `get_current_user` et
filtre **systématiquement** par `user.id`. Endpoints d'ingestion : dépendance
`get_ingest_identity("ingest:<domain>")`.
8. Pagination : toute liste utilise `PageParams`/`Page[T]`/`paginate` de `core.pagination`
pas de pagination maison. Tri via `?sort=` (préfixe `-` = desc), champs autorisés explicites.
9. Requêtes : `select()` SQLAlchemy 2.0 uniquement (pas de `session.query`). Le routeur/service
ne fait pas de `commit` partiel par ligne ; un `commit` par opération logique (le `get_db`
fournit la session, le service commit).
### C3. Ajouter un importeur de fichier
1. Dans `importers.py` du module métier concerné, sous-classer `BaseImporter`
(`app.core.importing.base`) et décorer avec `@register_importer`.
2. Renseigner `id` (snake_case unique, suffixe format : `_csv`, `_ofx`), `label` (français,
affiché dans l'UI), `domain`, `accepted_extensions`.
3. `sniff()` ne lève jamais et reste bon marché (extension + en-têtes dans les 4096 premiers
octets). `parse()` gère les encodages `utf-8-sig` puis `cp1252` et les CSV `;` à décimale
virgule ; il produit des `NormalizedRecord` en **unités canoniques** avec `external_id` si la
source en fournit un, sinon `dedupe_fields` pertinents (ex : `("posted_at", "amount_cents",
"label_raw")`).
4. `upsert()` retourne `INSERTED` / `UPDATED` / `DUPLICATE` — la détection de doublon se fait par
lookup sur les contraintes du C2.4 (pas par try/except d'IntegrityError en boucle).
5. Ne pas créer d'endpoint d'upload : `POST /api/imports` du module `imports` sert toutes les
sources.
### C4. Ajouter un handler d'ingestion JSON
Dans `ingest.py` du module : sous-classer `BaseIngestHandler`, décorer
`@register_ingest_handler`, déclarer `domain` et `record_types`, implémenter
`apply()` (mêmes règles de dédup que C3.4). Le endpoint `POST /api/ingest/{domain}` existe déjà ;
il exige le scope `ingest:<domain>` (ou `ingest:*`) pour les clés d'appareil.
### C5. Créer un module frontend
1. Créer `apps/web/src/modules/<name>/` avec `index.ts` dont l'**export default** est un
`ModuleManifest` (`src/types/module.ts`) : `id` = nom du dossier (anglais), `title` français,
`order` réservé (home 0, health 10, vape 20, finance 30, imports 80, settings 90 ; nouveaux
modules : dizaine libre), `routes` (chemins **absolus**, slugs **français** :
`/sante/...`, `/vape/...`, `/finances/...`), `nav` (labels français + icône `lucide-react`).
2. **Ne jamais modifier** `router.tsx`, `modules.ts`, `Sidebar.tsx` : la découverte
`import.meta.glob` est automatique. Ajouter une entrée de nav = ajouter un élément au tableau
`nav` du manifeste de son module.
3. Fichiers standard du module : `api.ts` (hooks React Query + types TS des schémas API),
`strings.ts` (chaînes françaises du module, export d'un objet constant — pas de français en
dur éparpillé dans le JSX pour les libellés réutilisés), `pages/`, `components/`.
4. Données : **toujours** via les hooks React Query de `api.ts` du module, qui appellent le
wrapper `api()` de `src/lib/api.ts` (jamais `fetch` direct). Clés de requête
`[moduleId, resource, params]` ; toute mutation invalide `[moduleId, resource]`.
5. Graphiques : exclusivement via `<EChart option={...}/>` (`src/components/charts/EChart.tsx`)
et le thème `lifetrack-dark` ; pas d'accès direct à `echarts.init` dans les pages ; formats
d'axes/tooltips via `src/lib/format.ts`.
6. UI : composants partagés de `src/components/ui/` d'abord ; classes Tailwind (palette sombre du
§7.5) ; pas de CSS externe, pas de CDN, pas de nouvelle dépendance sans l'ajouter à
`package.json` du repo.
7. Pages : nom `XxxPage.tsx`, chargées via `React.lazy` dans le manifeste ; états
vide/chargement/erreur systématiques (`EmptyState`, `Spinner`, message d'`ApiError.message`).
### C6. API — rappels contractuels
- Préfixe `/api/<module>` ; ressources au pluriel anglais ; `GET` liste paginée
(`Page[T]`), `POST` création (`201`), `PATCH` partiel, `PUT` singleton, `DELETE``204`.
- Forme d'erreur unique `{"error": {"code", "message", "details"}}``code` snake_case stable,
`message` en français.
- Datetimes : UTC ISO 8601 (`Z`) ; jours locaux : `YYYY-MM-DD` ; agrégations journalières :
paramètre `?tz=` (défaut `Europe/Paris`).
- Filtres temporels : `?from=`/`?to=` inclusifs.
### C7. Qualité
- Python : `ruff` (lint + format), type hints partout, pas d'import inutilisé ; tests `pytest`
dans `apps/api/app/tests/` (au minimum : contrat du routeur du module + dédup des importeurs).
- TypeScript : `strict: true`, pas de `any` non justifié ; build `npm run build` sans erreur.
- Aucune dépendance réseau à l'exécution côté web (fonts/icônes/librairies embarquées).
- Secrets uniquement via variables d'environnement ; rien de sensible commité (`.env` est
git-ignoré, `.env.example` documente).
### C8. Portabilité des types (tests SQLite)
La suite pytest s'exécute sur **SQLite en mémoire** (PostgreSQL en production) : tous les
modèles doivent donc utiliser des types de colonnes **portables**.
1. Énumérations : `sa.Enum(..., native_enum=False)` — jamais d'`ENUM` natif PostgreSQL.
2. UUID : `sa.Uuid` (type générique SQLAlchemy), pas `postgresql.UUID`.
3. JSON : `sa.JSON().with_variant(postgresql.JSONB, "postgresql")` — utiliser l'alias prêt à
l'emploi **`JSONB_V`** exporté par `app/core/mixins.py` (JSON simple sous SQLite, JSONB sous
PostgreSQL).
4. Index uniques partiels : déclarer **les deux** clauses `sqlite_where` **et**
`postgresql_where` (mêmes expressions) ; **jamais** `postgresql_nulls_not_distinct`.
5. Rollback centralisé des imports : toute table portant une colonne référençant
`import_runs.id` doit déclarer la clé étrangère avec **`ondelete="CASCADE"`**
(`ForeignKey("import_runs.id", ondelete="CASCADE")`) — c'est ce qui permet à
`DELETE /api/imports/{id}` de supprimer les lignes importées d'un lot. (Sous SQLite, le
moteur active `PRAGMA foreign_keys=ON` automatiquement — voir `app/core/database.py`.)
+59
View File
@@ -0,0 +1,59 @@
# LifeTrack — Synthèse du design (v1)
> Tracker de vie auto-hébergé : sport/calories/poids, arrêt de la cigarette (vape), et finances
> personnelles — interface web riche en graphiques, déployée via Docker Compose.
## Documents de référence
| Document | Contenu |
|---|---|
| [docs/design/architecture.md](docs/design/architecture.md) | Architecture technique complète : monorepo, FastAPI, React, auto-découverte des modules, framework de connecteurs, Docker. **§10 = CONVENTIONS (extrait dans [CONVENTIONS.md](CONVENTIONS.md))** |
| [docs/design/datamodel-health-vape.md](docs/design/datamodel-health-vape.md) | Tables + formules santé/nutrition/vape : BMR/TDEE, tendance EMA, projections, coût/ml, économies |
| [docs/design/datamodel-finance.md](docs/design/datamodel-finance.md) | Tables + logique finances : pipeline d'import, dédup, moteur de règles, virements, récurrents, budgets |
| [docs/design/ux-pages.md](docs/design/ux-pages.md) | Spec UX française page par page : ~30 graphiques ECharts, KPI, modales, palette validée daltonisme |
| [docs/research/health-connect.md](docs/research/health-connect.md) | Comment sortir les données de Health Connect (bridge webhook, Health Sync CSV, app compagnon v2) |
| [docs/research/nutrition-sources.md](docs/research/nutrition-sources.md) | Foodvisor (export RGPD), Open Food Facts, CIQUAL, importeurs CSV nutrition |
| [docs/research/finance-sources.md](docs/research/finance-sources.md) | Formats CSV réels de 10 banques FR + PayPal, OFX, Enable Banking (v2) |
## Stack (décidée)
- **Backend** : Python 3.12 (Docker) · FastAPI · SQLAlchemy 2.0 typé · PostgreSQL 16 · PyJWT + argon2
- **Frontend** : React 18 · TypeScript strict · Vite · TailwindCSS (thème sombre) · Apache ECharts · TanStack Query
- **Déploiement** : Docker Compose (postgres + api + web/nginx) ; dev = uvicorn --reload + vite
- **Langues** : code en anglais, UI et messages d'erreur en français (fr-FR : `1 234,56 €`, `dd/MM/yyyy`)
## Décisions structurantes
1. **Modules auto-découverts** des deux côtés (pkgutil côté API, `import.meta.glob` côté web) :
ajouter un module = créer un dossier, personne ne modifie `main.py`/`router.tsx` → chantiers
parallèles sans conflit, extensibilité maximale (« l'appli peut aller dans plein de sens »).
2. **Framework de connecteurs** à deux voies : importeurs de **fichiers** (`BaseImporter`,
`POST /api/imports`, suivi `ImportRun`, dédup idempotente) et handlers d'**ingestion JSON**
(`BaseIngestHandler`, `POST /api/ingest/{domain}`, clés d'appareil à scopes `ingest:<domain>`).
3. **Dédup normalisée** : `(user_id, source, external_id)` + `(user_id, content_hash)` pour les
événements ; `(user_id, day, source)` + upsert pour les agrégats journaliers → ré-imports sûrs.
4. **Health Connect = push, jamais pull** (données on-device, pas d'API cloud ; Google Fit API
morte). v1 : endpoint d'ingestion + app bridge `health-connect-webhook` + import CSV Health
Sync ; v2 : app compagnon Android (Kotlin, 3-5 j) ; v2/v3 : tapis via Web Bluetooth FTMS.
5. **Nutrition v1** : saisie rapide + recherche Open Food Facts (proxy backend + cache) + table
CIQUAL + import CSV Foodvisor (export RGPD) ; connecteur Foodvisor natif impossible (pas d'API).
6. **Finances v1 = fichiers** : mapper CSV générique + presets par banque (10 banques FR),
OFX, PayPal ; aperçu avant import + rollback par lot. **v2 = Enable Banking** (PSD2 gratuit
pour ses propres comptes — GoCardless/Nordigen ferme, ne pas construire dessus).
7. **Vape** : modèle de coût DIY historisé (base, boosters, arômes, résistances) → coût/ml,
coût/jour avec amortissement résistance, économies cumulées vs baseline cigarettes gelée.
8. **Temps** : stockage UTC `timestamptz`, jours locaux `Date` en Europe/Paris (`?tz=` sur les
agrégations). **Unités canoniques** : kg, cm/m, ml, kcal, secondes, centimes d'euro.
9. **v1 sans Alembic** : `metadata.create_all` au démarrage (migrations ajoutées quand le schéma
bougera). Tests pytest sur SQLite → types portables (voir CONVENTIONS C8).
10. **Graphiques** : palette 8 couleurs validée daltonisme sur fond sombre, jamais de double axe Y,
sémantique directionnelle (déficit/économies = vert), réponses stats déjà au format ECharts.
## Périmètre v1 (ce dépôt) / v2 / v3
- **v1** : tout le web + API ci-dessus, imports fichiers, ingestion JSON Health Connect (bridge),
dashboards complets, wizard premier démarrage.
- **v2** : app compagnon Android Health Connect, connecteur Enable Banking, calculateur DIY avancé,
Alembic, notifications.
- **v3** : enregistrement tapis Web Bluetooth FTMS, analyse photo repas (si API Foodvisor s'ouvre),
multi-utilisateurs complet, thème clair.
+565
View File
@@ -0,0 +1,565 @@
# LifeTrack
> Tracker de vie **auto-hébergé** : poids et calories, sevrage tabagique (vape) et finances
> personnelles — le tout dans une interface web sombre et riche en graphiques.
Vos données restent chez vous, dans votre PostgreSQL. Aucun compte tiers, aucune télémétrie,
aucune ressource chargée depuis Internet au moment de l'affichage.
Une seule dépendance réseau existe côté serveur, et elle est facultative : la recherche
d'aliments interroge **Open Food Facts** (base publique, sans clé d'API) et met chaque
produit consulté en cache local. Tout le reste fonctionne hors ligne.
---
## Sommaire
- [À quoi sert LifeTrack](#à-quoi-sert-lifetrack)
- [Fonctionnalités par module](#fonctionnalités-par-module)
- [Captures d'écran](#captures-décran)
- [Démarrage rapide (Docker Compose)](#démarrage-rapide-docker-compose)
- [Premier démarrage](#premier-démarrage)
- [Mode développement](#mode-développement)
- [Architecture](#architecture)
- [Stack technique](#stack-technique)
- [Feuille de route et écarts connus](#feuille-de-route-et-écarts-connus)
- [Documentation](#documentation)
- [Licence](#licence)
---
## À quoi sert LifeTrack
Trois besoins réunis dans une seule application, parce qu'ils s'alimentent mutuellement :
1. **Perdre du poids sans se tromper de chiffres.** LifeTrack calcule votre métabolisme de
base, votre dépense réelle et votre budget calorique quotidien, lisse vos pesées en une
tendance, et projette la date d'atteinte de votre objectif.
2. **Tenir l'arrêt de la cigarette.** Le module vape chiffre au centime près ce que vous
dépensez en e-liquide et en résistances, et le compare à ce que le tabac vous coûterait
aujourd'hui.
3. **Savoir où part l'argent.** Les relevés bancaires s'importent en quelques clics,
se catégorisent tout seuls et alimentent budgets, récurrents et graphiques.
L'application est **mono-utilisateur en pratique** : le premier compte créé est le vôtre,
il n'y a pas d'inscription publique.
---
## Fonctionnalités par module
L'interface est découpée en modules auto-découverts. Voici ce qui existe réellement
aujourd'hui.
### Tableau de bord (`/`)
- Carte « Aujourd'hui » : ce qui est planifié le jour même (pesée, séance, journal
alimentaire), fait ou non, avec les séries en cours.
- Ajout rapide d'une pesée sans quitter la page.
- Widgets KPI et graphiques de synthèse tirés des modules installés.
### Santé (`/sante/...`)
| Page | Contenu |
|---|---|
| **Poids & Objectif** (`/sante/poids`) | Historique des pesées, tendance lissée (EMA), projection de la date d'atteinte, mensurations, calendrier d'assiduité |
| **Nutrition** (`/sante/nutrition`) | Journal alimentaire par repas, recherche d'aliments, favoris, aliments récents, suivi de l'hydratation |
| **Activité & Sport** (`/sante/activite`) | Activité quotidienne (pas, distance, calories, minutes actives, étages), séances de sport |
| **Balance énergétique** (`/sante/balance`) | Calories ingérées contre calories dépensées, déficit cumulé, calibration adaptative du TDEE |
Détails utiles :
- **Profil corporel** (taille, sexe, date de naissance, niveau d'activité) → BMR par la
formule **Mifflin-St Jeor**, puis TDEE.
- **Objectif de poids** en trois modes : rythme hebdomadaire, date cible, ou maintien.
Budget calorique = TDEE lissé sur 7 jours (rythme × 7 700 kcal 7), avec un plancher
de sécurité.
- **Planning de suivi** : vous choisissez les jours de la semaine où vous prévoyez de vous
peser (`weigh_in`), de faire une séance (`workout`) et de tenir votre journal alimentaire
(`food_log`). LifeTrack en déduit assiduité, jours manqués et séries — sans case à cocher :
l'habitude est considérée faite dès qu'une donnée existe pour ce jour.
- **Recherche d'aliments** : cache local d'abord, puis **Open Food Facts** (recherche
plein texte et code-barres) via un proxy côté serveur. Chaque produit consulté est mis en
cache et n'est plus jamais redemandé.
- **Composition corporelle** : IMC, et estimation de masse grasse par la méthode US Navy
quand les mensurations nécessaires sont saisies.
### Vape / sevrage (`/vape`)
| Onglet | Contenu |
|---|---|
| **Consommation** | Recharges quotidiennes en ml, nicotine consommée, tendance, équivalent cigarettes |
| **Coûts & modèle** | Catalogue de produits (base, boosters, arômes, résistances), recettes DIY, coût de revient au ml, achats réels |
| **Résistances** | Changement de résistance en un clic, durée de vie moyenne, volume passé par résistance |
| **Économies** | Économies cumulées face à la référence tabac, cigarettes évitées, temps de vie récupéré, jalons santé |
- **Référence tabac gelée** : date d'arrêt, cigarettes par jour avant l'arrêt, cigarettes par
paquet, prix du paquet. C'est l'ancre de tout le calcul d'économies.
- **Modèle de coût DIY** : chaque recette additionne `quantité × (prix du flacon contenance)`
pour donner un coût au ml, auquel s'ajoute l'amortissement de la résistance.
- **Assistant de recette** : à partir d'un volume total, d'un taux de nicotine visé et d'un
booster, LifeTrack calcule les millilitres de booster, d'arôme et de base.
- **Jalons santé** : 12 étapes chronométrées depuis la date d'arrêt (20 minutes, 8 heures,
24 heures, 48 heures, 72 heures, 2 semaines, 3 mois, 9 mois, 1 an, 5 ans, 10 ans, 15 ans).
### Finances (`/finances`)
| Page | Contenu |
|---|---|
| **Aperçu** (`/finances`) | Flux de trésorerie, dépenses par catégorie, diagramme Sankey, principaux marchands |
| **Transactions** (`/finances/transactions`) | Liste filtrable, catégorisation unitaire ou en masse, saisie manuelle |
| **Budgets** (`/finances/budgets`) | Budget mensuel par catégorie, avancement, projection de fin de mois |
| **Récurrents** (`/finances/recurrents`) | Abonnements et prélèvements détectés automatiquement, prochaine échéance estimée |
| **Comptes** (`/finances/comptes`) | Comptes courants, épargne, PayPal, espèces |
| **Catégories** (`/finances/categories`) | Arborescence de catégories (un jeu français complet est créé automatiquement) |
- **Import de relevés** avec choix du compte, choix du profil bancaire, **aperçu avant
écriture** et **annulation d'un lot** en un clic.
- **Presets de banques intégrés** : CSV générique, BoursoBank / Boursorama, Crédit Agricole,
BNP Paribas, Société Générale, La Banque Postale, Caisse d'Épargne, Fortuneo, Revolut, N26,
OFX (toutes banques) et PayPal.
- **Déduplication** systématique : ré-importer le même fichier n'ajoute rien deux fois.
- **Moteur de règles** de catégorisation (libellé contenant, expression régulière, sens,
fourchette de montant, compte), avec priorité, aperçu et application en masse. Une
catégorisation manuelle n'est jamais écrasée.
- **Virements internes** : détection automatique des paires de transactions qui s'annulent
entre deux de vos comptes, pour ne pas les compter comme dépense.
### Imports et connecteurs (`/imports`)
- Assistant en trois étapes : **Fichier → Vérification → Import**, avec détection
automatique du format à partir de l'en-tête du fichier.
- Éditeur de mappage de colonnes (séparateur, encodage, format de date, colonnes) enregistré
dans un profil personnel.
- **Formats reconnus** : relevé bancaire CSV, relevé bancaire OFX, PayPal CSV,
Foodvisor CSV, Health Sync / Health Connect CSV, historique de poids CSV (date + poids).
- **Historique des imports** avec statistiques par lot et **annulation (rollback)** :
supprimer un lot supprime les lignes qu'il avait créées.
- Taille maximale d'un fichier : **20 Mio**.
- Carte « Connecteurs & API » : URL d'ingestion à coller dans une passerelle Android, lien
vers les clés d'appareil, et procédure d'export Foodvisor.
### Réglages (`/reglages`)
Cinq onglets : **Profil**, **Objectif**, **Vape**, **Appareils & API** (création et
révocation de clés d'API par appareil), **Application** (préférences locales au navigateur).
---
## Captures d'écran
**Il n'y en a pas, et ce n'est pas un oubli.**
L'application a été développée et testée intégralement hors ligne : la suite de tests
backend et la compilation du frontend passent, mais **la pile Docker Compose n'a jamais été
démarrée** (le démon Docker était indisponible pendant tout le développement). Aucune
capture n'a donc pu être prise sur une instance réelle.
Plutôt que de publier des images de synthèse trompeuses, cette section restera vide jusqu'à
votre premier démarrage. Une fois l'application lancée, les pages à capturer en priorité
sont : le tableau de bord, `/sante/poids`, `/vape/economies` et `/finances`.
---
## Démarrage rapide (Docker Compose)
### Prérequis
- **Docker Desktop installé ET démarré.** Vérifiez que la baleine est bien active dans la
barre des tâches ; `docker compose version` doit répondre sans erreur.
- Environ 2 Go d'espace disque pour les images et la base.
- Aucun autre service n'écoute sur le port choisi (80 par défaut).
> ⚠️ **À lire avant le premier lancement.** Le démon Docker était **hors service pendant
> toute la phase de développement** : la pile Compose n'a donc jamais été construite ni
> démarrée. Le code applicatif, lui, est validé (292 tests backend, compilation frontend).
> Attendez-vous éventuellement à un petit ajustement lors du tout premier `up` — le plus
> probable étant un temps de build long, ou une image de base à retélécharger. Si un
> conteneur refuse de démarrer, consultez ses journaux avec
> `docker compose logs api` avant toute autre chose.
### 1. Créer le fichier `.env`
Depuis la racine du dépôt :
```bash
cp .env.example .env
```
Sous Windows (PowerShell) :
```powershell
Copy-Item .env.example .env
```
### 2. Modifier les deux variables obligatoires
Ouvrez `.env` et changez **impérativement** ces deux valeurs :
| Variable | Valeur d'exemple à remplacer | Pourquoi |
|---|---|---|
| `LIFETRACK_JWT_SECRET` | `generate-a-long-random-string` | Signe vos jetons de session. Laissée telle quelle, n'importe qui pourrait forger une session valide. |
| `POSTGRES_PASSWORD` | `change-me` | Mot de passe de la base PostgreSQL. |
Pour générer un secret solide :
```bash
openssl rand -hex 32
```
Sans `openssl` (PowerShell) :
```powershell
-join ((1..64) | ForEach-Object { '0123456789abcdef'[(Get-Random -Maximum 16)] })
```
Les autres variables ont des valeurs par défaut utilisables telles quelles :
| Variable | Défaut | Rôle |
|---|---|---|
| `POSTGRES_USER` | `lifetrack` | Utilisateur PostgreSQL |
| `POSTGRES_DB` | `lifetrack` | Nom de la base |
| `LIFETRACK_TIMEZONE` | `Europe/Paris` | Fuseau des agrégations journalières |
| `WEB_PORT` | `80` | Port d'écoute de l'interface web |
Deux variables restent commentées et ne servent qu'au développement :
`LIFETRACK_CORS_ORIGINS` (origines autorisées quand Vite tourne sans proxy) et
`LIFETRACK_MODULES` (liste blanche de modules backend à charger ; vide = tous).
> `LIFETRACK_DATABASE_URL` n'est **pas** à renseigner dans `.env` : `docker-compose.yml` la
> compose automatiquement à partir de `POSTGRES_USER`, `POSTGRES_PASSWORD` et `POSTGRES_DB`.
### 3. Lancer la pile
```bash
docker compose up -d --build
```
Trois conteneurs démarrent : `postgres` (base de données), `api` (FastAPI) et `web`
(nginx servant le frontend compilé et relayant `/api/` vers l'API). Le premier build prend
plusieurs minutes.
Suivre le démarrage :
```bash
docker compose logs -f
```
### 4. Ouvrir l'application
<http://localhost>
Si vous avez changé `WEB_PORT`, adaptez l'adresse (par exemple `http://localhost:8080`).
Depuis un autre appareil du réseau local, utilisez l'adresse IP du serveur.
Deux adresses utiles :
- <http://localhost/api/healthz> — doit répondre `{"status":"ok"}`
- <http://localhost/api/docs> — documentation interactive de l'API
### Arrêter, mettre à jour, repartir de zéro
```bash
docker compose down # arrêt, les données sont conservées
docker compose up -d --build # reconstruction après mise à jour du code
docker compose down -v # ⚠️ supprime AUSSI le volume : toutes vos données
```
---
## Premier démarrage
Au premier accès, LifeTrack détecte qu'aucun compte n'existe et vous redirige vers l'écran
de création.
**Étape 1 — Compte administrateur.** Adresse e-mail, prénom ou pseudonyme (facultatif) et
mot de passe de **8 caractères minimum**. C'est le seul compte de l'instance : il n'y a pas
de page d'inscription publique. Vous êtes connecté immédiatement après validation et arrivez
sur le tableau de bord.
**Étape 2 — Profil corporel.** Rendez-vous dans **Réglages → Profil** et renseignez taille,
sexe, date de naissance et niveau d'activité. Sans ces informations, LifeTrack ne peut pas
calculer votre métabolisme de base, donc ni votre dépense estimée ni votre budget calorique.
**Étape 3 — Objectif.** Dans **Réglages → Objectif**, indiquez poids de départ, poids cible
et mode de calcul (rythme hebdomadaire, date cible ou maintien). Le budget calorique
quotidien et la date d'atteinte estimée s'affichent aussitôt. C'est aussi ici que vous
accédez à votre **planning de pesées et de séances**.
**Étape 4 (facultative) — Vape.** Dans **Réglages → Vape**, saisissez votre date d'arrêt du
tabac et votre consommation d'avant : sans cette référence, le module vape reste en mode
« non configuré » et n'affiche aucune économie.
Ensuite seulement : importez vos premières données depuis **Imports**, ou saisissez-les à la
main. Le [guide d'utilisation](docs/GUIDE.md) détaille chaque parcours.
---
## Mode développement
Commandes réellement exécutées et validées sur ce dépôt (Windows, PowerShell ou Git Bash).
### Base de données seule
Le démon Docker doit tourner. Depuis la racine :
```bash
docker compose -f docker-compose.yml -f docker-compose.dev.yml up postgres
```
L'overlay de développement publie le port `5432` sur l'hôte, ce qui permet de faire tourner
l'API directement sur la machine.
### Backend (FastAPI)
L'environnement virtuel Python est déjà présent dans `apps/api/.venv`.
```bash
cd apps/api
# Suite de tests — 292 tests, exécutés sur SQLite en mémoire
.venv/Scripts/python.exe -m pytest
# Lint et formatage
.venv/Scripts/ruff.exe check .
.venv/Scripts/ruff.exe format .
# Serveur de développement avec rechargement à chaud
.venv/Scripts/python.exe -m uvicorn app.main:app --reload --port 8000
```
L'API écoute alors sur <http://localhost:8000>, documentation sur
<http://localhost:8000/api/docs>.
Si l'environnement virtuel doit être recréé :
```bash
cd apps/api
python -m venv .venv
.venv/Scripts/python.exe -m pip install -r requirements.txt -r requirements-dev.txt
```
> Les tests tournent sur **SQLite en mémoire** alors que la production utilise
> **PostgreSQL 16** : tous les types de colonnes doivent rester portables (voir
> [CONVENTIONS.md](CONVENTIONS.md) §C8).
### Frontend (React + Vite)
```bash
cd apps/web
npm install # une seule fois
npm run dev # serveur de développement sur http://localhost:5173
npm run build # tsc --noEmit puis build de production
npm run preview # prévisualiser le build
```
**Proxy Vite** : `vite.config.ts` redirige tout `/api` vers `http://localhost:8000`. Vous
n'avez donc rien à configurer côté CORS tant que l'API tourne sur le port 8000 — le frontend
appelle simplement `/api/...` en chemin relatif, exactement comme en production derrière
nginx.
Si vous préférez ne pas utiliser le proxy, décommentez `LIFETRACK_CORS_ORIGINS` dans `.env` :
```
LIFETRACK_CORS_ORIGINS=["http://localhost:5173"]
```
### Tout en conteneurs, avec rechargement à chaud
```bash
docker compose -f docker-compose.yml -f docker-compose.dev.yml up
```
L'API tourne alors en `--reload` sur le port 8000 et Vite sur le port 5173, les sources
étant montées depuis l'hôte.
---
## Architecture
### Organisation du dépôt
```
LifeTrack/
├─ apps/
│ ├─ api/ # Backend FastAPI
│ │ ├─ app/
│ │ │ ├─ core/ # config, base, sécurité, pagination, erreurs,
│ │ │ │ # framework d'import et d'ingestion, chargeur de modules
│ │ │ ├─ modules/ # auth, health, vape, finance, imports
│ │ │ ├─ tests/ # 292 tests pytest
│ │ │ └─ main.py # création de l'app — ne jamais modifier pour un module
│ │ ├─ requirements.txt
│ │ └─ openapi.json # schéma OpenAPI exporté
│ └─ web/ # Frontend React + TypeScript
│ └─ src/
│ ├─ app/ # routeur, layout, authentification — ne pas modifier
│ ├─ components/ # composants partagés (ui/, charts/)
│ ├─ lib/ # wrapper API, formatage fr-FR
│ └─ modules/ # home, health, vape, finance, imports, settings
├─ docker/ # Dockerfiles et configuration nginx
├─ docs/ # documents de conception et de recherche
├─ docker-compose.yml
├─ docker-compose.dev.yml
├─ .env.example
├─ CONVENTIONS.md # règles obligatoires pour toute contribution
└─ DESIGN.md # synthèse des décisions de conception
```
### Modules auto-découverts
C'est la décision structurante du projet : **ajouter un module ne demande de modifier aucun
fichier existant.**
- **Côté backend**, `app/core/module_loader.py` parcourt `app/modules/` avec `pkgutil`. Tout
dossier qui expose un `router.py` contenant `router = APIRouter(prefix="/<nom>")` est monté
automatiquement sous `/api`. Les fichiers `models.py`, `importers.py` et `ingest.py` sont
importés au démarrage pour peupler les métadonnées SQLAlchemy et les registres de
connecteurs.
- **Côté frontend**, `src/app/modules.ts` utilise `import.meta.glob` sur
`src/modules/*/index.ts`. Chaque module exporte par défaut un `ModuleManifest` déclarant son
identifiant, son titre français, son ordre, ses routes et ses entrées de navigation.
Pour créer un module, suivez [CONVENTIONS.md](CONVENTIONS.md) §C2 (backend) et §C5
(frontend). Les règles y sont normatives : nommage, unités canoniques, pagination, gestion
des erreurs, portabilité SQLite.
### Où vivent les données
- **PostgreSQL 16**, dans le volume Docker nommé `pgdata`. Il survit à `docker compose down`
mais **pas** à `docker compose down -v`.
- Le schéma est créé au démarrage par `Base.metadata.create_all` — il n'y a **pas encore de
migrations Alembic** en v1.
- Conventions de stockage : horodatages en **UTC**, jours locaux en `Date`, poids en **kg**,
distances en **m**, volumes en **ml**, énergie en **kcal**, durées en **secondes**, argent
en **centimes d'euro**. L'affichage en `fr-FR` (virgule décimale, `€` après le montant) est
fait côté interface.
- Sauvegarde recommandée :
```bash
docker compose exec postgres pg_dump -U lifetrack lifetrack > sauvegarde.sql
```
### API
- Toutes les routes sont préfixées par **`/api`**, suivi du nom du module :
`/api/auth`, `/api/health`, `/api/vape`, `/api/finance`, `/api/imports`, `/api/ingest`.
- En production, nginx relaie `/api/` vers le conteneur `api` sur le port 8000 ; tout le
reste retombe sur `index.html` (application monopage).
- Format d'erreur unique : `{"error": {"code", "message", "details"}}`, `code` stable en
snake_case, `message` en français.
- Les dates-heures sont en UTC ISO 8601, les jours locaux en `YYYY-MM-DD`, et les agrégations
journalières acceptent un paramètre `?tz=` (défaut `Europe/Paris`).
### Authentification
Deux mécanismes, pour deux usages :
1. **JWT** — pour vous, dans le navigateur. Obtenu via `POST /api/auth/login`, envoyé dans
l'en-tête `Authorization: Bearer <jeton>`. Mots de passe hachés en argon2. Durée de
validité : **7 jours** (choix assumé pour un déploiement domestique).
2. **Clés d'appareil** — pour vos machines. Créées dans **Réglages → Appareils & API**,
préfixées `ltk_`, envoyées dans l'en-tête **`X-API-Key`**, et portant des **portées**
explicites (`ingest:health`, `ingest:*`, …). La clé en clair n'est affichée **qu'une seule
fois** à la création ; seul son empreinte est stockée. Chaque clé est révocable
individuellement.
Tout endpoint hors `POST /api/auth/login`, `POST /api/auth/setup`, `GET /api/auth/status` et
`/api/healthz` exige une authentification et filtre systématiquement les données par
utilisateur.
---
## Stack technique
| Couche | Technologies |
|---|---|
| Backend | Python 3.12 · FastAPI · SQLAlchemy 2.0 typé · Pydantic v2 · PyJWT · argon2 (pwdlib) |
| Base de données | PostgreSQL 16 (pilote psycopg 3) — SQLite en mémoire pour les tests |
| Frontend | React 18 · TypeScript strict · Vite 6 · TailwindCSS · Apache ECharts · TanStack Query · React Router |
| Déploiement | Docker Compose : `postgres` + `api` + `web` (nginx 1.27) |
| Qualité | pytest (292 tests) · ruff · `tsc --noEmit` |
Aucune ressource externe n'est chargée à l'exécution : polices, icônes et bibliothèques sont
embarquées dans le build.
---
## Feuille de route et écarts connus
Ce qui suit est **décrit dans les documents de conception mais absent du code
d'aujourd'hui**. Rien de tout cela n'est nécessaire pour utiliser LifeTrack, mais autant le
savoir avant de le chercher dans l'interface.
### Écarts entre la conception et le code actuel
- **Table CIQUAL absente.** La recherche d'aliments prévoyait une base d'aliments génériques
français (ANSES). Le code ne contient ni script d'import ni données : seule la recherche
Open Food Facts fonctionne. Les plats maison se saisissent donc à la main.
- **Adaptateur webhook sans en-tête d'authentification non implémenté.** La recherche
prévoyait une route acceptant un jeton dans l'URL, pour les passerelles Android incapables
de poser un en-tête HTTP. Elle n'existe pas : `POST /api/ingest/{domain}` exige
`X-API-Key` ou un jeton JWT. Conséquence pratique détaillée dans le
[guide](docs/GUIDE.md#connecter-health-connect).
- **Ingestion limitée au domaine `health`.** L'écran de création de clé propose les portées
`ingest:vape` et `ingest:finance`, mais aucun gestionnaire ne les traite :
`POST /api/ingest/vape` et `POST /api/ingest/finance` répondent « Domaine d'ingestion
inconnu ». Ces portées sont réservées pour plus tard.
- **Pas de table de charges utiles brutes.** Chaque ligne conserve sa charge utile d'origine
dans une colonne `raw`, mais il n'existe pas de table dédiée permettant de rejouer une
normalisation après coup.
- **Importeurs de séances absents** : pas de lecteur TCX, GPX ou FIT. Les séances de sport se
saisissent à la main ou arrivent par l'ingestion JSON.
- **Importeurs MyFitnessPal et Cronometer absents** : seul Foodvisor dispose d'un profil
nutrition dédié.
- **Pas de migrations Alembic** : le schéma est créé au démarrage. Une modification de schéma
sur une base existante devra être gérée manuellement.
### v2 — envisagé
- Application compagnon Android maison (Kotlin, SDK Health Connect), avec vraie
authentification par jeton et rattrapage d'historique au-delà de 48 heures.
- Connecteur bancaire **Enable Banking** (PSD2, gratuit pour ses propres comptes) en
remplacement des imports de fichiers.
- Calculateur DIY avancé, migrations Alembic, notifications.
### v3 — exploratoire
- Enregistrement direct des séances de tapis de course via **Web Bluetooth + FTMS**
(Chrome/Edge uniquement, HTTPS requis).
- Analyse photo des repas, multi-utilisateurs complet, thème clair.
---
## Documentation
| Document | Contenu |
|---|---|
| [docs/GUIDE.md](docs/GUIDE.md) | **Guide d'utilisation au quotidien** : connecter Health Connect, suivre poids et calories, nutrition, vape, finances, dépannage |
| [CONVENTIONS.md](CONVENTIONS.md) | Règles obligatoires pour contribuer au code |
| [DESIGN.md](DESIGN.md) | Synthèse des décisions de conception |
| [docs/design/architecture.md](docs/design/architecture.md) | Architecture technique complète |
| [docs/design/datamodel-health-vape.md](docs/design/datamodel-health-vape.md) | Modèle de données et formules santé, nutrition, vape |
| [docs/design/datamodel-finance.md](docs/design/datamodel-finance.md) | Modèle de données et logique finances |
| [docs/design/ux-pages.md](docs/design/ux-pages.md) | Spécification UX page par page |
| [docs/design/addendum-planning.md](docs/design/addendum-planning.md) | Planning d'habitudes, assiduité, séries |
| [docs/research/health-connect.md](docs/research/health-connect.md) | Comment sortir les données de Health Connect |
| [docs/research/nutrition-sources.md](docs/research/nutrition-sources.md) | Foodvisor, Open Food Facts, CIQUAL |
| [docs/research/finance-sources.md](docs/research/finance-sources.md) | Formats d'export réels des banques françaises |
---
## Licence
**Aucune licence n'est déclarée** : ce dépôt est un projet personnel auto-hébergé, il n'est
pas publié sous licence libre. En l'absence de fichier `LICENSE`, tous droits sont réservés
par défaut.
Notez les obligations attachées aux données tierces que vous consommerez :
- **Open Food Facts** — base sous **ODbL**, contenus sous DbCL. Un usage personnel ne pose
aucune difficulté ; une attribution « Données produits : Open Food Facts (ODbL) » reste la
bonne pratique.
- Les applications passerelles Android citées dans la documentation ont leurs propres
licences (AGPL-3.0 pour `health-connect-webhook`, GPL-3.0 pour HCGateway) — elles ne sont
pas distribuées avec LifeTrack.
View File
View File
+33
View File
@@ -0,0 +1,33 @@
from functools import lru_cache
from pydantic_settings import BaseSettings, SettingsConfigDict
class Settings(BaseSettings):
model_config = SettingsConfigDict(
env_prefix="LIFETRACK_", env_file=".env", extra="ignore"
)
# Database (psycopg v3 driver)
database_url: str = (
"postgresql+psycopg://lifetrack:lifetrack@localhost:5432/lifetrack"
)
# Auth
jwt_secret: str = "change-me-in-env" # MUST be overridden in production
jwt_algorithm: str = "HS256"
access_token_expire_minutes: int = 60 * 24 * 7 # 7 days (home deployment)
# Time
timezone: str = "Europe/Paris" # default tz for daily aggregations
# Imports
max_upload_bytes: int = 20 * 1024 * 1024 # 20 MiB
# Dev only: Vite dev server origin(s), comma-separated env value
cors_origins: list[str] = []
@lru_cache
def get_settings() -> Settings:
return Settings()
+45
View File
@@ -0,0 +1,45 @@
from collections.abc import Iterator
from typing import Any
from sqlalchemy import Engine, create_engine, event
from sqlalchemy.orm import DeclarativeBase, Session, sessionmaker
from sqlalchemy.pool import StaticPool
from app.core.config import get_settings
class Base(DeclarativeBase):
"""Single declarative base for the whole application."""
def _build_engine(url: str) -> Engine:
if url.startswith("sqlite"):
# Test configuration (pytest runs on SQLite in-memory): a StaticPool
# shares the single in-memory database across sessions/threads.
return create_engine(
url,
connect_args={"check_same_thread": False},
poolclass=StaticPool,
)
return create_engine(url, pool_pre_ping=True)
engine = _build_engine(get_settings().database_url)
if engine.dialect.name == "sqlite":
# SQLite does not enforce foreign keys (and thus ON DELETE CASCADE, used by
# the import rollback) unless the pragma is enabled per connection.
@event.listens_for(engine, "connect")
def _enable_sqlite_fks(dbapi_connection: Any, _record: Any) -> None:
cursor = dbapi_connection.cursor()
cursor.execute("PRAGMA foreign_keys=ON")
cursor.close()
SessionLocal = sessionmaker(bind=engine, autoflush=False, expire_on_commit=False)
def get_db() -> Iterator[Session]:
"""FastAPI dependency: one session per request, closed automatically."""
with SessionLocal() as session:
yield session
+81
View File
@@ -0,0 +1,81 @@
from collections.abc import Callable
from typing import TYPE_CHECKING, Annotated
from fastapi import Depends, Header
from fastapi.security import HTTPAuthorizationCredentials, HTTPBearer
from sqlalchemy.orm import Session
from app.core.database import get_db
from app.core.errors import ForbiddenError, UnauthorizedError
from app.core.security import decode_access_token, verify_device_key
if TYPE_CHECKING:
from app.modules.auth.models import User
bearer = HTTPBearer(auto_error=False)
INGEST_WILDCARD_SCOPE = "ingest:*"
def get_current_user(
credentials: Annotated[HTTPAuthorizationCredentials | None, Depends(bearer)],
db: Annotated[Session, Depends(get_db)],
) -> "User":
"""JWT Bearer -> User. Raises UnauthorizedError (401) otherwise."""
from app.modules.auth.models import User
if credentials is None:
raise UnauthorizedError("Authentification requise.")
user_id = decode_access_token(credentials.credentials)
user = db.get(User, user_id)
if user is None or not user.is_active:
raise UnauthorizedError("Session invalide ou expirée.")
return user
def authenticate_ingest(
db: Session,
credentials: HTTPAuthorizationCredentials | None,
api_key: str | None,
required_scope: str,
) -> "User":
"""Shared ingest authentication: JWT Bearer OR X-API-Key device key.
A device key must hold `required_scope` (ex: "ingest:health") or the
wildcard "ingest:*". Raises 401/403 accordingly.
"""
from app.modules.auth.models import User
if api_key:
key = verify_device_key(db, api_key)
scopes = key.scopes or []
if required_scope not in scopes and INGEST_WILDCARD_SCOPE not in scopes:
raise ForbiddenError(
"Cette clé d'appareil ne dispose pas du droit requis.",
details={"required_scope": required_scope},
)
user = db.get(User, key.user_id)
if user is None or not user.is_active:
raise UnauthorizedError("Clé d'appareil invalide.")
return user
if credentials is not None:
return get_current_user(credentials, db)
raise UnauthorizedError("Authentification requise.")
def get_ingest_identity(required_scope: str) -> Callable[..., "User"]:
"""Factory dependency for ingest endpoints.
Accepts EITHER:
- Authorization: Bearer <jwt> (interactive user), OR
- X-API-Key: ltk_... (device key)
"""
def dependency(
credentials: Annotated[HTTPAuthorizationCredentials | None, Depends(bearer)],
db: Annotated[Session, Depends(get_db)],
x_api_key: Annotated[str | None, Header(alias="X-API-Key")] = None,
) -> "User":
return authenticate_ingest(db, credentials, x_api_key, required_scope)
return dependency
+94
View File
@@ -0,0 +1,94 @@
from typing import Any
from fastapi import FastAPI, Request
from fastapi.encoders import jsonable_encoder
from fastapi.exceptions import RequestValidationError
from fastapi.responses import JSONResponse
from starlette.exceptions import HTTPException as StarletteHTTPException
class AppError(Exception):
status_code: int = 400
code: str = "bad_request"
def __init__(self, message: str, *, details: dict[str, Any] | None = None):
self.message = message
self.details = details or {}
super().__init__(message)
class NotFoundError(AppError):
status_code, code = 404, "not_found"
class ConflictError(AppError):
status_code, code = 409, "conflict"
class UnauthorizedError(AppError):
status_code, code = 401, "unauthorized"
class ForbiddenError(AppError):
status_code, code = 403, "forbidden"
class DomainValidationError(AppError):
status_code, code = 422, "validation_error"
class PayloadTooLargeError(AppError):
status_code, code = 413, "payload_too_large"
_HTTP_CODES = {
400: "bad_request",
401: "unauthorized",
403: "forbidden",
404: "not_found",
405: "method_not_allowed",
409: "conflict",
413: "payload_too_large",
422: "validation_error",
500: "internal_error",
}
def _payload(code: str, message: str, details: Any = None) -> dict[str, Any]:
return {"error": {"code": code, "message": message, "details": details or {}}}
def register_error_handlers(app: FastAPI) -> None:
@app.exception_handler(AppError)
async def app_error_handler(_: Request, exc: AppError) -> JSONResponse:
return JSONResponse(
status_code=exc.status_code,
content=_payload(exc.code, exc.message, exc.details),
)
@app.exception_handler(RequestValidationError)
async def validation_handler(
_: Request, exc: RequestValidationError
) -> JSONResponse:
return JSONResponse(
status_code=422,
content=_payload(
"validation_error",
"Les données envoyées sont invalides.",
{"errors": jsonable_encoder(exc.errors())},
),
)
@app.exception_handler(StarletteHTTPException)
async def http_exception_handler(
_: Request, exc: StarletteHTTPException
) -> JSONResponse:
# Normalize framework-raised HTTP errors (404 route, 405…) to the
# single error shape of §8.3.
code = _HTTP_CODES.get(exc.status_code, "error")
message = (
exc.detail if isinstance(exc.detail, str) else "Une erreur est survenue."
)
return JSONResponse(
status_code=exc.status_code, content=_payload(code, message)
)
+63
View File
@@ -0,0 +1,63 @@
from abc import ABC, abstractmethod
from collections.abc import Iterator
from dataclasses import dataclass, field
from enum import StrEnum
from typing import Any, ClassVar
from sqlalchemy.orm import Session
class ImporterParseError(Exception):
"""Fatal error: the whole file is unreadable/malformed for this importer."""
class RowError(Exception):
"""Non-fatal error on a single row; the run keeps going."""
class UpsertOutcome(StrEnum):
INSERTED = "inserted"
UPDATED = "updated" # daily aggregates replaced in place
DUPLICATE = "duplicate"
ERROR = "error"
@dataclass
class NormalizedRecord:
kind: str # ex: "weight", "food_entry", "transaction"
data: dict[str, Any] # normalized fields (canonical units, see CONVENTIONS)
external_id: str | None = None # stable source id if available
# Fields of `data` used to build content_hash when external_id is None.
dedupe_fields: tuple[str, ...] = field(default_factory=tuple)
class BaseImporter(ABC):
"""One importer = one (source, file format) pair. Stateless; instantiated per run."""
id: ClassVar[str] # unique snake_case, ex: "foodvisor_csv"
label: ClassVar[str] # French label shown in UI, ex: "Foodvisor (export CSV)"
domain: ClassVar[str] # "health" | "vape" | "finance"
accepted_extensions: ClassVar[tuple[str, ...]] # (".csv",), (".ofx",)…
# Set by the imports service (run_import) before the upsert loop, so that
# rows written by `upsert` can reference the ImportRun (FK ondelete CASCADE
# -> central rollback).
import_run_id: int | None = None
@classmethod
@abstractmethod
def sniff(cls, filename: str, head: bytes) -> bool:
"""Return True if this importer recognizes the file (used when source='auto').
`head` = first 4096 bytes. Must be cheap and never raise."""
@abstractmethod
def parse(self, data: bytes, filename: str) -> Iterator[NormalizedRecord]:
"""Decode bytes (handle encodings utf-8/cp1252 and csv dialects) and yield
normalized records. Raise ImporterParseError for a fatally malformed file;
yield-level row errors should raise RowError inside iteration."""
@abstractmethod
def upsert(
self, db: Session, user_id: int, record: NormalizedRecord
) -> UpsertOutcome:
"""Write one record into the module's tables, honoring dedupe rules."""
+10
View File
@@ -0,0 +1,10 @@
import hashlib
import json
from app.core.importing.base import NormalizedRecord
def content_hash(record: NormalizedRecord) -> str:
subset = {k: record.data.get(k) for k in sorted(record.dedupe_fields)}
payload = json.dumps(subset, sort_keys=True, default=str, ensure_ascii=False)
return hashlib.sha256(payload.encode("utf-8")).hexdigest()
+17
View File
@@ -0,0 +1,17 @@
from app.core.importing.base import BaseImporter
IMPORTER_REGISTRY: dict[str, type[BaseImporter]] = {}
def register_importer(cls: type[BaseImporter]) -> type[BaseImporter]:
if cls.id in IMPORTER_REGISTRY:
raise RuntimeError(f"Duplicate importer id: {cls.id}")
IMPORTER_REGISTRY[cls.id] = cls
return cls
def detect_importer(filename: str, head: bytes) -> type[BaseImporter] | None:
matches = [c for c in IMPORTER_REGISTRY.values() if c.sniff(filename, head)]
return (
matches[0] if len(matches) == 1 else None
) # ambiguous -> force explicit choice
+25
View File
@@ -0,0 +1,25 @@
from abc import ABC, abstractmethod
from dataclasses import dataclass
from typing import Any, ClassVar
from sqlalchemy.orm import Session
from app.core.importing.base import UpsertOutcome
@dataclass
class IngestRecord:
type: str # ex: "steps", "weight", "workout"
data: dict[str, Any]
external_id: str | None = None
source: str = "ingest" # sender-declared source (ex: "android_bridge")
class BaseIngestHandler(ABC):
domain: ClassVar[str] # "health"
record_types: ClassVar[tuple[str, ...]] # accepted `type` values
@abstractmethod
def apply(
self, db: Session, user_id: int, record: IngestRecord
) -> UpsertOutcome: ...
+11
View File
@@ -0,0 +1,11 @@
from app.core.ingest.base import BaseIngestHandler
INGEST_REGISTRY: dict[str, BaseIngestHandler] = {}
def register_ingest_handler(cls: type[BaseIngestHandler]) -> type[BaseIngestHandler]:
handler = cls()
if handler.domain in INGEST_REGISTRY:
raise RuntimeError(f"Duplicate ingest handler for domain: {handler.domain}")
INGEST_REGISTRY[handler.domain] = handler
return cls
+31
View File
@@ -0,0 +1,31 @@
from datetime import datetime
from sqlalchemy import JSON, DateTime, String, func
from sqlalchemy.dialects import postgresql
from sqlalchemy.orm import Mapped, mapped_column
# Portable JSON type (see CONVENTIONS C8): plain JSON on SQLite (tests),
# JSONB on PostgreSQL (production). Use this alias for every JSON column.
JSONB_V = JSON().with_variant(postgresql.JSONB(), "postgresql")
class TimestampMixin:
created_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), server_default=func.now()
)
updated_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), server_default=func.now(), onupdate=func.now()
)
class SourceMixin:
"""Provenance + dedupe columns for every imported/ingested row.
source: "manual" | importer id ("foodvisor_csv") | ingest source ("android_bridge")
external_id: stable id from the source system, if any
content_hash: sha256 of canonical payload, used when no external_id exists
"""
source: Mapped[str] = mapped_column(String(50), default="manual")
external_id: Mapped[str | None] = mapped_column(String(255), default=None)
content_hash: Mapped[str | None] = mapped_column(String(64), default=None)
+103
View File
@@ -0,0 +1,103 @@
import importlib
import os
import pkgutil
from collections.abc import Iterable, Iterator
from fastapi import APIRouter, FastAPI
from starlette.routing import BaseRoute
from app import modules as modules_pkg
# Optional side-effect files imported for every module (models -> metadata,
# importers/ingest -> registries populated by decorators).
_SIDE_FILES = ("models", "importers", "ingest")
# Optional comma-separated whitelist of module folder names; unset/empty means
# "discover everything". Used by parallel test runs to isolate modules.
_MODULES_ENV_VAR = "LIFETRACK_MODULES"
def _whitelist() -> set[str] | None:
raw = os.environ.get(_MODULES_ENV_VAR, "").strip()
if not raw:
return None
return {part.strip() for part in raw.split(",") if part.strip()}
def iter_module_names() -> list[str]:
allowed = _whitelist()
names = sorted(
info.name
for info in pkgutil.iter_modules(modules_pkg.__path__)
if not info.name.startswith("_")
)
if allowed is not None:
names = [name for name in names if name in allowed]
return names
def import_side_modules() -> None:
for name in iter_module_names():
for side in _SIDE_FILES:
dotted = f"app.modules.{name}.{side}"
try:
importlib.import_module(dotted)
except ModuleNotFoundError as exc:
# Only swallow "file does not exist"; re-raise real import errors
# coming from inside the file (missing dependency, typo…).
if exc.name != dotted:
raise
def register_routers(app: FastAPI) -> None:
for name in iter_module_names():
mod = importlib.import_module(f"app.modules.{name}.router")
router = getattr(mod, "router", None)
if not isinstance(router, APIRouter):
# RuntimeError (not TypeError): broken module contract, per §3.4.
raise RuntimeError( # noqa: TRY004
f"Module '{name}' must expose an APIRouter named 'router' in router.py"
)
app.include_router(router, prefix="/api")
# Optional extra routers mounted under a different prefix (ex: the
# imports module also serves /api/ingest/{domain}, see §5.5).
extra_routers = getattr(mod, "extra_routers", None) or []
for extra in extra_routers:
if not isinstance(extra, APIRouter):
# RuntimeError (not TypeError): broken module contract, per §3.4.
raise RuntimeError( # noqa: TRY004
f"Module '{name}': every item of 'extra_routers' must be an APIRouter"
)
app.include_router(extra, prefix="/api")
def iter_mounted_routes(app: FastAPI) -> Iterator[tuple[str, frozenset[str]]]:
"""Yield `(full_path, http_methods)` for every route mounted on `app`.
`app.routes` cannot be read directly: since FastAPI 0.140 `include_router()`
appends an opaque `_IncludedRouter` wrapper instead of copying the
sub-router's routes into the parent, so module endpoints never appear as
top-level `APIRoute` objects. This walks the wrappers (recursively, since a
module may itself include sub-routers — `imports` does) and rebuilds the
effective paths. Falls back to plain iteration on older FastAPI versions.
"""
yield from _walk_routes(app.routes, "")
def _walk_routes(
routes: Iterable[BaseRoute], prefix: str
) -> Iterator[tuple[str, frozenset[str]]]:
for route in routes:
included = getattr(route, "original_router", None)
if included is not None:
# `include_context.prefix` already carries the parent router's own
# prefix, so it is not accumulated twice.
context = getattr(route, "include_context", None)
yield from _walk_routes(
included.routes, prefix + getattr(context, "prefix", "")
)
continue
path = getattr(route, "path", None)
if path is None:
continue
yield prefix + path, frozenset(getattr(route, "methods", None) or ())
+35
View File
@@ -0,0 +1,35 @@
from typing import Generic, TypeVar
from fastapi import Query
from pydantic import BaseModel
from sqlalchemy import Select, func, select
from sqlalchemy.orm import Session
T = TypeVar("T")
class PageParams:
def __init__(
self,
page: int = Query(1, ge=1),
page_size: int = Query(50, ge=1, le=200),
):
self.page = page
self.page_size = page_size
@property
def offset(self) -> int:
return (self.page - 1) * self.page_size
class Page(BaseModel, Generic[T]):
items: list[T]
total: int
page: int
page_size: int
def paginate(db: Session, stmt: Select, params: PageParams) -> tuple[list, int]:
total = db.scalar(select(func.count()).select_from(stmt.subquery())) or 0
rows = db.scalars(stmt.offset(params.offset).limit(params.page_size)).all()
return list(rows), total
+90
View File
@@ -0,0 +1,90 @@
import hashlib
import hmac
import secrets
from datetime import timedelta
from typing import TYPE_CHECKING
import jwt
from pwdlib import PasswordHash
from sqlalchemy import select
from sqlalchemy.orm import Session
from app.core.config import get_settings
from app.core.errors import UnauthorizedError
from app.core.timeutils import utcnow
if TYPE_CHECKING:
from app.modules.auth.models import DeviceApiKey
password_hasher = PasswordHash.recommended()
DEVICE_KEY_PREFIX = "ltk"
def hash_password(password: str) -> str:
return password_hasher.hash(password)
def verify_password(password: str, password_hash: str) -> bool:
return password_hasher.verify(password, password_hash)
def create_access_token(user_id: int) -> str:
settings = get_settings()
now = utcnow()
payload = {
"sub": str(user_id),
"iat": now,
"exp": now + timedelta(minutes=settings.access_token_expire_minutes),
}
return jwt.encode(payload, settings.jwt_secret, algorithm=settings.jwt_algorithm)
def decode_access_token(token: str) -> int:
settings = get_settings()
try:
payload = jwt.decode(
token, settings.jwt_secret, algorithms=[settings.jwt_algorithm]
)
return int(payload["sub"])
except (jwt.PyJWTError, KeyError, ValueError) as exc:
raise UnauthorizedError("Session invalide ou expirée.") from exc
def generate_device_key() -> tuple[str, str, str]:
"""Return (full_key, prefix, key_hash) for a new device API key.
Format: ltk_<prefix>_<secret>. Only the sha256 of the full key is stored;
the plaintext key is shown to the user exactly once.
"""
prefix = secrets.token_hex(4) # 8 hex chars
secret = secrets.token_urlsafe(32)
full_key = f"{DEVICE_KEY_PREFIX}_{prefix}_{secret}"
key_hash = hashlib.sha256(full_key.encode("utf-8")).hexdigest()
return full_key, prefix, key_hash
def verify_device_key(db: Session, full_key: str) -> "DeviceApiKey":
"""Validate an X-API-Key value and return its DeviceApiKey row.
Lookup by prefix, constant-time hash comparison, rejects revoked keys,
updates last_used_at. Raises UnauthorizedError on any failure.
"""
from app.modules.auth.models import DeviceApiKey
invalid = UnauthorizedError("Clé d'appareil invalide.")
parts = full_key.split("_", 2)
if len(parts) != 3 or parts[0] != DEVICE_KEY_PREFIX:
raise invalid
prefix = parts[1]
key = db.scalar(select(DeviceApiKey).where(DeviceApiKey.key_prefix == prefix))
if key is None:
raise invalid
candidate_hash = hashlib.sha256(full_key.encode("utf-8")).hexdigest()
if not hmac.compare_digest(candidate_hash, key.key_hash):
raise invalid
if key.revoked_at is not None:
raise UnauthorizedError("Clé d'appareil révoquée.")
key.last_used_at = utcnow()
db.commit()
return key
+57
View File
@@ -0,0 +1,57 @@
from datetime import UTC, date, datetime
from typing import Annotated
from zoneinfo import ZoneInfo
from pydantic import AfterValidator
from app.core.config import get_settings
from app.core.errors import DomainValidationError
def utcnow() -> datetime:
return datetime.now(UTC)
def resolve_tz(tz: str | None) -> ZoneInfo:
name = tz or get_settings().timezone
try:
return ZoneInfo(name)
except Exception as exc:
raise DomainValidationError(f"Fuseau horaire inconnu : {name}") from exc
def local_day(dt_utc: datetime, tz: ZoneInfo) -> date:
return dt_utc.astimezone(tz).date()
def as_utc(value: datetime) -> datetime:
"""Normalise a *stored* instant to UTC-aware.
Every instant is persisted in UTC (CONVENTIONS C2.3), but the driver decides
what comes back: PostgreSQL `TIMESTAMPTZ` yields an aware datetime while
SQLite yields a naive one. Stamping UTC on naive values keeps responses
compliant with the `Z` suffix required by architecture §8.4 on both engines.
"""
return value.replace(tzinfo=UTC) if value.tzinfo is None else value.astimezone(UTC)
def require_utc(value: datetime) -> datetime:
"""Validate an *inbound* instant: naive is refused, aware is converted to UTC.
Architecture §8.4: "entrée avec offset acceptée et convertie en UTC ; entrée
naïve refusée (422)". Silently reading a naive datetime as UTC would record a
local wall-clock time off by the client's offset.
"""
if value.tzinfo is None or value.utcoffset() is None:
raise ValueError(
"Horodatage sans fuseau horaire : précisez le décalage "
"(par exemple « 2026-08-13T06:31:00Z »)."
)
return value.astimezone(UTC)
# Use in request schemas (XxxCreate/XxxUpdate) for any client-supplied instant.
UtcDatetime = Annotated[datetime, AfterValidator(require_utc)]
# Use in response schemas (XxxRead) for any instant read back from the database.
StoredUtcDatetime = Annotated[datetime, AfterValidator(as_utc)]
+63
View File
@@ -0,0 +1,63 @@
from contextlib import asynccontextmanager
from typing import Any
from fastapi import FastAPI
from fastapi.middleware.cors import CORSMiddleware
from fastapi.responses import JSONResponse
from app.core.config import get_settings
from app.core.database import Base, engine
from app.core.errors import register_error_handlers
from app.core.module_loader import import_side_modules, register_routers
@asynccontextmanager
async def lifespan(app: FastAPI):
# Import every module's models/importers/ingest so that:
# - Base.metadata knows all tables before create_all
# - importer & ingest registries are populated (decorator side effects)
import_side_modules()
# v1 table-creation strategy: create_all on startup (Alembic later).
Base.metadata.create_all(bind=engine)
yield
def create_app() -> FastAPI:
settings = get_settings()
app = FastAPI(
title="LifeTrack API",
version="1.0.0",
docs_url="/api/docs",
redoc_url=None,
openapi_url="/api/openapi.json",
lifespan=lifespan,
)
if settings.cors_origins:
app.add_middleware(
CORSMiddleware,
allow_origins=settings.cors_origins,
allow_credentials=True,
allow_methods=["*"],
allow_headers=["*"],
)
register_error_handlers(app)
register_routers(app) # auto-discovery, see architecture §3.4
@app.get("/api/healthz", include_in_schema=False)
def healthz() -> dict[str, str]:
return {"status": "ok"}
# Bare aliases (no /api prefix) for container/orchestrator probes.
@app.get("/healthz", include_in_schema=False)
def healthz_alias() -> dict[str, str]:
return {"status": "ok"}
@app.get("/openapi.json", include_in_schema=False)
def openapi_alias() -> JSONResponse:
schema: dict[str, Any] = app.openapi()
return JSONResponse(schema)
return app
app = create_app()
View File
+36
View File
@@ -0,0 +1,36 @@
from datetime import datetime
from sqlalchemy import DateTime, ForeignKey, String
from sqlalchemy.orm import Mapped, mapped_column
from app.core.database import Base
from app.core.mixins import JSONB_V, TimestampMixin
class User(TimestampMixin, Base):
__tablename__ = "users"
id: Mapped[int] = mapped_column(primary_key=True)
email: Mapped[str] = mapped_column(String(255), unique=True, index=True)
password_hash: Mapped[str] = mapped_column(String(255))
display_name: Mapped[str] = mapped_column(String(100))
is_active: Mapped[bool] = mapped_column(default=True)
class DeviceApiKey(TimestampMixin, Base):
"""Long-lived scoped token for device bridges (Android Health Connect…)."""
__tablename__ = "device_api_keys"
id: Mapped[int] = mapped_column(primary_key=True)
user_id: Mapped[int] = mapped_column(ForeignKey("users.id"), index=True)
name: Mapped[str] = mapped_column(
String(100)
) # ex: "Pixel 8 pont Health Connect"
key_prefix: Mapped[str] = mapped_column(String(12), unique=True, index=True)
key_hash: Mapped[str] = mapped_column(String(64)) # sha256 hex of the full key
scopes: Mapped[list[str]] = mapped_column(
JSONB_V, default=list
) # ["ingest:health"]
last_used_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
revoked_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
+80
View File
@@ -0,0 +1,80 @@
from typing import Annotated
from fastapi import APIRouter, Depends
from sqlalchemy.orm import Session
from app.core.database import get_db
from app.core.dependencies import get_current_user
from app.core.security import create_access_token
from app.modules.auth import service
from app.modules.auth.models import User
from app.modules.auth.schemas import (
AuthStatus,
DeviceKeyCreate,
DeviceKeyCreated,
DeviceKeyRead,
LoginRequest,
SetupRequest,
TokenResponse,
UserRead,
UserUpdate,
)
router = APIRouter(prefix="/auth", tags=["auth"])
DbDep = Annotated[Session, Depends(get_db)]
UserDep = Annotated[User, Depends(get_current_user)]
def _token_response(user: User) -> TokenResponse:
return TokenResponse(
access_token=create_access_token(user.id),
user=UserRead.model_validate(user),
)
@router.get("/status", response_model=AuthStatus)
def auth_status(db: DbDep) -> AuthStatus:
required = service.setup_required(db)
return AuthStatus(setup_required=required, needs_setup=required)
@router.post("/setup", response_model=TokenResponse, status_code=201)
def setup(payload: SetupRequest, db: DbDep) -> TokenResponse:
user = service.create_first_user(db, payload)
return _token_response(user)
@router.post("/login", response_model=TokenResponse)
def login(payload: LoginRequest, db: DbDep) -> TokenResponse:
user = service.authenticate(db, payload.email, payload.password)
return _token_response(user)
@router.get("/me", response_model=UserRead)
def me(user: UserDep) -> User:
return user
@router.patch("/me", response_model=UserRead)
def update_me(payload: UserUpdate, db: DbDep, user: UserDep) -> User:
return service.update_me(db, user, payload)
@router.get("/device-keys", response_model=list[DeviceKeyRead])
def list_device_keys(db: DbDep, user: UserDep) -> list[DeviceKeyRead]:
return service.list_device_keys(db, user.id)
@router.post("/device-keys", response_model=DeviceKeyCreated, status_code=201)
def create_device_key(
payload: DeviceKeyCreate, db: DbDep, user: UserDep
) -> DeviceKeyCreated:
key, plaintext = service.create_device_key(db, user.id, payload)
read = DeviceKeyRead.model_validate(key)
return DeviceKeyCreated(**read.model_dump(), key=plaintext)
@router.delete("/device-keys/{key_id}", status_code=204)
def revoke_device_key(key_id: int, db: DbDep, user: UserDep) -> None:
service.revoke_device_key(db, user.id, key_id)
+85
View File
@@ -0,0 +1,85 @@
from pydantic import BaseModel, ConfigDict, Field, field_validator
from app.core.timeutils import StoredUtcDatetime
class AuthStatus(BaseModel):
# `setup_required` is the documented flag (§4.4); `needs_setup` is kept as
# a synonym for tooling compatibility.
setup_required: bool
needs_setup: bool
class SetupRequest(BaseModel):
email: str = Field(
min_length=3, max_length=255, pattern=r"^[^@\s]+@[^@\s]+\.[^@\s]+$"
)
password: str = Field(min_length=8, max_length=128)
display_name: str = Field(min_length=1, max_length=100)
class LoginRequest(BaseModel):
email: str
password: str
class UserRead(BaseModel):
model_config = ConfigDict(from_attributes=True)
id: int
email: str
display_name: str
created_at: StoredUtcDatetime
class UserUpdate(BaseModel):
email: str | None = Field(
default=None,
min_length=3,
max_length=255,
pattern=r"^[^@\s]+@[^@\s]+\.[^@\s]+$",
)
display_name: str | None = Field(default=None, min_length=1, max_length=100)
password: str | None = Field(default=None, min_length=8, max_length=128)
current_password: str | None = None
class TokenResponse(BaseModel):
access_token: str
token_type: str = "bearer"
user: UserRead
class DeviceKeyCreate(BaseModel):
name: str = Field(min_length=1, max_length=100)
scopes: list[str] = Field(min_length=1)
@field_validator("scopes")
@classmethod
def check_scopes(cls, scopes: list[str]) -> list[str]:
for scope in scopes:
if scope != "ingest:*" and not (
scope.startswith("ingest:") and len(scope) > len("ingest:")
):
raise ValueError(
"Les droits doivent être de la forme « ingest:<domaine> » ou « ingest:* »."
)
return scopes
class DeviceKeyRead(BaseModel):
model_config = ConfigDict(from_attributes=True)
id: int
name: str
key_prefix: str
scopes: list[str]
last_used_at: StoredUtcDatetime | None
created_at: StoredUtcDatetime
revoked_at: StoredUtcDatetime | None
class DeviceKeyCreated(DeviceKeyRead):
"""Returned once at creation time: the only moment the plaintext key exists."""
key: str
+109
View File
@@ -0,0 +1,109 @@
from sqlalchemy import func, select
from sqlalchemy.orm import Session
from app.core.errors import (
ConflictError,
DomainValidationError,
ForbiddenError,
NotFoundError,
UnauthorizedError,
)
from app.core.security import (
generate_device_key,
hash_password,
verify_password,
)
from app.core.timeutils import utcnow
from app.modules.auth.models import DeviceApiKey, User
from app.modules.auth.schemas import DeviceKeyCreate, SetupRequest, UserUpdate
def setup_required(db: Session) -> bool:
return (db.scalar(select(func.count()).select_from(User)) or 0) == 0
def create_first_user(db: Session, data: SetupRequest) -> User:
if not setup_required(db):
raise ForbiddenError(
"Un utilisateur existe déjà : l'assistant de configuration est désactivé."
)
user = User(
email=data.email.strip().lower(),
password_hash=hash_password(data.password),
display_name=data.display_name.strip(),
)
db.add(user)
db.commit()
return user
def authenticate(db: Session, email: str, password: str) -> User:
user = db.scalar(select(User).where(User.email == email.strip().lower()))
if (
user is None
or not user.is_active
or not verify_password(password, user.password_hash)
):
raise UnauthorizedError("Adresse e-mail ou mot de passe incorrect.")
return user
def update_me(db: Session, user: User, data: UserUpdate) -> User:
if data.password is not None or (
data.email is not None and data.email.strip().lower() != user.email
):
if not data.current_password:
raise DomainValidationError(
"Le mot de passe actuel est requis pour cette modification."
)
if not verify_password(data.current_password, user.password_hash):
raise UnauthorizedError("Mot de passe actuel incorrect.")
if data.email is not None:
email = data.email.strip().lower()
if email != user.email:
existing = db.scalar(select(User).where(User.email == email))
if existing is not None:
raise ConflictError("Cette adresse e-mail est déjà utilisée.")
user.email = email
if data.display_name is not None:
user.display_name = data.display_name.strip()
if data.password is not None:
user.password_hash = hash_password(data.password)
db.commit()
return user
def list_device_keys(db: Session, user_id: int) -> list[DeviceApiKey]:
stmt = (
select(DeviceApiKey)
.where(DeviceApiKey.user_id == user_id)
.order_by(DeviceApiKey.created_at.desc(), DeviceApiKey.id.desc())
)
return list(db.scalars(stmt).all())
def create_device_key(
db: Session, user_id: int, data: DeviceKeyCreate
) -> tuple[DeviceApiKey, str]:
"""Create a device key; returns (row, plaintext_key). The plaintext key is
returned exactly once and never stored."""
full_key, prefix, key_hash = generate_device_key()
key = DeviceApiKey(
user_id=user_id,
name=data.name.strip(),
key_prefix=prefix,
key_hash=key_hash,
scopes=list(data.scopes),
)
db.add(key)
db.commit()
return key, full_key
def revoke_device_key(db: Session, user_id: int, key_id: int) -> None:
key = db.get(DeviceApiKey, key_id)
if key is None or key.user_id != user_id:
raise NotFoundError("Clé d'appareil introuvable.")
if key.revoked_at is None:
key.revoked_at = utcnow()
db.commit()
+577
View File
@@ -0,0 +1,577 @@
"""Rules engine, internal-transfer detection and recurring-series detection.
Reference: datamodel-finance.md §5 (rules), §6 (transfers), §7 (recurring).
Nothing here ever overwrites a manual categorisation (`category_source =
'user'`) unless the caller explicitly forces it (§10.6).
"""
import re
import uuid
from collections import Counter, defaultdict
from dataclasses import dataclass, field
from datetime import date, timedelta
from decimal import Decimal
from itertools import pairwise
from statistics import median
from typing import Any
from sqlalchemy import select
from sqlalchemy.orm import Session
from app.core.errors import DomainValidationError, NotFoundError
from app.core.timeutils import utcnow
from app.modules.finance.enums import CategoryKind, CategorySource
from app.modules.finance.models import FinCategory, FinRule, FinTransaction
from app.modules.finance.normalize import merchant_key, normalize_label_for_hash
TRANSFER_MAX_DAYS = 3
TRANSFER_LABEL_RE = re.compile(r"\bVIR(EMENT)?\b|\bVIRT\b|TRANSFERT|PAYPAL")
RULE_SCOPES = ("uncategorized", "all_non_manual", "all")
# (name, min_interval_days, max_interval_days, nominal_interval_days)
PERIODICITIES: tuple[tuple[str, int, int, int], ...] = (
("weekly", 6, 8, 7),
("monthly", 25, 35, 30),
("quarterly", 85, 97, 91),
("yearly", 350, 380, 365),
)
# Monthly normalisation factor used by `monthly_total_estimate` (§9.8).
MONTHLY_FACTOR: dict[str, Decimal] = {
"weekly": Decimal("4.33"),
"monthly": Decimal(1),
"quarterly": Decimal(1) / Decimal(3),
"yearly": Decimal(1) / Decimal(12),
}
MIN_OCCURRENCES = 3
REGULARITY_RATIO = 0.8
# ---------------------------------------------------------------------------
# Rules engine (§5)
# ---------------------------------------------------------------------------
def compile_matchers(matchers: dict[str, Any]) -> re.Pattern[str] | None:
"""Compile `label_regex` once; raises DomainValidationError if invalid."""
pattern = matchers.get("label_regex")
if not pattern:
return None
try:
return re.compile(pattern, re.IGNORECASE)
except re.error as exc:
raise DomainValidationError(f"Expression régulière invalide : {exc}") from exc
def validate_matchers(matchers: dict[str, Any]) -> None:
if not isinstance(matchers, dict):
raise DomainValidationError("Conditions de règle invalides.")
direction = matchers.get("direction")
if direction is not None and direction not in ("debit", "credit", "any"):
raise DomainValidationError(
"Sens invalide : utilisez « debit », « credit » ou « any »."
)
conditions = (
"label_contains",
"label_regex",
"amount_min",
"amount_max",
"account_id",
)
has_condition = any(
matchers.get(key) not in (None, [], "") for key in conditions
) or direction in ("debit", "credit")
if not has_condition:
raise DomainValidationError("La règle doit comporter au moins une condition.")
compile_matchers(matchers)
def validate_actions(actions: dict[str, Any]) -> None:
if not isinstance(actions, dict) or not any(
actions.get(key) not in (None, "", False)
for key in (
"set_category_id",
"set_label_clean",
"set_counterparty",
"mark_transfer",
)
):
raise DomainValidationError("La règle doit comporter au moins une action.")
@dataclass
class CompiledRule:
rule: FinRule
regex: re.Pattern[str] | None
contains: tuple[str, ...]
@property
def matchers(self) -> dict[str, Any]:
return self.rule.matchers or {}
@property
def actions(self) -> dict[str, Any]:
return self.rule.actions or {}
def compile_rules(rules: list[FinRule]) -> list[CompiledRule]:
compiled: list[CompiledRule] = []
for rule in rules:
matchers = rule.matchers or {}
try:
regex = compile_matchers(matchers)
except DomainValidationError:
regex = None # stored rule broke: ignore its regex, keep the rest
contains = tuple(
normalize_label_for_hash(str(needle))
for needle in (matchers.get("label_contains") or [])
if str(needle).strip()
)
compiled.append(CompiledRule(rule=rule, regex=regex, contains=contains))
return compiled
def load_enabled_rules(db: Session, user_id: int) -> list[CompiledRule]:
stmt = (
select(FinRule)
.where(FinRule.user_id == user_id, FinRule.enabled.is_(True))
.order_by(FinRule.priority.asc(), FinRule.created_at.asc(), FinRule.id.asc())
)
return compile_rules(list(db.scalars(stmt).all()))
def rule_matches(compiled: CompiledRule, tx: FinTransaction) -> bool:
m = compiled.matchers
account_id = m.get("account_id")
if account_id and str(tx.account_id) != str(account_id):
return False
direction = m.get("direction")
if direction == "debit" and tx.amount >= 0:
return False
if direction == "credit" and tx.amount <= 0:
return False
if m.get("amount_min") is not None and tx.amount < Decimal(str(m["amount_min"])):
return False
if m.get("amount_max") is not None and tx.amount > Decimal(str(m["amount_max"])):
return False
if compiled.contains:
hay = (
normalize_label_for_hash(tx.label_raw)
+ " | "
+ normalize_label_for_hash(tx.label_clean or "")
)
if not any(needle in hay for needle in compiled.contains):
return False
return compiled.regex is None or bool(
compiled.regex.search(tx.label_raw or "")
or compiled.regex.search(tx.label_clean or "")
)
def apply_rules_to_tx(
tx: FinTransaction,
rules: list[CompiledRule],
transfer_category_id: uuid.UUID | None,
*,
force: bool = False,
) -> list[uuid.UUID]:
"""Mutate `tx` in place; returns the ids of the rules that fired (§5.3)."""
fired: list[uuid.UUID] = []
for compiled in rules:
if not rule_matches(compiled, tx):
continue
actions = compiled.actions
protected = tx.category_source == CategorySource.USER and not force
if actions.get("set_category_id") and not protected:
tx.category_id = uuid.UUID(str(actions["set_category_id"]))
tx.category_source = CategorySource.RULE
tx.applied_rule_id = compiled.rule.id
if actions.get("set_label_clean"):
tx.label_clean = str(actions["set_label_clean"])
if actions.get("set_counterparty"):
tx.counterparty = str(actions["set_counterparty"])[:150]
if actions.get("mark_transfer") and not protected and transfer_category_id:
tx.category_id = transfer_category_id
tx.category_source = CategorySource.RULE
tx.applied_rule_id = compiled.rule.id
fired.append(compiled.rule.id)
compiled.rule.hit_count += 1
compiled.rule.last_applied_at = utcnow()
if compiled.rule.stop:
break
return fired
def get_transfer_category_id(db: Session, user_id: int) -> uuid.UUID | None:
stmt = select(FinCategory.id).where(
FinCategory.user_id == user_id,
FinCategory.kind == CategoryKind.TRANSFER,
)
return db.scalars(stmt.limit(1)).first()
@dataclass
class RuleApplyResult:
scanned: int = 0
matched: int = 0
updated: int = 0
dry_run: bool = False
by_rule: list[dict[str, Any]] = field(default_factory=list)
def apply_rules(
db: Session,
user_id: int,
*,
scope: str = "uncategorized",
date_from: date | None = None,
date_to: date | None = None,
account_id: uuid.UUID | None = None,
rule_id: uuid.UUID | None = None,
dry_run: bool = False,
force: bool = False,
) -> RuleApplyResult:
"""Re-run the rules over stored transactions (§5.4)."""
if scope not in RULE_SCOPES:
raise DomainValidationError(
"Portée inconnue : utilisez « uncategorized », « all_non_manual » "
"ou « all »."
)
if scope == "all" and not force:
raise DomainValidationError(
"La portée « all » écrase les catégories manuelles : "
"confirmez avec force=true."
)
rules = load_enabled_rules(db, user_id)
if rule_id is not None:
rules = [c for c in rules if c.rule.id == rule_id]
if not rules:
raise NotFoundError("Règle introuvable.")
stmt = select(FinTransaction).where(FinTransaction.user_id == user_id)
if scope == "uncategorized":
stmt = stmt.where(FinTransaction.category_id.is_(None))
elif scope == "all_non_manual":
stmt = stmt.where(
(FinTransaction.category_source.is_(None))
| (FinTransaction.category_source != CategorySource.USER)
)
if date_from is not None:
stmt = stmt.where(FinTransaction.booked_date >= date_from)
if date_to is not None:
stmt = stmt.where(FinTransaction.booked_date <= date_to)
if account_id is not None:
stmt = stmt.where(FinTransaction.account_id == account_id)
stmt = stmt.order_by(FinTransaction.booked_date.asc(), FinTransaction.id.asc())
transfer_cat = get_transfer_category_id(db, user_id)
result = RuleApplyResult(dry_run=dry_run)
per_rule: Counter[uuid.UUID] = Counter()
for tx in db.scalars(stmt.execution_options(yield_per=1000)):
result.scanned += 1
before = (tx.category_id, tx.category_source, tx.label_clean, tx.counterparty)
fired = apply_rules_to_tx(tx, rules, transfer_cat, force=force)
if fired:
result.matched += 1
for fired_id in fired:
per_rule[fired_id] += 1
after = (tx.category_id, tx.category_source, tx.label_clean, tx.counterparty)
if before != after:
result.updated += 1
result.by_rule = [
{
"rule_id": compiled.rule.id,
"name": compiled.rule.name,
"matched": per_rule.get(compiled.rule.id, 0),
}
for compiled in rules
if per_rule.get(compiled.rule.id, 0)
]
if dry_run:
db.rollback()
else:
db.commit()
return result
def preview_rule(
db: Session,
user_id: int,
matchers: dict[str, Any],
limit: int = 50,
) -> tuple[list[FinTransaction], int]:
"""Transactions that an unsaved rule would match (§9.4 `/rules/preview`)."""
validate_matchers(matchers)
probe = FinRule(
id=uuid.uuid4(),
user_id=user_id,
name="preview",
matchers=matchers,
actions={},
)
compiled = compile_rules([probe])[0]
stmt = (
select(FinTransaction)
.where(FinTransaction.user_id == user_id)
.order_by(FinTransaction.booked_date.desc(), FinTransaction.id.desc())
)
matches: list[FinTransaction] = []
total = 0
for tx in db.scalars(stmt.execution_options(yield_per=1000)):
if rule_matches(compiled, tx):
total += 1
if len(matches) < limit:
matches.append(tx)
return matches, total
# ---------------------------------------------------------------------------
# Internal transfers (§6)
# ---------------------------------------------------------------------------
def detect_transfers(
db: Session,
user_id: int,
date_min: date | None = None,
date_max: date | None = None,
) -> int:
"""Pair opposite transactions across accounts; returns the pairs created."""
stmt = select(FinTransaction).where(
FinTransaction.user_id == user_id,
FinTransaction.transfer_group_id.is_(None),
)
if date_min is not None:
stmt = stmt.where(FinTransaction.booked_date >= date_min)
if date_max is not None:
stmt = stmt.where(FinTransaction.booked_date <= date_max)
txs = list(db.scalars(stmt).all())
debits = [t for t in txs if t.amount < 0]
credits_by_amount: dict[Decimal, list[FinTransaction]] = defaultdict(list)
for t in txs:
if t.amount > 0:
credits_by_amount[t.amount].append(t)
pairs: list[tuple[int, FinTransaction, FinTransaction]] = []
for debit in debits:
for credit in credits_by_amount.get(-debit.amount, []):
if credit.account_id == debit.account_id:
continue
if credit.currency != debit.currency:
continue
delta = abs((credit.booked_date - debit.booked_date).days)
if delta > TRANSFER_MAX_DAYS:
continue
score = (TRANSFER_MAX_DAYS - delta) * 10
joined = normalize_label_for_hash(f"{debit.label_raw} {credit.label_raw}")
if TRANSFER_LABEL_RE.search(joined):
score += 5
pairs.append((score, debit, credit))
pairs.sort(key=lambda p: (-p[0], p[1].booked_date, str(p[1].id)))
transfer_cat = get_transfer_category_id(db, user_id)
used: set[uuid.UUID] = set()
created = 0
for _score, debit, credit in pairs:
if debit.id in used or credit.id in used:
continue
group_id = uuid.uuid4()
for leg in (debit, credit):
leg.transfer_group_id = group_id
if leg.category_source != CategorySource.USER:
leg.category_id = transfer_cat
leg.category_source = CategorySource.RULE
used |= {debit.id, credit.id}
created += 1
return created
def link_transfer(
db: Session, user_id: int, id_a: uuid.UUID, id_b: uuid.UUID
) -> uuid.UUID:
"""Manual pairing (§6): opposite amounts, different accounts."""
if id_a == id_b:
raise DomainValidationError("Sélectionnez deux transactions différentes.")
legs = list(
db.scalars(
select(FinTransaction).where(
FinTransaction.user_id == user_id,
FinTransaction.id.in_([id_a, id_b]),
)
).all()
)
if len(legs) != 2:
raise NotFoundError("Transaction introuvable.")
first, second = legs
if first.amount + second.amount != 0:
raise DomainValidationError(
"Les montants doivent être strictement opposés pour former "
"un virement interne."
)
if first.account_id == second.account_id:
raise DomainValidationError(
"Les deux jambes d'un virement doivent appartenir à des comptes différents."
)
if first.currency != second.currency:
raise DomainValidationError("Les deux jambes doivent être dans la même devise.")
group_id = uuid.uuid4()
transfer_cat = get_transfer_category_id(db, user_id)
for leg in legs:
leg.transfer_group_id = group_id
if leg.category_source != CategorySource.USER:
leg.category_id = transfer_cat
leg.category_source = CategorySource.RULE
db.commit()
return group_id
def unlink_transfer(db: Session, user_id: int, group_id: uuid.UUID) -> None:
legs = list(
db.scalars(
select(FinTransaction).where(
FinTransaction.user_id == user_id,
FinTransaction.transfer_group_id == group_id,
)
).all()
)
if not legs:
raise NotFoundError("Virement introuvable.")
transfer_cat = get_transfer_category_id(db, user_id)
for leg in legs:
leg.transfer_group_id = None
if (
leg.category_id == transfer_cat
and leg.category_source == CategorySource.RULE
):
leg.category_id = None
leg.category_source = None
db.commit()
# ---------------------------------------------------------------------------
# Recurring series (§7)
# ---------------------------------------------------------------------------
@dataclass
class RecurringSeries:
merchant_key: str
label_display: str
category_id: uuid.UUID | None
category_name: str | None
periodicity: str
occurrences: int
average_amount: Decimal
expected_amount: Decimal
last_date: date
next_date_predicted: date
is_active: bool
def _most_common(values: list[str]) -> str:
counter = Counter(v for v in values if v)
return counter.most_common(1)[0][0] if counter else ""
def detect_recurring(
db: Session,
user_id: int,
*,
direction: str = "debit",
lookback_months: int = 18,
include_inactive: bool = False,
today: date | None = None,
) -> list[RecurringSeries]:
"""On-the-fly recurring detection (§7.2) — no dedicated table in v1."""
today = today or utcnow().date()
since = today - timedelta(days=int(lookback_months * 30.44))
stmt = select(FinTransaction).where(
FinTransaction.user_id == user_id,
FinTransaction.booked_date >= since,
FinTransaction.transfer_group_id.is_(None),
)
stmt = stmt.where(
FinTransaction.amount < 0 if direction == "debit" else FinTransaction.amount > 0
)
txs = list(db.scalars(stmt).all())
if not txs:
return []
category_names = dict(
db.execute(
select(FinCategory.id, FinCategory.name).where(
FinCategory.user_id == user_id
)
).all()
)
transfer_cat = get_transfer_category_id(db, user_id)
groups: dict[str, list[FinTransaction]] = defaultdict(list)
for tx in txs:
if transfer_cat is not None and tx.category_id == transfer_cat:
continue
groups[merchant_key(tx.label_raw, tx.counterparty)].append(tx)
series: list[RecurringSeries] = []
for key, items in groups.items():
if not key or len(items) < MIN_OCCURRENCES:
continue
dates = sorted({t.booked_date for t in items})
if len(dates) < MIN_OCCURRENCES:
continue
intervals = [(b - a).days for a, b in pairwise(dates)]
med_int = median(intervals)
period = next(
(p for p in PERIODICITIES if p[1] <= med_int <= p[2]),
None,
)
if period is None:
continue
regular = sum(1 for i in intervals if period[1] <= i <= period[2])
if regular / len(intervals) < REGULARITY_RATIO:
continue
amounts = [abs(t.amount) for t in items]
med_amt = Decimal(str(median(amounts)))
mad = Decimal(str(median([abs(a - med_amt) for a in amounts])))
if mad > max(Decimal("1.00"), med_amt * Decimal("0.10")):
continue
cat_counter = Counter(t.category_id for t in items if t.category_id)
category_id = cat_counter.most_common(1)[0][0] if cat_counter else None
average = (sum(amounts) / Decimal(len(amounts))).quantize(Decimal("0.01"))
is_active = (today - dates[-1]).days <= period[2] * 2
if not is_active and not include_inactive:
continue
series.append(
RecurringSeries(
merchant_key=key,
label_display=_most_common([t.label_clean for t in items]) or key,
category_id=category_id,
category_name=category_names.get(category_id),
periodicity=period[0],
occurrences=len(dates),
average_amount=average,
expected_amount=med_amt.quantize(Decimal("0.01")),
last_date=dates[-1],
next_date_predicted=dates[-1]
+ timedelta(days=round(median(intervals))),
is_active=is_active,
)
)
series.sort(key=lambda s: (not s.is_active, s.next_date_predicted))
return series
def monthly_total_estimate(series: list[RecurringSeries]) -> Decimal:
total = sum(
(
s.expected_amount * MONTHLY_FACTOR.get(s.periodicity, Decimal(1))
for s in series
if s.is_active
),
Decimal(0),
)
return Decimal(total).quantize(Decimal("0.01"))
+33
View File
@@ -0,0 +1,33 @@
from enum import StrEnum
class AccountKind(StrEnum):
CHECKING = "checking"
SAVINGS = "savings"
PAYPAL = "paypal"
CASH = "cash"
OTHER = "other"
class CategoryKind(StrEnum):
INCOME = "income"
EXPENSE = "expense"
TRANSFER = "transfer"
class SourceKind(StrEnum):
CSV = "csv"
OFX = "ofx"
PAYPAL_CSV = "paypal_csv"
class ImportStatus(StrEnum):
PENDING = "pending"
RUNNING = "running"
COMPLETED = "completed"
FAILED = "failed"
class CategorySource(StrEnum):
RULE = "rule"
USER = "user"
+191
View File
@@ -0,0 +1,191 @@
"""File importers registered in the core registry (CONVENTIONS C3).
They make the finance formats available through the generic
`POST /api/imports` endpoint (source = importer id). The richer, account-aware
flow (profile choice, preview, per-run stats) lives in `POST /api/finance/imports`
and shares the same parsers/dedup helpers — see `pipeline.py`.
Because the generic endpoint carries no account, rows land in a per-format
default account created on first use; the user can rename it afterwards.
"""
from collections.abc import Iterator
from typing import Any, ClassVar
from sqlalchemy import select
from sqlalchemy.orm import Session
from app.core.importing.base import (
BaseImporter,
ImporterParseError,
NormalizedRecord,
UpsertOutcome,
)
from app.core.importing.registry import register_importer
from app.modules.finance import service
from app.modules.finance.categorize import (
apply_rules_to_tx,
get_transfer_category_id,
load_enabled_rules,
)
from app.modules.finance.enums import SourceKind
from app.modules.finance.models import FinTransaction
from app.modules.finance.normalize import compute_dedup_hash, occurrence_key
from app.modules.finance.parsers import NormalizedRow
from app.modules.finance.pipeline import build_transaction, parse_file
from app.modules.finance.presets import BUILTIN_PROFILES
# Header signatures used by `sniff()` (lowercased, accent-insensitive enough).
_BANK_HEADER_HINTS = (
"dateop;",
"date;date valeur",
"date;libell",
"date de comptabilisation;",
"date op",
"date_comptabilisation;",
"booking date",
"completed date",
"montant",
"d\xe9bit",
"debit;",
)
_PAYPAL_HINTS = ("transaction id", "num\xe9ro de transaction", "numero de transaction")
def _preset_config(slug: str) -> dict[str, Any]:
for preset in BUILTIN_PROFILES:
if preset["slug"] == slug:
return preset["config"]
return {}
class FinanceImporter(BaseImporter):
"""Shared upsert logic: dedup by (account, hash) then (account, external_id)."""
source_kind: ClassVar[str]
profile_slug: ClassVar[str]
default_account_name: ClassVar[str]
domain: ClassVar[str] = "finance"
def __init__(self) -> None:
self._occurrences: dict[str, int] = {}
def config(self) -> dict[str, Any]:
return _preset_config(self.profile_slug)
def parse(self, data: bytes, filename: str) -> Iterator[NormalizedRecord]:
parsed = parse_file(self.source_kind, self.config(), data)
# Only a file where every data row errored is unusable; rows filtered
# on purpose (§3.3) are a legitimate empty import.
if parsed.rows_total and parsed.rows_error == parsed.rows_total:
raise ImporterParseError(
"Aucune ligne exploitable — vérifiez le profil de source."
)
for row in parsed.rows:
yield NormalizedRecord(
kind="transaction",
data={"row": row},
external_id=row.external_id,
dedupe_fields=("booked_date", "amount", "label_raw"),
)
def upsert(
self, db: Session, user_id: int, record: NormalizedRecord
) -> UpsertOutcome:
row: NormalizedRow = record.data["row"]
service.ensure_seed(db, user_id)
account = service.account_for_source_kind(
db, user_id, self.source_kind, self.default_account_name
)
# Occurrence numbering of §4.3, kept across the whole file by the
# importer instance (one instance per run).
key = occurrence_key(row.booked_date, row.amount, row.label_raw)
occurrence = self._occurrences.get(key, 0)
self._occurrences[key] = occurrence + 1
dedup_hash = compute_dedup_hash(
account.id, row.booked_date, row.amount, row.label_raw, occurrence
)
if row.external_id:
exists = db.scalars(
select(FinTransaction.id).where(
FinTransaction.account_id == account.id,
FinTransaction.external_id == row.external_id,
)
).first()
if exists is not None:
return UpsertOutcome.DUPLICATE
exists = db.scalars(
select(FinTransaction.id).where(
FinTransaction.account_id == account.id,
FinTransaction.dedup_hash == dedup_hash,
)
).first()
if exists is not None:
return UpsertOutcome.DUPLICATE
tx = build_transaction(user_id, account, row, dedup_hash, self.import_run_id)
apply_rules_to_tx(
tx, load_enabled_rules(db, user_id), get_transfer_category_id(db, user_id)
)
db.add(tx)
db.flush()
return UpsertOutcome.INSERTED
@register_importer
class BankGenericCsvImporter(FinanceImporter):
id = "bank_generic_csv"
label = "Relevé bancaire (CSV générique)"
accepted_extensions = (".csv", ".txt")
source_kind = SourceKind.CSV
profile_slug = "generic"
default_account_name = "Compte importé (CSV)"
@classmethod
def sniff(cls, filename: str, head: bytes) -> bool:
if not filename.lower().endswith(cls.accepted_extensions):
return False
try:
text = head.decode("utf-8", errors="replace").lower()
except Exception: # noqa: BLE001 — sniff must never raise
return False
first = text.splitlines()[0] if text.splitlines() else ""
if any(hint in first for hint in _PAYPAL_HINTS):
return False
return any(hint in first for hint in _BANK_HEADER_HINTS)
@register_importer
class BankOfxImporter(FinanceImporter):
id = "bank_ofx"
label = "Relevé bancaire (OFX)"
accepted_extensions = (".ofx", ".qfx")
source_kind = SourceKind.OFX
profile_slug = "ofx"
default_account_name = "Compte importé (OFX)"
@classmethod
def sniff(cls, filename: str, head: bytes) -> bool:
if not filename.lower().endswith(cls.accepted_extensions):
return False
text = head.decode("latin-1", errors="replace").upper()
return "OFXHEADER" in text or "<OFX>" in text
@register_importer
class PaypalCsvImporter(FinanceImporter):
id = "paypal_csv"
label = "PayPal — rapport d'activité (CSV)"
accepted_extensions = (".csv",)
source_kind = SourceKind.PAYPAL_CSV
profile_slug = "paypal"
default_account_name = "PayPal"
@classmethod
def sniff(cls, filename: str, head: bytes) -> bool:
if not filename.lower().endswith(cls.accepted_extensions):
return False
text = head.decode("utf-8", errors="replace").lower()
first = text.splitlines()[0] if text.splitlines() else ""
return any(hint in first for hint in _PAYPAL_HINTS)
+319
View File
@@ -0,0 +1,319 @@
"""Finance module tables (datamodel-finance.md §2), portable SQLite/PostgreSQL.
Deviations from the normative DDL, imposed by the finished backend core
("real code wins"): `users.id` and the central `import_runs.id` are integers,
so every FK to those tables is an int. Import rollback is centralised: any
column referencing `import_runs.id` declares ondelete="CASCADE" (CONVENTIONS
C8.5). The pg_trgm GIN index of §2.5 is created on PostgreSQL only, best
effort — see `_TRGM_DDL` at the bottom of this module.
Deduplication follows §4.3 of the datamodel (which supersedes the generic
CONVENTIONS C2.4 pair for this module): UNIQUE(account_id, dedup_hash) plus a
partial UNIQUE(account_id, external_id) — the finance scope is the account, not
the user, and the hash embeds an occurrence counter so that two identical
purchases of the same day both survive.
"""
import uuid
from datetime import date, datetime
from decimal import Decimal
from typing import Any
from sqlalchemy import (
DDL,
CheckConstraint,
Date,
DateTime,
Enum,
ForeignKey,
Index,
Numeric,
String,
Text,
UniqueConstraint,
Uuid,
event,
text,
)
from sqlalchemy.orm import Mapped, mapped_column
from app.core.database import Base
from app.core.mixins import JSONB_V, TimestampMixin
from app.modules.finance.enums import (
AccountKind,
CategoryKind,
CategorySource,
ImportStatus,
SourceKind,
)
def _enum(enum_cls: type, name: str) -> Enum:
"""Portable enum column: VARCHAR + CHECK, never a native PG ENUM (C8.1).
`create_constraint=True` is explicit: SQLAlchemy defaults it to False, which
would degrade the column to a bare VARCHAR with no integrity check at all on
PostgreSQL as well as on SQLite.
"""
return Enum(
enum_cls,
name=name,
native_enum=False,
create_constraint=True,
length=20,
values_callable=lambda e: [m.value for m in e],
)
class FinAccount(TimestampMixin, Base):
__tablename__ = "fin_accounts"
__table_args__ = (
UniqueConstraint("user_id", "name", name="uq_fin_accounts_user_name"),
)
id: Mapped[uuid.UUID] = mapped_column(Uuid, primary_key=True, default=uuid.uuid4)
user_id: Mapped[int] = mapped_column(
ForeignKey("users.id", ondelete="CASCADE"), index=True
)
name: Mapped[str] = mapped_column(String(100))
kind: Mapped[AccountKind] = mapped_column(
_enum(AccountKind, "fin_account_kind"), default=AccountKind.CHECKING
)
currency: Mapped[str] = mapped_column(String(3), default="EUR")
institution: Mapped[str | None] = mapped_column(String(100), default=None)
# Never the full IBAN: "FR76 **** **** **** 1234" at most.
iban_masked: Mapped[str | None] = mapped_column(String(34), default=None)
initial_balance: Mapped[Decimal] = mapped_column(
Numeric(12, 2), default=Decimal("0.00")
)
is_archived: Mapped[bool] = mapped_column(default=False)
class FinCategory(TimestampMixin, Base):
__tablename__ = "fin_categories"
__table_args__ = (
UniqueConstraint(
"user_id", "parent_id", "name", name="uq_fin_categories_sibling"
),
)
id: Mapped[uuid.UUID] = mapped_column(Uuid, primary_key=True, default=uuid.uuid4)
user_id: Mapped[int] = mapped_column(
ForeignKey("users.id", ondelete="CASCADE"), index=True
)
parent_id: Mapped[uuid.UUID | None] = mapped_column(
ForeignKey("fin_categories.id", ondelete="CASCADE"), index=True, default=None
)
name: Mapped[str] = mapped_column(String(80))
icon: Mapped[str | None] = mapped_column(String(50), default=None)
color: Mapped[str | None] = mapped_column(String(7), default=None)
kind: Mapped[CategoryKind] = mapped_column(
_enum(CategoryKind, "fin_category_kind"), default=CategoryKind.EXPENSE
)
is_system: Mapped[bool] = mapped_column(default=False)
sort_order: Mapped[int] = mapped_column(default=0)
class FinSourceProfile(TimestampMixin, Base):
__tablename__ = "fin_source_profiles"
__table_args__ = (
CheckConstraint(
"(is_builtin = true AND user_id IS NULL)"
" OR (is_builtin = false AND user_id IS NOT NULL)",
name="ck_fin_source_profiles_builtin",
),
Index(
"uq_fin_source_profiles_builtin_name",
"name",
unique=True,
sqlite_where=text("user_id IS NULL"),
postgresql_where=text("user_id IS NULL"),
),
Index(
"uq_fin_source_profiles_user_name",
"user_id",
"name",
unique=True,
sqlite_where=text("user_id IS NOT NULL"),
postgresql_where=text("user_id IS NOT NULL"),
),
)
id: Mapped[uuid.UUID] = mapped_column(Uuid, primary_key=True, default=uuid.uuid4)
# NULL = built-in preset, visible by every user, read-only through the API.
user_id: Mapped[int | None] = mapped_column(
ForeignKey("users.id", ondelete="CASCADE"), default=None
)
name: Mapped[str] = mapped_column(String(100))
kind: Mapped[SourceKind] = mapped_column(_enum(SourceKind, "fin_source_kind"))
config: Mapped[dict[str, Any]] = mapped_column(JSONB_V, default=dict)
is_builtin: Mapped[bool] = mapped_column(default=False)
class FinImportRun(TimestampMixin, Base):
"""Finance-side record of an import run.
One-to-one with the central `import_runs` row (which owns the rollback
cascade); carries the finance-specific metadata and rich stats of §2.4.
"""
__tablename__ = "fin_import_runs"
__table_args__ = (
Index("ix_fin_import_runs_user_started", "user_id", "started_at"),
)
id: Mapped[uuid.UUID] = mapped_column(Uuid, primary_key=True, default=uuid.uuid4)
user_id: Mapped[int] = mapped_column(
ForeignKey("users.id", ondelete="CASCADE"), index=True
)
import_run_id: Mapped[int] = mapped_column(
ForeignKey("import_runs.id", ondelete="CASCADE"), unique=True, index=True
)
source_profile_id: Mapped[uuid.UUID | None] = mapped_column(
ForeignKey("fin_source_profiles.id", ondelete="SET NULL"), default=None
)
account_id: Mapped[uuid.UUID] = mapped_column(
ForeignKey("fin_accounts.id", ondelete="CASCADE")
)
filename: Mapped[str] = mapped_column(String(255))
file_sha256: Mapped[str] = mapped_column(String(64))
status: Mapped[ImportStatus] = mapped_column(
_enum(ImportStatus, "fin_import_status"), default=ImportStatus.PENDING
)
started_at: Mapped[datetime] = mapped_column(DateTime(timezone=True))
finished_at: Mapped[datetime | None] = mapped_column(
DateTime(timezone=True), default=None
)
stats: Mapped[dict[str, Any]] = mapped_column(JSONB_V, default=dict)
error_message: Mapped[str | None] = mapped_column(Text, default=None)
class FinTransaction(TimestampMixin, Base):
__tablename__ = "fin_transactions"
__table_args__ = (
UniqueConstraint("account_id", "dedup_hash", name="uq_fin_transactions_dedup"),
Index(
"uq_fin_transactions_external",
"account_id",
"external_id",
unique=True,
sqlite_where=text("external_id IS NOT NULL"),
postgresql_where=text("external_id IS NOT NULL"),
),
Index("ix_fin_transactions_user_date", "user_id", "booked_date"),
Index("ix_fin_transactions_account_date", "account_id", "booked_date"),
)
id: Mapped[uuid.UUID] = mapped_column(Uuid, primary_key=True, default=uuid.uuid4)
user_id: Mapped[int] = mapped_column(ForeignKey("users.id", ondelete="CASCADE"))
account_id: Mapped[uuid.UUID] = mapped_column(
ForeignKey("fin_accounts.id", ondelete="CASCADE")
)
booked_date: Mapped[date] = mapped_column(Date) # civil date, never tz-shifted
value_date: Mapped[date | None] = mapped_column(Date, default=None)
amount: Mapped[Decimal] = mapped_column(Numeric(12, 2)) # signed: debit < 0
currency: Mapped[str] = mapped_column(String(3), default="EUR")
label_raw: Mapped[str] = mapped_column(Text)
label_clean: Mapped[str] = mapped_column(Text)
counterparty: Mapped[str | None] = mapped_column(String(150), default=None)
category_id: Mapped[uuid.UUID | None] = mapped_column(
ForeignKey("fin_categories.id", ondelete="SET NULL"), index=True, default=None
)
category_source: Mapped[CategorySource | None] = mapped_column(
_enum(CategorySource, "fin_category_source"), default=None
)
# Logical FK to fin_rules.id (no constraint: rules are freely deletable).
applied_rule_id: Mapped[uuid.UUID | None] = mapped_column(Uuid, default=None)
notes: Mapped[str | None] = mapped_column(Text, default=None)
# Central import run (int id); NULL = manual entry. CASCADE = C8.5 rollback.
import_run_id: Mapped[int | None] = mapped_column(
ForeignKey("import_runs.id", ondelete="CASCADE"), index=True, default=None
)
external_id: Mapped[str | None] = mapped_column(String(255), default=None)
dedup_hash: Mapped[str] = mapped_column(String(64))
transfer_group_id: Mapped[uuid.UUID | None] = mapped_column(
Uuid, index=True, default=None
)
class FinRule(TimestampMixin, Base):
__tablename__ = "fin_rules"
__table_args__ = (
Index("ix_fin_rules_user_priority", "user_id", "priority", "created_at"),
)
id: Mapped[uuid.UUID] = mapped_column(Uuid, primary_key=True, default=uuid.uuid4)
user_id: Mapped[int] = mapped_column(ForeignKey("users.id", ondelete="CASCADE"))
name: Mapped[str] = mapped_column(String(100))
priority: Mapped[int] = mapped_column(default=100) # ascending = first
enabled: Mapped[bool] = mapped_column(default=True)
stop: Mapped[bool] = mapped_column(default=True)
matchers: Mapped[dict[str, Any]] = mapped_column(JSONB_V)
actions: Mapped[dict[str, Any]] = mapped_column(JSONB_V)
hit_count: Mapped[int] = mapped_column(default=0)
last_applied_at: Mapped[datetime | None] = mapped_column(
DateTime(timezone=True), default=None
)
class FinBudget(TimestampMixin, Base):
__tablename__ = "fin_budgets"
__table_args__ = (
CheckConstraint("monthly_amount > 0", name="ck_fin_budgets_positive"),
Index("ix_fin_budgets_user_cat", "user_id", "category_id"),
)
id: Mapped[uuid.UUID] = mapped_column(Uuid, primary_key=True, default=uuid.uuid4)
user_id: Mapped[int] = mapped_column(ForeignKey("users.id", ondelete="CASCADE"))
category_id: Mapped[uuid.UUID] = mapped_column(
ForeignKey("fin_categories.id", ondelete="CASCADE")
)
monthly_amount: Mapped[Decimal] = mapped_column(Numeric(12, 2))
start_month: Mapped[date] = mapped_column(Date) # always the 1st of the month
end_month: Mapped[date | None] = mapped_column(Date, default=None)
# --------------------------------------------------------------------------
# pg_trgm — trigram GIN index of datamodel-finance §2.5 (PostgreSQL only).
#
# `GET /api/finance/transactions?q=` searches with ILIKE '%…%' on label_clean;
# without a trigram index PostgreSQL can only answer with a sequential scan.
# The index cannot be declared as a plain `Index(...)`: `gin_trgm_ops` does not
# exist on SQLite (the test engine) and does not exist on a PostgreSQL where the
# extension was never installed.
#
# So it is emitted as an `after_create` DDL hook restricted to the PostgreSQL
# dialect, written so that it can never break `create_all` (§9.4 has no
# migration step in v1, the API creates its own schema at startup):
# - `CREATE EXTENSION` is attempted in a nested block: pg_trgm is a *trusted*
# extension since PostgreSQL 13, so the database owner (the compose
# POSTGRES_USER) can install it; if the deployment forbids it the exception
# is swallowed instead of aborting the whole transaction;
# - the index itself is only created when the extension really is present.
# `docker/postgres-init/10-extensions.sql` installs it at initdb time as well,
# for deployments where the API user is not the database owner.
# --------------------------------------------------------------------------
_TRGM_DDL = DDL(
"""
DO $$
BEGIN
BEGIN
CREATE EXTENSION IF NOT EXISTS pg_trgm;
EXCEPTION WHEN OTHERS THEN
RAISE NOTICE 'pg_trgm indisponible : recherche de libelles non indexee';
END;
IF EXISTS (SELECT 1 FROM pg_extension WHERE extname = 'pg_trgm') THEN
CREATE INDEX IF NOT EXISTS ix_fin_transactions_label_trgm
ON fin_transactions USING gin (label_clean gin_trgm_ops);
END IF;
END
$$;
"""
)
event.listen(
FinTransaction.__table__,
"after_create",
_TRGM_DDL.execute_if(dialect="postgresql"),
)
+139
View File
@@ -0,0 +1,139 @@
"""Label/amount normalisation and dedup hashing (datamodel-finance.md §4.3, §7.1)."""
import hashlib
import re
import unicodedata
from datetime import date, timedelta
from decimal import ROUND_HALF_UP, Decimal, InvalidOperation
from uuid import UUID
TWO_PLACES = Decimal("0.01")
# Purely technical bank prefixes stripped from label_clean (§4.3).
_TECH_PREFIX_RE = re.compile(
r"^(CARTE \d{2}/\d{2}(/\d{2,4})? |PAIEMENT (PSC |CB )?\d{4} |PRLV SEPA "
r"|VIR SEPA |VIR INST |ACHAT CB )",
re.IGNORECASE,
)
# Spaces found in French bank exports: NBSP, narrow NBSP, thin space.
_SPACES = " "
def normalize_label_for_hash(label: str) -> str:
"""STABLE and CONSERVATIVE normalisation used by the dedup hash.
Uppercase + accents stripped + whitespace collapsed, nothing else — any
more aggressive cleaning would merge distinct transactions.
"""
s = unicodedata.normalize("NFKD", label)
s = "".join(c for c in s if not unicodedata.combining(c))
s = s.upper()
return re.sub(r"\s+", " ", s).strip()
def light_clean(label_raw: str) -> str:
"""Initial label_clean: trim, collapse whitespace, drop technical prefixes."""
s = re.sub(r"\s+", " ", label_raw).strip()
s = _TECH_PREFIX_RE.sub("", s).strip()
return s or label_raw.strip()
def compute_dedup_hash(
account_id: UUID,
booked_date: date,
amount: Decimal,
label_raw: str,
occurrence: int,
) -> str:
canonical = "|".join(
[
str(account_id),
booked_date.isoformat(),
f"{amount:.2f}", # sign included
normalize_label_for_hash(label_raw),
str(occurrence), # rank among identical rows of the SAME file
]
)
return hashlib.sha256(canonical.encode("utf-8")).hexdigest()
def occurrence_key(booked_date: date, amount: Decimal, label_raw: str) -> tuple:
return (booked_date, f"{amount:.2f}", normalize_label_for_hash(label_raw))
def quantize2(value: Decimal) -> Decimal:
return value.quantize(TWO_PLACES, rounding=ROUND_HALF_UP)
def parse_amount(raw: str, decimal_separator: str = ",") -> Decimal:
"""Parse a French/English bank amount string into a signed Decimal.
Covers: "-6,4", "+3500,00", "-123 456,78", "1.234,56", "12,50 €",
"12,50" (U+2212), "226.68".
"""
s = raw.strip()
if not s:
raise ValueError("montant vide")
s = s.replace("", "-") # unicode minus
for sp in _SPACES:
s = s.replace(sp, "")
s = s.replace(" ", "")
s = re.sub(r"(?i)(€|EUR)", "", s).strip()
if decimal_separator == ",":
# Remove dot/space thousands separators, then turn the comma into a dot.
if "," in s:
s = s.replace(".", "").replace(",", ".")
# else: already dot-decimal (BoursoBank mixes both in one file)
else:
s = s.replace(",", "")
try:
return Decimal(s)
except InvalidOperation as exc:
raise ValueError(f"montant illisible : {raw!r}") from exc
_MERCHANT_NOISE_WORDS = re.compile(
r"\b(CB|CARTE|PRLV|SEPA|VIR|ECH|PAIEMENT|ACHAT|WEB|FACT(URE)?)\b"
)
def merchant_key(label_raw: str, counterparty: str | None = None) -> str:
"""Aggressive grouping key for recurring/merchant detection (§7.1)."""
if counterparty:
return normalize_label_for_hash(counterparty)[:60]
s = normalize_label_for_hash(label_raw)
s = re.sub(r"\b\d{2}/\d{2}(/\d{2,4})?\b", "", s) # dates inside the label
s = re.sub(r"\b\d{4,}\b", "", s) # card/reference numbers
s = _MERCHANT_NOISE_WORDS.sub("", s)
s = re.sub(r"[^A-Z0-9 ]", " ", s)
s = re.sub(r"\s+", " ", s).strip()
return s[:60]
def month_str(d: date) -> str:
return f"{d.year:04d}-{d.month:02d}"
def month_first_day(year: int, month: int) -> date:
return date(year, month, 1)
def add_months(d: date, months: int) -> date:
"""First day of the month `months` after (or before) d's month."""
total = d.year * 12 + (d.month - 1) + months
return date(total // 12, total % 12 + 1, 1)
def month_last_day(d: date) -> date:
return add_months(d, 1) - timedelta(days=1)
def parse_month(value: str) -> date:
m = re.fullmatch(r"(\d{4})-(\d{2})", value.strip())
if not m:
raise ValueError(f"mois invalide : {value!r}")
year, month = int(m.group(1)), int(m.group(2))
if not 1 <= month <= 12:
raise ValueError(f"mois invalide : {value!r}")
return date(year, month, 1)
+642
View File
@@ -0,0 +1,642 @@
"""File parsers for the finance import pipeline (datamodel-finance.md §3-4).
Three parsers (CSV generic driven by a source-profile config, OFX, PayPal CSV)
all produce the same `ParseResult` of `NormalizedRow`s. Row-level problems are
collected (never fatal); a structurally unreadable file raises
`ImporterParseError` with a French message.
"""
import csv
import io
import re
import unicodedata
from dataclasses import dataclass, field
from datetime import date, datetime
from decimal import Decimal
from app.core.importing.base import ImporterParseError
from app.modules.finance.normalize import parse_amount, quantize2
MAX_ERRORS = 50
_GENERIC_DATE_FORMATS = (
"%d/%m/%Y",
"%Y-%m-%d",
"%d.%m.%Y",
"%Y-%m-%d %H:%M:%S",
"%d/%m/%y",
)
@dataclass
class NormalizedRow:
booked_date: date
value_date: date | None
amount: Decimal # signed, quantized to 2 decimals
currency: str # ISO 4217 upper
label_raw: str # trimmed, newlines collapsed to spaces
counterparty: str | None = None # PayPal / "counterparty" column only
external_id: str | None = None
source_row_index: int = 0 # 1-based row index for error messages
@dataclass
class ParseResult:
rows: list[NormalizedRow] = field(default_factory=list)
errors: list[dict] = field(default_factory=list) # [{"row": int, "message": str}]
rows_filtered: int = 0 # rows intentionally skipped by the parser
# Every data row seen, blank lines and footers excluded. §2.4 requires
# rows_total == imported + skipped_duplicate + skipped_filtered + error, so
# the intentionally filtered rows are part of it.
rows_total: int = 0
errors_truncated: bool = False
@property
def rows_error(self) -> int:
return self.rows_total - self.rows_filtered - len(self.rows)
def add_error(self, row_index: int, message: str) -> None:
if len(self.errors) < MAX_ERRORS:
self.errors.append({"row": row_index, "message": message})
else:
self.errors_truncated = True
# ---------------------------------------------------------------------------
# Decoding
# ---------------------------------------------------------------------------
def decode_bytes(raw: bytes, encoding_cfg: str = "auto") -> str:
"""§4.1 + research §7.3: BOM first, then explicit encoding, then auto
(utf-8 strict -> charset-normalizer -> cp1252, which never fails)."""
if raw.startswith(b"\xef\xbb\xbf"):
return raw.decode("utf-8-sig")
if encoding_cfg and encoding_cfg != "auto":
return raw.decode(encoding_cfg, errors="replace")
try:
return raw.decode("utf-8")
except UnicodeDecodeError:
pass
try:
from charset_normalizer import from_bytes
best = from_bytes(raw).best()
if best is not None and best.encoding not in ("utf_8", "utf-8"):
return str(best)
except Exception: # noqa: BLE001, S110 — auto mode must never fail
pass
return raw.decode("cp1252")
def _norm_header(name: str) -> str:
"""Case/accent-insensitive header comparison key."""
s = unicodedata.normalize("NFKD", name)
s = "".join(c for c in s if not unicodedata.combining(c))
return re.sub(r"\s+", " ", s).strip().strip('"').casefold()
def _clean_cell(value: str) -> str:
return re.sub(r"\s+", " ", value).strip()
# ---------------------------------------------------------------------------
# Generic CSV parser (kind = 'csv')
# ---------------------------------------------------------------------------
def _sniff_delimiter(sample: str) -> str:
try:
return csv.Sniffer().sniff(sample, delimiters=";,\t|").delimiter
except csv.Error:
first = sample.splitlines()[0] if sample.splitlines() else ""
counts = {d: first.count(d) for d in (";", ",", "\t", "|")}
best = max(counts, key=lambda d: counts[d])
return best if counts[best] > 0 else ";"
def _resolve_column(spec: object, header_map: dict[str, int] | None) -> int | None:
"""Column spec: header name (str), 0-based index (int) or list of aliases."""
if spec is None or spec == "":
return None
if isinstance(spec, bool):
return None
if isinstance(spec, int):
return spec
if isinstance(spec, str):
if header_map is None:
return None
return header_map.get(_norm_header(spec))
if isinstance(spec, (list, tuple)):
for alias in spec:
idx = _resolve_column(alias, header_map)
if idx is not None:
return idx
return None
def _parse_date(value: str, config: dict) -> date:
v = value.strip().strip('"')
formats: list[str] = []
if config.get("date_format"):
formats.append(config["date_format"])
formats.extend(config.get("date_formats") or [])
formats.extend(f for f in _GENERIC_DATE_FORMATS if f not in formats)
for fmt in formats:
try:
# Civil date from a bank statement: never timezone-converted (§1).
return datetime.strptime(v, fmt).date() # noqa: DTZ007
except ValueError:
continue
raise ValueError(f"Date invalide : '{value.strip()}'")
def _cell(row: list[str], idx: int | None) -> str:
if idx is None or idx >= len(row):
return ""
return row[idx]
def parse_csv(config: dict, text: str, default_currency: str = "EUR") -> ParseResult:
result = ParseResult()
lines = text.splitlines(keepends=True)
# Preamble handling: fixed number of rows and/or "skip until header" marker.
start = int(config.get("skip_rows_top") or 0)
marker = config.get("skip_until_header_startswith")
if marker:
want = _norm_header(marker)
found = None
for i, line in enumerate(lines):
if _norm_header(line)[: len(want)] == want:
found = i
break
if found is None:
raise ImporterParseError(
"Ligne d'en-tête introuvable — vérifiez le profil de source."
)
start = found
if start >= len(lines) and lines:
raise ImporterParseError("Le fichier ne contient aucune ligne de données.")
body = "".join(lines[start:])
delimiter = config.get("delimiter") or "auto"
if delimiter == "auto":
delimiter = _sniff_delimiter("".join(lines[start : start + 6]))
quote_char = config.get("quote_char") or '"'
reader = csv.reader(io.StringIO(body), delimiter=delimiter, quotechar=quote_char)
try:
raw_rows = list(reader)
except csv.Error as exc:
raise ImporterParseError(
"Fichier CSV illisible — vérifiez le séparateur et les guillemets."
) from exc
skip_bottom = int(config.get("skip_rows_bottom") or 0)
if skip_bottom:
raw_rows = raw_rows[:-skip_bottom] if skip_bottom < len(raw_rows) else []
has_header = config.get("has_header", True)
header_map: dict[str, int] | None = None
data_start = 0
if has_header:
if not raw_rows:
raise ImporterParseError("Le fichier est vide.")
header_map = {
_norm_header(cell): i for i, cell in enumerate(raw_rows[0]) if cell.strip()
}
data_start = 1
columns = config.get("columns") or {}
col_booked = _resolve_column(columns.get("booked_date"), header_map)
col_value = _resolve_column(columns.get("value_date"), header_map)
col_label = _resolve_column(columns.get("label"), header_map)
col_amount = _resolve_column(columns.get("amount"), header_map)
col_debit = _resolve_column(columns.get("debit"), header_map)
col_credit = _resolve_column(columns.get("credit"), header_map)
col_currency = _resolve_column(columns.get("currency"), header_map)
col_external = _resolve_column(columns.get("external_id"), header_map)
col_counterparty = _resolve_column(columns.get("counterparty"), header_map)
col_fee = _resolve_column(columns.get("fee"), header_map)
amount_mode = config.get("amount_mode", "signed")
if amount_mode == "auto":
amount_mode = "signed" if col_amount is not None else "split"
if col_booked is None or col_label is None:
raise ImporterParseError(
"Colonnes obligatoires introuvables (date, libellé) — "
"vérifiez le profil de source."
)
if amount_mode == "signed" and col_amount is None:
raise ImporterParseError(
"Colonne de montant introuvable — vérifiez le profil de source."
)
if amount_mode == "split" and col_debit is None and col_credit is None:
raise ImporterParseError(
"Colonnes Débit/Crédit introuvables — vérifiez le profil de source."
)
decimal_sep = config.get("decimal_separator", ",")
invert = bool(config.get("invert_sign", False))
label_join = config.get("label_join") or []
join_idx = [
idx
for idx in (_resolve_column(c, header_map) for c in label_join)
if idx is not None
]
filters = config.get("skip_row_if") or []
filter_specs = []
for f in filters:
fidx = _resolve_column(f.get("column"), header_map)
if fidx is not None:
filter_specs.append((fidx, f))
stop_on_non_date = bool(config.get("stop_on_non_date_row", False))
for offset, row in enumerate(raw_rows[data_start:]):
row_index = start + data_start + offset + 1 # 1-based, file-relative
if not any(cell.strip() for cell in row):
continue
booked_raw = _cell(row, col_booked)
if stop_on_non_date and booked_raw.strip():
try:
_parse_date(booked_raw, config)
except ValueError:
break # footer reached (Crédit Agricole)
result.rows_total += 1
skip = False
for fidx, f in filter_specs:
cell_val = _clean_cell(_cell(row, fidx)).casefold()
if "equals" in f and cell_val == str(f["equals"]).casefold():
skip = True
if "not_equals" in f and cell_val != str(f["not_equals"]).casefold():
skip = True
if skip:
result.rows_filtered += 1
continue
try:
booked = _parse_date(booked_raw, config)
value_raw = _cell(row, col_value).strip()
value_d = _parse_date(value_raw, config) if value_raw else None
if amount_mode == "signed":
amount = parse_amount(_cell(row, col_amount), decimal_sep)
else:
debit_raw = _cell(row, col_debit).strip()
credit_raw = _cell(row, col_credit).strip()
if bool(debit_raw) == bool(credit_raw):
raise ValueError(
"exactement une des colonnes Débit/Crédit doit être remplie"
)
if credit_raw:
amount = abs(parse_amount(credit_raw, decimal_sep))
else:
amount = -abs(parse_amount(debit_raw, decimal_sep))
if col_fee is not None:
fee_raw = _cell(row, col_fee).strip()
if fee_raw:
amount -= parse_amount(fee_raw, decimal_sep)
if invert:
amount = -amount
label = _clean_cell(_cell(row, col_label))
for jidx in join_idx:
extra = _clean_cell(_cell(row, jidx))
if extra:
label = f"{label}{extra}" if label else extra
if not label:
raise ValueError("libellé vide")
currency = _clean_cell(_cell(row, col_currency)).upper() or (
default_currency
)
external = _clean_cell(_cell(row, col_external)) or None
counterparty = _clean_cell(_cell(row, col_counterparty)) or None
result.rows.append(
NormalizedRow(
booked_date=booked,
value_date=value_d,
amount=quantize2(amount),
currency=currency[:3],
label_raw=label,
counterparty=counterparty,
external_id=external,
source_row_index=row_index,
)
)
except ValueError as exc:
result.add_error(row_index, str(exc).capitalize())
return result
# ---------------------------------------------------------------------------
# OFX parser (kind = 'ofx')
# ---------------------------------------------------------------------------
_OFX_FIELD_RES = {
name: re.compile(rf"<{name}>([^<\r\n]*)", re.IGNORECASE)
for name in ("DTPOSTED", "TRNAMT", "FITID", "NAME", "MEMO")
}
_OFX_CURDEF_RE = re.compile(r"<CURDEF>([^<\r\n]*)", re.IGNORECASE)
_OFX_ACCTID_RE = re.compile(r"<ACCTID>([^<\r\n]*)", re.IGNORECASE)
def _ofx_decode(raw: bytes, fallback_encoding: str) -> str:
head = raw[:512].decode("latin-1", errors="replace")
m = re.search(r"CHARSET:\s*([A-Za-z0-9-]+)", head)
if m:
charset = m.group(1).strip().lower()
if charset in ("1252", "cp1252", "windows-1252"):
return raw.decode("cp1252", errors="replace")
if charset not in ("none",):
try:
return raw.decode(charset, errors="replace")
except LookupError:
pass
m = re.search(r'encoding="([^"]+)"', head, re.IGNORECASE)
if m:
try:
return raw.decode(m.group(1), errors="replace")
except LookupError:
pass
if re.search(r"ENCODING:\s*UTF-8", head, re.IGNORECASE):
return raw.decode("utf-8", errors="replace")
return raw.decode(fallback_encoding or "cp1252", errors="replace")
def _ofx_date(value: str) -> date:
digits = re.sub(r"[^0-9]", "", value)[:8]
if len(digits) < 8:
raise ValueError(f"Date invalide : '{value.strip()}'")
# DTPOSTED: keep the 8 leading digits, no timezone conversion (§3.2).
return datetime.strptime(digits, "%Y%m%d").date() # noqa: DTZ007
def _ofx_label(name: str, memo: str) -> str:
name, memo = _clean_cell(name), _clean_cell(memo)
if name and memo:
return f"{name}{memo}"
return name or memo
def _parse_ofx_with_ofxtools(
raw: bytes, config: dict, default_currency: str
) -> ParseResult | None:
try:
from ofxtools.Parser import OFXTree
tree = OFXTree()
tree.parse(io.BytesIO(raw))
ofx = tree.convert()
statements = list(getattr(ofx, "statements", []) or [])
if not statements:
return None
except Exception: # noqa: BLE001 — malformed French SGML: use the regex fallback
return None
wanted = config.get("account_match")
stmt = statements[0]
if wanted:
for st in statements:
acctid = getattr(getattr(st, "account", None), "acctid", None)
if acctid and str(acctid).strip() == str(wanted).strip():
stmt = st
break
result = ParseResult()
currency = (getattr(stmt, "curdef", None) or default_currency or "EUR").upper()
transactions = list(getattr(stmt, "transactions", None) or [])
for i, trn in enumerate(transactions, start=1):
result.rows_total += 1
try:
dtposted = getattr(trn, "dtposted", None)
if dtposted is None:
raise ValueError("Date absente (DTPOSTED)")
booked = dtposted.date() if hasattr(dtposted, "date") else dtposted
amount = getattr(trn, "trnamt", None)
if amount is None:
raise ValueError("Montant absent (TRNAMT)")
label = _ofx_label(
str(getattr(trn, "name", "") or ""), str(getattr(trn, "memo", "") or "")
)
if not label:
raise ValueError("Libellé vide")
fitid = str(getattr(trn, "fitid", "") or "").strip() or None
result.rows.append(
NormalizedRow(
booked_date=booked,
value_date=None,
amount=quantize2(Decimal(str(amount))),
currency=currency,
label_raw=label,
external_id=fitid,
source_row_index=i,
)
)
except ValueError as exc:
result.add_error(i, str(exc))
return result
def _parse_ofx_with_regex(
text: str, config: dict, default_currency: str
) -> ParseResult:
result = ParseResult()
m = _OFX_CURDEF_RE.search(text)
currency = (m.group(1).strip() if m else default_currency or "EUR").upper()
wanted = config.get("account_match")
scope = text
if wanted:
# Keep the statement block whose ACCTID matches, when identifiable.
parts = re.split(r"(?i)(?=<STMTRS>|<CCSTMTRS>)", text)
for part in parts:
am = _OFX_ACCTID_RE.search(part)
if am and am.group(1).strip() == str(wanted).strip():
scope = part
break
blocks = re.split(r"(?i)<STMTTRN>", scope)[1:]
if not blocks:
raise ImporterParseError("Aucune transaction OFX trouvée dans le fichier.")
for i, block in enumerate(blocks, start=1):
block = re.split(r"(?i)</STMTTRN>", block)[0]
result.rows_total += 1
fields = {}
for name, rx in _OFX_FIELD_RES.items():
fm = rx.search(block)
fields[name] = fm.group(1).strip() if fm else ""
try:
if not fields["DTPOSTED"]:
raise ValueError("Date absente (DTPOSTED)")
booked = _ofx_date(fields["DTPOSTED"])
if not fields["TRNAMT"]:
raise ValueError("Montant absent (TRNAMT)")
amount = parse_amount(fields["TRNAMT"], decimal_separator=".")
label = _ofx_label(fields["NAME"], fields["MEMO"])
if not label:
raise ValueError("Libellé vide")
result.rows.append(
NormalizedRow(
booked_date=booked,
value_date=None,
amount=quantize2(amount),
currency=currency,
label_raw=label,
external_id=fields["FITID"] or None,
source_row_index=i,
)
)
except ValueError as exc:
result.add_error(i, str(exc))
return result
def parse_ofx(config: dict, raw: bytes, default_currency: str = "EUR") -> ParseResult:
result = _parse_ofx_with_ofxtools(raw, config, default_currency)
if result is None:
text = _ofx_decode(raw, config.get("fallback_encoding", "cp1252"))
result = _parse_ofx_with_regex(text, config, default_currency)
_dedupe_fitids(result)
return result
def _dedupe_fitids(result: ParseResult) -> None:
"""LCL-style FITID collisions inside one file: suffix an occurrence counter
so the partial unique index (account_id, external_id) stays truthful."""
seen: dict[str, int] = {}
for row in result.rows:
if not row.external_id:
continue
n = seen.get(row.external_id, 0)
seen[row.external_id] = n + 1
if n:
row.external_id = f"{row.external_id}#{n}"
# ---------------------------------------------------------------------------
# PayPal activity CSV parser (kind = 'paypal_csv')
# ---------------------------------------------------------------------------
_PAYPAL_ALIASES = {
"date": ("date",),
"name": ("nom", "name"),
"type": ("type",),
"status": ("etat", "status", "statut", "state"),
"currency": ("devise", "currency"),
"gross": ("brut", "gross"),
"fee": ("frais", "fee"),
"net": ("net",),
"amount": ("montant", "amount"),
"transaction_id": ("numero de transaction", "transaction id"),
"item_title": ("titre de l'objet", "item title", "objet", "subject"),
"balance_impact": ("impact sur le solde", "balance impact"),
}
_PAYPAL_COMPLETED = {"effectue", "completed"}
_PAYPAL_CONVERSION_TYPES = {
"conversion de devise generale",
"general currency conversion",
}
def parse_paypal_csv(
config: dict, text: str, default_currency: str = "EUR"
) -> ParseResult:
reader = csv.reader(
io.StringIO(text), delimiter=config.get("delimiter", ","), quotechar='"'
)
try:
raw_rows = list(reader)
except csv.Error as exc:
raise ImporterParseError("Fichier CSV PayPal illisible.") from exc
if not raw_rows:
raise ImporterParseError("Le fichier est vide.")
header_map = {_norm_header(c): i for i, c in enumerate(raw_rows[0]) if c.strip()}
cols: dict[str, int | None] = {}
for key, aliases in _PAYPAL_ALIASES.items():
cols[key] = next((header_map[a] for a in aliases if a in header_map), None)
if cols["date"] is None or cols["transaction_id"] is None:
raise ImporterParseError(
"En-têtes PayPal introuvables (Date, Numéro de transaction) — "
"est-ce bien un rapport d'activité PayPal ?"
)
use_net = bool(config.get("use_net_amount", True))
amount_col = (
cols["net"]
if use_net and cols["net"] is not None
else cols["gross"]
if cols["gross"] is not None
else cols["amount"]
)
if amount_col is None:
raise ImporterParseError("Colonne de montant PayPal introuvable (Net/Brut).")
skip_types = {_norm_header(t) for t in (config.get("skip_types") or [])}
if config.get("conversion_as_skip", True):
skip_types |= _PAYPAL_CONVERSION_TYPES
only_completed = bool(config.get("skip_status_not_completed", True))
target_currency = (default_currency or "EUR").upper()
result = ParseResult()
for offset, row in enumerate(raw_rows[1:]):
row_index = offset + 2
if not any(cell.strip() for cell in row):
continue
result.rows_total += 1
typ = _norm_header(_cell(row, cols["type"]))
status = _norm_header(_cell(row, cols["status"]))
impact = _norm_header(_cell(row, cols["balance_impact"]))
currency = _clean_cell(_cell(row, cols["currency"])).upper()
if (
(skip_types and typ in skip_types)
or (
only_completed
and cols["status"] is not None
and status not in _PAYPAL_COMPLETED
)
or (cols["balance_impact"] is not None and impact == "memo")
or (currency and currency != target_currency)
):
result.rows_filtered += 1
continue
try:
booked = _parse_date(_cell(row, cols["date"]), config)
decimal_sep = config.get("decimal_separator", ",")
if decimal_sep == "auto":
decimal_sep = ","
amount = parse_amount(_cell(row, amount_col), decimal_sep)
name = _clean_cell(_cell(row, cols["name"]))
type_label = _clean_cell(_cell(row, cols["type"]))
title = _clean_cell(_cell(row, cols["item_title"]))
label = "".join(p for p in (name, type_label, title) if p)
if not label:
raise ValueError("libellé vide")
result.rows.append(
NormalizedRow(
booked_date=booked,
value_date=None,
amount=quantize2(amount),
currency=currency or target_currency,
label_raw=label,
counterparty=name or None,
external_id=_clean_cell(_cell(row, cols["transaction_id"])) or None,
source_row_index=row_index,
)
)
except ValueError as exc:
result.add_error(row_index, str(exc).capitalize())
return result
+456
View File
@@ -0,0 +1,456 @@
"""Import pipeline: decode -> parse -> normalize -> dedup -> rules -> insert.
Reference: datamodel-finance.md §4. The whole run is one SQL transaction
(atomicity, §4.4) and is idempotent thanks to the occurrence-aware dedup hash.
Every imported row references the central `import_runs` row (FK ON DELETE
CASCADE), which is what makes `DELETE /api/imports/{id}` a real rollback.
"""
import hashlib
import uuid
from collections import defaultdict
from dataclasses import dataclass, field
from datetime import date, timedelta
from decimal import Decimal
from typing import Any
from sqlalchemy import select
from sqlalchemy.orm import Session
from app.core.errors import DomainValidationError
from app.core.importing.base import ImporterParseError
from app.core.timeutils import utcnow
from app.modules.finance.categorize import (
apply_rules_to_tx,
detect_transfers,
get_transfer_category_id,
load_enabled_rules,
)
from app.modules.finance.enums import ImportStatus, SourceKind
from app.modules.finance.models import (
FinAccount,
FinImportRun,
FinSourceProfile,
FinTransaction,
)
from app.modules.finance.normalize import (
compute_dedup_hash,
light_clean,
occurrence_key,
)
from app.modules.finance.parsers import (
NormalizedRow,
ParseResult,
decode_bytes,
parse_csv,
parse_ofx,
parse_paypal_csv,
)
from app.modules.imports.models import ImportRun
TRANSFER_WINDOW_DAYS = 3
MAX_STATS_ERRORS = 50
# Importer ids registered in importers.py, per source kind.
IMPORTER_ID_BY_KIND: dict[str, str] = {
SourceKind.CSV: "bank_generic_csv",
SourceKind.OFX: "bank_ofx",
SourceKind.PAYPAL_CSV: "paypal_csv",
}
def file_sha256(raw: bytes) -> str:
return hashlib.sha256(raw).hexdigest()
def parse_file(
kind: str, config: dict[str, Any], raw: bytes, default_currency: str = "EUR"
) -> ParseResult:
"""Dispatch to the parser of the profile kind (§4.2)."""
if kind == SourceKind.OFX:
return parse_ofx(config or {}, raw, default_currency)
text = decode_bytes(raw, (config or {}).get("encoding", "auto"))
if kind == SourceKind.PAYPAL_CSV:
return parse_paypal_csv(config or {}, text, default_currency)
return parse_csv(config or {}, text, default_currency)
def prepare_rows(
account_id: uuid.UUID, rows: list[NormalizedRow]
) -> list[tuple[NormalizedRow, str]]:
"""Number identical tuples (occurrence) then hash each row (§4.3)."""
counter: dict[tuple, int] = defaultdict(int)
prepared: list[tuple[NormalizedRow, str]] = []
for row in rows:
key = occurrence_key(row.booked_date, row.amount, row.label_raw)
occurrence = counter[key]
counter[key] += 1
prepared.append(
(
row,
compute_dedup_hash(
account_id, row.booked_date, row.amount, row.label_raw, occurrence
),
)
)
return prepared
def _existing_keys(
db: Session,
account_id: uuid.UUID,
date_min: date | None,
date_max: date | None,
) -> tuple[set[str], set[str]]:
"""Set-based duplicate lookup: one query for hashes, one for external ids."""
hash_stmt = select(FinTransaction.dedup_hash).where(
FinTransaction.account_id == account_id
)
if date_min is not None and date_max is not None:
hash_stmt = hash_stmt.where(
FinTransaction.booked_date >= date_min,
FinTransaction.booked_date <= date_max,
)
hashes = set(db.scalars(hash_stmt).all())
ext_stmt = select(FinTransaction.external_id).where(
FinTransaction.account_id == account_id,
FinTransaction.external_id.is_not(None),
)
externals = {e for e in db.scalars(ext_stmt).all() if e}
return hashes, externals
def _date_bounds(rows: list[NormalizedRow]) -> tuple[date | None, date | None]:
if not rows:
return None, None
dates = [row.booked_date for row in rows]
return min(dates), max(dates)
def build_transaction(
user_id: int,
account: FinAccount,
row: NormalizedRow,
dedup_hash: str,
import_run_id: int | None,
) -> FinTransaction:
return FinTransaction(
id=uuid.uuid4(),
user_id=user_id,
account_id=account.id,
booked_date=row.booked_date,
value_date=row.value_date,
amount=row.amount,
currency=(row.currency or account.currency or "EUR").upper()[:3],
label_raw=row.label_raw,
label_clean=light_clean(row.label_raw),
counterparty=row.counterparty,
external_id=row.external_id,
dedup_hash=dedup_hash,
import_run_id=import_run_id,
)
@dataclass
class PreviewResult:
rows_preview: list[NormalizedRow] = field(default_factory=list)
rows_total: int = 0
rows_error: int = 0
rows_skipped_filtered: int = 0
would_skip_duplicates: int = 0
date_min: date | None = None
date_max: date | None = None
errors: list[dict[str, Any]] = field(default_factory=list)
duplicate_file_of: int | None = None
def preview_import(
db: Session,
user_id: int,
account: FinAccount,
profile: FinSourceProfile,
raw: bytes,
preview_size: int = 20,
) -> PreviewResult:
"""Parse without writing anything (§9.6 `POST /imports/preview`)."""
if not raw:
raise DomainValidationError("Le fichier envoyé est vide.")
try:
parsed = parse_file(profile.kind, profile.config or {}, raw, account.currency)
except ImporterParseError as exc:
raise DomainValidationError(str(exc)) from exc
prepared = prepare_rows(account.id, parsed.rows)
date_min, date_max = _date_bounds(parsed.rows)
hashes, externals = _existing_keys(db, account.id, date_min, date_max)
duplicates = 0
seen: set[str] = set()
for row, dedup_hash in prepared:
if (row.external_id and row.external_id in externals) or dedup_hash in hashes:
duplicates += 1
continue
if dedup_hash in seen:
duplicates += 1
continue
seen.add(dedup_hash)
previous = db.scalars(
select(FinImportRun.import_run_id)
.where(
FinImportRun.user_id == user_id,
FinImportRun.file_sha256 == file_sha256(raw),
FinImportRun.status == ImportStatus.COMPLETED,
)
.limit(1)
).first()
return PreviewResult(
rows_preview=parsed.rows[:preview_size],
rows_total=parsed.rows_total,
rows_error=parsed.rows_error,
rows_skipped_filtered=parsed.rows_filtered,
would_skip_duplicates=duplicates,
date_min=date_min,
date_max=date_max,
errors=parsed.errors,
duplicate_file_of=previous,
)
def _failed_run(
db: Session,
user_id: int,
account: FinAccount,
profile: FinSourceProfile,
filename: str,
raw: bytes,
message: str,
) -> FinImportRun:
"""Persist a failed run after a rollback (no transaction inserted)."""
db.rollback()
started = utcnow()
central = ImportRun(
user_id=user_id,
importer_id=IMPORTER_ID_BY_KIND.get(profile.kind, "bank_generic_csv"),
domain="finance",
filename=filename,
file_size=len(raw),
status="failed",
error_details=[{"row": 0, "message": message}],
started_at=started,
finished_at=utcnow(),
)
db.add(central)
db.flush()
run = FinImportRun(
id=uuid.uuid4(),
user_id=user_id,
import_run_id=central.id,
source_profile_id=profile.id,
account_id=account.id,
filename=filename,
file_sha256=file_sha256(raw),
status=ImportStatus.FAILED,
started_at=started,
finished_at=utcnow(),
stats={},
error_message=message,
)
db.add(run)
db.commit()
return run
def run_import(
db: Session,
user_id: int,
account: FinAccount,
profile: FinSourceProfile,
filename: str,
raw: bytes,
) -> FinImportRun:
"""Execute one synchronous import run (§4.4)."""
if not raw:
raise DomainValidationError("Le fichier envoyé est vide.")
started = utcnow()
checksum = file_sha256(raw)
duplicate_of = db.scalars(
select(FinImportRun.import_run_id)
.where(
FinImportRun.user_id == user_id,
FinImportRun.file_sha256 == checksum,
FinImportRun.status == ImportStatus.COMPLETED,
)
.limit(1)
).first()
central = ImportRun(
user_id=user_id,
importer_id=IMPORTER_ID_BY_KIND.get(profile.kind, "bank_generic_csv"),
domain="finance",
filename=filename,
file_size=len(raw),
status="completed",
error_details=[],
started_at=started,
)
db.add(central)
db.flush()
run = FinImportRun(
id=uuid.uuid4(),
user_id=user_id,
import_run_id=central.id,
source_profile_id=profile.id,
account_id=account.id,
filename=filename,
file_sha256=checksum,
status=ImportStatus.RUNNING,
started_at=started,
stats={},
)
db.add(run)
try:
parsed = parse_file(profile.kind, profile.config or {}, raw, account.currency)
except ImporterParseError as exc:
return _failed_run(db, user_id, account, profile, filename, raw, str(exc))
except Exception: # noqa: BLE001 — a broken file must not 500 the API
return _failed_run(
db,
user_id,
account,
profile,
filename,
raw,
"Erreur inattendue pendant la lecture du fichier.",
)
# §4.2: the run only fails when EVERY data row errored. A file whose rows
# were all filtered out on purpose (PayPal authorisations, non-EUR lines…)
# is a legitimate empty import, not a failure.
if parsed.rows_total and parsed.rows_error == parsed.rows_total:
return _failed_run(
db,
user_id,
account,
profile,
filename,
raw,
"Aucune ligne exploitable — vérifiez le profil de source.",
)
prepared = prepare_rows(account.id, parsed.rows)
date_min, date_max = _date_bounds(parsed.rows)
hashes, externals = _existing_keys(db, account.id, date_min, date_max)
rules = load_enabled_rules(db, user_id)
transfer_cat = get_transfer_category_id(db, user_id)
to_insert: list[FinTransaction] = []
skipped_duplicate = 0
rules_applied = 0
for row, dedup_hash in prepared:
if (row.external_id and row.external_id in externals) or dedup_hash in hashes:
skipped_duplicate += 1
continue
hashes.add(dedup_hash)
if row.external_id:
externals.add(row.external_id)
tx = build_transaction(user_id, account, row, dedup_hash, central.id)
if apply_rules_to_tx(tx, rules, transfer_cat):
rules_applied += 1
to_insert.append(tx)
# Atomicity (§4.4): the whole run is one SQL transaction; anything going
# wrong rolls every row back and the run is stored as failed.
transfers = 0
try:
db.add_all(to_insert)
db.flush()
if date_min is not None and date_max is not None and to_insert:
transfers = detect_transfers(
db,
user_id,
date_min - timedelta(days=TRANSFER_WINDOW_DAYS),
date_max + timedelta(days=TRANSFER_WINDOW_DAYS),
)
except Exception: # noqa: BLE001 — never leave a half-written run behind
return _failed_run(
db,
user_id,
account,
profile,
filename,
raw,
"L'enregistrement des transactions a échoué : import annulé.",
)
stats: dict[str, Any] = {
"rows_total": parsed.rows_total,
"rows_imported": len(to_insert),
"rows_skipped_duplicate": skipped_duplicate,
"rows_skipped_filtered": parsed.rows_filtered,
"rows_error": parsed.rows_error,
"rules_applied": rules_applied,
"transfers_detected": transfers,
"date_min": date_min.isoformat() if date_min else None,
"date_max": date_max.isoformat() if date_max else None,
"errors": parsed.errors[:MAX_STATS_ERRORS],
}
if parsed.errors_truncated or len(parsed.errors) > MAX_STATS_ERRORS:
stats["errors_truncated"] = True
if duplicate_of is not None:
stats["duplicate_file_of"] = duplicate_of
run.status = ImportStatus.COMPLETED
run.finished_at = utcnow()
run.stats = stats
central.rows_total = parsed.rows_total
central.rows_inserted = len(to_insert)
central.rows_duplicates = skipped_duplicate
central.rows_errors = parsed.rows_error
central.error_details = parsed.errors[:MAX_STATS_ERRORS]
central.finished_at = run.finished_at
try:
db.commit()
except Exception: # noqa: BLE001 — never leave a half-written run behind
return _failed_run(
db,
user_id,
account,
profile,
filename,
raw,
"L'enregistrement des transactions a échoué : import annulé.",
)
return run
def transaction_dedup_hash(
db: Session,
account_id: uuid.UUID,
booked_date: date,
amount: Decimal,
label_raw: str,
) -> str:
"""Hash for a manually entered transaction: occurrence = count of twins."""
occurrence = 0
stmt = select(FinTransaction.dedup_hash).where(
FinTransaction.account_id == account_id,
FinTransaction.booked_date == booked_date,
FinTransaction.amount == amount,
)
existing = set(db.scalars(stmt).all())
candidate = compute_dedup_hash(
account_id, booked_date, amount, label_raw, occurrence
)
while candidate in existing:
occurrence += 1
candidate = compute_dedup_hash(
account_id, booked_date, amount, label_raw, occurrence
)
return candidate
+439
View File
@@ -0,0 +1,439 @@
"""Built-in seed data: source profiles (§3.4) and the default category tree (§2.2).
Both are lazy-seeded on first access to the module (see `service.ensure_seed`)
because v1 has no Alembic data migration (DESIGN §9).
"""
from copy import deepcopy
from typing import Any
from app.modules.finance.enums import CategoryKind, SourceKind
# Chart palette (ux-pages.md §7.5), fixed slot order, colour-blind validated.
PALETTE = (
"#3987E5",
"#D95926",
"#199E70",
"#C98500",
"#D55181",
"#008300",
"#9085E9",
"#E66767",
)
MUTED_COLOR = "#898781"
UNCATEGORIZED_COLOR = "#9ca3af"
UNCATEGORIZED_LABEL = "Non catégorisé"
TRANSFER_CATEGORY_NAME = "Virements internes"
SAVINGS_NODE_NAME = "Épargne du mois"
DEFICIT_NODE_NAME = "Découvert / réserves"
INCOME_NODE_NAME = "Revenus"
# --------------------------------------------------------------------------
# Source profiles — presets shipped with the app (datamodel §3.4 + research §2)
# --------------------------------------------------------------------------
# Generic CSV defaults (§3.1). Every preset below is this dict updated.
GENERIC_CSV_CONFIG: dict[str, Any] = {
"encoding": "auto",
"delimiter": "auto",
"quote_char": '"',
"decimal_separator": ",",
"thousands_separator": " ",
"date_format": None,
"date_formats": ["%d/%m/%Y", "%Y-%m-%d", "%d.%m.%Y", "%Y-%m-%d %H:%M:%S"],
"has_header": True,
"skip_rows_top": 0,
"skip_rows_bottom": 0,
"skip_until_header_startswith": None,
"stop_on_non_date_row": False,
"columns": {
"booked_date": [
"dateOp",
"Date",
"Date de comptabilisation",
"Date opération",
"Date de l'opération",
"date_comptabilisation",
"Booking Date",
"Completed Date",
],
"value_date": ["dateVal", "Date valeur", "Date de valeur", "Value Date"],
"label": [
"label",
"Libellé",
"Libelle operation",
"libellé_complet_operation",
"Détail de l'écriture",
"Description",
"Partner Name",
],
"amount": [
"amount",
"Montant",
"Montant(EUROS)",
"Montant de l'opération",
"montant_operation",
"Amount",
"Amount (EUR)",
],
"debit": ["Débit", "Débit euros", "Debit", "Débit Euros"],
"credit": ["Crédit", "Crédit euros", "Credit", "Crédit Euros"],
"currency": ["Devise", "devise", "Currency"],
"external_id": None,
"counterparty": None,
"fee": None,
},
"amount_mode": "auto",
"invert_sign": False,
"label_join": [],
"skip_row_if": [],
}
def _csv_config(**overrides: Any) -> dict[str, Any]:
config = deepcopy(GENERIC_CSV_CONFIG)
columns = config["columns"]
columns.update(overrides.pop("columns", {}))
config.update(overrides)
config["columns"] = columns
return config
BUILTIN_PROFILES: tuple[dict[str, Any], ...] = (
{
"slug": "generic",
"name": "CSV générique",
"kind": SourceKind.CSV,
"config": _csv_config(),
},
{
"slug": "boursobank",
"name": "BoursoBank / Boursorama (CSV)",
"kind": SourceKind.CSV,
"config": _csv_config(
encoding="utf-8-sig",
delimiter=";",
date_format="%Y-%m-%d",
amount_mode="signed",
label_join=["supplierFound"],
columns={
"booked_date": "dateOp",
"value_date": "dateVal",
"label": "label",
"amount": "amount",
"debit": None,
"credit": None,
},
),
},
{
"slug": "credit-agricole",
"name": "Crédit Agricole (CSV)",
"kind": SourceKind.CSV,
# Variable-length preamble: locate the header line instead of skipping
# a fixed number of rows; a footer may follow the data (research §2.2).
"config": _csv_config(
encoding="cp1252",
delimiter=";",
date_format="%d/%m/%Y",
amount_mode="split",
skip_until_header_startswith="Date;",
stop_on_non_date_row=True,
columns={
"booked_date": "Date",
"value_date": "Date valeur",
"label": "Libellé",
"amount": None,
"debit": ["Débit euros", "Débit Euros", "Debit euros"],
"credit": ["Crédit euros", "Crédit Euros", "Credit euros"],
},
),
},
{
"slug": "bnp-paribas",
"name": "BNP Paribas (CSV)",
"kind": SourceKind.CSV,
# First line = balance metadata, no column header: positional mapping.
"config": _csv_config(
encoding="cp1252",
delimiter=";",
date_format="%d/%m/%Y",
has_header=False,
skip_rows_top=1,
amount_mode="signed",
columns={
"booked_date": 0,
"value_date": None,
"label": 3,
"amount": 4,
"debit": None,
"credit": None,
"currency": None,
},
),
},
{
"slug": "societe-generale",
"name": "Société Générale (CSV)",
"kind": SourceKind.CSV,
"config": _csv_config(
encoding="cp1252",
delimiter=";",
date_format="%d/%m/%Y",
skip_rows_top=1,
amount_mode="signed",
columns={
"booked_date": ["date_comptabilisation", "Date de l'opération", "Date"],
"value_date": None,
"label": [
"libellé_complet_operation",
"Libellé",
"Détail de l'écriture",
],
"amount": [
"montant_operation",
"Montant de l'opération",
"Montant",
],
"debit": None,
"credit": None,
"currency": ["devise", "Devise"],
},
),
},
{
"slug": "banque-postale",
"name": "La Banque Postale (CSV)",
"kind": SourceKind.CSV,
"config": _csv_config(
encoding="cp1252",
delimiter=";",
date_format="%d/%m/%Y",
skip_until_header_startswith="Date;",
amount_mode="signed",
columns={
"booked_date": "Date",
"value_date": None,
"label": "Libellé",
"amount": ["Montant(EUROS)", "Montant (EUROS)", "Montant"],
"debit": None,
"credit": None,
},
),
},
{
"slug": "caisse-epargne",
"name": "Caisse d'Épargne (CSV)",
"kind": SourceKind.CSV,
# 2024+ layout: debit already negative, credit prefixed with '+'.
"config": _csv_config(
encoding="cp1252",
delimiter=";",
date_format="%d/%m/%Y",
amount_mode="split",
columns={
"booked_date": ["Date de comptabilisation", "Date operation"],
"value_date": "Date de valeur",
"label": ["Libelle operation", "Libellé opération"],
"amount": None,
"debit": "Debit",
"credit": "Credit",
"counterparty": ["Libelle simplifie", "Libellé simplifié"],
},
),
},
{
"slug": "fortuneo",
"name": "Fortuneo (CSV)",
"kind": SourceKind.CSV,
"config": _csv_config(
encoding="auto",
delimiter=";",
date_format="%d/%m/%Y",
amount_mode="split",
columns={
"booked_date": ["Date opération", "Date operation"],
"value_date": "Date valeur",
"label": ["libellé", "Libellé"],
"amount": None,
"debit": "Débit",
"credit": "Crédit",
},
),
},
{
"slug": "revolut",
"name": "Revolut (CSV)",
"kind": SourceKind.CSV,
# Amount is gross: Fee is subtracted to get the real balance impact.
"config": _csv_config(
encoding="utf-8",
delimiter=",",
decimal_separator=".",
thousands_separator="",
date_format="%Y-%m-%d %H:%M:%S",
amount_mode="signed",
skip_row_if=[{"column": "State", "not_equals": "COMPLETED"}],
columns={
"booked_date": ["Completed Date", "Started Date"],
"value_date": None,
"label": "Description",
"amount": "Amount",
"debit": None,
"credit": None,
"currency": "Currency",
"fee": "Fee",
},
),
},
{
"slug": "n26",
"name": "N26 (CSV)",
"kind": SourceKind.CSV,
"config": _csv_config(
encoding="utf-8",
delimiter=",",
decimal_separator=".",
thousands_separator="",
date_format="%Y-%m-%d",
amount_mode="signed",
# Column names are resolved case/accent-insensitively.
label_join=["Payment Reference"],
columns={
"booked_date": ["Booking Date", "Date"],
"value_date": "Value Date",
"label": ["Partner Name", "Payee"],
"amount": ["Amount (EUR)", "Amount(EUR)"],
"debit": None,
"credit": None,
"counterparty": ["Partner Name", "Payee"],
},
),
},
{
"slug": "ofx",
"name": "OFX (toutes banques)",
"kind": SourceKind.OFX,
"config": {"fallback_encoding": "cp1252", "account_match": None},
},
{
"slug": "paypal",
"name": "PayPal — rapport d'activité (CSV)",
"kind": SourceKind.PAYPAL_CSV,
"config": {
"encoding": "utf-8-sig",
"delimiter": ",",
"decimal_separator": "auto",
"date_format": "%d/%m/%Y",
"date_formats": ["%d/%m/%Y", "%m/%d/%Y", "%Y-%m-%d"],
"use_net_amount": True,
"skip_types": [
"Autorisation",
"Authorization",
"Commande",
"Order",
"Annulation d'autorisation",
"Void of Authorization",
"Retenue pour vérification par PayPal",
"Payment Review Hold",
"Annulation de la retenue",
"Payment Review Release",
],
"skip_status_not_completed": True,
"conversion_as_skip": True,
},
},
)
# --------------------------------------------------------------------------
# Default category tree (§2.2). (name, icon, children)
# --------------------------------------------------------------------------
EXPENSE_TREE: tuple[tuple[str, str, tuple[str, ...]], ...] = (
("Alimentation", "shopping-cart", ("Courses", "Restaurants & bars", "Livraison")),
(
"Logement",
"home",
(
"Loyer / Crédit",
"Énergie",
"Eau",
"Internet & mobile",
"Assurance habitation",
"Entretien",
),
),
(
"Transports",
"car",
(
"Carburant",
"Péages & parking",
"Transports en commun",
"Entretien véhicule",
"Assurance auto",
),
),
("Santé", "heart-pulse", ("Pharmacie", "Médecin", "Mutuelle")),
(
"Loisirs",
"gamepad-2",
("Abonnements & streaming", "Jeux vidéo", "Sorties", "Sport", "Vacances"),
),
("Shopping", "shirt", ("Vêtements", "High-tech", "Maison")),
("Vape & tabac", "cigarette", ()),
("Banque & frais", "landmark", ("Frais bancaires", "Intérêts")),
("Impôts & taxes", "receipt", ()),
("Enfants & famille", "baby", ()),
("Animaux", "paw-print", ()),
("Dons & cadeaux", "gift", ()),
("Autres dépenses", "circle-ellipsis", ()),
)
INCOME_TREE: tuple[tuple[str, str, tuple[str, ...]], ...] = (
("Salaire", "wallet", ()),
("Aides & prestations", "hand-coins", ()),
("Remboursements", "undo-2", ("Santé", "Autres")),
("Ventes", "tag", ()),
("Intérêts & placements", "trending-up", ()),
("Autres revenus", "circle-plus", ()),
)
TRANSFER_TREE: tuple[tuple[str, str, tuple[str, ...]], ...] = (
(TRANSFER_CATEGORY_NAME, "arrow-left-right", ()),
)
def default_category_tree() -> list[dict[str, Any]]:
"""Flat description of the seed tree, roots first, in display order."""
out: list[dict[str, Any]] = []
order = 0
palette_index = 0
for kind, tree in (
(CategoryKind.INCOME, INCOME_TREE),
(CategoryKind.EXPENSE, EXPENSE_TREE),
(CategoryKind.TRANSFER, TRANSFER_TREE),
):
for name, icon, children in tree:
color = (
MUTED_COLOR
if kind is CategoryKind.TRANSFER
else PALETTE[palette_index % len(PALETTE)]
)
palette_index += 1
out.append(
{
"name": name,
"icon": icon,
"color": color,
"kind": kind,
"sort_order": order,
"is_system": kind is CategoryKind.TRANSFER,
"children": [
{"name": child, "sort_order": i}
for i, child in enumerate(children)
],
}
)
order += 1
return out
+653
View File
@@ -0,0 +1,653 @@
"""HTTP contract of the finance module (datamodel-finance.md §9).
Mounted automatically under `/api/finance` by the module loader; every endpoint
depends on the JWT user and filters by `user.id` (CONVENTIONS C2.7).
"""
import uuid
from datetime import date
from decimal import Decimal
from typing import Annotated, Any, Literal
from fastapi import APIRouter, Depends, File, Form, Query, UploadFile
from sqlalchemy.orm import Session
from app.core.config import get_settings
from app.core.database import get_db
from app.core.dependencies import get_current_user
from app.core.errors import PayloadTooLargeError
from app.core.pagination import Page, PageParams, paginate
from app.core.timeutils import resolve_tz, utcnow
from app.modules.auth.models import User
from app.modules.finance import categorize, pipeline, service
from app.modules.finance import stats as stats_service
from app.modules.finance.schemas import (
AccountCreate,
AccountRead,
AccountUpdate,
BudgetCreate,
BudgetProgressResponse,
BudgetRead,
BudgetUpdate,
BulkCategorizeRequest,
BulkCategorizeResponse,
CashflowResponse,
CategoryCreate,
CategoryRead,
CategoryUpdate,
DashboardResponse,
ImportPreviewResponse,
ImportRunRead,
MonthlyByCategoryResponse,
RecurringResponse,
RuleApplyRequest,
RuleApplyResponse,
RuleCreate,
RulePreviewRequest,
RulePreviewResponse,
RuleRead,
RuleReorderRequest,
RuleUpdate,
SankeyResponse,
SourceProfileCreate,
SourceProfileRead,
SourceProfileUpdate,
TopMerchantsResponse,
TransactionCreate,
TransactionRead,
TransactionUpdate,
TransferDetectRequest,
TransferDetectResponse,
TransferLinkRequest,
TransferLinkResponse,
)
router = APIRouter(prefix="/finance", tags=["finance"])
DbDep = Annotated[Session, Depends(get_db)]
def seeded_user(db: DbDep, user: Annotated[User, Depends(get_current_user)]) -> User:
"""Lazy seed of the built-in profiles and of the user's category tree."""
service.ensure_seed(db, user.id)
return user
UserDep = Annotated[User, Depends(seeded_user)]
AccountFilter = Annotated[list[uuid.UUID] | None, Query(alias="account_id")]
def _today() -> date:
"""Local civil day (Europe/Paris by default) used by month-based stats."""
return utcnow().astimezone(resolve_tz(get_settings().timezone)).date()
# ---------------------------------------------------------------------------
# Accounts
# ---------------------------------------------------------------------------
@router.get("/accounts", response_model=list[AccountRead])
def list_accounts(
db: DbDep,
user: UserDep,
include_archived: Annotated[bool, Query()] = False,
) -> list[AccountRead]:
return [
AccountRead.model_validate(item)
for item in service.list_accounts(db, user.id, include_archived)
]
@router.post("/accounts", response_model=AccountRead, status_code=201)
def create_account(payload: AccountCreate, db: DbDep, user: UserDep) -> AccountRead:
account = service.create_account(db, user.id, payload)
return AccountRead.model_validate(service.account_detail(db, user.id, account.id))
@router.patch("/accounts/{account_id}", response_model=AccountRead)
def update_account(
account_id: uuid.UUID, payload: AccountUpdate, db: DbDep, user: UserDep
) -> AccountRead:
service.update_account(db, user.id, account_id, payload)
return AccountRead.model_validate(service.account_detail(db, user.id, account_id))
@router.delete("/accounts/{account_id}", status_code=204)
def delete_account(account_id: uuid.UUID, db: DbDep, user: UserDep) -> None:
service.delete_account(db, user.id, account_id)
# ---------------------------------------------------------------------------
# Categories
# ---------------------------------------------------------------------------
@router.get("/categories", response_model=list[CategoryRead])
def list_categories(db: DbDep, user: UserDep) -> list[CategoryRead]:
return [
CategoryRead.model_validate(node) for node in service.category_tree(db, user.id)
]
@router.post("/categories", response_model=CategoryRead, status_code=201)
def create_category(payload: CategoryCreate, db: DbDep, user: UserDep) -> CategoryRead:
return CategoryRead.model_validate(service.create_category(db, user.id, payload))
@router.patch("/categories/{category_id}", response_model=CategoryRead)
def update_category(
category_id: uuid.UUID, payload: CategoryUpdate, db: DbDep, user: UserDep
) -> CategoryRead:
return CategoryRead.model_validate(
service.update_category(db, user.id, category_id, payload)
)
@router.delete("/categories/{category_id}", status_code=204)
def delete_category(category_id: uuid.UUID, db: DbDep, user: UserDep) -> None:
service.delete_category(db, user.id, category_id)
# ---------------------------------------------------------------------------
# Source profiles
# ---------------------------------------------------------------------------
@router.get("/source-profiles", response_model=list[SourceProfileRead])
def list_source_profiles(db: DbDep, user: UserDep) -> list[SourceProfileRead]:
return [
SourceProfileRead.model_validate(profile)
for profile in service.list_source_profiles(db, user.id)
]
@router.post("/source-profiles", response_model=SourceProfileRead, status_code=201)
def create_source_profile(
payload: SourceProfileCreate, db: DbDep, user: UserDep
) -> SourceProfileRead:
return SourceProfileRead.model_validate(
service.create_source_profile(db, user.id, payload)
)
@router.post(
"/source-profiles/{profile_id}/clone",
response_model=SourceProfileRead,
status_code=201,
)
def clone_source_profile(
profile_id: uuid.UUID, db: DbDep, user: UserDep
) -> SourceProfileRead:
return SourceProfileRead.model_validate(
service.clone_source_profile(db, user.id, profile_id)
)
@router.patch("/source-profiles/{profile_id}", response_model=SourceProfileRead)
def update_source_profile(
profile_id: uuid.UUID, payload: SourceProfileUpdate, db: DbDep, user: UserDep
) -> SourceProfileRead:
return SourceProfileRead.model_validate(
service.update_source_profile(db, user.id, profile_id, payload)
)
@router.delete("/source-profiles/{profile_id}", status_code=204)
def delete_source_profile(profile_id: uuid.UUID, db: DbDep, user: UserDep) -> None:
service.delete_source_profile(db, user.id, profile_id)
# ---------------------------------------------------------------------------
# Transactions
# ---------------------------------------------------------------------------
@router.get("/transactions", response_model=Page[TransactionRead])
def list_transactions(
db: DbDep,
user: UserDep,
params: Annotated[PageParams, Depends()],
date_from: Annotated[date | None, Query()] = None,
date_to: Annotated[date | None, Query()] = None,
account_id: AccountFilter = None,
category_id: Annotated[list[str] | None, Query()] = None,
q: Annotated[str | None, Query()] = None,
direction: Annotated[Literal["debit", "credit"] | None, Query()] = None,
amount_min: Annotated[Decimal | None, Query()] = None,
amount_max: Annotated[Decimal | None, Query()] = None,
is_transfer: Annotated[bool | None, Query()] = None,
import_run_id: Annotated[int | None, Query()] = None,
sort: Annotated[str | None, Query()] = None,
) -> Page[TransactionRead]:
stmt = service.transactions_query(
db,
user.id,
date_from=date_from,
date_to=date_to,
account_ids=account_id,
category_ids=category_id,
q=q,
direction=direction,
amount_min=amount_min,
amount_max=amount_max,
is_transfer=is_transfer,
import_run_id=import_run_id,
sort=sort,
)
items, total = paginate(db, stmt, params)
return Page(
items=[
TransactionRead.model_validate(row)
for row in service.serialize_transactions(db, user.id, items)
],
total=total,
page=params.page,
page_size=params.page_size,
)
@router.post("/transactions", response_model=TransactionRead, status_code=201)
def create_transaction(
payload: TransactionCreate, db: DbDep, user: UserDep
) -> TransactionRead:
tx = service.create_transaction(db, user.id, payload)
return TransactionRead.model_validate(
service.serialize_transactions(db, user.id, [tx])[0]
)
@router.patch("/transactions/{transaction_id}", response_model=TransactionRead)
def update_transaction(
transaction_id: uuid.UUID, payload: TransactionUpdate, db: DbDep, user: UserDep
) -> TransactionRead:
tx = service.update_transaction(db, user.id, transaction_id, payload)
return TransactionRead.model_validate(
service.serialize_transactions(db, user.id, [tx])[0]
)
@router.delete("/transactions/{transaction_id}", status_code=204)
def delete_transaction(transaction_id: uuid.UUID, db: DbDep, user: UserDep) -> None:
service.delete_transaction(db, user.id, transaction_id)
@router.post("/transactions/bulk-categorize", response_model=BulkCategorizeResponse)
def bulk_categorize(
payload: BulkCategorizeRequest, db: DbDep, user: UserDep
) -> BulkCategorizeResponse:
updated = service.bulk_categorize(
db, user.id, payload.transaction_ids, payload.category_id
)
return BulkCategorizeResponse(updated=updated)
# ---------------------------------------------------------------------------
# Rules
# ---------------------------------------------------------------------------
@router.get("/rules", response_model=list[RuleRead])
def list_rules(db: DbDep, user: UserDep) -> list[RuleRead]:
return [RuleRead.model_validate(rule) for rule in service.list_rules(db, user.id)]
@router.post("/rules", response_model=RuleRead, status_code=201)
def create_rule(payload: RuleCreate, db: DbDep, user: UserDep) -> RuleRead:
return RuleRead.model_validate(service.create_rule(db, user.id, payload))
@router.patch("/rules/{rule_id}", response_model=RuleRead)
def update_rule(
rule_id: uuid.UUID, payload: RuleUpdate, db: DbDep, user: UserDep
) -> RuleRead:
return RuleRead.model_validate(service.update_rule(db, user.id, rule_id, payload))
@router.delete("/rules/{rule_id}", status_code=204)
def delete_rule(rule_id: uuid.UUID, db: DbDep, user: UserDep) -> None:
service.delete_rule(db, user.id, rule_id)
@router.post("/rules/reorder", response_model=list[RuleRead])
def reorder_rules(
payload: RuleReorderRequest, db: DbDep, user: UserDep
) -> list[RuleRead]:
service.reorder_rules(db, user.id, payload.ordered_ids)
return [RuleRead.model_validate(rule) for rule in service.list_rules(db, user.id)]
@router.post("/rules/apply", response_model=RuleApplyResponse)
def apply_rules(
payload: RuleApplyRequest, db: DbDep, user: UserDep
) -> RuleApplyResponse:
result = categorize.apply_rules(
db,
user.id,
scope=payload.scope,
date_from=payload.date_from,
date_to=payload.date_to,
account_id=payload.account_id,
rule_id=payload.rule_id,
dry_run=payload.dry_run,
force=payload.force,
)
return RuleApplyResponse.model_validate(
{
"scanned": result.scanned,
"matched": result.matched,
"updated": result.updated,
"dry_run": result.dry_run,
"by_rule": result.by_rule,
}
)
@router.post("/rules/preview", response_model=RulePreviewResponse)
def preview_rule(
payload: RulePreviewRequest, db: DbDep, user: UserDep
) -> RulePreviewResponse:
matches, total = categorize.preview_rule(db, user.id, payload.matchers)
return RulePreviewResponse(
items=[
TransactionRead.model_validate(row)
for row in service.serialize_transactions(db, user.id, matches)
],
total_matched=total,
)
# ---------------------------------------------------------------------------
# Budgets
# ---------------------------------------------------------------------------
@router.get("/budgets", response_model=list[BudgetRead])
def list_budgets(
db: DbDep,
user: UserDep,
month: Annotated[str | None, Query()] = None,
) -> list[BudgetRead]:
target = service.parse_month_param(month, _today())
return [
BudgetRead.model_validate(item)
for item in service.list_budgets(db, user.id, target)
]
@router.post("/budgets", response_model=BudgetRead, status_code=201)
def create_budget(payload: BudgetCreate, db: DbDep, user: UserDep) -> BudgetRead:
budget = service.create_budget(db, user.id, payload)
return BudgetRead.model_validate(budget)
@router.patch("/budgets/{budget_id}", response_model=list[BudgetRead])
def update_budget(
budget_id: uuid.UUID, payload: BudgetUpdate, db: DbDep, user: UserDep
) -> list[BudgetRead]:
budgets = service.update_budget(db, user.id, budget_id, payload)
return [BudgetRead.model_validate(budget) for budget in budgets]
@router.delete("/budgets/{budget_id}", status_code=204)
def delete_budget(budget_id: uuid.UUID, db: DbDep, user: UserDep) -> None:
service.delete_budget(db, user.id, budget_id)
# ---------------------------------------------------------------------------
# Imports
# ---------------------------------------------------------------------------
def _read_upload(file: UploadFile) -> bytes:
data = file.file.read()
if len(data) > get_settings().max_upload_bytes:
raise PayloadTooLargeError("Fichier trop volumineux (limite : 20 Mio).")
return data
@router.post("/imports/preview", response_model=ImportPreviewResponse)
def preview_import(
db: DbDep,
user: UserDep,
file: Annotated[UploadFile, File()],
account_id: Annotated[uuid.UUID, Form()],
source_profile_id: Annotated[uuid.UUID | None, Form()] = None,
) -> ImportPreviewResponse:
account = service.get_account(db, user.id, account_id)
profile = service.resolve_profile(db, user.id, source_profile_id)
result = pipeline.preview_import(db, user.id, account, profile, _read_upload(file))
return ImportPreviewResponse.model_validate(
{
"rows_preview": result.rows_preview,
"rows_total": result.rows_total,
"rows_error": result.rows_error,
"rows_skipped_filtered": result.rows_skipped_filtered,
"would_skip_duplicates": result.would_skip_duplicates,
"date_min": result.date_min,
"date_max": result.date_max,
"errors": result.errors,
"duplicate_file_of": result.duplicate_file_of,
}
)
@router.post("/imports", response_model=ImportRunRead, status_code=201)
def run_import(
db: DbDep,
user: UserDep,
file: Annotated[UploadFile, File()],
account_id: Annotated[uuid.UUID, Form()],
source_profile_id: Annotated[uuid.UUID | None, Form()] = None,
) -> ImportRunRead:
account = service.get_account(db, user.id, account_id)
profile = service.resolve_profile(db, user.id, source_profile_id)
run = pipeline.run_import(
db,
user.id,
account,
profile,
file.filename or "import.csv",
_read_upload(file),
)
return ImportRunRead.model_validate(run)
@router.get("/imports", response_model=Page[ImportRunRead])
def list_imports(
db: DbDep,
user: UserDep,
params: Annotated[PageParams, Depends()],
) -> Page[ImportRunRead]:
items, total = paginate(db, service.import_runs_query(user.id), params)
return Page(
items=[ImportRunRead.model_validate(run) for run in items],
total=total,
page=params.page,
page_size=params.page_size,
)
@router.get("/imports/{run_id}", response_model=ImportRunRead)
def get_import(run_id: uuid.UUID, db: DbDep, user: UserDep) -> ImportRunRead:
return ImportRunRead.model_validate(service.get_import_run(db, user.id, run_id))
@router.delete("/imports/{run_id}", status_code=204)
def rollback_import(
run_id: uuid.UUID,
db: DbDep,
user: UserDep,
force: Annotated[bool, Query()] = False,
) -> None:
service.rollback_import(db, user.id, run_id, force=force)
# ---------------------------------------------------------------------------
# Transfers
# ---------------------------------------------------------------------------
@router.post("/transfers/detect", response_model=TransferDetectResponse)
def detect_transfers(
db: DbDep,
user: UserDep,
payload: TransferDetectRequest | None = None,
) -> TransferDetectResponse:
payload = payload or TransferDetectRequest()
created = categorize.detect_transfers(
db, user.id, payload.date_from, payload.date_to
)
db.commit()
return TransferDetectResponse(pairs_created=created)
@router.post("/transfers/link", response_model=TransferLinkResponse)
def link_transfer(
payload: TransferLinkRequest, db: DbDep, user: UserDep
) -> TransferLinkResponse:
group_id = categorize.link_transfer(
db, user.id, payload.transaction_id_a, payload.transaction_id_b
)
return TransferLinkResponse(transfer_group_id=group_id)
@router.delete("/transfers/{transfer_group_id}", status_code=204)
def unlink_transfer(transfer_group_id: uuid.UUID, db: DbDep, user: UserDep) -> None:
categorize.unlink_transfer(db, user.id, transfer_group_id)
# ---------------------------------------------------------------------------
# Stats
# ---------------------------------------------------------------------------
@router.get("/stats/monthly-by-category", response_model=MonthlyByCategoryResponse)
def stats_monthly_by_category(
db: DbDep,
user: UserDep,
months: Annotated[int, Query(ge=1, le=60)] = 12,
level: Annotated[Literal["root", "child"], Query()] = "root",
direction: Annotated[Literal["debit", "credit"], Query()] = "debit",
account_id: AccountFilter = None,
) -> MonthlyByCategoryResponse:
return MonthlyByCategoryResponse.model_validate(
stats_service.monthly_by_category(
db,
user.id,
months=months,
level=level,
direction=direction,
account_ids=account_id,
today=_today(),
)
)
@router.get("/stats/cashflow", response_model=CashflowResponse)
def stats_cashflow(
db: DbDep,
user: UserDep,
months: Annotated[int, Query(ge=1, le=60)] = 12,
account_id: AccountFilter = None,
) -> CashflowResponse:
return CashflowResponse.model_validate(
stats_service.cashflow(
db, user.id, months=months, account_ids=account_id, today=_today()
)
)
@router.get("/stats/top-merchants", response_model=TopMerchantsResponse)
def stats_top_merchants(
db: DbDep,
user: UserDep,
months: Annotated[int, Query(ge=1, le=60)] = 3,
limit: Annotated[int, Query(ge=1, le=100)] = 15,
direction: Annotated[Literal["debit", "credit"], Query()] = "debit",
account_id: AccountFilter = None,
) -> TopMerchantsResponse:
data = stats_service.top_merchants(
db,
user.id,
months=months,
limit=limit,
direction=direction,
account_ids=account_id,
today=_today(),
)
return TopMerchantsResponse.model_validate(
{"period": _period(data["period"]), "items": data["items"]}
)
@router.get("/stats/recurring", response_model=RecurringResponse)
def stats_recurring(
db: DbDep,
user: UserDep,
direction: Annotated[Literal["debit", "credit"], Query()] = "debit",
include_inactive: Annotated[bool, Query()] = False,
) -> RecurringResponse:
return RecurringResponse.model_validate(
stats_service.recurring(
db,
user.id,
direction=direction,
include_inactive=include_inactive,
today=_today(),
)
)
@router.get("/stats/budget-progress", response_model=BudgetProgressResponse)
def stats_budget_progress(
db: DbDep,
user: UserDep,
month: Annotated[str | None, Query()] = None,
account_id: AccountFilter = None,
) -> BudgetProgressResponse:
today = _today()
target = service.parse_month_param(month, today)
return BudgetProgressResponse.model_validate(
stats_service.budget_progress(
db, user.id, target, account_ids=account_id, today=today
)
)
@router.get("/stats/sankey", response_model=SankeyResponse)
def stats_sankey(
db: DbDep,
user: UserDep,
month: Annotated[str | None, Query()] = None,
months: Annotated[int, Query(ge=1, le=60)] = 1,
account_id: AccountFilter = None,
) -> SankeyResponse:
today = _today()
target = service.parse_month_param(month, today) if month else None
data = stats_service.sankey(
db,
user.id,
month=target,
months=months,
account_ids=account_id,
today=today,
)
return SankeyResponse.model_validate(
{
"period": _period(data["period"]),
"nodes": data["nodes"],
"links": data["links"],
}
)
@router.get("/dashboard", response_model=DashboardResponse)
def dashboard(db: DbDep, user: UserDep) -> DashboardResponse:
return DashboardResponse.model_validate(
stats_service.dashboard(db, user.id, today=_today())
)
def _period(period: dict[str, Any]) -> dict[str, Any]:
return {"from": period["from"], "to": period["to"]}
+501
View File
@@ -0,0 +1,501 @@
"""Pydantic v2 schemas of the finance module (datamodel-finance.md §9).
Amounts travel as JSON numbers with 2 decimals (`Money`), dates as
`YYYY-MM-DD`, months as `YYYY-MM`.
"""
import uuid
from datetime import date
from decimal import Decimal
from typing import Annotated, Any, Literal
from pydantic import BaseModel, ConfigDict, Field, PlainSerializer, field_validator
from app.core.timeutils import StoredUtcDatetime
from app.modules.finance.enums import (
AccountKind,
CategoryKind,
CategorySource,
ImportStatus,
SourceKind,
)
Money = Annotated[
Decimal,
PlainSerializer(
lambda v: float(round(Decimal(v), 2)), return_type=float, when_used="json"
),
]
MAX_BULK_IDS = 500
class ORMModel(BaseModel):
model_config = ConfigDict(from_attributes=True)
# ---------------------------------------------------------------------------
# Accounts
# ---------------------------------------------------------------------------
class AccountCreate(BaseModel):
name: str = Field(min_length=1, max_length=100)
kind: AccountKind = AccountKind.CHECKING
currency: str = Field(default="EUR", min_length=3, max_length=3)
institution: str | None = Field(default=None, max_length=100)
iban_masked: str | None = Field(default=None, max_length=34)
initial_balance: Money = Decimal("0.00")
class AccountUpdate(BaseModel):
name: str | None = Field(default=None, min_length=1, max_length=100)
kind: AccountKind | None = None
currency: str | None = Field(default=None, min_length=3, max_length=3)
institution: str | None = Field(default=None, max_length=100)
iban_masked: str | None = Field(default=None, max_length=34)
initial_balance: Money | None = None
is_archived: bool | None = None
class AccountRead(ORMModel):
id: uuid.UUID
name: str
kind: AccountKind
currency: str
institution: str | None
iban_masked: str | None
initial_balance: Money
is_archived: bool
balance: Money
transaction_count: int
last_transaction_date: date | None = None
# ---------------------------------------------------------------------------
# Categories
# ---------------------------------------------------------------------------
class CategoryCreate(BaseModel):
name: str = Field(min_length=1, max_length=80)
parent_id: uuid.UUID | None = None
icon: str | None = Field(default=None, max_length=50)
color: str | None = Field(default=None, max_length=7)
kind: CategoryKind = CategoryKind.EXPENSE
sort_order: int = 0
class CategoryUpdate(BaseModel):
name: str | None = Field(default=None, min_length=1, max_length=80)
parent_id: uuid.UUID | None = None
icon: str | None = Field(default=None, max_length=50)
color: str | None = Field(default=None, max_length=7)
sort_order: int | None = None
class CategoryRead(ORMModel):
id: uuid.UUID
parent_id: uuid.UUID | None
name: str
icon: str | None
color: str | None
kind: CategoryKind
is_system: bool
sort_order: int
transaction_count: int = 0
children: list["CategoryRead"] = Field(default_factory=list)
# ---------------------------------------------------------------------------
# Source profiles
# ---------------------------------------------------------------------------
class SourceProfileCreate(BaseModel):
name: str = Field(min_length=1, max_length=100)
kind: SourceKind
config: dict[str, Any] = Field(default_factory=dict)
class SourceProfileUpdate(BaseModel):
name: str | None = Field(default=None, min_length=1, max_length=100)
config: dict[str, Any] | None = None
class SourceProfileRead(ORMModel):
id: uuid.UUID
name: str
kind: SourceKind
config: dict[str, Any]
is_builtin: bool
# ---------------------------------------------------------------------------
# Transactions
# ---------------------------------------------------------------------------
class TransactionRead(BaseModel):
id: uuid.UUID
account_id: uuid.UUID
account_name: str | None = None
booked_date: date
value_date: date | None = None
amount: Money
currency: str
label_raw: str
label_clean: str
counterparty: str | None = None
category_id: uuid.UUID | None = None
category_name: str | None = None
category_color: str | None = None
category_source: CategorySource | None = None
notes: str | None = None
transfer_group_id: uuid.UUID | None = None
external_id: str | None = None
import_run_id: int | None = None
class TransactionCreate(BaseModel):
account_id: uuid.UUID
booked_date: date
amount: Money
label_clean: str = Field(min_length=1)
value_date: date | None = None
currency: str | None = Field(default=None, min_length=3, max_length=3)
category_id: uuid.UUID | None = None
counterparty: str | None = Field(default=None, max_length=150)
notes: str | None = None
class TransactionUpdate(BaseModel):
model_config = ConfigDict(extra="forbid")
label_clean: str | None = Field(default=None, min_length=1)
counterparty: str | None = Field(default=None, max_length=150)
category_id: uuid.UUID | None = None
notes: str | None = None
booked_date: date | None = None
amount: Money | None = None
class BulkCategorizeRequest(BaseModel):
transaction_ids: list[uuid.UUID] = Field(min_length=1, max_length=MAX_BULK_IDS)
category_id: uuid.UUID | None = None
class BulkCategorizeResponse(BaseModel):
updated: int
# ---------------------------------------------------------------------------
# Rules
# ---------------------------------------------------------------------------
class RuleCreate(BaseModel):
name: str = Field(min_length=1, max_length=100)
priority: int = 100
enabled: bool = True
stop: bool = True
matchers: dict[str, Any]
actions: dict[str, Any]
class RuleUpdate(BaseModel):
name: str | None = Field(default=None, min_length=1, max_length=100)
priority: int | None = None
enabled: bool | None = None
stop: bool | None = None
matchers: dict[str, Any] | None = None
actions: dict[str, Any] | None = None
class RuleRead(ORMModel):
id: uuid.UUID
name: str
priority: int
enabled: bool
stop: bool
matchers: dict[str, Any]
actions: dict[str, Any]
hit_count: int
# Stored instants: aware on PostgreSQL, naive on SQLite -> always UTC "Z".
last_applied_at: StoredUtcDatetime | None
class RuleReorderRequest(BaseModel):
ordered_ids: list[uuid.UUID] = Field(min_length=1)
class RuleApplyRequest(BaseModel):
scope: Literal["uncategorized", "all_non_manual", "all"] = "uncategorized"
date_from: date | None = None
date_to: date | None = None
account_id: uuid.UUID | None = None
rule_id: uuid.UUID | None = None
dry_run: bool = False
force: bool = False
class RuleApplyByRule(BaseModel):
rule_id: uuid.UUID
name: str
matched: int
class RuleApplyResponse(BaseModel):
scanned: int
matched: int
updated: int
dry_run: bool
by_rule: list[RuleApplyByRule]
class RulePreviewRequest(BaseModel):
matchers: dict[str, Any]
class RulePreviewResponse(BaseModel):
items: list[TransactionRead]
total_matched: int
# ---------------------------------------------------------------------------
# Budgets
# ---------------------------------------------------------------------------
def _first_of_month(value: date | None) -> date | None:
if value is None:
return None
return value.replace(day=1)
class BudgetCreate(BaseModel):
category_id: uuid.UUID
monthly_amount: Money = Field(gt=0)
start_month: date
end_month: date | None = None
@field_validator("start_month", "end_month")
@classmethod
def normalize_month(cls, value: date | None) -> date | None:
return _first_of_month(value)
class BudgetUpdate(BaseModel):
monthly_amount: Money | None = Field(default=None, gt=0)
end_month: date | None = None
effective_from: date | None = None
@field_validator("end_month", "effective_from")
@classmethod
def normalize_month(cls, value: date | None) -> date | None:
return _first_of_month(value)
class BudgetRead(ORMModel):
id: uuid.UUID
category_id: uuid.UUID
category_name: str | None = None
category_color: str | None = None
monthly_amount: Money
start_month: date
end_month: date | None
actual: Money = Decimal("0.00")
remaining: Money = Decimal("0.00")
progress_pct: float = 0.0
projected_eom: Money | None = None
status: str = "ok"
# ---------------------------------------------------------------------------
# Imports
# ---------------------------------------------------------------------------
class NormalizedRowRead(ORMModel):
booked_date: date
value_date: date | None
amount: Money
currency: str
label_raw: str
counterparty: str | None
external_id: str | None
source_row_index: int
class ImportPreviewResponse(BaseModel):
rows_preview: list[NormalizedRowRead]
rows_total: int
rows_error: int
rows_skipped_filtered: int
would_skip_duplicates: int
date_min: date | None
date_max: date | None
errors: list[dict[str, Any]]
duplicate_file_of: int | None = None
class ImportRunRead(ORMModel):
id: uuid.UUID
import_run_id: int
account_id: uuid.UUID
source_profile_id: uuid.UUID | None
filename: str
file_sha256: str
status: ImportStatus
# Stored instants: aware on PostgreSQL, naive on SQLite -> always UTC "Z".
started_at: StoredUtcDatetime
finished_at: StoredUtcDatetime | None
stats: dict[str, Any]
error_message: str | None
# ---------------------------------------------------------------------------
# Transfers
# ---------------------------------------------------------------------------
class TransferDetectRequest(BaseModel):
date_from: date | None = None
date_to: date | None = None
class TransferDetectResponse(BaseModel):
pairs_created: int
class TransferLinkRequest(BaseModel):
transaction_id_a: uuid.UUID
transaction_id_b: uuid.UUID
class TransferLinkResponse(BaseModel):
transfer_group_id: uuid.UUID
# ---------------------------------------------------------------------------
# Stats (ECharts-ready)
# ---------------------------------------------------------------------------
class MonthlySeries(BaseModel):
category_id: uuid.UUID | None
name: str
color: str
data: list[Money]
class MonthlyByCategoryResponse(BaseModel):
months: list[str]
series: list[MonthlySeries]
totals: list[Money]
class CashflowResponse(BaseModel):
months: list[str]
income: list[Money]
expenses: list[Money]
net: list[Money]
cumulative_net: list[Money]
class PeriodRead(BaseModel):
from_: date = Field(alias="from")
to: date
model_config = ConfigDict(populate_by_name=True)
class MerchantRead(BaseModel):
merchant: str
total: Money
count: int
average: Money
category_name: str | None = None
category_color: str | None = None
class TopMerchantsResponse(BaseModel):
period: PeriodRead
items: list[MerchantRead]
class RecurringItem(BaseModel):
model_config = ConfigDict(from_attributes=True)
merchant_key: str
label_display: str
category_id: uuid.UUID | None
category_name: str | None
periodicity: str
occurrences: int
average_amount: Money
expected_amount: Money
last_date: date
next_date_predicted: date
is_active: bool
class RecurringResponse(BaseModel):
items: list[RecurringItem]
monthly_total_estimate: Money
class BudgetProgressItem(BaseModel):
budget_id: uuid.UUID
category_id: uuid.UUID
category_name: str
category_color: str
budget: Money
actual: Money
remaining: Money
progress_pct: float
projected_eom: Money | None
status: str
class BudgetProgressTotals(BaseModel):
budget: Money
actual: Money
progress_pct: float
class BudgetProgressResponse(BaseModel):
month: str
items: list[BudgetProgressItem]
totals: BudgetProgressTotals
class SankeyNode(BaseModel):
name: str
color: str
class SankeyLink(BaseModel):
source: str
target: str
value: Money
class SankeyResponse(BaseModel):
period: PeriodRead
nodes: list[SankeyNode]
links: list[SankeyLink]
class DashboardResponse(BaseModel):
month: str
total_balance: Money
accounts_count: int
month_expenses: Money
month_income: Money
month_net: Money
average_expenses_6m: Money
budget_total: Money
budget_actual: Money
budget_progress_pct: float
uncategorized_count: int
File diff suppressed because it is too large Load Diff
+713
View File
@@ -0,0 +1,713 @@
"""Chart-ready aggregates (datamodel-finance.md §8 and §9.8).
Every expense figure is returned as a POSITIVE number (the semantics carry the
sign, §10.7). The common scope of §8.1 excludes internal transfers, non-EUR
transactions and archived accounts (unless accounts are given explicitly).
"""
import uuid
from collections import defaultdict
from dataclasses import dataclass, field
from datetime import date, timedelta
from decimal import ROUND_HALF_UP, Decimal
from typing import Any
import sqlalchemy as sa
from sqlalchemy import Select, func, or_, select
from sqlalchemy.orm import Session, aliased
from app.modules.finance.categorize import (
detect_recurring,
monthly_total_estimate,
)
from app.modules.finance.enums import CategoryKind
from app.modules.finance.models import (
FinAccount,
FinBudget,
FinCategory,
FinTransaction,
)
from app.modules.finance.normalize import add_months, merchant_key, month_str
from app.modules.finance.presets import (
DEFICIT_NODE_NAME,
INCOME_NODE_NAME,
MUTED_COLOR,
PALETTE,
SAVINGS_NODE_NAME,
UNCATEGORIZED_COLOR,
UNCATEGORIZED_LABEL,
)
ZERO = Decimal("0.00")
ONE_DAY = timedelta(days=1)
BUDGET_WARNING_RATIO = Decimal(80)
BUDGET_OVER_RATIO = Decimal(100)
def money(value: Any) -> Decimal:
"""Coerce a SQL aggregate (float on SQLite, Decimal on PG) to 2 decimals."""
if value is None:
return ZERO
if not isinstance(value, Decimal):
value = Decimal(str(value))
return value.quantize(Decimal("0.01"), rounding=ROUND_HALF_UP)
def month_range(months: int, end: date) -> list[str]:
"""The `months` last month keys ending with `end`'s month (inclusive)."""
first = add_months(end.replace(day=1), -(months - 1))
return [month_str(add_months(first, i)) for i in range(months)]
@dataclass
class StatsScope:
"""Aliases + predicates shared by every aggregate (§8.1)."""
category: Any
parent: Any
stmt: Select
def scope(user_id: int, account_ids: list[uuid.UUID] | None) -> StatsScope:
category = aliased(FinCategory)
parent = aliased(FinCategory)
stmt = (
select()
.select_from(FinTransaction)
.join(FinAccount, FinAccount.id == FinTransaction.account_id)
.outerjoin(category, category.id == FinTransaction.category_id)
.outerjoin(parent, parent.id == category.parent_id)
.where(
FinTransaction.user_id == user_id,
FinTransaction.currency == "EUR",
FinTransaction.transfer_group_id.is_(None),
or_(category.kind.is_(None), category.kind != CategoryKind.TRANSFER),
)
)
if account_ids:
stmt = stmt.where(FinTransaction.account_id.in_(account_ids))
else:
stmt = stmt.where(FinAccount.is_archived.is_(False))
return StatsScope(category=category, parent=parent, stmt=stmt)
def _year_month() -> tuple[Any, Any]:
"""Portable month extraction (SQLite -> strftime, PostgreSQL -> EXTRACT)."""
return (
sa.extract("year", FinTransaction.booked_date),
sa.extract("month", FinTransaction.booked_date),
)
def _key(year: Any, month: Any) -> str:
return f"{int(year):04d}-{int(month):02d}"
def _direction_clause(direction: str) -> Any:
if direction == "credit":
return FinTransaction.amount > 0
return FinTransaction.amount < 0
def _color_for(index: int, color: str | None) -> str:
return color or PALETTE[index % len(PALETTE)]
# ---------------------------------------------------------------------------
# GET /stats/monthly-by-category
# ---------------------------------------------------------------------------
def monthly_by_category(
db: Session,
user_id: int,
*,
months: int = 12,
level: str = "root",
direction: str = "debit",
account_ids: list[uuid.UUID] | None = None,
today: date | None = None,
) -> dict[str, Any]:
today = today or date.today() # noqa: DTZ011 — civil day, no tz conversion
keys = month_range(months, today)
start = date.fromisoformat(f"{keys[0]}-01")
end = add_months(date.fromisoformat(f"{keys[-1]}-01"), 1)
sc = scope(user_id, account_ids)
year_col, month_col = _year_month()
if level == "child":
cat_id = sc.category.id
cat_name = sc.category.name
cat_color = func.coalesce(sc.category.color, sc.parent.color)
else:
cat_id = func.coalesce(sc.parent.id, sc.category.id)
cat_name = func.coalesce(sc.parent.name, sc.category.name)
cat_color = func.coalesce(sc.parent.color, sc.category.color)
stmt = (
sc.stmt.add_columns(
year_col.label("y"),
month_col.label("m"),
cat_id.label("cid"),
cat_name.label("cname"),
cat_color.label("ccolor"),
func.sum(FinTransaction.amount).label("total"),
)
.where(
FinTransaction.booked_date >= start,
FinTransaction.booked_date < end,
_direction_clause(direction),
)
.group_by(year_col, month_col, cat_id, cat_name, cat_color)
)
buckets: dict[tuple[uuid.UUID | None, str, str | None], dict[str, Decimal]] = (
defaultdict(lambda: defaultdict(lambda: ZERO))
)
for row in db.execute(stmt):
key = _key(row.y, row.m)
ident = (row.cid, row.cname or UNCATEGORIZED_LABEL, row.ccolor)
buckets[ident][key] = money(abs(money(row.total)))
series: list[dict[str, Any]] = []
for index, (ident, per_month) in enumerate(
sorted(buckets.items(), key=lambda kv: -sum(kv[1].values()))
):
cid, name, color = ident
series.append(
{
"category_id": cid,
"name": name if cid else UNCATEGORIZED_LABEL,
"color": UNCATEGORIZED_COLOR if not cid else _color_for(index, color),
"data": [per_month.get(key, ZERO) for key in keys],
}
)
totals = [
money(sum((s["data"][i] for s in series), ZERO)) for i in range(len(keys))
]
return {"months": keys, "series": series, "totals": totals}
# ---------------------------------------------------------------------------
# GET /stats/cashflow
# ---------------------------------------------------------------------------
def cashflow(
db: Session,
user_id: int,
*,
months: int = 12,
account_ids: list[uuid.UUID] | None = None,
today: date | None = None,
) -> dict[str, Any]:
today = today or date.today() # noqa: DTZ011 — civil day, no tz conversion
keys = month_range(months, today)
start = date.fromisoformat(f"{keys[0]}-01")
end = add_months(date.fromisoformat(f"{keys[-1]}-01"), 1)
sc = scope(user_id, account_ids)
year_col, month_col = _year_month()
positive = func.sum(
sa.case((FinTransaction.amount > 0, FinTransaction.amount), else_=0)
)
negative = func.sum(
sa.case((FinTransaction.amount < 0, FinTransaction.amount), else_=0)
)
stmt = (
sc.stmt.add_columns(
year_col.label("y"),
month_col.label("m"),
positive.label("income"),
negative.label("expenses"),
)
.where(FinTransaction.booked_date >= start, FinTransaction.booked_date < end)
.group_by(year_col, month_col)
)
income_by_month: dict[str, Decimal] = {}
expenses_by_month: dict[str, Decimal] = {}
for row in db.execute(stmt):
key = _key(row.y, row.m)
income_by_month[key] = money(row.income)
expenses_by_month[key] = money(abs(money(row.expenses)))
income = [income_by_month.get(key, ZERO) for key in keys]
expenses = [expenses_by_month.get(key, ZERO) for key in keys]
net = [money(i - e) for i, e in zip(income, expenses, strict=True)]
cumulative: list[Decimal] = []
running = ZERO
for value in net:
running = money(running + value)
cumulative.append(running)
return {
"months": keys,
"income": income,
"expenses": expenses,
"net": net,
"cumulative_net": cumulative,
}
# ---------------------------------------------------------------------------
# GET /stats/top-merchants
# ---------------------------------------------------------------------------
def top_merchants(
db: Session,
user_id: int,
*,
months: int = 3,
limit: int = 15,
direction: str = "debit",
account_ids: list[uuid.UUID] | None = None,
today: date | None = None,
) -> dict[str, Any]:
today = today or date.today() # noqa: DTZ011 — civil day, no tz conversion
keys = month_range(months, today)
start = date.fromisoformat(f"{keys[0]}-01")
end = add_months(date.fromisoformat(f"{keys[-1]}-01"), 1)
sc = scope(user_id, account_ids)
stmt = sc.stmt.add_columns(
FinTransaction.counterparty,
FinTransaction.label_clean,
FinTransaction.label_raw,
FinTransaction.amount,
sc.category.name.label("cname"),
func.coalesce(sc.category.color, sc.parent.color).label("ccolor"),
).where(
FinTransaction.booked_date >= start,
FinTransaction.booked_date < end,
_direction_clause(direction),
)
groups: dict[str, dict[str, Any]] = {}
for row in db.execute(stmt):
name = row.counterparty or merchant_key(row.label_clean or row.label_raw)
if not name:
name = UNCATEGORIZED_LABEL
entry = groups.setdefault(
name,
{
"merchant": name,
"total": ZERO,
"count": 0,
"category_name": row.cname,
"category_color": row.ccolor,
},
)
entry["total"] = money(entry["total"] + abs(money(row.amount)))
entry["count"] += 1
if entry["category_name"] is None and row.cname:
entry["category_name"] = row.cname
entry["category_color"] = row.ccolor
items = sorted(groups.values(), key=lambda e: -e["total"])[:limit]
for entry in items:
entry["average"] = money(entry["total"] / entry["count"])
return {
"period": {"from": start, "to": end - ONE_DAY},
"items": items,
}
# ---------------------------------------------------------------------------
# GET /stats/budget-progress (§8.3)
# ---------------------------------------------------------------------------
def _children_map(db: Session, user_id: int) -> dict[uuid.UUID, list[uuid.UUID]]:
rows = db.execute(
select(FinCategory.id, FinCategory.parent_id).where(
FinCategory.user_id == user_id
)
).all()
children: dict[uuid.UUID, list[uuid.UUID]] = defaultdict(list)
for cid, parent_id in rows:
if parent_id is not None:
children[parent_id].append(cid)
return children
def expenses_by_category(
db: Session,
user_id: int,
start: date,
end: date,
account_ids: list[uuid.UUID] | None = None,
) -> dict[uuid.UUID | None, Decimal]:
"""Absolute expense total per (leaf) category over [start, end)."""
sc = scope(user_id, account_ids)
stmt = (
sc.stmt.add_columns(
FinTransaction.category_id.label("cid"),
func.sum(FinTransaction.amount).label("total"),
)
.where(
FinTransaction.booked_date >= start,
FinTransaction.booked_date < end,
FinTransaction.amount < 0,
)
.group_by(FinTransaction.category_id)
)
return {row.cid: money(abs(money(row.total))) for row in db.execute(stmt)}
def budget_progress(
db: Session,
user_id: int,
month: date,
*,
account_ids: list[uuid.UUID] | None = None,
today: date | None = None,
) -> dict[str, Any]:
today = today or date.today() # noqa: DTZ011 — civil day, no tz conversion
start = month.replace(day=1)
end = add_months(start, 1)
budgets = list(
db.scalars(
select(FinBudget)
.where(
FinBudget.user_id == user_id,
FinBudget.start_month <= start,
or_(FinBudget.end_month.is_(None), FinBudget.end_month >= start),
)
.order_by(FinBudget.start_month.asc())
).all()
)
if not budgets:
return {
"month": month_str(start),
"items": [],
"totals": {"budget": ZERO, "actual": ZERO, "progress_pct": 0.0},
}
actuals = expenses_by_category(db, user_id, start, end, account_ids)
children = _children_map(db, user_id)
categories = {
c.id: c
for c in db.scalars(
select(FinCategory).where(FinCategory.user_id == user_id)
).all()
}
is_current = start <= today < end
days_elapsed = (today - start).days + 1 if is_current else None
days_in_month = (end - start).days
items: list[dict[str, Any]] = []
total_budget = ZERO
total_actual = ZERO
for budget in budgets:
category = categories.get(budget.category_id)
subtree = [budget.category_id, *children.get(budget.category_id, [])]
actual = money(sum((actuals.get(cid, ZERO) for cid in subtree), ZERO))
amount = money(budget.monthly_amount)
progress = (
float((actual / amount * 100).quantize(Decimal("0.1"))) if amount else 0.0
)
projected = None
if is_current and days_elapsed:
projected = money(actual / Decimal(days_elapsed) * Decimal(days_in_month))
status = "ok"
if Decimal(str(progress)) > BUDGET_OVER_RATIO:
status = "over"
elif Decimal(str(progress)) >= BUDGET_WARNING_RATIO or (
projected is not None and projected > amount
):
status = "warning"
items.append(
{
"budget_id": budget.id,
"category_id": budget.category_id,
"category_name": category.name if category else UNCATEGORIZED_LABEL,
"category_color": (category.color if category else None)
or UNCATEGORIZED_COLOR,
"budget": amount,
"actual": actual,
"remaining": money(amount - actual),
"progress_pct": progress,
"projected_eom": projected,
"status": status,
}
)
total_budget = money(total_budget + amount)
total_actual = money(total_actual + actual)
totals_pct = (
float((total_actual / total_budget * 100).quantize(Decimal("0.1")))
if total_budget
else 0.0
)
return {
"month": month_str(start),
"items": items,
"totals": {
"budget": total_budget,
"actual": total_actual,
"progress_pct": totals_pct,
},
}
# ---------------------------------------------------------------------------
# GET /stats/sankey (§9.8)
# ---------------------------------------------------------------------------
@dataclass
class _SankeyBuilder:
nodes: list[dict[str, Any]] = field(default_factory=list)
links: list[dict[str, Any]] = field(default_factory=list)
seen: set[str] = field(default_factory=set)
def node(self, name: str, color: str) -> str:
if name not in self.seen:
self.seen.add(name)
self.nodes.append({"name": name, "color": color})
return name
def link(self, source: str, target: str, value: Decimal) -> None:
if value > 0:
self.links.append(
{"source": source, "target": target, "value": money(value)}
)
def sankey(
db: Session,
user_id: int,
*,
month: date | None = None,
months: int = 1,
account_ids: list[uuid.UUID] | None = None,
today: date | None = None,
) -> dict[str, Any]:
today = today or date.today() # noqa: DTZ011 — civil day, no tz conversion
if month is not None:
start = month.replace(day=1)
end = add_months(start, 1)
else:
keys = month_range(months, today)
start = date.fromisoformat(f"{keys[0]}-01")
end = add_months(date.fromisoformat(f"{keys[-1]}-01"), 1)
sc = scope(user_id, account_ids)
stmt = (
sc.stmt.add_columns(
sc.category.id.label("cid"),
sc.category.name.label("cname"),
sc.category.kind.label("ckind"),
sc.category.color.label("ccolor"),
sc.parent.id.label("pid"),
sc.parent.name.label("pname"),
sc.parent.color.label("pcolor"),
func.sum(FinTransaction.amount).label("total"),
func.sum(
sa.case((FinTransaction.amount > 0, FinTransaction.amount), else_=0)
).label("credit"),
func.sum(
sa.case((FinTransaction.amount < 0, -FinTransaction.amount), else_=0)
).label("debit"),
)
.where(FinTransaction.booked_date >= start, FinTransaction.booked_date < end)
.group_by(
sc.category.id,
sc.category.name,
sc.category.kind,
sc.category.color,
sc.parent.id,
sc.parent.name,
sc.parent.color,
)
)
income_roots: dict[str, dict[str, Any]] = {}
expense_roots: dict[str, dict[str, Any]] = {}
children: dict[str, dict[str, Decimal]] = defaultdict(
lambda: defaultdict(lambda: ZERO)
)
child_colors: dict[str, str] = {}
total_income = ZERO
total_expense = ZERO
for row in db.execute(stmt):
root_name = row.pname or row.cname
root_color = row.pcolor or row.ccolor
# Uncategorized rows are TWO independent buckets (§9.8): credits feed
# "Autres revenus", debits feed "Non catégorisé". Netting them against
# each other would silently drop the uncategorized expenses AND shrink
# the income side by the same amount. A real category keeps netting so
# that a refund simply lowers that category's expense (and a category
# never ends up on both sides of the hub, which would build a cycle).
if row.cid is None:
credit, debit = money(row.credit), money(row.debit)
else:
total = money(row.total)
credit = total if total > 0 else ZERO
debit = -total if total < 0 else ZERO
if credit > 0:
name = root_name or "Autres revenus"
entry = income_roots.setdefault(
name, {"total": ZERO, "color": root_color or PALETTE[0]}
)
entry["total"] = money(entry["total"] + credit)
total_income = money(total_income + credit)
if debit > 0:
name = root_name or UNCATEGORIZED_LABEL
entry = expense_roots.setdefault(
name,
{
"total": ZERO,
"color": (root_color or UNCATEGORIZED_COLOR)
if row.cid
else UNCATEGORIZED_COLOR,
},
)
entry["total"] = money(entry["total"] + debit)
total_expense = money(total_expense + debit)
if row.pid is not None and row.cname:
child_name = row.cname
if child_name in expense_roots or child_name in income_roots:
child_name = f"{row.pname} · {row.cname}"
children[name][child_name] = money(children[name][child_name] + debit)
child_colors[child_name] = row.ccolor or row.pcolor or PALETTE[0]
builder = _SankeyBuilder()
hub = builder.node(INCOME_NODE_NAME, MUTED_COLOR)
for index, (name, entry) in enumerate(
sorted(income_roots.items(), key=lambda kv: -kv[1]["total"])
):
builder.node(name, _color_for(index, entry["color"]))
builder.link(name, hub, entry["total"])
if total_expense > total_income:
builder.node(DEFICIT_NODE_NAME, "#D03B3B")
builder.link(DEFICIT_NODE_NAME, hub, money(total_expense - total_income))
for index, (name, entry) in enumerate(
sorted(expense_roots.items(), key=lambda kv: -kv[1]["total"])
):
builder.node(name, _color_for(index, entry["color"]))
builder.link(hub, name, entry["total"])
for child_name, value in sorted(
children.get(name, {}).items(), key=lambda kv: -kv[1]
):
builder.node(child_name, child_colors.get(child_name, PALETTE[index % 8]))
builder.link(name, child_name, value)
if total_income > total_expense:
builder.node(SAVINGS_NODE_NAME, "#0CA30C")
builder.link(hub, SAVINGS_NODE_NAME, money(total_income - total_expense))
return {
"period": {"from": start, "to": end - ONE_DAY},
"nodes": builder.nodes,
"links": builder.links,
}
# ---------------------------------------------------------------------------
# GET /stats/recurring + GET /dashboard
# ---------------------------------------------------------------------------
def recurring(
db: Session,
user_id: int,
*,
direction: str = "debit",
include_inactive: bool = False,
today: date | None = None,
) -> dict[str, Any]:
series = detect_recurring(
db,
user_id,
direction=direction,
include_inactive=include_inactive,
today=today,
)
return {
"items": series,
"monthly_total_estimate": monthly_total_estimate(series),
}
def account_balances(db: Session, user_id: int) -> dict[uuid.UUID, Decimal]:
rows = db.execute(
select(
FinTransaction.account_id,
func.sum(FinTransaction.amount),
)
.where(FinTransaction.user_id == user_id)
.group_by(FinTransaction.account_id)
).all()
return {account_id: money(total) for account_id, total in rows}
def account_transaction_counts(db: Session, user_id: int) -> dict[uuid.UUID, int]:
rows = db.execute(
select(FinTransaction.account_id, func.count())
.where(FinTransaction.user_id == user_id)
.group_by(FinTransaction.account_id)
).all()
return {account_id: int(count) for account_id, count in rows}
def dashboard(
db: Session,
user_id: int,
*,
today: date | None = None,
) -> dict[str, Any]:
"""KPIs of the finance home tab (ux-pages §13.1)."""
today = today or date.today() # noqa: DTZ011 — civil day, no tz conversion
accounts = list(
db.scalars(
select(FinAccount).where(
FinAccount.user_id == user_id, FinAccount.is_archived.is_(False)
)
).all()
)
balances = account_balances(db, user_id)
total_balance = money(
sum(
(money(a.initial_balance) + balances.get(a.id, ZERO) for a in accounts),
ZERO,
)
)
flow = cashflow(db, user_id, months=6, today=today)
current_key = month_str(today)
index = flow["months"].index(current_key)
month_expenses = flow["expenses"][index]
month_income = flow["income"][index]
month_net = flow["net"][index]
past = [value for i, value in enumerate(flow["expenses"]) if i != index]
average_expenses = money(sum(past, ZERO) / Decimal(len(past))) if past else ZERO
budgets = budget_progress(db, user_id, today.replace(day=1), today=today)
uncategorized = (
db.scalar(
select(func.count())
.select_from(FinTransaction)
.where(
FinTransaction.user_id == user_id,
FinTransaction.category_id.is_(None),
FinTransaction.transfer_group_id.is_(None),
)
)
or 0
)
return {
"month": current_key,
"total_balance": total_balance,
"accounts_count": len(accounts),
"month_expenses": month_expenses,
"month_income": month_income,
"month_net": month_net,
"average_expenses_6m": average_expenses,
"budget_total": budgets["totals"]["budget"],
"budget_actual": budgets["totals"]["actual"],
"budget_progress_pct": budgets["totals"]["progress_pct"],
"uncategorized_count": int(uncategorized),
}
+496
View File
@@ -0,0 +1,496 @@
"""Pure health/nutrition calculations (datamodel-health-vape.md §5).
Every function here is side-effect free and unit-tested with exact values
(app/tests/modules/test_health_calculations.py). No SQLAlchemy imports.
"""
import math
from dataclasses import dataclass, field
from datetime import date, timedelta
# Alias for the annotations of dataclasses that own a field *named* `date`
# (Projection). In a class body Python binds the default value before it
# evaluates the annotation, so `date: date | None = None` would read the field
# itself (None) instead of datetime.date and raise at import time on Python
# 3.12 — the runtime of docker/api.Dockerfile. Python 3.14 only hides the bug
# by deferring annotation evaluation (PEP 649).
DateT = date
# --- Central constants (datamodel §10) ---------------------------------------
KCAL_PER_KG_FAT = 7700 # 1 kg of body mass ~ 7700 kcal
EMA_ALPHA = 0.1 # weight trend smoothing (Hacker's Diet)
MIN_SLOPE_KG_PER_DAY = 0.005 # below this, slope is considered null
ACTIVITY_FACTORS: dict[str, float] = {
"sedentary": 1.2,
"light": 1.375,
"moderate": 1.55,
"active": 1.725,
"very_active": 1.9,
}
CALORIE_FLOOR_MALE = 1500
CALORIE_FLOOR_FEMALE = 1200
WORKOUT_OVERLAP_THRESHOLD = 0.8 # cross-source workout dedup (§3.6)
ADAPTIVE_TDEE_MIN_DAYS = 21 # minimal tracked window for §5.7
TDEE_SMOOTHING_DAYS = 7 # moving average window for the budget
MEASURED_TOTAL_MIN_RATIO = 0.8 # total_kcal plausibility guard vs BMR
# Field-by-field merge priority for activity_daily (§3.5). Unknown sources
# rank after every listed one.
ACTIVITY_SOURCE_PRIORITY: list[str] = [
"manual",
"health_connect",
"fitshow",
"csv_import",
"api",
]
MERGE_FIELDS: tuple[str, ...] = (
"steps",
"active_kcal",
"total_kcal",
"distance_m",
"active_minutes",
"floors",
)
def source_priority(source: str) -> int:
"""Rank of a source in the merge order; unknown sources go last."""
try:
return ACTIVITY_SOURCE_PRIORITY.index(source)
except ValueError:
return len(ACTIVITY_SOURCE_PRIORITY)
# --- BMR / TDEE / budget ------------------------------------------------------
def age_on(day: date, birthdate: date) -> int:
"""Completed years on `day`."""
years = day.year - birthdate.year
if (day.month, day.day) < (birthdate.month, birthdate.day):
years -= 1
return years
def bmr_mifflin(weight_kg: float, height_cm: float, age: int, sex: str) -> float:
"""Mifflin-St Jeor BMR; sex in {male, female, other} (§5.1)."""
base = 10 * weight_kg + 6.25 * height_cm - 5 * age
offset = {"male": 5.0, "female": -161.0, "other": -78.0}[sex]
return base + offset
@dataclass
class TdeeResult:
kcal: float
method: str # "measured_total" | "bmr_plus_active" | "estimated"
def tdee_effective(
bmr: float,
activity_level: str,
total_kcal: float | None = None,
active_kcal: float | None = None,
) -> TdeeResult:
"""Effective TDEE of one day, 3-tier preference (§5.2):
1. measured total_kcal, if plausible (> 0.8 x BMR);
2. BMR + active_kcal;
3. BMR x activity factor.
"""
if total_kcal is not None and total_kcal > MEASURED_TOTAL_MIN_RATIO * bmr:
return TdeeResult(kcal=float(total_kcal), method="measured_total")
if active_kcal is not None:
return TdeeResult(kcal=bmr + float(active_kcal), method="bmr_plus_active")
return TdeeResult(kcal=bmr * ACTIVITY_FACTORS[activity_level], method="estimated")
def moving_average(
values: list[float | None], window: int = TDEE_SMOOTHING_DAYS
) -> list[float | None]:
"""Trailing moving average over consecutive daily values; None entries are
skipped (a day stays None only when no value exists in its window)."""
out: list[float | None] = []
for i in range(len(values)):
chunk = [v for v in values[max(0, i - window + 1) : i + 1] if v is not None]
out.append(sum(chunk) / len(chunk) if chunk else None)
return out
def clamp(value: float, lo: float, hi: float) -> float:
return max(lo, min(hi, value))
@dataclass
class GoalRate:
weekly_rate_kg: float # positive = loss
rate_clamped: bool = False
def resolve_goal_rate(
mode: str,
day: date,
trend_now: float,
target_weight_kg: float,
weekly_rate_kg: float | None,
target_date: date | None,
) -> GoalRate:
"""Weekly rate implied by the active goal on `day` (§5.3)."""
if mode == "maintain":
return GoalRate(0.0)
if mode == "weekly_rate":
return GoalRate(float(weekly_rate_kg or 0.0))
# target_date: rate recomputed daily from the current trend.
if target_date is None:
return GoalRate(0.0)
weeks_left = max((target_date - day).days / 7, 1.0)
rate = (trend_now - target_weight_kg) / weeks_left
clamped = clamp(rate, -0.5, 1.0)
return GoalRate(clamped, rate_clamped=clamped != rate)
@dataclass
class BudgetResult:
kcal: float
deficit_target: float
floor_applied: bool
rate_clamped: bool = False
def daily_budget(
tdee_smoothed_kcal: float,
rate: GoalRate,
sex: str,
calorie_floor_kcal: int | None = None,
) -> BudgetResult:
"""Daily calorie budget = TDEE - weekly_rate x 7700 / 7, floored (§5.3)."""
deficit = rate.weekly_rate_kg * KCAL_PER_KG_FAT / 7 # = rate x 1100
budget = tdee_smoothed_kcal - deficit
floor = calorie_floor_kcal or (
CALORIE_FLOOR_MALE if sex == "male" else CALORIE_FLOOR_FEMALE
)
return BudgetResult(
kcal=max(budget, float(floor)),
deficit_target=deficit,
floor_applied=budget < floor,
rate_clamped=rate.rate_clamped,
)
# --- Weight trend (EMA), slope, projection ------------------------------------
def weight_trend(
entries: list[tuple[date, float]], alpha: float = EMA_ALPHA
) -> list[tuple[date, float]]:
"""EMA trend with gap correction (§5.5).
`entries` = one (day, weight) per local day, ascending — the FIRST
weigh-in of each day. Missing days are handled through
alpha_eff = 1 - (1 - alpha) ** gap_days.
Returned trend values are rounded to 2 decimals; the running trend keeps
full precision.
"""
out: list[tuple[date, float]] = []
prev_date: date | None = None
trend: float | None = None
for d, w in entries:
if trend is None or prev_date is None:
trend = w
else:
gap = (d - prev_date).days
alpha_eff = 1 - (1 - alpha) ** gap
trend = trend + alpha_eff * (w - trend)
out.append((d, round(trend, 2)))
prev_date = d
return out
def trend_at_day(trend: list[tuple[date, float]], day: date) -> float | None:
"""Trend value carried forward to `day` (last known point <= day)."""
value: float | None = None
for point_day, point_value in trend:
if point_day > day:
break
value = point_value
return value
def regression_slope(points: list[tuple[date, float]]) -> float | None:
"""OLS slope in kg/day over (day, value) points; None if < 3 points (§5.5)."""
if len(points) < 3:
return None
t0 = points[0][0]
ts = [float((d - t0).days) for d, _ in points]
ws = [w for _, w in points]
t_mean = sum(ts) / len(ts)
w_mean = sum(ws) / len(ws)
denom = sum((t - t_mean) ** 2 for t in ts)
if denom == 0:
return None
num = sum((t - t_mean) * (w - w_mean) for t, w in zip(ts, ws, strict=True))
return num / denom
@dataclass
class Projection:
status: str # "reached" | "not_converging" | "ok"
date: DateT | None = None
def project_target_date(
trend_now: float,
target_weight: float,
slope_kg_day: float | None,
today: date,
) -> Projection:
"""Projected date of reaching the target at the observed slope (§5.6)."""
delta = trend_now - target_weight # > 0 while weight remains to lose
if abs(delta) < 0.1:
return Projection(status="reached")
losing_needed = delta > 0
if (
slope_kg_day is None
or abs(slope_kg_day) < MIN_SLOPE_KG_PER_DAY
or (losing_needed and slope_kg_day >= 0)
or (not losing_needed and slope_kg_day <= 0)
):
return Projection(status="not_converging")
days = abs(delta / slope_kg_day)
if days > 3650:
return Projection(status="not_converging")
return Projection(status="ok", date=today + timedelta(days=round(days)))
# --- Energy balance & TDEE calibration (§5.4 / §5.7) --------------------------
def energy_balance(intake_kcal: float | None, tdee_kcal: float | None) -> float | None:
"""balance = intake - TDEE (negative = deficit); None when untracked."""
if intake_kcal is None or tdee_kcal is None:
return None
return intake_kcal - tdee_kcal
@dataclass
class CalibrationResult:
status: str # "ok" | "insufficient_data"
tracked_days: int = 0
expected_change_kg: float | None = None
actual_change_kg: float | None = None
gap_kg: float | None = None
tdee_adaptive_kcal: float | None = None
tdee_correction_kcal: float | None = None
def tdee_calibration(
tracked: list[tuple[float, float]],
trend_start: float | None,
trend_end: float | None,
) -> CalibrationResult:
"""Cumulative deficit vs actual (trend) loss + adaptive TDEE (§5.7).
`tracked` = [(intake_kcal, tdee_effective_kcal)] for tracked days only
(days with at least one food entry). Requires >= 21 tracked days.
"""
tracked_days = len(tracked)
if (
tracked_days < ADAPTIVE_TDEE_MIN_DAYS
or trend_start is None
or trend_end is None
):
return CalibrationResult(status="insufficient_data", tracked_days=tracked_days)
expected = sum(intake - tdee for intake, tdee in tracked) / KCAL_PER_KG_FAT
actual = trend_end - trend_start
mean_intake = sum(intake for intake, _ in tracked) / tracked_days
mean_tdee = sum(tdee for _, tdee in tracked) / tracked_days
tdee_adaptive = mean_intake - actual * KCAL_PER_KG_FAT / tracked_days
return CalibrationResult(
status="ok",
tracked_days=tracked_days,
expected_change_kg=expected,
actual_change_kg=actual,
gap_kg=actual - expected,
tdee_adaptive_kcal=tdee_adaptive,
tdee_correction_kcal=tdee_adaptive - mean_tdee,
)
# --- Activity merge & workout overlap -----------------------------------------
@dataclass
class MergedActivity:
date: date
steps: int | None = None
active_kcal: float | None = None
total_kcal: float | None = None
distance_m: int | None = None
active_minutes: int | None = None
floors: int | None = None
field_sources: dict[str, str] = field(default_factory=dict)
def merge_activity_day(rows: list) -> MergedActivity:
"""Field-by-field merge of one day's rows by source priority (§3.5).
`rows` = every activity row of the same (user, date), any source order;
objects only need `.date`, `.source` and the MERGE_FIELDS attributes.
Never sums two sources (double-count risk).
"""
ordered = sorted(rows, key=lambda r: (source_priority(r.source), r.source))
merged = MergedActivity(date=ordered[0].date)
for field_name in MERGE_FIELDS:
for row in ordered:
value = getattr(row, field_name)
if value is not None:
if field_name in ("active_kcal", "total_kcal"):
value = float(value)
setattr(merged, field_name, value)
merged.field_sources[field_name] = row.source
break
return merged
def overlap_ratio(a_start, a_end, b_start, b_end) -> float:
"""Overlap seconds / duration of the SHORTER interval (datetimes)."""
inter = (min(a_end, b_end) - max(a_start, b_start)).total_seconds()
if inter <= 0:
return 0.0
shorter = min((a_end - a_start).total_seconds(), (b_end - b_start).total_seconds())
if shorter <= 0:
return 0.0
return inter / shorter
# --- Body composition ---------------------------------------------------------
def bmi(weight_kg: float, height_cm: float) -> float | None:
"""Body mass index = kg / m²."""
if height_cm <= 0:
return None
height_m = height_cm / 100
return weight_kg / (height_m * height_m)
def navy_body_fat_pct(
sex: str,
height_cm: float,
waist_cm: float | None,
neck_cm: float | None,
hips_cm: float | None = None,
) -> float | None:
"""US Navy body-fat estimate (§3.3); None when inputs are missing."""
if not waist_cm or not neck_cm or height_cm <= 0:
return None
if sex == "female":
if not hips_cm:
return None
inner = waist_cm + hips_cm - neck_cm
if inner <= 0:
return None
denom = 1.29579 - 0.35004 * math.log10(inner) + 0.22100 * math.log10(height_cm)
else:
inner = waist_cm - neck_cm
if inner <= 0:
return None
denom = 1.0324 - 0.19077 * math.log10(inner) + 0.15456 * math.log10(height_cm)
if denom == 0:
return None
return 495 / denom - 450
# --- Planning / adherence (addendum-planning.md) ------------------------------
# Habit kinds; "done" is DERIVED from the data tables, there is no check-in
# table in v1: weigh_in -> WeightEntry, workout -> Workout, food_log -> FoodEntry.
SCHEDULE_KINDS: tuple[str, ...] = ("weigh_in", "workout", "food_log")
@dataclass
class DayAdherence:
day: date
planned: bool
done: bool
@property
def missed(self) -> bool:
return self.planned and not self.done
@property
def status(self) -> str:
"""Calendar-heatmap status: planned+done / planned+missed / off-plan."""
if self.planned:
return "done" if self.done else "missed"
return "done_unplanned" if self.done else "rest"
def is_planned(weekdays: list[int] | None, day: date, enabled: bool = True) -> bool:
"""True when `day` falls on a planned weekday (0 = Monday … 6 = Sunday)."""
if not enabled or not weekdays:
return False
return day.weekday() in set(weekdays)
def date_range(start: date, end: date) -> list[date]:
"""Inclusive list of local days from `start` to `end`."""
if end < start:
return []
return [start + timedelta(days=i) for i in range((end - start).days + 1)]
def build_adherence(
start: date,
end: date,
weekdays: list[int] | None,
enabled: bool,
done_days: set[date],
) -> list[DayAdherence]:
"""Per-day planned/done status over an inclusive local-day range."""
return [
DayAdherence(
day=day,
planned=is_planned(weekdays, day, enabled),
done=day in done_days,
)
for day in date_range(start, end)
]
def adherence_pct(days: list[DayAdherence]) -> float | None:
"""done ÷ planned, in percent; None when nothing was planned."""
planned = [d for d in days if d.planned]
if not planned:
return None
return 100.0 * sum(1 for d in planned if d.done) / len(planned)
@dataclass
class StreakResult:
current: int = 0
best: int = 0
def compute_streaks(days: list[DayAdherence], today: date) -> StreakResult:
"""Streaks over PLANNED days only (addendum-planning.md).
`best` = longest run of consecutive planned-and-done days. `current` = run
ending at the most recent planned day; a planned day equal to `today` that
is not done yet does not break the streak (the day is not over).
"""
planned = sorted((d for d in days if d.planned), key=lambda d: d.day)
best = run = 0
for entry in planned:
run = run + 1 if entry.done else 0
best = max(best, run)
current = 0
for entry in reversed(planned):
if entry.day > today:
continue # future planned days are not part of the current streak
if entry.day == today and not entry.done:
continue # grace period: today is still open
if not entry.done:
break
current += 1
return StreakResult(current=current, best=best)
+231
View File
@@ -0,0 +1,231 @@
"""Food referential: local `food_items` cache first, Open Food Facts proxy next.
Rules from docs/research/nutrition-sources.md §4.3:
- every OFF call goes through the backend (never the browser);
- mandatory custom User-Agent `AppName/Version (ContactEmail)`;
- each fetched product is cached in `food_items` so it is never fetched twice;
- OFF unreachable must degrade gracefully (French 503 only when nothing local).
Tests inject an `httpx.MockTransport` through `set_http_transport` — the suite
never touches the network.
"""
from decimal import Decimal
from typing import Any
import httpx
from sqlalchemy import func, or_, select
from sqlalchemy.orm import Session
from app.core.errors import AppError
from app.modules.health.models import FoodItem
from app.modules.health.schemas import FoodSearchItem, FoodSearchResponse
OFF_SEARCH_URL = "https://search.openfoodfacts.org/search"
OFF_PRODUCT_URL = "https://world.openfoodfacts.org/api/v2/product/{barcode}.json"
OFF_USER_AGENT = "LifeTrack/1.0 (meejayproduction@gmail.com)"
OFF_TIMEOUT_S = 5.0
OFF_SOURCE = "off"
OFF_FIELDS = (
"code,product_name,product_name_fr,brands,quantity,serving_size,"
"serving_quantity,nutriments"
)
_TRANSPORT: httpx.BaseTransport | None = None
class ServiceUnavailableError(AppError):
status_code, code = 503, "service_unavailable"
def set_http_transport(transport: httpx.BaseTransport | None) -> None:
"""Test seam: inject an httpx transport (MockTransport) for OFF calls."""
global _TRANSPORT
_TRANSPORT = transport
def _client() -> httpx.Client:
return httpx.Client(
timeout=OFF_TIMEOUT_S,
headers={"User-Agent": OFF_USER_AGENT},
transport=_TRANSPORT,
)
def _num(value: Any) -> Decimal | None:
if value is None or value == "":
return None
try:
return Decimal(str(float(value)))
except (TypeError, ValueError, ArithmeticError):
return None
def _text(value: Any) -> str:
"""OFF fields are user-contributed: a field documented as a string sometimes
arrives as a list (multi-value) or a number. Coerce anything to a clean string."""
if value is None:
return ""
if isinstance(value, (list, tuple)):
value = value[0] if value else ""
return str(value).strip()
def _kcal_100g(nutriments: dict[str, Any]) -> Decimal | None:
"""kcal per 100 g, recomputed from kJ when the kcal field is missing."""
kcal = _num(nutriments.get("energy-kcal_100g"))
if kcal is not None:
return kcal
kj = _num(nutriments.get("energy_100g")) or _num(nutriments.get("energy-kj_100g"))
if kj is None:
return None
return Decimal(str(round(float(kj) / 4.184, 1)))
def normalize_off_product(product: dict[str, Any]) -> dict[str, Any] | None:
"""Map one OFF product payload to `food_items` columns (per 100 g)."""
name = _text(product.get("product_name_fr")) or _text(product.get("product_name"))
if not name:
return None
nutriments = product.get("nutriments")
if not isinstance(nutriments, dict):
nutriments = {}
return {
"source": OFF_SOURCE,
"source_id": _text(product.get("code")) or None,
"name": name[:200],
"brand": _text(product.get("brands")).split(",")[0].strip()[:100] or None,
"energy_kcal_100g": _kcal_100g(nutriments),
"protein_g_100g": _num(nutriments.get("proteins_100g")),
"carbs_g_100g": _num(nutriments.get("carbohydrates_100g")),
"sugar_g_100g": _num(nutriments.get("sugars_100g")),
"fat_g_100g": _num(nutriments.get("fat_100g")),
"sat_fat_g_100g": _num(nutriments.get("saturated-fat_100g")),
"fiber_g_100g": _num(nutriments.get("fiber_100g")),
"salt_g_100g": _num(nutriments.get("salt_100g")),
"serving_size_g": _num(product.get("serving_quantity")),
"raw": product,
}
def cache_product(db: Session, data: dict[str, Any]) -> FoodItem:
"""Upsert one product into the local cache, keyed by (source, source_id)."""
item: FoodItem | None = None
if data["source_id"]:
item = db.scalar(
select(FoodItem).where(
FoodItem.source == data["source"],
FoodItem.source_id == data["source_id"],
)
)
if item is None:
item = FoodItem(source=data["source"], source_id=data["source_id"])
db.add(item)
for key, value in data.items():
if key not in {"source", "source_id"}:
setattr(item, key, value)
return item
def search_local(db: Session, query: str, limit: int) -> list[FoodItem]:
pattern = f"%{query.lower()}%"
stmt = (
select(FoodItem)
.where(
or_(
func.lower(FoodItem.name).like(pattern),
func.lower(func.coalesce(FoodItem.brand, "")).like(pattern),
)
)
.order_by(FoodItem.name.asc())
.limit(limit)
)
return list(db.scalars(stmt).all())
def fetch_off_search(query: str, limit: int) -> list[dict[str, Any]]:
"""Full-text search on Open Food Facts (Search-a-licious)."""
with _client() as client:
response = client.get(
OFF_SEARCH_URL,
params={"q": query, "langs": "fr", "page_size": limit},
)
response.raise_for_status()
payload = response.json()
hits = payload.get("hits")
if hits is None:
hits = payload.get("products") or []
return [hit for hit in hits if isinstance(hit, dict)]
def fetch_off_product(barcode: str) -> dict[str, Any] | None:
with _client() as client:
response = client.get(
OFF_PRODUCT_URL.format(barcode=barcode), params={"fields": OFF_FIELDS}
)
response.raise_for_status()
payload = response.json()
if payload.get("status") in (0, "failure"):
return None
return payload.get("product")
def search_foods(db: Session, query: str, limit: int = 20) -> FoodSearchResponse:
"""Local cache first, then Open Food Facts; caches every OFF hit."""
query = query.strip()
if len(query) < 2:
raise AppError("Saisissez au moins 2 caractères pour rechercher un aliment.")
local = search_local(db, query, limit)
items = [FoodSearchItem.model_validate(row) for row in local]
if len(items) >= limit:
return FoodSearchResponse(query=query, items=items, origin="cache")
known = {(row.source, row.source_id) for row in local}
try:
hits = fetch_off_search(query, limit - len(items))
except (httpx.HTTPError, ValueError) as exc:
if items:
return FoodSearchResponse(query=query, items=items, origin="cache")
raise ServiceUnavailableError(
"La base Open Food Facts est momentanément injoignable. "
"Réessayez plus tard ou saisissez l'aliment manuellement."
) from exc
for hit in hits:
data = normalize_off_product(hit)
if data is None or (data["source"], data["source_id"]) in known:
continue
known.add((data["source"], data["source_id"]))
cached = cache_product(db, data)
items.append(FoodSearchItem.model_validate(cached))
db.commit()
return FoodSearchResponse(query=query, items=items[:limit], origin="cache+off")
def food_by_barcode(db: Session, barcode: str) -> FoodSearchItem:
"""Cache lookup, then OFF product endpoint."""
barcode = barcode.strip()
cached = db.scalar(
select(FoodItem).where(
FoodItem.source == OFF_SOURCE, FoodItem.source_id == barcode
)
)
if cached is not None:
return FoodSearchItem.model_validate(cached)
try:
product = fetch_off_product(barcode)
except (httpx.HTTPError, ValueError) as exc:
raise ServiceUnavailableError(
"La base Open Food Facts est momentanément injoignable. "
"Réessayez plus tard ou saisissez l'aliment manuellement."
) from exc
data = normalize_off_product(product or {})
if data is None:
from app.core.errors import NotFoundError
raise NotFoundError("Aucun produit ne correspond à ce code-barres.")
item = cache_product(db, data)
db.commit()
db.refresh(item)
return FoodSearchItem.model_validate(item)
+673
View File
@@ -0,0 +1,673 @@
"""File importers for the health module (CONVENTIONS C3).
Three profiles:
- `foodvisor_csv` : Foodvisor RGPD/in-app export -> FoodEntry;
- `health_sync_csv` : Health Sync CSV export -> WeightEntry / DailyActivity;
- `weight_generic_csv` : plain `date;poids` history -> WeightEntry.
All of them are tolerant by design (docs/research/nutrition-sources.md §2.2 and
§7.3): utf-8-sig then cp1252, `,`/`;`/tab delimiters, comma decimals, FR/EN
header aliases, timestamps without offset interpreted in Europe/Paris.
When the source has no native id, `external_id = sha256(normalized row)[:32]`.
"""
import csv
import datetime as dt
import hashlib
import io
import json
import re
import unicodedata
from collections.abc import Iterator
from decimal import Decimal
from typing import Any, ClassVar
from zoneinfo import ZoneInfo
from sqlalchemy import select
from sqlalchemy.orm import Session
from app.core.importing.base import (
BaseImporter,
ImporterParseError,
NormalizedRecord,
RowError,
UpsertOutcome,
)
from app.core.importing.hashing import content_hash
from app.core.importing.registry import register_importer
from app.modules.health.models import (
DailyActivity,
FoodEntry,
MealType,
WeightEntry,
)
DEFAULT_TZ = ZoneInfo("Europe/Paris")
EXTERNAL_ID_LEN = 32
# --- Decoding / parsing helpers -----------------------------------------------
def decode_bytes(data: bytes) -> str:
"""utf-8-sig first, cp1252 as documented fallback (C3.3)."""
for encoding in ("utf-8-sig", "cp1252"):
try:
return data.decode(encoding)
except UnicodeDecodeError:
continue
return data.decode("utf-8", errors="replace")
def sniff_delimiter(sample: str) -> str:
header = sample.splitlines()[0] if sample.splitlines() else ""
counts = {sep: header.count(sep) for sep in (";", ",", "\t", "|")}
best = max(counts, key=lambda sep: counts[sep])
return best if counts[best] > 0 else ","
def norm_key(value: str) -> str:
"""Header normalisation: lowercase, accent-free, snake_case."""
text = unicodedata.normalize("NFKD", value or "")
text = "".join(ch for ch in text if not unicodedata.combining(ch))
text = re.sub(r"[^0-9a-zA-Z]+", "_", text.lower())
return text.strip("_")
def pick(row: dict[str, Any], aliases: tuple[str, ...]) -> Any:
"""First non-empty value among the normalized aliases (exact then prefix)."""
for alias in aliases:
if alias in row and str(row[alias]).strip() != "":
return row[alias]
for alias in aliases:
for key, value in row.items():
if key.startswith(alias) and str(value).strip() != "":
return value
return None
def to_decimal(value: Any) -> Decimal | None:
"""Tolerant number parsing: comma decimals, spaces, units, `traces`, `< x`."""
if value is None:
return None
text = str(value).strip()
if text == "" or text in {"-", "--", "NA", "N/A", "null"}:
return None
if "trace" in text.lower():
return Decimal(0)
text = text.replace("<", "").replace(" ", "").replace(" ", "")
text = text.replace(",", ".")
match = re.search(r"-?\d+(?:\.\d+)?", text)
if match is None:
return None
try:
return Decimal(match.group(0))
except ArithmeticError:
return None
def to_float(value: Any) -> float | None:
decimal = to_decimal(value)
return None if decimal is None else float(decimal)
_DATE_FORMATS = (
"%Y-%m-%d %H:%M:%S",
"%Y-%m-%d %H:%M",
"%Y-%m-%d",
"%d/%m/%Y %H:%M:%S",
"%d/%m/%Y %H:%M",
"%d/%m/%Y",
"%d-%m-%Y %H:%M",
"%d-%m-%Y",
"%d.%m.%Y",
"%Y/%m/%d",
)
def parse_datetime(value: Any, tz: ZoneInfo = DEFAULT_TZ) -> dt.datetime | None:
"""Parse a timestamp to aware UTC; naive input is read as local time."""
if value is None:
return None
text = str(value).strip()
if not text:
return None
iso = text.replace("Z", "+00:00")
parsed: dt.datetime | None = None
try:
parsed = dt.datetime.fromisoformat(iso)
except ValueError:
for fmt in _DATE_FORMATS:
try:
# These legacy formats carry no offset on purpose; the tz is
# applied a few lines below (Europe/Paris by default).
parsed = dt.datetime.strptime(text, fmt) # noqa: DTZ007
break
except ValueError:
continue
if parsed is None:
return None
if parsed.tzinfo is None:
parsed = parsed.replace(tzinfo=tz)
return parsed.astimezone(dt.UTC)
def parse_date(value: Any, tz: ZoneInfo = DEFAULT_TZ) -> dt.date | None:
parsed = parse_datetime(value, tz)
return None if parsed is None else parsed.astimezone(tz).date()
def row_external_id(prefix: str, row: dict[str, Any]) -> str:
"""sha256 of the normalized row (§1.5), namespaced by importer."""
payload = json.dumps(
{key: str(value) for key, value in sorted(row.items())},
sort_keys=True,
ensure_ascii=False,
)
digest = hashlib.sha256(f"{prefix}|{payload}".encode()).hexdigest()
return digest[:EXTERNAL_ID_LEN]
def read_rows(data: bytes) -> tuple[list[str], list[dict[str, Any]]]:
"""Decode + parse a CSV into normalized-header dicts."""
text = decode_bytes(data)
if not text.strip():
raise ImporterParseError("Fichier vide.")
delimiter = sniff_delimiter(text)
reader = csv.DictReader(io.StringIO(text), delimiter=delimiter)
if not reader.fieldnames:
raise ImporterParseError("En-têtes de colonnes introuvables.")
headers = [norm_key(name) for name in reader.fieldnames]
rows: list[dict[str, Any]] = []
for raw in reader:
row = {
norm_key(key): ("" if value is None else value)
for key, value in raw.items()
if key is not None
}
if any(str(value).strip() for value in row.values()):
rows.append(row)
return headers, rows
def head_headers(head: bytes) -> set[str]:
"""Normalized headers of the first line — used by `sniff` (never raises)."""
try:
text = decode_bytes(head)
first = text.splitlines()[0] if text.splitlines() else ""
delimiter = sniff_delimiter(text)
return {norm_key(part) for part in first.split(delimiter)}
except Exception: # noqa: BLE001 — sniff must never raise (C3.3)
return set()
def has_any(headers: set[str], aliases: tuple[str, ...]) -> bool:
return any(
header == alias or header.startswith(alias)
for header in headers
for alias in aliases
)
# --- Column alias tables -------------------------------------------------------
DATE_ALIASES = (
"date",
"jour",
"day",
"datetime",
"date_heure",
"horodatage",
"timestamp",
)
TIME_ALIASES = ("heure", "time", "hour")
MEAL_ALIASES = ("repas", "meal", "type_de_repas", "meal_type", "moment", "categorie")
NAME_ALIASES = (
"aliment",
"nom",
"name",
"food",
"food_name",
"libelle",
"produit",
"product",
)
BRAND_ALIASES = ("marque", "brand")
QUANTITY_ALIASES = ("quantite", "quantity", "portion", "amount", "serving", "poids_g")
UNIT_ALIASES = ("unite", "unit")
KCAL_ALIASES = ("calories", "kcal", "energie", "energy", "energie_kcal", "energy_kcal")
PROTEIN_ALIASES = ("proteines", "protein", "proteins", "prot")
CARBS_ALIASES = (
"glucides",
"carbs",
"carbohydrates",
"carbohydrate",
"total_carbohydrate",
)
FAT_ALIASES = ("lipides", "fat", "fats", "total_fat", "matieres_grasses")
FIBER_ALIASES = ("fibres", "fiber", "dietary_fiber", "fibre")
SUGAR_ALIASES = ("sucres", "sugar", "sugars", "sucre")
SATFAT_ALIASES = ("acides_gras_satures", "saturated_fat", "satures", "ags", "sat_fat")
SODIUM_ALIASES = ("sodium", "sel", "salt")
WEIGHT_ALIASES = ("poids", "weight", "poids_kg", "weight_kg", "masse", "body_weight")
STEPS_ALIASES = ("pas", "steps", "step_count", "nombre_de_pas")
DISTANCE_ALIASES = ("distance", "distance_m", "distance_km", "distance_metres")
ACTIVE_KCAL_ALIASES = (
"calories_actives",
"active_calories",
"active_energy",
"active_kcal",
"calories_brulees",
)
TOTAL_KCAL_ALIASES = (
"calories_totales",
"total_calories",
"total_energy",
"total_kcal",
)
ACTIVE_MIN_ALIASES = (
"minutes_actives",
"active_minutes",
"move_minutes",
"duree_activite",
)
BODYFAT_ALIASES = ("masse_grasse", "body_fat", "fat_percentage", "graisse")
MEAL_MAP: tuple[tuple[tuple[str, ...], MealType], ...] = (
(("petit_dejeuner", "petit_dej", "breakfast", "matin"), MealType.BREAKFAST),
(("collation", "snack", "gouter", "encas", "en_cas"), MealType.SNACK),
(("dejeuner", "lunch", "midi", "déjeuner"), MealType.LUNCH),
(("diner", "dinner", "souper", "soir"), MealType.DINNER),
)
def map_meal(value: Any) -> MealType:
key = norm_key(str(value or ""))
for aliases, meal in MEAL_MAP:
if any(alias in key for alias in aliases):
return meal
return MealType.SNACK
# --- Shared upsert helpers ----------------------------------------------------
def _find_event(
db: Session,
model: type,
user_id: int,
source: str,
external_id: str | None,
digest: str | None,
):
if external_id is not None:
row = db.scalar(
select(model).where(
model.user_id == user_id,
model.source == source,
model.external_id == external_id,
)
)
if row is not None:
return row
if digest is not None:
row = db.scalar(
select(model).where(model.user_id == user_id, model.content_hash == digest)
)
if row is not None:
return row
return None
def _dedupe_keys(record: NormalizedRecord) -> tuple[str | None, str | None]:
"""(external_id, content_hash) per architecture §5.1.
The content hash is only the FALLBACK used when the source provides no id
— never both, otherwise the `(user_id, content_hash)` constraint would also
dedupe across importers/sources, which the spec keeps distinct.
"""
if record.external_id:
return record.external_id[:255], None
return None, content_hash(record)
def _weight_slot_taken(
db: Session, user_id: int, source: str, measured_at: dt.datetime
) -> bool:
"""`uq_weight_entries_user_ts_source` guard — never let an import crash on it."""
return (
db.scalar(
select(WeightEntry).where(
WeightEntry.user_id == user_id,
WeightEntry.source == source,
WeightEntry.measured_at == measured_at,
)
)
is not None
)
def _upsert_daily_activity(
db: Session,
user_id: int,
source: str,
day: dt.date,
values: dict[str, Any],
raw: dict[str, Any],
import_run_id: int | None,
) -> UpsertOutcome:
"""Natural key (user, day, source) upsert-replacement (architecture §5.1).
Returns DUPLICATE when every value is already identical (re-importing the
same file is a no-op), UPDATED when the source refreshed its numbers.
"""
row = db.scalar(
select(DailyActivity).where(
DailyActivity.user_id == user_id,
DailyActivity.date == day,
DailyActivity.source == source,
)
)
if row is None:
db.add(
DailyActivity(
user_id=user_id,
date=day,
source=source,
import_run_id=import_run_id,
raw=raw,
**values,
)
)
db.flush() # make the row visible to the next lookups of the same run
return UpsertOutcome.INSERTED
changed = False
for field, value in values.items():
if value is None:
continue
if getattr(row, field) != value:
setattr(row, field, value)
changed = True
if not changed:
return UpsertOutcome.DUPLICATE
row.raw = raw
row.import_run_id = import_run_id
return UpsertOutcome.UPDATED
# --- foodvisor_csv -------------------------------------------------------------
@register_importer
class FoodvisorCsvImporter(BaseImporter):
id: ClassVar[str] = "foodvisor_csv"
label: ClassVar[str] = "Foodvisor (export CSV)"
domain: ClassVar[str] = "health"
accepted_extensions: ClassVar[tuple[str, ...]] = (".csv",)
source: ClassVar[str] = "foodvisor"
@classmethod
def sniff(cls, filename: str, head: bytes) -> bool:
if not filename.lower().endswith(".csv"):
return False
headers = head_headers(head)
return (
has_any(headers, MEAL_ALIASES)
and has_any(headers, KCAL_ALIASES)
and has_any(headers, NAME_ALIASES)
)
def parse(self, data: bytes, filename: str) -> Iterator[NormalizedRecord]:
_, rows = read_rows(data)
if not rows:
raise ImporterParseError("Aucune ligne exploitable dans le fichier.")
for row in rows:
name = pick(row, NAME_ALIASES)
kcal = to_decimal(pick(row, KCAL_ALIASES))
when = parse_datetime(pick(row, DATE_ALIASES))
if when is None:
time_part = pick(row, TIME_ALIASES)
date_part = pick(row, DATE_ALIASES)
if date_part and time_part:
when = parse_datetime(f"{date_part} {time_part}")
if not name or kcal is None or when is None:
raise RowError(
"Ligne incomplète : date, aliment et calories sont requis."
)
quantity = to_decimal(pick(row, QUANTITY_ALIASES)) or Decimal(100)
yield NormalizedRecord(
kind="food_entry",
external_id=str(pick(row, ("id", "entry_id", "uuid")) or "")
or row_external_id(self.id, row),
dedupe_fields=("eaten_at", "name", "kcal", "meal"),
data={
"eaten_at": when,
"meal": map_meal(pick(row, MEAL_ALIASES)),
"name": str(name)[:200],
"brand": (str(pick(row, BRAND_ALIASES) or "") or None),
"quantity": quantity,
"unit": str(pick(row, UNIT_ALIASES) or "g")[:20],
"kcal": kcal,
"protein_g": to_decimal(pick(row, PROTEIN_ALIASES)),
"carbs_g": to_decimal(pick(row, CARBS_ALIASES)),
"fat_g": to_decimal(pick(row, FAT_ALIASES)),
"fiber_g": to_decimal(pick(row, FIBER_ALIASES)),
"sugar_g": to_decimal(pick(row, SUGAR_ALIASES)),
"sat_fat_g": to_decimal(pick(row, SATFAT_ALIASES)),
"sodium_mg": to_decimal(pick(row, SODIUM_ALIASES)),
"raw": {key: str(value) for key, value in row.items()},
},
)
def upsert(
self, db: Session, user_id: int, record: NormalizedRecord
) -> UpsertOutcome:
external_id, digest = _dedupe_keys(record)
if _find_event(db, FoodEntry, user_id, self.source, external_id, digest):
return UpsertOutcome.DUPLICATE
data = dict(record.data)
raw = data.pop("raw", None)
db.add(
FoodEntry(
user_id=user_id,
source=self.source,
external_id=external_id,
content_hash=digest,
import_run_id=self.import_run_id,
raw=raw,
**data,
)
)
db.flush()
return UpsertOutcome.INSERTED
# --- health_sync_csv -----------------------------------------------------------
@register_importer
class HealthSyncCsvImporter(BaseImporter):
"""Health Sync exports: one row per day (or per measure) with a flexible
set of metric columns (poids, pas, distance, calories…)."""
id: ClassVar[str] = "health_sync_csv"
label: ClassVar[str] = "Health Sync / Health Connect (export CSV)"
domain: ClassVar[str] = "health"
accepted_extensions: ClassVar[tuple[str, ...]] = (".csv",)
source: ClassVar[str] = "csv_import"
_METRIC_GROUPS = (
STEPS_ALIASES,
DISTANCE_ALIASES,
ACTIVE_KCAL_ALIASES,
TOTAL_KCAL_ALIASES,
ACTIVE_MIN_ALIASES,
WEIGHT_ALIASES,
)
@classmethod
def sniff(cls, filename: str, head: bytes) -> bool:
if not filename.lower().endswith(".csv"):
return False
headers = head_headers(head)
if not has_any(headers, DATE_ALIASES):
return False
if has_any(headers, MEAL_ALIASES):
return False # nutrition export -> foodvisor_csv
matched = sum(1 for group in cls._METRIC_GROUPS if has_any(headers, group))
return matched >= 2
def parse(self, data: bytes, filename: str) -> Iterator[NormalizedRecord]:
_, rows = read_rows(data)
if not rows:
raise ImporterParseError("Aucune ligne exploitable dans le fichier.")
for row in rows:
when = parse_datetime(pick(row, DATE_ALIASES))
if when is None:
raise RowError("Date illisible sur cette ligne.")
time_part = pick(row, TIME_ALIASES)
if time_part:
combined = parse_datetime(
f"{parse_date(pick(row, DATE_ALIASES))} {time_part}"
)
if combined is not None:
when = combined
day = when.astimezone(DEFAULT_TZ).date()
raw = {key: str(value) for key, value in row.items()}
weight = to_decimal(pick(row, WEIGHT_ALIASES))
if weight is not None and 20 < float(weight) < 400:
yield NormalizedRecord(
kind="weight",
external_id=row_external_id(f"{self.id}:weight", row),
dedupe_fields=("measured_at", "weight_kg"),
data={
"measured_at": when,
"weight_kg": weight,
"body_fat_pct": to_decimal(pick(row, BODYFAT_ALIASES)),
"raw": raw,
},
)
distance_key = next(
(key for key in row if has_any({key}, DISTANCE_ALIASES)), None
)
distance = to_decimal(row.get(distance_key)) if distance_key else None
if distance is not None and distance_key.endswith("km"):
distance = distance * 1000 # Health Sync exports km on some locales
values = {
"steps": int(to_float(pick(row, STEPS_ALIASES)) or 0)
if pick(row, STEPS_ALIASES) is not None
else None,
"active_kcal": to_decimal(pick(row, ACTIVE_KCAL_ALIASES)),
"total_kcal": to_decimal(pick(row, TOTAL_KCAL_ALIASES)),
"distance_m": int(distance) if distance is not None else None,
"active_minutes": int(to_float(pick(row, ACTIVE_MIN_ALIASES)) or 0)
if pick(row, ACTIVE_MIN_ALIASES) is not None
else None,
}
if any(value is not None for value in values.values()):
yield NormalizedRecord(
kind="daily_activity",
external_id=row_external_id(f"{self.id}:activity", row),
dedupe_fields=("day", "steps", "active_kcal", "distance_m"),
data={"day": day, **values, "raw": raw},
)
def upsert(
self, db: Session, user_id: int, record: NormalizedRecord
) -> UpsertOutcome:
external_id, digest = _dedupe_keys(record)
data = dict(record.data)
raw = data.pop("raw", None)
if record.kind == "weight":
if _find_event(
db, WeightEntry, user_id, self.source, external_id, digest
) or _weight_slot_taken(db, user_id, self.source, data["measured_at"]):
return UpsertOutcome.DUPLICATE
db.add(
WeightEntry(
user_id=user_id,
source=self.source,
external_id=external_id,
content_hash=digest,
import_run_id=self.import_run_id,
raw=raw,
**data,
)
)
db.flush()
return UpsertOutcome.INSERTED
day = data.pop("day")
return _upsert_daily_activity(
db, user_id, self.source, day, data, raw, self.import_run_id
)
# --- weight_generic_csv --------------------------------------------------------
@register_importer
class WeightGenericCsvImporter(BaseImporter):
"""Minimal `date;poids` history typed by the user."""
id: ClassVar[str] = "weight_generic_csv"
label: ClassVar[str] = "Historique de poids (CSV date/poids)"
domain: ClassVar[str] = "health"
accepted_extensions: ClassVar[tuple[str, ...]] = (".csv",)
source: ClassVar[str] = "csv_import"
@classmethod
def sniff(cls, filename: str, head: bytes) -> bool:
if not filename.lower().endswith(".csv"):
return False
headers = {header for header in head_headers(head) if header}
if len(headers) > 3:
return False
return has_any(headers, DATE_ALIASES) and has_any(headers, WEIGHT_ALIASES)
def parse(self, data: bytes, filename: str) -> Iterator[NormalizedRecord]:
_, rows = read_rows(data)
if not rows:
raise ImporterParseError("Aucune ligne exploitable dans le fichier.")
for row in rows:
when = parse_datetime(pick(row, DATE_ALIASES))
weight = to_decimal(pick(row, WEIGHT_ALIASES))
if when is None or weight is None:
raise RowError("Ligne incomplète : date et poids sont requis.")
if not 20 < float(weight) < 400:
raise RowError("Poids hors bornes (20-400 kg).")
yield NormalizedRecord(
kind="weight",
external_id=row_external_id(self.id, row),
dedupe_fields=("measured_at", "weight_kg"),
data={
"measured_at": when,
"weight_kg": weight,
"raw": {key: str(value) for key, value in row.items()},
},
)
def upsert(
self, db: Session, user_id: int, record: NormalizedRecord
) -> UpsertOutcome:
external_id, digest = _dedupe_keys(record)
data = dict(record.data)
raw = data.pop("raw", None)
if _find_event(
db, WeightEntry, user_id, self.source, external_id, digest
) or _weight_slot_taken(db, user_id, self.source, data["measured_at"]):
return UpsertOutcome.DUPLICATE
db.add(
WeightEntry(
user_id=user_id,
source=self.source,
external_id=external_id,
content_hash=digest,
import_run_id=self.import_run_id,
raw=raw,
**data,
)
)
db.flush()
return UpsertOutcome.INSERTED
+592
View File
@@ -0,0 +1,592 @@
"""JSON ingestion handler for the health domain (CONVENTIONS C4).
`POST /api/ingest/health` (device key scope `ingest:health`) accepts BOTH:
1. the canonical normalized shape of architecture §5.5
`{"type": "steps", "external_id": "...", "data": {"day": "2026-08-12",
"steps": 9421, "calories_kcal": 2350, "distance_m": 6800}}`;
2. the health-connect-webhook bridge shape (snake_case Health Connect records,
docs/research/health-connect.md §4.1) where the payload of one array item is
passed as `data` — `{"start_time", "end_time", "count"/"energy"/"volume",
"metadata": {"id": ...}, "origin_app": ...}`.
The adapter below flattens both into the same field names. The full incoming
payload is always kept in the `raw` JSON column so the mapping can be replayed.
"""
import datetime as dt
import hashlib
import json
from decimal import Decimal
from typing import Any, ClassVar
from zoneinfo import ZoneInfo
from sqlalchemy import select
from sqlalchemy.orm import Session
from app.core.importing.base import RowError, UpsertOutcome
from app.core.ingest.base import BaseIngestHandler, IngestRecord
from app.core.ingest.registry import register_ingest_handler
from app.core.timeutils import resolve_tz
from app.modules.health.models import (
DailyActivity,
FoodEntry,
MealType,
SportType,
WaterEntry,
WeightEntry,
Workout,
)
DEFAULT_TZ = ZoneInfo("Europe/Paris")
# Sender-declared source -> DataSource registry value (§1.5). Anything else is
# kept verbatim so an unknown bridge still ranks last in the merge priority.
SOURCE_ALIASES = {
"android_bridge": "health_connect",
"health-connect-webhook": "health_connect",
"health_connect_webhook": "health_connect",
"hc-webhook": "health_connect",
"hc_webhook": "health_connect",
"companion-app": "health_connect",
"companion_app": "health_connect",
"healthconnect": "health_connect",
"health_connect": "health_connect",
"fitshow": "fitshow",
"manual": "manual",
"ingest": "api",
"": "api",
}
ACTIVITY_TYPES: dict[str, str] = {
"steps": "steps",
"distance": "distance_m",
"active_calories": "active_kcal",
"total_calories": "total_kcal",
}
SPORT_ALIASES: dict[str, SportType] = {
"running_treadmill": SportType.TREADMILL_RUN,
"treadmill_run": SportType.TREADMILL_RUN,
"walking_treadmill": SportType.TREADMILL_WALK,
"treadmill_walk": SportType.TREADMILL_WALK,
"treadmill": SportType.TREADMILL_WALK,
"running": SportType.RUNNING,
"run": SportType.RUNNING,
"walking": SportType.WALKING,
"walk": SportType.WALKING,
"biking": SportType.CYCLING,
"biking_stationary": SportType.CYCLING,
"cycling": SportType.CYCLING,
"swimming_pool": SportType.SWIMMING,
"swimming_open_water": SportType.SWIMMING,
"swimming": SportType.SWIMMING,
"strength_training": SportType.STRENGTH,
"weightlifting": SportType.STRENGTH,
"strength": SportType.STRENGTH,
"high_intensity_interval_training": SportType.HIIT,
"hiit": SportType.HIIT,
"yoga": SportType.YOGA,
"hiking": SportType.HIKING,
}
MEAL_BY_CODE = {
1: MealType.BREAKFAST,
2: MealType.LUNCH,
3: MealType.DINNER,
4: MealType.SNACK,
}
MEAL_BY_NAME = {
"breakfast": MealType.BREAKFAST,
"petit_dejeuner": MealType.BREAKFAST,
"lunch": MealType.LUNCH,
"dejeuner": MealType.LUNCH,
"dinner": MealType.DINNER,
"diner": MealType.DINNER,
"snack": MealType.SNACK,
"collation": MealType.SNACK,
}
# --- Adapter helpers ----------------------------------------------------------
def normalize_source(source: str | None) -> str:
key = (source or "").strip().lower()
return SOURCE_ALIASES.get(key, key or "api")[:50]
def flatten(data: dict[str, Any]) -> dict[str, Any]:
"""Merge the bridge's nested `value`/`metadata` objects into a flat view."""
flat: dict[str, Any] = {}
nested = data.get("value")
if isinstance(nested, dict):
flat.update(nested)
metadata = data.get("metadata")
if isinstance(metadata, dict):
for key, value in metadata.items():
flat[f"metadata_{key}"] = value
for key, value in data.items():
if key not in {"value", "metadata"} or not isinstance(value, dict):
flat[key] = value
return flat
def first(flat: dict[str, Any], *keys: str) -> Any:
for key in keys:
value = flat.get(key)
if value is not None and value != "":
return value
return None
def as_decimal(value: Any) -> Decimal | None:
if value is None or value == "":
return None
try:
return Decimal(str(float(value)))
except (TypeError, ValueError, ArithmeticError):
return None
def as_int(value: Any) -> int | None:
number = as_decimal(value)
return None if number is None else int(number)
def as_datetime(value: Any) -> dt.datetime | None:
if value is None or value == "":
return None
if isinstance(value, dt.datetime):
parsed = value
else:
text = str(value).strip().replace("Z", "+00:00")
try:
parsed = dt.datetime.fromisoformat(text)
except ValueError:
try: # epoch millis/seconds sent by some bridges
number = float(text)
except ValueError:
return None
if number > 1e11:
number /= 1000
parsed = dt.datetime.fromtimestamp(number, tz=dt.UTC)
if parsed.tzinfo is None:
parsed = parsed.replace(tzinfo=DEFAULT_TZ)
return parsed.astimezone(dt.UTC)
def resolve_day(flat: dict[str, Any], tz: ZoneInfo) -> dt.date | None:
raw_day = first(flat, "day", "date", "local_date")
if raw_day is not None:
try:
return dt.date.fromisoformat(str(raw_day)[:10])
except ValueError:
return None
moment = as_datetime(
first(flat, "start_time", "started_at", "time", "measured_at", "end_time")
)
return None if moment is None else moment.astimezone(tz).date()
def external_id_of(record: IngestRecord, flat: dict[str, Any]) -> str | None:
return record.external_id or first(
flat, "metadata_id", "external_id", "id", "uuid", "client_record_id"
)
def content_digest(record_type: str, payload: dict[str, Any]) -> str:
"""Fallback dedupe key when the source carries no id (architecture §5.1)."""
body = json.dumps(payload, sort_keys=True, default=str, ensure_ascii=False)
return hashlib.sha256(f"{record_type}|{body}".encode()).hexdigest()
def meal_of(flat: dict[str, Any], moment: dt.datetime, tz: ZoneInfo) -> MealType:
raw = first(flat, "meal_type", "meal", "meal_name")
if isinstance(raw, int) or (isinstance(raw, str) and raw.isdigit()):
mapped = MEAL_BY_CODE.get(int(raw))
if mapped is not None:
return mapped
if isinstance(raw, str):
mapped = MEAL_BY_NAME.get(raw.strip().lower())
if mapped is not None:
return mapped
hour = moment.astimezone(tz).hour
if hour < 11:
return MealType.BREAKFAST
if hour < 15:
return MealType.LUNCH
if hour < 18:
return MealType.SNACK
return MealType.DINNER
def sport_of(flat: dict[str, Any]) -> tuple[SportType, str | None]:
raw = first(
flat, "exercise_type", "sport_type", "activity_type", "exercise", "type"
)
key = str(raw or "").strip().lower().replace("-", "_").replace(" ", "_")
mapped = SPORT_ALIASES.get(key)
if mapped is not None:
return mapped, None
return SportType.OTHER, (str(raw)[:100] if raw else None)
# --- Handler ------------------------------------------------------------------
@register_ingest_handler
class HealthIngestHandler(BaseIngestHandler):
domain: ClassVar[str] = "health"
record_types: ClassVar[tuple[str, ...]] = (
"weight",
"steps",
"distance",
"active_calories",
"total_calories",
"exercise_session",
"nutrition",
"hydration",
)
def apply(self, db: Session, user_id: int, record: IngestRecord) -> UpsertOutcome:
tz = self._user_tz(db, user_id)
source = normalize_source(record.source)
flat = flatten(record.data or {})
if record.type in ACTIVITY_TYPES:
return self._apply_activity(db, user_id, source, record, flat, tz)
if record.type == "weight":
return self._apply_weight(db, user_id, source, record, flat)
if record.type == "exercise_session":
return self._apply_workout(db, user_id, source, record, flat)
if record.type == "nutrition":
return self._apply_nutrition(db, user_id, source, record, flat, tz)
if record.type == "hydration":
return self._apply_hydration(db, user_id, source, record, flat)
raise RowError("Type d'enregistrement non pris en charge.")
# -- shared plumbing --
@staticmethod
def _user_tz(db: Session, user_id: int) -> ZoneInfo:
from app.modules.health.models import HealthProfile
profile = db.scalar(
select(HealthProfile).where(HealthProfile.user_id == user_id)
)
try:
return resolve_tz(profile.timezone if profile else None)
except Exception: # noqa: BLE001 — a broken profile tz must not fail ingest
return DEFAULT_TZ
@staticmethod
def _dedupe_keys(
record: IngestRecord, flat: dict[str, Any], kind: str, payload: dict[str, Any]
) -> tuple[str | None, str | None]:
"""(external_id, content_hash) per architecture §5.1: the content hash
is only the FALLBACK when the source carries no id — never both, so two
sources describing the same event stay distinct rows (the cross-source
overlap rule handles workouts)."""
external_id = external_id_of(record, flat)
if external_id is not None:
return str(external_id)[:255], None
return None, content_digest(kind, payload)
@staticmethod
def _existing(
db: Session,
model: type,
user_id: int,
source: str,
external_id: str | None,
digest: str | None,
):
if external_id is not None:
return db.scalar(
select(model).where(
model.user_id == user_id,
model.source == source,
model.external_id == external_id,
)
)
return db.scalar(
select(model).where(model.user_id == user_id, model.content_hash == digest)
)
# -- record types --
def _apply_activity(
self,
db: Session,
user_id: int,
source: str,
record: IngestRecord,
flat: dict[str, Any],
tz: ZoneInfo,
) -> UpsertOutcome:
day = resolve_day(flat, tz)
if day is None:
raise RowError("Jour introuvable dans l'enregistrement d'activité.")
distance = as_decimal(first(flat, "distance_m", "distance", "meters"))
if distance is None:
km = as_decimal(first(flat, "distance_km", "kilometers"))
distance = km * 1000 if km is not None else None
values: dict[str, Any] = {
"steps": as_int(first(flat, "steps", "count", "step_count")),
"distance_m": int(distance) if distance is not None else None,
"active_kcal": as_decimal(
first(flat, "active_kcal", "active_calories", "calories_kcal")
),
"total_kcal": as_decimal(
first(flat, "total_kcal", "total_calories", "total_energy_kcal")
),
"active_minutes": as_int(first(flat, "active_minutes", "move_minutes")),
"floors": as_int(first(flat, "floors", "floors_climbed")),
}
primary = ACTIVITY_TYPES[record.type]
if values[primary] is None:
fallback = first(flat, "value", "amount", "energy_kcal", "energy", "kcal")
if primary == "steps":
values[primary] = as_int(fallback)
elif primary == "distance_m":
values[primary] = (
int(as_decimal(fallback))
if as_decimal(fallback) is not None
else None
)
else:
values[primary] = as_decimal(fallback)
if values[primary] is None:
raise RowError("Valeur manquante pour cet enregistrement d'activité.")
row = db.scalar(
select(DailyActivity).where(
DailyActivity.user_id == user_id,
DailyActivity.date == day,
DailyActivity.source == source,
)
)
raw = dict(record.data or {})
if row is None:
db.add(
DailyActivity(
user_id=user_id,
date=day,
source=source,
external_id=external_id_of(record, flat),
raw=raw,
**values,
)
)
db.flush()
return UpsertOutcome.INSERTED
changed = False
for field, value in values.items():
if value is None:
continue
if getattr(row, field) != value:
setattr(row, field, value)
changed = True
if not changed:
return UpsertOutcome.DUPLICATE
row.raw = raw
db.flush()
return UpsertOutcome.UPDATED
def _apply_weight(
self,
db: Session,
user_id: int,
source: str,
record: IngestRecord,
flat: dict[str, Any],
) -> UpsertOutcome:
moment = as_datetime(
first(flat, "measured_at", "time", "start_time", "end_time", "date")
)
weight = as_decimal(first(flat, "weight_kg", "weight", "kilograms", "value"))
if moment is None or weight is None:
raise RowError("Pesée incomplète : horodatage et poids sont requis.")
if not 20 < float(weight) < 400:
raise RowError("Poids hors bornes (20-400 kg).")
external_id, digest = self._dedupe_keys(
record, flat, "weight", {"t": moment.isoformat(), "w": str(weight)}
)
if self._existing(db, WeightEntry, user_id, source, external_id, digest):
return UpsertOutcome.DUPLICATE
slot_taken = db.scalar(
select(WeightEntry).where(
WeightEntry.user_id == user_id,
WeightEntry.source == source,
WeightEntry.measured_at == moment,
)
)
if slot_taken is not None:
return UpsertOutcome.DUPLICATE
db.add(
WeightEntry(
user_id=user_id,
source=source,
external_id=external_id,
content_hash=digest,
measured_at=moment,
weight_kg=weight,
body_fat_pct=as_decimal(first(flat, "body_fat_pct", "body_fat")),
raw=dict(record.data or {}),
)
)
db.flush()
return UpsertOutcome.INSERTED
def _apply_workout(
self,
db: Session,
user_id: int,
source: str,
record: IngestRecord,
flat: dict[str, Any],
) -> UpsertOutcome:
started = as_datetime(first(flat, "start_time", "started_at", "begin"))
ended = as_datetime(first(flat, "end_time", "ended_at", "finish"))
if started is None:
raise RowError("Séance sans horodatage de début.")
if ended is None:
duration = as_int(first(flat, "duration_s", "duration_seconds"))
ended = started + dt.timedelta(seconds=duration) if duration else None
if ended is None or ended <= started:
raise RowError("Séance sans durée exploitable.")
external_id, digest = self._dedupe_keys(
record,
flat,
"exercise_session",
{"s": started.isoformat(), "e": ended.isoformat()},
)
if self._existing(db, Workout, user_id, source, external_id, digest):
return UpsertOutcome.DUPLICATE
sport, label = sport_of(flat)
distance = as_decimal(first(flat, "distance_m", "distance", "meters"))
workout = Workout(
user_id=user_id,
source=source,
external_id=external_id,
content_hash=digest,
started_at=started,
ended_at=ended,
sport_type=sport,
sport_label=label,
kcal=as_decimal(
first(flat, "energy_kcal", "calories_kcal", "kcal", "calories")
),
distance_m=int(distance) if distance is not None else None,
steps=as_int(first(flat, "steps", "count")),
avg_hr=as_int(
first(flat, "avg_hr", "average_heart_rate", "heart_rate_avg")
),
max_hr=as_int(first(flat, "max_hr", "max_heart_rate")),
raw=dict(record.data or {}),
)
db.add(workout)
db.flush()
self._flag_overlaps(db, workout)
return UpsertOutcome.INSERTED
@staticmethod
def _flag_overlaps(db: Session, workout: Workout) -> None:
from app.modules.health.service import flag_overlapping_duplicates
flag_overlapping_duplicates(db, workout)
def _apply_nutrition(
self,
db: Session,
user_id: int,
source: str,
record: IngestRecord,
flat: dict[str, Any],
tz: ZoneInfo,
) -> UpsertOutcome:
moment = as_datetime(first(flat, "eaten_at", "start_time", "time", "date"))
kcal = as_decimal(first(flat, "energy_kcal", "energy", "calories", "kcal"))
if moment is None or kcal is None:
raise RowError("Repas incomplet : horodatage et calories sont requis.")
name = str(first(flat, "name", "food_name", "label") or "Repas")[:200]
sodium_mg = as_decimal(first(flat, "sodium_mg"))
if sodium_mg is None:
sodium_g = as_decimal(first(flat, "sodium_g", "sodium"))
sodium_mg = sodium_g * 1000 if sodium_g is not None else None
external_id, digest = self._dedupe_keys(
record,
flat,
"nutrition",
{"t": moment.isoformat(), "n": name, "k": str(kcal)},
)
if self._existing(db, FoodEntry, user_id, source, external_id, digest):
return UpsertOutcome.DUPLICATE
db.add(
FoodEntry(
user_id=user_id,
source=source,
external_id=external_id,
content_hash=digest,
eaten_at=moment,
meal=meal_of(flat, moment, tz),
name=name,
brand=(str(first(flat, "brand") or "")[:100] or None),
quantity=as_decimal(first(flat, "quantity", "serving_quantity"))
or Decimal(1),
unit=str(first(flat, "unit") or "portion")[:20],
kcal=kcal,
protein_g=as_decimal(first(flat, "protein_g", "protein")),
carbs_g=as_decimal(
first(flat, "carbs_g", "total_carbohydrate_g", "total_carbohydrate")
),
fat_g=as_decimal(first(flat, "fat_g", "total_fat_g", "total_fat")),
fiber_g=as_decimal(
first(flat, "fiber_g", "dietary_fiber_g", "dietary_fiber")
),
sugar_g=as_decimal(first(flat, "sugar_g", "sugar")),
sat_fat_g=as_decimal(
first(flat, "sat_fat_g", "saturated_fat_g", "saturated_fat")
),
sodium_mg=sodium_mg,
raw=dict(record.data or {}),
)
)
db.flush()
return UpsertOutcome.INSERTED
def _apply_hydration(
self,
db: Session,
user_id: int,
source: str,
record: IngestRecord,
flat: dict[str, Any],
) -> UpsertOutcome:
moment = as_datetime(first(flat, "drunk_at", "start_time", "time", "date"))
volume = as_decimal(first(flat, "volume_ml", "volume", "milliliters"))
if volume is None:
liters = as_decimal(first(flat, "volume_liters", "liters", "value"))
volume = liters * 1000 if liters is not None else None
if moment is None or volume is None:
raise RowError("Hydratation incomplète : horodatage et volume requis.")
volume_ml = int(volume)
if not 0 < volume_ml <= 5000:
raise RowError("Volume hors bornes (1-5000 ml).")
external_id, digest = self._dedupe_keys(
record, flat, "hydration", {"t": moment.isoformat(), "v": volume_ml}
)
if self._existing(db, WaterEntry, user_id, source, external_id, digest):
return UpsertOutcome.DUPLICATE
db.add(
WaterEntry(
user_id=user_id,
source=source,
external_id=external_id,
content_hash=digest,
drunk_at=moment,
volume_ml=volume_ml,
)
)
db.flush()
return UpsertOutcome.INSERTED
+499
View File
@@ -0,0 +1,499 @@
"""Health module models (santé + nutrition + planning).
Tables follow docs/design/datamodel-health-vape.md (§3-§4), the planning
addendum (docs/design/addendum-planning.md) and CONVENTIONS C2/C8:
- portable column types (suite runs on SQLite): non-native enums, JSONB_V;
- SourceMixin + (user_id, source, external_id) / (user_id, content_hash)
unique constraints on every connector-fed table;
- FK to import_runs.id with ondelete="CASCADE" (central import rollback).
"""
import datetime as dt
import enum
from decimal import Decimal
from typing import Any
from sqlalchemy import (
Boolean,
CheckConstraint,
Date,
DateTime,
ForeignKey,
Index,
Integer,
Numeric,
String,
UniqueConstraint,
text,
)
from sqlalchemy import (
Enum as SAEnum,
)
from sqlalchemy.orm import Mapped, mapped_column
from sqlalchemy.types import TypeDecorator
from app.core.database import Base
from app.core.mixins import JSONB_V, SourceMixin, TimestampMixin
class UTCDateTime(TypeDecorator):
"""`timestamptz` that stays UTC-aware on SQLite too (CONVENTIONS C8).
SQLite has no timezone-aware storage: SQLAlchemy drops the offset on write
and returns naive datetimes on read, which would silently shift any input
sent with a non-UTC offset and break `astimezone()` on read. This decorator
normalizes to UTC on the way in and re-attaches UTC on the way out, so
business code always manipulates aware UTC datetimes on both backends.
"""
impl = DateTime(timezone=True)
cache_ok = True
def process_bind_param(
self, value: dt.datetime | None, dialect: Any
) -> dt.datetime | None:
if value is None:
return None
if value.tzinfo is None:
return value.replace(tzinfo=dt.UTC)
return value.astimezone(dt.UTC)
def process_result_value(
self, value: dt.datetime | None, dialect: Any
) -> dt.datetime | None:
if value is None:
return None
if value.tzinfo is None:
return value.replace(tzinfo=dt.UTC)
return value.astimezone(dt.UTC)
def _enum(enum_cls: type[enum.Enum], name: str) -> SAEnum:
"""Portable enum column: VARCHAR + CHECK, persists the *values* (C8.1).
`create_constraint=True` is explicit: SQLAlchemy defaults it to False, and
without it the column is a bare VARCHAR with no CHECK — on PostgreSQL too.
"""
return SAEnum(
enum_cls,
name=name,
native_enum=False,
create_constraint=True,
validate_strings=True,
values_callable=lambda e: [m.value for m in e],
)
class Sex(str, enum.Enum):
MALE = "male"
FEMALE = "female"
OTHER = "other" # BMR formula: average of male/female offsets (§5.1)
class ActivityLevel(str, enum.Enum):
SEDENTARY = "sedentary" # factor 1.2
LIGHT = "light" # 1.375
MODERATE = "moderate" # 1.55
ACTIVE = "active" # 1.725
VERY_ACTIVE = "very_active" # 1.9
class MealType(str, enum.Enum):
BREAKFAST = "breakfast"
LUNCH = "lunch"
DINNER = "dinner"
SNACK = "snack"
class GoalMode(str, enum.Enum):
WEEKLY_RATE = "weekly_rate" # user sets kg/week -> budget derived
TARGET_DATE = "target_date" # user sets the date -> rate derived
MAINTAIN = "maintain" # budget = TDEE
class GoalStatus(str, enum.Enum):
ACTIVE = "active"
COMPLETED = "completed"
ABANDONED = "abandoned"
class SportType(str, enum.Enum):
TREADMILL_WALK = "treadmill_walk"
TREADMILL_RUN = "treadmill_run"
WALKING = "walking"
RUNNING = "running"
CYCLING = "cycling"
SWIMMING = "swimming"
STRENGTH = "strength"
HIIT = "hiit"
YOGA = "yoga"
HIKING = "hiking"
OTHER = "other"
class ScheduleKind(str, enum.Enum):
WEIGH_IN = "weigh_in"
WORKOUT = "workout"
FOOD_LOG = "food_log"
class HealthProfile(TimestampMixin, Base):
"""One row per user (1-1): physiological parameters for BMR/TDEE."""
__tablename__ = "health_profiles"
__table_args__ = (
UniqueConstraint("user_id", name="uq_health_profiles_user"),
CheckConstraint(
"height_cm > 0 AND height_cm < 300", name="ck_health_profiles_height"
),
)
id: Mapped[int] = mapped_column(primary_key=True)
user_id: Mapped[int] = mapped_column(
ForeignKey("users.id", ondelete="CASCADE"), index=True
)
height_cm: Mapped[Decimal] = mapped_column(Numeric(4, 1))
sex: Mapped[Sex] = mapped_column(_enum(Sex, "sex"))
birthdate: Mapped[dt.date] = mapped_column(Date)
activity_level: Mapped[ActivityLevel] = mapped_column(
_enum(ActivityLevel, "activity_level"), default=ActivityLevel.SEDENTARY
)
timezone: Mapped[str] = mapped_column(String(64), default="Europe/Paris")
water_goal_ml: Mapped[int | None] = mapped_column(Integer, default=2000)
calorie_floor_kcal: Mapped[int | None] = mapped_column(Integer, default=None)
class WeightEntry(TimestampMixin, SourceMixin, Base):
"""Weigh-ins; trend uses the first weigh-in of each local day (§3.2)."""
__tablename__ = "weight_entries"
__table_args__ = (
UniqueConstraint(
"user_id", "source", "external_id", name="uq_weight_entries_external"
),
UniqueConstraint("user_id", "content_hash", name="uq_weight_entries_hash"),
UniqueConstraint(
"user_id", "measured_at", "source", name="uq_weight_entries_user_ts_source"
),
CheckConstraint(
"weight_kg > 20 AND weight_kg < 400", name="ck_weight_entries_range"
),
Index("ix_weight_entries_user_measured", "user_id", "measured_at"),
)
id: Mapped[int] = mapped_column(primary_key=True)
user_id: Mapped[int] = mapped_column(
ForeignKey("users.id", ondelete="CASCADE"), index=True
)
import_run_id: Mapped[int | None] = mapped_column(
ForeignKey("import_runs.id", ondelete="CASCADE"), default=None
)
measured_at: Mapped[dt.datetime] = mapped_column(UTCDateTime())
weight_kg: Mapped[Decimal] = mapped_column(Numeric(5, 2))
body_fat_pct: Mapped[Decimal | None] = mapped_column(Numeric(4, 1), default=None)
muscle_mass_kg: Mapped[Decimal | None] = mapped_column(Numeric(5, 2), default=None)
water_pct: Mapped[Decimal | None] = mapped_column(Numeric(4, 1), default=None)
note: Mapped[str | None] = mapped_column(String(255), default=None)
raw: Mapped[dict[str, Any] | None] = mapped_column(JSONB_V, default=None)
class BodyMeasurement(TimestampMixin, SourceMixin, Base):
"""One row = one measuring session; every site is optional (§3.3)."""
__tablename__ = "body_measurements"
__table_args__ = (
UniqueConstraint(
"user_id", "source", "external_id", name="uq_body_measurements_external"
),
UniqueConstraint("user_id", "content_hash", name="uq_body_measurements_hash"),
Index("ix_body_measurements_user_measured", "user_id", "measured_at"),
)
id: Mapped[int] = mapped_column(primary_key=True)
user_id: Mapped[int] = mapped_column(
ForeignKey("users.id", ondelete="CASCADE"), index=True
)
import_run_id: Mapped[int | None] = mapped_column(
ForeignKey("import_runs.id", ondelete="CASCADE"), default=None
)
measured_at: Mapped[dt.datetime] = mapped_column(UTCDateTime())
neck_cm: Mapped[Decimal | None] = mapped_column(Numeric(4, 1), default=None)
chest_cm: Mapped[Decimal | None] = mapped_column(Numeric(4, 1), default=None)
waist_cm: Mapped[Decimal | None] = mapped_column(Numeric(4, 1), default=None)
hips_cm: Mapped[Decimal | None] = mapped_column(Numeric(4, 1), default=None)
biceps_left_cm: Mapped[Decimal | None] = mapped_column(Numeric(4, 1), default=None)
biceps_right_cm: Mapped[Decimal | None] = mapped_column(Numeric(4, 1), default=None)
thigh_left_cm: Mapped[Decimal | None] = mapped_column(Numeric(4, 1), default=None)
thigh_right_cm: Mapped[Decimal | None] = mapped_column(Numeric(4, 1), default=None)
calf_left_cm: Mapped[Decimal | None] = mapped_column(Numeric(4, 1), default=None)
calf_right_cm: Mapped[Decimal | None] = mapped_column(Numeric(4, 1), default=None)
note: Mapped[str | None] = mapped_column(String(255), default=None)
class DailyActivity(TimestampMixin, SourceMixin, Base):
"""Daily activity aggregates, one row per (user, local day, source) (§3.4).
Cross-source merge happens at read time, field by field, by source
priority (calculations.ACTIVITY_SOURCE_PRIORITY) — rows from one source
never overwrite another source's row.
"""
__tablename__ = "activity_daily"
__table_args__ = (
UniqueConstraint(
"user_id", "date", "source", name="uq_activity_daily_user_date_source"
),
CheckConstraint("steps IS NULL OR steps >= 0", name="ck_activity_daily_steps"),
Index("ix_activity_daily_user_date", "user_id", "date"),
)
id: Mapped[int] = mapped_column(primary_key=True)
user_id: Mapped[int] = mapped_column(
ForeignKey("users.id", ondelete="CASCADE"), index=True
)
import_run_id: Mapped[int | None] = mapped_column(
ForeignKey("import_runs.id", ondelete="CASCADE"), default=None
)
date: Mapped[dt.date] = mapped_column(Date) # local civil day (Europe/Paris)
steps: Mapped[int | None] = mapped_column(Integer, default=None)
active_kcal: Mapped[Decimal | None] = mapped_column(Numeric(7, 1), default=None)
total_kcal: Mapped[Decimal | None] = mapped_column(Numeric(7, 1), default=None)
distance_m: Mapped[int | None] = mapped_column(Integer, default=None)
active_minutes: Mapped[int | None] = mapped_column(Integer, default=None)
floors: Mapped[int | None] = mapped_column(Integer, default=None)
raw: Mapped[dict[str, Any] | None] = mapped_column(JSONB_V, default=None)
class Workout(TimestampMixin, SourceMixin, Base):
"""Sport sessions; cross-source 80 %-overlap dedup flags is_hidden (§3.6)."""
__tablename__ = "workouts"
__table_args__ = (
UniqueConstraint(
"user_id", "source", "external_id", name="uq_workouts_external"
),
UniqueConstraint("user_id", "content_hash", name="uq_workouts_hash"),
CheckConstraint("ended_at > started_at", name="ck_workouts_duration"),
Index("ix_workouts_user_started", "user_id", "started_at"),
)
id: Mapped[int] = mapped_column(primary_key=True)
user_id: Mapped[int] = mapped_column(
ForeignKey("users.id", ondelete="CASCADE"), index=True
)
import_run_id: Mapped[int | None] = mapped_column(
ForeignKey("import_runs.id", ondelete="CASCADE"), default=None
)
started_at: Mapped[dt.datetime] = mapped_column(UTCDateTime())
ended_at: Mapped[dt.datetime] = mapped_column(UTCDateTime())
sport_type: Mapped[SportType] = mapped_column(_enum(SportType, "sport_type"))
sport_label: Mapped[str | None] = mapped_column(String(100), default=None)
kcal: Mapped[Decimal | None] = mapped_column(Numeric(7, 1), default=None)
distance_m: Mapped[int | None] = mapped_column(Integer, default=None)
steps: Mapped[int | None] = mapped_column(Integer, default=None)
avg_hr: Mapped[int | None] = mapped_column(Integer, default=None)
max_hr: Mapped[int | None] = mapped_column(Integer, default=None)
avg_speed_kmh: Mapped[Decimal | None] = mapped_column(Numeric(4, 1), default=None)
elevation_m: Mapped[int | None] = mapped_column(Integer, default=None)
is_hidden: Mapped[bool] = mapped_column(Boolean, default=False)
note: Mapped[str | None] = mapped_column(String(255), default=None)
raw: Mapped[dict[str, Any] | None] = mapped_column(JSONB_V, default=None)
@property
def duration_s(self) -> int:
"""Session length in seconds (derived, never stored)."""
return int((self.ended_at - self.started_at).total_seconds())
class Goal(TimestampMixin, Base):
"""Weight goals, historised; a single active goal per user (§3.7)."""
__tablename__ = "goals"
__table_args__ = (
Index(
"uq_goals_user_active",
"user_id",
unique=True,
sqlite_where=text("status = 'active'"),
postgresql_where=text("status = 'active'"),
),
CheckConstraint(
"weekly_rate_kg IS NULL OR "
"(weekly_rate_kg > -1.01 AND weekly_rate_kg <= 1.5)",
name="ck_goals_rate_sane",
),
CheckConstraint(
"mode <> 'target_date' OR target_date IS NOT NULL",
name="ck_goals_target_date",
),
CheckConstraint(
"mode <> 'weekly_rate' OR weekly_rate_kg IS NOT NULL",
name="ck_goals_weekly_rate",
),
)
id: Mapped[int] = mapped_column(primary_key=True)
user_id: Mapped[int] = mapped_column(
ForeignKey("users.id", ondelete="CASCADE"), index=True
)
mode: Mapped[GoalMode] = mapped_column(_enum(GoalMode, "goal_mode"))
start_date: Mapped[dt.date] = mapped_column(Date)
start_weight_kg: Mapped[Decimal] = mapped_column(Numeric(5, 2))
target_weight_kg: Mapped[Decimal] = mapped_column(Numeric(5, 2))
target_date: Mapped[dt.date | None] = mapped_column(Date, default=None)
weekly_rate_kg: Mapped[Decimal | None] = mapped_column(Numeric(4, 2), default=None)
status: Mapped[GoalStatus] = mapped_column(
_enum(GoalStatus, "goal_status"), default=GoalStatus.ACTIVE
)
note: Mapped[str | None] = mapped_column(String(255), default=None)
class FoodEntry(TimestampMixin, SourceMixin, Base):
"""Food journal; all nutritional values are ABSOLUTE for the eaten
quantity, never per-100 g (§4.1)."""
__tablename__ = "food_entries"
__table_args__ = (
UniqueConstraint(
"user_id", "source", "external_id", name="uq_food_entries_external"
),
UniqueConstraint("user_id", "content_hash", name="uq_food_entries_hash"),
CheckConstraint("kcal >= 0 AND quantity > 0", name="ck_food_entries_positive"),
Index("ix_food_entries_user_eaten", "user_id", "eaten_at"),
)
id: Mapped[int] = mapped_column(primary_key=True)
user_id: Mapped[int] = mapped_column(
ForeignKey("users.id", ondelete="CASCADE"), index=True
)
import_run_id: Mapped[int | None] = mapped_column(
ForeignKey("import_runs.id", ondelete="CASCADE"), default=None
)
eaten_at: Mapped[dt.datetime] = mapped_column(UTCDateTime())
meal: Mapped[MealType] = mapped_column(_enum(MealType, "meal_type"))
name: Mapped[str] = mapped_column(String(200))
brand: Mapped[str | None] = mapped_column(String(100), default=None)
quantity: Mapped[Decimal] = mapped_column(Numeric(8, 2), default=Decimal(1))
unit: Mapped[str] = mapped_column(String(20), default="g")
kcal: Mapped[Decimal] = mapped_column(Numeric(7, 1))
protein_g: Mapped[Decimal | None] = mapped_column(Numeric(6, 1), default=None)
carbs_g: Mapped[Decimal | None] = mapped_column(Numeric(6, 1), default=None)
fat_g: Mapped[Decimal | None] = mapped_column(Numeric(6, 1), default=None)
fiber_g: Mapped[Decimal | None] = mapped_column(Numeric(6, 1), default=None)
sugar_g: Mapped[Decimal | None] = mapped_column(Numeric(6, 1), default=None)
sat_fat_g: Mapped[Decimal | None] = mapped_column(Numeric(6, 1), default=None)
sodium_mg: Mapped[Decimal | None] = mapped_column(Numeric(8, 1), default=None)
raw: Mapped[dict[str, Any] | None] = mapped_column(JSONB_V, default=None)
class FoodFavorite(TimestampMixin, Base):
"""Favorite foods for quick entry; macros stored for default_quantity
(same absolute convention as food_entries, §4.2)."""
__tablename__ = "food_favorites"
__table_args__ = (
UniqueConstraint(
"user_id", "name", "brand", name="uq_food_favorites_user_name_brand"
),
Index("ix_food_favorites_user_usage", "user_id", "use_count"),
)
id: Mapped[int] = mapped_column(primary_key=True)
user_id: Mapped[int] = mapped_column(
ForeignKey("users.id", ondelete="CASCADE"), index=True
)
name: Mapped[str] = mapped_column(String(200))
brand: Mapped[str | None] = mapped_column(String(100), default=None)
default_quantity: Mapped[Decimal] = mapped_column(Numeric(8, 2))
unit: Mapped[str] = mapped_column(String(20), default="g")
kcal: Mapped[Decimal] = mapped_column(Numeric(7, 1))
protein_g: Mapped[Decimal | None] = mapped_column(Numeric(6, 1), default=None)
carbs_g: Mapped[Decimal | None] = mapped_column(Numeric(6, 1), default=None)
fat_g: Mapped[Decimal | None] = mapped_column(Numeric(6, 1), default=None)
fiber_g: Mapped[Decimal | None] = mapped_column(Numeric(6, 1), default=None)
default_meal: Mapped[MealType | None] = mapped_column(
_enum(MealType, "meal_type"), default=None
)
use_count: Mapped[int] = mapped_column(Integer, default=0)
last_used_at: Mapped[dt.datetime | None] = mapped_column(
UTCDateTime(), default=None
)
class WaterEntry(TimestampMixin, SourceMixin, Base):
"""Hydration log (§4.3)."""
__tablename__ = "water_entries"
__table_args__ = (
UniqueConstraint(
"user_id", "source", "external_id", name="uq_water_entries_external"
),
UniqueConstraint("user_id", "content_hash", name="uq_water_entries_hash"),
CheckConstraint(
"volume_ml > 0 AND volume_ml <= 5000", name="ck_water_entries_volume"
),
Index("ix_water_entries_user_drunk", "user_id", "drunk_at"),
)
id: Mapped[int] = mapped_column(primary_key=True)
user_id: Mapped[int] = mapped_column(
ForeignKey("users.id", ondelete="CASCADE"), index=True
)
import_run_id: Mapped[int | None] = mapped_column(
ForeignKey("import_runs.id", ondelete="CASCADE"), default=None
)
drunk_at: Mapped[dt.datetime] = mapped_column(UTCDateTime())
volume_ml: Mapped[int] = mapped_column(Integer)
class FoodItem(TimestampMixin, Base):
"""Local food referential/cache (Open Food Facts / CIQUAL results).
Deliberately NOT user-scoped: it caches public product data shared by all
users (nutrition-sources.md §7.2), keyed by (source, source_id). All
values are per 100 g.
"""
__tablename__ = "food_items"
__table_args__ = (
UniqueConstraint("source", "source_id", name="uq_food_items_source_id"),
Index("ix_food_items_name", "name"),
)
id: Mapped[int] = mapped_column(primary_key=True)
source: Mapped[str] = mapped_column(String(32)) # 'off' | 'ciqual' | 'custom'
source_id: Mapped[str | None] = mapped_column(String(64), default=None)
name: Mapped[str] = mapped_column(String(200))
brand: Mapped[str | None] = mapped_column(String(100), default=None)
energy_kcal_100g: Mapped[Decimal | None] = mapped_column(
Numeric(7, 1), default=None
)
protein_g_100g: Mapped[Decimal | None] = mapped_column(Numeric(6, 1), default=None)
carbs_g_100g: Mapped[Decimal | None] = mapped_column(Numeric(6, 1), default=None)
sugar_g_100g: Mapped[Decimal | None] = mapped_column(Numeric(6, 1), default=None)
fat_g_100g: Mapped[Decimal | None] = mapped_column(Numeric(6, 1), default=None)
sat_fat_g_100g: Mapped[Decimal | None] = mapped_column(Numeric(6, 1), default=None)
fiber_g_100g: Mapped[Decimal | None] = mapped_column(Numeric(6, 1), default=None)
salt_g_100g: Mapped[Decimal | None] = mapped_column(Numeric(6, 2), default=None)
serving_size_g: Mapped[Decimal | None] = mapped_column(Numeric(6, 1), default=None)
raw: Mapped[dict[str, Any] | None] = mapped_column(JSONB_V, default=None)
class TrackingSchedule(TimestampMixin, Base):
"""Weekly habit planning (addendum-planning.md): weigh-in / workout /
food-log days. done/not-done is DERIVED from the data tables — no
check-in table in v1."""
__tablename__ = "tracking_schedules"
__table_args__ = (
UniqueConstraint("user_id", "kind", name="uq_tracking_schedules_user_kind"),
)
id: Mapped[int] = mapped_column(primary_key=True)
user_id: Mapped[int] = mapped_column(
ForeignKey("users.id", ondelete="CASCADE"), index=True
)
kind: Mapped[ScheduleKind] = mapped_column(_enum(ScheduleKind, "schedule_kind"))
# JSON list of ints, 0 = Monday … 6 = Sunday.
weekdays: Mapped[list[int]] = mapped_column(JSONB_V, default=list)
enabled: Mapped[bool] = mapped_column(Boolean, default=True)
+670
View File
@@ -0,0 +1,670 @@
"""HTTP surface of the health module — prefix /api/health (CONVENTIONS C2.1).
Routers stay thin: dependencies + service/stats call + response schema.
Static sub-paths are declared before their `/{id}` siblings on purpose.
"""
import datetime as dt
from typing import Annotated
from fastapi import APIRouter, Depends, Query
from sqlalchemy.orm import Session
from app.core.database import get_db
from app.core.dependencies import get_current_user
from app.core.pagination import Page, PageParams, paginate
from app.modules.auth.models import User
from app.modules.health import calculations as calc
from app.modules.health import foods as foods_service
from app.modules.health import service, stats
from app.modules.health.models import (
GoalStatus,
MealType,
ScheduleKind,
SportType,
)
from app.modules.health.schemas import (
ActiveGoalRead,
AdherenceResponse,
BodyMeasurementCreate,
BodyMeasurementRead,
BodyMeasurementUpdate,
DailyActivityRead,
DailyActivityUpsert,
DashboardResponse,
FoodEntryCreate,
FoodEntryRead,
FoodEntryUpdate,
FoodFavoriteCreate,
FoodFavoriteRead,
FoodFavoriteUpdate,
FoodSearchItem,
FoodSearchResponse,
GoalCreate,
GoalRead,
GoalUpdate,
HealthProfileRead,
HealthProfileUpdate,
MergedActivityRead,
NutritionDayDetail,
NutritionDayRead,
RecentFoodRead,
ScheduleRead,
ScheduleUpdate,
StatsResponse,
TodayResponse,
WaterEntryCreate,
WaterEntryRead,
WeightEntryCreate,
WeightEntryRead,
WeightEntryUpdate,
WorkoutCreate,
WorkoutRead,
WorkoutUpdate,
)
router = APIRouter(prefix="/health", tags=["health"])
DbDep = Annotated[Session, Depends(get_db)]
UserDep = Annotated[User, Depends(get_current_user)]
PageDep = Annotated[PageParams, Depends()]
FromQ = Annotated[dt.date | None, Query(alias="from")]
ToQ = Annotated[dt.date | None, Query()]
TzQ = Annotated[str | None, Query()]
SortQ = Annotated[str | None, Query()]
def _page(items: list, total: int, params: PageParams, schema) -> Page:
return Page(
items=[schema.model_validate(item) for item in items],
total=total,
page=params.page,
page_size=params.page_size,
)
# --- Profile ------------------------------------------------------------------
def _profile_read(db: Session, user_id: int) -> HealthProfileRead:
profile = service.require_profile(db, user_id)
read = HealthProfileRead.model_validate(profile)
tz = service.profile_tz(db, user_id, None)
today = service.today_local(tz)
read.age = calc.age_on(today, profile.birthdate)
trend = calc.weight_trend(service.daily_weights(db, user_id, tz))
weight = trend[-1][1] if trend else None
read.current_weight_kg = None if weight is None else round(weight, 2)
if weight is not None:
read.bmr_kcal = round(
calc.bmr_mifflin(
weight, float(profile.height_cm), read.age, profile.sex.value
),
1,
)
read.tdee_estimated_kcal = round(
read.bmr_kcal * calc.ACTIVITY_FACTORS[profile.activity_level.value], 1
)
bmi = calc.bmi(weight, float(profile.height_cm))
read.bmi = None if bmi is None else round(bmi, 1)
return read
@router.get("/profile", response_model=HealthProfileRead)
def get_profile(db: DbDep, user: UserDep) -> HealthProfileRead:
return _profile_read(db, user.id)
@router.put("/profile", response_model=HealthProfileRead)
def put_profile(
payload: HealthProfileUpdate, db: DbDep, user: UserDep
) -> HealthProfileRead:
service.upsert_profile(db, user.id, payload)
return _profile_read(db, user.id)
# --- Weights ------------------------------------------------------------------
@router.get("/weights/stats", response_model=StatsResponse)
def weights_stats(
db: DbDep,
user: UserDep,
from_: FromQ = None,
to: ToQ = None,
tz: TzQ = None,
) -> StatsResponse:
zone = service.profile_tz(db, user.id, tz)
start, end = service.resolve_range(from_, to, zone, default_days=90)
return stats.weight_stats(db, user.id, start, end, zone)
@router.get("/weights", response_model=Page[WeightEntryRead])
def list_weights(
db: DbDep,
user: UserDep,
params: PageDep,
from_: FromQ = None,
to: ToQ = None,
tz: TzQ = None,
sort: SortQ = None,
) -> Page[WeightEntryRead]:
zone = service.profile_tz(db, user.id, tz)
stmt = service.weights_query(user.id, from_, to, zone, sort)
items, total = paginate(db, stmt, params)
return _page(items, total, params, WeightEntryRead)
@router.post("/weights", response_model=WeightEntryRead, status_code=201)
def create_weight(
payload: WeightEntryCreate, db: DbDep, user: UserDep
) -> WeightEntryRead:
return WeightEntryRead.model_validate(service.create_weight(db, user.id, payload))
@router.patch("/weights/{entry_id}", response_model=WeightEntryRead)
@router.put("/weights/{entry_id}", response_model=WeightEntryRead)
def update_weight(
entry_id: int, payload: WeightEntryUpdate, db: DbDep, user: UserDep
) -> WeightEntryRead:
return WeightEntryRead.model_validate(
service.update_weight(db, user.id, entry_id, payload)
)
@router.delete("/weights/{entry_id}", status_code=204)
def delete_weight(entry_id: int, db: DbDep, user: UserDep) -> None:
service.delete_weight(db, user.id, entry_id)
# --- Body measurements --------------------------------------------------------
@router.get("/measurements/stats", response_model=StatsResponse)
def measurements_stats(
db: DbDep,
user: UserDep,
from_: FromQ = None,
to: ToQ = None,
tz: TzQ = None,
) -> StatsResponse:
zone = service.profile_tz(db, user.id, tz)
start, end = service.resolve_range(from_, to, zone, default_days=180)
return stats.measurement_stats(db, user.id, start, end, zone)
@router.get("/measurements", response_model=Page[BodyMeasurementRead])
def list_measurements(
db: DbDep,
user: UserDep,
params: PageDep,
from_: FromQ = None,
to: ToQ = None,
tz: TzQ = None,
sort: SortQ = None,
) -> Page[BodyMeasurementRead]:
zone = service.profile_tz(db, user.id, tz)
stmt = service.measurements_query(user.id, from_, to, zone, sort)
items, total = paginate(db, stmt, params)
return _page(items, total, params, BodyMeasurementRead)
@router.post("/measurements", response_model=BodyMeasurementRead, status_code=201)
def create_measurement(
payload: BodyMeasurementCreate, db: DbDep, user: UserDep
) -> BodyMeasurementRead:
return BodyMeasurementRead.model_validate(
service.create_measurement(db, user.id, payload)
)
@router.patch("/measurements/{row_id}", response_model=BodyMeasurementRead)
@router.put("/measurements/{row_id}", response_model=BodyMeasurementRead)
def update_measurement(
row_id: int, payload: BodyMeasurementUpdate, db: DbDep, user: UserDep
) -> BodyMeasurementRead:
return BodyMeasurementRead.model_validate(
service.update_measurement(db, user.id, row_id, payload)
)
@router.delete("/measurements/{row_id}", status_code=204)
def delete_measurement(row_id: int, db: DbDep, user: UserDep) -> None:
service.delete_measurement(db, user.id, row_id)
# --- Daily activity -----------------------------------------------------------
@router.get("/activity/stats", response_model=StatsResponse)
def activity_stats(
db: DbDep,
user: UserDep,
from_: FromQ = None,
to: ToQ = None,
tz: TzQ = None,
) -> StatsResponse:
zone = service.profile_tz(db, user.id, tz)
start, end = service.resolve_range(from_, to, zone)
return stats.activity_stats(db, user.id, start, end, zone)
@router.get("/activity", response_model=None)
def list_activity(
db: DbDep,
user: UserDep,
from_: FromQ = None,
to: ToQ = None,
tz: TzQ = None,
raw: Annotated[bool, Query()] = False,
) -> list[MergedActivityRead] | list[DailyActivityRead]:
zone = service.profile_tz(db, user.id, tz)
start, end = service.resolve_range(from_, to, zone)
if raw:
return [
DailyActivityRead.model_validate(row)
for row in service.activity_rows(db, user.id, start, end)
]
merged = service.merged_activity(db, user.id, start, end)
return [
MergedActivityRead(
date=day,
steps=value.steps,
active_kcal=value.active_kcal,
total_kcal=value.total_kcal,
distance_m=value.distance_m,
active_minutes=value.active_minutes,
floors=value.floors,
field_sources=value.field_sources,
)
for day, value in sorted(merged.items())
]
@router.post("/activity", response_model=DailyActivityRead, status_code=201)
def upsert_activity(
payload: DailyActivityUpsert, db: DbDep, user: UserDep
) -> DailyActivityRead:
return DailyActivityRead.model_validate(
service.upsert_manual_activity(db, user.id, payload)
)
@router.delete("/activity/{row_id}", status_code=204)
def delete_activity(row_id: int, db: DbDep, user: UserDep) -> None:
service.delete_activity(db, user.id, row_id)
# --- Workouts -----------------------------------------------------------------
@router.get("/workouts/stats", response_model=StatsResponse)
def workouts_stats(
db: DbDep,
user: UserDep,
from_: FromQ = None,
to: ToQ = None,
tz: TzQ = None,
) -> StatsResponse:
zone = service.profile_tz(db, user.id, tz)
start, end = service.resolve_range(from_, to, zone, default_days=90)
return stats.workout_stats(db, user.id, start, end, zone)
@router.get("/workouts", response_model=Page[WorkoutRead])
def list_workouts(
db: DbDep,
user: UserDep,
params: PageDep,
from_: FromQ = None,
to: ToQ = None,
tz: TzQ = None,
sport_type: Annotated[SportType | None, Query()] = None,
include_hidden: Annotated[bool, Query()] = False,
sort: SortQ = None,
) -> Page[WorkoutRead]:
zone = service.profile_tz(db, user.id, tz)
stmt = service.workouts_query(
user.id,
from_,
to,
zone,
sport_type.value if sport_type else None,
include_hidden,
sort,
)
items, total = paginate(db, stmt, params)
return _page(items, total, params, WorkoutRead)
@router.post("/workouts", response_model=WorkoutRead, status_code=201)
def create_workout(payload: WorkoutCreate, db: DbDep, user: UserDep) -> WorkoutRead:
return WorkoutRead.model_validate(service.create_workout(db, user.id, payload))
@router.patch("/workouts/{workout_id}", response_model=WorkoutRead)
@router.put("/workouts/{workout_id}", response_model=WorkoutRead)
def update_workout(
workout_id: int, payload: WorkoutUpdate, db: DbDep, user: UserDep
) -> WorkoutRead:
return WorkoutRead.model_validate(
service.update_workout(db, user.id, workout_id, payload)
)
@router.delete("/workouts/{workout_id}", status_code=204)
def delete_workout(workout_id: int, db: DbDep, user: UserDep) -> None:
service.delete_workout(db, user.id, workout_id)
# --- Goals --------------------------------------------------------------------
@router.get("/goals/active", response_model=ActiveGoalRead)
def get_active_goal(db: DbDep, user: UserDep, tz: TzQ = None) -> ActiveGoalRead:
zone = service.profile_tz(db, user.id, tz)
return stats.active_goal_view(db, user.id, zone)
@router.get("/goals", response_model=Page[GoalRead])
def list_goals(
db: DbDep,
user: UserDep,
params: PageDep,
status: Annotated[GoalStatus | None, Query()] = None,
sort: SortQ = None,
) -> Page[GoalRead]:
stmt = service.goals_query(user.id, status.value if status else None, sort)
items, total = paginate(db, stmt, params)
return _page(items, total, params, GoalRead)
@router.post("/goals", response_model=GoalRead, status_code=201)
def create_goal(
payload: GoalCreate,
db: DbDep,
user: UserDep,
replace_active: Annotated[bool, Query()] = False,
) -> GoalRead:
return GoalRead.model_validate(
service.create_goal(db, user.id, payload, replace_active)
)
@router.post("/goals/{goal_id}/activate", response_model=GoalRead)
def activate_goal(goal_id: int, db: DbDep, user: UserDep) -> GoalRead:
return GoalRead.model_validate(service.activate_goal(db, user.id, goal_id))
@router.patch("/goals/{goal_id}", response_model=GoalRead)
@router.put("/goals/{goal_id}", response_model=GoalRead)
def update_goal(
goal_id: int, payload: GoalUpdate, db: DbDep, user: UserDep
) -> GoalRead:
return GoalRead.model_validate(service.update_goal(db, user.id, goal_id, payload))
@router.delete("/goals/{goal_id}", status_code=204)
def delete_goal(goal_id: int, db: DbDep, user: UserDep) -> None:
service.delete_goal(db, user.id, goal_id)
# --- Energy balance & dashboard -----------------------------------------------
@router.get("/energy-balance", response_model=StatsResponse)
def energy_balance(
db: DbDep,
user: UserDep,
from_: FromQ = None,
to: ToQ = None,
tz: TzQ = None,
) -> StatsResponse:
zone = service.profile_tz(db, user.id, tz)
start, end = service.resolve_range(from_, to, zone)
return stats.energy_balance_stats(db, user.id, start, end, zone)
@router.get("/dashboard", response_model=DashboardResponse)
def dashboard(db: DbDep, user: UserDep, tz: TzQ = None) -> DashboardResponse:
zone = service.profile_tz(db, user.id, tz)
return stats.dashboard(db, user.id, zone)
# --- Nutrition: journal --------------------------------------------------------
@router.get("/nutrition/stats", response_model=StatsResponse)
def nutrition_stats(
db: DbDep,
user: UserDep,
from_: FromQ = None,
to: ToQ = None,
tz: TzQ = None,
) -> StatsResponse:
zone = service.profile_tz(db, user.id, tz)
start, end = service.resolve_range(from_, to, zone)
return stats.nutrition_stats(db, user.id, start, end, zone)
@router.get("/nutrition/days", response_model=list[NutritionDayRead])
def nutrition_days(
db: DbDep,
user: UserDep,
from_: FromQ = None,
to: ToQ = None,
tz: TzQ = None,
) -> list[NutritionDayRead]:
zone = service.profile_tz(db, user.id, tz)
start, end = service.resolve_range(from_, to, zone)
return stats.nutrition_days(db, user.id, start, end, zone)
@router.get("/nutrition/days/{day}", response_model=NutritionDayDetail)
def nutrition_day(
day: dt.date, db: DbDep, user: UserDep, tz: TzQ = None
) -> NutritionDayDetail:
zone = service.profile_tz(db, user.id, tz)
return stats.nutrition_day_detail(db, user.id, day, zone)
@router.get("/nutrition/recent", response_model=list[RecentFoodRead])
def recent_foods(
db: DbDep,
user: UserDep,
limit: Annotated[int, Query(ge=1, le=50)] = 20,
) -> list[RecentFoodRead]:
return [
RecentFoodRead(
name=entry.name,
brand=entry.brand,
unit=entry.unit,
quantity=float(entry.quantity),
kcal=float(entry.kcal),
protein_g=float(entry.protein_g) if entry.protein_g is not None else None,
carbs_g=float(entry.carbs_g) if entry.carbs_g is not None else None,
fat_g=float(entry.fat_g) if entry.fat_g is not None else None,
meal=entry.meal,
last_eaten_at=entry.eaten_at,
)
for entry in service.recent_foods(db, user.id, limit)
]
@router.get("/nutrition/favorites", response_model=Page[FoodFavoriteRead])
def list_favorites(
db: DbDep, user: UserDep, params: PageDep, sort: SortQ = None
) -> Page[FoodFavoriteRead]:
stmt = service.favorites_query(user.id, sort)
items, total = paginate(db, stmt, params)
return _page(items, total, params, FoodFavoriteRead)
@router.post("/nutrition/favorites", response_model=FoodFavoriteRead, status_code=201)
def create_favorite(
payload: FoodFavoriteCreate, db: DbDep, user: UserDep
) -> FoodFavoriteRead:
return FoodFavoriteRead.model_validate(
service.create_favorite(db, user.id, payload)
)
@router.patch("/nutrition/favorites/{favorite_id}", response_model=FoodFavoriteRead)
@router.put("/nutrition/favorites/{favorite_id}", response_model=FoodFavoriteRead)
def update_favorite(
favorite_id: int, payload: FoodFavoriteUpdate, db: DbDep, user: UserDep
) -> FoodFavoriteRead:
return FoodFavoriteRead.model_validate(
service.update_favorite(db, user.id, favorite_id, payload)
)
@router.delete("/nutrition/favorites/{favorite_id}", status_code=204)
def delete_favorite(favorite_id: int, db: DbDep, user: UserDep) -> None:
service.delete_favorite(db, user.id, favorite_id)
@router.get("/nutrition/water/stats", response_model=StatsResponse)
def water_stats(
db: DbDep,
user: UserDep,
from_: FromQ = None,
to: ToQ = None,
tz: TzQ = None,
) -> StatsResponse:
zone = service.profile_tz(db, user.id, tz)
start, end = service.resolve_range(from_, to, zone)
return stats.water_stats(db, user.id, start, end, zone)
@router.get("/nutrition/water", response_model=Page[WaterEntryRead])
def list_water(
db: DbDep,
user: UserDep,
params: PageDep,
from_: FromQ = None,
to: ToQ = None,
tz: TzQ = None,
sort: SortQ = None,
) -> Page[WaterEntryRead]:
zone = service.profile_tz(db, user.id, tz)
stmt = service.water_query(user.id, from_, to, zone, sort)
items, total = paginate(db, stmt, params)
return _page(items, total, params, WaterEntryRead)
@router.post("/nutrition/water", response_model=WaterEntryRead, status_code=201)
def create_water(payload: WaterEntryCreate, db: DbDep, user: UserDep) -> WaterEntryRead:
return WaterEntryRead.model_validate(service.create_water(db, user.id, payload))
@router.delete("/nutrition/water/{entry_id}", status_code=204)
def delete_water(entry_id: int, db: DbDep, user: UserDep) -> None:
service.delete_water(db, user.id, entry_id)
@router.get("/nutrition/entries", response_model=Page[FoodEntryRead])
def list_food_entries(
db: DbDep,
user: UserDep,
params: PageDep,
day: Annotated[dt.date | None, Query()] = None,
from_: FromQ = None,
to: ToQ = None,
tz: TzQ = None,
meal: Annotated[MealType | None, Query()] = None,
q: Annotated[str | None, Query()] = None,
sort: SortQ = None,
) -> Page[FoodEntryRead]:
zone = service.profile_tz(db, user.id, tz)
stmt = service.food_entries_query(
user.id, zone, day, from_, to, meal.value if meal else None, q, sort
)
items, total = paginate(db, stmt, params)
return _page(items, total, params, FoodEntryRead)
@router.post("/nutrition/entries", response_model=FoodEntryRead, status_code=201)
def create_food_entry(
payload: FoodEntryCreate, db: DbDep, user: UserDep
) -> FoodEntryRead:
return FoodEntryRead.model_validate(service.create_food_entry(db, user.id, payload))
@router.patch("/nutrition/entries/{entry_id}", response_model=FoodEntryRead)
@router.put("/nutrition/entries/{entry_id}", response_model=FoodEntryRead)
def update_food_entry(
entry_id: int, payload: FoodEntryUpdate, db: DbDep, user: UserDep
) -> FoodEntryRead:
return FoodEntryRead.model_validate(
service.update_food_entry(db, user.id, entry_id, payload)
)
@router.delete("/nutrition/entries/{entry_id}", status_code=204)
def delete_food_entry(entry_id: int, db: DbDep, user: UserDep) -> None:
service.delete_food_entry(db, user.id, entry_id)
# --- Food referential (Open Food Facts proxy + local cache) -------------------
@router.get("/foods/search", response_model=FoodSearchResponse)
def search_foods(
db: DbDep,
user: UserDep,
q: Annotated[str, Query(min_length=1, max_length=100)],
limit: Annotated[int, Query(ge=1, le=50)] = 20,
) -> FoodSearchResponse:
return foods_service.search_foods(db, q, limit)
@router.get("/foods/barcode/{barcode}", response_model=FoodSearchItem)
def food_by_barcode(barcode: str, db: DbDep, user: UserDep) -> FoodSearchItem:
return foods_service.food_by_barcode(db, barcode)
# --- Planning (addendum-planning.md) ------------------------------------------
@router.get("/schedules", response_model=list[ScheduleRead])
def list_schedules(db: DbDep, user: UserDep) -> list[ScheduleRead]:
return [
ScheduleRead(
kind=row.kind, weekdays=list(row.weekdays or []), enabled=row.enabled
)
for row in service.list_schedules(db, user.id)
]
@router.put("/schedules/{kind}", response_model=ScheduleRead)
def put_schedule(
kind: ScheduleKind, payload: ScheduleUpdate, db: DbDep, user: UserDep
) -> ScheduleRead:
row = service.upsert_schedule(db, user.id, kind, payload)
return ScheduleRead(
kind=row.kind, weekdays=list(row.weekdays or []), enabled=row.enabled
)
@router.get("/today", response_model=TodayResponse)
def today(db: DbDep, user: UserDep, tz: TzQ = None) -> TodayResponse:
zone = service.profile_tz(db, user.id, tz)
return stats.today_view(db, user.id, zone)
@router.get("/stats/adherence", response_model=AdherenceResponse)
def adherence(
db: DbDep,
user: UserDep,
from_: FromQ = None,
to: ToQ = None,
tz: TzQ = None,
kind: Annotated[ScheduleKind | None, Query()] = None,
) -> AdherenceResponse:
zone = service.profile_tz(db, user.id, tz)
start, end = service.resolve_range(from_, to, zone, default_days=90)
return stats.adherence_stats(
db, user.id, start, end, zone, kind.value if kind else None
)
+604
View File
@@ -0,0 +1,604 @@
"""Pydantic v2 schemas for the health module (CONVENTIONS C2.5).
Numeric columns are exposed as `float` on purpose: the API feeds ECharts
directly and JSON numbers are required (Pydantic serialises Decimal as a
string). Precision stays in the database (Numeric columns).
"""
import datetime as dt
from typing import Any
from pydantic import BaseModel, ConfigDict, Field
from app.core.timeutils import UtcDatetime
from app.modules.health.models import (
ActivityLevel,
GoalMode,
GoalStatus,
MealType,
ScheduleKind,
Sex,
SportType,
)
ORM = ConfigDict(from_attributes=True)
# --- Profile ------------------------------------------------------------------
class HealthProfileUpdate(BaseModel):
"""PUT /health/profile — full singleton upsert."""
height_cm: float = Field(gt=0, lt=300)
sex: Sex
birthdate: dt.date
activity_level: ActivityLevel = ActivityLevel.SEDENTARY
timezone: str = Field(default="Europe/Paris", max_length=64)
water_goal_ml: int | None = Field(default=2000, ge=0, le=20000)
calorie_floor_kcal: int | None = Field(default=None, ge=800, le=5000)
class HealthProfileRead(BaseModel):
model_config = ORM
id: int
height_cm: float
sex: Sex
birthdate: dt.date
activity_level: ActivityLevel
timezone: str
water_goal_ml: int | None
calorie_floor_kcal: int | None
# Derived (not columns)
age: int | None = None
bmr_kcal: float | None = None
tdee_estimated_kcal: float | None = None
current_weight_kg: float | None = None
bmi: float | None = None
# --- Weights ------------------------------------------------------------------
class WeightEntryCreate(BaseModel):
measured_at: UtcDatetime
weight_kg: float = Field(gt=20, lt=400)
body_fat_pct: float | None = Field(default=None, ge=0, le=100)
muscle_mass_kg: float | None = Field(default=None, ge=0, le=200)
water_pct: float | None = Field(default=None, ge=0, le=100)
note: str | None = Field(default=None, max_length=255)
class WeightEntryUpdate(BaseModel):
measured_at: UtcDatetime | None = None
weight_kg: float | None = Field(default=None, gt=20, lt=400)
body_fat_pct: float | None = Field(default=None, ge=0, le=100)
muscle_mass_kg: float | None = Field(default=None, ge=0, le=200)
water_pct: float | None = Field(default=None, ge=0, le=100)
note: str | None = Field(default=None, max_length=255)
class WeightEntryRead(BaseModel):
model_config = ORM
id: int
measured_at: dt.datetime
weight_kg: float
body_fat_pct: float | None
muscle_mass_kg: float | None
water_pct: float | None
note: str | None
source: str
# --- Body measurements --------------------------------------------------------
MEASUREMENT_SITES: tuple[str, ...] = (
"neck_cm",
"chest_cm",
"waist_cm",
"hips_cm",
"biceps_left_cm",
"biceps_right_cm",
"thigh_left_cm",
"thigh_right_cm",
"calf_left_cm",
"calf_right_cm",
)
class BodyMeasurementCreate(BaseModel):
measured_at: UtcDatetime
neck_cm: float | None = Field(default=None, gt=0, lt=300)
chest_cm: float | None = Field(default=None, gt=0, lt=300)
waist_cm: float | None = Field(default=None, gt=0, lt=300)
hips_cm: float | None = Field(default=None, gt=0, lt=300)
biceps_left_cm: float | None = Field(default=None, gt=0, lt=300)
biceps_right_cm: float | None = Field(default=None, gt=0, lt=300)
thigh_left_cm: float | None = Field(default=None, gt=0, lt=300)
thigh_right_cm: float | None = Field(default=None, gt=0, lt=300)
calf_left_cm: float | None = Field(default=None, gt=0, lt=300)
calf_right_cm: float | None = Field(default=None, gt=0, lt=300)
note: str | None = Field(default=None, max_length=255)
class BodyMeasurementUpdate(BodyMeasurementCreate):
measured_at: UtcDatetime | None = None
class BodyMeasurementRead(BaseModel):
model_config = ORM
id: int
measured_at: dt.datetime
neck_cm: float | None
chest_cm: float | None
waist_cm: float | None
hips_cm: float | None
biceps_left_cm: float | None
biceps_right_cm: float | None
thigh_left_cm: float | None
thigh_right_cm: float | None
calf_left_cm: float | None
calf_right_cm: float | None
note: str | None
source: str
# --- Daily activity -----------------------------------------------------------
class DailyActivityUpsert(BaseModel):
"""POST /health/activity — manual upsert of one local day."""
date: dt.date
steps: int | None = Field(default=None, ge=0)
active_kcal: float | None = Field(default=None, ge=0)
total_kcal: float | None = Field(default=None, ge=0)
distance_m: int | None = Field(default=None, ge=0)
active_minutes: int | None = Field(default=None, ge=0)
floors: int | None = Field(default=None, ge=0)
class DailyActivityRead(BaseModel):
model_config = ORM
id: int
date: dt.date
steps: int | None
active_kcal: float | None
total_kcal: float | None
distance_m: int | None
active_minutes: int | None
floors: int | None
source: str
class MergedActivityRead(BaseModel):
"""Cross-source merged day (§3.5) with per-field provenance."""
date: dt.date
steps: int | None = None
active_kcal: float | None = None
total_kcal: float | None = None
distance_m: int | None = None
active_minutes: int | None = None
floors: int | None = None
field_sources: dict[str, str] = Field(default_factory=dict)
# --- Workouts -----------------------------------------------------------------
class WorkoutCreate(BaseModel):
started_at: UtcDatetime
ended_at: UtcDatetime
sport_type: SportType
sport_label: str | None = Field(default=None, max_length=100)
kcal: float | None = Field(default=None, ge=0)
distance_m: int | None = Field(default=None, ge=0)
steps: int | None = Field(default=None, ge=0)
avg_hr: int | None = Field(default=None, ge=0, le=300)
max_hr: int | None = Field(default=None, ge=0, le=300)
avg_speed_kmh: float | None = Field(default=None, ge=0, le=100)
elevation_m: int | None = None
note: str | None = Field(default=None, max_length=255)
class WorkoutUpdate(BaseModel):
started_at: UtcDatetime | None = None
ended_at: UtcDatetime | None = None
sport_type: SportType | None = None
sport_label: str | None = Field(default=None, max_length=100)
kcal: float | None = Field(default=None, ge=0)
distance_m: int | None = Field(default=None, ge=0)
steps: int | None = Field(default=None, ge=0)
avg_hr: int | None = Field(default=None, ge=0, le=300)
max_hr: int | None = Field(default=None, ge=0, le=300)
avg_speed_kmh: float | None = Field(default=None, ge=0, le=100)
elevation_m: int | None = None
note: str | None = Field(default=None, max_length=255)
is_hidden: bool | None = None
class WorkoutRead(BaseModel):
model_config = ORM
id: int
started_at: dt.datetime
ended_at: dt.datetime
duration_s: int
sport_type: SportType
sport_label: str | None
kcal: float | None
distance_m: int | None
steps: int | None
avg_hr: int | None
max_hr: int | None
avg_speed_kmh: float | None
elevation_m: int | None
is_hidden: bool
note: str | None
source: str
# --- Goals --------------------------------------------------------------------
class GoalCreate(BaseModel):
mode: GoalMode
start_date: dt.date | None = None
start_weight_kg: float | None = Field(default=None, gt=20, lt=400)
target_weight_kg: float = Field(gt=20, lt=400)
target_date: dt.date | None = None
weekly_rate_kg: float | None = Field(default=None, gt=-1.01, le=1.5)
note: str | None = Field(default=None, max_length=255)
class GoalUpdate(BaseModel):
mode: GoalMode | None = None
start_date: dt.date | None = None
start_weight_kg: float | None = Field(default=None, gt=20, lt=400)
target_weight_kg: float | None = Field(default=None, gt=20, lt=400)
target_date: dt.date | None = None
weekly_rate_kg: float | None = Field(default=None, gt=-1.01, le=1.5)
status: GoalStatus | None = None
note: str | None = Field(default=None, max_length=255)
class GoalRead(BaseModel):
model_config = ORM
id: int
mode: GoalMode
start_date: dt.date
start_weight_kg: float
target_weight_kg: float
target_date: dt.date | None
weekly_rate_kg: float | None
status: GoalStatus
note: str | None
class ProjectionRead(BaseModel):
status: str # "ok" | "reached" | "not_converging"
date: dt.date | None = None
class BudgetRead(BaseModel):
kcal: float
deficit_target_kcal: float
floor_applied: bool
rate_clamped: bool = False
tdee_kcal: float
tdee_method: str
class ActiveGoalRead(BaseModel):
goal: GoalRead | None = None
budget: BudgetRead | None = None
projection: ProjectionRead | None = None
trend_weight_kg: float | None = None
done_kg: float | None = None
remaining_kg: float | None = None
progress_pct: float | None = None
# --- Nutrition ----------------------------------------------------------------
class FoodEntryCreate(BaseModel):
eaten_at: UtcDatetime
meal: MealType
name: str = Field(min_length=1, max_length=200)
brand: str | None = Field(default=None, max_length=100)
quantity: float = Field(default=100.0, gt=0)
unit: str = Field(default="g", max_length=20)
kcal: float = Field(ge=0)
protein_g: float | None = Field(default=None, ge=0)
carbs_g: float | None = Field(default=None, ge=0)
fat_g: float | None = Field(default=None, ge=0)
fiber_g: float | None = Field(default=None, ge=0)
sugar_g: float | None = Field(default=None, ge=0)
sat_fat_g: float | None = Field(default=None, ge=0)
sodium_mg: float | None = Field(default=None, ge=0)
# Optional shortcut: build the entry from a favorite, prorated on quantity.
favorite_id: int | None = None
class FoodEntryUpdate(BaseModel):
eaten_at: UtcDatetime | None = None
meal: MealType | None = None
name: str | None = Field(default=None, min_length=1, max_length=200)
brand: str | None = Field(default=None, max_length=100)
quantity: float | None = Field(default=None, gt=0)
unit: str | None = Field(default=None, max_length=20)
kcal: float | None = Field(default=None, ge=0)
protein_g: float | None = Field(default=None, ge=0)
carbs_g: float | None = Field(default=None, ge=0)
fat_g: float | None = Field(default=None, ge=0)
fiber_g: float | None = Field(default=None, ge=0)
sugar_g: float | None = Field(default=None, ge=0)
sat_fat_g: float | None = Field(default=None, ge=0)
sodium_mg: float | None = Field(default=None, ge=0)
class FoodEntryRead(BaseModel):
model_config = ORM
id: int
eaten_at: dt.datetime
meal: MealType
name: str
brand: str | None
quantity: float
unit: str
kcal: float
protein_g: float | None
carbs_g: float | None
fat_g: float | None
fiber_g: float | None
sugar_g: float | None
sat_fat_g: float | None
sodium_mg: float | None
source: str
class FoodFavoriteCreate(BaseModel):
name: str = Field(min_length=1, max_length=200)
brand: str | None = Field(default=None, max_length=100)
default_quantity: float = Field(default=100.0, gt=0)
unit: str = Field(default="g", max_length=20)
kcal: float = Field(ge=0)
protein_g: float | None = Field(default=None, ge=0)
carbs_g: float | None = Field(default=None, ge=0)
fat_g: float | None = Field(default=None, ge=0)
fiber_g: float | None = Field(default=None, ge=0)
default_meal: MealType | None = None
class FoodFavoriteUpdate(BaseModel):
name: str | None = Field(default=None, min_length=1, max_length=200)
brand: str | None = Field(default=None, max_length=100)
default_quantity: float | None = Field(default=None, gt=0)
unit: str | None = Field(default=None, max_length=20)
kcal: float | None = Field(default=None, ge=0)
protein_g: float | None = Field(default=None, ge=0)
carbs_g: float | None = Field(default=None, ge=0)
fat_g: float | None = Field(default=None, ge=0)
fiber_g: float | None = Field(default=None, ge=0)
default_meal: MealType | None = None
class FoodFavoriteRead(BaseModel):
model_config = ORM
id: int
name: str
brand: str | None
default_quantity: float
unit: str
kcal: float
protein_g: float | None
carbs_g: float | None
fat_g: float | None
fiber_g: float | None
default_meal: MealType | None
use_count: int
last_used_at: dt.datetime | None
class RecentFoodRead(BaseModel):
"""Distinct (name, brand) recently logged — computed from food_entries."""
name: str
brand: str | None = None
unit: str = "g"
quantity: float | None = None
kcal: float | None = None
protein_g: float | None = None
carbs_g: float | None = None
fat_g: float | None = None
meal: MealType | None = None
last_eaten_at: dt.datetime
class WaterEntryCreate(BaseModel):
drunk_at: UtcDatetime
volume_ml: int = Field(gt=0, le=5000)
class WaterEntryRead(BaseModel):
model_config = ORM
id: int
drunk_at: dt.datetime
volume_ml: int
source: str
class MealTotals(BaseModel):
meal: MealType
kcal: float = 0.0
protein_g: float = 0.0
carbs_g: float = 0.0
fat_g: float = 0.0
entries: list[FoodEntryRead] = Field(default_factory=list)
class NutritionDayRead(BaseModel):
date: dt.date
kcal: float | None = None
protein_g: float | None = None
carbs_g: float | None = None
fat_g: float | None = None
fiber_g: float | None = None
budget_kcal: float | None = None
vs_budget_kcal: float | None = None
class NutritionDayDetail(BaseModel):
date: dt.date
totals: NutritionDayRead
water_ml: int = 0
water_goal_ml: int | None = None
meals: list[MealTotals] = Field(default_factory=list)
# --- Food referential / search -------------------------------------------------
class FoodSearchItem(BaseModel):
model_config = ORM
id: int | None = None
source: str
source_id: str | None = None
name: str
brand: str | None = None
energy_kcal_100g: float | None = None
protein_g_100g: float | None = None
carbs_g_100g: float | None = None
sugar_g_100g: float | None = None
fat_g_100g: float | None = None
sat_fat_g_100g: float | None = None
fiber_g_100g: float | None = None
salt_g_100g: float | None = None
serving_size_g: float | None = None
class FoodSearchResponse(BaseModel):
query: str
items: list[FoodSearchItem] = Field(default_factory=list)
# "cache" when Open Food Facts was not called, "cache+off" otherwise.
origin: str = "cache"
# --- Planning (addendum-planning.md) ------------------------------------------
class ScheduleRead(BaseModel):
model_config = ORM
kind: ScheduleKind
weekdays: list[int] = Field(default_factory=list)
enabled: bool = False
class ScheduleUpdate(BaseModel):
weekdays: list[int] = Field(default_factory=list)
enabled: bool = True
class StreakRead(BaseModel):
current: int = 0
best: int = 0
class TodayItem(BaseModel):
kind: ScheduleKind
planned: bool
done: bool
value: float | None = None # kg / sessions count / kcal
class TodayResponse(BaseModel):
date: dt.date
items: list[TodayItem] = Field(default_factory=list)
streaks: dict[str, StreakRead] = Field(default_factory=dict)
class AdherenceDay(BaseModel):
date: dt.date
planned: bool
done: bool
status: str # "done" | "missed" | "done_unplanned" | "rest"
class AdherenceKind(BaseModel):
kind: ScheduleKind
weekdays: list[int] = Field(default_factory=list)
enabled: bool = False
planned_days: int = 0
done_days: int = 0
missed_days: int = 0
adherence_pct: float | None = None
streak: StreakRead = Field(default_factory=StreakRead)
days: list[AdherenceDay] = Field(default_factory=list)
# --- Chart-ready stats contract (§8.1) ----------------------------------------
class Series(BaseModel):
name: str
type: str # "line" | "bar" | "scatter" | "pie" | "heatmap"
points: list[list[Any]] = Field(default_factory=list) # [[date_iso, value|null]]
class StatsResponse(BaseModel):
model_config = ConfigDict(populate_by_name=True)
from_: dt.date = Field(alias="from")
to: dt.date
unit: str
series: list[Series] = Field(default_factory=list)
meta: dict[str, Any] = Field(default_factory=dict)
class AdherenceResponse(StatsResponse):
"""`/health/stats/adherence` — a `stats/*` endpoint, so it carries the §8.1
envelope (one calendar-heatmap series per habit) plus the richer per-habit
breakdown of addendum-planning.md that the planning editor needs."""
kinds: list[AdherenceKind] = Field(default_factory=list)
# --- Dashboard ----------------------------------------------------------------
class DashboardResponse(BaseModel):
date: dt.date
weight_kg: float | None = None
weight_measured_at: dt.datetime | None = None
trend_weight_kg: float | None = None
trend_delta_7d_kg: float | None = None
bmi: float | None = None
intake_kcal: float | None = None
budget_kcal: float | None = None
remaining_kcal: float | None = None
tdee_kcal: float | None = None
tdee_method: str | None = None
balance_kcal: float | None = None
cumulative_balance_30d_kcal: float | None = None
steps: int | None = None
active_kcal: float | None = None
distance_m: int | None = None
water_ml: int = 0
water_goal_ml: int | None = None
workouts_this_week: int = 0
goal: GoalRead | None = None
projection: ProjectionRead | None = None
weight_series: list[list[Any]] = Field(default_factory=list)
today: TodayResponse | None = None
File diff suppressed because it is too large Load Diff
+945
View File
@@ -0,0 +1,945 @@
"""Chart-ready aggregations for the health module.
Every `stats/*` response follows the contract of datamodel-health-vape.md §8.1:
{from, to, unit, series:[{name, type, points:[[date, value|null]]}], meta}.
Points are ready for `dataset.source` in ECharts; series names are stable
identifiers (the UI maps them to French labels).
"""
import datetime as dt
from dataclasses import dataclass
from zoneinfo import ZoneInfo
from sqlalchemy.orm import Session
from app.modules.health import calculations as calc
from app.modules.health import service
from app.modules.health.models import Goal, HealthProfile, MealType, ScheduleKind
from app.modules.health.schemas import (
AdherenceDay,
AdherenceKind,
AdherenceResponse,
BudgetRead,
DashboardResponse,
GoalRead,
MealTotals,
NutritionDayDetail,
NutritionDayRead,
ProjectionRead,
Series,
StatsResponse,
StreakRead,
TodayItem,
TodayResponse,
)
STREAK_WINDOW_DAYS = 365
MACRO_KCAL = {"protein_g": 4.0, "carbs_g": 4.0, "fat_g": 9.0}
# Calendar-heatmap encoding of an adherence day. `None` (rest) keeps the cell
# empty in ECharts; the UI colours 0/1/2 via visualMap pieces.
ADHERENCE_STATUS_CODES: dict[str, int | None] = {
"rest": None,
"missed": 0,
"done_unplanned": 1,
"done": 2,
}
def _iso(day: dt.date) -> str:
return day.isoformat()
def _round(value: float | None, digits: int = 1) -> float | None:
return None if value is None else round(value, digits)
# --- Daily energy model -------------------------------------------------------
@dataclass
class DailyModel:
day: dt.date
trend_weight_kg: float | None = None
bmr_kcal: float | None = None
tdee_kcal: float | None = None
tdee_method: str | None = None
tdee_smoothed_kcal: float | None = None
intake_kcal: float | None = None
budget_kcal: float | None = None
deficit_target_kcal: float | None = None
balance_kcal: float | None = None
steps: int | None = None
active_kcal: float | None = None
total_kcal: float | None = None
distance_m: int | None = None
floor_applied: bool = False
rate_clamped: bool = False
def build_daily_models(
db: Session,
user_id: int,
start: dt.date,
end: dt.date,
tz: ZoneInfo,
profile: HealthProfile | None = None,
goal: Goal | None = None,
) -> list[DailyModel]:
"""One model per local day of [start, end], TDEE smoothed over 7 days."""
if profile is None:
profile = service.get_profile(db, user_id)
if goal is None:
goal = service.active_goal(db, user_id)
warmup = start - dt.timedelta(days=calc.TDEE_SMOOTHING_DAYS - 1)
days = calc.date_range(warmup, end)
trend = calc.weight_trend(service.daily_weights(db, user_id, tz, until=end))
activity = service.merged_activity(db, user_id, warmup, end)
intake = service.intake_by_day(db, user_id, warmup, end, tz)
models: list[DailyModel] = []
for day in days:
model = DailyModel(day=day)
merged = activity.get(day)
if merged is not None:
model.steps = merged.steps
model.active_kcal = merged.active_kcal
model.total_kcal = merged.total_kcal
model.distance_m = merged.distance_m
model.trend_weight_kg = calc.trend_at_day(trend, day)
if profile is not None and model.trend_weight_kg is not None:
model.bmr_kcal = calc.bmr_mifflin(
model.trend_weight_kg,
float(profile.height_cm),
calc.age_on(day, profile.birthdate),
profile.sex.value,
)
tdee = calc.tdee_effective(
model.bmr_kcal,
profile.activity_level.value,
total_kcal=model.total_kcal,
active_kcal=model.active_kcal,
)
model.tdee_kcal = tdee.kcal
model.tdee_method = tdee.method
bucket = intake.get(day)
model.intake_kcal = bucket["kcal"] if bucket else None
models.append(model)
smoothed = calc.moving_average(
[m.tdee_kcal for m in models], calc.TDEE_SMOOTHING_DAYS
)
for model, value in zip(models, smoothed, strict=True):
model.tdee_smoothed_kcal = value
model.balance_kcal = calc.energy_balance(model.intake_kcal, model.tdee_kcal)
if profile is None or value is None:
continue
rate = calc.resolve_goal_rate(
mode=goal.mode.value if goal else "maintain",
day=model.day,
trend_now=model.trend_weight_kg or 0.0,
target_weight_kg=float(goal.target_weight_kg) if goal else 0.0,
weekly_rate_kg=float(goal.weekly_rate_kg)
if goal and goal.weekly_rate_kg is not None
else None,
target_date=goal.target_date if goal else None,
)
budget = calc.daily_budget(
value, rate, profile.sex.value, profile.calorie_floor_kcal
)
model.budget_kcal = budget.kcal
model.deficit_target_kcal = budget.deficit_target
model.floor_applied = budget.floor_applied
model.rate_clamped = budget.rate_clamped
return [m for m in models if m.day >= start]
def plan_end_date(goal: Goal) -> dt.date | None:
"""Day the PLAN reaches the target weight (§5.6b) — never the chart bound.
`target_date` goals carry it; `weekly_rate` goals derive it from the rate
(`days = delta / (weekly_rate_kg / 7)`). Returns None when the plan cannot
converge (maintain, zero rate, or a rate pushing away from the target).
"""
if goal.target_date is not None:
return goal.target_date
rate = float(goal.weekly_rate_kg) if goal.weekly_rate_kg is not None else 0.0
delta = float(goal.start_weight_kg) - float(goal.target_weight_kg)
if rate == 0.0 or delta == 0.0 or delta / rate <= 0:
return None
return goal.start_date + dt.timedelta(days=round(delta / rate * 7))
# --- Weight -------------------------------------------------------------------
def weight_stats(
db: Session, user_id: int, start: dt.date, end: dt.date, tz: ZoneInfo
) -> StatsResponse:
raw_all = service.daily_weights(db, user_id, tz, until=end)
trend_all = calc.weight_trend(raw_all)
raw = [(d, w) for d, w in raw_all if start <= d <= end]
trend = [(d, w) for d, w in trend_all if start <= d <= end]
goal = service.active_goal(db, user_id)
series = [
Series(
name="weight_raw",
type="scatter",
points=[[_iso(d), w] for d, w in raw],
),
Series(
name="weight_trend",
type="line",
points=[[_iso(d), w] for d, w in trend],
),
]
slope_14 = calc.regression_slope(
[p for p in trend_all if p[0] > end - dt.timedelta(days=14)]
)
slope_30 = calc.regression_slope(
[p for p in trend_all if p[0] > end - dt.timedelta(days=30)]
)
trend_now = trend_all[-1][1] if trend_all else None
projection = calc.Projection(status="not_converging")
if trend_now is not None and goal is not None:
projection = calc.project_target_date(
trend_now, float(goal.target_weight_kg), slope_30 or slope_14, end
)
plan_end = plan_end_date(goal)
if plan_end is not None:
series.append(
Series(
name="plan_line",
type="line",
points=[
[_iso(goal.start_date), float(goal.start_weight_kg)],
[_iso(plan_end), float(goal.target_weight_kg)],
],
)
)
if projection.status == "ok" and projection.date is not None:
series.append(
Series(
name="projection",
type="line",
points=[
[_iso(trend_all[-1][0]), trend_now],
[_iso(projection.date), float(goal.target_weight_kg)],
],
)
)
profile = service.get_profile(db, user_id)
bmi = (
calc.bmi(trend_now, float(profile.height_cm))
if trend_now is not None and profile is not None
else None
)
meta = {
"trend_now_kg": _round(trend_now, 2),
"last_weight_kg": _round(raw_all[-1][1], 2) if raw_all else None,
"slope_14d_kg_day": _round(slope_14, 4),
"slope_14d_kg_week": _round(slope_14 * 7, 3) if slope_14 is not None else None,
"slope_30d_kg_day": _round(slope_30, 4),
"slope_30d_kg_week": _round(slope_30 * 7, 3) if slope_30 is not None else None,
"total_change_kg": _round(trend[-1][1] - trend[0][1], 2)
if len(trend) > 1
else None,
"bmi": _round(bmi, 1),
"target_weight_kg": float(goal.target_weight_kg) if goal else None,
"plan_end_date": _iso(plan_end_date(goal))
if goal and plan_end_date(goal)
else None,
"projection": {
"status": projection.status,
"date": _iso(projection.date) if projection.date else None,
},
"count": len(raw),
}
return StatsResponse(from_=start, to=end, unit="kg", series=series, meta=meta)
def measurement_stats(
db: Session, user_id: int, start: dt.date, end: dt.date, tz: ZoneInfo
) -> StatsResponse:
from app.modules.health.schemas import MEASUREMENT_SITES
rows = service.measurements_between(db, user_id, start, end, tz)
series: list[Series] = []
for site in MEASUREMENT_SITES:
points = [
[_iso(service.local_day_of(row.measured_at, tz)), float(getattr(row, site))]
for row in rows
if getattr(row, site) is not None
]
if points:
series.append(Series(name=site, type="line", points=points))
profile = service.get_profile(db, user_id)
navy_points: list[list] = []
if profile is not None:
for row in rows:
value = calc.navy_body_fat_pct(
profile.sex.value,
float(profile.height_cm),
float(row.waist_cm) if row.waist_cm is not None else None,
float(row.neck_cm) if row.neck_cm is not None else None,
float(row.hips_cm) if row.hips_cm is not None else None,
)
if value is not None:
navy_points.append(
[_iso(service.local_day_of(row.measured_at, tz)), round(value, 1)]
)
if navy_points:
series.append(Series(name="body_fat_navy_pct", type="line", points=navy_points))
return StatsResponse(
from_=start,
to=end,
unit="cm",
series=series,
meta={"count": len(rows), "sites": [s.name for s in series]},
)
# --- Activity -----------------------------------------------------------------
def activity_stats(
db: Session, user_id: int, start: dt.date, end: dt.date, tz: ZoneInfo
) -> StatsResponse:
merged = service.merged_activity(db, user_id, start, end)
days = calc.date_range(start, end)
fields = ("steps", "active_kcal", "total_kcal", "distance_m")
series: list[Series] = []
meta: dict = {}
for field in fields:
values: list[float | None] = []
for day in days:
row = merged.get(day)
value = getattr(row, field) if row else None
values.append(float(value) if value is not None else None)
series.append(
Series(
name=field,
type="bar" if field in {"steps", "active_kcal"} else "line",
points=[[_iso(d), v] for d, v in zip(days, values, strict=True)],
)
)
ma = calc.moving_average(values, 7)
series.append(
Series(
name=f"{field}_ma7",
type="line",
points=[[_iso(d), _round(v)] for d, v in zip(days, ma, strict=True)],
)
)
tracked = [v for v in values if v is not None]
meta[f"avg_{field}"] = _round(sum(tracked) / len(tracked)) if tracked else None
meta[f"total_{field}"] = _round(sum(tracked)) if tracked else None
meta["tracked_days"] = len(merged)
meta["days"] = len(days)
return StatsResponse(from_=start, to=end, unit="mixed", series=series, meta=meta)
def workout_stats(
db: Session, user_id: int, start: dt.date, end: dt.date, tz: ZoneInfo
) -> StatsResponse:
workouts = service.workouts_between(db, user_id, start, end, tz)
weekly: dict[dt.date, dict[str, float]] = {}
by_sport: dict[str, dict[str, float]] = {}
for workout in workouts:
day = service.local_day_of(workout.started_at, tz)
week_start = day - dt.timedelta(days=day.weekday())
bucket = weekly.setdefault(
week_start,
{
"sessions_count": 0.0,
"total_kcal": 0.0,
"total_distance_m": 0.0,
"total_duration_min": 0.0,
},
)
bucket["sessions_count"] += 1
bucket["total_kcal"] += float(workout.kcal or 0)
bucket["total_distance_m"] += float(workout.distance_m or 0)
bucket["total_duration_min"] += service.workout_duration_s(workout) / 60
sport = workout.sport_type.value
sbucket = by_sport.setdefault(
sport, {"sessions": 0.0, "duration_min": 0.0, "kcal": 0.0}
)
sbucket["sessions"] += 1
sbucket["duration_min"] += service.workout_duration_s(workout) / 60
sbucket["kcal"] += float(workout.kcal or 0)
weeks = sorted(weekly)
series = [
Series(
name=name,
type="bar",
points=[[_iso(week), _round(weekly[week][name])] for week in weeks],
)
for name in (
"sessions_count",
"total_kcal",
"total_distance_m",
"total_duration_min",
)
]
series.append(
Series(
name="by_sport_duration_min",
type="pie",
points=[
[sport, _round(values["duration_min"])]
for sport, values in sorted(
by_sport.items(),
key=lambda item: item[1]["duration_min"],
reverse=True,
)
],
)
)
total_duration = sum(service.workout_duration_s(w) for w in workouts) / 60
meta = {
"sessions": len(workouts),
"total_duration_min": _round(total_duration),
"total_kcal": _round(sum(float(w.kcal or 0) for w in workouts)),
"total_distance_m": _round(sum(float(w.distance_m or 0) for w in workouts)),
"by_sport": by_sport,
}
return StatsResponse(from_=start, to=end, unit="mixed", series=series, meta=meta)
# --- Energy balance -----------------------------------------------------------
def energy_balance_stats(
db: Session, user_id: int, start: dt.date, end: dt.date, tz: ZoneInfo
) -> StatsResponse:
profile = service.get_profile(db, user_id)
models = build_daily_models(db, user_id, start, end, tz, profile=profile)
days = [m.day for m in models]
cumulative: list[float | None] = []
running = 0.0
for model in models:
if model.balance_kcal is not None:
running += model.balance_kcal
cumulative.append(round(running, 1))
series = [
Series(
name="intake_kcal",
type="bar",
points=[[_iso(m.day), _round(m.intake_kcal)] for m in models],
),
Series(
name="tdee_kcal",
type="line",
points=[[_iso(m.day), _round(m.tdee_kcal)] for m in models],
),
Series(
name="balance_kcal",
type="bar",
points=[[_iso(m.day), _round(m.balance_kcal)] for m in models],
),
Series(
name="budget_kcal",
type="line",
points=[[_iso(m.day), _round(m.budget_kcal)] for m in models],
),
Series(
name="cumulative_balance_kcal",
type="line",
points=[[_iso(d), v] for d, v in zip(days, cumulative, strict=True)],
),
]
tracked = [
(m.intake_kcal, m.tdee_kcal)
for m in models
if m.intake_kcal is not None and m.tdee_kcal is not None
]
trend_all = calc.weight_trend(service.daily_weights(db, user_id, tz, until=end))
calibration = calc.tdee_calibration(
tracked,
calc.trend_at_day(trend_all, start),
calc.trend_at_day(trend_all, end),
)
meta = {
"tdee_methods": {
_iso(m.day): m.tdee_method for m in models if m.tdee_method is not None
},
"cumulative_balance_kcal": _round(running),
"cumulative_kg_equivalent": _round(running / calc.KCAL_PER_KG_FAT, 2),
# Daily deficit aimed at by the active goal on the last day of the range
# (the UI draws it as the target line of the balance chart).
"deficit_target_kcal": _round(models[-1].deficit_target_kcal)
if models
else None,
"tracked_days": calibration.tracked_days,
"days": len(models),
"profile_missing": profile is None,
"calibration_status": calibration.status,
"expected_change_kg": _round(calibration.expected_change_kg, 2),
"actual_change_kg": _round(calibration.actual_change_kg, 2),
"gap_kg": _round(calibration.gap_kg, 2),
"tdee_adaptive_kcal": _round(calibration.tdee_adaptive_kcal),
"tdee_correction_kcal": _round(calibration.tdee_correction_kcal),
}
return StatsResponse(from_=start, to=end, unit="kcal", series=series, meta=meta)
# --- Nutrition ----------------------------------------------------------------
def nutrition_stats(
db: Session, user_id: int, start: dt.date, end: dt.date, tz: ZoneInfo
) -> StatsResponse:
models = build_daily_models(db, user_id, start, end, tz)
budgets = {m.day: m.budget_kcal for m in models}
totals = service.intake_by_day(db, user_id, start, end, tz)
meals = service.meal_kcal_by_day(db, user_id, start, end, tz)
days = calc.date_range(start, end)
series = [
Series(
name="kcal",
type="bar",
points=[
[_iso(d), _round(totals[d]["kcal"]) if d in totals else None]
for d in days
],
),
Series(
name="budget_kcal",
type="line",
points=[[_iso(d), _round(budgets.get(d))] for d in days],
),
]
for macro, factor in MACRO_KCAL.items():
series.append(
Series(
name=f"{macro}_kcal",
type="bar",
points=[
[
_iso(d),
_round(totals[d][macro] * factor) if d in totals else None,
]
for d in days
],
)
)
series.append(
Series(
name=macro,
type="bar",
points=[
[_iso(d), _round(totals[d][macro]) if d in totals else None]
for d in days
],
)
)
for meal in MealType:
series.append(
Series(
name=f"meal_{meal.value}_kcal",
type="bar",
points=[
[
_iso(d),
_round(meals[d][meal.value]) if d in meals else None,
]
for d in days
],
)
)
ranked = service.top_foods(db, user_id, start, end, tz)
series.append(
Series(
name="top_foods_kcal",
type="bar",
points=[[item["name"], _round(item["kcal"])] for item in ranked],
)
)
tracked = [totals[d] for d in days if d in totals]
tracked_kcal = [t["kcal"] for t in tracked]
macro_totals = {macro: sum(t[macro] for t in tracked) for macro in MACRO_KCAL}
macro_kcal_total = sum(macro_totals[m] * MACRO_KCAL[m] for m in MACRO_KCAL)
gaps = [
totals[d]["kcal"] - budgets[d]
for d in days
if d in totals and budgets.get(d) is not None
]
meta = {
"tracked_days": len(tracked),
"days": len(days),
"avg_kcal": _round(sum(tracked_kcal) / len(tracked_kcal))
if tracked_kcal
else None,
"avg_protein_g": _round(macro_totals["protein_g"] / len(tracked))
if tracked
else None,
"avg_carbs_g": _round(macro_totals["carbs_g"] / len(tracked))
if tracked
else None,
"avg_fat_g": _round(macro_totals["fat_g"] / len(tracked)) if tracked else None,
"macro_split_pct": {
macro: _round(
100 * macro_totals[macro] * MACRO_KCAL[macro] / macro_kcal_total
)
for macro in MACRO_KCAL
}
if macro_kcal_total > 0
else {},
"days_in_budget": sum(1 for gap in gaps if gap <= 0),
"days_with_budget": len(gaps),
"avg_gap_kcal": _round(sum(gaps) / len(gaps)) if gaps else None,
# Rounded like every other kcal figure: raw float sums otherwise leak
# binary artefacts (3667.9000000000005) straight into the UI.
"top_foods": [{**item, "kcal": _round(item["kcal"])} for item in ranked],
}
return StatsResponse(from_=start, to=end, unit="kcal", series=series, meta=meta)
def nutrition_days(
db: Session, user_id: int, start: dt.date, end: dt.date, tz: ZoneInfo
) -> list[NutritionDayRead]:
models = build_daily_models(db, user_id, start, end, tz)
totals = service.intake_by_day(db, user_id, start, end, tz)
out: list[NutritionDayRead] = []
for model in models:
bucket = totals.get(model.day)
vs_budget = (
bucket["kcal"] - model.budget_kcal
if bucket is not None and model.budget_kcal is not None
else None
)
out.append(
NutritionDayRead(
date=model.day,
kcal=_round(bucket["kcal"]) if bucket else None,
protein_g=_round(bucket["protein_g"]) if bucket else None,
carbs_g=_round(bucket["carbs_g"]) if bucket else None,
fat_g=_round(bucket["fat_g"]) if bucket else None,
fiber_g=_round(bucket["fiber_g"]) if bucket else None,
budget_kcal=_round(model.budget_kcal),
vs_budget_kcal=_round(vs_budget),
)
)
return out
def nutrition_day_detail(
db: Session, user_id: int, day: dt.date, tz: ZoneInfo
) -> NutritionDayDetail:
from app.modules.health.schemas import FoodEntryRead
entries = service.food_entries_between(db, user_id, day, day, tz)
meals: dict[str, MealTotals] = {
meal.value: MealTotals(meal=meal) for meal in MealType
}
for entry in entries:
bucket = meals[entry.meal.value]
bucket.kcal += float(entry.kcal)
bucket.protein_g += float(entry.protein_g or 0)
bucket.carbs_g += float(entry.carbs_g or 0)
bucket.fat_g += float(entry.fat_g or 0)
bucket.entries.append(FoodEntryRead.model_validate(entry))
for bucket in meals.values():
bucket.kcal = round(bucket.kcal, 1)
bucket.protein_g = round(bucket.protein_g, 1)
bucket.carbs_g = round(bucket.carbs_g, 1)
bucket.fat_g = round(bucket.fat_g, 1)
totals = nutrition_days(db, user_id, day, day, tz)[0]
profile = service.get_profile(db, user_id)
water = sum(
row.volume_ml for row in service.water_between(db, user_id, day, day, tz)
)
return NutritionDayDetail(
date=day,
totals=totals,
water_ml=water,
water_goal_ml=profile.water_goal_ml if profile else None,
meals=list(meals.values()),
)
def water_stats(
db: Session, user_id: int, start: dt.date, end: dt.date, tz: ZoneInfo
) -> StatsResponse:
rows = service.water_between(db, user_id, start, end, tz)
per_day: dict[dt.date, int] = {}
for row in rows:
day = service.local_day_of(row.drunk_at, tz)
per_day[day] = per_day.get(day, 0) + row.volume_ml
days = calc.date_range(start, end)
profile = service.get_profile(db, user_id)
goal_ml = profile.water_goal_ml if profile else None
series = [
Series(
name="volume_ml",
type="bar",
points=[[_iso(d), per_day.get(d, 0)] for d in days],
),
Series(
name="goal_ml",
type="line",
points=[[_iso(d), goal_ml] for d in days],
),
]
tracked = [v for v in per_day.values()]
meta = {
"goal_ml": goal_ml,
"avg_ml": _round(sum(tracked) / len(tracked)) if tracked else None,
"tracked_days": len(tracked),
"days_goal_reached": sum(1 for v in tracked if goal_ml and v >= goal_ml),
}
return StatsResponse(from_=start, to=end, unit="ml", series=series, meta=meta)
# --- Planning: today & adherence ----------------------------------------------
def today_view(db: Session, user_id: int, tz: ZoneInfo) -> TodayResponse:
today = service.today_local(tz)
start = today - dt.timedelta(days=STREAK_WINDOW_DAYS)
schedules = service.schedule_map(db, user_id)
done = service.done_days_by_kind(db, user_id, start, today, tz)
items: list[TodayItem] = []
streaks: dict[str, StreakRead] = {}
for kind in calc.SCHEDULE_KINDS:
schedule = schedules[kind]
days = calc.build_adherence(
start, today, schedule.weekdays, schedule.enabled, set(done[kind])
)
streak = calc.compute_streaks(days, today)
streaks[kind] = StreakRead(current=streak.current, best=streak.best)
items.append(
TodayItem(
kind=ScheduleKind(kind),
planned=calc.is_planned(schedule.weekdays, today, schedule.enabled),
done=today in done[kind],
value=_round(done[kind].get(today), 2),
)
)
return TodayResponse(date=today, items=items, streaks=streaks)
def adherence_stats(
db: Session,
user_id: int,
start: dt.date,
end: dt.date,
tz: ZoneInfo,
kind: str | None = None,
) -> AdherenceResponse:
today = service.today_local(tz)
schedules = service.schedule_map(db, user_id)
# Streaks look further back than the requested window on purpose.
streak_start = min(start, end - dt.timedelta(days=STREAK_WINDOW_DAYS))
done = service.done_days_by_kind(db, user_id, streak_start, end, tz)
kinds: list[AdherenceKind] = []
for name in calc.SCHEDULE_KINDS:
if kind and name != kind:
continue
schedule = schedules[name]
done_days = set(done[name])
window = calc.build_adherence(
start, end, schedule.weekdays, schedule.enabled, done_days
)
full = calc.build_adherence(
streak_start, end, schedule.weekdays, schedule.enabled, done_days
)
streak = calc.compute_streaks(full, today)
planned = [d for d in window if d.planned]
kinds.append(
AdherenceKind(
kind=ScheduleKind(name),
weekdays=list(schedule.weekdays or []),
enabled=schedule.enabled,
planned_days=len(planned),
done_days=sum(1 for d in planned if d.done),
missed_days=sum(1 for d in planned if d.missed),
adherence_pct=_round(calc.adherence_pct(window)),
streak=StreakRead(current=streak.current, best=streak.best),
days=[
AdherenceDay(
date=d.day, planned=d.planned, done=d.done, status=d.status
)
for d in window
],
)
)
# §8.1 envelope: one calendar-heatmap series per habit, points [day, code].
series = [
Series(
name=item.kind.value,
type="heatmap",
points=[
[_iso(d.date), ADHERENCE_STATUS_CODES[d.status]] for d in item.days
],
)
for item in kinds
]
meta = {
"status_codes": ADHERENCE_STATUS_CODES,
"kinds": {
item.kind.value: {
"enabled": item.enabled,
"weekdays": item.weekdays,
"planned_days": item.planned_days,
"done_days": item.done_days,
"missed_days": item.missed_days,
"adherence_pct": item.adherence_pct,
"streak_current": item.streak.current,
"streak_best": item.streak.best,
}
for item in kinds
},
}
return AdherenceResponse(
from_=start, to=end, unit="day", series=series, meta=meta, kinds=kinds
)
# --- Dashboard ----------------------------------------------------------------
def dashboard(db: Session, user_id: int, tz: ZoneInfo) -> DashboardResponse:
today = service.today_local(tz)
start = today - dt.timedelta(days=29)
profile = service.get_profile(db, user_id)
goal = service.active_goal(db, user_id)
models = build_daily_models(
db, user_id, start, today, tz, profile=profile, goal=goal
)
current = models[-1] if models else DailyModel(day=today)
raw_all = service.daily_weights(db, user_id, tz, until=today)
trend_all = calc.weight_trend(raw_all)
trend_now = trend_all[-1][1] if trend_all else None
trend_7d_ago = calc.trend_at_day(trend_all, today - dt.timedelta(days=7))
last_entry = db.scalar(
service.weights_query(user_id, None, None, tz, "-measured_at").limit(1)
)
cumulative = sum(m.balance_kcal for m in models if m.balance_kcal is not None)
week_start = today - dt.timedelta(days=today.weekday())
workouts_week = service.workouts_between(db, user_id, week_start, today, tz)
water_today = sum(
row.volume_ml for row in service.water_between(db, user_id, today, today, tz)
)
projection = None
if goal is not None and trend_now is not None:
slope = calc.regression_slope(
[p for p in trend_all if p[0] > today - dt.timedelta(days=30)]
) or calc.regression_slope(
[p for p in trend_all if p[0] > today - dt.timedelta(days=14)]
)
result = calc.project_target_date(
trend_now, float(goal.target_weight_kg), slope, today
)
projection = ProjectionRead(status=result.status, date=result.date)
remaining = (
current.budget_kcal - current.intake_kcal
if current.budget_kcal is not None and current.intake_kcal is not None
else current.budget_kcal
)
return DashboardResponse(
date=today,
weight_kg=_round(float(last_entry.weight_kg), 2) if last_entry else None,
weight_measured_at=last_entry.measured_at if last_entry else None,
trend_weight_kg=_round(trend_now, 2),
trend_delta_7d_kg=_round(trend_now - trend_7d_ago, 2)
if trend_now is not None and trend_7d_ago is not None
else None,
bmi=_round(calc.bmi(trend_now, float(profile.height_cm)), 1)
if trend_now is not None and profile is not None
else None,
intake_kcal=_round(current.intake_kcal),
budget_kcal=_round(current.budget_kcal),
remaining_kcal=_round(remaining),
tdee_kcal=_round(current.tdee_kcal),
tdee_method=current.tdee_method,
balance_kcal=_round(current.balance_kcal),
cumulative_balance_30d_kcal=_round(cumulative),
steps=current.steps,
active_kcal=_round(current.active_kcal),
distance_m=current.distance_m,
water_ml=water_today,
water_goal_ml=profile.water_goal_ml if profile else None,
workouts_this_week=len(workouts_week),
goal=GoalRead.model_validate(goal) if goal else None,
projection=projection,
weight_series=[[_iso(d), w] for d, w in trend_all if d >= start],
today=today_view(db, user_id, tz),
)
def active_goal_view(db: Session, user_id: int, tz: ZoneInfo):
"""Active goal + budget of the day + projection + progress (§8.2)."""
from app.modules.health.schemas import ActiveGoalRead
goal = service.active_goal(db, user_id)
if goal is None:
return ActiveGoalRead()
today = service.today_local(tz)
profile = service.get_profile(db, user_id)
models = build_daily_models(
db, user_id, today, today, tz, profile=profile, goal=goal
)
model = models[-1] if models else DailyModel(day=today)
trend_all = calc.weight_trend(service.daily_weights(db, user_id, tz, until=today))
trend_now = trend_all[-1][1] if trend_all else None
budget = None
if model.budget_kcal is not None and model.tdee_kcal is not None:
budget = BudgetRead(
kcal=_round(model.budget_kcal),
# The deficit AIMED AT by the goal (rate x 7700 / 7), not the one the
# floored budget happens to leave — §5.3 BudgetResult.deficit_target.
deficit_target_kcal=_round(model.deficit_target_kcal or 0.0),
floor_applied=model.floor_applied,
rate_clamped=model.rate_clamped,
tdee_kcal=_round(model.tdee_kcal),
tdee_method=model.tdee_method or "estimated",
)
projection = None
if trend_now is not None:
slope = calc.regression_slope(
[p for p in trend_all if p[0] > today - dt.timedelta(days=30)]
)
result = calc.project_target_date(
trend_now, float(goal.target_weight_kg), slope, today
)
projection = ProjectionRead(status=result.status, date=result.date)
start_w = float(goal.start_weight_kg)
target_w = float(goal.target_weight_kg)
done_kg = remaining_kg = progress = None
if trend_now is not None:
done_kg = start_w - trend_now
remaining_kg = trend_now - target_w
span = start_w - target_w
progress = 100 * done_kg / span if span else None
progress = None if progress is None else max(0.0, min(100.0, progress))
return ActiveGoalRead(
goal=GoalRead.model_validate(goal),
budget=budget,
projection=projection,
trend_weight_kg=_round(trend_now, 2),
done_kg=_round(done_kg, 2),
remaining_kg=_round(remaining_kg, 2),
progress_pct=_round(progress),
)
+29
View File
@@ -0,0 +1,29 @@
from datetime import datetime
from typing import Any
from sqlalchemy import DateTime, ForeignKey, String
from sqlalchemy.orm import Mapped, mapped_column
from app.core.database import Base
from app.core.mixins import JSONB_V, TimestampMixin
class ImportRun(TimestampMixin, Base):
__tablename__ = "import_runs"
id: Mapped[int] = mapped_column(primary_key=True)
user_id: Mapped[int] = mapped_column(ForeignKey("users.id"), index=True)
importer_id: Mapped[str] = mapped_column(String(50)) # ex: "foodvisor_csv"
domain: Mapped[str] = mapped_column(String(20)) # "health" | "vape" | "finance"
filename: Mapped[str] = mapped_column(String(255))
file_size: Mapped[int] = mapped_column()
status: Mapped[str] = mapped_column(String(20)) # "completed" | "failed"
rows_total: Mapped[int] = mapped_column(default=0)
rows_inserted: Mapped[int] = mapped_column(default=0)
rows_updated: Mapped[int] = mapped_column(default=0)
rows_duplicates: Mapped[int] = mapped_column(default=0)
rows_errors: Mapped[int] = mapped_column(default=0)
# [{row, message}] capped at 100
error_details: Mapped[list[dict[str, Any]]] = mapped_column(JSONB_V, default=list)
started_at: Mapped[datetime] = mapped_column(DateTime(timezone=True))
finished_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
+89
View File
@@ -0,0 +1,89 @@
from typing import Annotated
from fastapi import APIRouter, Depends, File, Form, Header, Query, UploadFile
from fastapi.security import HTTPAuthorizationCredentials
from sqlalchemy.orm import Session
from app.core.config import get_settings
from app.core.database import get_db
from app.core.dependencies import authenticate_ingest, bearer, get_current_user
from app.core.errors import PayloadTooLargeError
from app.core.pagination import Page, PageParams, paginate
from app.modules.auth.models import User
from app.modules.imports import service
from app.modules.imports.schemas import (
ImportRunRead,
IngestPayload,
IngestResponse,
SourceInfo,
)
router = APIRouter(prefix="/imports", tags=["imports"])
ingest_router = APIRouter(prefix="/ingest", tags=["ingest"])
# Mounted by the module loader in addition to `router` (architecture §5.5).
extra_routers = [ingest_router]
DbDep = Annotated[Session, Depends(get_db)]
UserDep = Annotated[User, Depends(get_current_user)]
@router.get("/sources", response_model=list[SourceInfo])
def list_sources(user: UserDep) -> list[SourceInfo]:
return service.list_sources()
@router.post("", response_model=ImportRunRead, status_code=201)
def upload_import(
db: DbDep,
user: UserDep,
file: Annotated[UploadFile, File()],
source: Annotated[str, Form()],
) -> ImportRunRead:
data = file.file.read()
if len(data) > get_settings().max_upload_bytes:
raise PayloadTooLargeError("Fichier trop volumineux (limite : 20 Mio).")
filename = file.filename or "import"
importer_id = service.resolve_importer_id(source, filename, data[:4096])
run = service.run_import(db, user, importer_id, filename, data)
return ImportRunRead.model_validate(run)
@router.get("", response_model=Page[ImportRunRead])
def list_imports(
db: DbDep,
user: UserDep,
params: Annotated[PageParams, Depends()],
domain: Annotated[str | None, Query()] = None,
sort: Annotated[str | None, Query()] = None,
) -> Page[ImportRunRead]:
stmt = service.runs_query(user.id, domain, sort)
items, total = paginate(db, stmt, params)
return Page(
items=[ImportRunRead.model_validate(run) for run in items],
total=total,
page=params.page,
page_size=params.page_size,
)
@router.get("/{run_id}", response_model=ImportRunRead)
def get_import(run_id: int, db: DbDep, user: UserDep) -> ImportRunRead:
return ImportRunRead.model_validate(service.get_run(db, user.id, run_id))
@router.delete("/{run_id}", status_code=204)
def rollback_import(run_id: int, db: DbDep, user: UserDep) -> None:
service.rollback_run(db, user.id, run_id)
@ingest_router.post("/{domain}", response_model=IngestResponse)
def ingest(
domain: str,
payload: IngestPayload,
db: DbDep,
credentials: Annotated[HTTPAuthorizationCredentials | None, Depends(bearer)],
x_api_key: Annotated[str | None, Header(alias="X-API-Key")] = None,
) -> IngestResponse:
user = authenticate_ingest(db, credentials, x_api_key, f"ingest:{domain}")
return service.run_ingest(db, user, domain, payload)
+60
View File
@@ -0,0 +1,60 @@
from typing import Any
from pydantic import BaseModel, ConfigDict, Field
from app.core.timeutils import StoredUtcDatetime
MAX_INGEST_RECORDS = 1000
class SourceInfo(BaseModel):
id: str
label: str
domain: str
accepted_extensions: tuple[str, ...]
class ImportRunRead(BaseModel):
model_config = ConfigDict(from_attributes=True)
id: int
importer_id: str
domain: str
filename: str
file_size: int
status: str
rows_total: int
rows_inserted: int
rows_updated: int
rows_duplicates: int
rows_errors: int
error_details: list[dict[str, Any]]
started_at: StoredUtcDatetime
finished_at: StoredUtcDatetime | None
created_at: StoredUtcDatetime
class IngestRecordIn(BaseModel):
type: str
data: dict[str, Any]
external_id: str | None = None
class IngestPayload(BaseModel):
source: str = Field(default="ingest", min_length=1, max_length=50)
records: list[IngestRecordIn] = Field(min_length=1, max_length=MAX_INGEST_RECORDS)
class IngestRecordError(BaseModel):
index: int
type: str
message: str
class IngestResponse(BaseModel):
domain: str
received: int
inserted: int
updated: int
duplicates: int
errors: list[IngestRecordError]
+246
View File
@@ -0,0 +1,246 @@
from typing import Any
from sqlalchemy import select
from sqlalchemy.orm import Session
from app.core.errors import DomainValidationError, NotFoundError
from app.core.importing.base import (
ImporterParseError,
RowError,
UpsertOutcome,
)
from app.core.importing.registry import IMPORTER_REGISTRY, detect_importer
from app.core.ingest.base import IngestRecord
from app.core.ingest.registry import INGEST_REGISTRY
from app.core.timeutils import utcnow
from app.modules.auth.models import User
from app.modules.imports.models import ImportRun
from app.modules.imports.schemas import (
IngestPayload,
IngestRecordError,
IngestResponse,
SourceInfo,
)
MAX_ERROR_DETAILS = 100
def list_sources() -> list[SourceInfo]:
return [
SourceInfo(
id=cls.id,
label=cls.label,
domain=cls.domain,
accepted_extensions=cls.accepted_extensions,
)
for cls in sorted(IMPORTER_REGISTRY.values(), key=lambda c: c.id)
]
def resolve_importer_id(source: str, filename: str, head: bytes) -> str:
"""Resolve the form field `source` ("auto" or an importer id) to an id."""
if source == "auto":
detected = detect_importer(filename, head)
if detected is None:
raise DomainValidationError(
"Format non reconnu, choisissez un profil de source."
)
return detected.id
if source not in IMPORTER_REGISTRY:
raise DomainValidationError(
"Profil de source inconnu.", details={"source": source}
)
return source
def run_import(
db: Session, user: User, importer_id: str, filename: str, data: bytes
) -> ImportRun:
"""Orchestrate one synchronous import run (see architecture §5.3).
Row-level errors increment counters without stopping the run; a fatal
parse error marks the run as failed (partial domain rows rolled back).
A single commit ends the run.
"""
importer_cls = IMPORTER_REGISTRY[importer_id]
run = ImportRun(
user_id=user.id,
importer_id=importer_id,
domain=importer_cls.domain,
filename=filename,
file_size=len(data),
status="completed",
started_at=utcnow(),
)
db.add(run)
db.flush() # assign run.id so imported rows can reference it
importer = importer_cls()
importer.import_run_id = run.id
counters = {outcome: 0 for outcome in UpsertOutcome}
error_details: list[dict[str, Any]] = []
rows_total = 0
fatal_message: str | None = None
def record_error(row: int, message: str) -> None:
counters[UpsertOutcome.ERROR] += 1
if len(error_details) < MAX_ERROR_DETAILS:
error_details.append({"row": row, "message": message})
iterator = importer.parse(data, filename)
row = 0
while fatal_message is None:
row += 1
try:
record = next(iterator)
except StopIteration:
break
except RowError as exc:
rows_total += 1
record_error(row, str(exc))
continue
except ImporterParseError as exc:
fatal_message = str(exc) or "Fichier illisible pour ce profil de source."
break
except Exception: # noqa: BLE001 — importer bug must fail the run, not the API
fatal_message = "Erreur inattendue pendant la lecture du fichier."
break
rows_total += 1
try:
outcome = importer.upsert(db, user.id, record)
except RowError as exc:
record_error(row, str(exc))
continue
except Exception: # noqa: BLE001 — a broken row must not stop the run
record_error(row, "Erreur inattendue sur cette ligne.")
continue
if outcome is UpsertOutcome.ERROR:
record_error(row, "Ligne rejetée par l'importeur.")
else:
counters[outcome] += 1
if fatal_message is not None:
# Drop partial domain rows AND the flushed run, then persist a failed run.
db.rollback()
run = ImportRun(
user_id=user.id,
importer_id=importer_id,
domain=importer_cls.domain,
filename=filename,
file_size=len(data),
status="failed",
rows_total=rows_total,
rows_inserted=0,
rows_updated=0,
rows_duplicates=0,
rows_errors=counters[UpsertOutcome.ERROR],
error_details=(error_details + [{"row": row, "message": fatal_message}])[
:MAX_ERROR_DETAILS
],
started_at=utcnow(),
finished_at=utcnow(),
)
db.add(run)
db.commit()
return run
run.rows_total = rows_total
run.rows_inserted = counters[UpsertOutcome.INSERTED]
run.rows_updated = counters[UpsertOutcome.UPDATED]
run.rows_duplicates = counters[UpsertOutcome.DUPLICATE]
run.rows_errors = counters[UpsertOutcome.ERROR]
run.error_details = error_details
run.finished_at = utcnow()
db.commit()
return run
def get_run(db: Session, user_id: int, run_id: int) -> ImportRun:
run = db.get(ImportRun, run_id)
if run is None or run.user_id != user_id:
raise NotFoundError("Import introuvable.")
return run
def runs_query(user_id: int, domain: str | None, sort: str | None):
stmt = select(ImportRun).where(ImportRun.user_id == user_id)
if domain:
stmt = stmt.where(ImportRun.domain == domain)
sort = sort or "-started_at"
allowed = {"started_at": ImportRun.started_at}
descending = sort.startswith("-")
field_name = sort.lstrip("-")
column = allowed.get(field_name)
if column is None:
raise DomainValidationError(
"Champ de tri non autorisé.", details={"sort": sort}
)
order = column.desc() if descending else column.asc()
return stmt.order_by(order, ImportRun.id.desc())
def rollback_run(db: Session, user_id: int, run_id: int) -> None:
"""Delete a run; imported domain rows disappear via FK ON DELETE CASCADE."""
run = get_run(db, user_id, run_id)
db.delete(run)
db.commit()
def run_ingest(
db: Session, user: User, domain: str, payload: IngestPayload
) -> IngestResponse:
handler = INGEST_REGISTRY.get(domain)
if handler is None:
raise NotFoundError("Domaine d'ingestion inconnu.", details={"domain": domain})
counters = {outcome: 0 for outcome in UpsertOutcome}
errors: list[IngestRecordError] = []
for index, item in enumerate(payload.records):
if item.type not in handler.record_types:
errors.append(
IngestRecordError(
index=index,
type=item.type,
message="Type d'enregistrement non pris en charge.",
)
)
continue
record = IngestRecord(
type=item.type,
data=item.data,
external_id=item.external_id,
source=payload.source,
)
try:
outcome = handler.apply(db, user.id, record)
except RowError as exc:
errors.append(
IngestRecordError(index=index, type=item.type, message=str(exc))
)
continue
except Exception: # noqa: BLE001 — a broken record must not fail the batch
errors.append(
IngestRecordError(
index=index,
type=item.type,
message="Erreur inattendue sur cet enregistrement.",
)
)
continue
if outcome is UpsertOutcome.ERROR:
errors.append(
IngestRecordError(
index=index, type=item.type, message="Enregistrement rejeté."
)
)
else:
counters[outcome] += 1
db.commit()
return IngestResponse(
domain=domain,
received=len(payload.records),
inserted=counters[UpsertOutcome.INSERTED],
updated=counters[UpsertOutcome.UPDATED],
duplicates=counters[UpsertOutcome.DUPLICATE],
errors=errors,
)
+548
View File
@@ -0,0 +1,548 @@
"""Pure vape calculations (datamodel-health-vape.md §7) — unit-testable.
Money is expressed in euro cents; derived rates (cost/ml, cost/day, savings)
may be fractional cents (floats/Decimals), only stored amounts are ints.
Every function here is pure: no database, no HTTP, no application errors.
"""
from collections.abc import Iterable, Iterator, Mapping, Sequence
from dataclasses import dataclass
from datetime import UTC, date, datetime, time, timedelta
from decimal import Decimal
from itertools import pairwise
from math import floor
from statistics import fmean
from zoneinfo import ZoneInfo
DEFAULT_COIL_LIFESPAN_DAYS = 14.0
COIL_AVG_LAST_N = 5
NICOTINE_MG_PER_CIG = 12.0
MINUTES_PER_CIG = 11
NICOTINE_MISMATCH_TOLERANCE = 0.10 # 10 % gap target vs recomputed
TREND_SLOPE_THRESHOLD = 0.05 # mg/day per day: below = stable
PURCHASE_FALLBACK_WINDOW_DAYS = 90 # §7.1 fallback cost/ml window
SECONDS_PER_DAY = 86400.0
def date_range(start: date, end: date) -> Iterator[date]:
"""Every local day from `start` to `end`, both bounds included."""
day = start
while day <= end:
yield day
day += timedelta(days=1)
# ---------------------------------------------------------------------------
# Mix / recipe costing (§6.3, §7.1)
# ---------------------------------------------------------------------------
@dataclass(frozen=True)
class ComponentInput:
"""One recipe line: quantity in the product's size_unit + catalog price."""
quantity: Decimal
price_cents: int
size_value: Decimal
nicotine_mg_ml: Decimal | None = None # boosters only
vg_pct: Decimal | None = None
def component_cost_cents(component: ComponentInput) -> Decimal:
"""quantity × (package price / package size), in cents."""
return component.quantity * Decimal(component.price_cents) / component.size_value
def mix_cost_total_cents(components: Iterable[ComponentInput]) -> Decimal:
"""Σ quantity × (price / size_value), in cents."""
total = Decimal(0)
for c in components:
total += component_cost_cents(c)
return total
def mix_cost_per_ml_cents(
components: Iterable[ComponentInput], total_ml: Decimal
) -> Decimal | None:
if total_ml <= 0:
return None
return mix_cost_total_cents(components) / total_ml
def mix_nicotine_check_mg_ml(
components: Iterable[ComponentInput], total_ml: Decimal
) -> Decimal:
"""Σ (booster qty × booster mg/ml) / total_ml (§6.3)."""
if total_ml <= 0:
return Decimal(0)
total_mg = Decimal(0)
for c in components:
if c.nicotine_mg_ml is not None:
total_mg += c.quantity * c.nicotine_mg_ml
return total_mg / total_ml
def mix_vg_pct(
components: Iterable[ComponentInput], total_ml: Decimal
) -> Decimal | None:
"""Weighted VG % — only when every component declares vg_pct."""
comps = list(components)
if total_ml <= 0 or not comps or any(c.vg_pct is None for c in comps):
return None
weighted = sum(
(c.quantity * c.vg_pct for c in comps if c.vg_pct is not None),
Decimal(0),
)
return weighted / total_ml
def nicotine_mismatch_warning(
target_mg_ml: Decimal, check_mg_ml: Decimal
) -> str | None:
"""'nicotine_mismatch' when the recomputed rate drifts > 10 % from target."""
tolerance = abs(target_mg_ml) * Decimal(str(NICOTINE_MISMATCH_TOLERANCE))
if abs(check_mg_ml - target_mg_ml) > tolerance:
return "nicotine_mismatch"
return None
@dataclass(frozen=True)
class RecipeQuantities:
booster_ml: Decimal
aroma_ml: Decimal
base_ml: Decimal
def recipe_quantities(
total_ml: Decimal,
target_nicotine_mg_ml: Decimal,
booster_nicotine_mg_ml: Decimal,
aroma_pct: Decimal,
) -> RecipeQuantities:
"""Stateless recipe assistant (§6.3).
booster_ml = total × target / n_booster ; aroma_ml = total × aroma_pct/100 ;
base_ml = total booster aroma. Raises ValueError when the recipe is
impossible (booster too weak, aroma dosage too high).
"""
if total_ml <= 0:
raise ValueError("total_ml must be > 0")
if booster_nicotine_mg_ml <= 0:
raise ValueError("booster nicotine rate must be > 0")
booster_ml = total_ml * target_nicotine_mg_ml / booster_nicotine_mg_ml
aroma_ml = total_ml * aroma_pct / Decimal(100)
base_ml = total_ml - booster_ml - aroma_ml
if base_ml <= 0:
raise ValueError("base volume would be <= 0")
return RecipeQuantities(booster_ml=booster_ml, aroma_ml=aroma_ml, base_ml=base_ml)
def fallback_cost_per_ml_cents(
purchases: Iterable[tuple[Decimal, Decimal]],
) -> Decimal | None:
"""Weighted average cost/ml of liquid purchases (§7.1 fallback).
`purchases` = (total_cents, ml_bought) pairs. None when nothing was bought.
"""
total_cents = Decimal(0)
total_ml = Decimal(0)
for cents, ml in purchases:
total_cents += cents
total_ml += ml
if total_ml <= 0:
return None
return total_cents / total_ml
# ---------------------------------------------------------------------------
# Daily consumption (§6.4, §7.2)
# ---------------------------------------------------------------------------
def effective_daily_ml(
entries: Iterable[tuple[str, Decimal]],
) -> Decimal | None:
"""Daily consumption for ONE day: a daily_total overrides the refill sum.
`entries` = (kind, ml) pairs of that day. None = untracked day.
"""
rows = list(entries)
totals = [ml for kind, ml in rows if kind == "daily_total"]
if totals:
return totals[0]
refills = [ml for kind, ml in rows if kind == "refill"]
if not refills:
return None
return sum(refills, Decimal(0))
def mean_ml_per_day(values: Iterable[Decimal | None]) -> Decimal | None:
"""Mean over tracked days only (§7.2); None when nothing is tracked."""
vals = [v for v in values if v is not None]
if not vals:
return None
return sum(vals, Decimal(0)) / Decimal(len(vals))
def tracked_days_ratio(values: Sequence[Decimal | None]) -> float:
"""Share of tracked days in the window — reliability indicator (§7.2)."""
if not values:
return 0.0
return sum(1 for v in values if v is not None) / len(values)
def sum_ml_between(
daily_ml_by_date: Mapping[date, Decimal | None], start: date, end: date
) -> Decimal | None:
"""Σ tracked daily ml over [start, end) — volume through one coil (§6.5).
None when no day of the interval is tracked.
"""
values = [
daily_ml_by_date.get(day)
for day in date_range(start, end - timedelta(days=1))
if daily_ml_by_date.get(day) is not None
]
if not values:
return None
return sum((v for v in values if v is not None), Decimal(0))
# ---------------------------------------------------------------------------
# Coils (§7.3)
# ---------------------------------------------------------------------------
def coil_intervals_days(changed_ats: Sequence[datetime]) -> list[float]:
"""Days between consecutive changes, oldest first."""
ordered = sorted(changed_ats)
return [
(nxt - prev).total_seconds() / SECONDS_PER_DAY
for prev, nxt in pairwise(ordered)
]
def avg_coil_lifespan_days(
intervals_days: Sequence[float],
last_n: int = COIL_AVG_LAST_N,
default: float = DEFAULT_COIL_LIFESPAN_DAYS,
) -> tuple[float, bool]:
"""Mean of the last `last_n` cycles; (default, True) when < 2 changes."""
if not intervals_days:
return default, True
recent = list(intervals_days)[-last_n:]
return fmean(recent), False
def coil_cost_per_day_cents(
coil_unit_price_cents: float | Decimal | None, avg_lifespan_days: float
) -> float:
"""Amortized coil cost; 0 when no coil product/price is known."""
if coil_unit_price_cents is None or avg_lifespan_days <= 0:
return 0.0
return float(coil_unit_price_cents) / avg_lifespan_days
def coil_age_days(changed_at: datetime, now_utc: datetime) -> float:
"""Provisional lifespan of the coil currently installed."""
return (now_utc - changed_at).total_seconds() / SECONDS_PER_DAY
# ---------------------------------------------------------------------------
# Daily costs & cigarette baseline (§7.4, §7.5)
# ---------------------------------------------------------------------------
def vape_cost_per_day_cents(
ml_per_day: Decimal | float | None,
cost_per_ml_cents: Decimal | float | None,
coil_cpd_cents: float,
) -> float | None:
"""ml/day × cost/ml + coil amortization; None when cost/ml is unknown."""
if cost_per_ml_cents is None or ml_per_day is None:
return None
return float(ml_per_day) * float(cost_per_ml_cents) + coil_cpd_cents
def cig_cost_per_day_cents(
cigs_per_day: Decimal | float,
cigs_per_pack: int,
pack_price_cents: int,
) -> float:
"""Frozen cigarette baseline: cigs/day ÷ cigs/pack × pack price."""
if cigs_per_pack <= 0:
return 0.0
return float(cigs_per_day) / cigs_per_pack * pack_price_cents
def savings_per_day_cents(
cig_cpd_cents: float, vape_cpd_cents: float | None
) -> float | None:
"""Daily gain of vaping over the frozen cigarette baseline (§7.5)."""
if vape_cpd_cents is None:
return None
return cig_cpd_cents - vape_cpd_cents
def real_cost_per_day_cents(total_spend_cents: float, window_days: int) -> float | None:
"""Purchase-based cost/day over a window (§7.4)."""
if window_days <= 0:
return None
return total_spend_cents / window_days
def theoretical_vape_cost_cents(
day_ml: Decimal | None,
imputed_ml_per_day: Decimal | None,
cost_per_ml_cents: Decimal | float | None,
coil_cpd_cents: float,
) -> float:
"""Theoretical cost of one day, mean-imputed when untracked (§7.5).
Untracked days use ml_per_day(30) so tracking gaps do not inflate savings.
An unknown cost/ml contributes 0 (cost metrics themselves are null'ed
upstream per §7.1) — coil amortization still applies.
"""
ml = day_ml if day_ml is not None else (imputed_ml_per_day or Decimal(0))
liquid = float(ml) * float(cost_per_ml_cents or 0)
return liquid + coil_cpd_cents
def cumulative_savings_theoretical(
quit_date: date,
today: date,
daily_ml_by_date: Mapping[date, Decimal | None],
imputed_ml_per_day: Decimal | None,
cost_per_ml_cents: Decimal | float | None,
coil_cpd_cents: float,
cig_cpd_cents: float,
) -> list[tuple[date, float]]:
"""Frozen-baseline theoretical savings, cumulative point per day (§7.5).
cum(d) = cig_cpd × (d quit_date).days Σ_{x=quit}^{d} vape_cost(x)
"""
if today < quit_date:
return []
out: list[tuple[date, float]] = []
vape_sum = 0.0
for day in date_range(quit_date, today):
vape_sum += theoretical_vape_cost_cents(
daily_ml_by_date.get(day),
imputed_ml_per_day,
cost_per_ml_cents,
coil_cpd_cents,
)
elapsed = (day - quit_date).days
out.append((day, cig_cpd_cents * elapsed - vape_sum))
return out
def cumulative_savings_real(
quit_date: date,
today: date,
spend_cents_by_date: Mapping[date, Decimal | int | float],
cig_cpd_cents: float,
) -> list[tuple[date, float]]:
"""Purchase-based savings: cig baseline minus real spending, cumulative."""
if today < quit_date:
return []
out: list[tuple[date, float]] = []
spent = 0.0
for day in date_range(quit_date, today):
spent += float(spend_cents_by_date.get(day, 0))
elapsed = (day - quit_date).days
out.append((day, cig_cpd_cents * elapsed - spent))
return out
# ---------------------------------------------------------------------------
# Nicotine (§7.6) & avoided cigarettes (§7.7)
# ---------------------------------------------------------------------------
def nicotine_mg_for_day(
entries: Iterable[tuple[Decimal, Decimal | None]],
) -> float | None:
"""Σ ml × effective mg/ml over the day's entries; None if untracked.
Entries without a resolvable nicotine rate contribute 0 mg.
"""
rows = list(entries)
if not rows:
return None
return float(sum((ml * (nic or Decimal(0)) for ml, nic in rows), Decimal(0)))
def cig_equivalent(nicotine_mg: float) -> float:
"""Informative equivalence (~12 mg nicotine per cigarette)."""
return nicotine_mg / NICOTINE_MG_PER_CIG
def days_since_quit(quit_date: date, today: date) -> int:
"""Whole local days elapsed since the quit date (never negative)."""
return max(0, (today - quit_date).days)
def cigarettes_avoided(elapsed_days: int, cigs_per_day: Decimal | float) -> int:
return floor(elapsed_days * float(cigs_per_day))
def packs_avoided(cigs_avoided: int, cigs_per_pack: int) -> float:
if cigs_per_pack <= 0:
return 0.0
return cigs_avoided / cigs_per_pack
def time_regained_minutes(cigs_avoided: int) -> int:
return cigs_avoided * MINUTES_PER_CIG
# ---------------------------------------------------------------------------
# Series helpers
# ---------------------------------------------------------------------------
def moving_average(
values: Sequence[float | None], window: int = 7
) -> list[float | None]:
"""Trailing moving average ignoring None gaps (None when window is empty)."""
out: list[float | None] = []
for i in range(len(values)):
chunk = [v for v in values[max(0, i - window + 1) : i + 1] if v is not None]
out.append(fmean(chunk) if chunk else None)
return out
def regression_slope(points: Sequence[tuple[date, float]]) -> float | None:
"""OLS slope in unit/day over (day, value) points (§5.5); None if < 3 pts."""
if len(points) < 3:
return None
t0 = points[0][0]
ts = [float((d - t0).days) for d, _ in points]
ws = [w for _, w in points]
t_mean = fmean(ts)
w_mean = fmean(ws)
denom = sum((t - t_mean) ** 2 for t in ts)
if denom == 0:
return None
num = sum((t - t_mean) * (w - w_mean) for t, w in zip(ts, ws, strict=True))
return num / denom
def trend_status(slope: float | None, threshold: float = TREND_SLOPE_THRESHOLD) -> str:
"""'down' / 'stable' / 'up' — stable identifiers mapped to labels by the UI."""
if slope is None or abs(slope) < threshold:
return "stable"
return "down" if slope < 0 else "up"
# ---------------------------------------------------------------------------
# Health milestones (§7.8 — WHO-style timeline, static, French labels)
# ---------------------------------------------------------------------------
@dataclass(frozen=True)
class Milestone:
code: str
offset: timedelta
label_fr: str
MILESTONES: tuple[Milestone, ...] = (
Milestone(
"hr_bp_normal",
timedelta(minutes=20),
"Fréquence cardiaque et tension redescendent",
),
Milestone(
"co_halved",
timedelta(hours=8),
"Le monoxyde de carbone sanguin diminue de moitié",
),
Milestone(
"co_normal",
timedelta(hours=24),
"Monoxyde de carbone éliminé ; les poumons commencent à évacuer les résidus",
),
Milestone(
"nicotine_out",
timedelta(hours=48),
"Plus de nicotine dans le corps ; goût et odorat s'améliorent",
),
Milestone(
"breathing_easier",
timedelta(hours=72),
"Respiration plus facile, énergie en hausse (bronches détendues)",
),
Milestone("circulation", timedelta(days=14), "Circulation sanguine améliorée"),
Milestone(
"lung_function",
timedelta(days=90),
"Fonction pulmonaire améliorée jusqu'à +30 %",
),
Milestone(
"cilia_recovery",
timedelta(days=270),
"Cils bronchiques régénérés ; toux et essoufflement diminuent",
),
Milestone(
"chd_risk_half",
timedelta(days=365),
"Risque de maladie coronarienne réduit de moitié",
),
Milestone(
"stroke_risk_normal",
timedelta(days=5 * 365),
"Risque d'AVC ramené à celui d'un non-fumeur",
),
Milestone(
"lung_cancer_half",
timedelta(days=10 * 365),
"Risque de cancer du poumon réduit de moitié",
),
Milestone(
"chd_risk_normal",
timedelta(days=15 * 365),
"Risque coronarien équivalent à celui d'un non-fumeur",
),
)
@dataclass(frozen=True)
class MilestoneStatus:
code: str
label_fr: str
reached_at: datetime # UTC
achieved: bool
progress_pct: float
def milestone_statuses(
quit_date: date, tz: ZoneInfo, now_utc: datetime
) -> list[MilestoneStatus]:
"""Milestones measured from local midnight of quit_date (§7.8)."""
t0 = datetime.combine(quit_date, time(0, 0), tzinfo=tz)
elapsed = (now_utc - t0).total_seconds()
out: list[MilestoneStatus] = []
for m in MILESTONES:
reached_at = (t0 + m.offset).astimezone(UTC)
progress = 100.0 * elapsed / m.offset.total_seconds()
out.append(
MilestoneStatus(
code=m.code,
label_fr=m.label_fr,
reached_at=reached_at,
achieved=now_utc >= reached_at,
progress_pct=max(0.0, min(100.0, progress)),
)
)
return out
def next_milestone(statuses: Sequence[MilestoneStatus]) -> MilestoneStatus | None:
"""First milestone not reached yet (None once the timeline is complete)."""
for status in statuses:
if not status.achieved:
return status
return None
+291
View File
@@ -0,0 +1,291 @@
"""Vape module tables (datamodel-health-vape.md §6, money in euro cents).
Portable column types (CONVENTIONS C8): non-native enums, partial unique
indexes declared with BOTH sqlite_where and postgresql_where.
"""
import enum
from datetime import date, datetime
from decimal import Decimal
from sqlalchemy import (
CheckConstraint,
Date,
DateTime,
Enum,
ForeignKey,
Index,
Integer,
Numeric,
String,
UniqueConstraint,
text,
)
from sqlalchemy.orm import Mapped, mapped_column, relationship
from app.core.database import Base
from app.core.mixins import SourceMixin, TimestampMixin
class ProductKind(str, enum.Enum):
COIL = "coil" # résistance
BASE = "base" # base PG/VG
BOOSTER = "booster" # booster de nicotine
AROMA = "aroma" # arôme concentré
HARDWARE = "hardware" # matériel (box, clearomiseur…) — hors coût/ml
POD = "pod" # cartouche pod
class SizeUnit(str, enum.Enum):
ML = "ml"
UNIT = "unit" # à l'unité (boîte de 5 résistances -> size_value=5)
G = "g"
class LiquidEntryKind(str, enum.Enum):
REFILL = "refill"
DAILY_TOTAL = "daily_total"
#: Product kinds that make up a DIY recipe (hardware/coil/pod are excluded).
LIQUID_KINDS = (ProductKind.BASE, ProductKind.BOOSTER, ProductKind.AROMA)
def _enum(enum_cls: type[enum.Enum], name: str) -> Enum:
"""Portable string enum persisting the *values* (lowercase), never native.
VARCHAR + CHECK on both engines: `create_constraint=True` is required, its
SQLAlchemy default is False (bare VARCHAR, no integrity check).
"""
return Enum(
enum_cls,
name=name,
native_enum=False,
create_constraint=True,
validate_strings=True,
values_callable=lambda e: [m.value for m in e],
)
class VapeSettings(TimestampMixin, Base):
"""Singleton per user: quit reference + defaults (§6.1)."""
__tablename__ = "vape_settings"
__table_args__ = (
UniqueConstraint("user_id", name="uq_vape_settings_user"),
CheckConstraint(
"cigs_per_pack > 0 AND cig_pack_price_cents >= 0"
" AND cigs_per_day_before >= 0",
name="ck_vape_settings_pack",
),
)
id: Mapped[int] = mapped_column(primary_key=True)
user_id: Mapped[int] = mapped_column(
ForeignKey("users.id", ondelete="CASCADE"), index=True
)
quit_date: Mapped[date] = mapped_column(Date)
cigs_per_day_before: Mapped[Decimal] = mapped_column(Numeric(4, 1))
cig_pack_price_cents: Mapped[int] = mapped_column(Integer)
cigs_per_pack: Mapped[int] = mapped_column(Integer, default=20)
default_nicotine_mg_ml: Mapped[Decimal] = mapped_column(Numeric(4, 1))
currency: Mapped[str] = mapped_column(String(3), default="EUR")
class Product(TimestampMixin, Base):
"""Vape catalog: coils, bases, boosters, aromas, hardware, pods (§6.2)."""
__tablename__ = "products"
__table_args__ = (
UniqueConstraint(
"user_id",
"kind",
"name",
"brand",
name="uq_products_user_kind_name_brand",
),
Index("ix_products_user_kind", "user_id", "kind"),
CheckConstraint(
"price_cents >= 0 AND size_value > 0", name="ck_products_price_size"
),
CheckConstraint(
"kind <> 'booster' OR nicotine_mg_ml IS NOT NULL",
name="ck_products_booster_nic",
),
)
id: Mapped[int] = mapped_column(primary_key=True)
user_id: Mapped[int] = mapped_column(
ForeignKey("users.id", ondelete="CASCADE"), index=True
)
kind: Mapped[ProductKind] = mapped_column(_enum(ProductKind, "product_kind"))
name: Mapped[str] = mapped_column(String(150))
brand: Mapped[str | None] = mapped_column(String(100), default=None)
price_cents: Mapped[int] = mapped_column(Integer) # package catalog price (cents)
size_value: Mapped[Decimal] = mapped_column(Numeric(8, 2))
size_unit: Mapped[SizeUnit] = mapped_column(_enum(SizeUnit, "size_unit"))
nicotine_mg_ml: Mapped[Decimal | None] = mapped_column(
Numeric(4, 1), default=None
) # boosters only
vg_pct: Mapped[Decimal | None] = mapped_column(Numeric(4, 1), default=None)
ohm: Mapped[Decimal | None] = mapped_column(Numeric(4, 2), default=None)
is_archived: Mapped[bool] = mapped_column(default=False)
note: Mapped[str | None] = mapped_column(String(255), default=None)
@property
def unit_price_cents(self) -> Decimal:
"""Cents per ml/unit/g of the package (derived, never stored)."""
return Decimal(self.price_cents) / Decimal(self.size_value)
class Mix(TimestampMixin, Base):
"""DIY recipe; cost is always computed from catalog prices (§6.3)."""
__tablename__ = "mixes"
__table_args__ = (
Index(
"uq_mixes_user_active",
"user_id",
unique=True,
postgresql_where=text("is_active"),
sqlite_where=text("is_active"),
),
CheckConstraint("total_ml > 0", name="ck_mixes_total_ml"),
)
id: Mapped[int] = mapped_column(primary_key=True)
user_id: Mapped[int] = mapped_column(
ForeignKey("users.id", ondelete="CASCADE"), index=True
)
name: Mapped[str] = mapped_column(String(150))
total_ml: Mapped[Decimal] = mapped_column(Numeric(7, 1))
target_nicotine_mg_ml: Mapped[Decimal] = mapped_column(Numeric(4, 1))
is_active: Mapped[bool] = mapped_column(default=False)
is_archived: Mapped[bool] = mapped_column(default=False)
note: Mapped[str | None] = mapped_column(String(255), default=None)
components: Mapped[list["MixComponent"]] = relationship(
back_populates="mix",
cascade="all, delete-orphan",
order_by="MixComponent.id",
lazy="selectin",
)
class MixComponent(Base):
"""One product line of a recipe; ownership goes through mixes.user_id."""
__tablename__ = "mix_components"
__table_args__ = (
UniqueConstraint("mix_id", "product_id", name="uq_mix_components_mix_product"),
CheckConstraint("quantity > 0", name="ck_mix_components_qty"),
)
id: Mapped[int] = mapped_column(primary_key=True)
mix_id: Mapped[int] = mapped_column(
ForeignKey("mixes.id", ondelete="CASCADE"), index=True
)
product_id: Mapped[int] = mapped_column(
ForeignKey("products.id", ondelete="RESTRICT")
)
quantity: Mapped[Decimal] = mapped_column(Numeric(7, 2)) # in product.size_unit
mix: Mapped[Mix] = relationship(back_populates="components")
product: Mapped[Product] = relationship(lazy="joined")
class LiquidEntry(TimestampMixin, SourceMixin, Base):
"""Liquid consumption journal: refills or direct daily totals (§6.4)."""
__tablename__ = "liquid_entries"
__table_args__ = (
Index("ix_liquid_entries_user_date", "user_id", "entry_date"),
Index(
"uq_liquid_entries_user_date_dailytotal",
"user_id",
"entry_date",
unique=True,
postgresql_where=text("kind = 'daily_total'"),
sqlite_where=text("kind = 'daily_total'"),
),
UniqueConstraint(
"user_id", "source", "external_id", name="uq_liquid_entries_external"
),
UniqueConstraint("user_id", "content_hash", name="uq_liquid_entries_hash"),
CheckConstraint("ml > 0 AND ml <= 100", name="ck_liquid_entries_ml"),
)
id: Mapped[int] = mapped_column(primary_key=True)
user_id: Mapped[int] = mapped_column(
ForeignKey("users.id", ondelete="CASCADE"), index=True
)
import_run_id: Mapped[int | None] = mapped_column(
ForeignKey("import_runs.id", ondelete="CASCADE"), default=None
)
entry_date: Mapped[date] = mapped_column(Date) # local civil day
kind: Mapped[LiquidEntryKind] = mapped_column(
_enum(LiquidEntryKind, "liquid_entry_kind"),
default=LiquidEntryKind.REFILL,
)
ml: Mapped[Decimal] = mapped_column(Numeric(6, 2))
nicotine_mg_ml: Mapped[Decimal | None] = mapped_column(
Numeric(4, 1), default=None
) # override; fallback: mix target, then settings default
mix_id: Mapped[int | None] = mapped_column(
ForeignKey("mixes.id", ondelete="SET NULL"), default=None
)
note: Mapped[str | None] = mapped_column(String(255), default=None)
class CoilChange(TimestampMixin, Base):
"""A new coil installed at `changed_at`; lifespan derived between rows (§6.5)."""
__tablename__ = "coil_changes"
__table_args__ = (Index("ix_coil_changes_user_changed", "user_id", "changed_at"),)
id: Mapped[int] = mapped_column(primary_key=True)
user_id: Mapped[int] = mapped_column(
ForeignKey("users.id", ondelete="CASCADE"), index=True
)
changed_at: Mapped[datetime] = mapped_column(DateTime(timezone=True))
product_id: Mapped[int | None] = mapped_column(
ForeignKey("products.id", ondelete="RESTRICT"), default=None
)
reason: Mapped[str | None] = mapped_column(String(50), default=None)
note: Mapped[str | None] = mapped_column(String(255), default=None)
product: Mapped[Product | None] = relationship(lazy="joined")
class Purchase(TimestampMixin, Base):
"""Real vape spending, for real savings and real cost/ml (§6.6)."""
__tablename__ = "purchases"
__table_args__ = (
Index("ix_purchases_user_date", "user_id", "purchased_on"),
CheckConstraint("qty > 0", name="ck_purchases_qty"),
)
id: Mapped[int] = mapped_column(primary_key=True)
user_id: Mapped[int] = mapped_column(
ForeignKey("users.id", ondelete="CASCADE"), index=True
)
purchased_on: Mapped[date] = mapped_column(Date)
product_id: Mapped[int] = mapped_column(
ForeignKey("products.id", ondelete="RESTRICT")
)
qty: Mapped[Decimal] = mapped_column(Numeric(6, 2), default=Decimal(1))
unit_price_cents: Mapped[int | None] = mapped_column(Integer, default=None)
note: Mapped[str | None] = mapped_column(String(255), default=None)
product: Mapped[Product] = relationship(lazy="joined")
@property
def total_cents(self) -> Decimal:
"""qty × coalesce(unit_price_cents, product.price_cents), derived."""
unit = (
Decimal(self.unit_price_cents)
if self.unit_price_cents is not None
else Decimal(self.product.price_cents)
)
return Decimal(self.qty) * unit
+322
View File
@@ -0,0 +1,322 @@
"""Vape API (datamodel-health-vape.md §8.4) — thin routing layer.
Static sub-paths (/mixes/calculator, /coils/stats, /coils/change) are declared
before their `{id}` siblings so FastAPI never routes them to the item handler.
"""
from datetime import date
from typing import Annotated
from fastapi import APIRouter, Depends, Query
from sqlalchemy.orm import Session
from app.core.database import get_db
from app.core.dependencies import get_current_user
from app.core.pagination import Page, PageParams
from app.modules.auth.models import User
from app.modules.vape import service
from app.modules.vape.models import LiquidEntryKind, ProductKind
from app.modules.vape.schemas import (
CoilChangeCreate,
CoilChangeCreated,
CoilChangeRead,
CoilChangeUpdate,
LiquidEntryCreate,
LiquidEntryRead,
LiquidEntryUpdate,
MilestoneRead,
MixCalculatorRequest,
MixCalculatorResult,
MixCreate,
MixRead,
MixUpdate,
ProductCreate,
ProductRead,
ProductUpdate,
PurchaseCreate,
PurchaseRead,
PurchaseUpdate,
StatsResponse,
VapeDashboard,
VapeSettingsPut,
VapeSettingsRead,
)
router = APIRouter(prefix="/vape", tags=["vape"])
DbDep = Annotated[Session, Depends(get_db)]
UserDep = Annotated[User, Depends(get_current_user)]
PageDep = Annotated[PageParams, Depends()]
FromQuery = Annotated[date | None, Query(alias="from")]
ToQuery = Annotated[date | None, Query()]
TzQuery = Annotated[str | None, Query()]
SortQuery = Annotated[str | None, Query()]
# --------------------------------------------------------------------- settings
@router.get("/settings", response_model=VapeSettingsRead)
def read_settings(db: DbDep, user: UserDep) -> VapeSettingsRead:
return VapeSettingsRead.model_validate(service.get_settings(db, user.id))
@router.put("/settings", response_model=VapeSettingsRead)
def write_settings(
payload: VapeSettingsPut, db: DbDep, user: UserDep
) -> VapeSettingsRead:
return VapeSettingsRead.model_validate(service.put_settings(db, user.id, payload))
# --------------------------------------------------------------------- products
@router.get("/products", response_model=Page[ProductRead])
def list_products(
db: DbDep,
user: UserDep,
params: PageDep,
kind: Annotated[ProductKind | None, Query()] = None,
include_archived: Annotated[bool, Query()] = False,
sort: SortQuery = None,
) -> Page[ProductRead]:
return service.list_products(db, user.id, params, kind, include_archived, sort)
@router.post("/products", response_model=ProductRead, status_code=201)
def create_product(payload: ProductCreate, db: DbDep, user: UserDep) -> ProductRead:
return service.create_product(db, user.id, payload)
@router.patch("/products/{product_id}", response_model=ProductRead)
def update_product(
product_id: int, payload: ProductUpdate, db: DbDep, user: UserDep
) -> ProductRead:
return service.update_product(db, user.id, product_id, payload)
@router.delete("/products/{product_id}", status_code=204)
def delete_product(product_id: int, db: DbDep, user: UserDep) -> None:
service.delete_product(db, user.id, product_id)
# ------------------------------------------------------------------------ mixes
@router.get("/mixes", response_model=Page[MixRead])
def list_mixes(
db: DbDep,
user: UserDep,
params: PageDep,
include_archived: Annotated[bool, Query()] = False,
sort: SortQuery = None,
) -> Page[MixRead]:
return service.list_mixes(db, user.id, params, include_archived, sort)
@router.post("/mixes", response_model=MixRead, status_code=201)
def create_mix(payload: MixCreate, db: DbDep, user: UserDep) -> MixRead:
return service.create_mix(db, user.id, payload)
@router.post("/mixes/calculator", response_model=MixCalculatorResult)
def mix_calculator(
payload: MixCalculatorRequest, db: DbDep, user: UserDep
) -> MixCalculatorResult:
"""Stateless recipe assistant — persists nothing (§6.3)."""
return service.mix_calculator(db, user.id, payload)
@router.post("/mixes/{mix_id}/activate", response_model=MixRead)
def activate_mix(mix_id: int, db: DbDep, user: UserDep) -> MixRead:
return service.activate_mix(db, user.id, mix_id)
@router.patch("/mixes/{mix_id}", response_model=MixRead)
def update_mix(mix_id: int, payload: MixUpdate, db: DbDep, user: UserDep) -> MixRead:
return service.update_mix(db, user.id, mix_id, payload)
@router.delete("/mixes/{mix_id}", status_code=204)
def delete_mix(mix_id: int, db: DbDep, user: UserDep) -> None:
service.delete_mix(db, user.id, mix_id)
# ---------------------------------------------------------------------- liquids
@router.get("/liquids", response_model=Page[LiquidEntryRead])
def list_liquids(
db: DbDep,
user: UserDep,
params: PageDep,
from_: FromQuery = None,
to: ToQuery = None,
kind: Annotated[LiquidEntryKind | None, Query()] = None,
sort: SortQuery = None,
) -> Page[LiquidEntryRead]:
return service.list_liquids(db, user.id, params, from_, to, kind, sort)
@router.post("/liquids", response_model=LiquidEntryRead, status_code=201)
def create_liquid(
payload: LiquidEntryCreate, db: DbDep, user: UserDep
) -> LiquidEntryRead:
return service.create_liquid(db, user.id, payload)
@router.patch("/liquids/{entry_id}", response_model=LiquidEntryRead)
def update_liquid(
entry_id: int, payload: LiquidEntryUpdate, db: DbDep, user: UserDep
) -> LiquidEntryRead:
return service.update_liquid(db, user.id, entry_id, payload)
@router.delete("/liquids/{entry_id}", status_code=204)
def delete_liquid(entry_id: int, db: DbDep, user: UserDep) -> None:
service.delete_liquid(db, user.id, entry_id)
# ------------------------------------------------------------------------ coils
@router.get("/coils", response_model=Page[CoilChangeRead])
def list_coils(
db: DbDep,
user: UserDep,
params: PageDep,
from_: FromQuery = None,
to: ToQuery = None,
sort: SortQuery = None,
tz: TzQuery = None,
) -> Page[CoilChangeRead]:
return service.list_coils(db, user.id, params, from_, to, sort, tz)
@router.get("/coils/stats", response_model=StatsResponse)
def coil_stats(db: DbDep, user: UserDep, tz: TzQuery = None) -> StatsResponse:
return service.coil_stats(db, user.id, tz)
@router.post("/coils/change", response_model=CoilChangeCreated, status_code=201)
def quick_coil_change(
db: DbDep,
user: UserDep,
payload: CoilChangeCreate | None = None,
tz: TzQuery = None,
) -> CoilChangeCreated:
"""One-click « Résistance changée » : registers a change at `now` (§12.6)."""
return service.create_coil(db, user.id, payload or CoilChangeCreate(), tz)
@router.post("/coils", response_model=CoilChangeCreated, status_code=201)
def create_coil(
payload: CoilChangeCreate, db: DbDep, user: UserDep, tz: TzQuery = None
) -> CoilChangeCreated:
return service.create_coil(db, user.id, payload, tz)
@router.patch("/coils/{coil_id}", response_model=CoilChangeRead)
def update_coil(
coil_id: int,
payload: CoilChangeUpdate,
db: DbDep,
user: UserDep,
tz: TzQuery = None,
) -> CoilChangeRead:
return service.update_coil(db, user.id, coil_id, payload, tz)
@router.delete("/coils/{coil_id}", status_code=204)
def delete_coil(coil_id: int, db: DbDep, user: UserDep) -> None:
service.delete_coil(db, user.id, coil_id)
# -------------------------------------------------------------------- purchases
@router.get("/purchases", response_model=Page[PurchaseRead])
def list_purchases(
db: DbDep,
user: UserDep,
params: PageDep,
from_: FromQuery = None,
to: ToQuery = None,
product_id: Annotated[int | None, Query()] = None,
sort: SortQuery = None,
) -> Page[PurchaseRead]:
return service.list_purchases(db, user.id, params, from_, to, product_id, sort)
@router.post("/purchases", response_model=PurchaseRead, status_code=201)
def create_purchase(payload: PurchaseCreate, db: DbDep, user: UserDep) -> PurchaseRead:
return service.create_purchase(db, user.id, payload)
@router.patch("/purchases/{purchase_id}", response_model=PurchaseRead)
def update_purchase(
purchase_id: int, payload: PurchaseUpdate, db: DbDep, user: UserDep
) -> PurchaseRead:
return service.update_purchase(db, user.id, purchase_id, payload)
@router.delete("/purchases/{purchase_id}", status_code=204)
def delete_purchase(purchase_id: int, db: DbDep, user: UserDep) -> None:
service.delete_purchase(db, user.id, purchase_id)
# ------------------------------------------------------------- stats & overview
@router.get("/stats/consumption", response_model=StatsResponse)
def consumption_stats(
db: DbDep,
user: UserDep,
from_: FromQuery = None,
to: ToQuery = None,
tz: TzQuery = None,
) -> StatsResponse:
return service.consumption_stats(db, user.id, from_, to, tz)
@router.get("/stats/nicotine", response_model=StatsResponse)
def nicotine_stats(
db: DbDep,
user: UserDep,
from_: FromQuery = None,
to: ToQuery = None,
tz: TzQuery = None,
) -> StatsResponse:
return service.nicotine_stats(db, user.id, from_, to, tz)
@router.get("/stats/costs", response_model=StatsResponse)
def cost_stats(
db: DbDep,
user: UserDep,
from_: FromQuery = None,
to: ToQuery = None,
tz: TzQuery = None,
) -> StatsResponse:
return service.cost_stats(db, user.id, from_, to, tz)
@router.get("/stats/savings", response_model=StatsResponse)
def savings_stats(
db: DbDep,
user: UserDep,
from_: FromQuery = None,
to: ToQuery = None,
tz: TzQuery = None,
) -> StatsResponse:
return service.savings_stats(db, user.id, from_, to, tz)
@router.get("/milestones", response_model=list[MilestoneRead])
def milestones(db: DbDep, user: UserDep, tz: TzQuery = None) -> list[MilestoneRead]:
return service.milestones(db, user.id, tz)
@router.get("/dashboard", response_model=VapeDashboard)
def dashboard(db: DbDep, user: UserDep, tz: TzQuery = None) -> VapeDashboard:
return service.dashboard(db, user.id, tz)
+329
View File
@@ -0,0 +1,329 @@
"""Pydantic v2 schemas for the vape module (API contract §8.4).
Money crosses the API in euro cents: integers for stored amounts,
floats for derived rates (cost/ml, cost/day, cumulative savings).
"""
from datetime import date, datetime
from decimal import Decimal
from typing import Annotated, Any
from pydantic import BaseModel, ConfigDict, Field, PlainSerializer
from app.core.timeutils import UtcDatetime
from app.modules.vape.models import LiquidEntryKind, ProductKind, SizeUnit
# `Numeric` columns (ml, mg/ml, %, Ω, quantities) cross the API as JSON *numbers*,
# never strings: Pydantic serialises `Decimal` as a string by default, which would
# break the fr-FR formatters and the ECharts series of the frontend. Validation
# still runs on Decimal, so precision is kept end to end — same trade-off as the
# `Money` alias of the finance module and the float columns of `health`.
Quantity = Annotated[
Decimal, PlainSerializer(float, return_type=float, when_used="json")
]
# ---------------------------------------------------------------------------
# Settings (singleton per user)
# ---------------------------------------------------------------------------
class VapeSettingsPut(BaseModel):
quit_date: date
cigs_per_day_before: Quantity = Field(ge=0, le=200)
cig_pack_price_cents: int = Field(ge=0)
cigs_per_pack: int = Field(default=20, gt=0)
default_nicotine_mg_ml: Quantity = Field(ge=0, le=100)
currency: str = Field(default="EUR", min_length=3, max_length=3)
class VapeSettingsRead(BaseModel):
model_config = ConfigDict(from_attributes=True)
id: int
quit_date: date
cigs_per_day_before: Quantity
cig_pack_price_cents: int
cigs_per_pack: int
default_nicotine_mg_ml: Quantity
currency: str
# ---------------------------------------------------------------------------
# Products
# ---------------------------------------------------------------------------
class ProductCreate(BaseModel):
kind: ProductKind
name: str = Field(min_length=1, max_length=150)
brand: str | None = Field(default=None, max_length=100)
price_cents: int = Field(ge=0)
size_value: Quantity = Field(gt=0)
size_unit: SizeUnit
nicotine_mg_ml: Quantity | None = Field(default=None, ge=0, le=100)
vg_pct: Quantity | None = Field(default=None, ge=0, le=100)
ohm: Quantity | None = Field(default=None, gt=0)
is_archived: bool = False
note: str | None = Field(default=None, max_length=255)
class ProductUpdate(BaseModel):
kind: ProductKind | None = None
name: str | None = Field(default=None, min_length=1, max_length=150)
brand: str | None = Field(default=None, max_length=100)
price_cents: int | None = Field(default=None, ge=0)
size_value: Quantity | None = Field(default=None, gt=0)
size_unit: SizeUnit | None = None
nicotine_mg_ml: Quantity | None = Field(default=None, ge=0, le=100)
vg_pct: Quantity | None = Field(default=None, ge=0, le=100)
ohm: Quantity | None = Field(default=None, gt=0)
is_archived: bool | None = None
note: str | None = Field(default=None, max_length=255)
class ProductRead(BaseModel):
model_config = ConfigDict(from_attributes=True)
id: int
kind: ProductKind
name: str
brand: str | None
price_cents: int
size_value: Quantity
size_unit: SizeUnit
nicotine_mg_ml: Quantity | None
vg_pct: Quantity | None
ohm: Quantity | None
is_archived: bool
note: str | None
unit_price_cents: float # derived: price_cents / size_value
# ---------------------------------------------------------------------------
# Mixes
# ---------------------------------------------------------------------------
class MixComponentIn(BaseModel):
product_id: int
quantity: Quantity = Field(gt=0)
class MixCreate(BaseModel):
name: str = Field(min_length=1, max_length=150)
total_ml: Quantity = Field(gt=0)
target_nicotine_mg_ml: Quantity = Field(ge=0, le=100)
note: str | None = Field(default=None, max_length=255)
components: list[MixComponentIn] = Field(default_factory=list)
class MixUpdate(BaseModel):
name: str | None = Field(default=None, min_length=1, max_length=150)
total_ml: Quantity | None = Field(default=None, gt=0)
target_nicotine_mg_ml: Quantity | None = Field(default=None, ge=0, le=100)
is_archived: bool | None = None
note: str | None = Field(default=None, max_length=255)
# Full replacement when provided (§8.4)
components: list[MixComponentIn] | None = None
class MixComponentRead(BaseModel):
id: int
product_id: int
product_name: str
product_kind: ProductKind
quantity: Quantity
cost_cents: float
class MixRead(BaseModel):
id: int
name: str
total_ml: Quantity
target_nicotine_mg_ml: Quantity
is_active: bool
is_archived: bool
note: str | None
components: list[MixComponentRead]
cost_total_cents: float
cost_per_ml_cents: float | None
nicotine_check_mg_ml: float
vg_pct_mix: float | None
warning: str | None # "nicotine_mismatch" when > 10 % off target
class MixCalculatorRequest(BaseModel):
total_ml: Quantity = Field(gt=0)
target_nicotine_mg_ml: Quantity = Field(ge=0, le=100)
booster_product_id: int
base_product_id: int
aroma_pct: Quantity = Field(default=Decimal(0), ge=0, le=100)
aroma_product_id: int | None = None
class MixCalculatorResult(BaseModel):
total_ml: Quantity
target_nicotine_mg_ml: Quantity
booster_ml: float
aroma_ml: float
base_ml: float
nicotine_check_mg_ml: float
cost_total_cents: float
cost_per_ml_cents: float
# ---------------------------------------------------------------------------
# Liquid entries
# ---------------------------------------------------------------------------
class LiquidEntryCreate(BaseModel):
entry_date: date
ml: Quantity = Field(gt=0, le=100)
kind: LiquidEntryKind = LiquidEntryKind.REFILL
nicotine_mg_ml: Quantity | None = Field(default=None, ge=0, le=100)
mix_id: int | None = None
note: str | None = Field(default=None, max_length=255)
class LiquidEntryUpdate(BaseModel):
entry_date: date | None = None
ml: Quantity | None = Field(default=None, gt=0, le=100)
kind: LiquidEntryKind | None = None
nicotine_mg_ml: Quantity | None = Field(default=None, ge=0, le=100)
mix_id: int | None = None
note: str | None = Field(default=None, max_length=255)
class LiquidEntryRead(BaseModel):
model_config = ConfigDict(from_attributes=True)
id: int
entry_date: date
kind: LiquidEntryKind
ml: Quantity
nicotine_mg_ml: Quantity | None
nicotine_effective_mg_ml: float | None # override -> mix -> settings default
nicotine_mg: float | None # ml × effective rate
mix_id: int | None
source: str
note: str | None
# ---------------------------------------------------------------------------
# Coil changes
# ---------------------------------------------------------------------------
class CoilChangeCreate(BaseModel):
changed_at: UtcDatetime | None = None # default: now (one-click)
product_id: int | None = None
reason: str | None = Field(default=None, max_length=50)
note: str | None = Field(default=None, max_length=255)
class CoilChangeUpdate(BaseModel):
changed_at: UtcDatetime | None = None
product_id: int | None = None
reason: str | None = Field(default=None, max_length=50)
note: str | None = Field(default=None, max_length=255)
class CoilChangeRead(BaseModel):
id: int
changed_at: datetime
product_id: int | None
product_name: str | None
reason: str | None
note: str | None
is_current: bool
lifespan_days: float | None # provisional (now changed_at) for the current one
ml_through: float | None # Σ daily ml over the coil's interval
class CoilChangeCreated(CoilChangeRead):
previous_lifespan_days: float | None # for the one-click toast
# ---------------------------------------------------------------------------
# Purchases
# ---------------------------------------------------------------------------
class PurchaseCreate(BaseModel):
purchased_on: date
product_id: int
qty: Quantity = Field(default=Decimal(1), gt=0)
unit_price_cents: int | None = Field(default=None, ge=0)
note: str | None = Field(default=None, max_length=255)
class PurchaseUpdate(BaseModel):
purchased_on: date | None = None
product_id: int | None = None
qty: Quantity | None = Field(default=None, gt=0)
unit_price_cents: int | None = Field(default=None, ge=0)
note: str | None = Field(default=None, max_length=255)
class PurchaseRead(BaseModel):
id: int
purchased_on: date
product_id: int
product_name: str
qty: Quantity
unit_price_cents: int | None
total_cents: float # qty × coalesce(unit_price_cents, catalog price)
note: str | None
# ---------------------------------------------------------------------------
# Stats (ECharts-ready, §8.1) / milestones / dashboard
# ---------------------------------------------------------------------------
class SeriesModel(BaseModel):
name: str
type: str # "line" | "scatter" | "bar"
points: list[tuple[str, float | None]]
class StatsResponse(BaseModel):
model_config = ConfigDict(populate_by_name=True)
from_: date = Field(alias="from")
to: date
unit: str # "ml" | "mg" | "cents" | "days"
series: list[SeriesModel]
meta: dict[str, Any]
class MilestoneRead(BaseModel):
code: str
label_fr: str
reached_at: datetime
achieved: bool
progress_pct: float
class VapeDashboard(BaseModel):
quit_date: date
days_since_quit: int
ml_today: float | None
ml_per_day_7: float | None
ml_per_day_30: float | None
nicotine_today_mg: float | None
cost_per_ml_cents: float | None
coil_cost_per_day_cents: float
vape_cost_per_day_cents: float | None
cig_cost_per_day_cents: float
savings_theoretical_cents: float
savings_real_cents: float
savings_display_cents: float # real when a purchase exists, else theoretical
has_purchases: bool
cigarettes_avoided: int
packs_avoided: float
current_coil_age_days: float | None
coil_avg_lifespan_days: float
coil_lifespan_is_default: bool
next_milestone: MilestoneRead | None
File diff suppressed because it is too large Load Diff
View File
+103
View File
@@ -0,0 +1,103 @@
"""Shared test fixtures.
The suite runs on SQLite in-memory (StaticPool, see app.core.database), so the
environment MUST be configured before any app import. Works with any
LIFETRACK_MODULES subset: `auth` and `imports` are always forced in because the
shared fixtures (user, tokens, device keys, import runs) rely on them.
"""
import os
os.environ.setdefault("LIFETRACK_DATABASE_URL", "sqlite+pysqlite:///:memory:")
os.environ.setdefault("LIFETRACK_JWT_SECRET", "test-secret-not-for-production-32byte")
_mods = os.environ.get("LIFETRACK_MODULES", "").strip()
if _mods:
_names = {part.strip() for part in _mods.split(",") if part.strip()}
_names.update({"auth", "imports"})
os.environ["LIFETRACK_MODULES"] = ",".join(sorted(_names))
from collections.abc import Callable, Iterator
from dataclasses import dataclass
import pytest
from fastapi.testclient import TestClient
from sqlalchemy.orm import Session
from app.core.database import Base, SessionLocal, engine
from app.core.module_loader import import_side_modules
from app.core.security import create_access_token
from app.main import app
from app.modules.auth import service as auth_service
from app.modules.auth.models import User
from app.modules.auth.schemas import DeviceKeyCreate, SetupRequest
TEST_USER_EMAIL = "test@lifetrack.local"
TEST_USER_PASSWORD = "correct-horse-battery"
@pytest.fixture()
def _schema() -> Iterator[None]:
"""Fresh tables for every test (all modules' models are in the metadata)."""
import_side_modules()
Base.metadata.drop_all(bind=engine)
Base.metadata.create_all(bind=engine)
yield
@pytest.fixture()
def client(_schema: None) -> Iterator[TestClient]:
with TestClient(app) as test_client:
yield test_client
@pytest.fixture()
def db(_schema: None) -> Iterator[Session]:
with SessionLocal() as session:
yield session
@pytest.fixture()
def user(db: Session) -> User:
return auth_service.create_first_user(
db,
SetupRequest(
email=TEST_USER_EMAIL,
password=TEST_USER_PASSWORD,
display_name="Testeur",
),
)
@pytest.fixture()
def make_auth_headers() -> Callable[[int], dict[str, str]]:
"""Helper building an Authorization header for any user id."""
def _make(user_id: int) -> dict[str, str]:
return {"Authorization": f"Bearer {create_access_token(user_id)}"}
return _make
@pytest.fixture()
def auth_headers(
user: User, make_auth_headers: Callable[[int], dict[str, str]]
) -> dict[str, str]:
return make_auth_headers(user.id)
@dataclass
class DeviceKeyFixture:
id: int
plaintext: str
headers: dict[str, str]
@pytest.fixture()
def device_key(db: Session, user: User) -> DeviceKeyFixture:
"""A device API key holding the wildcard ingest scope."""
key, plaintext = auth_service.create_device_key(
db, user.id, DeviceKeyCreate(name="Clé de test", scopes=["ingest:*"])
)
return DeviceKeyFixture(
id=key.id, plaintext=plaintext, headers={"X-API-Key": plaintext}
)
View File
+10
View File
@@ -0,0 +1,10 @@
Numéro Compte ;12345678901
Type ;COMPTE
Compte tenu en ;euros
Date ;30/06/2026
Solde (EUROS) ;1 234,56
Solde (FRANCS) ;8 098,12
Date;Libellé;Montant(EUROS);Montant(FRANCS)
20/06/2026;ACHAT CB LIDL 1906;-32,15;-210,89
21/06/2026;VIR SEPA CAF PRESTATIONS;185,00;1213,52
1 Numéro Compte ;12345678901
2 Type ;COMPTE
3 Compte tenu en ;euros
4 Date ;30/06/2026
5 Solde (EUROS) ;1 234,56
6 Solde (FRANCS) ;8 098,12
7 Date;Libellé;Montant(EUROS);Montant(FRANCS)
8 20/06/2026;ACHAT CB LIDL 1906;-32,15;-210,89
9 21/06/2026;VIR SEPA CAF PRESTATIONS;185,00;1213,52
+4
View File
@@ -0,0 +1,4 @@
"Cr&eacute;dit immobilier";"Cr&amp;eacute;dit immobilier";****1234;18/03/2026;;-123 456,78
05/01/2026;;; AMORTISSEMENT PRET 1234;-70,93
06/01/2026;;; CARTE 05/01 LECLERC;-45,20
07/01/2026;;; VIREMENT SALAIRE;2 450,00
1 Cr&eacute;dit immobilier Cr&amp;eacute;dit immobilier ****1234 18/03/2026 -123 456,78
2 05/01/2026 AMORTISSEMENT PRET 1234 -70,93
3 06/01/2026 CARTE 05/01 LECLERC -45,20
4 07/01/2026 VIREMENT SALAIRE 2 450,00
+9
View File
@@ -0,0 +1,9 @@
dateOp;dateVal;label;category;categoryParent;amount;comment;accountNum;accountLabel;accountbalance
2026-06-01;2026-06-01;"CARTE 31/05 CARREFOUR CITY PARIS";"Alimentation";"Vie quotidienne";-42,90;;001234;"BOURSORAMA BANQUE";1226.68
2026-06-02;2026-06-02;"VIR SEPA LOYER JUIN";"Logement";"Logement";-750,00;;001234;"BOURSORAMA BANQUE";476.68
2026-06-03;2026-06-03;"VIR INST SALAIRE MAI";"Salaire";"Revenus";2450,00;;001234;"BOURSORAMA BANQUE";2926.68
2026-06-05;2026-06-05;"CARTE 04/06 BOULANGERIE DU COIN";"Alimentation";"Vie quotidienne";-2,50;;001234;"BOURSORAMA BANQUE";2924.18
2026-06-05;2026-06-05;"CARTE 04/06 BOULANGERIE DU COIN";"Alimentation";"Vie quotidienne";-2,50;;001234;"BOURSORAMA BANQUE";2921.68
2026-06-10;2026-06-10;"PRLV SEPA NETFLIX.COM";"Loisirs";"Loisirs";-13,49;;001234;"BOURSORAMA BANQUE";2908.19
2026-06-12;2026-06-12;"VIR SEPA VERS PAYPAL EUROPE";"Virements";"Virements";-50,00;;001234;"BOURSORAMA BANQUE";2858.19
date-invalide;2026-06-15;"LIGNE CASSEE";"";"";-1,00;;001234;"BOURSORAMA BANQUE";2857.19
1 dateOp dateVal label category categoryParent amount comment accountNum accountLabel accountbalance
2 2026-06-01 2026-06-01 CARTE 31/05 CARREFOUR CITY PARIS Alimentation Vie quotidienne -42,90 001234 BOURSORAMA BANQUE 1226.68
3 2026-06-02 2026-06-02 VIR SEPA LOYER JUIN Logement Logement -750,00 001234 BOURSORAMA BANQUE 476.68
4 2026-06-03 2026-06-03 VIR INST SALAIRE MAI Salaire Revenus 2450,00 001234 BOURSORAMA BANQUE 2926.68
5 2026-06-05 2026-06-05 CARTE 04/06 BOULANGERIE DU COIN Alimentation Vie quotidienne -2,50 001234 BOURSORAMA BANQUE 2924.18
6 2026-06-05 2026-06-05 CARTE 04/06 BOULANGERIE DU COIN Alimentation Vie quotidienne -2,50 001234 BOURSORAMA BANQUE 2921.68
7 2026-06-10 2026-06-10 PRLV SEPA NETFLIX.COM Loisirs Loisirs -13,49 001234 BOURSORAMA BANQUE 2908.19
8 2026-06-12 2026-06-12 VIR SEPA VERS PAYPAL EUROPE Virements Virements -50,00 001234 BOURSORAMA BANQUE 2858.19
9 date-invalide 2026-06-15 LIGNE CASSEE -1,00 001234 BOURSORAMA BANQUE 2857.19
@@ -0,0 +1,3 @@
Date de comptabilisation;Libelle simplifie;Libelle operation;Reference;Informations complementaires;Type operation;Categorie;Sous categorie;Debit;Credit;Date operation;Date de valeur;Pointage operation
15/11/2026;SUPERMARCHE;CB SUPERMARCHE CENTRAL FACT 141126;;;Carte bancaire;Alimentation;Hyper/supermarche;-45,50;;14/11/2026;15/11/2026;0
10/11/2026;EMPLOYEUR SA;VIR INST Employeur SA;REF123456;Salaire Novembre-;Virement recu;Revenus;Salaires;;+3500,00;09/11/2026;09/11/2026;0
1 Date de comptabilisation Libelle simplifie Libelle operation Reference Informations complementaires Type operation Categorie Sous categorie Debit Credit Date operation Date de valeur Pointage operation
2 15/11/2026 SUPERMARCHE CB SUPERMARCHE CENTRAL FACT 141126 Carte bancaire Alimentation Hyper/supermarche -45,50 14/11/2026 15/11/2026 0
3 10/11/2026 EMPLOYEUR SA VIR INST Employeur SA REF123456 Salaire Novembre- Virement recu Revenus Salaires +3500,00 09/11/2026 09/11/2026 0
+13
View File
@@ -0,0 +1,13 @@
Liste des opérations
Compte de dépôt n° 12345678901
Période du 01/06/2026 au 30/06/2026
Solde au 30/06/2026;1 234,56
Date;Date valeur;Libellé;Débit euros;Crédit euros;
02/06/2026;02/06/2026;"PAIEMENT CB 0106 INTERMARCHE
FACT 010626";58,20;;
03/06/2026;04/06/2026;VIREMENT DE M DUPONT;;1 500,00;
05/06/2026;05/06/2026;PRLV SEPA EDF ENERGIE;89,00;;
06/06/2026;06/06/2026;CARTE 05/06 TOTAL ENERGIES;45,10;;
Total des opérations;;;192,30;1 500,00;
1 Liste des opérations
2 Compte de dépôt n° 12345678901
3 Période du 01/06/2026 au 30/06/2026
4 Solde au 30/06/2026 1 234,56
5 Date Date valeur Libellé Débit euros Crédit euros
6 02/06/2026 02/06/2026 PAIEMENT CB 0106 INTERMARCHE FACT 010626 58,20
7 03/06/2026 04/06/2026 VIREMENT DE M DUPONT 1 500,00
8 05/06/2026 05/06/2026 PRLV SEPA EDF ENERGIE 89,00
9 06/06/2026 06/06/2026 CARTE 05/06 TOTAL ENERGIES 45,10
10 Total des opérations 192,30 1 500,00
+3
View File
@@ -0,0 +1,3 @@
Date opération;Date valeur;libellé;Débit;Crédit;
13/12/2026;13/12/2026;CARTE 12/12 FNAC METZ;-6,4;;
20/12/2026;20/12/2026;VIREMENT SALAIRE DECEMBRE;;2 500,00;
1 Date opération Date valeur libellé Débit Crédit
2 13/12/2026 13/12/2026 CARTE 12/12 FNAC METZ -6,4
3 20/12/2026 20/12/2026 VIREMENT SALAIRE DECEMBRE 2 500,00
+3
View File
@@ -0,0 +1,3 @@
date,libelle,montant
2026-06-02,Achat supermarche,-25.40
2026-06-03,Remboursement mutuelle,42.10
1 date libelle montant
2 2026-06-02 Achat supermarche -25.40
3 2026-06-03 Remboursement mutuelle 42.10
+3
View File
@@ -0,0 +1,3 @@
"Booking Date","Value Date","Partner Name","Partner Iban",Type,"Payment Reference","Account Name","Amount (EUR)","Original Amount","Original Currency","Exchange Rate"
"2026-02-01","2026-02-01","Netflix","DE1234",Debit,"Abonnement fevrier","Main Account","-13.49","","",""
"2026-02-03","2026-02-03","Employeur SA","DE9999",Credit,"Salaire","Main Account","2500.00","","",""
1 Booking Date Value Date Partner Name Partner Iban Type Payment Reference Account Name Amount (EUR) Original Amount Original Currency Exchange Rate
2 2026-02-01 2026-02-01 Netflix DE1234 Debit Abonnement fevrier Main Account -13.49
3 2026-02-03 2026-02-03 Employeur SA DE9999 Credit Salaire Main Account 2500.00
+8
View File
@@ -0,0 +1,8 @@
Date,Heure,Fuseau horaire,Nom,Type,État,Devise,Brut,Frais,Net,Adresse email de l'expéditeur,Adresse email du destinataire,Numéro de transaction,Titre de l'objet,Numéro de la transaction de référence,Solde,Impact sur le solde
04/06/2026,10:12:00,CEST,Steam Games,"Paiement express",Effectué,EUR,"-12,99","0,00","-12,99",moi@example.com,pay@steam.com,5AB12345CD678901E,"Half-Life 3",,"37,01",Débit
05/06/2026,11:00:00,CEST,Boutique Vape,"Paiement",Autorisation,EUR,"-25,00","0,00","-25,00",moi@example.com,shop@vape.fr,5AB12345CD678902E,"Kit",,"37,01",Mémo
06/06/2026,09:30:00,CEST,Client Vinted,"Paiement",En attente,EUR,"18,00","-0,50","17,50",client@example.com,moi@example.com,5AB12345CD678903E,"Veste",,"54,51",Crédit
07/06/2026,09:30:00,CEST,Client Vinted,"Paiement",Effectué,EUR,"18,00","-0,50","17,50",client@example.com,moi@example.com,5AB12345CD678904E,"Veste",,"54,51",Crédit
08/06/2026,08:00:00,CEST,Amazon US,"Paiement",Effectué,USD,"-30,00","0,00","-30,00",moi@example.com,pay@amazon.com,5AB12345CD678905E,"Livre",,"12,00",Débit
09/06/2026,08:05:00,CEST,PayPal,"Conversion de devise générale",Effectué,EUR,"-27,10","0,00","-27,10",moi@example.com,,5AB12345CD678906E,,5AB12345CD678905E,"9,91",Débit
12/06/2026,07:00:00,CEST,Ma Banque,"Virement depuis un compte bancaire",Effectué,EUR,"50,00","0,00","50,00",moi@example.com,,5AB12345CD678907E,,,"59,91",Crédit
1 Date Heure Fuseau horaire Nom Type État Devise Brut Frais Net Adresse email de l'expéditeur Adresse email du destinataire Numéro de transaction Titre de l'objet Numéro de la transaction de référence Solde Impact sur le solde
2 04/06/2026 10:12:00 CEST Steam Games Paiement express Effectué EUR -12,99 0,00 -12,99 moi@example.com pay@steam.com 5AB12345CD678901E Half-Life 3 37,01 Débit
3 05/06/2026 11:00:00 CEST Boutique Vape Paiement Autorisation EUR -25,00 0,00 -25,00 moi@example.com shop@vape.fr 5AB12345CD678902E Kit 37,01 Mémo
4 06/06/2026 09:30:00 CEST Client Vinted Paiement En attente EUR 18,00 -0,50 17,50 client@example.com moi@example.com 5AB12345CD678903E Veste 54,51 Crédit
5 07/06/2026 09:30:00 CEST Client Vinted Paiement Effectué EUR 18,00 -0,50 17,50 client@example.com moi@example.com 5AB12345CD678904E Veste 54,51 Crédit
6 08/06/2026 08:00:00 CEST Amazon US Paiement Effectué USD -30,00 0,00 -30,00 moi@example.com pay@amazon.com 5AB12345CD678905E Livre 12,00 Débit
7 09/06/2026 08:05:00 CEST PayPal Conversion de devise générale Effectué EUR -27,10 0,00 -27,10 moi@example.com 5AB12345CD678906E 5AB12345CD678905E 9,91 Débit
8 12/06/2026 07:00:00 CEST Ma Banque Virement depuis un compte bancaire Effectué EUR 50,00 0,00 50,00 moi@example.com 5AB12345CD678907E 59,91 Crédit
+4
View File
@@ -0,0 +1,4 @@
Type,Product,Started Date,Completed Date,Description,Amount,Fee,Currency,State,Balance
CARD_PAYMENT,Current,2026-01-05 14:00:40,2026-01-05 14:00:41,Uber Eats,-15.00,0.50,EUR,COMPLETED,74.43
TOPUP,Current,2026-01-06 09:00:00,2026-01-06 09:00:01,Payment from M Dupont,10.00,0.00,EUR,COMPLETED,84.43
CARD_PAYMENT,Current,2026-01-07 10:00:00,,Boutique en attente,-20.00,0.00,EUR,PENDING,64.43
1 Type Product Started Date Completed Date Description Amount Fee Currency State Balance
2 CARD_PAYMENT Current 2026-01-05 14:00:40 2026-01-05 14:00:41 Uber Eats -15.00 0.50 EUR COMPLETED 74.43
3 TOPUP Current 2026-01-06 09:00:00 2026-01-06 09:00:01 Payment from M Dupont 10.00 0.00 EUR COMPLETED 84.43
4 CARD_PAYMENT Current 2026-01-07 10:00:00 Boutique en attente -20.00 0.00 EUR PENDING 64.43
+25
View File
@@ -0,0 +1,25 @@
OFXHEADER:100
DATA:OFXSGML
VERSION:102
SECURITY:NONE
ENCODING:USASCII
CHARSET:1252
COMPRESSION:NONE
OLDFILEUID:NONE
NEWFILEUID:NONE
<OFX>
<SIGNONMSGSRSV1><SONRS><STATUS><CODE>0<SEVERITY>INFO</STATUS>
<DTSERVER>20260630120000<LANGUAGE>FRA</SONRS></SIGNONMSGSRSV1>
<BANKMSGSRSV1><STMTTRNRS><TRNUID>1<STATUS><CODE>0<SEVERITY>INFO</STATUS>
<STMTRS><CURDEF>EUR
<BANKACCTFROM><BANKID>30002<BRANCHID>00550<ACCTID>0000123456X<ACCTTYPE>CHECKING</BANKACCTFROM>
<BANKTRANLIST><DTSTART>20260601<DTEND>20260630
<STMTTRN><TRNTYPE>DEBIT<DTPOSTED>20260604<TRNAMT>-12.75<FITID>948 040626 -1275<NAME>CB FNAC METZ</STMTTRN>
<STMTTRN><TRNTYPE>DEBIT<DTPOSTED>20260604<TRNAMT>-12.75<FITID>948 040626 -1275<NAME>CB FNAC METZ</STMTTRN>
<STMTTRN><TRNTYPE>DEBIT<DTPOSTED>20260608<TRNAMT>-59.90<FITID>948 080626 -5990<NAME>PRLV SEPA ORANGE<MEMO>FACTURE JUIN</STMTTRN>
<STMTTRN><TRNTYPE>CREDIT<DTPOSTED>20260610<TRNAMT>1500.00<FITID>948 100626 150000<NAME>VIREMENT RECU<MEMO>SALAIRE MAI</STMTTRN>
</BANKTRANLIST>
<LEDGERBAL><BALAMT>1414.60<DTASOF>20260630</LEDGERBAL>
</STMTRS></STMTTRNRS></BANKMSGSRSV1>
</OFX>
@@ -0,0 +1,6 @@
="0201900016400270";01/06/2026;30/06/2026;
date_comptabilisation;libellé_complet_operation;montant_operation;devise;
15/06/2026;CARTE X7527 15/06 METRO ;-14,90;EUR;
16/06/2026;PRLV SEPA FREE MOBILE ;-19,99;EUR;
17/06/2026;VIR SEPA SALAIRE JUIN ;2450,00;EUR;
18/06/2026;CARTE X7527 18/06 AMAZON EU ;-1 234,56;EUR;
Can't render this file because it contains an unexpected character in line 1 and column 2.
@@ -0,0 +1,5 @@
Date;Repas;Aliment;Marque;Quantité;Unité;Calories (kcal);Protéines (g);Glucides (g);Lipides (g);Fibres (g);Sucres (g)
2026-08-10 08:15;Petit-déjeuner;Flocons d'avoine;Quaker;60;g;228,0;8,1;39,0;4,2;6,0;0,8
2026-08-10 12:45;Déjeuner;Poulet rôti;;150;g;248,5;46,2;0,0;5,4;0,0;0,0
2026-08-10 20:10;Dîner;Pâtes complètes;Barilla;100;g;350,0;13,0;66,0;2,5;8,0;3,0
2026-08-11 16:00;Collation;Pomme;;120;g;62,0;0,4;16,0;0,2;2,9;12,0
1 Date Repas Aliment Marque Quantité Unité Calories (kcal) Protéines (g) Glucides (g) Lipides (g) Fibres (g) Sucres (g)
2 2026-08-10 08:15 Petit-déjeuner Flocons d'avoine Quaker 60 g 228,0 8,1 39,0 4,2 6,0 0,8
3 2026-08-10 12:45 Déjeuner Poulet rôti 150 g 248,5 46,2 0,0 5,4 0,0 0,0
4 2026-08-10 20:10 Dîner Pâtes complètes Barilla 100 g 350,0 13,0 66,0 2,5 8,0 3,0
5 2026-08-11 16:00 Collation Pomme 120 g 62,0 0,4 16,0 0,2 2,9 12,0
+4
View File
@@ -0,0 +1,4 @@
Date;Pas;Distance (km);Calories actives;Poids (kg)
2026-08-10;9421;6,80;520;92,40
2026-08-11;12034;8,10;640;
2026-08-12;7800;5,20;410;92,10
1 Date Pas Distance (km) Calories actives Poids (kg)
2 2026-08-10 9421 6,80 520 92,40
3 2026-08-11 12034 8,10 640
4 2026-08-12 7800 5,20 410 92,10
+4
View File
@@ -0,0 +1,4 @@
date;poids
2026-07-01;95,20
2026-07-08;94,10
2026-07-15;93,40
1 date poids
2 2026-07-01 95,20
3 2026-07-08 94,10
4 2026-07-15 93,40
@@ -0,0 +1,787 @@
"""HTTP contract of the finance module (datamodel-finance.md §9).
Covers the endpoints an agent/UI can call: seeding, CRUD, import (preview +
run + rollback), rules, budgets, transfers and the chart-ready stats.
"""
from io import BytesIO
from pathlib import Path
from typing import Any
from fastapi.testclient import TestClient
FIXTURES = Path(__file__).resolve().parents[1] / "fixtures" / "finance"
API = "/api/finance"
def upload(name: str) -> tuple[str, BytesIO, str]:
return (name, BytesIO((FIXTURES / name).read_bytes()), "text/csv")
def create_account(
client: TestClient, headers: dict[str, str], name: str = "BoursoBank"
) -> dict[str, Any]:
response = client.post(
f"{API}/accounts",
json={"name": name, "kind": "checking", "initial_balance": 100},
headers=headers,
)
assert response.status_code == 201, response.text
return response.json()
def profile_id(client: TestClient, headers: dict[str, str], name: str) -> str:
profiles = client.get(f"{API}/source-profiles", headers=headers).json()
return next(p["id"] for p in profiles if p["name"] == name)
def import_boursobank(
client: TestClient, headers: dict[str, str], account_id: str
) -> dict[str, Any]:
response = client.post(
f"{API}/imports",
files={"file": upload("boursobank.csv")},
data={
"account_id": account_id,
"source_profile_id": profile_id(
client, headers, "BoursoBank / Boursorama (CSV)"
),
},
headers=headers,
)
assert response.status_code == 201, response.text
return response.json()
# ---------------------------------------------------------------------------
# Auth + seeding
# ---------------------------------------------------------------------------
def test_endpoints_require_authentication(client: TestClient) -> None:
response = client.get(f"{API}/accounts")
assert response.status_code == 401
assert response.json()["error"]["code"] == "unauthorized"
def test_categories_are_seeded_on_first_access(
client: TestClient, auth_headers: dict[str, str]
) -> None:
response = client.get(f"{API}/categories", headers=auth_headers)
assert response.status_code == 200
tree = response.json()
names = {node["name"] for node in tree}
assert {"Alimentation", "Logement", "Transports", "Salaire"} <= names
alimentation = next(n for n in tree if n["name"] == "Alimentation")
assert {c["name"] for c in alimentation["children"]} >= {"Courses", "Livraison"}
transfer = next(n for n in tree if n["kind"] == "transfer")
assert transfer["name"] == "Virements internes"
assert transfer["is_system"] is True
def test_builtin_source_profiles_are_seeded_and_read_only(
client: TestClient, auth_headers: dict[str, str]
) -> None:
profiles = client.get(f"{API}/source-profiles", headers=auth_headers).json()
builtin = [p for p in profiles if p["is_builtin"]]
assert len(builtin) >= 12
assert {"csv", "ofx", "paypal_csv"} == {p["kind"] for p in builtin}
target = next(p for p in builtin if p["name"] == "Fortuneo (CSV)")
forbidden = client.patch(
f"{API}/source-profiles/{target['id']}",
json={"name": "Bidouille"},
headers=auth_headers,
)
assert forbidden.status_code == 403
clone = client.post(
f"{API}/source-profiles/{target['id']}/clone", headers=auth_headers
)
assert clone.status_code == 201
assert clone.json()["is_builtin"] is False
assert clone.json()["name"] == "Fortuneo (CSV) (copie)"
patched = client.patch(
f"{API}/source-profiles/{clone.json()['id']}",
json={"name": "Fortuneo perso"},
headers=auth_headers,
)
assert patched.status_code == 200
def test_deleting_a_category_cleans_transactions_budgets_and_rules(
client: TestClient, auth_headers: dict[str, str]
) -> None:
account = create_account(client, auth_headers)
import_boursobank(client, auth_headers, account["id"])
tree = client.get(f"{API}/categories", headers=auth_headers).json()
alimentation = next(n for n in tree if n["name"] == "Alimentation")
courses = next(c for c in alimentation["children"] if c["name"] == "Courses")
rule = client.post(
f"{API}/rules",
json={
"name": "Courses Carrefour",
"matchers": {"label_contains": ["CARREFOUR"]},
"actions": {"set_category_id": courses["id"]},
},
headers=auth_headers,
).json()
client.post(
f"{API}/rules/apply", json={"scope": "uncategorized"}, headers=auth_headers
)
client.post(
f"{API}/budgets",
json={
"category_id": alimentation["id"],
"monthly_amount": 400,
"start_month": "2026-01-01",
},
headers=auth_headers,
)
system = next(n for n in tree if n["is_system"])
refused = client.delete(f"{API}/categories/{system['id']}", headers=auth_headers)
assert refused.status_code == 422
deleted = client.delete(
f"{API}/categories/{alimentation['id']}", headers=auth_headers
)
assert deleted.status_code == 204
tree_after = client.get(f"{API}/categories", headers=auth_headers).json()
names = {node["name"] for node in tree_after}
assert "Alimentation" not in names
tx = client.get(
f"{API}/transactions", params={"q": "carrefour"}, headers=auth_headers
).json()["items"][0]
assert tx["category_id"] is None
assert tx["category_source"] is None
assert client.get(f"{API}/budgets", headers=auth_headers).json() == []
rules = client.get(f"{API}/rules", headers=auth_headers).json()
orphan = next(r for r in rules if r["id"] == rule["id"])
assert orphan["actions"].get("set_category_id") is None
assert orphan["enabled"] is False
def test_category_depth_is_limited_to_two_levels(
client: TestClient, auth_headers: dict[str, str]
) -> None:
tree = client.get(f"{API}/categories", headers=auth_headers).json()
child = next(n for n in tree if n["name"] == "Alimentation")["children"][0]
response = client.post(
f"{API}/categories",
json={"name": "Trop profond", "parent_id": child["id"]},
headers=auth_headers,
)
assert response.status_code == 422
assert "deux niveaux" in response.json()["error"]["message"]
# ---------------------------------------------------------------------------
# Accounts
# ---------------------------------------------------------------------------
def test_accounts_crud(client: TestClient, auth_headers: dict[str, str]) -> None:
account = create_account(client, auth_headers)
assert account["balance"] == 100.0
assert account["transaction_count"] == 0
duplicate = client.post(
f"{API}/accounts", json={"name": "BoursoBank"}, headers=auth_headers
)
assert duplicate.status_code == 409
patched = client.patch(
f"{API}/accounts/{account['id']}",
json={"institution": "BoursoBank", "is_archived": True},
headers=auth_headers,
)
assert patched.status_code == 200
assert patched.json()["is_archived"] is True
assert client.get(f"{API}/accounts", headers=auth_headers).json() == []
listed = client.get(
f"{API}/accounts", params={"include_archived": True}, headers=auth_headers
).json()
assert len(listed) == 1
assert (
client.delete(
f"{API}/accounts/{account['id']}", headers=auth_headers
).status_code
== 204
)
def test_account_with_transactions_cannot_be_deleted(
client: TestClient, auth_headers: dict[str, str]
) -> None:
account = create_account(client, auth_headers)
import_boursobank(client, auth_headers, account["id"])
response = client.delete(f"{API}/accounts/{account['id']}", headers=auth_headers)
assert response.status_code == 409
assert "Archivez" in response.json()["error"]["message"]
# ---------------------------------------------------------------------------
# Imports
# ---------------------------------------------------------------------------
def test_import_preview_writes_nothing(
client: TestClient, auth_headers: dict[str, str]
) -> None:
account = create_account(client, auth_headers)
response = client.post(
f"{API}/imports/preview",
files={"file": upload("boursobank.csv")},
data={
"account_id": account["id"],
"source_profile_id": profile_id(
client, auth_headers, "BoursoBank / Boursorama (CSV)"
),
},
headers=auth_headers,
)
assert response.status_code == 200
body = response.json()
assert body["rows_total"] == 8
assert body["rows_error"] == 1
assert body["would_skip_duplicates"] == 0
assert body["date_min"] == "2026-06-01"
assert len(body["rows_preview"]) == 7
assert body["rows_preview"][0]["amount"] == -42.9
listed = client.get(f"{API}/transactions", headers=auth_headers).json()
assert listed["total"] == 0
def test_import_then_rollback(client: TestClient, auth_headers: dict[str, str]) -> None:
account = create_account(client, auth_headers)
run = import_boursobank(client, auth_headers, account["id"])
assert run["status"] == "completed"
assert run["stats"]["rows_imported"] == 7
runs = client.get(f"{API}/imports", headers=auth_headers).json()
assert runs["total"] == 1
detail = client.get(f"{API}/imports/{run['id']}", headers=auth_headers)
assert detail.status_code == 200
# A duplicate re-upload is reported, not inserted twice.
again = import_boursobank(client, auth_headers, account["id"])
assert again["stats"]["rows_imported"] == 0
assert again["stats"]["duplicate_file_of"] == run["import_run_id"]
deleted = client.delete(f"{API}/imports/{run['id']}", headers=auth_headers)
assert deleted.status_code == 204
remaining = client.get(f"{API}/transactions", headers=auth_headers).json()
assert remaining["total"] == 0
def test_rollback_refuses_manually_edited_rows_without_force(
client: TestClient, auth_headers: dict[str, str]
) -> None:
account = create_account(client, auth_headers)
run = import_boursobank(client, auth_headers, account["id"])
tx = client.get(f"{API}/transactions", headers=auth_headers).json()["items"][0]
categories = client.get(f"{API}/categories", headers=auth_headers).json()
courses = next(
c for node in categories for c in node["children"] if c["name"] == "Courses"
)
client.patch(
f"{API}/transactions/{tx['id']}",
json={"category_id": courses["id"]},
headers=auth_headers,
)
blocked = client.delete(f"{API}/imports/{run['id']}", headers=auth_headers)
assert blocked.status_code == 409
forced = client.delete(
f"{API}/imports/{run['id']}", params={"force": True}, headers=auth_headers
)
assert forced.status_code == 204
def test_generic_imports_endpoint_serves_the_finance_importers(
client: TestClient, auth_headers: dict[str, str]
) -> None:
"""The registered importers (C3) also work through POST /api/imports."""
sources = client.get("/api/imports/sources", headers=auth_headers).json()
finance_ids = {s["id"] for s in sources if s["domain"] == "finance"}
assert {"bank_generic_csv", "bank_ofx", "paypal_csv"} <= finance_ids
first = client.post(
"/api/imports",
files={"file": upload("sample.ofx")},
data={"source": "bank_ofx"},
headers=auth_headers,
)
assert first.status_code == 201, first.text
assert first.json()["rows_inserted"] == 4
assert first.json()["status"] == "completed"
# A default account was created for that format…
accounts = client.get(f"{API}/accounts", headers=auth_headers).json()
assert [a["name"] for a in accounts] == ["Compte importé (OFX)"]
assert accounts[0]["transaction_count"] == 4
# …and re-importing the same file stays idempotent.
again = client.post(
"/api/imports",
files={"file": upload("sample.ofx")},
data={"source": "bank_ofx"},
headers=auth_headers,
)
assert again.json()["rows_inserted"] == 0
assert again.json()["rows_duplicates"] == 4
# Central rollback removes the finance rows (FK ON DELETE CASCADE).
assert (
client.delete(
f"/api/imports/{first.json()['id']}", headers=auth_headers
).status_code
== 204
)
assert client.get(f"{API}/transactions", headers=auth_headers).json()["total"] == 0
# ---------------------------------------------------------------------------
# Transactions
# ---------------------------------------------------------------------------
def test_transactions_filters_and_manual_flag(
client: TestClient, auth_headers: dict[str, str]
) -> None:
account = create_account(client, auth_headers)
import_boursobank(client, auth_headers, account["id"])
page = client.get(
f"{API}/transactions",
params={"page_size": 3, "sort": "booked_date"},
headers=auth_headers,
).json()
assert page["total"] == 7
assert len(page["items"]) == 3
assert page["items"][0]["booked_date"] == "2026-06-01"
assert page["items"][0]["account_name"] == "BoursoBank"
credits = client.get(
f"{API}/transactions", params={"direction": "credit"}, headers=auth_headers
).json()
assert credits["total"] == 1
assert credits["items"][0]["amount"] == 2450.0
searched = client.get(
f"{API}/transactions", params={"q": "netflix"}, headers=auth_headers
).json()
assert searched["total"] == 1
window = client.get(
f"{API}/transactions",
params={"date_from": "2026-06-05", "date_to": "2026-06-10"},
headers=auth_headers,
).json()
assert window["total"] == 3
uncategorized = client.get(
f"{API}/transactions", params={"category_id": "none"}, headers=auth_headers
).json()
assert uncategorized["total"] == 7
bad_sort = client.get(
f"{API}/transactions", params={"sort": "hacker"}, headers=auth_headers
)
assert bad_sort.status_code == 422
def test_patch_and_bulk_categorize_set_user_source(
client: TestClient, auth_headers: dict[str, str]
) -> None:
account = create_account(client, auth_headers)
import_boursobank(client, auth_headers, account["id"])
items = client.get(f"{API}/transactions", headers=auth_headers).json()["items"]
categories = client.get(f"{API}/categories", headers=auth_headers).json()
courses = next(
c for node in categories for c in node["children"] if c["name"] == "Courses"
)
patched = client.patch(
f"{API}/transactions/{items[0]['id']}",
json={"category_id": courses["id"], "notes": "Vérifié"},
headers=auth_headers,
).json()
assert patched["category_source"] == "user"
assert patched["category_name"] == "Courses"
assert patched["notes"] == "Vérifié"
# Date/amount of an imported row stay read-only.
refused = client.patch(
f"{API}/transactions/{items[0]['id']}",
json={"amount": -1},
headers=auth_headers,
)
assert refused.status_code == 422
bulk = client.post(
f"{API}/transactions/bulk-categorize",
json={
"transaction_ids": [item["id"] for item in items[:3]],
"category_id": courses["id"],
},
headers=auth_headers,
).json()
assert bulk["updated"] == 3
# Deleting an imported row is refused (roll the import back instead).
assert (
client.delete(
f"{API}/transactions/{items[0]['id']}", headers=auth_headers
).status_code
== 422
)
def test_manual_transaction_lifecycle(
client: TestClient, auth_headers: dict[str, str]
) -> None:
account = create_account(client, auth_headers)
created = client.post(
f"{API}/transactions",
json={
"account_id": account["id"],
"booked_date": "2026-08-02",
"amount": -12.5,
"label_clean": "Café du coin",
},
headers=auth_headers,
)
assert created.status_code == 201
body = created.json()
assert body["amount"] == -12.5
assert body["import_run_id"] is None
twin = client.post(
f"{API}/transactions",
json={
"account_id": account["id"],
"booked_date": "2026-08-02",
"amount": -12.5,
"label_clean": "Café du coin",
},
headers=auth_headers,
)
assert twin.status_code == 201 # occurrence 1, not a conflict
assert (
client.delete(
f"{API}/transactions/{body['id']}", headers=auth_headers
).status_code
== 204
)
# ---------------------------------------------------------------------------
# Rules
# ---------------------------------------------------------------------------
def test_rules_crud_preview_and_apply(
client: TestClient, auth_headers: dict[str, str]
) -> None:
account = create_account(client, auth_headers)
import_boursobank(client, auth_headers, account["id"])
categories = client.get(f"{API}/categories", headers=auth_headers).json()
courses = next(
c for node in categories for c in node["children"] if c["name"] == "Courses"
)
invalid = client.post(
f"{API}/rules",
json={
"name": "Regex cassée",
"matchers": {"label_regex": "["},
"actions": {"set_category_id": courses["id"]},
},
headers=auth_headers,
)
assert invalid.status_code == 422
empty = client.post(
f"{API}/rules",
json={"name": "Vide", "matchers": {}, "actions": {}},
headers=auth_headers,
)
assert empty.status_code == 422
preview = client.post(
f"{API}/rules/preview",
json={"matchers": {"label_contains": ["CARREFOUR"]}},
headers=auth_headers,
).json()
assert preview["total_matched"] == 1
created = client.post(
f"{API}/rules",
json={
"name": "Courses Carrefour",
"priority": 10,
"matchers": {"label_contains": ["CARREFOUR"], "direction": "debit"},
"actions": {
"set_category_id": courses["id"],
"set_counterparty": "Carrefour",
},
},
headers=auth_headers,
)
assert created.status_code == 201
rule = created.json()
second = client.post(
f"{API}/rules",
json={
"name": "Loyer",
"matchers": {"label_contains": ["LOYER"]},
"actions": {"set_label_clean": "Loyer"},
},
headers=auth_headers,
).json()
reordered = client.post(
f"{API}/rules/reorder",
json={"ordered_ids": [second["id"], rule["id"]]},
headers=auth_headers,
).json()
assert [r["priority"] for r in reordered] == [0, 10]
dry = client.post(
f"{API}/rules/apply",
json={"scope": "uncategorized", "dry_run": True},
headers=auth_headers,
).json()
assert dry["matched"] == 2
assert dry["dry_run"] is True
applied = client.post(
f"{API}/rules/apply", json={"scope": "uncategorized"}, headers=auth_headers
).json()
assert applied["updated"] == 2
assert {entry["name"] for entry in applied["by_rule"]} == {
"Courses Carrefour",
"Loyer",
}
tx = client.get(
f"{API}/transactions", params={"q": "carrefour"}, headers=auth_headers
).json()["items"][0]
assert tx["category_name"] == "Courses"
assert tx["category_source"] == "rule"
assert tx["counterparty"] == "Carrefour"
forced = client.post(
f"{API}/rules/apply", json={"scope": "all"}, headers=auth_headers
)
assert forced.status_code == 422 # force=true required
assert (
client.delete(f"{API}/rules/{rule['id']}", headers=auth_headers).status_code
== 204
)
# ---------------------------------------------------------------------------
# Budgets
# ---------------------------------------------------------------------------
def test_budgets_overlap_and_effective_from(
client: TestClient, auth_headers: dict[str, str]
) -> None:
categories = client.get(f"{API}/categories", headers=auth_headers).json()
alimentation = next(c for c in categories if c["name"] == "Alimentation")
salaire = next(c for c in categories if c["name"] == "Salaire")
created = client.post(
f"{API}/budgets",
json={
"category_id": alimentation["id"],
"monthly_amount": 450,
"start_month": "2026-01-01",
},
headers=auth_headers,
)
assert created.status_code == 201
overlap = client.post(
f"{API}/budgets",
json={
"category_id": alimentation["id"],
"monthly_amount": 500,
"start_month": "2026-06-01",
},
headers=auth_headers,
)
assert overlap.status_code == 409
income_budget = client.post(
f"{API}/budgets",
json={
"category_id": salaire["id"],
"monthly_amount": 100,
"start_month": "2026-01-01",
},
headers=auth_headers,
)
assert income_budget.status_code == 422
split = client.patch(
f"{API}/budgets/{created.json()['id']}",
json={"monthly_amount": 520, "effective_from": "2026-08-01"},
headers=auth_headers,
).json()
assert len(split) == 2
assert split[0]["end_month"] == "2026-07-01"
assert split[1]["start_month"] == "2026-08-01"
assert split[1]["monthly_amount"] == 520.0
listed = client.get(
f"{API}/budgets", params={"month": "2026-08"}, headers=auth_headers
).json()
assert len(listed) == 1
assert listed[0]["monthly_amount"] == 520.0
assert listed[0]["category_name"] == "Alimentation"
assert (
client.delete(
f"{API}/budgets/{split[1]['id']}", headers=auth_headers
).status_code
== 204
)
# ---------------------------------------------------------------------------
# Transfers + stats
# ---------------------------------------------------------------------------
def test_transfer_endpoints(client: TestClient, auth_headers: dict[str, str]) -> None:
bank = create_account(client, auth_headers, "BoursoBank")
paypal = create_account(client, auth_headers, "PayPal")
import_boursobank(client, auth_headers, bank["id"])
response = client.post(
f"{API}/imports",
files={"file": upload("paypal_fr.csv")},
data={
"account_id": paypal["id"],
"source_profile_id": profile_id(
client, auth_headers, "PayPal — rapport d'activité (CSV)"
),
},
headers=auth_headers,
)
assert response.status_code == 201
assert response.json()["stats"]["transfers_detected"] == 1
transfers = client.get(
f"{API}/transactions", params={"is_transfer": True}, headers=auth_headers
).json()
assert transfers["total"] == 2
group_id = transfers["items"][0]["transfer_group_id"]
assert transfers["items"][0]["category_name"] == "Virements internes"
unlinked = client.delete(f"{API}/transfers/{group_id}", headers=auth_headers)
assert unlinked.status_code == 204
assert (
client.get(
f"{API}/transactions", params={"is_transfer": True}, headers=auth_headers
).json()["total"]
== 0
)
detected = client.post(
f"{API}/transfers/detect", json={}, headers=auth_headers
).json()
assert detected["pairs_created"] == 1
legs = client.get(
f"{API}/transactions", params={"is_transfer": True}, headers=auth_headers
).json()["items"]
client.delete(
f"{API}/transfers/{legs[0]['transfer_group_id']}", headers=auth_headers
)
linked = client.post(
f"{API}/transfers/link",
json={
"transaction_id_a": legs[0]["id"],
"transaction_id_b": legs[1]["id"],
},
headers=auth_headers,
)
assert linked.status_code == 200
assert linked.json()["transfer_group_id"]
def test_stats_endpoints_are_chart_ready(
client: TestClient, auth_headers: dict[str, str]
) -> None:
account = create_account(client, auth_headers)
import_boursobank(client, auth_headers, account["id"])
monthly = client.get(
f"{API}/stats/monthly-by-category",
params={"months": 6},
headers=auth_headers,
).json()
assert len(monthly["months"]) == 6
assert all(len(series["data"]) == 6 for series in monthly["series"])
assert monthly["series"][0]["color"].startswith("#")
cashflow = client.get(
f"{API}/stats/cashflow", params={"months": 4}, headers=auth_headers
).json()
assert set(cashflow) == {
"months",
"income",
"expenses",
"net",
"cumulative_net",
}
assert len(cashflow["cumulative_net"]) == 4
merchants = client.get(
f"{API}/stats/top-merchants", params={"months": 24}, headers=auth_headers
).json()
assert merchants["period"]["from"] <= merchants["period"]["to"]
assert merchants["items"]
recurring = client.get(f"{API}/stats/recurring", headers=auth_headers).json()
assert recurring["items"] == []
assert recurring["monthly_total_estimate"] == 0.0
budgets = client.get(
f"{API}/stats/budget-progress",
params={"month": "2026-06"},
headers=auth_headers,
).json()
assert budgets["month"] == "2026-06"
sankey = client.get(
f"{API}/stats/sankey", params={"month": "2026-06"}, headers=auth_headers
).json()
names = [node["name"] for node in sankey["nodes"]]
assert "Revenus" in names
assert len(names) == len(set(names))
assert all(link["value"] > 0 for link in sankey["links"])
dashboard = client.get(f"{API}/dashboard", headers=auth_headers).json()
assert dashboard["accounts_count"] == 1
assert dashboard["uncategorized_count"] == 7
# 100 € initial balance + the net of the 7 imported rows.
assert dashboard["total_balance"] == 1688.61
assert dashboard["month_expenses"] == 0.0 # the file covers June 2026
bad_month = client.get(
f"{API}/stats/budget-progress",
params={"month": "2026-13"},
headers=auth_headers,
)
assert bad_month.status_code == 422
@@ -0,0 +1,400 @@
"""Calculation tests: recurring detection, budget rollup and chart aggregates.
Reference: datamodel-finance.md §7 (recurring), §8 (aggregates), §9.8 (stats).
Every function takes an explicit `today`, so the expectations are deterministic.
"""
import uuid
from datetime import date, timedelta
from decimal import Decimal
from sqlalchemy import select
from sqlalchemy.orm import Session
from app.modules.auth.models import User
from app.modules.finance import service, stats
from app.modules.finance.categorize import (
detect_recurring,
monthly_total_estimate,
)
from app.modules.finance.enums import AccountKind, CategorySource
from app.modules.finance.models import (
FinAccount,
FinBudget,
FinCategory,
FinTransaction,
)
from app.modules.finance.normalize import add_months, compute_dedup_hash
TODAY = date(2026, 8, 13)
THIS_MONTH = date(2026, 8, 1)
def account(db: Session, user: User, name: str = "Compte courant") -> FinAccount:
row = FinAccount(
id=uuid.uuid4(),
user_id=user.id,
name=name,
kind=AccountKind.CHECKING,
currency="EUR",
initial_balance=Decimal("1000.00"),
)
db.add(row)
db.commit()
return row
def category(db: Session, user: User, name: str) -> FinCategory:
service.ensure_seed(db, user.id)
found = db.scalars(
select(FinCategory).where(
FinCategory.user_id == user.id, FinCategory.name == name
)
).first()
assert found is not None, name
return found
def add_tx(
db: Session,
user: User,
acc: FinAccount,
day: date,
amount: str,
label: str,
cat: FinCategory | None = None,
counterparty: str | None = None,
) -> FinTransaction:
value = Decimal(amount)
tx = FinTransaction(
id=uuid.uuid4(),
user_id=user.id,
account_id=acc.id,
booked_date=day,
amount=value,
currency="EUR",
label_raw=label,
label_clean=label,
counterparty=counterparty,
category_id=cat.id if cat else None,
category_source=CategorySource.RULE if cat else None,
dedup_hash=compute_dedup_hash(acc.id, day, value, label, 0)
+ uuid.uuid4().hex[:4],
)
db.add(tx)
db.commit()
return tx
# ---------------------------------------------------------------------------
# §7 — recurring detection
# ---------------------------------------------------------------------------
def test_recurring_detects_a_monthly_subscription(db: Session, user: User) -> None:
acc = account(db, user)
abo = category(db, user, "Abonnements & streaming")
for months_ago in range(6, 0, -1):
day = add_months(THIS_MONTH, -months_ago) + timedelta(days=27)
amount = "-13.49" if months_ago != 3 else "-13.99"
add_tx(db, user, acc, day, amount, "PRLV SEPA NETFLIX.COM", abo)
# noise: a one-off purchase must not create a series
add_tx(db, user, acc, date(2026, 7, 4), "-89.00", "ACHAT CB DARTY 4444")
series = detect_recurring(db, user.id, today=TODAY)
assert len(series) == 1
netflix = series[0]
assert netflix.merchant_key == "NETFLIX COM"
assert netflix.periodicity == "monthly"
assert netflix.occurrences == 6
assert netflix.expected_amount == Decimal("13.49")
assert netflix.category_id == abo.id
assert netflix.is_active is True
assert netflix.next_date_predicted > netflix.last_date
assert monthly_total_estimate(series) == Decimal("13.49")
def test_recurring_ignores_irregular_and_unstable_series(
db: Session, user: User
) -> None:
acc = account(db, user)
# irregular intervals
for day in (date(2026, 3, 2), date(2026, 3, 20), date(2026, 7, 15)):
add_tx(db, user, acc, day, "-30.00", "CARTE BOUTIQUE ALPHA")
# regular but wildly unstable amounts
for index, amount in enumerate(("-10.00", "-90.00", "-200.00")):
add_tx(
db,
user,
acc,
add_months(THIS_MONTH, -(3 - index)) + timedelta(days=4),
amount,
"CARTE BOUTIQUE BETA",
)
assert detect_recurring(db, user.id, today=TODAY) == []
def test_recurring_on_income_direction(db: Session, user: User) -> None:
acc = account(db, user)
salaire = category(db, user, "Salaire")
for months_ago in range(5, 0, -1):
add_tx(
db,
user,
acc,
add_months(THIS_MONTH, -months_ago) + timedelta(days=1),
"2450.00",
"VIR SEPA SALAIRE EMPLOYEUR SA",
salaire,
)
series = detect_recurring(db, user.id, direction="credit", today=TODAY)
assert len(series) == 1
assert series[0].periodicity == "monthly"
assert series[0].expected_amount == Decimal("2450.00")
def test_recurring_marks_inactive_series(db: Session, user: User) -> None:
acc = account(db, user)
for months_ago in range(12, 8, -1):
add_tx(
db,
user,
acc,
add_months(THIS_MONTH, -months_ago) + timedelta(days=9),
"-9.99",
"PRLV SEPA VIEUX SERVICE",
)
assert detect_recurring(db, user.id, today=TODAY) == []
inactive = detect_recurring(db, user.id, include_inactive=True, today=TODAY)
assert len(inactive) == 1
assert inactive[0].is_active is False
# ---------------------------------------------------------------------------
# §8.3 — budgets
# ---------------------------------------------------------------------------
def test_budget_rollup_root_covers_children(db: Session, user: User) -> None:
acc = account(db, user)
alimentation = category(db, user, "Alimentation")
courses = category(db, user, "Courses")
restaurants = category(db, user, "Restaurants & bars")
db.add(
FinBudget(
id=uuid.uuid4(),
user_id=user.id,
category_id=alimentation.id,
monthly_amount=Decimal("450.00"),
start_month=date(2026, 1, 1),
)
)
db.add(
FinBudget(
id=uuid.uuid4(),
user_id=user.id,
category_id=courses.id,
monthly_amount=Decimal("200.00"),
start_month=date(2026, 1, 1),
)
)
db.commit()
add_tx(db, user, acc, date(2026, 8, 3), "-300.00", "COURSES", courses)
add_tx(db, user, acc, date(2026, 8, 6), "-100.00", "RESTO", restaurants)
add_tx(db, user, acc, date(2026, 7, 6), "-500.00", "MOIS PRECEDENT", courses)
progress = stats.budget_progress(db, user.id, THIS_MONTH, today=TODAY)
by_category = {item["category_name"]: item for item in progress["items"]}
root = by_category["Alimentation"]
assert root["actual"] == Decimal("400.00") # child + sibling child
assert root["remaining"] == Decimal("50.00")
assert root["progress_pct"] == 88.9
assert root["status"] == "warning"
assert root["projected_eom"] is not None # current month -> linear projection
child = by_category["Courses"]
assert child["actual"] == Decimal("300.00")
assert child["status"] == "over"
assert progress["totals"]["budget"] == Decimal("650.00")
def test_budget_progress_of_a_past_month_has_no_projection(
db: Session, user: User
) -> None:
acc = account(db, user)
courses = category(db, user, "Courses")
db.add(
FinBudget(
id=uuid.uuid4(),
user_id=user.id,
category_id=courses.id,
monthly_amount=Decimal("200.00"),
start_month=date(2026, 1, 1),
end_month=date(2026, 7, 1),
)
)
db.commit()
add_tx(db, user, acc, date(2026, 7, 6), "-120.00", "COURSES", courses)
progress = stats.budget_progress(db, user.id, date(2026, 7, 1), today=TODAY)
assert progress["items"][0]["projected_eom"] is None
assert progress["items"][0]["status"] == "ok"
# the budget ended in July: it no longer applies in August
assert stats.budget_progress(db, user.id, THIS_MONTH, today=TODAY)["items"] == []
# ---------------------------------------------------------------------------
# §9.8 — chart-ready aggregates
# ---------------------------------------------------------------------------
def _sample_month(db: Session, user: User) -> FinAccount:
acc = account(db, user)
courses = category(db, user, "Courses")
restaurants = category(db, user, "Restaurants & bars")
salaire = category(db, user, "Salaire")
add_tx(
db, user, acc, date(2026, 8, 2), "-120.00", "CARREFOUR", courses, "Carrefour"
)
add_tx(db, user, acc, date(2026, 8, 9), "-80.00", "CARREFOUR", courses, "Carrefour")
add_tx(db, user, acc, date(2026, 8, 10), "-60.00", "BRASSERIE", restaurants)
add_tx(db, user, acc, date(2026, 8, 11), "-25.00", "INCONNU SANS CATEGORIE")
add_tx(db, user, acc, date(2026, 8, 1), "2450.00", "SALAIRE", salaire)
return acc
def test_monthly_by_category_rolls_children_into_roots(db: Session, user: User) -> None:
_sample_month(db, user)
data = stats.monthly_by_category(db, user.id, months=3, today=TODAY)
assert data["months"][-1] == "2026-08"
series = {item["name"]: item for item in data["series"]}
assert series["Alimentation"]["data"][-1] == Decimal("260.00")
assert series["Non catégorisé"]["data"][-1] == Decimal("25.00")
assert series["Non catégorisé"]["color"] == "#9ca3af"
assert data["totals"][-1] == Decimal("285.00")
child_level = stats.monthly_by_category(
db, user.id, months=1, level="child", today=TODAY
)
names = {item["name"] for item in child_level["series"]}
assert {"Courses", "Restaurants & bars"} <= names
def test_cashflow_series(db: Session, user: User) -> None:
_sample_month(db, user)
data = stats.cashflow(db, user.id, months=2, today=TODAY)
assert data["months"] == ["2026-07", "2026-08"]
assert data["income"] == [Decimal("0.00"), Decimal("2450.00")]
assert data["expenses"] == [Decimal("0.00"), Decimal("285.00")]
assert data["net"][-1] == Decimal("2165.00")
assert data["cumulative_net"][-1] == Decimal("2165.00")
def test_transfers_are_excluded_from_stats(db: Session, user: User) -> None:
acc = _sample_month(db, user)
other = account(db, user, "Livret A")
group = uuid.uuid4()
leg_a = add_tx(db, user, acc, date(2026, 8, 12), "-500.00", "VIR VERS LIVRET")
leg_b = add_tx(db, user, other, date(2026, 8, 12), "500.00", "VIR DEPUIS COURANT")
for leg in (leg_a, leg_b):
leg.transfer_group_id = group
db.commit()
data = stats.cashflow(db, user.id, months=1, today=TODAY)
assert data["expenses"] == [Decimal("285.00")]
assert data["income"] == [Decimal("2450.00")]
def test_top_merchants_groups_by_counterparty(db: Session, user: User) -> None:
_sample_month(db, user)
data = stats.top_merchants(db, user.id, months=1, today=TODAY)
top = data["items"][0]
assert top["merchant"] == "Carrefour"
assert top["total"] == Decimal("200.00")
assert top["count"] == 2
assert top["average"] == Decimal("100.00")
assert top["category_name"] == "Courses"
def test_sankey_structure(db: Session, user: User) -> None:
_sample_month(db, user)
data = stats.sankey(db, user.id, month=THIS_MONTH, today=TODAY)
names = [node["name"] for node in data["nodes"]]
assert "Revenus" in names
assert "Salaire" in names
assert "Alimentation" in names
assert "Épargne du mois" in names
links = {(link["source"], link["target"]): link["value"] for link in data["links"]}
assert links[("Salaire", "Revenus")] == Decimal("2450.00")
assert links[("Revenus", "Alimentation")] == Decimal("260.00")
assert links[("Alimentation", "Courses")] == Decimal("200.00")
assert links[("Revenus", "Épargne du mois")] == Decimal("2165.00")
assert len(names) == len(set(names)) # ECharts requires unique node names
def test_sankey_never_nets_uncategorized_credits_against_debits(
db: Session, user: User
) -> None:
"""§9.8: uncategorized credits feed « Autres revenus », uncategorized debits
feed « Non catégorisé » netting the two buckets would drop the expense
AND shrink the income side by the same amount."""
acc = _sample_month(db, user) # already holds a -25.00 uncategorized debit
add_tx(db, user, acc, date(2026, 8, 6), "300.00", "VIR RECU SANS CATEGORIE")
data = stats.sankey(db, user.id, month=THIS_MONTH, today=TODAY)
names = [node["name"] for node in data["nodes"]]
assert "Non catégorisé" in names
assert "Autres revenus" in names
links = {(link["source"], link["target"]): link["value"] for link in data["links"]}
assert links[("Autres revenus", "Revenus")] == Decimal("300.00")
assert links[("Revenus", "Non catégorisé")] == Decimal("25.00")
income = sum(v for (_s, t), v in links.items() if t == "Revenus")
expenses = sum(
v for (s, t), v in links.items() if s == "Revenus" and t != "Épargne du mois"
)
assert income == Decimal("2750.00") # 2450 salary + 300 uncategorized credit
assert expenses == Decimal("285.00") # 120 + 80 + 60 + 25
assert links[("Revenus", "Épargne du mois")] == Decimal("2465.00")
assert len(names) == len(set(names))
def test_sankey_nets_a_refund_inside_its_own_category(db: Session, user: User) -> None:
"""A credit on an EXPENSE category is a refund: it lowers that category's
expense instead of opening an income node with the same name (which would
make the ECharts sankey cyclic)."""
acc = _sample_month(db, user)
courses = category(db, user, "Courses")
add_tx(
db, user, acc, date(2026, 8, 12), "50.00", "REMBOURSEMENT CARREFOUR", courses
)
data = stats.sankey(db, user.id, month=THIS_MONTH, today=TODAY)
links = {(link["source"], link["target"]): link["value"] for link in data["links"]}
assert ("Alimentation", "Revenus") not in links # no cycle through the hub
assert links[("Revenus", "Alimentation")] == Decimal("210.00") # 260 - 50
assert links[("Alimentation", "Courses")] == Decimal("150.00") # 200 - 50
names = [node["name"] for node in data["nodes"]]
assert len(names) == len(set(names))
def test_dashboard_kpis(db: Session, user: User) -> None:
_sample_month(db, user)
courses = category(db, user, "Courses")
db.add(
FinBudget(
id=uuid.uuid4(),
user_id=user.id,
category_id=courses.id,
monthly_amount=Decimal("400.00"),
start_month=date(2026, 1, 1),
)
)
db.commit()
kpi = stats.dashboard(db, user.id, today=TODAY)
assert kpi["month"] == "2026-08"
assert kpi["accounts_count"] == 1
assert kpi["total_balance"] == Decimal("3165.00") # 1000 initial + net
assert kpi["month_expenses"] == Decimal("285.00")
assert kpi["month_income"] == Decimal("2450.00")
assert kpi["month_net"] == Decimal("2165.00")
assert kpi["uncategorized_count"] == 1
assert kpi["budget_total"] == Decimal("400.00")
assert kpi["budget_actual"] == Decimal("200.00")
@@ -0,0 +1,252 @@
"""Parser/normalisation tests against synthetic files of the real bank layouts.
Fixtures live in app/tests/fixtures/finance and reproduce the header lines,
encodings and quirks documented in docs/research/finance-sources.md §2-4.
"""
from datetime import date
from decimal import Decimal
from pathlib import Path
import pytest
from app.modules.finance.normalize import (
compute_dedup_hash,
light_clean,
merchant_key,
normalize_label_for_hash,
parse_amount,
)
from app.modules.finance.parsers import decode_bytes
from app.modules.finance.pipeline import parse_file
from app.modules.finance.presets import BUILTIN_PROFILES
FIXTURES = Path(__file__).resolve().parents[1] / "fixtures" / "finance"
ACCOUNT_UUID = __import__("uuid").UUID("00000000-0000-0000-0000-0000000000aa")
def preset(slug: str) -> tuple[str, dict]:
for item in BUILTIN_PROFILES:
if item["slug"] == slug:
return item["kind"], item["config"]
raise AssertionError(f"preset {slug} missing")
def parse_fixture(slug: str, filename: str):
kind, config = preset(slug)
return parse_file(kind, config, (FIXTURES / filename).read_bytes(), "EUR")
# ---------------------------------------------------------------------------
# Amounts / labels
# ---------------------------------------------------------------------------
@pytest.mark.parametrize(
("raw", "expected"),
[
("-6,4", "-6.4"),
("+3500,00", "3500.00"),
("-123 456,78", "-123456.78"),
("1 234,56", "1234.56"),
("1.234,56", "1234.56"),
("226.68", "226.68"),
("12,50 €", "12.50"),
("12,50", "-12.50"),
("1 234,56", "1234.56"),
],
)
def test_parse_amount_french_variants(raw: str, expected: str) -> None:
assert parse_amount(raw) == Decimal(expected)
def test_parse_amount_dot_decimal_profile() -> None:
assert parse_amount("1,234.56", decimal_separator=".") == Decimal("1234.56")
def test_parse_amount_rejects_garbage() -> None:
with pytest.raises(ValueError, match="montant"):
parse_amount("abc")
def test_normalize_label_is_conservative() -> None:
assert normalize_label_for_hash(" Café Crème\n2 ") == "CAFE CREME 2"
def test_light_clean_strips_technical_prefixes() -> None:
assert light_clean("CARTE 04/06 BOULANGERIE") == "BOULANGERIE"
assert light_clean("PRLV SEPA NETFLIX.COM") == "NETFLIX.COM"
assert light_clean("VIR INST SALAIRE MAI") == "SALAIRE MAI"
def test_merchant_key_drops_dates_and_numbers() -> None:
key = merchant_key("CARTE 04/06 CARREFOUR CITY 123456", None)
assert key == "CARREFOUR CITY"
assert merchant_key("n'importe quoi", "Netflix") == "NETFLIX"
def test_dedup_hash_depends_on_occurrence() -> None:
args = (ACCOUNT_UUID, date(2026, 6, 5), Decimal("-2.50"), "CAFE")
assert compute_dedup_hash(*args, 0) != compute_dedup_hash(*args, 1)
assert compute_dedup_hash(*args, 0) == compute_dedup_hash(*args, 0)
def test_dedup_hash_ignores_accents_and_case() -> None:
args = (ACCOUNT_UUID, date(2026, 6, 5), Decimal("-2.50"))
assert compute_dedup_hash(*args, "Café Crème", 0) == compute_dedup_hash(
*args, "CAFE CREME", 0
)
def test_decode_bytes_bom_and_cp1252() -> None:
sample = (
"Date;Libellé;Débit euros;Crédit euros;\n"
"02/06/2026;PRLV SEPA EDF ÉNERGIE;89,00;;\n"
"03/06/2026;VIREMENT DE Mme DÉSIRÉE;;1 500,00;\n"
)
# BOM wins over the configured encoding.
assert decode_bytes(b"\xef\xbb\xbfdate;label", "cp1252") == "date;label"
# "auto" never fails on a legacy French export…
assert decode_bytes(sample.encode("cp1252"), "auto") == sample
# …nor on UTF-8, and honours an explicit encoding.
assert decode_bytes(sample.encode(), "auto") == sample
assert decode_bytes(sample.encode("cp1252"), "cp1252") == sample
# ---------------------------------------------------------------------------
# Bank presets
# ---------------------------------------------------------------------------
def test_boursobank_csv() -> None:
result = parse_fixture("boursobank", "boursobank.csv")
assert len(result.rows) == 7
assert result.rows_total == 8
assert result.rows_error == 1
assert result.errors[0]["message"].startswith("Date invalide")
first = result.rows[0]
assert first.booked_date == date(2026, 6, 1)
assert first.value_date == date(2026, 6, 1)
assert first.amount == Decimal("-42.90")
assert first.label_raw == "CARTE 31/05 CARREFOUR CITY PARIS"
# Two identical purchases the same day survive as two rows.
twins = [r for r in result.rows if r.amount == Decimal("-2.50")]
assert len(twins) == 2
def test_credit_agricole_split_columns_multiline_and_footer() -> None:
result = parse_fixture("credit-agricole", "credit_agricole.csv")
assert len(result.rows) == 4 # footer row stopped the parsing
assert result.rows[0].amount == Decimal("-58.20")
# multi-line quoted label collapsed into a single line
assert result.rows[0].label_raw == "PAIEMENT CB 0106 INTERMARCHE FACT 010626"
assert result.rows[1].amount == Decimal("1500.00") # thousands space
assert result.rows[1].value_date == date(2026, 6, 4)
def test_societe_generale_preamble_and_padding() -> None:
result = parse_fixture("societe-generale", "societe_generale.csv")
assert len(result.rows) == 4
assert result.rows[0].label_raw == "CARTE X7527 15/06 METRO"
assert result.rows[0].currency == "EUR"
assert result.rows[3].amount == Decimal("-1234.56")
def test_banque_postale_skip_until_header() -> None:
result = parse_fixture("banque-postale", "banque_postale.csv")
assert [r.amount for r in result.rows] == [
Decimal("-32.15"),
Decimal("185.00"),
]
def test_paypal_filters_and_net_amount() -> None:
result = parse_fixture("paypal", "paypal_fr.csv")
assert len(result.rows) == 3
# Authorization + pending + USD + currency conversion are filtered out.
assert result.rows_filtered == 4
steam = result.rows[0]
assert steam.amount == Decimal("-12.99")
assert steam.external_id == "5AB12345CD678901E"
assert steam.counterparty == "Steam Games"
vinted = result.rows[1]
assert vinted.amount == Decimal("17.50") # Net (gross 18,00 - fee 0,50)
def test_ofx_sgml_fitid_collision_gets_occurrence_suffix() -> None:
result = parse_fixture("ofx", "sample.ofx")
assert len(result.rows) == 4
ids = [row.external_id for row in result.rows]
assert ids[0] == "948 040626 -1275"
assert ids[1] == "948 040626 -1275#1"
assert len(set(ids)) == 4
assert result.rows[2].label_raw == "PRLV SEPA ORANGE — FACTURE JUIN"
assert result.rows[3].amount == Decimal("1500.00")
def test_bnp_positional_mapping_without_column_header() -> None:
result = parse_fixture("bnp-paribas", "bnp_paribas.csv")
# the balance line is skipped, the 3 operations are mapped by position
assert len(result.rows) == 3
assert result.rows[0].label_raw == "AMORTISSEMENT PRET 1234"
assert result.rows[0].amount == Decimal("-70.93")
assert result.rows[2].amount == Decimal("2450.00") # thousands space
def test_caisse_epargne_signed_debit_and_plus_prefixed_credit() -> None:
result = parse_fixture("caisse-epargne", "caisse_epargne.csv")
assert [r.amount for r in result.rows] == [
Decimal("-45.50"),
Decimal("3500.00"),
]
assert result.rows[0].counterparty == "SUPERMARCHE" # "Libelle simplifie"
assert result.rows[1].value_date == date(2026, 11, 9)
def test_fortuneo_split_columns_and_short_decimals() -> None:
result = parse_fixture("fortuneo", "fortuneo.csv")
assert result.rows[0].amount == Decimal("-6.40")
assert result.rows[1].amount == Decimal("2500.00")
def test_revolut_net_of_fee_and_completed_only() -> None:
result = parse_fixture("revolut", "revolut.csv")
assert len(result.rows) == 2
assert result.rows_filtered == 1 # the PENDING row is ignored
assert result.rows[0].amount == Decimal("-15.50") # amount 15,00 + fee 0,50
assert result.rows[0].currency == "EUR"
def test_n26_label_join_and_counterparty() -> None:
result = parse_fixture("n26", "n26.csv")
assert result.rows[0].label_raw == "Netflix — Abonnement fevrier"
assert result.rows[0].counterparty == "Netflix"
assert result.rows[0].amount == Decimal("-13.49")
def test_generic_preset_autodetects_delimiter_and_columns() -> None:
result = parse_fixture("generic", "generic.csv")
assert [r.amount for r in result.rows] == [
Decimal("-25.40"),
Decimal("42.10"),
]
def test_every_builtin_preset_is_declared_once() -> None:
slugs = [item["slug"] for item in BUILTIN_PROFILES]
assert len(slugs) == len(set(slugs))
expected = {
"generic",
"boursobank",
"credit-agricole",
"bnp-paribas",
"societe-generale",
"banque-postale",
"caisse-epargne",
"fortuneo",
"revolut",
"n26",
"ofx",
"paypal",
}
assert expected.issubset(set(slugs))
@@ -0,0 +1,481 @@
"""Import pipeline: dedup/idempotence, occurrence, rules, transfers, rollback.
Reference: datamodel-finance.md §4 (pipeline), §5 (rules), §6 (transfers),
§10.3 (same-day twins) and §10.6 (manual categorisation is sacred).
"""
import uuid
from decimal import Decimal
from pathlib import Path
import pytest
from sqlalchemy import func, select
from sqlalchemy.orm import Session
from app.core.errors import DomainValidationError
from app.modules.auth.models import User
from app.modules.finance import categorize, pipeline, service
from app.modules.finance.enums import AccountKind, CategorySource, ImportStatus
from app.modules.finance.models import (
FinAccount,
FinCategory,
FinRule,
FinSourceProfile,
FinTransaction,
)
FIXTURES = Path(__file__).resolve().parents[1] / "fixtures" / "finance"
def make_account(
db: Session, user: User, name: str, kind: AccountKind = AccountKind.CHECKING
) -> FinAccount:
account = FinAccount(
id=uuid.uuid4(),
user_id=user.id,
name=name,
kind=kind,
currency="EUR",
initial_balance=Decimal("0.00"),
)
db.add(account)
db.commit()
return account
def builtin(db: Session, name: str) -> FinSourceProfile:
profile = service.find_profile_by_name(db, name)
assert profile is not None
return profile
def import_fixture(
db: Session, user: User, account: FinAccount, profile_name: str, filename: str
):
service.ensure_seed(db, user.id)
return pipeline.run_import(
db,
user.id,
account,
builtin(db, profile_name),
filename,
(FIXTURES / filename).read_bytes(),
)
def transactions(db: Session, user: User) -> list[FinTransaction]:
return list(
db.scalars(
select(FinTransaction)
.where(FinTransaction.user_id == user.id)
.order_by(FinTransaction.booked_date, FinTransaction.amount)
).all()
)
# ---------------------------------------------------------------------------
# Import + idempotence
# ---------------------------------------------------------------------------
def test_import_boursobank_counts(db: Session, user: User) -> None:
account = make_account(db, user, "BoursoBank")
run = import_fixture(
db, user, account, "BoursoBank / Boursorama (CSV)", "boursobank.csv"
)
assert run.status == ImportStatus.COMPLETED
assert run.stats["rows_total"] == 8
assert run.stats["rows_imported"] == 7
assert run.stats["rows_error"] == 1
assert run.stats["rows_skipped_duplicate"] == 0
assert run.stats["date_min"] == "2026-06-01"
assert run.stats["date_max"] == "2026-06-12"
assert len(run.stats["errors"]) == 1
rows = transactions(db, user)
assert len(rows) == 7
assert all(tx.import_run_id == run.import_run_id for tx in rows)
assert all(tx.currency == "EUR" for tx in rows)
carrefour = next(tx for tx in rows if "CARREFOUR" in tx.label_raw)
assert carrefour.amount == Decimal("-42.90")
assert carrefour.label_clean == "CARREFOUR CITY PARIS" # technical prefix dropped
def test_reimporting_the_same_file_inserts_nothing(db: Session, user: User) -> None:
account = make_account(db, user, "BoursoBank")
first = import_fixture(
db, user, account, "BoursoBank / Boursorama (CSV)", "boursobank.csv"
)
second = import_fixture(
db, user, account, "BoursoBank / Boursorama (CSV)", "boursobank.csv"
)
assert second.stats["rows_imported"] == 0
assert second.stats["rows_skipped_duplicate"] == 7
assert second.stats["duplicate_file_of"] == first.import_run_id
assert len(transactions(db, user)) == 7
def test_same_day_identical_purchases_are_kept(db: Session, user: User) -> None:
account = make_account(db, user, "BoursoBank")
import_fixture(db, user, account, "BoursoBank / Boursorama (CSV)", "boursobank.csv")
twins = [tx for tx in transactions(db, user) if tx.amount == Decimal("-2.50")]
assert len(twins) == 2
assert twins[0].dedup_hash != twins[1].dedup_hash
def test_ofx_duplicate_fitid_is_not_a_conflict(db: Session, user: User) -> None:
account = make_account(db, user, "LCL")
run = import_fixture(db, user, account, "OFX (toutes banques)", "sample.ofx")
assert run.stats["rows_imported"] == 4
again = import_fixture(db, user, account, "OFX (toutes banques)", "sample.ofx")
assert again.stats["rows_imported"] == 0
assert again.stats["rows_skipped_duplicate"] == 4
def test_paypal_import_uses_transaction_id(db: Session, user: User) -> None:
account = make_account(db, user, "PayPal", AccountKind.PAYPAL)
run = import_fixture(
db, user, account, "PayPal — rapport d'activité (CSV)", "paypal_fr.csv"
)
assert run.stats["rows_imported"] == 3
assert run.stats["rows_skipped_filtered"] == 4
external = {tx.external_id for tx in transactions(db, user)}
assert "5AB12345CD678901E" in external
def test_stats_counters_add_up_to_rows_total(db: Session, user: User) -> None:
"""§2.4: rows_total == imported + skipped_duplicate + skipped_filtered
+ error. Rows dropped on purpose must stay visible in the total, otherwise
the import summary silently hides them."""
account = make_account(db, user, "PayPal", AccountKind.PAYPAL)
for _ in range(2): # second pass turns the 3 kept rows into duplicates
run = import_fixture(
db, user, account, "PayPal — rapport d'activité (CSV)", "paypal_fr.csv"
)
stats = run.stats
assert stats["rows_skipped_filtered"] == 4
assert stats["rows_total"] == (
stats["rows_imported"]
+ stats["rows_skipped_duplicate"]
+ stats["rows_skipped_filtered"]
+ stats["rows_error"]
)
assert run.stats["rows_skipped_duplicate"] == 3
def test_a_fully_filtered_file_is_an_empty_import_not_a_failure(
db: Session, user: User
) -> None:
"""A PayPal export made only of authorisations/pending lines is legitimate:
nothing to import, but the run must not be marked `failed` (§4.2 fails only
when every row ERRORED)."""
service.ensure_seed(db, user.id)
account = make_account(db, user, "PayPal", AccountKind.PAYPAL)
header = (
(FIXTURES / "paypal_fr.csv").read_text(encoding="utf-8-sig").splitlines()[0]
)
only_filtered = (
header + "\n"
"01/06/2026,10:00:00,CEST,Fnac,Autorisation,Autorisation,EUR,"
'"-120,00","0,00","-120,00",moi@example.com,pay@fnac.com,'
'9ZZ11111AA222222B,"Casque",,"0,00",Mémo\n'
)
run = pipeline.run_import(
db,
user.id,
account,
builtin(db, "PayPal — rapport d'activité (CSV)"),
"paypal_vide.csv",
only_filtered.encode("utf-8-sig"),
)
assert run.status == ImportStatus.COMPLETED
assert run.stats["rows_total"] == 1
assert run.stats["rows_skipped_filtered"] == 1
assert run.stats["rows_imported"] == 0
assert run.stats["rows_error"] == 0
def test_credit_agricole_import(db: Session, user: User) -> None:
account = make_account(db, user, "Crédit Agricole")
run = import_fixture(
db, user, account, "Crédit Agricole (CSV)", "credit_agricole.csv"
)
assert run.stats["rows_imported"] == 4
amounts = sorted(tx.amount for tx in transactions(db, user))
assert amounts[0] == Decimal("-89.00")
assert amounts[-1] == Decimal("1500.00")
def test_rollback_removes_the_imported_rows(db: Session, user: User) -> None:
account = make_account(db, user, "BoursoBank")
run = import_fixture(
db, user, account, "BoursoBank / Boursorama (CSV)", "boursobank.csv"
)
service.rollback_import(db, user.id, run.id)
assert transactions(db, user) == []
assert db.scalar(select(func.count()).select_from(FinTransaction)) == 0
def test_wrong_profile_marks_the_run_as_failed(db: Session, user: User) -> None:
account = make_account(db, user, "BoursoBank")
service.ensure_seed(db, user.id)
run = pipeline.run_import(
db,
user.id,
account,
builtin(db, "PayPal — rapport d'activité (CSV)"),
"boursobank.csv",
(FIXTURES / "boursobank.csv").read_bytes(),
)
assert run.status == ImportStatus.FAILED
assert "PayPal" in run.error_message
assert transactions(db, user) == []
def test_empty_file_is_rejected(db: Session, user: User) -> None:
account = make_account(db, user, "BoursoBank")
service.ensure_seed(db, user.id)
with pytest.raises(DomainValidationError):
pipeline.run_import(
db, user.id, account, builtin(db, "CSV générique"), "vide.csv", b""
)
# ---------------------------------------------------------------------------
# Rules engine
# ---------------------------------------------------------------------------
def make_rule(
db: Session,
user: User,
name: str,
matchers: dict,
actions: dict,
priority: int = 100,
stop: bool = True,
) -> FinRule:
rule = FinRule(
id=uuid.uuid4(),
user_id=user.id,
name=name,
priority=priority,
enabled=True,
stop=stop,
matchers=matchers,
actions=actions,
)
db.add(rule)
db.commit()
return rule
def category_named(db: Session, user: User, name: str) -> FinCategory:
service.ensure_seed(db, user.id)
category = db.scalars(
select(FinCategory).where(
FinCategory.user_id == user.id, FinCategory.name == name
)
).first()
assert category is not None, name
return category
def test_rules_are_applied_before_insert(db: Session, user: User) -> None:
account = make_account(db, user, "BoursoBank")
courses = category_named(db, user, "Courses")
rule = make_rule(
db,
user,
"Courses Carrefour",
{"label_contains": ["CARREFOUR"], "direction": "debit"},
{"set_category_id": str(courses.id), "set_counterparty": "Carrefour"},
)
run = import_fixture(
db, user, account, "BoursoBank / Boursorama (CSV)", "boursobank.csv"
)
assert run.stats["rules_applied"] == 1
tx = next(tx for tx in transactions(db, user) if "CARREFOUR" in tx.label_raw)
assert tx.category_id == courses.id
assert tx.category_source == CategorySource.RULE
assert tx.applied_rule_id == rule.id
assert tx.counterparty == "Carrefour"
db.refresh(rule)
assert rule.hit_count == 1
assert rule.last_applied_at is not None
def test_rule_priority_and_stop_flag(db: Session, user: User) -> None:
account = make_account(db, user, "BoursoBank")
loisirs = category_named(db, user, "Abonnements & streaming")
autres = category_named(db, user, "Autres dépenses")
make_rule(
db,
user,
"Netflix",
{"label_contains": ["NETFLIX"]},
{"set_category_id": str(loisirs.id), "set_label_clean": "Netflix"},
priority=10,
)
make_rule(
db,
user,
"Tout le reste",
{"direction": "debit"},
{"set_category_id": str(autres.id)},
priority=90,
)
import_fixture(db, user, account, "BoursoBank / Boursorama (CSV)", "boursobank.csv")
netflix = next(tx for tx in transactions(db, user) if "NETFLIX" in tx.label_raw)
assert netflix.category_id == loisirs.id # priority 10 stopped the chain
assert netflix.label_clean == "Netflix"
other = next(tx for tx in transactions(db, user) if "LOYER" in tx.label_raw)
assert other.category_id == autres.id
def test_amount_bounds_and_regex_matchers(db: Session, user: User) -> None:
account = make_account(db, user, "BoursoBank")
loyer = category_named(db, user, "Loyer / Crédit")
make_rule(
db,
user,
"Loyer",
{"label_regex": r"^vir\s+sepa\s+loyer", "amount_max": -100},
{"set_category_id": str(loyer.id)},
)
import_fixture(db, user, account, "BoursoBank / Boursorama (CSV)", "boursobank.csv")
rows = transactions(db, user)
tagged = [tx for tx in rows if tx.category_id == loyer.id]
assert len(tagged) == 1
assert tagged[0].amount == Decimal("-750.00")
def test_manual_categorisation_is_never_overwritten(db: Session, user: User) -> None:
account = make_account(db, user, "BoursoBank")
courses = category_named(db, user, "Courses")
restaurants = category_named(db, user, "Restaurants & bars")
import_fixture(db, user, account, "BoursoBank / Boursorama (CSV)", "boursobank.csv")
tx = next(tx for tx in transactions(db, user) if "CARREFOUR" in tx.label_raw)
tx.category_id = restaurants.id
tx.category_source = CategorySource.USER
db.commit()
make_rule(
db,
user,
"Courses Carrefour",
{"label_contains": ["CARREFOUR"]},
{"set_category_id": str(courses.id)},
)
result = categorize.apply_rules(db, user.id, scope="all_non_manual")
db.refresh(tx)
assert tx.category_id == restaurants.id
assert result.updated == 0 or tx.category_source == CategorySource.USER
with pytest.raises(DomainValidationError):
categorize.apply_rules(db, user.id, scope="all")
categorize.apply_rules(db, user.id, scope="all", force=True)
db.refresh(tx)
assert tx.category_id == courses.id
def test_apply_rules_dry_run_writes_nothing(db: Session, user: User) -> None:
account = make_account(db, user, "BoursoBank")
courses = category_named(db, user, "Courses")
import_fixture(db, user, account, "BoursoBank / Boursorama (CSV)", "boursobank.csv")
make_rule(
db,
user,
"Courses Carrefour",
{"label_contains": ["CARREFOUR"]},
{"set_category_id": str(courses.id)},
)
result = categorize.apply_rules(db, user.id, scope="uncategorized", dry_run=True)
assert result.matched == 1
assert result.dry_run is True
assert result.by_rule[0]["matched"] == 1
tx = next(tx for tx in transactions(db, user) if "CARREFOUR" in tx.label_raw)
assert tx.category_id is None
# ---------------------------------------------------------------------------
# Internal transfers
# ---------------------------------------------------------------------------
def test_transfer_pairing_between_two_accounts(db: Session, user: User) -> None:
bank = make_account(db, user, "BoursoBank")
paypal = make_account(db, user, "PayPal", AccountKind.PAYPAL)
import_fixture(db, user, bank, "BoursoBank / Boursorama (CSV)", "boursobank.csv")
run = import_fixture(
db, user, paypal, "PayPal — rapport d'activité (CSV)", "paypal_fr.csv"
)
assert run.stats["transfers_detected"] == 1
legs = [tx for tx in transactions(db, user) if tx.transfer_group_id is not None]
assert len(legs) == 2
assert legs[0].transfer_group_id == legs[1].transfer_group_id
assert {leg.account_id for leg in legs} == {bank.id, paypal.id}
assert {leg.amount for leg in legs} == {Decimal("-50.00"), Decimal("50.00")}
transfer_cat = service.transfer_category(db, user.id)
assert all(leg.category_id == transfer_cat.id for leg in legs)
def test_manual_link_and_unlink(db: Session, user: User) -> None:
bank = make_account(db, user, "Compte A")
other = make_account(db, user, "Compte B")
service.ensure_seed(db, user.id)
a = pipeline.build_transaction(
user.id,
bank,
_row("2026-07-01", "-120.00", "VIREMENT VERS LIVRET"),
"hash-a",
None,
)
b = pipeline.build_transaction(
user.id,
other,
_row("2026-07-02", "120.00", "VIREMENT RECU"),
"hash-b",
None,
)
db.add_all([a, b])
db.commit()
group_id = categorize.link_transfer(db, user.id, a.id, b.id)
db.refresh(a)
db.refresh(b)
assert a.transfer_group_id == group_id == b.transfer_group_id
categorize.unlink_transfer(db, user.id, group_id)
db.refresh(a)
assert a.transfer_group_id is None
assert a.category_id is None
def test_link_rejects_non_opposite_amounts(db: Session, user: User) -> None:
bank = make_account(db, user, "Compte A")
other = make_account(db, user, "Compte B")
a = pipeline.build_transaction(
user.id, bank, _row("2026-07-01", "-120.00", "A"), "h1", None
)
b = pipeline.build_transaction(
user.id, other, _row("2026-07-01", "119.00", "B"), "h2", None
)
db.add_all([a, b])
db.commit()
with pytest.raises(DomainValidationError):
categorize.link_transfer(db, user.id, a.id, b.id)
def _row(day: str, amount: str, label: str):
from datetime import date
from app.modules.finance.parsers import NormalizedRow
return NormalizedRow(
booked_date=date.fromisoformat(day),
value_date=None,
amount=Decimal(amount),
@@ -0,0 +1,364 @@
"""Unit tests for the pure health calculations (datamodel-health-vape.md §5
and the planning addendum). Exact formula values, no database."""
import datetime as dt
from dataclasses import dataclass
import pytest
from app.modules.health import calculations as calc
D = dt.date
# --- BMR / TDEE / budget ------------------------------------------------------
def test_bmr_mifflin_exact_values_per_sex() -> None:
# 10*80 + 6.25*180 - 5*30 = 1775
assert calc.bmr_mifflin(80, 180, 30, "male") == pytest.approx(1780.0)
assert calc.bmr_mifflin(80, 180, 30, "female") == pytest.approx(1614.0)
assert calc.bmr_mifflin(80, 180, 30, "other") == pytest.approx(1697.0)
def test_age_on_handles_birthday_not_reached() -> None:
assert calc.age_on(D(2026, 8, 13), D(1990, 8, 13)) == 36
assert calc.age_on(D(2026, 8, 12), D(1990, 8, 13)) == 35
assert calc.age_on(D(2026, 8, 14), D(1990, 8, 13)) == 36
def test_tdee_three_tier_preference() -> None:
bmr = 1780.0
measured = calc.tdee_effective(bmr, "sedentary", total_kcal=2600, active_kcal=400)
assert (measured.kcal, measured.method) == (2600.0, "measured_total")
# 1000 kcal is below the 0.8 x BMR plausibility guard -> fall through
partial = calc.tdee_effective(bmr, "sedentary", total_kcal=1000, active_kcal=400)
assert partial.method == "bmr_plus_active"
assert partial.kcal == pytest.approx(2180.0)
estimated = calc.tdee_effective(bmr, "moderate")
assert estimated.method == "estimated"
assert estimated.kcal == pytest.approx(1780.0 * 1.55)
def test_activity_factors_table() -> None:
assert calc.ACTIVITY_FACTORS == {
"sedentary": 1.2,
"light": 1.375,
"moderate": 1.55,
"active": 1.725,
"very_active": 1.9,
}
def test_moving_average_trailing_window_skips_none() -> None:
values = [None, 100.0, 200.0, None, 300.0]
assert calc.moving_average(values, 3) == [
None,
100.0,
150.0,
150.0,
250.0,
]
def test_daily_budget_deficit_and_floor() -> None:
rate = calc.GoalRate(0.5)
budget = calc.daily_budget(2400.0, rate, "male")
assert budget.deficit_target == pytest.approx(550.0) # 0.5 * 7700 / 7
assert budget.kcal == pytest.approx(1850.0)
assert budget.floor_applied is False
floored = calc.daily_budget(1800.0, calc.GoalRate(1.0), "male")
assert floored.kcal == pytest.approx(1500.0)
assert floored.floor_applied is True
female = calc.daily_budget(1500.0, calc.GoalRate(1.0), "female")
assert female.kcal == pytest.approx(1200.0)
custom = calc.daily_budget(1500.0, calc.GoalRate(1.0), "male", 1400)
assert custom.kcal == pytest.approx(1400.0)
def test_daily_budget_surplus_when_rate_is_negative() -> None:
budget = calc.daily_budget(2400.0, calc.GoalRate(-0.25), "male")
assert budget.kcal == pytest.approx(2675.0)
def test_resolve_goal_rate_target_date_is_clamped() -> None:
rate = calc.resolve_goal_rate(
"target_date", D(2026, 8, 13), 92.0, 78.0, None, D(2026, 9, 13)
)
assert rate.weekly_rate_kg == pytest.approx(1.0)
assert rate.rate_clamped is True
steady = calc.resolve_goal_rate(
"target_date", D(2026, 8, 13), 92.0, 88.0, None, D(2026, 10, 22)
)
assert steady.rate_clamped is False
assert steady.weekly_rate_kg == pytest.approx(4.0 / 10.0)
assert (
calc.resolve_goal_rate(
"maintain", D(2026, 8, 13), 92.0, 78.0, 0.5, None
).weekly_rate_kg
== 0.0
)
assert calc.resolve_goal_rate(
"weekly_rate", D(2026, 8, 13), 92.0, 78.0, 0.75, None
).weekly_rate_kg == pytest.approx(0.75)
# --- Weight trend / slope / projection ----------------------------------------
def test_weight_trend_ema_alpha_and_gap_correction() -> None:
trend = calc.weight_trend([(D(2026, 8, 1), 92.0), (D(2026, 8, 2), 91.0)])
assert trend[0] == (D(2026, 8, 1), 92.0)
assert trend[1][1] == pytest.approx(91.9) # 92 + 0.1 * (91 - 92)
gapped = calc.weight_trend([(D(2026, 8, 1), 92.0), (D(2026, 8, 6), 90.0)])
# alpha_eff = 1 - 0.9**5 = 0.40951 -> 92 - 0.40951 * 2 = 91.18098
assert gapped[1][1] == pytest.approx(91.18, abs=1e-2)
def test_weight_trend_empty_and_single_point() -> None:
assert calc.weight_trend([]) == []
assert calc.weight_trend([(D(2026, 8, 1), 88.5)]) == [(D(2026, 8, 1), 88.5)]
def test_regression_slope_exact_and_guards() -> None:
points = [(D(2026, 8, 1), 90.0), (D(2026, 8, 2), 89.0), (D(2026, 8, 3), 88.0)]
assert calc.regression_slope(points) == pytest.approx(-1.0)
assert calc.regression_slope(points[:2]) is None
flat = [(D(2026, 8, 1), 90.0)] * 3
assert calc.regression_slope(flat) is None # denom == 0
def test_trend_at_day_carries_forward() -> None:
trend = [(D(2026, 8, 1), 92.0), (D(2026, 8, 5), 91.0)]
assert calc.trend_at_day(trend, D(2026, 7, 31)) is None
assert calc.trend_at_day(trend, D(2026, 8, 3)) == 92.0
assert calc.trend_at_day(trend, D(2026, 8, 9)) == 91.0
def test_projection_ok_reached_and_not_converging() -> None:
today = D(2026, 8, 13)
ok = calc.project_target_date(90.0, 85.0, -0.05, today)
assert ok.status == "ok"
assert ok.date == today + dt.timedelta(days=100)
assert calc.project_target_date(85.05, 85.0, -0.05, today).status == "reached"
# slope below MIN_SLOPE_KG_PER_DAY
assert (
calc.project_target_date(90.0, 85.0, -0.001, today).status == "not_converging"
)
# gaining while a loss is needed
assert calc.project_target_date(90.0, 85.0, 0.05, today).status == "not_converging"
assert calc.project_target_date(90.0, 85.0, None, today).status == "not_converging"
# more than 10 years away
assert (
calc.project_target_date(90.0, 85.0, -0.001_2, today).status == "not_converging"
)
def test_projection_upwards_when_gaining_is_the_goal() -> None:
today = D(2026, 8, 13)
result = calc.project_target_date(70.0, 75.0, 0.05, today)
assert result.status == "ok"
assert result.date == today + dt.timedelta(days=100)
# --- Energy balance & calibration ---------------------------------------------
def test_energy_balance_is_none_on_untracked_days() -> None:
assert calc.energy_balance(1800.0, 2300.0) == pytest.approx(-500.0)
assert calc.energy_balance(None, 2300.0) is None
assert calc.energy_balance(1800.0, None) is None
def test_tdee_calibration_requires_21_tracked_days() -> None:
short = calc.tdee_calibration([(2000.0, 2500.0)] * 20, 92.0, 91.0)
assert short.status == "insufficient_data"
assert short.tracked_days == 20
def test_tdee_calibration_exact_values() -> None:
result = calc.tdee_calibration([(2000.0, 2500.0)] * 21, 92.0, 91.0)
assert result.status == "ok"
assert result.tracked_days == 21
assert result.expected_change_kg == pytest.approx(21 * -500 / 7700)
assert result.actual_change_kg == pytest.approx(-1.0)
assert result.gap_kg == pytest.approx(-1.0 - (21 * -500 / 7700))
assert result.tdee_adaptive_kcal == pytest.approx(2000 + 7700 / 21)
assert result.tdee_correction_kcal == pytest.approx(2000 + 7700 / 21 - 2500)
def test_constants_match_the_spec() -> None:
assert calc.KCAL_PER_KG_FAT == 7700
assert calc.EMA_ALPHA == 0.1
assert calc.MIN_SLOPE_KG_PER_DAY == 0.005
assert calc.CALORIE_FLOOR_MALE == 1500
assert calc.CALORIE_FLOOR_FEMALE == 1200
assert calc.WORKOUT_OVERLAP_THRESHOLD == 0.8
assert calc.ADAPTIVE_TDEE_MIN_DAYS == 21
# --- Activity merge & workout overlap -----------------------------------------
@dataclass
class Row:
date: dt.date
source: str
steps: int | None = None
active_kcal: float | None = None
total_kcal: float | None = None
distance_m: int | None = None
active_minutes: int | None = None
floors: int | None = None
def test_merge_activity_is_field_by_field_by_priority() -> None:
day = D(2026, 8, 12)
merged = calc.merge_activity_day(
[
Row(day, "csv_import", steps=5000, distance_m=3000),
Row(day, "health_connect", steps=9421, active_kcal=520.0),
Row(day, "manual", steps=10000),
]
)
assert merged.steps == 10000 # manual wins
assert merged.active_kcal == pytest.approx(520.0) # only health_connect has it
assert merged.distance_m == 3000 # only csv_import has it
assert merged.field_sources == {
"steps": "manual",
"active_kcal": "health_connect",
"distance_m": "csv_import",
}
def test_merge_puts_unknown_sources_last() -> None:
day = D(2026, 8, 12)
merged = calc.merge_activity_day(
[Row(day, "some_bridge", steps=1), Row(day, "api", steps=2)]
)
assert merged.steps == 2
assert calc.source_priority("manual") < calc.source_priority("health_connect")
assert calc.source_priority("api") < calc.source_priority("unknown")
def test_overlap_ratio_uses_the_shorter_session() -> None:
base = dt.datetime(2026, 8, 12, 18, 0, tzinfo=dt.UTC)
a_end = base + dt.timedelta(minutes=60)
b_start = base + dt.timedelta(minutes=5)
b_end = base + dt.timedelta(minutes=55)
assert calc.overlap_ratio(base, a_end, b_start, b_end) == pytest.approx(1.0)
far = base + dt.timedelta(hours=5)
assert calc.overlap_ratio(base, a_end, far, far + dt.timedelta(hours=1)) == 0.0
# --- Body composition ---------------------------------------------------------
def test_bmi_and_navy_body_fat() -> None:
assert calc.bmi(80.0, 180.0) == pytest.approx(24.69, abs=1e-2)
assert calc.bmi(80.0, 0) is None
male = calc.navy_body_fat_pct("male", 180.0, 90.0, 38.0)
assert male == pytest.approx(19.8, abs=0.2)
assert calc.navy_body_fat_pct("female", 165.0, 75.0, 32.0) is None # hips missing
assert calc.navy_body_fat_pct("female", 165.0, 75.0, 32.0, 98.0) is not None
assert calc.navy_body_fat_pct("male", 180.0, None, 38.0) is None
# --- Planning: schedules, adherence, streaks ----------------------------------
def test_schedule_kinds_are_the_three_habits() -> None:
assert calc.SCHEDULE_KINDS == ("weigh_in", "workout", "food_log")
def test_is_planned_uses_monday_zero_convention() -> None:
monday = D(2026, 7, 6)
assert monday.weekday() == 0
assert calc.is_planned([0, 2, 4], monday) is True
assert calc.is_planned([1, 3], monday) is False
assert calc.is_planned([0], monday, enabled=False) is False
assert calc.is_planned([], monday) is False
assert calc.is_planned(None, monday) is False
def test_build_adherence_marks_planned_done_and_missed() -> None:
start, end = D(2026, 7, 6), D(2026, 7, 12) # a full Monday-Sunday week
days = calc.build_adherence(start, end, [0, 2], True, {D(2026, 7, 6)})
assert len(days) == 7
statuses = {day.day: day.status for day in days}
assert statuses[D(2026, 7, 6)] == "done" # Monday planned + done
assert statuses[D(2026, 7, 8)] == "missed" # Wednesday planned, not done
assert statuses[D(2026, 7, 7)] == "rest"
assert calc.adherence_pct(days) == pytest.approx(50.0)
def test_adherence_pct_is_none_without_planned_days() -> None:
days = calc.build_adherence(D(2026, 7, 6), D(2026, 7, 12), [], True, set())
assert calc.adherence_pct(days) is None
def test_streaks_count_planned_days_only_across_weeks() -> None:
start, end = D(2026, 7, 6), D(2026, 8, 3) # 5 Mondays
mondays = [
D(2026, 7, 6),
D(2026, 7, 13),
D(2026, 7, 20),
D(2026, 7, 27),
D(2026, 8, 3),
]
assert all(day.weekday() == 0 for day in mondays)
done = {mondays[0], mondays[1], mondays[3], mondays[4]} # 07-20 missed
days = calc.build_adherence(start, end, [0], True, done)
streak = calc.compute_streaks(days, today=end)
assert streak.best == 2
assert streak.current == 2
def test_streak_grace_period_on_today() -> None:
mondays = [
D(2026, 7, 6),
D(2026, 7, 13),
D(2026, 7, 20),
D(2026, 7, 27),
D(2026, 8, 3),
]
days = calc.build_adherence(
D(2026, 7, 6), D(2026, 8, 3), [0], True, set(mondays[:-1])
)
# Today is a planned day that is not done yet: the streak is not broken.
streak = calc.compute_streaks(days, today=D(2026, 8, 3))
assert streak.current == 4
assert streak.best == 4
# One day later the missed Monday does break it.
assert calc.compute_streaks(days, today=D(2026, 8, 4)).current == 0
def test_streaks_ignore_unplanned_done_days() -> None:
days = calc.build_adherence(
D(2026, 7, 6),
D(2026, 7, 12),
[0],
True,
{D(2026, 7, 7), D(2026, 7, 8)}, # done on two unplanned days
)
streak = calc.compute_streaks(days, today=D(2026, 7, 12))
assert streak.best == 0
assert streak.current == 0
def test_date_range_is_inclusive_and_safe() -> None:
assert calc.date_range(D(2026, 8, 1), D(2026, 8, 3)) == [
D(2026, 8, 1),
D(2026, 8, 2),
D(2026, 8, 3),
]
assert calc.date_range(D(2026, 8, 3), D(2026, 8, 1)) == []
@@ -0,0 +1,228 @@
"""Food search: local `food_items` cache first, Open Food Facts proxy next.
Every OFF call is served by an httpx.MockTransport the suite never reaches
the network.
"""
from collections.abc import Iterator
import httpx
import pytest
from fastapi.testclient import TestClient
from sqlalchemy import select
from sqlalchemy.orm import Session
from app.modules.health import foods
from app.modules.health.models import FoodItem
URL = "/api/health/foods/search"
OFF_HIT = {
"code": "3175680011480",
"product_name": "Petit Beurre",
"product_name_fr": "Véritable Petit Beurre",
"brands": "LU, Mondelez",
"serving_quantity": 8.3,
"nutriments": {
"energy-kcal_100g": 442,
"proteins_100g": 7.2,
"carbohydrates_100g": 74.5,
"sugars_100g": 22.1,
"fat_100g": 12.3,
"saturated-fat_100g": 6.4,
"fiber_100g": 2.6,
"salt_100g": 0.9,
},
}
@pytest.fixture(autouse=True)
def _reset_transport() -> Iterator[None]:
yield
foods.set_http_transport(None)
def use_transport(handler) -> list[httpx.Request]:
"""Install a MockTransport and return the list of captured requests."""
seen: list[httpx.Request] = []
def wrapped(request: httpx.Request) -> httpx.Response:
seen.append(request)
return handler(request)
foods.set_http_transport(httpx.MockTransport(wrapped))
return seen
def ok_search(hits: list[dict]):
return lambda request: httpx.Response(200, json={"hits": hits, "count": len(hits)})
# --- Normalisation -------------------------------------------------------------
def test_normalize_prefers_the_french_name_and_first_brand() -> None:
data = foods.normalize_off_product(OFF_HIT)
assert data["name"] == "Véritable Petit Beurre"
assert data["brand"] == "LU"
assert float(data["energy_kcal_100g"]) == 442.0
assert float(data["salt_g_100g"]) == 0.9
assert float(data["serving_size_g"]) == 8.3
def test_normalize_recomputes_kcal_from_kilojoules() -> None:
data = foods.normalize_off_product(
{"code": "1", "product_name": "Truc", "nutriments": {"energy_100g": 1000}}
)
assert float(data["energy_kcal_100g"]) == pytest.approx(239.0, abs=0.1)
def test_normalize_skips_nameless_products() -> None:
assert foods.normalize_off_product({"code": "1", "nutriments": {}}) is None
# --- Search --------------------------------------------------------------------
def test_search_queries_off_and_caches_the_hit(
client: TestClient, db: Session, auth_headers: dict[str, str]
) -> None:
seen = use_transport(ok_search([OFF_HIT]))
body = client.get(URL, params={"q": "petit beurre"}, headers=auth_headers).json()
assert body["origin"] == "cache+off"
assert body["items"][0]["name"] == "Véritable Petit Beurre"
assert body["items"][0]["source"] == "off"
assert body["items"][0]["source_id"] == "3175680011480"
request = seen[0]
assert request.url.host == "search.openfoodfacts.org"
assert request.url.params["q"] == "petit beurre"
assert request.url.params["langs"] == "fr"
assert request.headers["User-Agent"] == "LifeTrack/1.0 (meejayproduction@gmail.com)"
cached = db.scalars(select(FoodItem)).all()
assert len(cached) == 1
assert cached[0].source_id == "3175680011480"
def test_second_search_is_served_from_the_cache(
client: TestClient, db: Session, auth_headers: dict[str, str]
) -> None:
db.add(
FoodItem(
source="off",
source_id="3175680011480",
name="Véritable Petit Beurre",
brand="LU",
)
)
db.commit()
seen = use_transport(ok_search([OFF_HIT]))
body = client.get(
URL, params={"q": "beurre", "limit": 1}, headers=auth_headers
).json()
assert body["origin"] == "cache"
assert len(body["items"]) == 1
assert seen == [] # OFF was never called
def test_cached_products_are_not_duplicated(
client: TestClient, db: Session, auth_headers: dict[str, str]
) -> None:
use_transport(ok_search([OFF_HIT]))
client.get(URL, params={"q": "petit beurre"}, headers=auth_headers)
client.get(URL, params={"q": "petit beurre"}, headers=auth_headers)
assert len(db.scalars(select(FoodItem)).all()) == 1
def test_search_matches_the_brand_locally(
client: TestClient, db: Session, auth_headers: dict[str, str]
) -> None:
db.add(FoodItem(source="ciqual", source_id="1234", name="Pomme, crue", brand=None))
db.commit()
use_transport(ok_search([]))
body = client.get(URL, params={"q": "pomme"}, headers=auth_headers).json()
assert [item["name"] for item in body["items"]] == ["Pomme, crue"]
def test_search_requires_two_characters(
client: TestClient, auth_headers: dict[str, str]
) -> None:
response = client.get(URL, params={"q": "a"}, headers=auth_headers)
assert response.status_code == 400
assert "2 caractères" in response.json()["error"]["message"]
def test_search_requires_authentication(client: TestClient) -> None:
assert client.get(URL, params={"q": "pomme"}).status_code == 401
# --- Degraded mode -------------------------------------------------------------
def test_unreachable_off_returns_a_french_503(
client: TestClient, auth_headers: dict[str, str]
) -> None:
def boom(request: httpx.Request) -> httpx.Response:
raise httpx.ConnectError("boom", request=request)
use_transport(boom)
response = client.get(URL, params={"q": "petit beurre"}, headers=auth_headers)
assert response.status_code == 503
error = response.json()["error"]
assert error["code"] == "service_unavailable"
assert "Open Food Facts" in error["message"]
assert "injoignable" in error["message"]
def test_unreachable_off_still_serves_local_results(
client: TestClient, db: Session, auth_headers: dict[str, str]
) -> None:
db.add(FoodItem(source="ciqual", source_id="1234", name="Pomme, crue"))
db.commit()
def boom(request: httpx.Request) -> httpx.Response:
raise httpx.ConnectError("boom", request=request)
use_transport(boom)
body = client.get(URL, params={"q": "pomme"}, headers=auth_headers).json()
assert body["origin"] == "cache"
assert len(body["items"]) == 1
def test_off_http_error_is_degraded_too(
client: TestClient, auth_headers: dict[str, str]
) -> None:
use_transport(lambda request: httpx.Response(429, json={"error": "rate limited"}))
response = client.get(URL, params={"q": "petit beurre"}, headers=auth_headers)
assert response.status_code == 503
# --- Barcode -------------------------------------------------------------------
def test_barcode_lookup_caches_and_then_hits_the_cache(
client: TestClient, db: Session, auth_headers: dict[str, str]
) -> None:
seen = use_transport(
lambda request: httpx.Response(200, json={"status": 1, "product": OFF_HIT})
)
body = client.get(
"/api/health/foods/barcode/3175680011480", headers=auth_headers
).json()
assert body["name"] == "Véritable Petit Beurre"
assert len(seen) == 1
assert seen[0].url.host == "world.openfoodfacts.org"
client.get("/api/health/foods/barcode/3175680011480", headers=auth_headers)
assert len(seen) == 1 # served from food_items
assert len(db.scalars(select(FoodItem)).all()) == 1
def test_unknown_barcode_is_a_french_404(
client: TestClient, auth_headers: dict[str, str]
) -> None:
use_transport(lambda request: httpx.Response(200, json={"status": 0}))
response = client.get("/api/health/foods/barcode/0000", headers=auth_headers)
assert response.status_code == 404
assert "code-barres" in response.json()["error"]["message"]
@@ -0,0 +1,236 @@
"""File importers of the health module: sniffing, parsing tolerance, and
idempotent deduplication (re-importing the same file changes nothing)."""
import datetime as dt
from pathlib import Path
import pytest
from sqlalchemy import select
from sqlalchemy.orm import Session
from app.core.importing.registry import IMPORTER_REGISTRY, detect_importer
from app.modules.auth.models import User
from app.modules.health import importers as health_importers
from app.modules.health.models import DailyActivity, FoodEntry, WeightEntry
from app.modules.imports import service as imports_service
from app.modules.imports.models import ImportRun
FIXTURES = Path(__file__).resolve().parents[1] / "fixtures" / "health"
def fixture_bytes(name: str) -> bytes:
return (FIXTURES / name).read_bytes()
def run(db: Session, user: User, importer_id: str, name: str) -> ImportRun:
return imports_service.run_import(db, user, importer_id, name, fixture_bytes(name))
# --- Registration & sniffing ---------------------------------------------------
def test_the_three_importers_are_registered() -> None:
for importer_id in ("foodvisor_csv", "health_sync_csv", "weight_generic_csv"):
assert importer_id in IMPORTER_REGISTRY
assert IMPORTER_REGISTRY[importer_id].domain == "health"
assert IMPORTER_REGISTRY[importer_id].accepted_extensions == (".csv",)
# French label, shown as-is in the source picker
assert IMPORTER_REGISTRY[importer_id].label
@pytest.mark.parametrize(
("filename", "expected"),
[
("foodvisor_export.csv", "foodvisor_csv"),
("health_sync.csv", "health_sync_csv"),
("weight_history.csv", "weight_generic_csv"),
],
)
def test_sniffing_is_unambiguous(filename: str, expected: str) -> None:
detected = detect_importer(filename, fixture_bytes(filename)[:4096])
assert detected is not None, f"{filename} not detected"
assert detected.id == expected
def test_sniff_never_raises_on_garbage() -> None:
for cls in (
health_importers.FoodvisorCsvImporter,
health_importers.HealthSyncCsvImporter,
health_importers.WeightGenericCsvImporter,
):
assert cls.sniff("x.pdf", b"%PDF-1.7") is False
assert cls.sniff("x.csv", b"\xff\xfe\x00binary") is False
assert cls.sniff("x.csv", b"") is False
# --- Parsing helpers -----------------------------------------------------------
def test_decoding_falls_back_to_cp1252() -> None:
assert health_importers.decode_bytes("Protéines".encode()) == "Protéines"
assert health_importers.decode_bytes("Protéines".encode("cp1252")) == "Protéines"
def test_number_parsing_is_tolerant() -> None:
to_decimal = health_importers.to_decimal
assert float(to_decimal("1 234,5")) == 1234.5
assert float(to_decimal("< 0,5")) == 0.5
assert float(to_decimal("traces")) == 0.0
assert to_decimal("-") is None
assert to_decimal("") is None
assert float(to_decimal("228.0")) == 228.0
def test_naive_timestamps_are_read_as_paris_time() -> None:
parsed = health_importers.parse_datetime("2026-08-10 08:15")
assert parsed == dt.datetime(2026, 8, 10, 6, 15, tzinfo=dt.UTC) # CEST = UTC+2
assert health_importers.parse_datetime("10/08/2026") is not None
assert health_importers.parse_datetime("2026-08-10T08:15:00Z") == dt.datetime(
2026, 8, 10, 8, 15, tzinfo=dt.UTC
)
assert health_importers.parse_datetime("n'importe quoi") is None
def test_meal_mapping_fr_en() -> None:
assert health_importers.map_meal("Petit-déjeuner").value == "breakfast"
assert health_importers.map_meal("Déjeuner").value == "lunch"
assert health_importers.map_meal("Dîner").value == "dinner"
assert health_importers.map_meal("Collation").value == "snack"
assert health_importers.map_meal("Breakfast").value == "breakfast"
assert health_importers.map_meal("inconnu").value == "snack"
# --- Foodvisor -----------------------------------------------------------------
def test_foodvisor_import_maps_meals_and_macros(db: Session, user: User) -> None:
result = run(db, user, "foodvisor_csv", "foodvisor_export.csv")
assert result.status == "completed"
assert (result.rows_total, result.rows_inserted, result.rows_errors) == (4, 4, 0)
entries = db.scalars(select(FoodEntry).order_by(FoodEntry.eaten_at.asc())).all()
assert [entry.meal.value for entry in entries] == [
"breakfast",
"lunch",
"dinner",
"snack",
]
first = entries[0]
assert first.name == "Flocons d'avoine"
assert first.brand == "Quaker"
assert float(first.kcal) == 228.0
assert float(first.protein_g) == 8.1
assert float(first.quantity) == 60.0
assert first.source == "foodvisor"
assert first.external_id is not None
assert first.import_run_id == result.id
assert first.raw["aliment"] == "Flocons d'avoine"
# 2026-08-10 08:15 Paris -> 06:15 UTC
assert first.eaten_at == dt.datetime(2026, 8, 10, 6, 15, tzinfo=dt.UTC)
assert entries[1].brand is None
def test_foodvisor_reimport_is_fully_duplicate(db: Session, user: User) -> None:
run(db, user, "foodvisor_csv", "foodvisor_export.csv")
second = run(db, user, "foodvisor_csv", "foodvisor_export.csv")
assert second.rows_total == 4
assert second.rows_duplicates == 4
assert second.rows_inserted == 0
assert db.scalar(select(FoodEntry).where(FoodEntry.user_id == user.id)) is not None
assert len(db.scalars(select(FoodEntry)).all()) == 4
def test_foodvisor_rejects_incomplete_rows(db: Session, user: User) -> None:
data = b"Date;Repas;Aliment;Calories (kcal)\n;Dejeuner;;\n"
result = imports_service.run_import(db, user, "foodvisor_csv", "x.csv", data)
assert result.rows_errors == 1
assert "requis" in result.error_details[0]["message"]
# --- Health Sync ---------------------------------------------------------------
def test_health_sync_import_splits_weights_and_activity(
db: Session, user: User
) -> None:
result = run(db, user, "health_sync_csv", "health_sync.csv")
assert result.status == "completed"
# 3 rows -> 3 activity records + 2 weights (one row has no weight)
assert result.rows_total == 5
assert result.rows_inserted == 5
activities = db.scalars(
select(DailyActivity).order_by(DailyActivity.date.asc())
).all()
assert [row.date for row in activities] == [
dt.date(2026, 8, 10),
dt.date(2026, 8, 11),
dt.date(2026, 8, 12),
]
assert activities[0].steps == 9421
assert activities[0].distance_m == 6800 # 6,80 km -> metres
assert float(activities[0].active_kcal) == 520.0
assert activities[0].source == "csv_import"
assert activities[0].import_run_id == result.id
weights = db.scalars(
select(WeightEntry).order_by(WeightEntry.measured_at.asc())
).all()
assert [float(row.weight_kg) for row in weights] == [92.4, 92.1]
def test_health_sync_reimport_is_fully_duplicate(db: Session, user: User) -> None:
run(db, user, "health_sync_csv", "health_sync.csv")
second = run(db, user, "health_sync_csv", "health_sync.csv")
assert second.rows_total == 5
assert second.rows_duplicates == 5
assert second.rows_inserted == 0
assert second.rows_updated == 0
def test_health_sync_updated_values_replace_the_day(db: Session, user: User) -> None:
run(db, user, "health_sync_csv", "health_sync.csv")
later = b"Date;Pas;Distance (km);Calories actives\n2026-08-10;11000;7,50;600\n"
result = imports_service.run_import(db, user, "health_sync_csv", "hs.csv", later)
assert result.rows_updated == 1
assert result.rows_inserted == 0
row = db.scalar(
select(DailyActivity).where(DailyActivity.date == dt.date(2026, 8, 10))
)
assert row.steps == 11000
assert row.distance_m == 7500
# --- Generic weight history ----------------------------------------------------
def test_weight_generic_import_and_reimport(db: Session, user: User) -> None:
result = run(db, user, "weight_generic_csv", "weight_history.csv")
assert (result.rows_total, result.rows_inserted) == (3, 3)
weights = db.scalars(
select(WeightEntry).order_by(WeightEntry.measured_at.asc())
).all()
assert [float(row.weight_kg) for row in weights] == [95.2, 94.1, 93.4]
assert weights[0].source == "csv_import"
assert weights[0].measured_at == dt.datetime(2026, 6, 30, 22, 0, tzinfo=dt.UTC)
second = run(db, user, "weight_generic_csv", "weight_history.csv")
assert second.rows_duplicates == 3
assert len(db.scalars(select(WeightEntry)).all()) == 3
def test_weight_generic_rejects_out_of_range(db: Session, user: User) -> None:
data = b"date;poids\n2026-07-01;950,0\n"
result = imports_service.run_import(db, user, "weight_generic_csv", "w.csv", data)
assert result.rows_errors == 1
assert "bornes" in result.error_details[0]["message"]
# --- Central rollback ----------------------------------------------------------
def test_deleting_an_import_run_removes_its_rows(db: Session, user: User) -> None:
result = run(db, user, "foodvisor_csv", "foodvisor_export.csv")
assert len(db.scalars(select(FoodEntry)).all()) == 4
imports_service.rollback_run(db, user.id, result.id)
assert db.scalars(select(FoodEntry)).all() == []
@@ -0,0 +1,412 @@
"""JSON ingestion for the health domain: canonical shape, health-connect-webhook
bridge shape, idempotence and per-record error reporting."""
import datetime as dt
from fastapi.testclient import TestClient
from sqlalchemy import select
from sqlalchemy.orm import Session
from app.core.ingest.registry import INGEST_REGISTRY
from app.modules.health import (
ingest as health_ingest, # noqa: F401 — registers the handler
)
from app.modules.health.models import (
DailyActivity,
FoodEntry,
WaterEntry,
WeightEntry,
Workout,
)
URL = "/api/ingest/health"
def post(
client: TestClient, headers: dict[str, str], *records, source="android_bridge"
):
response = client.post(
URL, json={"source": source, "records": list(records)}, headers=headers
)
assert response.status_code == 200, response.text
return response.json()
# --- Registration & auth -------------------------------------------------------
def test_handler_is_registered_with_every_record_type() -> None:
handler = INGEST_REGISTRY["health"]
assert set(handler.record_types) == {
"weight",
"steps",
"distance",
"active_calories",
"total_calories",
"exercise_session",
"nutrition",
"hydration",
}
def test_ingest_requires_a_credential(client: TestClient) -> None:
response = client.post(
URL, json={"source": "x", "records": [{"type": "steps", "data": {}}]}
)
assert response.status_code == 401
def test_ingest_accepts_the_device_key(client: TestClient, device_key) -> None:
body = post(
client,
device_key.headers,
{
"type": "steps",
"external_id": "hc:2026-08-12",
"data": {"day": "2026-08-12", "steps": 9421},
},
)
assert body == {
"domain": "health",
"received": 1,
"inserted": 1,
"updated": 0,
"duplicates": 0,
"errors": [],
}
# --- Canonical shape (architecture §5.5) ---------------------------------------
def test_canonical_steps_record_fills_the_daily_row(
client: TestClient, db: Session, device_key
) -> None:
post(
client,
device_key.headers,
{
"type": "steps",
"external_id": "hc:2026-08-12",
"data": {
"day": "2026-08-12",
"steps": 9421,
"calories_kcal": 2350,
"distance_m": 6800,
},
},
)
row = db.scalar(select(DailyActivity))
assert row.date == dt.date(2026, 8, 12)
assert row.steps == 9421
assert float(row.active_kcal) == 2350.0 # bridge convention: active calories
assert row.distance_m == 6800
assert row.source == "health_connect" # android_bridge is a known alias
assert row.external_id == "hc:2026-08-12"
assert row.raw["day"] == "2026-08-12"
def test_reposting_the_same_day_is_idempotent_then_updates(
client: TestClient, db: Session, device_key
) -> None:
record = {
"type": "steps",
"external_id": "hc:2026-08-12",
"data": {"day": "2026-08-12", "steps": 9421},
}
assert post(client, device_key.headers, record)["inserted"] == 1
assert post(client, device_key.headers, record)["duplicates"] == 1
grown = {**record, "data": {**record["data"], "steps": 12500}}
assert post(client, device_key.headers, grown)["updated"] == 1
rows = db.scalars(select(DailyActivity)).all()
assert len(rows) == 1
assert rows[0].steps == 12500
def test_each_activity_type_targets_its_own_field(
client: TestClient, db: Session, device_key
) -> None:
post(
client,
device_key.headers,
{"type": "steps", "data": {"day": "2026-08-12", "steps": 100}},
{"type": "distance", "data": {"day": "2026-08-12", "value": 4200}},
{"type": "active_calories", "data": {"day": "2026-08-12", "value": 480}},
{"type": "total_calories", "data": {"day": "2026-08-12", "value": 2600}},
)
rows = db.scalars(select(DailyActivity)).all()
assert len(rows) == 1 # same (user, day, source)
row = rows[0]
assert (row.steps, row.distance_m) == (100, 4200)
assert float(row.active_kcal) == 480.0
assert float(row.total_kcal) == 2600.0
def test_canonical_weight_record(client: TestClient, db: Session, device_key) -> None:
post(
client,
device_key.headers,
{
"type": "weight",
"external_id": "hc:w:1755012345",
"data": {"measured_at": "2026-08-13T06:31:00Z", "weight_kg": 91.4},
},
)
row = db.scalar(select(WeightEntry))
assert float(row.weight_kg) == 91.4
assert row.measured_at == dt.datetime(2026, 8, 13, 6, 31, tzinfo=dt.UTC)
assert row.source == "health_connect"
# --- health-connect-webhook bridge shape ---------------------------------------
def test_bridge_weight_record_with_nested_value_and_metadata(
client: TestClient, db: Session, device_key
) -> None:
post(
client,
device_key.headers,
{
"type": "weight",
"data": {
"time": "2026-08-13T06:31:00Z",
"value": {"weight": 91.4, "unit": "kg"},
"metadata": {
"id": "hc-uuid-1",
"data_origin": "com.sec.android.app.shealth",
},
},
},
source="health-connect-webhook",
)
row = db.scalar(select(WeightEntry))
assert float(row.weight_kg) == 91.4
assert row.external_id == "hc-uuid-1"
assert row.source == "health_connect"
assert row.raw["metadata"]["id"] == "hc-uuid-1"
# Re-pushing the same record (48 h sliding window) must not duplicate it.
body = post(
client,
device_key.headers,
{
"type": "weight",
"data": {
"time": "2026-08-13T06:31:00Z",
"value": {"weight": 91.4},
"metadata": {"id": "hc-uuid-1"},
},
},
source="health-connect-webhook",
)
assert body["duplicates"] == 1
def test_bridge_steps_record_uses_count_and_start_time(
client: TestClient, db: Session, device_key
) -> None:
post(
client,
device_key.headers,
{
"type": "steps",
"data": {
"start_time": "2026-08-12T05:00:00Z",
"end_time": "2026-08-12T20:00:00Z",
"count": 9421,
"metadata": {"id": "hc-steps-1"},
},
},
source="health-connect-webhook",
)
row = db.scalar(select(DailyActivity))
assert row.date == dt.date(2026, 8, 12) # local day in Europe/Paris
assert row.steps == 9421
def test_bridge_exercise_session_becomes_a_workout(
client: TestClient, db: Session, device_key
) -> None:
post(
client,
device_key.headers,
{
"type": "exercise_session",
"data": {
"start_time": "2026-08-12T16:00:00Z",
"end_time": "2026-08-12T17:00:00Z",
"exercise_type": "RUNNING_TREADMILL",
"energy_kcal": 620,
"distance_m": 9500,
"metadata": {"id": "hc-ex-1"},
},
},
source="health-connect-webhook",
)
row = db.scalar(select(Workout))
assert row.sport_type.value == "treadmill_run"
assert row.duration_s == 3600
assert float(row.kcal) == 620.0
assert row.distance_m == 9500
assert row.external_id == "hc-ex-1"
assert row.is_hidden is False
def test_overlapping_sessions_from_two_sources_hide_the_weaker_one(
client: TestClient, db: Session, device_key
) -> None:
session = {
"type": "exercise_session",
"data": {
"start_time": "2026-08-12T16:00:00Z",
"end_time": "2026-08-12T17:00:00Z",
"exercise_type": "RUNNING_TREADMILL",
"metadata": {"id": "fs-1"},
},
}
post(client, device_key.headers, session, source="fitshow")
post(
client,
device_key.headers,
{**session, "data": {**session["data"], "metadata": {"id": "hc-1"}}},
source="health-connect-webhook",
)
rows = {row.source: row for row in db.scalars(select(Workout)).all()}
assert len(rows) == 2
assert rows["health_connect"].is_hidden is False # higher priority source
assert rows["fitshow"].is_hidden is True
def test_bridge_nutrition_record_maps_meal_and_grams(
client: TestClient, db: Session, device_key
) -> None:
post(
client,
device_key.headers,
{
"type": "nutrition",
"data": {
"start_time": "2026-08-13T11:45:00Z",
"end_time": "2026-08-13T12:15:00Z",
"meal_type": 2,
"name": "Salade poulet",
"energy_kcal": 620.0,
"protein_g": 42.1,
"total_carbohydrate_g": 51.0,
"total_fat_g": 24.3,
"dietary_fiber_g": 7.0,
"sodium_g": 1.1,
"metadata": {"id": "hc-nut-1"},
},
},
source="health-connect-webhook",
)
row = db.scalar(select(FoodEntry))
assert row.meal.value == "lunch"
assert row.name == "Salade poulet"
assert float(row.kcal) == 620.0
assert float(row.carbs_g) == 51.0
assert float(row.sodium_mg) == 1100.0 # Health Connect sends grams
def test_nutrition_meal_falls_back_on_the_local_hour(
client: TestClient, db: Session, device_key
) -> None:
post(
client,
device_key.headers,
{
"type": "nutrition",
"data": {
"start_time": "2026-08-13T06:00:00Z", # 08:00 Paris
"name": "Café",
"energy_kcal": 5,
"meal_type": 0,
},
},
)
assert db.scalar(select(FoodEntry)).meal.value == "breakfast"
def test_bridge_hydration_record(client: TestClient, db: Session, device_key) -> None:
post(
client,
device_key.headers,
{
"type": "hydration",
"data": {
"start_time": "2026-08-13T09:00:00Z",
"volume_liters": 0.5,
"metadata": {"id": "hc-hyd-1"},
},
},
source="health-connect-webhook",
)
row = db.scalar(select(WaterEntry))
assert row.volume_ml == 500
assert row.external_id == "hc-hyd-1"
def test_records_without_external_id_dedupe_on_content(
client: TestClient, db: Session, device_key
) -> None:
record = {
"type": "hydration",
"data": {"start_time": "2026-08-13T09:00:00Z", "volume_ml": 250},
}
assert post(client, device_key.headers, record)["inserted"] == 1
assert post(client, device_key.headers, record)["duplicates"] == 1
assert len(db.scalars(select(WaterEntry)).all()) == 1
# --- Error handling ------------------------------------------------------------
def test_unsupported_type_is_reported_per_record(
client: TestClient, device_key
) -> None:
body = post(
client,
device_key.headers,
{"type": "sleep", "data": {"day": "2026-08-12"}},
{"type": "steps", "data": {"day": "2026-08-12", "steps": 10}},
)
assert body["inserted"] == 1
assert body["errors"][0]["index"] == 0
assert body["errors"][0]["type"] == "sleep"
def test_invalid_records_do_not_fail_the_batch(
client: TestClient, db: Session, device_key
) -> None:
body = post(
client,
device_key.headers,
{"type": "weight", "data": {"measured_at": "2026-08-13T06:31:00Z"}},
{
"type": "weight",
"data": {"measured_at": "2026-08-13T06:31:00Z", "weight_kg": 900},
},
{"type": "steps", "data": {"steps": 10}},
{
"type": "weight",
"data": {"measured_at": "2026-08-13T07:00:00Z", "weight_kg": 91.4},
},
)
assert body["inserted"] == 1
assert len(body["errors"]) == 3
messages = " ".join(error["message"] for error in body["errors"])
assert "requis" in messages
assert "bornes" in messages
assert len(db.scalars(select(WeightEntry)).all()) == 1
def test_unknown_domain_is_a_404(client: TestClient, device_key) -> None:
response = client.post(
"/api/ingest/sleep",
json={"source": "x", "records": [{"type": "steps", "data": {}}]},
headers=device_key.headers,
)
assert response.status_code == 404
@@ -0,0 +1,280 @@
"""Planning (addendum-planning.md) at service level: derived "done" days,
adherence over crafted weeks, streaks across week boundaries, timezone edges."""
import datetime as dt
from decimal import Decimal
from zoneinfo import ZoneInfo
from sqlalchemy.orm import Session
from app.modules.auth.models import User
from app.modules.health import service, stats
from app.modules.health.models import (
FoodEntry,
ScheduleKind,
TrackingSchedule,
WeightEntry,
Workout,
)
PARIS = ZoneInfo("Europe/Paris")
def utc_at(day: dt.date, hour: int, minute: int = 0) -> dt.datetime:
"""UTC instant for a Paris wall-clock time."""
return dt.datetime(
day.year, day.month, day.day, hour, minute, tzinfo=PARIS
).astimezone(dt.UTC)
def schedule(db: Session, user: User, kind: str, weekdays: list[int], enabled=True):
db.add(
TrackingSchedule(
user_id=user.id,
kind=ScheduleKind(kind),
weekdays=weekdays,
enabled=enabled,
)
)
db.commit()
def add_weight(db: Session, user: User, day: dt.date, kg: float, hour: int = 7) -> None:
db.add(
WeightEntry(
user_id=user.id,
source="manual",
measured_at=utc_at(day, hour),
weight_kg=Decimal(str(kg)),
)
)
db.commit()
def add_workout(db: Session, user: User, day: dt.date, hidden: bool = False) -> None:
start = utc_at(day, 18)
db.add(
Workout(
user_id=user.id,
source="manual",
started_at=start,
ended_at=start + dt.timedelta(hours=1),
sport_type="running",
is_hidden=hidden,
)
)
db.commit()
def add_food(
db: Session, user: User, day: dt.date, kcal: float, hour: int = 13
) -> None:
db.add(
FoodEntry(
user_id=user.id,
source="manual",
eaten_at=utc_at(day, hour),
meal="lunch",
name="Repas",
quantity=Decimal(100),
unit="g",
kcal=Decimal(str(kcal)),
)
)
db.commit()
# --- Derived "done" days -------------------------------------------------------
def test_done_days_are_derived_from_the_three_tables(db: Session, user: User) -> None:
day = dt.date(2026, 8, 12)
add_weight(db, user, day, 91.6)
add_weight(db, user, day, 92.4, hour=20) # second weigh-in of the day
add_workout(db, user, day)
add_food(db, user, day, 620.0)
add_food(db, user, day, 380.0)
done = service.done_days_by_kind(db, user.id, day, day, PARIS)
assert done["weigh_in"] == {day: 91.6} # FIRST weigh-in of the local day
assert done["workout"] == {day: 1.0}
assert done["food_log"] == {day: 1000.0}
def test_hidden_workouts_do_not_count_as_done(db: Session, user: User) -> None:
day = dt.date(2026, 8, 12)
add_workout(db, user, day, hidden=True)
done = service.done_days_by_kind(db, user.id, day, day, PARIS)
assert done["workout"] == {}
def test_local_day_boundary_uses_the_profile_timezone(db: Session, user: User) -> None:
# 22:30 UTC on 12/08 is 00:30 on 13/08 in Paris (CEST = UTC+2).
db.add(
WeightEntry(
user_id=user.id,
source="manual",
measured_at=dt.datetime(2026, 8, 12, 22, 30, tzinfo=dt.UTC),
weight_kg=Decimal("91.6"),
)
)
db.commit()
paris = service.done_days_by_kind(
db, user.id, dt.date(2026, 8, 12), dt.date(2026, 8, 13), PARIS
)
assert dt.date(2026, 8, 13) in paris["weigh_in"]
assert dt.date(2026, 8, 12) not in paris["weigh_in"]
utc = service.done_days_by_kind(
db, user.id, dt.date(2026, 8, 12), dt.date(2026, 8, 13), ZoneInfo("UTC")
)
assert dt.date(2026, 8, 12) in utc["weigh_in"]
# --- Schedules -----------------------------------------------------------------
def test_missing_schedules_are_returned_disabled(db: Session, user: User) -> None:
rows = service.list_schedules(db, user.id)
assert [row.kind.value for row in rows] == ["weigh_in", "workout", "food_log"]
assert all(row.enabled is False and row.weekdays == [] for row in rows)
def test_upsert_schedule_is_idempotent_per_kind(db: Session, user: User) -> None:
from app.modules.health.schemas import ScheduleUpdate
first = service.upsert_schedule(
db,
user.id,
ScheduleKind.WEIGH_IN,
ScheduleUpdate(weekdays=[4, 0], enabled=True),
)
assert first.weekdays == [0, 4]
second = service.upsert_schedule(
db, user.id, ScheduleKind.WEIGH_IN, ScheduleUpdate(weekdays=[2], enabled=False)
)
assert first.id == second.id
assert second.weekdays == [2]
assert second.enabled is False
# --- Adherence over crafted weeks ---------------------------------------------
def test_adherence_over_four_weeks_of_mondays(db: Session, user: User) -> None:
today = service.today_local(PARIS)
weekday = today.weekday()
schedule(db, user, "weigh_in", [weekday])
planned = [today - dt.timedelta(days=7 * n) for n in range(4)]
for day in planned[:3]: # the oldest planned day is missed
add_weight(db, user, day, 92.0)
body = stats.adherence_stats(db, user.id, planned[-1], today, PARIS, "weigh_in")
assert [kind.kind.value for kind in body.kinds] == ["weigh_in"]
weigh_in = body.kinds[0]
assert weigh_in.weekdays == [weekday]
assert weigh_in.planned_days == 4
assert weigh_in.done_days == 3
assert weigh_in.missed_days == 1
assert weigh_in.adherence_pct == 75.0
assert weigh_in.streak.current == 3
assert weigh_in.streak.best == 3
assert len(weigh_in.days) == 22 # inclusive range
statuses = {row.date: row.status for row in weigh_in.days}
assert statuses[today] == "done"
assert statuses[planned[-1]] == "missed"
assert statuses[today - dt.timedelta(days=1)] == "rest"
def test_streak_spans_week_boundaries_with_two_days_a_week(
db: Session, user: User
) -> None:
today = service.today_local(PARIS)
# Two consecutive weekdays: for a Sunday anchor the pair straddles two ISO weeks.
first_day, second_day = today.weekday(), (today.weekday() + 1) % 7
schedule(db, user, "workout", sorted({first_day, second_day}))
start = today - dt.timedelta(days=21)
for offset in range((today - start).days + 1):
day = start + dt.timedelta(days=offset)
if day.weekday() in {first_day, second_day}:
add_workout(db, user, day)
body = stats.adherence_stats(db, user.id, start, today, PARIS, "workout")
workout = body.kinds[0]
assert workout.planned_days == workout.done_days
assert workout.missed_days == 0
assert workout.adherence_pct == 100.0
assert workout.streak.current == workout.planned_days
assert workout.streak.best >= workout.planned_days
def test_adherence_without_schedule_reports_no_planned_day(
db: Session, user: User
) -> None:
today = service.today_local(PARIS)
add_weight(db, user, today, 92.0)
body = stats.adherence_stats(
db, user.id, today - dt.timedelta(days=6), today, PARIS
)
kinds = {kind.kind.value: kind for kind in body.kinds}
assert set(kinds) == {"weigh_in", "workout", "food_log"}
weigh_in = kinds["weigh_in"]
assert weigh_in.planned_days == 0
assert weigh_in.adherence_pct is None
assert weigh_in.streak.current == 0
# An unplanned weigh-in is still reported for the heatmap.
statuses = {row.date: row.status for row in weigh_in.days}
assert statuses[today] == "done_unplanned"
def test_disabled_schedule_plans_nothing(db: Session, user: User) -> None:
today = service.today_local(PARIS)
schedule(db, user, "weigh_in", [today.weekday()], enabled=False)
body = stats.adherence_stats(
db, user.id, today - dt.timedelta(days=6), today, PARIS, "weigh_in"
)
assert body.kinds[0].planned_days == 0
assert body.kinds[0].enabled is False
# --- Today ---------------------------------------------------------------------
def test_today_reports_planned_done_and_values(db: Session, user: User) -> None:
today = service.today_local(PARIS)
weekday = today.weekday()
schedule(db, user, "weigh_in", [weekday])
schedule(db, user, "workout", [(weekday + 3) % 7]) # not today
schedule(db, user, "food_log", [weekday])
add_weight(db, user, today, 91.6)
add_food(db, user, today, 620.0)
body = stats.today_view(db, user.id, PARIS)
assert body.date == today
items = {item.kind.value: item for item in body.items}
assert items["weigh_in"].planned is True
assert items["weigh_in"].done is True
assert items["weigh_in"].value == 91.6
assert items["food_log"].value == 620.0
assert items["workout"].planned is False
assert items["workout"].done is False
assert items["workout"].value is None
assert body.streaks["weigh_in"].current == 1
assert body.streaks["workout"].current == 0
def test_today_streak_survives_a_planned_but_unfinished_day(
db: Session, user: User
) -> None:
today = service.today_local(PARIS)
schedule(db, user, "weigh_in", [today.weekday()])
for offset in (7, 14):
add_weight(db, user, today - dt.timedelta(days=offset), 92.0)
body = stats.today_view(db, user.id, PARIS)
items = {item.kind.value: item for item in body.items}
assert items["weigh_in"].planned is True
assert items["weigh_in"].done is False
# Today is not over yet: the two previous planned days still count.
assert body.streaks["weigh_in"].current == 2
assert body.streaks["weigh_in"].best == 2
@@ -0,0 +1,275 @@
"""Regression tests for defects found while walking the live health journey.
Each test below fails on the code as it was before the matching fix:
1. an unknown value on an enum-backed query filter (`sport_type`, `meal`,
`status`) reached SQLAlchemy and raised `LookupError` -> HTTP 500 with no
French error envelope;
2. `/health/weights/stats` drew `plan_line` from the goal start to the LAST DAY
OF THE REQUESTED WINDOW, so the plan reference line of the weight chart
moved with the chart period and was ~7x too steep for a `weekly_rate` goal
(datamodel-health-vape.md §5.6b: `days = delta / (weekly_rate_kg / 7)`);
3. `/health/goals/active` reported `deficit_target_kcal` as
`tdee_smoothed - budget`, i.e. the deficit left AFTER the calorie floor
clamped the budget, instead of the deficit AIMED AT by the goal
(§5.3 `BudgetResult.deficit_target = rate x 7700 / 7`).
"""
import datetime as dt
from zoneinfo import ZoneInfo
from fastapi.testclient import TestClient
BASE = "/api/health"
PARIS = ZoneInfo("Europe/Paris")
PROFILE = {
"height_cm": 178.0,
"sex": "male",
"birthdate": "1990-05-12",
"activity_level": "moderate",
"timezone": "Europe/Paris",
}
def _today() -> dt.date:
return dt.datetime.now(dt.UTC).astimezone(PARIS).date()
def _at(day: dt.date, hour: int) -> str:
return (
dt.datetime(day.year, day.month, day.day, hour, tzinfo=PARIS)
.astimezone(dt.UTC)
.isoformat()
.replace("+00:00", "Z")
)
def _seed(client: TestClient, headers: dict[str, str]) -> None:
assert (
client.put(f"{BASE}/profile", json=PROFILE, headers=headers).status_code == 200
)
today = _today()
for offset, weight in ((6, 95.0), (4, 94.6), (2, 94.3), (0, 94.0)):
response = client.post(
f"{BASE}/weights",
json={
"measured_at": _at(today - dt.timedelta(days=offset), 7),
"weight_kg": weight,
},
headers=headers,
)
assert response.status_code == 201, response.text
# --- 1. Unknown enum filter values must be rejected, never crash --------------
def test_unknown_sport_type_filter_is_rejected(
client: TestClient, auth_headers: dict[str, str]
) -> None:
response = client.get(
f"{BASE}/workouts", params={"sport_type": "quidditch"}, headers=auth_headers
)
assert response.status_code == 422, response.text
assert response.json()["error"]["code"] == "validation_error"
def test_unknown_meal_filter_is_rejected(
client: TestClient, auth_headers: dict[str, str]
) -> None:
response = client.get(
f"{BASE}/nutrition/entries", params={"meal": "brunch"}, headers=auth_headers
)
assert response.status_code == 422, response.text
assert response.json()["error"]["code"] == "validation_error"
def test_unknown_goal_status_filter_is_rejected(
client: TestClient, auth_headers: dict[str, str]
) -> None:
response = client.get(
f"{BASE}/goals", params={"status": "zzz"}, headers=auth_headers
)
assert response.status_code == 422, response.text
assert response.json()["error"]["code"] == "validation_error"
def test_known_enum_filters_still_work(
client: TestClient, auth_headers: dict[str, str]
) -> None:
assert (
client.get(
f"{BASE}/workouts", params={"sport_type": "cycling"}, headers=auth_headers
).status_code
== 200
)
assert (
client.get(
f"{BASE}/nutrition/entries", params={"meal": "lunch"}, headers=auth_headers
).status_code
== 200
)
assert (
client.get(
f"{BASE}/goals", params={"status": "active"}, headers=auth_headers
).status_code
== 200
)
# --- 2. plan_line must follow the goal rate, not the chart window -------------
def test_plan_line_follows_the_weekly_rate_not_the_window(
client: TestClient, auth_headers: dict[str, str]
) -> None:
_seed(client, auth_headers)
today = _today()
start = today - dt.timedelta(days=6)
created = client.post(
f"{BASE}/goals",
json={
"mode": "weekly_rate",
"start_date": start.isoformat(),
"start_weight_kg": 95.0,
"target_weight_kg": 85.0,
"weekly_rate_kg": 0.5,
},
headers=auth_headers,
)
assert created.status_code == 201, created.text
# 10 kg at 0.5 kg/week = 20 weeks = 140 days after the goal start date.
expected_end = (start + dt.timedelta(days=140)).isoformat()
def plan_line(params: dict) -> list:
response = client.get(
f"{BASE}/weights/stats", params=params, headers=auth_headers
)
assert response.status_code == 200, response.text
body = response.json()
found = [s for s in body["series"] if s["name"] == "plan_line"]
assert found, "plan_line series missing"
assert body["meta"]["plan_end_date"] == expected_end
return found[0]["points"]
wide = plan_line({"from": start.isoformat(), "to": today.isoformat()})
narrow = plan_line(
{"from": (today - dt.timedelta(days=2)).isoformat(), "to": today.isoformat()}
)
assert wide == narrow, "plan_line must not depend on the requested window"
assert wide[0] == [start.isoformat(), 95.0]
assert wide[1] == [expected_end, 85.0]
def test_plan_line_uses_target_date_when_the_goal_has_one(
client: TestClient, auth_headers: dict[str, str]
) -> None:
_seed(client, auth_headers)
today = _today()
start = today - dt.timedelta(days=6)
target_date = today + dt.timedelta(days=200)
assert (
client.post(
f"{BASE}/goals",
json={
"mode": "target_date",
"start_date": start.isoformat(),
"start_weight_kg": 95.0,
"target_weight_kg": 85.0,
"target_date": target_date.isoformat(),
},
headers=auth_headers,
).status_code
== 201
)
body = client.get(
f"{BASE}/weights/stats",
params={"from": start.isoformat(), "to": today.isoformat()},
headers=auth_headers,
).json()
points = next(s for s in body["series"] if s["name"] == "plan_line")["points"]
assert points[1] == [target_date.isoformat(), 85.0]
def test_maintain_goal_has_no_plan_line(
client: TestClient, auth_headers: dict[str, str]
) -> None:
_seed(client, auth_headers)
today = _today()
assert (
client.post(
f"{BASE}/goals",
json={
"mode": "maintain",
"start_date": (today - dt.timedelta(days=6)).isoformat(),
"start_weight_kg": 95.0,
"target_weight_kg": 95.0,
},
headers=auth_headers,
).status_code
== 201
)
body = client.get(f"{BASE}/weights/stats", headers=auth_headers).json()
assert [s for s in body["series"] if s["name"] == "plan_line"] == []
assert body["meta"]["plan_end_date"] is None
# --- 3. deficit_target_kcal is the AIMED deficit, floor or not ----------------
def test_deficit_target_is_the_goal_target_even_when_the_floor_applies(
client: TestClient, auth_headers: dict[str, str]
) -> None:
_seed(client, auth_headers)
today = _today()
assert (
client.post(
f"{BASE}/goals",
json={
"mode": "weekly_rate",
"start_date": (today - dt.timedelta(days=6)).isoformat(),
"start_weight_kg": 95.0,
"target_weight_kg": 85.0,
"weekly_rate_kg": 1.5,
},
headers=auth_headers,
).status_code
== 201
)
budget = client.get(f"{BASE}/goals/active", headers=auth_headers).json()["budget"]
# 1.5 kg/week -> 1.5 * 7700 / 7 = 1650 kcal/day, whatever the floor does.
assert budget["deficit_target_kcal"] == 1650.0
assert budget["floor_applied"] is True
assert budget["kcal"] == 1500.0
def test_energy_balance_meta_exposes_the_target_deficit(
client: TestClient, auth_headers: dict[str, str]
) -> None:
_seed(client, auth_headers)
today = _today()
assert (
client.post(
f"{BASE}/goals",
json={
"mode": "weekly_rate",
"start_date": (today - dt.timedelta(days=6)).isoformat(),
"start_weight_kg": 95.0,
"target_weight_kg": 85.0,
"weekly_rate_kg": 0.5,
},
headers=auth_headers,
).status_code
== 201
)
meta = client.get(
f"{BASE}/energy-balance",
params={
"from": (today - dt.timedelta(days=6)).isoformat(),
"to": today.isoformat(),
},
headers=auth_headers,
).json()["meta"]
assert meta["deficit_target_kcal"] == 550.0 # 0.5 * 7700 / 7
File diff suppressed because it is too large Load Diff
+914
View File
@@ -0,0 +1,914 @@
"""Contract tests of the vape module API (datamodel-health-vape.md §8.4)."""
from collections.abc import Callable
from datetime import UTC, datetime, timedelta
from typing import Any
from zoneinfo import ZoneInfo
import pytest
from fastapi.testclient import TestClient
from sqlalchemy.orm import Session
from app.modules.auth.models import User
PARIS = ZoneInfo("Europe/Paris")
BASE = "/api/vape"
def _today() -> Any:
return datetime.now(PARIS).date()
def _day(offset: int) -> str:
return (_today() + timedelta(days=offset)).isoformat()
def _settings_payload(**overrides: Any) -> dict[str, Any]:
payload = {
"quit_date": _day(-10),
"cigs_per_day_before": 15,
"cig_pack_price_cents": 1250,
"cigs_per_pack": 20,
"default_nicotine_mg_ml": 6,
}
payload.update(overrides)
return payload
def _put_settings(client: TestClient, headers: dict[str, str], **overrides: Any) -> Any:
response = client.put(
f"{BASE}/settings", json=_settings_payload(**overrides), headers=headers
)
assert response.status_code == 200, response.text
return response.json()
def _create_product(
client: TestClient, headers: dict[str, str], **payload: Any
) -> dict[str, Any]:
response = client.post(f"{BASE}/products", json=payload, headers=headers)
assert response.status_code == 201, response.text
return response.json()
def _catalog(client: TestClient, headers: dict[str, str]) -> dict[str, dict[str, Any]]:
"""Base 1 L / booster 10 ml 20 mg / aroma 30 ml / coil box of 5."""
return {
"base": _create_product(
client,
headers,
kind="base",
name="Base 50/50 1 L",
price_cents=1200,
size_value=1000,
size_unit="ml",
vg_pct=50,
),
"booster": _create_product(
client,
headers,
kind="booster",
name="Booster 20 mg",
price_cents=90,
size_value=10,
size_unit="ml",
nicotine_mg_ml=20,
vg_pct=50,
),
"aroma": _create_product(
client,
headers,
kind="aroma",
name="Arôme fraise",
price_cents=600,
size_value=30,
size_unit="ml",
vg_pct=0,
),
"coil": _create_product(
client,
headers,
kind="coil",
name="GT Mesh 0,6 Ω",
price_cents=1950,
size_value=5,
size_unit="unit",
ohm="0.6",
),
}
def _create_mix(
client: TestClient, headers: dict[str, str], catalog: dict[str, dict[str, Any]]
) -> dict[str, Any]:
response = client.post(
f"{BASE}/mixes",
json={
"name": "Fraise 6 mg 50/50",
"total_ml": 260,
"target_nicotine_mg_ml": 6,
"components": [
{"product_id": catalog["base"]["id"], "quantity": 156},
{"product_id": catalog["booster"]["id"], "quantity": 78},
{"product_id": catalog["aroma"]["id"], "quantity": 26},
],
},
headers=headers,
)
assert response.status_code == 201, response.text
return response.json()
# --- Settings ---------------------------------------------------------------
def test_settings_404_then_upsert(
client: TestClient, auth_headers: dict[str, str]
) -> None:
missing = client.get(f"{BASE}/settings", headers=auth_headers)
assert missing.status_code == 404
error = missing.json()["error"]
assert error["code"] == "not_found"
assert error["message"] == "Paramètres vape non configurés."
created = _put_settings(client, auth_headers)
assert created["cig_pack_price_cents"] == 1250
assert created["cigs_per_pack"] == 20
updated = _put_settings(client, auth_headers, cig_pack_price_cents=1300)
assert updated["id"] == created["id"] # upsert, not a second row
assert updated["cig_pack_price_cents"] == 1300
read = client.get(f"{BASE}/settings", headers=auth_headers)
assert read.status_code == 200
assert read.json()["cig_pack_price_cents"] == 1300
def test_endpoints_require_authentication(client: TestClient) -> None:
for path in ("/settings", "/products", "/dashboard", "/milestones"):
assert client.get(f"{BASE}{path}").status_code == 401
# --- Products ---------------------------------------------------------------
def test_products_crud_pagination_and_conflicts(
client: TestClient, auth_headers: dict[str, str]
) -> None:
catalog = _catalog(client, auth_headers)
assert catalog["coil"]["unit_price_cents"] == 390.0 # 19,50 € / 5
listing = client.get(f"{BASE}/products", headers=auth_headers).json()
assert listing["total"] == 4
assert {"items", "total", "page", "page_size"} <= set(listing)
filtered = client.get(f"{BASE}/products?kind=coil", headers=auth_headers).json()
assert [item["kind"] for item in filtered["items"]] == ["coil"]
duplicate = client.post(
f"{BASE}/products",
json={
"kind": "base",
"name": "Base 50/50 1 L",
"price_cents": 1200,
"size_value": 1000,
"size_unit": "ml",
},
headers=auth_headers,
)
assert duplicate.status_code == 409
booster_without_rate = client.post(
f"{BASE}/products",
json={
"kind": "booster",
"name": "Booster sans taux",
"price_cents": 90,
"size_value": 10,
"size_unit": "ml",
},
headers=auth_headers,
)
assert booster_without_rate.status_code == 422
patched = client.patch(
f"{BASE}/products/{catalog['coil']['id']}",
json={"price_cents": 2000, "is_archived": True},
headers=auth_headers,
)
assert patched.status_code == 200
assert patched.json()["unit_price_cents"] == 400.0
hidden = client.get(f"{BASE}/products", headers=auth_headers).json()
assert hidden["total"] == 3
shown = client.get(
f"{BASE}/products?include_archived=true", headers=auth_headers
).json()
assert shown["total"] == 4
unknown_sort = client.get(f"{BASE}/products?sort=secret", headers=auth_headers)
assert unknown_sort.status_code == 422
assert (
client.delete(
f"{BASE}/products/{catalog['aroma']['id']}", headers=auth_headers
).status_code
== 204
)
def test_delete_referenced_product_conflicts(
client: TestClient, auth_headers: dict[str, str]
) -> None:
catalog = _catalog(client, auth_headers)
_create_mix(client, auth_headers, catalog)
response = client.delete(
f"{BASE}/products/{catalog['base']['id']}", headers=auth_headers
)
assert response.status_code == 409
assert "archivez" in response.json()["error"]["message"]
# --- Mixes ------------------------------------------------------------------
def test_mix_cost_nicotine_check_and_activation(
client: TestClient, auth_headers: dict[str, str]
) -> None:
catalog = _catalog(client, auth_headers)
mix = _create_mix(client, auth_headers, catalog)
# 156×1,2 + 78×9 + 26×20 = 1409,2 cents over 260 ml
assert mix["cost_total_cents"] == 1409.2
assert mix["cost_per_ml_cents"] == 5.42
assert mix["nicotine_check_mg_ml"] == 6.0
assert mix["warning"] is None
assert mix["is_active"] is False
assert len(mix["components"]) == 3
assert mix["components"][1]["cost_cents"] == 702.0
activated = client.post(f"{BASE}/mixes/{mix['id']}/activate", headers=auth_headers)
assert activated.status_code == 200
assert activated.json()["is_active"] is True
second = _create_mix(client, auth_headers, catalog)
client.patch(
f"{BASE}/mixes/{second['id']}",
json={"name": "Fraise 3 mg", "target_nicotine_mg_ml": 3},
headers=auth_headers,
)
activated_second = client.post(
f"{BASE}/mixes/{second['id']}/activate", headers=auth_headers
).json()
assert activated_second["is_active"] is True
assert activated_second["warning"] == "nicotine_mismatch" # 6 mg/ml for a 3 target
listing = client.get(f"{BASE}/mixes", headers=auth_headers).json()
assert [item["is_active"] for item in listing["items"]].count(True) == 1
def test_archiving_the_active_mix_deactivates_it(
client: TestClient, auth_headers: dict[str, str]
) -> None:
"""An archived recipe is hidden from the lists: it must stop being active,
otherwise it keeps driving cost_per_ml while the UI shows no active mix."""
_put_settings(client, auth_headers)
catalog = _catalog(client, auth_headers)
mix = _create_mix(client, auth_headers, catalog)
activated = client.post(f"{BASE}/mixes/{mix['id']}/activate", headers=auth_headers)
assert activated.json()["is_active"] is True
archived = client.patch(
f"{BASE}/mixes/{mix['id']}", json={"is_archived": True}, headers=auth_headers
)
assert archived.status_code == 200, archived.text
assert archived.json()["is_archived"] is True
assert archived.json()["is_active"] is False
listing = client.get(f"{BASE}/mixes?include_archived=true", headers=auth_headers)
assert [item["is_active"] for item in listing.json()["items"]].count(True) == 0
dashboard = client.get(f"{BASE}/dashboard", headers=auth_headers).json()
assert dashboard["cost_per_ml_cents"] is None # no active mix, no purchase
def test_mix_component_replacement_and_validation(
client: TestClient, auth_headers: dict[str, str]
) -> None:
catalog = _catalog(client, auth_headers)
mix = _create_mix(client, auth_headers, catalog)
replaced = client.patch(
f"{BASE}/mixes/{mix['id']}",
json={
"components": [
{"product_id": catalog["base"]["id"], "quantity": 100},
{"product_id": catalog["booster"]["id"], "quantity": 30},
]
},
headers=auth_headers,
)
assert replaced.status_code == 200
assert len(replaced.json()["components"]) == 2
hardware_component = client.patch(
f"{BASE}/mixes/{mix['id']}",
json={"components": [{"product_id": catalog["coil"]["id"], "quantity": 1}]},
headers=auth_headers,
)
assert hardware_component.status_code == 422
twice = client.patch(
f"{BASE}/mixes/{mix['id']}",
json={
"components": [
{"product_id": catalog["base"]["id"], "quantity": 10},
{"product_id": catalog["base"]["id"], "quantity": 20},
]
},
headers=auth_headers,
)
assert twice.status_code == 422
assert (
client.delete(f"{BASE}/mixes/{mix['id']}", headers=auth_headers).status_code
== 204
)
assert client.get(f"{BASE}/mixes", headers=auth_headers).json()["total"] == 0
def test_mix_calculator_is_stateless(
client: TestClient, auth_headers: dict[str, str]
) -> None:
catalog = _catalog(client, auth_headers)
response = client.post(
f"{BASE}/mixes/calculator",
json={
"total_ml": 260,
"target_nicotine_mg_ml": 6,
"booster_product_id": catalog["booster"]["id"],
"base_product_id": catalog["base"]["id"],
"aroma_pct": 10,
"aroma_product_id": catalog["aroma"]["id"],
},
headers=auth_headers,
)
assert response.status_code == 200, response.text
body = response.json()
assert body["booster_ml"] == 78.0
assert body["aroma_ml"] == 26.0
assert body["base_ml"] == 156.0
assert body["nicotine_check_mg_ml"] == 6.0
assert body["cost_per_ml_cents"] == 5.42
assert client.get(f"{BASE}/mixes", headers=auth_headers).json()["total"] == 0
impossible = client.post(
f"{BASE}/mixes/calculator",
json={
"total_ml": 100,
"target_nicotine_mg_ml": 18,
"booster_product_id": catalog["booster"]["id"],
"base_product_id": catalog["base"]["id"],
"aroma_pct": 20,
},
headers=auth_headers,
)
assert impossible.status_code == 422
# --- Liquid entries ---------------------------------------------------------
def test_daily_total_overrides_refills_and_is_unique(
client: TestClient, auth_headers: dict[str, str]
) -> None:
_put_settings(client, auth_headers)
for ml in (4, 3):
created = client.post(
f"{BASE}/liquids",
json={"entry_date": _day(-1), "ml": ml, "kind": "refill"},
headers=auth_headers,
)
assert created.status_code == 201, created.text
assert created.json()["nicotine_effective_mg_ml"] == 6.0
stats = client.get(
f"{BASE}/stats/consumption?from={_day(-1)}&to={_day(-1)}", headers=auth_headers
).json()
assert stats["series"][0]["points"] == [[_day(-1), 7.0]]
total = client.post(
f"{BASE}/liquids",
json={"entry_date": _day(-1), "ml": 9, "kind": "daily_total"},
headers=auth_headers,
)
assert total.status_code == 201
overridden = client.get(
f"{BASE}/stats/consumption?from={_day(-1)}&to={_day(-1)}", headers=auth_headers
).json()
assert overridden["series"][0]["points"] == [[_day(-1), 9.0]]
assert overridden["meta"]["tracked_days_ratio"] == 1.0
duplicate = client.post(
f"{BASE}/liquids",
json={"entry_date": _day(-1), "ml": 5, "kind": "daily_total"},
headers=auth_headers,
)
assert duplicate.status_code == 409
assert "total quotidien" in duplicate.json()["error"]["message"]
# the same day of another date is fine
other_day = client.post(
f"{BASE}/liquids",
json={"entry_date": _day(-2), "ml": 5, "kind": "daily_total"},
headers=auth_headers,
)
assert other_day.status_code == 201
listing = client.get(
f"{BASE}/liquids?kind=daily_total", headers=auth_headers
).json()
assert listing["total"] == 2
patched = client.patch(
f"{BASE}/liquids/{total.json()['id']}",
json={"ml": 11, "nicotine_mg_ml": 3},
headers=auth_headers,
)
assert patched.status_code == 200
assert patched.json()["nicotine_mg"] == 33.0
assert (
client.delete(
f"{BASE}/liquids/{total.json()['id']}", headers=auth_headers
).status_code
== 204
)
assert (
client.get(
f"{BASE}/liquids/{total.json()['id']}", headers=auth_headers
).status_code
== 405
)
def test_liquid_entry_inherits_the_active_mix(
client: TestClient, auth_headers: dict[str, str]
) -> None:
_put_settings(client, auth_headers, default_nicotine_mg_ml=12)
catalog = _catalog(client, auth_headers)
mix = _create_mix(client, auth_headers, catalog)
client.post(f"{BASE}/mixes/{mix['id']}/activate", headers=auth_headers)
entry = client.post(
f"{BASE}/liquids",
json={"entry_date": _day(0), "ml": 4},
headers=auth_headers,
).json()
assert entry["mix_id"] == mix["id"]
assert entry["nicotine_effective_mg_ml"] == 6.0 # mix target beats the default
assert entry["nicotine_mg"] == 24.0
# --- Coils ------------------------------------------------------------------
def test_one_click_coil_change_reports_previous_lifespan(
client: TestClient, auth_headers: dict[str, str]
) -> None:
catalog = _catalog(client, auth_headers)
first = client.post(
f"{BASE}/coils",
json={
"changed_at": (datetime.now(UTC) - timedelta(days=20)).isoformat(),
"product_id": catalog["coil"]["id"],
},
headers=auth_headers,
)
assert first.status_code == 201, first.text
assert first.json()["previous_lifespan_days"] is None
assert first.json()["is_current"] is True
quick = client.post(f"{BASE}/coils/change", headers=auth_headers)
assert quick.status_code == 201, quick.text
body = quick.json()
assert body["previous_lifespan_days"] == pytest.approx(20.0, abs=0.01)
assert body["is_current"] is True
listing = client.get(f"{BASE}/coils", headers=auth_headers).json()
assert listing["total"] == 2
finished = next(item for item in listing["items"] if not item["is_current"])
assert finished["lifespan_days"] == pytest.approx(20.0, abs=0.01)
assert finished["product_name"] == "GT Mesh 0,6 Ω"
def test_coil_stats_average_and_amortization(
client: TestClient, auth_headers: dict[str, str]
) -> None:
catalog = _catalog(client, auth_headers)
for offset in (30, 20, 10):
client.post(
f"{BASE}/coils",
json={
"changed_at": (datetime.now(UTC) - timedelta(days=offset)).isoformat(),
"product_id": catalog["coil"]["id"],
},
headers=auth_headers,
)
client.post(
f"{BASE}/liquids",
json={"entry_date": _day(-15), "ml": 6},
headers=auth_headers,
)
stats = client.get(f"{BASE}/coils/stats", headers=auth_headers).json()
assert stats["unit"] == "days"
assert stats["meta"]["avg_lifespan_days"] == pytest.approx(10.0, abs=0.01)
assert stats["meta"]["avg_lifespan_is_default"] is False
assert stats["meta"]["coil_unit_price_cents"] == 390.0
assert stats["meta"]["coil_cost_per_day_cents"] == pytest.approx(39.0, abs=0.05)
assert stats["meta"]["current_coil_age_days"] == pytest.approx(10.0, abs=0.01)
lifespans = stats["series"][0]["points"]
assert len(lifespans) == 3
# the 6 ml logged 15 days ago belong to the coil installed 20 days ago
ml_through = dict(stats["series"][1]["points"])
assert (
ml_through[(datetime.now(UTC) - timedelta(days=20)).date().isoformat()] == 6.0
)
def test_avg_ml_through_coil_uses_completed_cycles_only(
client: TestClient, auth_headers: dict[str, str]
) -> None:
"""§7.3: the running coil (partially used) must not drag the average down,
and only the last COIL_AVG_LAST_N finished cycles are averaged."""
catalog = _catalog(client, auth_headers)
for offset in (20, 10):
client.post(
f"{BASE}/coils",
json={
"changed_at": (datetime.now(UTC) - timedelta(days=offset)).isoformat(),
"product_id": catalog["coil"]["id"],
},
headers=auth_headers,
)
# 8 ml on the finished cycle, 2 ml on the coil currently in use
for day, ml in ((-15, 8), (-5, 2)):
created = client.post(
f"{BASE}/liquids",
json={"entry_date": _day(day), "ml": ml},
headers=auth_headers,
)
assert created.status_code == 201, created.text
stats = client.get(f"{BASE}/coils/stats", headers=auth_headers).json()
ml_points = dict(stats["series"][1]["points"])
assert ml_points[(datetime.now(UTC) - timedelta(days=20)).date().isoformat()] == 8.0
assert ml_points[(datetime.now(UTC) - timedelta(days=10)).date().isoformat()] == 2.0
assert stats["meta"]["avg_ml_through_coil"] == 8.0 # not (8 + 2) / 2 = 5.0
def test_coil_changed_at_is_stored_in_utc_and_filterable(
client: TestClient, auth_headers: dict[str, str]
) -> None:
local_instant = datetime.now(PARIS) - timedelta(days=2)
created = client.post(
f"{BASE}/coils",
json={"changed_at": local_instant.isoformat()},
headers=auth_headers,
)
assert created.status_code == 201, created.text
returned = datetime.fromisoformat(created.json()["changed_at"])
assert returned.utcoffset() == timedelta(0) # always UTC on the wire
assert returned == local_instant
day = local_instant.date().isoformat()
listing = client.get(f"{BASE}/coils?from={day}&to={day}", headers=auth_headers)
assert listing.json()["total"] == 1
empty = client.get(
f"{BASE}/coils?from={_day(-1)}&to={_day(0)}", headers=auth_headers
)
assert empty.json()["total"] == 0
def test_coil_stats_without_any_change_uses_the_default_lifespan(
client: TestClient, auth_headers: dict[str, str]
) -> None:
stats = client.get(f"{BASE}/coils/stats", headers=auth_headers).json()
assert stats["meta"]["avg_lifespan_days"] == 14.0
assert stats["meta"]["avg_lifespan_is_default"] is True
assert stats["meta"]["current_coil_age_days"] is None
assert stats["series"][0]["points"] == []
# --- Purchases & savings ----------------------------------------------------
def test_purchases_crud_and_totals(
client: TestClient, auth_headers: dict[str, str]
) -> None:
catalog = _catalog(client, auth_headers)
created = client.post(
f"{BASE}/purchases",
json={
"purchased_on": _day(-5),
"product_id": catalog["coil"]["id"],
"qty": 2,
},
headers=auth_headers,
)
assert created.status_code == 201, created.text
assert created.json()["total_cents"] == 3900.0 # 2 × 19,50 € catalog price
overridden = client.patch(
f"{BASE}/purchases/{created.json()['id']}",
json={"unit_price_cents": 1500},
headers=auth_headers,
)
assert overridden.json()["total_cents"] == 3000.0
listing = client.get(
f"{BASE}/purchases?from={_day(-6)}&to={_day(-4)}", headers=auth_headers
).json()
assert listing["total"] == 1
assert listing["items"][0]["product_name"] == "GT Mesh 0,6 Ω"
assert (
client.delete(
f"{BASE}/purchases/{created.json()['id']}", headers=auth_headers
).status_code
== 204
)
def test_savings_theoretical_and_real(
client: TestClient, auth_headers: dict[str, str]
) -> None:
_put_settings(client, auth_headers, quit_date=_day(-4))
catalog = _catalog(client, auth_headers)
mix = _create_mix(client, auth_headers, catalog)
client.post(f"{BASE}/mixes/{mix['id']}/activate", headers=auth_headers)
for offset in (-4, -3, -2, -1, 0):
client.post(
f"{BASE}/liquids",
json={"entry_date": _day(offset), "ml": 4, "kind": "daily_total"},
headers=auth_headers,
)
client.post(
f"{BASE}/purchases",
json={"purchased_on": _day(-4), "product_id": catalog["base"]["id"], "qty": 1},
headers=auth_headers,
)
stats = client.get(f"{BASE}/stats/savings", headers=auth_headers).json()
assert stats["unit"] == "cents"
assert stats["from"] == _day(-4)
assert stats["to"] == _day(0)
names = [series["name"] for series in stats["series"]]
assert names == ["savings_theoretical", "savings_real"]
assert len(stats["series"][0]["points"]) == 5
meta = stats["meta"]
# 15 cig/day ÷ 20 × 12,50 € = 937,5 cents/day, frozen since the quit date
assert meta["cig_cost_per_day_cents"] == 937.5
assert meta["days_since_quit"] == 4
assert meta["cigarettes_avoided"] == 60
assert meta["packs_avoided"] == 3.0
assert meta["time_regained_minutes"] == 660
assert meta["has_purchases"] is True
# daily vape cost = 4 ml × 5,42 c + coil amortization (no coil -> 0)
theoretical_points = stats["series"][0]["points"]
assert theoretical_points[0][1] == pytest.approx(-21.68, abs=0.01)
assert theoretical_points[4][1] == pytest.approx(937.5 * 4 - 5 * 21.68, abs=0.05)
real_points = stats["series"][1]["points"]
assert real_points[0][1] == pytest.approx(-1200.0, abs=0.01)
assert real_points[4][1] == pytest.approx(937.5 * 4 - 1200.0, abs=0.05)
def test_savings_requires_settings(
client: TestClient, auth_headers: dict[str, str]
) -> None:
for path in ("/stats/savings", "/milestones", "/dashboard"):
response = client.get(f"{BASE}{path}", headers=auth_headers)
assert response.status_code == 404
assert response.json()["error"]["message"] == "Paramètres vape non configurés."
# --- Stats contract ---------------------------------------------------------
def test_stats_endpoints_follow_the_series_contract(
client: TestClient, auth_headers: dict[str, str]
) -> None:
_put_settings(client, auth_headers)
catalog = _catalog(client, auth_headers)
mix = _create_mix(client, auth_headers, catalog)
client.post(f"{BASE}/mixes/{mix['id']}/activate", headers=auth_headers)
client.post(
f"{BASE}/liquids",
json={"entry_date": _day(-1), "ml": 4, "kind": "daily_total"},
headers=auth_headers,
)
for path in (
"/stats/consumption",
"/stats/nicotine",
"/stats/costs",
"/stats/savings",
"/coils/stats",
):
body = client.get(f"{BASE}{path}", headers=auth_headers).json()
assert set(body) == {"from", "to", "unit", "series", "meta"}, path
for series in body["series"]:
assert set(series) == {"name", "type", "points"}
for point in series["points"]:
assert len(point) == 2
assert isinstance(point[0], str)
assert point[1] is None or isinstance(point[1], (int, float))
consumption = client.get(
f"{BASE}/stats/consumption?from={_day(-6)}&to={_day(0)}", headers=auth_headers
).json()
assert len(consumption["series"][0]["points"]) == 7
assert consumption["meta"]["ml_per_day_7"] == 4.0
assert consumption["meta"]["tracked_days_ratio"] == pytest.approx(1 / 7, abs=0.001)
assert consumption["series"][1]["name"] == "ml_ma7"
nicotine = client.get(
f"{BASE}/stats/nicotine?from={_day(-1)}&to={_day(-1)}", headers=auth_headers
).json()
assert nicotine["unit"] == "mg"
assert nicotine["series"][0]["points"] == [[_day(-1), 24.0]] # 4 ml × 6 mg/ml
assert nicotine["meta"]["cig_equivalent_per_day"] == 2.0
assert nicotine["meta"]["trend_status"] in {"down", "stable", "up"}
costs = client.get(f"{BASE}/stats/costs", headers=auth_headers).json()
assert costs["unit"] == "cents"
assert costs["meta"]["cost_per_ml_cents"] == 5.42
assert costs["meta"]["cost_per_ml_source"] == "active_mix"
assert costs["meta"]["coil_cost_per_day_cents"] == 0.0
assert [series["name"] for series in costs["series"]] == [
"theoretical_cost",
"real_spend",
]
invalid = client.get(
f"{BASE}/stats/consumption?from={_day(0)}&to={_day(-5)}", headers=auth_headers
)
assert invalid.status_code == 422
def test_theoretical_cost_is_not_imputed_before_the_quit_date(
client: TestClient, auth_headers: dict[str, str]
) -> None:
"""Mean-imputation fills tracking gaps — it must not invent a vape cost for
the months the user was still smoking (nor for the future)."""
_put_settings(client, auth_headers, quit_date=_day(-3))
catalog = _catalog(client, auth_headers)
mix = _create_mix(client, auth_headers, catalog)
client.post(f"{BASE}/mixes/{mix['id']}/activate", headers=auth_headers)
client.post(
f"{BASE}/liquids",
json={"entry_date": _day(-1), "ml": 4, "kind": "daily_total"},
headers=auth_headers,
)
# A window that starts well before the quit date and ends in the future
costs = client.get(
f"{BASE}/stats/costs?from={_day(-40)}&to={_day(30)}", headers=auth_headers
).json()
theoretical = dict(costs["series"][0]["points"])
total = sum(value or 0.0 for value in theoretical.values())
# 4 days from the quit date to today: 1 tracked (4 ml) + 3 imputed at 4 ml
assert total == pytest.approx(4 * 4 * 5.42, abs=0.05)
def test_savings_do_not_project_into_the_future(
client: TestClient, auth_headers: dict[str, str]
) -> None:
_put_settings(client, auth_headers, quit_date=_day(-2))
body = client.get(
f"{BASE}/stats/savings?to={_day(60)}", headers=auth_headers
).json()
assert body["to"] == _day(0)
assert body["series"][0]["points"][-1][0] == _day(0)
# 15 cig/day ÷ 20 × 12,50 € × 2 days, nothing vaped yet
assert body["meta"]["savings_theoretical_cents"] == pytest.approx(937.5 * 2)
def test_cost_per_ml_falls_back_to_purchases(
client: TestClient, auth_headers: dict[str, str]
) -> None:
_put_settings(client, auth_headers)
catalog = _catalog(client, auth_headers)
client.post(
f"{BASE}/purchases",
json={"purchased_on": _day(-3), "product_id": catalog["base"]["id"], "qty": 1},
headers=auth_headers,
)
costs = client.get(f"{BASE}/stats/costs", headers=auth_headers).json()
# 12,00 € for 1 000 ml -> 1,2 cent/ml, no active mix
assert costs["meta"]["cost_per_ml_cents"] == 1.2
assert costs["meta"]["cost_per_ml_source"] == "purchases"
# --- Milestones & dashboard -------------------------------------------------
def test_milestones_timeline(client: TestClient, auth_headers: dict[str, str]) -> None:
_put_settings(client, auth_headers, quit_date=_day(-20))
body = client.get(f"{BASE}/milestones", headers=auth_headers).json()
assert len(body) == 12
assert set(body[0]) == {
"code",
"label_fr",
"reached_at",
"achieved",
"progress_pct",
}
by_code = {item["code"]: item for item in body}
assert by_code["hr_bp_normal"]["achieved"] is True
assert by_code["circulation"]["achieved"] is True
assert by_code["lung_function"]["achieved"] is False
assert by_code["chd_risk_normal"]["label_fr"].startswith("Risque coronarien")
assert 0 < by_code["lung_function"]["progress_pct"] < 100
def test_dashboard_aggregates_the_home_kpis(
client: TestClient, auth_headers: dict[str, str]
) -> None:
_put_settings(client, auth_headers, quit_date=_day(-10))
catalog = _catalog(client, auth_headers)
mix = _create_mix(client, auth_headers, catalog)
client.post(f"{BASE}/mixes/{mix['id']}/activate", headers=auth_headers)
client.post(
f"{BASE}/liquids",
json={"entry_date": _day(0), "ml": 4, "kind": "daily_total"},
headers=auth_headers,
)
client.post(
f"{BASE}/coils",
json={
"changed_at": (datetime.now(UTC) - timedelta(days=6)).isoformat(),
"product_id": catalog["coil"]["id"],
},
headers=auth_headers,
)
body = client.get(f"{BASE}/dashboard", headers=auth_headers).json()
assert body["quit_date"] == _day(-10)
assert body["days_since_quit"] == 10
assert body["ml_today"] == 4.0
assert body["nicotine_today_mg"] == 24.0
assert body["cost_per_ml_cents"] == 5.42
assert body["cigarettes_avoided"] == 150
assert body["packs_avoided"] == 7.5
assert body["cig_cost_per_day_cents"] == 937.5
assert body["coil_avg_lifespan_days"] == 14.0 # default: a single change so far
assert body["coil_lifespan_is_default"] is True
assert body["coil_cost_per_day_cents"] == pytest.approx(390 / 14, abs=0.01)
assert body["current_coil_age_days"] == pytest.approx(6.0, abs=0.01)
assert body["has_purchases"] is False
assert body["savings_display_cents"] == body["savings_theoretical_cents"]
assert body["savings_theoretical_cents"] > 0
assert body["next_milestone"]["code"] == "circulation" # 14 days, quit 10 days ago
# --- Multi-user isolation ---------------------------------------------------
def test_user_isolation(
client: TestClient,
db: Session,
auth_headers: dict[str, str],
make_auth_headers: Callable[[int], dict[str, str]],
) -> None:
_put_settings(client, auth_headers)
catalog = _catalog(client, auth_headers)
intruder = User(
email="autre@lifetrack.local",
password_hash="x" * 20,
display_name="Autre",
)
db.add(intruder)
db.commit()
other_headers = make_auth_headers(intruder.id)
assert client.get(f"{BASE}/settings", headers=other_headers).status_code == 404
assert client.get(f"{BASE}/products", headers=other_headers).json()["total"] == 0
assert (
client.get(
f"{BASE}/products/{catalog['base']['id']}", headers=other_headers
).status_code
== 405 # no item GET route: the module never exposes another user's row
)
assert (
client.patch(
f"{BASE}/products/{catalog['base']['id']}",
json={"name": "Volé"},
headers=other_headers,
).status_code
== 404
)
assert (
client.delete(
f"{BASE}/products/{catalog['base']['id']}", headers=other_headers
).status_code
== 404
)
@@ -0,0 +1,316 @@
"""Unit tests of the pure vape calculations (datamodel-health-vape.md §7)."""
from datetime import UTC, date, datetime, timedelta
from decimal import Decimal
from zoneinfo import ZoneInfo
import pytest
from app.modules.vape import calculations as calc
PARIS = ZoneInfo("Europe/Paris")
def _component(
quantity: str, price_cents: int, size_value: str, **kwargs: object
) -> calc.ComponentInput:
return calc.ComponentInput(
quantity=Decimal(quantity),
price_cents=price_cents,
size_value=Decimal(size_value),
**kwargs, # type: ignore[arg-type]
)
# --- Recipe costing (§6.3, §7.1) -------------------------------------------
def _recipe() -> list[calc.ComponentInput]:
"""260 ml at 6 mg/ml: 156 ml base + 78 ml booster (20 mg/ml) + 26 ml aroma."""
return [
# 1 L base bottle at 12,00 € -> 1.2 cents/ml
_component("156", 1200, "1000", vg_pct=Decimal(50)),
# 10 ml booster at 0,90 € -> 9 cents/ml
_component("78", 90, "10", nicotine_mg_ml=Decimal(20), vg_pct=Decimal(50)),
# 30 ml aroma at 6,00 € -> 20 cents/ml
_component("26", 600, "30", vg_pct=Decimal(0)),
]
def test_mix_cost_total_and_per_ml_are_exact() -> None:
components = _recipe()
# 156×1.2 + 78×9 + 26×20 = 187.2 + 702 + 520 = 1409.2 cents
assert calc.mix_cost_total_cents(components) == Decimal("1409.2")
per_ml = calc.mix_cost_per_ml_cents(components, Decimal(260))
assert per_ml == Decimal("1409.2") / Decimal(260)
assert per_ml is not None
assert round(float(per_ml), 4) == 5.4200
def test_mix_cost_per_ml_is_none_without_volume() -> None:
assert calc.mix_cost_per_ml_cents(_recipe(), Decimal(0)) is None
def test_nicotine_check_matches_target_and_warns_when_off() -> None:
components = _recipe()
check = calc.mix_nicotine_check_mg_ml(components, Decimal(260))
assert check == Decimal(6) # 78 ml × 20 mg/ml / 260 ml
assert calc.nicotine_mismatch_warning(Decimal(6), check) is None
assert calc.nicotine_mismatch_warning(Decimal(3), check) == "nicotine_mismatch"
# 10 % tolerance boundary: 6.6 for a 6 mg/ml target is still accepted
assert calc.nicotine_mismatch_warning(Decimal(6), Decimal("6.6")) is None
assert (
calc.nicotine_mismatch_warning(Decimal(6), Decimal("6.7"))
== "nicotine_mismatch"
)
def test_mix_vg_pct_needs_every_component() -> None:
components = _recipe()
vg = calc.mix_vg_pct(components, Decimal(260))
assert vg == (Decimal(156) * 50 + Decimal(78) * 50) / Decimal(260)
partial = [*components[:2], _component("26", 600, "30")]
assert calc.mix_vg_pct(partial, Decimal(260)) is None
def test_recipe_quantities_assistant() -> None:
result = calc.recipe_quantities(Decimal(260), Decimal(6), Decimal(20), Decimal(10))
assert result.booster_ml == Decimal(78)
assert result.aroma_ml == Decimal(26)
assert result.base_ml == Decimal(156)
def test_recipe_quantities_rejects_impossible_recipes() -> None:
with pytest.raises(ValueError, match="base volume"):
calc.recipe_quantities(Decimal(100), Decimal(18), Decimal(20), Decimal(20))
with pytest.raises(ValueError, match="booster nicotine"):
calc.recipe_quantities(Decimal(100), Decimal(6), Decimal(0), Decimal(10))
def test_fallback_cost_per_ml_is_volume_weighted() -> None:
pairs = [(Decimal(1200), Decimal(1000)), (Decimal(540), Decimal(60))]
assert calc.fallback_cost_per_ml_cents(pairs) == Decimal(1740) / Decimal(1060)
assert calc.fallback_cost_per_ml_cents([]) is None
# --- Daily consumption (§6.4, §7.2) ----------------------------------------
def test_daily_total_overrides_refills() -> None:
entries = [
("refill", Decimal(4)),
("refill", Decimal(3)),
("daily_total", Decimal(9)),
]
assert calc.effective_daily_ml(entries) == Decimal(9)
def test_refills_are_summed_and_untracked_day_is_none() -> None:
entries = [("refill", Decimal(4)), ("refill", Decimal("3.5"))]
assert calc.effective_daily_ml(entries) == Decimal("7.5")
assert calc.effective_daily_ml([]) is None
def test_mean_and_tracked_ratio_ignore_untracked_days() -> None:
values = [Decimal(4), None, Decimal(6), None]
assert calc.mean_ml_per_day(values) == Decimal(5)
assert calc.tracked_days_ratio(values) == 0.5
assert calc.mean_ml_per_day([None, None]) is None
def test_sum_ml_between_excludes_the_end_day() -> None:
daily = {
date(2026, 8, 1): Decimal(4),
date(2026, 8, 2): Decimal(5),
date(2026, 8, 3): Decimal(6),
}
assert calc.sum_ml_between(daily, date(2026, 8, 1), date(2026, 8, 3)) == Decimal(9)
assert calc.sum_ml_between(daily, date(2026, 9, 1), date(2026, 9, 5)) is None
# --- Coils (§7.3) ----------------------------------------------------------
def _dt(day: int) -> datetime:
return datetime(2026, 1, day, 12, 0, tzinfo=UTC)
def test_coil_intervals_and_average_of_last_five_cycles() -> None:
# changes every 10, 12, 14, 16, 18 and 20 days
starts = [0, 10, 22, 36, 52, 70, 90]
changed = [_dt(1) + timedelta(days=offset) for offset in starts]
intervals = calc.coil_intervals_days(changed)
assert intervals == [10.0, 12.0, 14.0, 16.0, 18.0, 20.0]
avg, is_default = calc.avg_coil_lifespan_days(intervals)
assert avg == 16.0 # mean of the last five: 12, 14, 16, 18, 20
assert is_default is False
def test_coil_average_falls_back_to_default_without_two_changes() -> None:
avg, is_default = calc.avg_coil_lifespan_days([])
assert avg == calc.DEFAULT_COIL_LIFESPAN_DAYS == 14.0
assert is_default is True
def test_coil_cost_per_day_amortization() -> None:
# 19,50 € box of 5 coils -> 390 cents each, 16-day average lifespan
assert calc.coil_cost_per_day_cents(Decimal(390), 16.0) == 24.375
assert calc.coil_cost_per_day_cents(None, 16.0) == 0.0
assert calc.coil_cost_per_day_cents(390, 0.0) == 0.0
# --- Daily cost and cigarette baseline (§7.4, §7.5) ------------------------
def test_vape_cost_per_day_combines_liquid_and_coil() -> None:
assert calc.vape_cost_per_day_cents(Decimal(4), Decimal(5), 24.375) == 44.375
assert calc.vape_cost_per_day_cents(Decimal(4), None, 24.375) is None
assert calc.vape_cost_per_day_cents(None, Decimal(5), 24.375) is None
def test_cig_baseline_per_day() -> None:
# 15 cig/day, 20 per pack, 12,50 € pack -> 9,375 €/day
assert calc.cig_cost_per_day_cents(Decimal(15), 20, 1250) == 937.5
assert calc.savings_per_day_cents(937.5, 44.375) == 893.125
assert calc.savings_per_day_cents(937.5, None) is None
def test_theoretical_cost_imputes_untracked_days() -> None:
tracked = calc.theoretical_vape_cost_cents(Decimal(4), Decimal(5), Decimal(5), 10.0)
assert tracked == 30.0
imputed = calc.theoretical_vape_cost_cents(None, Decimal(5), Decimal(5), 10.0)
assert imputed == 35.0 # mean-imputed 5 ml × 5 cents + 10 cents of coil
def test_cumulative_theoretical_savings_with_imputation() -> None:
quit_date = date(2026, 1, 1)
today = date(2026, 1, 5)
daily = {date(2026, 1, 1): Decimal(4), date(2026, 1, 3): Decimal(6)}
series = calc.cumulative_savings_theoretical(
quit_date,
today,
daily,
imputed_ml_per_day=Decimal(5),
cost_per_ml_cents=Decimal(5),
coil_cpd_cents=0.0,
cig_cpd_cents=100.0,
)
# vape costs: 20, 25, 30, 25, 25 (untracked days imputed at 5 ml)
assert [round(value, 3) for _, value in series] == [
-20.0,
55.0,
125.0,
200.0,
275.0,
]
assert [day for day, _ in series] == [date(2026, 1, d) for d in range(1, 6)]
def test_cumulative_real_savings_uses_purchases() -> None:
quit_date = date(2026, 1, 1)
spend = {date(2026, 1, 2): 300.0}
series = calc.cumulative_savings_real(
quit_date, date(2026, 1, 4), spend, cig_cpd_cents=100.0
)
assert [round(value, 3) for _, value in series] == [0.0, -200.0, -100.0, 0.0]
def test_savings_series_is_empty_before_the_quit_date() -> None:
assert (
calc.cumulative_savings_real(date(2026, 2, 1), date(2026, 1, 1), {}, 100.0)
== []
)
def test_real_cost_per_day() -> None:
assert calc.real_cost_per_day_cents(3000.0, 30) == 100.0
assert calc.real_cost_per_day_cents(3000.0, 0) is None
# --- Nicotine and avoided cigarettes (§7.6, §7.7) --------------------------
def test_nicotine_of_a_day() -> None:
entries = [(Decimal(4), Decimal(6)), (Decimal(2), Decimal(3))]
assert calc.nicotine_mg_for_day(entries) == 30.0
assert calc.nicotine_mg_for_day([(Decimal(4), None)]) == 0.0
assert calc.nicotine_mg_for_day([]) is None
def test_cigarette_equivalent_and_counters() -> None:
assert calc.cig_equivalent(24.0) == 2.0
assert calc.days_since_quit(date(2026, 1, 1), date(2026, 1, 11)) == 10
assert calc.days_since_quit(date(2026, 2, 1), date(2026, 1, 11)) == 0
avoided = calc.cigarettes_avoided(10, Decimal("15.5"))
assert avoided == 155
assert calc.cigarettes_avoided(3, Decimal("15.5")) == 46 # floor(46.5)
assert calc.packs_avoided(155, 20) == 7.75
assert calc.time_regained_minutes(155) == 155 * 11
# --- Series helpers --------------------------------------------------------
def test_moving_average_skips_gaps() -> None:
values: list[float | None] = [3.0, None, 6.0, 9.0]
assert calc.moving_average(values, 3) == [3.0, 3.0, 4.5, 7.5]
assert calc.moving_average([None, None], 3) == [None, None]
def test_regression_slope_and_trend_status() -> None:
points = [(date(2026, 1, 1) + timedelta(days=i), 10.0 - i) for i in range(5)]
slope = calc.regression_slope(points)
assert slope == pytest.approx(-1.0)
assert calc.trend_status(slope) == "down"
assert calc.trend_status(0.0) == "stable"
assert calc.trend_status(None) == "stable"
assert calc.trend_status(0.2) == "up"
assert calc.regression_slope(points[:2]) is None
# --- Health milestones (§7.8) ---------------------------------------------
def test_twelve_milestones_from_twenty_minutes_to_fifteen_years() -> None:
assert len(calc.MILESTONES) == 12
assert calc.MILESTONES[0].offset == timedelta(minutes=20)
assert calc.MILESTONES[-1].offset == timedelta(days=15 * 365)
assert calc.MILESTONES[0].code == "hr_bp_normal"
assert calc.MILESTONES[-1].code == "chd_risk_normal"
assert all(m.label_fr and m.label_fr[0].isupper() for m in calc.MILESTONES)
def test_milestone_statuses_reached_and_pending() -> None:
quit_date = date(2026, 1, 1)
now = datetime(2026, 1, 20, 12, 0, tzinfo=UTC) # 19 days later
statuses = calc.milestone_statuses(quit_date, PARIS, now)
by_code = {status.code: status for status in statuses}
assert by_code["hr_bp_normal"].achieved is True
assert by_code["circulation"].achieved is True # 14 days
assert by_code["lung_function"].achieved is False # 90 days
assert by_code["lung_function"].progress_pct == pytest.approx(
100 * (19 + 23 / 24) / 90, abs=0.5
)
assert by_code["hr_bp_normal"].progress_pct == 100.0
# local midnight of the quit day, expressed in UTC (Paris = UTC+1 in January)
assert by_code["hr_bp_normal"].reached_at == datetime(
2025, 12, 31, 23, 20, tzinfo=UTC
)
def test_next_milestone_is_the_first_pending_one() -> None:
statuses = calc.milestone_statuses(
date(2026, 1, 1), PARIS, datetime(2026, 1, 20, 12, 0, tzinfo=UTC)
)
upcoming = calc.next_milestone(statuses)
assert upcoming is not None
assert upcoming.code == "lung_function"
assert calc.next_milestone([]) is None
def test_progress_is_zero_before_the_quit_date() -> None:
statuses = calc.milestone_statuses(
date(2026, 6, 1), PARIS, datetime(2026, 1, 1, tzinfo=UTC)
)
assert all(status.progress_pct == 0.0 for status in statuses)
assert all(status.achieved is False for status in statuses)
+424
View File
@@ -0,0 +1,424 @@
"""Cross-module API contract tests (architecture §8, CONVENTIONS C6).
These assert the rules every module must honour, so a regression in one module
is caught even if that module's own suite still passes: the `Z` suffix on every
datetime, the single error envelope, the `Page[T]` envelope on unbounded lists,
the chart-ready `stats/*` shape, and the presence of the dashboards the home
page calls.
"""
import re
from collections.abc import Callable
from typing import Any
from fastapi.testclient import TestClient
from app.core.module_loader import iter_mounted_routes
from app.main import app
# An ISO datetime whose time part carries no offset at all.
NAIVE_ISO = re.compile(r"^\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}(\.\d+)?$")
# A JSON string that is really a number — the tell-tale of an unserialised
# `Decimal` (Pydantic renders it as `"6.0"`, not `6.0`).
STRINGIFIED_NUMBER = re.compile(r"^-?\d+\.\d+$")
STATS_CONTRACT_KEYS = {"from", "to", "unit", "series", "meta"}
def _find_strings(node: Any, pattern: re.Pattern[str], path: str = "") -> list[str]:
"""Every `path = value` in a JSON tree whose string value matches `pattern`."""
if isinstance(node, dict):
return [
f for k, v in node.items() for f in _find_strings(v, pattern, f"{path}.{k}")
]
if isinstance(node, list):
return [
f
for i, v in enumerate(node)
for f in _find_strings(v, pattern, f"{path}[{i}]")
]
if isinstance(node, str) and pattern.match(node):
return [f"{path} = {node}"]
return []
def _naive_datetimes(node: Any, path: str = "") -> list[str]:
return _find_strings(node, NAIVE_ISO, path)
# --- Datetimes: UTC ISO 8601 with the `Z` suffix (§8.4) ------------------------
def _seed_every_module(client: TestClient, headers: dict[str, str]) -> None:
"""One row in each datetime-bearing table, so read paths return data."""
response = client.put(
"/api/health/profile",
headers=headers,
json={
"height_cm": 180.0,
"sex": "male",
"birthdate": "1990-08-13",
"activity_level": "moderate",
"timezone": "Europe/Paris",
},
)
assert response.status_code == 200, response.text
for path, payload in (
(
"/api/health/weights",
{"measured_at": "2026-08-13T06:31:00Z", "weight_kg": 90.0},
),
(
"/api/health/measurements",
{"measured_at": "2026-08-13T07:00:00Z", "waist_cm": 90},
),
(
"/api/health/workouts",
{
"started_at": "2026-08-13T18:00:00Z",
"ended_at": "2026-08-13T19:00:00Z",
"sport_type": "running",
},
),
(
"/api/health/nutrition/entries",
{
"eaten_at": "2026-08-13T12:00:00Z",
"name": "Poulet",
"meal": "lunch",
"kcal": 500,
},
),
(
"/api/health/nutrition/water",
{"drunk_at": "2026-08-13T09:00:00Z", "volume_ml": 500},
),
("/api/health/nutrition/favorites", {"name": "Favori", "kcal": 100}),
("/api/health/activity", {"date": "2026-08-13", "steps": 9000}),
(
"/api/health/goals",
{
"mode": "weekly_rate",
"start_date": "2026-01-01",
"start_weight_kg": 95,
"target_weight_kg": 80,
"weekly_rate_kg": -0.5,
},
),
("/api/auth/device-keys", {"name": "Pont", "scopes": ["ingest:health"]}),
("/api/vape/coils", {"changed_at": "2026-08-13T07:00:00Z"}),
(
"/api/vape/liquids",
{"entry_date": "2026-08-13", "kind": "daily_total", "ml": 4.0},
),
("/api/finance/accounts", {"name": "Compte courant", "kind": "checking"}),
):
response = client.post(path, headers=headers, json=payload)
assert response.status_code in (200, 201), f"{path}: {response.text}"
csv = b"date;poids\n2026-08-01;91,2\n"
run = client.post(
"/api/imports",
headers=headers,
files={"file": ("poids.csv", csv, "text/csv")},
data={"source": "weight_generic_csv"},
)
assert run.status_code == 201, run.text
def test_no_endpoint_returns_a_naive_datetime(
client: TestClient, auth_headers: dict[str, str]
) -> None:
"""Sweeps every parameterless GET for datetimes missing their offset.
SQLite hands back naive datetimes for `DateTime(timezone=True)` columns, so
an unguarded read path emits `2026-08-13T18:58:59` which `new Date(...)`
parses as *local* time in the browser, a silent 2 h shift in Paris.
"""
_configure_vape(client, auth_headers)
_seed_every_module(client, auth_headers)
offenders: list[str] = []
checked = 0
for path, operations in app.openapi()["paths"].items():
if "get" not in operations or "{" in path:
continue
response = client.get(path, headers=auth_headers)
if response.status_code != 200:
continue
checked += 1
offenders += [f"{path}{field}" for field in _naive_datetimes(response.json())]
assert offenders == []
assert checked > 30, checked
def test_no_endpoint_returns_a_number_as_a_string(
client: TestClient, auth_headers: dict[str, str]
) -> None:
"""`Numeric` columns must reach the frontend as JSON numbers.
Pydantic serialises a bare `Decimal` as a string, so `4.25` ml would arrive
as `"4.25"` ECharts plots nothing and `Intl.NumberFormat` throws. Modules
wrap them (`finance.Money`, `vape.Quantity`) or expose floats (`health`).
"""
_configure_vape(client, auth_headers)
_seed_every_module(client, auth_headers)
offenders: list[str] = []
for path, operations in app.openapi()["paths"].items():
if "get" not in operations or "{" in path:
continue
response = client.get(path, headers=auth_headers)
if response.status_code != 200:
continue
offenders += [
f"{path}{field}"
for field in _find_strings(response.json(), STRINGIFIED_NUMBER)
]
assert offenders == []
def test_naive_datetime_input_is_refused(
client: TestClient, auth_headers: dict[str, str]
) -> None:
"""§8.4: naive input is refused (422) rather than silently read as UTC."""
response = client.post(
"/api/health/weights",
headers=auth_headers,
json={"measured_at": "2026-08-13T06:31:00", "weight_kg": 90.2},
)
assert response.status_code == 422, response.text
assert response.json()["error"]["code"] == "validation_error"
def test_offset_datetime_input_is_converted_to_utc(
client: TestClient, auth_headers: dict[str, str]
) -> None:
response = client.post(
"/api/health/weights",
headers=auth_headers,
json={"measured_at": "2026-08-13T08:31:00+02:00", "weight_kg": 90.2},
)
assert response.status_code == 201, response.text
assert response.json()["measured_at"] == "2026-08-13T06:31:00Z"
# --- Error envelope (§8.3) ----------------------------------------------------
def test_every_module_uses_the_single_error_envelope(client: TestClient) -> None:
for path in (
"/api/health/weights",
"/api/vape/liquids",
"/api/finance/transactions",
"/api/imports",
):
response = client.get(path)
assert response.status_code == 401, path
error = response.json()["error"]
assert set(error) == {"code", "message", "details"}, path
assert error["code"] == "unauthorized", path
# French, user-facing message (C1).
assert error["message"].endswith("."), path
# --- Home page: one dashboard per module (§7.1 of ux-pages) -------------------
def test_dashboard_endpoints_exist_for_every_business_module() -> None:
paths = {path for path, _ in iter_mounted_routes(app)}
for module in ("health", "vape", "finance"):
assert f"/api/{module}/dashboard" in paths, module
def test_health_and_finance_dashboards_work_on_a_blank_account(
client: TestClient, auth_headers: dict[str, str]
) -> None:
"""A brand-new user must not get a 500 on the home page."""
for path in ("/api/health/dashboard", "/api/finance/dashboard"):
response = client.get(path, headers=auth_headers)
assert response.status_code == 200, f"{path}: {response.text}"
def test_vape_dashboard_signals_setup_instead_of_failing(
client: TestClient, auth_headers: dict[str, str]
) -> None:
"""Vape needs a baseline before any figure means anything (§8.4 of the
datamodel): it answers 404 + `setup_required` so the home page can show the
« Module vape non configuré » empty state of ux-pages §16."""
response = client.get("/api/vape/dashboard", headers=auth_headers)
assert response.status_code == 404
error = response.json()["error"]
assert error["code"] == "not_found"
assert error["details"] == {"setup_required": True}
# --- Chart-ready stats contract (datamodel-health-vape §8.1) ------------------
def _configure_vape(client: TestClient, headers: dict[str, str]) -> None:
response = client.put(
"/api/vape/settings",
headers=headers,
json={
"quit_date": "2026-01-01",
"cigs_per_day_before": 20,
"cig_pack_price_cents": 1200,
"cigs_per_pack": 20,
"default_nicotine_mg_ml": 6.0,
"currency": "EUR",
},
)
assert response.status_code == 200, response.text
def test_health_and_vape_stats_follow_the_series_contract(
client: TestClient, auth_headers: dict[str, str]
) -> None:
_configure_vape(client, auth_headers)
endpoints = (
"/api/health/weights/stats",
"/api/health/measurements/stats",
"/api/health/activity/stats",
"/api/health/workouts/stats",
"/api/health/nutrition/stats",
"/api/health/nutrition/water/stats",
"/api/health/stats/adherence",
"/api/health/energy-balance",
"/api/vape/stats/consumption",
"/api/vape/stats/nicotine",
"/api/vape/stats/costs",
"/api/vape/stats/savings",
)
for path in endpoints:
response = client.get(path, headers=auth_headers)
assert response.status_code == 200, f"{path}: {response.text}"
body = response.json()
assert STATS_CONTRACT_KEYS <= set(body), path
assert isinstance(body["meta"], dict), path
for serie in body["series"]:
assert {"name", "type", "points"} <= set(serie), path
for point in serie["points"]:
# [date_iso, value|null] — ready for ECharts `dataset.source`.
assert len(point) == 2, f"{path}/{serie['name']}"
assert isinstance(point[0], str), f"{path}/{serie['name']}"
def test_finance_stats_follow_their_documented_shapes(
client: TestClient, auth_headers: dict[str, str]
) -> None:
"""datamodel-finance §9.8 defines per-endpoint shapes, not the §8.1 envelope."""
expected = {
"/api/finance/stats/monthly-by-category": {"months", "series", "totals"},
"/api/finance/stats/cashflow": {
"months",
"income",
"expenses",
"net",
"cumulative_net",
},
"/api/finance/stats/top-merchants": {"period", "items"},
"/api/finance/stats/recurring": {"items", "monthly_total_estimate"},
"/api/finance/stats/budget-progress": {"month", "items", "totals"},
"/api/finance/stats/sankey": {"period", "nodes", "links"},
}
for path, keys in expected.items():
response = client.get(path, headers=auth_headers)
assert response.status_code == 200, f"{path}: {response.text}"
assert keys <= set(response.json()), path
# --- Pagination envelope (C2.8) ----------------------------------------------
def test_unbounded_lists_use_the_page_envelope(
client: TestClient, auth_headers: dict[str, str]
) -> None:
"""Growing collections are paginated. Bounded config/selector lists
(`/finance/categories` tree, `/imports/sources`, `/vape/milestones`,
`/health/schedules`, `/finance/rules` whose `reorder` takes the whole
ordered set) stay plain arrays on purpose."""
for path in (
"/api/health/weights",
"/api/health/measurements",
"/api/health/workouts",
"/api/health/nutrition/entries",
"/api/health/nutrition/water",
"/api/health/goals",
"/api/vape/liquids",
"/api/vape/products",
"/api/vape/coils",
"/api/vape/purchases",
"/api/finance/transactions",
"/api/imports",
):
response = client.get(path, headers=auth_headers)
assert response.status_code == 200, f"{path}: {response.text}"
body = response.json()
assert {"items", "total", "page", "page_size"} <= set(body), path
# --- Ingest (§5.5) ------------------------------------------------------------
def test_unknown_ingest_domain_is_404(
client: TestClient, auth_headers: dict[str, str]
) -> None:
response = client.post(
"/api/ingest/unknown-domain",
headers=auth_headers,
json={
"source": "probe",
"records": [
{
"type": "weight",
"data": {"measured_at": "2026-08-13T06:31:00Z", "weight_kg": 90.0},
}
],
},
)
assert response.status_code == 404, response.text
assert response.json()["error"]["code"] == "not_found"
# --- Auth boundary (C2.7) -----------------------------------------------------
def test_business_endpoints_reject_anonymous_calls(client: TestClient) -> None:
"""Every documented route except the public auth ones and the probes."""
public = {
"/api/auth/status",
"/api/auth/setup",
"/api/auth/login",
"/api/healthz",
"/healthz",
"/openapi.json",
"/api/openapi.json",
"/api/docs",
}
schema = app.openapi()
unprotected: list[str] = []
checked = 0
caller: dict[str, Callable[..., Any]] = {
"get": client.get,
"post": client.post,
"patch": client.patch,
"put": client.put,
"delete": client.delete,
}
for path, operations in schema["paths"].items():
if path in public or "{" in path:
continue
for verb, call in caller.items():
if verb not in operations:
continue
response = call(path)
checked += 1
if response.status_code != 401:
unprotected.append(f"{verb.upper()} {path} -> {response.status_code}")
assert unprotected == []
# Guard against the sweep silently becoming vacuous.
assert checked > 50, checked
+149
View File
@@ -0,0 +1,149 @@
from fastapi.testclient import TestClient
from app.tests.conftest import TEST_USER_EMAIL, TEST_USER_PASSWORD
def test_status_reports_setup_required_then_not(client: TestClient) -> None:
res = client.get("/api/auth/status")
assert res.status_code == 200
body = res.json()
assert body["setup_required"] is True
assert body["needs_setup"] is True
res = client.post(
"/api/auth/setup",
json={
"email": "first@lifetrack.local",
"password": "long-enough-pw",
"display_name": "Premier",
},
)
assert res.status_code == 201
body = res.json()
assert body["token_type"] == "bearer"
assert body["access_token"]
assert body["user"]["email"] == "first@lifetrack.local"
res = client.get("/api/auth/status")
assert res.json()["setup_required"] is False
def test_setup_forbidden_once_a_user_exists(client: TestClient, user) -> None:
res = client.post(
"/api/auth/setup",
json={
"email": "other@lifetrack.local",
"password": "long-enough-pw",
"display_name": "Intrus",
},
)
assert res.status_code == 403
body = res.json()
assert body["error"]["code"] == "forbidden"
assert body["error"]["message"]
def test_login_ok_and_wrong_password(client: TestClient, user) -> None:
res = client.post(
"/api/auth/login",
json={"email": TEST_USER_EMAIL, "password": TEST_USER_PASSWORD},
)
assert res.status_code == 200
assert res.json()["access_token"]
res = client.post(
"/api/auth/login",
json={"email": TEST_USER_EMAIL, "password": "wrong-password"},
)
assert res.status_code == 401
assert res.json()["error"]["code"] == "unauthorized"
def test_me_requires_auth(client: TestClient, user, auth_headers) -> None:
res = client.get("/api/auth/me")
assert res.status_code == 401
assert res.json()["error"]["code"] == "unauthorized"
res = client.get("/api/auth/me", headers=auth_headers)
assert res.status_code == 200
body = res.json()
assert body["email"] == TEST_USER_EMAIL
assert "password_hash" not in body
def test_patch_me_password_change_requires_current(
client: TestClient, user, auth_headers
) -> None:
res = client.patch(
"/api/auth/me",
headers=auth_headers,
json={"password": "new-password-123"},
)
assert res.status_code == 422
res = client.patch(
"/api/auth/me",
headers=auth_headers,
json={
"password": "new-password-123",
"current_password": TEST_USER_PASSWORD,
"display_name": "Renommé",
},
)
assert res.status_code == 200
assert res.json()["display_name"] == "Renommé"
res = client.post(
"/api/auth/login",
json={"email": TEST_USER_EMAIL, "password": "new-password-123"},
)
assert res.status_code == 200
def test_device_key_lifecycle(client: TestClient, user, auth_headers) -> None:
# Create: plaintext key returned exactly once, shaped ltk_<prefix>_<secret>.
res = client.post(
"/api/auth/device-keys",
headers=auth_headers,
json={"name": "Pixel 8 pont Health Connect", "scopes": ["ingest:health"]},
)
assert res.status_code == 201
created = res.json()
key = created["key"]
assert key.startswith("ltk_")
prefix = key.split("_", 2)[1]
assert len(prefix) == 8
assert created["key_prefix"] == prefix
assert created["scopes"] == ["ingest:health"]
# List never exposes the plaintext key.
res = client.get("/api/auth/device-keys", headers=auth_headers)
assert res.status_code == 200
keys = res.json()
assert len(keys) == 1
assert "key" not in keys[0]
assert keys[0]["key_prefix"] == prefix
assert keys[0]["revoked_at"] is None
# Revoke (DELETE keeps the row, sets revoked_at).
res = client.delete(f"/api/auth/device-keys/{created['id']}", headers=auth_headers)
assert res.status_code == 204
res = client.get("/api/auth/device-keys", headers=auth_headers)
assert res.json()[0]["revoked_at"] is not None
def test_device_key_invalid_scope_rejected(
client: TestClient, user, auth_headers
) -> None:
res = client.post(
"/api/auth/device-keys",
headers=auth_headers,
json={"name": "Mauvaise clé", "scopes": ["admin:*"]},
)
assert res.status_code == 422
def test_error_shape_on_unknown_route(client: TestClient) -> None:
res = client.get("/api/definitely-not-a-route")
assert res.status_code == 404
assert set(res.json()["error"].keys()) == {"code", "message", "details"}
+368
View File
@@ -0,0 +1,368 @@
"""Contract tests for the imports/ingest framework.
They use a self-contained sample domain (model + importer + ingest handler
defined below) so they do not depend on any business module.
"""
from collections.abc import Iterator
from io import BytesIO
from fastapi.testclient import TestClient
from sqlalchemy import ForeignKey, String, UniqueConstraint, select
from sqlalchemy.orm import Mapped, Session, mapped_column
from app.core.database import Base
from app.core.importing.base import (
BaseImporter,
ImporterParseError,
NormalizedRecord,
RowError,
UpsertOutcome,
)
from app.core.importing.hashing import content_hash
from app.core.importing.registry import IMPORTER_REGISTRY, register_importer
from app.core.ingest.base import BaseIngestHandler, IngestRecord
from app.core.ingest.registry import INGEST_REGISTRY, register_ingest_handler
from app.core.mixins import SourceMixin, TimestampMixin
from app.modules.auth import service as auth_service
from app.modules.auth.schemas import DeviceKeyCreate
class SampleItem(TimestampMixin, SourceMixin, Base):
__tablename__ = "test_sample_items"
__table_args__ = (
UniqueConstraint(
"user_id", "source", "external_id", name="uq_test_sample_items_external"
),
UniqueConstraint("user_id", "content_hash", name="uq_test_sample_items_hash"),
)
id: Mapped[int] = mapped_column(primary_key=True)
user_id: Mapped[int] = mapped_column(ForeignKey("users.id"), index=True)
import_run_id: Mapped[int | None] = mapped_column(
ForeignKey("import_runs.id", ondelete="CASCADE"), default=None
)
label: Mapped[str] = mapped_column(String(100))
value: Mapped[float] = mapped_column()
def _upsert_sample(
db: Session,
user_id: int,
record: NormalizedRecord,
source: str,
import_run_id: int | None,
) -> UpsertOutcome:
if record.data["value"] < 0:
raise RowError("Valeur négative interdite.")
if record.external_id is not None:
digest = None
existing = db.scalar(
select(SampleItem).where(
SampleItem.user_id == user_id,
SampleItem.source == source,
SampleItem.external_id == record.external_id,
)
)
else:
digest = content_hash(record)
existing = db.scalar(
select(SampleItem).where(
SampleItem.user_id == user_id,
SampleItem.content_hash == digest,
)
)
if existing is not None:
return UpsertOutcome.DUPLICATE
db.add(
SampleItem(
user_id=user_id,
source=source,
external_id=record.external_id,
content_hash=digest,
import_run_id=import_run_id,
label=record.data["label"],
value=record.data["value"],
)
)
db.flush()
return UpsertOutcome.INSERTED
if "test_sample_csv" not in IMPORTER_REGISTRY:
@register_importer
class SampleCsvImporter(BaseImporter):
id = "test_sample_csv"
label = "Échantillon de test (CSV)"
domain = "sample"
accepted_extensions = (".csv",)
@classmethod
def sniff(cls, filename: str, head: bytes) -> bool:
return filename.lower().endswith(".csv") and head.startswith(b"label;value")
def parse(self, data: bytes, filename: str) -> Iterator[NormalizedRecord]:
lines = [ln for ln in data.decode("utf-8").splitlines() if ln.strip()]
if not lines or not lines[0].startswith("label;value"):
raise ImporterParseError("En-têtes introuvables dans le fichier.")
for line in lines[1:]:
label, raw_value, external_id = line.split(";")
yield NormalizedRecord(
kind="sample",
data={"label": label, "value": float(raw_value.replace(",", "."))},
external_id=external_id or None,
dedupe_fields=("label", "value"),
)
def upsert(
self, db: Session, user_id: int, record: NormalizedRecord
) -> UpsertOutcome:
return _upsert_sample(db, user_id, record, self.id, self.import_run_id)
if "sample" not in INGEST_REGISTRY:
@register_ingest_handler
class SampleIngestHandler(BaseIngestHandler):
domain = "sample"
record_types = ("sample",)
def apply(
self, db: Session, user_id: int, record: IngestRecord
) -> UpsertOutcome:
normalized = NormalizedRecord(
kind=record.type,
data=record.data,
external_id=record.external_id,
dedupe_fields=("label", "value"),
)
return _upsert_sample(db, user_id, normalized, record.source, None)
CSV_OK = "label;value;ext\nCafé;1,5;a1\nThé;2;a2\nJus;3;\n"
def _upload(client: TestClient, headers: dict[str, str], content: str, source: str):
return client.post(
"/api/imports",
headers=headers,
files={"file": ("sample.csv", BytesIO(content.encode("utf-8")), "text/csv")},
data={"source": source},
)
def test_sources_endpoint_lists_importers(
client: TestClient, user, auth_headers
) -> None:
res = client.get("/api/imports/sources", headers=auth_headers)
assert res.status_code == 200
by_id = {src["id"]: src for src in res.json()}
assert by_id["test_sample_csv"]["label"] == "Échantillon de test (CSV)"
assert by_id["test_sample_csv"]["domain"] == "sample"
assert by_id["test_sample_csv"]["accepted_extensions"] == [".csv"]
def test_import_then_reimport_is_idempotent(
client: TestClient, db: Session, user, auth_headers
) -> None:
res = _upload(client, auth_headers, CSV_OK, "test_sample_csv")
assert res.status_code == 201
run = res.json()
assert run["status"] == "completed"
assert run["rows_total"] == 3
assert run["rows_inserted"] == 3
assert run["rows_duplicates"] == 0
# Same file again: everything deduped (external_id + content_hash paths).
res = _upload(client, auth_headers, CSV_OK, "test_sample_csv")
assert res.status_code == 201
rerun = res.json()
assert rerun["rows_inserted"] == 0
assert rerun["rows_duplicates"] == 3
assert db.scalar(select(SampleItem.id).limit(1)) is not None
rows = db.scalars(select(SampleItem)).all()
assert len(rows) == 3
assert all(row.import_run_id == run["id"] for row in rows)
def test_auto_sniff_detects_and_rejects(client: TestClient, user, auth_headers) -> None:
res = _upload(client, auth_headers, CSV_OK, "auto")
assert res.status_code == 201
assert res.json()["importer_id"] == "test_sample_csv"
res = _upload(client, auth_headers, "something;else\n1;2\n", "auto")
assert res.status_code == 422
assert (
res.json()["error"]["message"]
== "Format non reconnu, choisissez un profil de source."
)
def test_unknown_source_rejected(client: TestClient, user, auth_headers) -> None:
res = _upload(client, auth_headers, CSV_OK, "nope_csv")
assert res.status_code == 422
assert res.json()["error"]["code"] == "validation_error"
def test_row_errors_do_not_stop_the_run(client: TestClient, user, auth_headers) -> None:
csv = "label;value;ext\nCafé;1,5;b1\nMauvais;-4;b2\nThé;2;b3\n"
res = _upload(client, auth_headers, csv, "test_sample_csv")
assert res.status_code == 201
run = res.json()
assert run["status"] == "completed"
assert run["rows_total"] == 3
assert run["rows_inserted"] == 2
assert run["rows_errors"] == 1
assert run["error_details"][0]["row"] == 2
assert "négative" in run["error_details"][0]["message"]
def test_fatal_parse_error_marks_run_failed(
client: TestClient, db: Session, user, auth_headers
) -> None:
res = _upload(client, auth_headers, "garbage without headers\n", "test_sample_csv")
assert res.status_code == 201
run = res.json()
assert run["status"] == "failed"
assert run["error_details"]
# No partial domain rows are kept for a failed run.
assert db.scalars(select(SampleItem)).all() == []
def test_history_pagination_and_filters(client: TestClient, user, auth_headers) -> None:
_upload(client, auth_headers, CSV_OK, "test_sample_csv")
_upload(client, auth_headers, CSV_OK, "test_sample_csv")
res = client.get("/api/imports", headers=auth_headers)
assert res.status_code == 200
page = res.json()
assert set(page.keys()) == {"items", "total", "page", "page_size"}
assert page["total"] == 2
res = client.get("/api/imports?domain=finance", headers=auth_headers)
assert res.json()["total"] == 0
res = client.get("/api/imports?sort=bogus", headers=auth_headers)
assert res.status_code == 422
run_id = page["items"][0]["id"]
res = client.get(f"/api/imports/{run_id}", headers=auth_headers)
assert res.status_code == 200
assert res.json()["id"] == run_id
def test_rollback_deletes_run_and_cascades(
client: TestClient, db: Session, user, auth_headers
) -> None:
run_id = _upload(client, auth_headers, CSV_OK, "test_sample_csv").json()["id"]
assert len(db.scalars(select(SampleItem)).all()) == 3
res = client.delete(f"/api/imports/{run_id}", headers=auth_headers)
assert res.status_code == 204
db.expire_all()
assert db.scalars(select(SampleItem)).all() == []
res = client.get(f"/api/imports/{run_id}", headers=auth_headers)
assert res.status_code == 404
assert res.json()["error"]["code"] == "not_found"
def test_upload_too_large_rejected(
client: TestClient, user, auth_headers, monkeypatch
) -> None:
from app.core.config import get_settings
monkeypatch.setattr(get_settings(), "max_upload_bytes", 10)
res = _upload(client, auth_headers, CSV_OK, "test_sample_csv")
assert res.status_code == 413
assert res.json()["error"]["code"] == "payload_too_large"
def test_ingest_with_device_key_and_dedup(client: TestClient, user, device_key) -> None:
payload = {
"source": "android_bridge",
"records": [
{"type": "sample", "external_id": "x1", "data": {"label": "A", "value": 1}},
{"type": "sample", "data": {"label": "B", "value": 2}},
],
}
res = client.post("/api/ingest/sample", headers=device_key.headers, json=payload)
assert res.status_code == 200
body = res.json()
assert body == {
"domain": "sample",
"received": 2,
"inserted": 2,
"updated": 0,
"duplicates": 0,
"errors": [],
}
res = client.post("/api/ingest/sample", headers=device_key.headers, json=payload)
assert res.json()["duplicates"] == 2
def test_ingest_works_with_jwt_too(client: TestClient, user, auth_headers) -> None:
payload = {"records": [{"type": "sample", "data": {"label": "J", "value": 9}}]}
res = client.post("/api/ingest/sample", headers=auth_headers, json=payload)
assert res.status_code == 200
assert res.json()["inserted"] == 1
def test_ingest_auth_and_scope_rules(
client: TestClient, db: Session, user, device_key
) -> None:
payload = {"records": [{"type": "sample", "data": {"label": "S", "value": 4}}]}
# No credentials at all -> 401.
res = client.post("/api/ingest/sample", json=payload)
assert res.status_code == 401
# Key scoped to another domain -> 403.
_, other_plaintext = auth_service.create_device_key(
db, user.id, DeviceKeyCreate(name="Autre", scopes=["ingest:health"])
)
res = client.post(
"/api/ingest/sample", headers={"X-API-Key": other_plaintext}, json=payload
)
assert res.status_code == 403
# Revoked wildcard key -> 401.
auth_service.revoke_device_key(db, user.id, device_key.id)
res = client.post("/api/ingest/sample", headers=device_key.headers, json=payload)
assert res.status_code == 401
def test_ingest_unknown_domain_and_bad_type(
client: TestClient, user, device_key
) -> None:
payload = {"records": [{"type": "sample", "data": {"label": "D", "value": 1}}]}
res = client.post("/api/ingest/nope", headers=device_key.headers, json=payload)
assert res.status_code == 404
payload = {
"records": [
{"type": "sample", "data": {"label": "OK", "value": 1}},
{"type": "unknown_type", "data": {}},
{"type": "sample", "data": {"label": "KO", "value": -1}},
]
}
res = client.post("/api/ingest/sample", headers=device_key.headers, json=payload)
assert res.status_code == 200
body = res.json()
assert body["received"] == 3
assert body["inserted"] == 1
assert {err["index"] for err in body["errors"]} == {1, 2}
def test_ingest_record_limit(client: TestClient, user, device_key) -> None:
records = [
{"type": "sample", "data": {"label": f"r{i}", "value": i}} for i in range(1001)
]
res = client.post(
"/api/ingest/sample", headers=device_key.headers, json={"records": records}
)
assert res.status_code == 422
assert res.json()["error"]["code"] == "validation_error"

Some files were not shown because too many files have changed in this diff Show More