replace Supabase with Postgres + Drizzle ORM
- Drop @supabase/supabase-js entirely; add drizzle-orm + postgres (porsager) driver - New packages/db: schema (pgSchema ordinarthur_os), client factory, migrate runner, drizzle-kit config - SQL migrations: 0000_init (pgcrypto + schema), 0001_jobs (jobs + job_search_criteria, no RLS) - Rewrite apps/api db module with DI symbols DB/DB_HANDLE + @InjectDb() decorator - Rewrite jobs.service.ts with Drizzle queries (upsert via onConflictDoUpdate, arrayOverlaps for stack filter) - Replace SUPABASE_* env vars with DATABASE_URL in env config + .env.example - Add docker-compose.yml (Postgres 16-alpine, dev only) - Add deploy/k8s/postgres.yaml (StatefulSet + PVC), migrate.job.yaml, updated secrets.template.yaml - Update all docs (README, PLAN, ARCHITECTURE, CLAUDE.md, AGENTS.md, packages/db/README.md) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
parent
eb430b59e6
commit
9c93e74318
@ -13,7 +13,7 @@ Avant toute action, lire dans cet ordre :
|
||||
- **Single-user.** Bearer token statique, pas de multi-tenant.
|
||||
- **Design = portfolio arthurbarre.fr** (cream `#F5F1EA`, ink `#0F0F0F`, accent orange `#FF4A1C`, mono labels, bordures, italique = orange). PAS le violet/cyan du HTML jobs.
|
||||
- **FR only** côté IA (prompts + STT lang=fr).
|
||||
- **Schéma Postgres dédié `ordinarthur_os`** dans Supabase self-hosted (`supabase.arthurbarre.fr`).
|
||||
- **Postgres standalone dans le k3s** (plus de Supabase), schéma dédié `ordinarthur_os`. ORM = **Drizzle** (via `packages/db`).
|
||||
|
||||
## Phases
|
||||
|
||||
@ -23,7 +23,7 @@ Voir `PLAN.md`. Implémentation séquentielle Phase 0 → 7. Phase 8 (finance) r
|
||||
|
||||
- Monorepo pnpm + Turborepo
|
||||
- `apps/pwa` Vite + React + TanStack Router/Query + Tailwind + shadcn
|
||||
- `apps/api` NestJS (modules par domaine), `@supabase/supabase-js` (pas d'ORM)
|
||||
- `apps/api` NestJS (modules par domaine), Drizzle ORM via `@ordinarthur-os/db`
|
||||
- `packages/shared` types + zod DTOs partagés PWA ↔ API
|
||||
- `packages/db/migrations` SQL versionné, appliqué manuellement sur Supabase pour l'instant
|
||||
- `packages/db` schéma Drizzle + `migrations/` SQL versionnées (runner `pnpm --filter @ordinarthur-os/db migrate`)
|
||||
- Pas de fichier `.env` commité, juste `.env.example`
|
||||
|
||||
@ -12,15 +12,15 @@
|
||||
┌──────────────┬──────────────────┼────────────────┬──────────────┐
|
||||
▼ ▼ ▼ ▼ ▼
|
||||
┌────────────┐ ┌─────────────┐ ┌──────────────┐ ┌────────────┐ ┌────────────┐
|
||||
│ Supabase │ │ Mistral AI │ │ Groq Whisper │ │ Google Cal │ │ Telegram │
|
||||
│ self-host │ │ (small) │ │ (STT FR) │ │ OAuth │ │ Bot API │
|
||||
│ Postgres │ │ Mistral AI │ │ Groq Whisper │ │ Google Cal │ │ Telegram │
|
||||
│ (k3s STS) │ │ (small) │ │ (STT FR) │ │ OAuth │ │ Bot API │
|
||||
│ schema │ │ function- │ │ │ │ │ │ │
|
||||
│ ordinarthur│ │ calling │ │ │ │ │ │ │
|
||||
│ _os │ │ │ │ │ │ │ │ │
|
||||
└────────────┘ └─────────────┘ └──────────────┘ └────────────┘ └────────────┘
|
||||
```
|
||||
|
||||
La PWA ne parle **jamais** directement à Supabase ni aux APIs externes. Tout passe par le BFF NestJS, protégé par bearer token.
|
||||
La PWA ne parle **jamais** directement à Postgres ni aux APIs externes. Tout passe par le BFF NestJS, protégé par bearer token. Le service Postgres est en ClusterIP interne au k3s, jamais exposé.
|
||||
|
||||
## 2. Monorepo
|
||||
|
||||
@ -52,20 +52,28 @@ ordinarthur-os/
|
||||
│ │ ├── ai/ # command, voice, function dispatcher
|
||||
│ │ ├── telegram/
|
||||
│ │ └── sync/ # replay client_mutations, dedup
|
||||
│ ├── db/ # supabase client factory
|
||||
│ ├── db/ # Drizzle client factory (@ordinarthur-os/db)
|
||||
│ ├── config/ # env schema (zod)
|
||||
│ └── main.ts
|
||||
│
|
||||
├── packages/
|
||||
│ ├── shared/ # Types + zod DTOs partagés pwa/api
|
||||
│ └── db/
|
||||
│ └── migrations/ # 0001_schema.sql, 0002_jobs.sql, …
|
||||
│ └── db/ # Drizzle ORM : schema TS + migrations SQL
|
||||
│ ├── drizzle.config.ts
|
||||
│ ├── src/
|
||||
│ │ ├── schema/ # Définitions tables (pgSchema ordinarthur_os)
|
||||
│ │ ├── client.ts # createDb(connectionString)
|
||||
│ │ └── migrate.ts # runner DATABASE_URL-driven
|
||||
│ └── migrations/ # 0000_init.sql, 0001_jobs.sql, … + meta/_journal.json
|
||||
│
|
||||
├── deploy/
|
||||
│ └── k8s/ # Manifests à aligner sur conf Gitea d'Arthur
|
||||
│ ├── namespace.yaml
|
||||
│ ├── postgres.yaml # StatefulSet Postgres 16 + PVC
|
||||
│ ├── api.deployment.yaml
|
||||
│ ├── pwa.deployment.yaml
|
||||
│ ├── ingress.yaml
|
||||
│ ├── migrate.job.yaml # Job one-shot drizzle migrate
|
||||
│ ├── secrets.template.yaml
|
||||
│ └── backup.cronjob.yaml
|
||||
│
|
||||
@ -124,6 +132,8 @@ theme: {
|
||||
|
||||
## 4. Schéma Postgres `ordinarthur_os`
|
||||
|
||||
Source de vérité : les définitions Drizzle dans [`packages/db/src/schema/`](./packages/db/src/schema/). Le SQL ci-dessous est l'équivalent dénormalisé, à titre de référence — les migrations réelles vivent dans `packages/db/migrations/`.
|
||||
|
||||
```sql
|
||||
create schema if not exists ordinarthur_os;
|
||||
set search_path to ordinarthur_os;
|
||||
@ -283,7 +293,7 @@ create table client_mutations (
|
||||
);
|
||||
```
|
||||
|
||||
**RLS** : activée partout avec policy unique `using (auth.role() = 'service_role')`. Seul le Nest (avec la service key) peut lire/écrire. La PWA est derrière le bearer Nest.
|
||||
**Pas de RLS.** La base n'est jamais exposée : le seul client SQL est le backend NestJS (ClusterIP interne au k3s, credentials Postgres classiques). La PWA est derrière le bearer Nest.
|
||||
|
||||
## 5. API NestJS — routes
|
||||
|
||||
@ -427,10 +437,9 @@ spec:
|
||||
|
||||
### Secrets k8s attendus
|
||||
|
||||
`ordinarthur-os-secrets` :
|
||||
`ordinarthur-os-secrets` (consommé par l'API) :
|
||||
- `API_BEARER_TOKEN`
|
||||
- `SUPABASE_URL` (`https://supabase.arthurbarre.fr`)
|
||||
- `SUPABASE_SERVICE_ROLE_KEY`
|
||||
- `DATABASE_URL` (`postgres://<user>:<pwd>@postgres.ordinarthur-os.svc.cluster.local:5432/ordinarthur_os`)
|
||||
- `MISTRAL_API_KEY`
|
||||
- `GROQ_API_KEY`
|
||||
- `GOOGLE_OAUTH_CLIENT_ID`
|
||||
@ -440,6 +449,13 @@ spec:
|
||||
- `TELEGRAM_WEBHOOK_SECRET`
|
||||
- `ICAL_FEED_SECRET`
|
||||
|
||||
`ordinarthur-os-db-secrets` (consommé par le StatefulSet Postgres) :
|
||||
- `POSTGRES_USER`, `POSTGRES_PASSWORD`, `POSTGRES_DB`
|
||||
|
||||
`ordinarthur-os-backup-secrets` (CronJob) :
|
||||
- `PGURL` (même valeur que `DATABASE_URL`)
|
||||
- `RCLONE_REMOTE` (ex. `b2:ordinarthur-os-backups`)
|
||||
|
||||
### CronJob backup
|
||||
|
||||
```yaml
|
||||
@ -462,10 +478,11 @@ spec:
|
||||
args:
|
||||
- |
|
||||
pg_dump "$PGURL" --schema=ordinarthur_os --format=c | gzip > /tmp/dump.gz
|
||||
# upload vers S3/B2 via rclone (à confirmer avec Arthur)
|
||||
rclone copy /tmp/dump.gz remote:ordinarthur-os-backups/$(date +%F).gz
|
||||
rclone copy /tmp/dump.gz "$RCLONE_REMOTE/$(date +%F).gz"
|
||||
```
|
||||
|
||||
> `PGURL` = même chaîne `postgres://…` que `DATABASE_URL`. Le schéma reste dédié (`--schema=ordinarthur_os`) pour faciliter d'éventuelles restaurations sélectives.
|
||||
|
||||
### Pipeline Gitea (à aligner avec le skill `/deploy` d'Arthur)
|
||||
|
||||
```yaml
|
||||
@ -500,7 +517,7 @@ jobs:
|
||||
- Bearer token Nest : stocké seulement dans les secrets k8s, jamais dans le bundle PWA → la PWA demande le token à l'utilisateur (écran d'onboarding) et le stocke dans `localStorage` (accès par Arthur uniquement sur son device)
|
||||
- Webhook Telegram : vérification de `X-Telegram-Bot-Api-Secret-Token`
|
||||
- Feed iCal : path contient un secret rotable (`ICAL_FEED_SECRET`)
|
||||
- Supabase : schema dédié + RLS service-role-only
|
||||
- Postgres : service ClusterIP interne au k3s, jamais exposé ; credentials via Secret k8s
|
||||
- Backups : quotidien, chiffrés au repos côté bucket, rétention 30 jours
|
||||
|
||||
## 10. Observabilité (phase ultérieure)
|
||||
@ -519,9 +536,7 @@ jobs:
|
||||
NODE_ENV=production
|
||||
PORT=3000
|
||||
API_BEARER_TOKEN=
|
||||
SUPABASE_URL=https://supabase.arthurbarre.fr
|
||||
SUPABASE_SERVICE_ROLE_KEY=
|
||||
SUPABASE_SCHEMA=ordinarthur_os
|
||||
DATABASE_URL=postgres://ordinarthur:changeme@postgres.ordinarthur-os.svc.cluster.local:5432/ordinarthur_os
|
||||
MISTRAL_API_KEY=
|
||||
MISTRAL_MODEL=mistral-small-latest
|
||||
GROQ_API_KEY=
|
||||
|
||||
@ -13,7 +13,7 @@ Avant toute action, lire dans cet ordre :
|
||||
- **Single-user.** Bearer token statique, pas de multi-tenant.
|
||||
- **Design = portfolio arthurbarre.fr** (cream `#F5F1EA`, ink `#0F0F0F`, accent orange `#FF4A1C`, mono labels, bordures, italique = orange). PAS le violet/cyan du HTML jobs.
|
||||
- **FR only** côté IA (prompts + STT lang=fr).
|
||||
- **Schéma Postgres dédié `ordinarthur_os`** dans Supabase self-hosted (`supabase.arthurbarre.fr`).
|
||||
- **Postgres standalone dans le k3s** (plus de Supabase), schéma dédié `ordinarthur_os`. ORM = **Drizzle** (via `packages/db`).
|
||||
|
||||
## Phases
|
||||
|
||||
@ -23,7 +23,7 @@ Voir `PLAN.md`. Implémentation séquentielle Phase 0 → 7. Phase 8 (finance) r
|
||||
|
||||
- Monorepo pnpm + Turborepo
|
||||
- `apps/pwa` Vite + React + TanStack Router/Query + Tailwind + shadcn
|
||||
- `apps/api` NestJS (modules par domaine), `@supabase/supabase-js` (pas d'ORM)
|
||||
- `apps/api` NestJS (modules par domaine), Drizzle ORM via `@ordinarthur-os/db`
|
||||
- `packages/shared` types + zod DTOs partagés PWA ↔ API
|
||||
- `packages/db/migrations` SQL versionné, appliqué manuellement sur Supabase pour l'instant
|
||||
- `packages/db` schéma Drizzle + `migrations/` SQL versionnées (runner `pnpm --filter @ordinarthur-os/db migrate`)
|
||||
- Pas de fichier `.env` commité, juste `.env.example`
|
||||
|
||||
12
PLAN.md
12
PLAN.md
@ -16,7 +16,7 @@ Un assistant personnel qui aide Arthur à s'organiser **sans le déresponsabilis
|
||||
1. **Self-hosted, open-source**. Pas de Vercel, pas de Next.js. Tout tourne sur le k3s d'Arthur.
|
||||
2. **Single-user**. Pas de multi-tenant, pas d'invitations, pas de partage. Bearer token unique pour protéger l'API.
|
||||
3. **PWA installable iOS**. Vite + React, pas de SSR. Service worker + mutation queue pour l'offline.
|
||||
4. **BFF unique**. La PWA ne parle qu'au NestJS. Le Nest parle à Supabase, Mistral, Groq, Google Calendar, Telegram.
|
||||
4. **BFF unique**. La PWA ne parle qu'au NestJS. Le Nest parle à Postgres, Mistral, Groq, Google Calendar, Telegram.
|
||||
5. **Design éditorial / Swiss-brutalist** — mirror du portfolio arthurbarre.fr (cream, ink, orange, borders, mono labels).
|
||||
|
||||
## Stack verrouillée
|
||||
@ -25,8 +25,8 @@ Un assistant personnel qui aide Arthur à s'organiser **sans le déresponsabilis
|
||||
| --- | --- |
|
||||
| Monorepo | pnpm workspaces + Turborepo |
|
||||
| Frontend | Vite + React 18 + TanStack Router + TanStack Query + Tailwind + shadcn/ui |
|
||||
| Backend | NestJS + `@supabase/supabase-js` (pas d'ORM) |
|
||||
| DB | Postgres via Supabase self-hosted, schéma dédié `ordinarthur_os` |
|
||||
| Backend | NestJS + Drizzle ORM (driver `postgres`) |
|
||||
| DB | Postgres 16 standalone (k3s StatefulSet + PVC), schéma dédié `ordinarthur_os` |
|
||||
| Auth | Bearer token statique (single-user), middleware Nest |
|
||||
| IA LLM | Mistral `mistral-small-latest` (low-cost) via API |
|
||||
| STT | Groq `whisper-large-v3-turbo` |
|
||||
@ -41,9 +41,9 @@ Un assistant personnel qui aide Arthur à s'organiser **sans le déresponsabilis
|
||||
### Phase 0 — Scaffold (prio immédiate)
|
||||
- Monorepo `pnpm-workspace.yaml`, `turbo.json`
|
||||
- `apps/pwa` : Vite + React + Tailwind + shadcn + manifest PWA + service worker placeholder + routing TanStack
|
||||
- `apps/api` : NestJS + module `health` + middleware bearer + client supabase initialisé
|
||||
- `apps/api` : NestJS + module `health` + middleware bearer + client Drizzle initialisé
|
||||
- `packages/shared` : types et zod schemas partagés
|
||||
- `packages/db` : premier fichier de migration SQL créant le schéma `ordinarthur_os`
|
||||
- `packages/db` : schéma Drizzle + premières migrations SQL (`0000_init` crée le schéma `ordinarthur_os`)
|
||||
- `deploy/k8s` : manifests génériques (à adapter ensuite à la conf Gitea/Traefik d'Arthur)
|
||||
- Design system : composants primitifs (`<Label>`, `<SectionHeader>`, `<GridFrame>`, `<DataChip>`, `<MetaRow>`) qui reproduisent le style arthurbarre.fr
|
||||
- Routes `GET /health` et `POST /auth/verify`
|
||||
@ -106,7 +106,7 @@ Pour reprendre ce projet avec Claude Code (Sonnet) :
|
||||
4. Avant de scaffolder, récupérer de l'utilisateur :
|
||||
- Le dossier `/Users/arthurbarre/dev/perso/proxmox` (conf k3s) pour aligner les manifests
|
||||
- Le skill `/deploy` ou `/create-deployment` qu'Arthur utilise pour ses autres déploiements Gitea
|
||||
- Les secrets nécessaires : `MISTRAL_API_KEY`, `GROQ_API_KEY`, `TELEGRAM_BOT_TOKEN`, `GOOGLE_OAUTH_CLIENT_ID/SECRET`, `SUPABASE_URL`, `SUPABASE_SERVICE_ROLE_KEY`, `API_BEARER_TOKEN`
|
||||
- Les secrets nécessaires : `MISTRAL_API_KEY`, `GROQ_API_KEY`, `TELEGRAM_BOT_TOKEN`, `GOOGLE_OAUTH_CLIENT_ID/SECRET`, `DATABASE_URL` (+ `POSTGRES_USER`/`POSTGRES_PASSWORD`/`POSTGRES_DB` côté StatefulSet), `API_BEARER_TOKEN`
|
||||
- Le choix du stockage backup S3-compatible (B2 / Scaleway / autre)
|
||||
5. Attaquer par la Phase 0 (scaffold), puis Phase 1 (jobs) — c'est explicitement prioritaire dans la tête d'Arthur.
|
||||
|
||||
|
||||
@ -1,6 +1,6 @@
|
||||
# ordinarthur-os
|
||||
|
||||
Assistant personnel self-hosted d'Arthur Barré. PWA installable + backend NestJS, déployés sur k3s personnel, branchés sur Supabase self-hosted (`supabase.arthurbarre.fr`).
|
||||
Assistant personnel self-hosted d'Arthur Barré. PWA installable + backend NestJS, déployés sur k3s personnel, adossés à un Postgres standalone (schéma `ordinarthur_os`) piloté via Drizzle ORM.
|
||||
|
||||
**But** : aider Arthur à être rigoureux (todos, projets, agenda, recherche d'emploi, santé) **sans le déresponsabiliser**. Toutes les actions IA passent par une confirmation explicite.
|
||||
|
||||
|
||||
@ -4,10 +4,9 @@ PORT=3000
|
||||
# Single-user bearer (génère via `openssl rand -hex 32`)
|
||||
API_BEARER_TOKEN=
|
||||
|
||||
# Supabase self-hosted
|
||||
SUPABASE_URL=https://supabase.arthurbarre.fr
|
||||
SUPABASE_SERVICE_ROLE_KEY=
|
||||
SUPABASE_SCHEMA=ordinarthur_os
|
||||
# Postgres standalone (k3s ou local docker). Le schéma `ordinarthur_os`
|
||||
# est géré par Drizzle, pas besoin de `search_path` dans l'URL.
|
||||
DATABASE_URL=postgres://ordinarthur:changeme@localhost:5432/ordinarthur_os
|
||||
|
||||
# Phase 5+
|
||||
MISTRAL_API_KEY=
|
||||
|
||||
@ -13,8 +13,10 @@
|
||||
"@nestjs/common": "^10.4.4",
|
||||
"@nestjs/core": "^10.4.4",
|
||||
"@nestjs/platform-express": "^10.4.4",
|
||||
"@ordinarthur-os/db": "workspace:*",
|
||||
"@ordinarthur-os/shared": "workspace:*",
|
||||
"@supabase/supabase-js": "^2.45.4",
|
||||
"drizzle-orm": "^0.36.4",
|
||||
"postgres": "^3.4.5",
|
||||
"reflect-metadata": "^0.2.2",
|
||||
"rxjs": "^7.8.1",
|
||||
"zod": "^3.23.8"
|
||||
|
||||
@ -8,9 +8,11 @@ const EnvSchema = z.object({
|
||||
|
||||
API_BEARER_TOKEN: z.string().min(16, "API_BEARER_TOKEN must be at least 16 chars"),
|
||||
|
||||
SUPABASE_URL: z.string().url(),
|
||||
SUPABASE_SERVICE_ROLE_KEY: z.string().min(1),
|
||||
SUPABASE_SCHEMA: z.string().default("ordinarthur_os"),
|
||||
// Postgres standalone (k3s) — pas de Supabase.
|
||||
// Le driver `postgres` impose d'inclure le search_path via
|
||||
// `?options=-c%20search_path%3Dordinarthur_os%2Cpublic` n'est PAS nécessaire
|
||||
// car Drizzle préfixe lui-même le schéma (`pgSchema`).
|
||||
DATABASE_URL: z.string().url(),
|
||||
|
||||
MISTRAL_API_KEY: z.string().optional(),
|
||||
MISTRAL_MODEL: z.string().default("mistral-small-latest"),
|
||||
|
||||
@ -1,27 +1,35 @@
|
||||
import { Global, Module, Inject } from "@nestjs/common";
|
||||
import { createClient, SupabaseClient } from "@supabase/supabase-js";
|
||||
import { Global, Inject, Module, type OnApplicationShutdown } from "@nestjs/common";
|
||||
import { createDb, type Db, type DbHandle } from "@ordinarthur-os/db";
|
||||
import { APP_CONFIG } from "../config/config.module";
|
||||
import type { AppConfig } from "../config/env";
|
||||
|
||||
export const SUPABASE = Symbol("SUPABASE");
|
||||
export type Supabase = SupabaseClient<any, any, any, any, any>;
|
||||
export const DB = Symbol("DB");
|
||||
export const DB_HANDLE = Symbol("DB_HANDLE");
|
||||
export type { Db };
|
||||
|
||||
@Global()
|
||||
@Module({
|
||||
providers: [
|
||||
{
|
||||
provide: SUPABASE,
|
||||
provide: DB_HANDLE,
|
||||
inject: [APP_CONFIG],
|
||||
useFactory: (config: AppConfig): Supabase =>
|
||||
createClient(config.SUPABASE_URL, config.SUPABASE_SERVICE_ROLE_KEY, {
|
||||
auth: { persistSession: false, autoRefreshToken: false },
|
||||
db: { schema: config.SUPABASE_SCHEMA },
|
||||
}),
|
||||
useFactory: (config: AppConfig): DbHandle => createDb(config.DATABASE_URL),
|
||||
},
|
||||
{
|
||||
provide: DB,
|
||||
inject: [DB_HANDLE],
|
||||
useFactory: (handle: DbHandle): Db => handle.db,
|
||||
},
|
||||
],
|
||||
exports: [SUPABASE],
|
||||
exports: [DB, DB_HANDLE],
|
||||
})
|
||||
export class DbModule {}
|
||||
export class DbModule implements OnApplicationShutdown {
|
||||
constructor(@Inject(DB_HANDLE) private readonly handle: DbHandle) {}
|
||||
|
||||
// Convenience decorator : `constructor(@InjectSupabase() private db: Supabase) {}`
|
||||
export const InjectSupabase = () => Inject(SUPABASE);
|
||||
async onApplicationShutdown() {
|
||||
await this.handle.close();
|
||||
}
|
||||
}
|
||||
|
||||
/** `constructor(@InjectDb() private db: Db) {}` */
|
||||
export const InjectDb = () => Inject(DB);
|
||||
|
||||
@ -1,4 +1,5 @@
|
||||
import { Injectable, NotFoundException } from "@nestjs/common";
|
||||
import { schema } from "@ordinarthur-os/db";
|
||||
import {
|
||||
Job,
|
||||
JobIngestDto,
|
||||
@ -8,61 +9,60 @@ import {
|
||||
JobSearchCriteria,
|
||||
JobSearchCriteriaUpsert,
|
||||
} from "@ordinarthur-os/shared";
|
||||
import { InjectSupabase, type Supabase } from "../../db/db.module";
|
||||
import { and, arrayOverlaps, asc, desc, eq, gte, lt, sql } from "drizzle-orm";
|
||||
import { InjectDb, type Db } from "../../db/db.module";
|
||||
|
||||
const { jobs, jobSearchCriteria } = schema;
|
||||
const RETENTION_DAYS = 30;
|
||||
|
||||
@Injectable()
|
||||
export class JobsService {
|
||||
constructor(@InjectSupabase() private readonly db: Supabase) {}
|
||||
constructor(@InjectDb() private readonly db: Db) {}
|
||||
|
||||
async ingest(jobs: JobIngestDto[]): Promise<JobIngestResponse> {
|
||||
// ---- ingest / list / patch --------------------------------------------
|
||||
|
||||
async ingest(input: JobIngestDto[]): Promise<JobIngestResponse> {
|
||||
let inserted = 0;
|
||||
let updated = 0;
|
||||
|
||||
// On traite job par job pour distinguer insert vs update.
|
||||
// Volume attendu : ~quelques dizaines/jour, c'est OK.
|
||||
for (const j of jobs) {
|
||||
const { data: existing } = await this.db
|
||||
.from("jobs")
|
||||
.select("id")
|
||||
.eq("source_url", j.source_url)
|
||||
.maybeSingle();
|
||||
|
||||
if (existing) {
|
||||
await this.db
|
||||
.from("jobs")
|
||||
.update({
|
||||
title: j.title,
|
||||
company: j.company ?? null,
|
||||
description: j.description ?? null,
|
||||
location: j.location ?? null,
|
||||
remote_type: j.remote_type ?? null,
|
||||
salary_min: j.salary_min ?? null,
|
||||
salary_max: j.salary_max ?? null,
|
||||
stack: j.stack ?? [],
|
||||
apply_url: j.apply_url ?? null,
|
||||
last_seen_at: new Date().toISOString(),
|
||||
})
|
||||
.eq("id", existing.id);
|
||||
updated++;
|
||||
} else {
|
||||
const { error } = await this.db.from("jobs").insert({
|
||||
// Upsert sur `source_url` (clé de dedup). On insère et, en cas de conflit,
|
||||
// on met à jour les champs mutables + bump `last_seen_at`. `xmax` permet
|
||||
// de savoir si la ligne résulte d'un INSERT (xmax=0) ou d'un UPDATE (xmax≠0).
|
||||
for (const j of input) {
|
||||
const rows = await this.db
|
||||
.insert(jobs)
|
||||
.values({
|
||||
source: j.source,
|
||||
source_url: j.source_url,
|
||||
sourceUrl: j.source_url,
|
||||
title: j.title,
|
||||
company: j.company ?? null,
|
||||
description: j.description ?? null,
|
||||
location: j.location ?? null,
|
||||
remote_type: j.remote_type ?? null,
|
||||
salary_min: j.salary_min ?? null,
|
||||
salary_max: j.salary_max ?? null,
|
||||
remoteType: j.remote_type ?? null,
|
||||
salaryMin: j.salary_min ?? null,
|
||||
salaryMax: j.salary_max ?? null,
|
||||
stack: j.stack ?? [],
|
||||
apply_url: j.apply_url ?? null,
|
||||
});
|
||||
if (error) throw error;
|
||||
inserted++;
|
||||
}
|
||||
applyUrl: j.apply_url ?? null,
|
||||
})
|
||||
.onConflictDoUpdate({
|
||||
target: jobs.sourceUrl,
|
||||
set: {
|
||||
title: j.title,
|
||||
company: j.company ?? null,
|
||||
description: j.description ?? null,
|
||||
location: j.location ?? null,
|
||||
remoteType: j.remote_type ?? null,
|
||||
salaryMin: j.salary_min ?? null,
|
||||
salaryMax: j.salary_max ?? null,
|
||||
stack: j.stack ?? [],
|
||||
applyUrl: j.apply_url ?? null,
|
||||
lastSeenAt: new Date(),
|
||||
},
|
||||
})
|
||||
.returning({ wasUpdate: sql<boolean>`xmax <> 0` });
|
||||
|
||||
if (rows[0]?.wasUpdate) updated++;
|
||||
else inserted++;
|
||||
}
|
||||
|
||||
const archived = await this.archiveStale();
|
||||
@ -71,79 +71,147 @@ export class JobsService {
|
||||
|
||||
/** Soft-delete des jobs non revus depuis RETENTION_DAYS jours. */
|
||||
private async archiveStale(): Promise<number> {
|
||||
const cutoff = new Date(Date.now() - RETENTION_DAYS * 86400_000).toISOString();
|
||||
const { data, error } = await this.db
|
||||
.from("jobs")
|
||||
.update({ archived: true })
|
||||
.lt("last_seen_at", cutoff)
|
||||
.eq("archived", false)
|
||||
.select("id");
|
||||
if (error) throw error;
|
||||
return data?.length ?? 0;
|
||||
const cutoff = new Date(Date.now() - RETENTION_DAYS * 86400_000);
|
||||
const rows = await this.db
|
||||
.update(jobs)
|
||||
.set({ archived: true })
|
||||
.where(and(lt(jobs.lastSeenAt, cutoff), eq(jobs.archived, false)))
|
||||
.returning({ id: jobs.id });
|
||||
return rows.length;
|
||||
}
|
||||
|
||||
async list(q: JobListQuery): Promise<Job[]> {
|
||||
let query = this.db.from("jobs").select("*").order("last_seen_at", { ascending: false });
|
||||
const filters = [eq(jobs.archived, q.archived ?? false)];
|
||||
|
||||
if (typeof q.archived === "boolean") query = query.eq("archived", q.archived);
|
||||
else query = query.eq("archived", false); // défaut : non archivés
|
||||
if (q.remote_type) filters.push(eq(jobs.remoteType, q.remote_type));
|
||||
if (q.starred !== undefined) filters.push(eq(jobs.starred, q.starred));
|
||||
if (q.since) filters.push(gte(jobs.lastSeenAt, new Date(q.since)));
|
||||
if (q.stack?.length) filters.push(arrayOverlaps(jobs.stack, q.stack));
|
||||
|
||||
if (q.remote_type) query = query.eq("remote_type", q.remote_type);
|
||||
if (q.starred !== undefined) query = query.eq("starred", q.starred);
|
||||
if (q.since) query = query.gte("last_seen_at", q.since);
|
||||
if (q.stack?.length) query = query.overlaps("stack", q.stack);
|
||||
const rows = await this.db
|
||||
.select()
|
||||
.from(jobs)
|
||||
.where(and(...filters))
|
||||
.orderBy(desc(jobs.lastSeenAt))
|
||||
.limit(500);
|
||||
|
||||
const { data, error } = await query.limit(500);
|
||||
if (error) throw error;
|
||||
return (data ?? []) as Job[];
|
||||
return rows.map(toJobDto);
|
||||
}
|
||||
|
||||
async patch(id: string, patch: JobPatchDto): Promise<Job> {
|
||||
const { data, error } = await this.db
|
||||
.from("jobs")
|
||||
.update(patch)
|
||||
.eq("id", id)
|
||||
.select("*")
|
||||
.maybeSingle();
|
||||
if (error) throw error;
|
||||
if (!data) throw new NotFoundException(`Job ${id} not found`);
|
||||
return data as Job;
|
||||
const rows = await this.db
|
||||
.update(jobs)
|
||||
.set({
|
||||
...(patch.starred !== undefined && { starred: patch.starred }),
|
||||
...(patch.archived !== undefined && { archived: patch.archived }),
|
||||
...(patch.applied_at !== undefined && {
|
||||
appliedAt: patch.applied_at === null ? null : new Date(patch.applied_at),
|
||||
}),
|
||||
...(patch.notes !== undefined && { notes: patch.notes }),
|
||||
})
|
||||
.where(eq(jobs.id, id))
|
||||
.returning();
|
||||
|
||||
const row = rows[0];
|
||||
if (!row) throw new NotFoundException(`Job ${id} not found`);
|
||||
return toJobDto(row);
|
||||
}
|
||||
|
||||
// ---- criteria ----------------------------------------------------------
|
||||
|
||||
async listCriteria(activeOnly = false): Promise<JobSearchCriteria[]> {
|
||||
let q = this.db.from("job_search_criteria").select("*").order("created_at", { ascending: true });
|
||||
if (activeOnly) q = q.eq("active", true);
|
||||
const { data, error } = await q;
|
||||
if (error) throw error;
|
||||
return (data ?? []) as JobSearchCriteria[];
|
||||
const rows = await (activeOnly
|
||||
? this.db
|
||||
.select()
|
||||
.from(jobSearchCriteria)
|
||||
.where(eq(jobSearchCriteria.active, true))
|
||||
.orderBy(asc(jobSearchCriteria.createdAt))
|
||||
: this.db.select().from(jobSearchCriteria).orderBy(asc(jobSearchCriteria.createdAt)));
|
||||
|
||||
return rows.map(toCriteriaDto);
|
||||
}
|
||||
|
||||
async createCriteria(input: JobSearchCriteriaUpsert): Promise<JobSearchCriteria> {
|
||||
const { data, error } = await this.db
|
||||
.from("job_search_criteria")
|
||||
.insert(input)
|
||||
.select("*")
|
||||
.single();
|
||||
if (error) throw error;
|
||||
return data as JobSearchCriteria;
|
||||
const rows = await this.db
|
||||
.insert(jobSearchCriteria)
|
||||
.values({
|
||||
name: input.name ?? null,
|
||||
titles: input.titles,
|
||||
locations: input.locations,
|
||||
stack: input.stack,
|
||||
remoteTypes: input.remote_types,
|
||||
salaryMin: input.salary_min ?? null,
|
||||
active: input.active,
|
||||
})
|
||||
.returning();
|
||||
return toCriteriaDto(rows[0]!);
|
||||
}
|
||||
|
||||
async updateCriteria(id: string, input: JobSearchCriteriaUpsert): Promise<JobSearchCriteria> {
|
||||
const { data, error } = await this.db
|
||||
.from("job_search_criteria")
|
||||
.update({ ...input, updated_at: new Date().toISOString() })
|
||||
.eq("id", id)
|
||||
.select("*")
|
||||
.maybeSingle();
|
||||
if (error) throw error;
|
||||
if (!data) throw new NotFoundException(`Criteria ${id} not found`);
|
||||
return data as JobSearchCriteria;
|
||||
const rows = await this.db
|
||||
.update(jobSearchCriteria)
|
||||
.set({
|
||||
name: input.name ?? null,
|
||||
titles: input.titles,
|
||||
locations: input.locations,
|
||||
stack: input.stack,
|
||||
remoteTypes: input.remote_types,
|
||||
salaryMin: input.salary_min ?? null,
|
||||
active: input.active,
|
||||
updatedAt: new Date(),
|
||||
})
|
||||
.where(eq(jobSearchCriteria.id, id))
|
||||
.returning();
|
||||
|
||||
const row = rows[0];
|
||||
if (!row) throw new NotFoundException(`Criteria ${id} not found`);
|
||||
return toCriteriaDto(row);
|
||||
}
|
||||
|
||||
async deleteCriteria(id: string): Promise<void> {
|
||||
const { error } = await this.db.from("job_search_criteria").delete().eq("id", id);
|
||||
if (error) throw error;
|
||||
await this.db.delete(jobSearchCriteria).where(eq(jobSearchCriteria.id, id));
|
||||
}
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Mapping row (snake_case en DB, camelCase en drizzle) → DTO (snake_case
|
||||
// côté API — la PWA consomme telle quelle).
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
function toJobDto(row: schema.JobRow): Job {
|
||||
return {
|
||||
id: row.id,
|
||||
source: row.source,
|
||||
source_url: row.sourceUrl,
|
||||
title: row.title,
|
||||
company: row.company,
|
||||
description: row.description,
|
||||
location: row.location,
|
||||
remote_type: row.remoteType as Job["remote_type"],
|
||||
salary_min: row.salaryMin,
|
||||
salary_max: row.salaryMax,
|
||||
stack: row.stack,
|
||||
apply_url: row.applyUrl,
|
||||
first_seen_at: row.firstSeenAt.toISOString(),
|
||||
last_seen_at: row.lastSeenAt.toISOString(),
|
||||
archived: row.archived,
|
||||
starred: row.starred,
|
||||
applied_at: row.appliedAt?.toISOString() ?? null,
|
||||
notes: row.notes,
|
||||
};
|
||||
}
|
||||
|
||||
function toCriteriaDto(row: schema.JobSearchCriteriaRow): JobSearchCriteria {
|
||||
return {
|
||||
id: row.id,
|
||||
name: row.name,
|
||||
titles: row.titles,
|
||||
locations: row.locations,
|
||||
stack: row.stack,
|
||||
remote_types: row.remoteTypes as JobSearchCriteria["remote_types"],
|
||||
salary_min: row.salaryMin,
|
||||
active: row.active,
|
||||
created_at: row.createdAt.toISOString(),
|
||||
updated_at: row.updatedAt.toISOString(),
|
||||
};
|
||||
}
|
||||
|
||||
@ -46,7 +46,7 @@ function Dashboard() {
|
||||
<div className="flex flex-wrap gap-2">
|
||||
<DataChip>VITE · REACT</DataChip>
|
||||
<DataChip>NESTJS</DataChip>
|
||||
<DataChip>SUPABASE</DataChip>
|
||||
<DataChip>POSTGRES · DRIZZLE</DataChip>
|
||||
<DataChip>K3S</DataChip>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
@ -2,12 +2,23 @@
|
||||
|
||||
Manifests Kubernetes pour le k3s perso d'Arthur.
|
||||
|
||||
## Stack
|
||||
|
||||
- `postgres.yaml` — StatefulSet Postgres 16 + PVC 5 Gi (single-user, faible volume).
|
||||
- `api.deployment.yaml` / `pwa.deployment.yaml` — services applicatifs.
|
||||
- `ingress.yaml` — routage Traefik TLS vers `os.arthurbarre.fr` et `api.os.arthurbarre.fr`.
|
||||
- `migrate.job.yaml` — job one-shot `drizzle-orm` pour appliquer les migrations.
|
||||
- `backup.cronjob.yaml` — `pg_dump` quotidien vers un bucket S3-compatible via `rclone`.
|
||||
|
||||
## Ordre d'application initial
|
||||
|
||||
```bash
|
||||
kubectl apply -f namespace.yaml
|
||||
# Copier secrets.template.yaml -> secrets.yaml, remplir, puis :
|
||||
kubectl apply -f secrets.yaml
|
||||
kubectl apply -f postgres.yaml
|
||||
# Attendre que le pod postgres soit Ready.
|
||||
kubectl apply -f migrate.job.yaml # crée le schéma ordinarthur_os + tables
|
||||
kubectl apply -f api.deployment.yaml
|
||||
kubectl apply -f pwa.deployment.yaml
|
||||
kubectl apply -f ingress.yaml
|
||||
@ -20,5 +31,7 @@ kubectl apply -f backup.cronjob.yaml
|
||||
- Cluster issuer cert-manager (`letsencrypt-prod` ?)
|
||||
- Entrée Traefik (`websecure` ?)
|
||||
- DNS : `os.arthurbarre.fr` et `api.os.arthurbarre.fr` doivent pointer sur l'IP du load-balancer k3s
|
||||
- StorageClass du PVC postgres (k3s fournit `local-path` par défaut — OK pour single-node)
|
||||
- Bucket S3-compatible pour les backups (B2 / Scaleway / autre)
|
||||
- Image registry : Gitea CR (défaut) — credentials pull peuvent nécessiter un `imagePullSecrets`
|
||||
- Image pour le `migrate.job` : soit re-utiliser l'image API et `node ./node_modules/.../migrate.ts` via tsx, soit builder une image dédiée `ordinarthur-os-migrate` qui embarque `packages/db` compilé.
|
||||
|
||||
@ -1,6 +1,7 @@
|
||||
# Backup quotidien du schéma ordinarthur_os.
|
||||
# `PGURL` et `RCLONE_REMOTE` à fournir via secret séparé `ordinarthur-os-backup-secrets`
|
||||
# (voir secrets.template.yaml — à splitter quand le bucket S3 est choisi avec Arthur).
|
||||
# Backup quotidien du Postgres ordinarthur-os.
|
||||
# Secrets dans `ordinarthur-os-backup-secrets` :
|
||||
# PGURL → chaîne `postgres://…` (copie de DATABASE_URL)
|
||||
# RCLONE_REMOTE → ex. `b2:ordinarthur-os-backups`
|
||||
apiVersion: batch/v1
|
||||
kind: CronJob
|
||||
metadata:
|
||||
|
||||
27
deploy/k8s/migrate.job.yaml
Normal file
27
deploy/k8s/migrate.job.yaml
Normal file
@ -0,0 +1,27 @@
|
||||
# Job one-shot qui applique les migrations Drizzle.
|
||||
# À rejouer manuellement après chaque déploiement qui contient une migration :
|
||||
# kubectl -n ordinarthur-os delete job ordinarthur-os-migrate --ignore-not-found
|
||||
# kubectl -n ordinarthur-os apply -f migrate.job.yaml
|
||||
#
|
||||
# (Peut aussi être branché dans le pipeline Gitea pour être auto-déclenché.)
|
||||
apiVersion: batch/v1
|
||||
kind: Job
|
||||
metadata:
|
||||
name: ordinarthur-os-migrate
|
||||
namespace: ordinarthur-os
|
||||
spec:
|
||||
backoffLimit: 2
|
||||
ttlSecondsAfterFinished: 86400
|
||||
template:
|
||||
spec:
|
||||
restartPolicy: OnFailure
|
||||
containers:
|
||||
- name: migrate
|
||||
image: gitea.arthurbarre.fr/arthurbarre/ordinarthur-os-migrate:latest
|
||||
imagePullPolicy: Always
|
||||
envFrom:
|
||||
- secretRef: { name: ordinarthur-os-secrets }
|
||||
command: ["node", "dist/migrate.js"]
|
||||
resources:
|
||||
requests: { cpu: 50m, memory: 128Mi }
|
||||
limits: { cpu: 300m, memory: 256Mi }
|
||||
66
deploy/k8s/postgres.yaml
Normal file
66
deploy/k8s/postgres.yaml
Normal file
@ -0,0 +1,66 @@
|
||||
# Postgres standalone pour ordinarthur-os.
|
||||
# Single-user, faible volume → 1 replica + PVC. Backup via backup.cronjob.yaml.
|
||||
#
|
||||
# Secrets attendus dans `ordinarthur-os-db-secrets` (cf. secrets.template.yaml) :
|
||||
# POSTGRES_USER, POSTGRES_PASSWORD, POSTGRES_DB
|
||||
#
|
||||
# `DATABASE_URL` consommé par l'API est injecté depuis `ordinarthur-os-secrets`
|
||||
# et doit pointer vers `postgres.ordinarthur-os.svc.cluster.local:5432/<POSTGRES_DB>`.
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
name: postgres
|
||||
namespace: ordinarthur-os
|
||||
spec:
|
||||
clusterIP: None
|
||||
selector: { app: postgres }
|
||||
ports:
|
||||
- name: postgres
|
||||
port: 5432
|
||||
targetPort: 5432
|
||||
---
|
||||
apiVersion: apps/v1
|
||||
kind: StatefulSet
|
||||
metadata:
|
||||
name: postgres
|
||||
namespace: ordinarthur-os
|
||||
spec:
|
||||
serviceName: postgres
|
||||
replicas: 1
|
||||
selector: { matchLabels: { app: postgres } }
|
||||
template:
|
||||
metadata: { labels: { app: postgres } }
|
||||
spec:
|
||||
containers:
|
||||
- name: postgres
|
||||
image: postgres:16-alpine
|
||||
ports:
|
||||
- containerPort: 5432
|
||||
name: postgres
|
||||
envFrom:
|
||||
- secretRef: { name: ordinarthur-os-db-secrets }
|
||||
env:
|
||||
- { name: PGDATA, value: /var/lib/postgresql/data/pgdata }
|
||||
volumeMounts:
|
||||
- name: data
|
||||
mountPath: /var/lib/postgresql/data
|
||||
readinessProbe:
|
||||
exec: { command: ["pg_isready", "-U", "$(POSTGRES_USER)", "-d", "$(POSTGRES_DB)"] }
|
||||
initialDelaySeconds: 5
|
||||
periodSeconds: 10
|
||||
livenessProbe:
|
||||
exec: { command: ["pg_isready", "-U", "$(POSTGRES_USER)", "-d", "$(POSTGRES_DB)"] }
|
||||
initialDelaySeconds: 30
|
||||
periodSeconds: 30
|
||||
resources:
|
||||
requests: { cpu: 50m, memory: 128Mi }
|
||||
limits: { cpu: 1000m, memory: 512Mi }
|
||||
volumeClaimTemplates:
|
||||
- metadata:
|
||||
name: data
|
||||
spec:
|
||||
accessModes: ["ReadWriteOnce"]
|
||||
resources:
|
||||
requests: { storage: 5Gi }
|
||||
# storageClassName: à définir selon le cluster (local-path par défaut sur k3s)
|
||||
@ -1,6 +1,7 @@
|
||||
# NE PAS COMMITER LES VRAIES VALEURS.
|
||||
# Ce fichier est un template — appliquer une copie remplie via :
|
||||
# Deux Secrets sont attendus côté cluster — dupliquer, remplir, puis :
|
||||
# kubectl -n ordinarthur-os apply -f secrets.yaml
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: Secret
|
||||
metadata:
|
||||
@ -9,9 +10,9 @@ metadata:
|
||||
type: Opaque
|
||||
stringData:
|
||||
API_BEARER_TOKEN: ""
|
||||
SUPABASE_URL: "https://supabase.arthurbarre.fr"
|
||||
SUPABASE_SERVICE_ROLE_KEY: ""
|
||||
SUPABASE_SCHEMA: "ordinarthur_os"
|
||||
# Postgres standalone dans le cluster (cf. postgres.yaml).
|
||||
# Format : postgres://<user>:<password>@postgres.ordinarthur-os.svc.cluster.local:5432/<db>
|
||||
DATABASE_URL: ""
|
||||
MISTRAL_API_KEY: ""
|
||||
MISTRAL_MODEL: "mistral-small-latest"
|
||||
GROQ_API_KEY: ""
|
||||
@ -22,3 +23,31 @@ stringData:
|
||||
ICAL_FEED_SECRET: ""
|
||||
TELEGRAM_BOT_TOKEN: ""
|
||||
TELEGRAM_WEBHOOK_SECRET: ""
|
||||
---
|
||||
# Credentials consommés par le StatefulSet postgres.
|
||||
# Les mêmes valeurs doivent composer DATABASE_URL ci-dessus.
|
||||
apiVersion: v1
|
||||
kind: Secret
|
||||
metadata:
|
||||
name: ordinarthur-os-db-secrets
|
||||
namespace: ordinarthur-os
|
||||
type: Opaque
|
||||
stringData:
|
||||
POSTGRES_USER: "ordinarthur"
|
||||
POSTGRES_PASSWORD: ""
|
||||
POSTGRES_DB: "ordinarthur_os"
|
||||
---
|
||||
# Credentials du CronJob de backup (bucket S3-compatible à choisir avec Arthur).
|
||||
apiVersion: v1
|
||||
kind: Secret
|
||||
metadata:
|
||||
name: ordinarthur-os-backup-secrets
|
||||
namespace: ordinarthur-os
|
||||
type: Opaque
|
||||
stringData:
|
||||
# Même valeur que DATABASE_URL (utilisable par pg_dump).
|
||||
PGURL: ""
|
||||
# rclone remote name + bucket, ex. "b2:ordinarthur-os-backups"
|
||||
RCLONE_REMOTE: ""
|
||||
# Contenu d'un rclone.conf — monté ensuite côté cronjob si besoin.
|
||||
RCLONE_CONFIG: ""
|
||||
|
||||
35
docker-compose.yml
Normal file
35
docker-compose.yml
Normal file
@ -0,0 +1,35 @@
|
||||
# Compose dev-only pour ordinarthur-os.
|
||||
# Pas destiné à la prod (la prod tourne sur k3s via deploy/k8s/postgres.yaml).
|
||||
#
|
||||
# Usage :
|
||||
# docker compose up -d # lance Postgres sur :5432
|
||||
# pnpm --filter @ordinarthur-os/db migrate # applique les migrations
|
||||
# docker compose logs -f postgres # suivre les logs
|
||||
# docker compose down # stop (garde le volume)
|
||||
# docker compose down -v # stop + wipe data
|
||||
#
|
||||
# Les credentials matchent `apps/api/.env.example` pour que `DATABASE_URL`
|
||||
# fonctionne sans config supplémentaire.
|
||||
|
||||
services:
|
||||
postgres:
|
||||
image: postgres:16-alpine
|
||||
container_name: ordinarthur-os-postgres
|
||||
restart: unless-stopped
|
||||
ports:
|
||||
- "5432:5432"
|
||||
environment:
|
||||
POSTGRES_USER: ordinarthur
|
||||
POSTGRES_PASSWORD: changeme
|
||||
POSTGRES_DB: ordinarthur_os
|
||||
volumes:
|
||||
- postgres-data:/var/lib/postgresql/data
|
||||
healthcheck:
|
||||
test: ["CMD-SHELL", "pg_isready -U ordinarthur -d ordinarthur_os"]
|
||||
interval: 5s
|
||||
timeout: 5s
|
||||
retries: 10
|
||||
|
||||
volumes:
|
||||
postgres-data:
|
||||
name: ordinarthur-os-postgres-data
|
||||
@ -1,15 +1,46 @@
|
||||
# @ordinarthur-os/db
|
||||
|
||||
Migrations SQL versionnées pour le schéma `ordinarthur_os` sur Supabase self-hosted.
|
||||
Schéma Postgres + migrations versionnées pour ordinarthur-os, via [Drizzle ORM](https://orm.drizzle.team).
|
||||
|
||||
Pas d'outil de migration automatique pour l'instant : appliquer manuellement via le SQL editor de Supabase, dans l'ordre numérique. Chaque fichier doit être idempotent autant que possible (`create … if not exists`).
|
||||
La base tourne dans le cluster k3s d'Arthur (cf. [`deploy/k8s/postgres.yaml`](../../deploy/k8s/postgres.yaml)). Plus de Supabase — l'API NestJS parle directement à Postgres.
|
||||
|
||||
## Ordre
|
||||
## Arbo
|
||||
|
||||
- `0001_schema.sql` — création du schéma + RLS service-role-only (Phase 0)
|
||||
- `0002_jobs.sql` — tables `jobs`, `job_search_criteria` (Phase 1)
|
||||
- `0003_todos.sql` — table `todos` + `client_mutations` (Phase 2)
|
||||
- `0004_projects.sql` — `projects`, `project_steps`, `project_ideas` (Phase 3)
|
||||
- `0005_agenda.sql` — `calendar_events`, `google_oauth_tokens` (Phase 4)
|
||||
- `0006_ai.sql` — `ai_actions` (Phase 5)
|
||||
- `0007_health.sql` — `daily_checkins` (Phase 7)
|
||||
- `src/schema/` — définitions Drizzle (TypeScript). Une table = un fichier, réexporté depuis `schema/index.ts`.
|
||||
- `src/client.ts` — factory `createDb(connectionString)` utilisée par l'API.
|
||||
- `src/migrate.ts` — runner des migrations (consomme `DATABASE_URL`).
|
||||
- `drizzle.config.ts` — config drizzle-kit.
|
||||
- `migrations/` — SQL versionné (`0000_init.sql`, `0001_jobs.sql`, …) + `meta/_journal.json` exploité par le runner.
|
||||
|
||||
## Commandes
|
||||
|
||||
Depuis la racine du monorepo :
|
||||
|
||||
```bash
|
||||
# 0. Lancer le Postgres de dev (cf. docker-compose.yml à la racine).
|
||||
docker compose up -d
|
||||
|
||||
# 1. Générer un diff SQL à partir du schéma TS (nouvelle migration).
|
||||
pnpm --filter @ordinarthur-os/db generate
|
||||
|
||||
# 2. Appliquer les migrations pendantes sur la base pointée par DATABASE_URL.
|
||||
# Le .env de apps/api suffit : le script lit process.env.DATABASE_URL.
|
||||
pnpm --filter @ordinarthur-os/db migrate
|
||||
|
||||
# 3. Ouvrir Drizzle Studio (inspection UI).
|
||||
pnpm --filter @ordinarthur-os/db studio
|
||||
```
|
||||
|
||||
## Convention migrations
|
||||
|
||||
- `0000_init.sql` — extension `pgcrypto` + création du schéma `ordinarthur_os`. Idempotent (à ne PAS régénérer via drizzle-kit).
|
||||
- `0001_jobs.sql` — tables `jobs` + `job_search_criteria` (Phase 1).
|
||||
- `0002_todos.sql` — `todos` + `client_mutations` (Phase 2).
|
||||
- `0003_projects.sql` — `projects`, `project_steps`, `project_ideas` (Phase 3).
|
||||
- `0004_agenda.sql` — `calendar_events`, `google_oauth_tokens` (Phase 4).
|
||||
- `0005_ai.sql` — `ai_actions` (Phase 5).
|
||||
- `0006_health.sql` — `daily_checkins` (Phase 7).
|
||||
|
||||
## Plus de RLS
|
||||
|
||||
Contrairement au setup Supabase initial, la DB n'est **pas** exposée publiquement : le seul consommateur est l'API NestJS, protégée par bearer token. On ne déploie donc aucune policy RLS — c'est du pur isolement réseau (service ClusterIP interne) + contrôle d'accès Postgres classique.
|
||||
|
||||
22
packages/db/drizzle.config.ts
Normal file
22
packages/db/drizzle.config.ts
Normal file
@ -0,0 +1,22 @@
|
||||
import type { Config } from "drizzle-kit";
|
||||
|
||||
/**
|
||||
* Config drizzle-kit.
|
||||
*
|
||||
* Usage :
|
||||
* - `pnpm --filter @ordinarthur-os/db generate` → génère un SQL diff dans `migrations/`
|
||||
* - `pnpm --filter @ordinarthur-os/db migrate` → applique les migrations pendantes
|
||||
*
|
||||
* `DATABASE_URL` attendu au format `postgres://user:pass@host:5432/dbname`.
|
||||
*/
|
||||
export default {
|
||||
schema: "./src/schema/index.ts",
|
||||
out: "./migrations",
|
||||
dialect: "postgresql",
|
||||
schemaFilter: ["ordinarthur_os"],
|
||||
dbCredentials: {
|
||||
url: process.env.DATABASE_URL ?? "",
|
||||
},
|
||||
strict: true,
|
||||
verbose: true,
|
||||
} satisfies Config;
|
||||
10
packages/db/migrations/0000_init.sql
Normal file
10
packages/db/migrations/0000_init.sql
Normal file
@ -0,0 +1,10 @@
|
||||
-- 0000_init.sql — Phase 0
|
||||
-- Pose le socle: extension pgcrypto + schéma dédié.
|
||||
-- Ces instructions sont idempotentes et sûres à ré-exécuter.
|
||||
|
||||
CREATE EXTENSION IF NOT EXISTS pgcrypto;
|
||||
|
||||
CREATE SCHEMA IF NOT EXISTS "ordinarthur_os";
|
||||
|
||||
COMMENT ON SCHEMA "ordinarthur_os" IS
|
||||
'Assistant personnel single-user d''Arthur. Accès applicatif exclusivement via l''API NestJS.';
|
||||
51
packages/db/migrations/0001_jobs.sql
Normal file
51
packages/db/migrations/0001_jobs.sql
Normal file
@ -0,0 +1,51 @@
|
||||
-- 0001_jobs.sql — Phase 1
|
||||
-- Tables jobs + job_search_criteria.
|
||||
|
||||
CREATE TABLE IF NOT EXISTS "ordinarthur_os"."jobs" (
|
||||
"id" uuid PRIMARY KEY DEFAULT gen_random_uuid() NOT NULL,
|
||||
"source" text NOT NULL,
|
||||
"source_url" text NOT NULL,
|
||||
"title" text NOT NULL,
|
||||
"company" text,
|
||||
"description" text,
|
||||
"location" text,
|
||||
"remote_type" text,
|
||||
"salary_min" integer,
|
||||
"salary_max" integer,
|
||||
"stack" text[] DEFAULT '{}'::text[] NOT NULL,
|
||||
"apply_url" text,
|
||||
"first_seen_at" timestamp with time zone DEFAULT now() NOT NULL,
|
||||
"last_seen_at" timestamp with time zone DEFAULT now() NOT NULL,
|
||||
"archived" boolean DEFAULT false NOT NULL,
|
||||
"starred" boolean DEFAULT false NOT NULL,
|
||||
"applied_at" timestamp with time zone,
|
||||
"notes" text,
|
||||
CONSTRAINT "jobs_source_url_unique" UNIQUE("source_url"),
|
||||
CONSTRAINT "jobs_remote_type_check"
|
||||
CHECK ("remote_type" IS NULL OR "remote_type" IN ('remote','hybrid','onsite'))
|
||||
);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS "jobs_last_seen_idx"
|
||||
ON "ordinarthur_os"."jobs" ("last_seen_at" DESC);
|
||||
CREATE INDEX IF NOT EXISTS "jobs_archived_idx"
|
||||
ON "ordinarthur_os"."jobs" ("archived");
|
||||
CREATE INDEX IF NOT EXISTS "jobs_remote_type_idx"
|
||||
ON "ordinarthur_os"."jobs" ("remote_type");
|
||||
CREATE INDEX IF NOT EXISTS "jobs_stack_gin"
|
||||
ON "ordinarthur_os"."jobs" USING gin ("stack");
|
||||
|
||||
CREATE TABLE IF NOT EXISTS "ordinarthur_os"."job_search_criteria" (
|
||||
"id" uuid PRIMARY KEY DEFAULT gen_random_uuid() NOT NULL,
|
||||
"name" text,
|
||||
"titles" text[] DEFAULT '{}'::text[] NOT NULL,
|
||||
"locations" text[] DEFAULT '{}'::text[] NOT NULL,
|
||||
"stack" text[] DEFAULT '{}'::text[] NOT NULL,
|
||||
"remote_types" text[] DEFAULT '{}'::text[] NOT NULL,
|
||||
"salary_min" integer,
|
||||
"active" boolean DEFAULT true NOT NULL,
|
||||
"created_at" timestamp with time zone DEFAULT now() NOT NULL,
|
||||
"updated_at" timestamp with time zone DEFAULT now() NOT NULL
|
||||
);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS "job_criteria_active_idx"
|
||||
ON "ordinarthur_os"."job_search_criteria" ("active");
|
||||
@ -1,20 +0,0 @@
|
||||
-- 0001_schema.sql — Phase 0
|
||||
-- Crée le schéma dédié `ordinarthur_os`. Les tables métier arrivent dans
|
||||
-- les migrations suivantes (0002+). On pose ici uniquement le socle.
|
||||
|
||||
create schema if not exists ordinarthur_os;
|
||||
|
||||
-- pgcrypto pour gen_random_uuid()
|
||||
create extension if not exists pgcrypto;
|
||||
|
||||
comment on schema ordinarthur_os is
|
||||
'Assistant personnel single-user d''Arthur. Accès via service_role uniquement.';
|
||||
|
||||
-- Helper appliqué à chaque table créée dans les migrations suivantes :
|
||||
--
|
||||
-- alter table ordinarthur_os.<t> enable row level security;
|
||||
-- create policy "<t>_service_role" on ordinarthur_os.<t>
|
||||
-- for all using (auth.role() = 'service_role')
|
||||
-- with check (auth.role() = 'service_role');
|
||||
--
|
||||
-- (Repris explicitement dans chaque migration de table.)
|
||||
@ -1,54 +0,0 @@
|
||||
-- 0002_jobs.sql — Phase 1
|
||||
-- Tables: jobs, job_search_criteria
|
||||
set search_path to ordinarthur_os, public;
|
||||
|
||||
create table if not exists ordinarthur_os.jobs (
|
||||
id uuid primary key default gen_random_uuid(),
|
||||
source text not null, -- 'Indeed','WeLoveDevs',...
|
||||
source_url text not null unique, -- clé de dedup
|
||||
title text not null,
|
||||
company text,
|
||||
description text,
|
||||
location text,
|
||||
remote_type text check (remote_type in ('remote','hybrid','onsite')),
|
||||
salary_min int,
|
||||
salary_max int,
|
||||
stack text[] not null default '{}',
|
||||
apply_url text,
|
||||
first_seen_at timestamptz not null default now(),
|
||||
last_seen_at timestamptz not null default now(),
|
||||
archived boolean not null default false,
|
||||
starred boolean not null default false,
|
||||
applied_at timestamptz,
|
||||
notes text
|
||||
);
|
||||
create index if not exists jobs_last_seen_idx on ordinarthur_os.jobs(last_seen_at desc);
|
||||
create index if not exists jobs_archived_idx on ordinarthur_os.jobs(archived);
|
||||
create index if not exists jobs_remote_type_idx on ordinarthur_os.jobs(remote_type);
|
||||
create index if not exists jobs_stack_gin on ordinarthur_os.jobs using gin (stack);
|
||||
|
||||
alter table ordinarthur_os.jobs enable row level security;
|
||||
drop policy if exists jobs_service_role on ordinarthur_os.jobs;
|
||||
create policy jobs_service_role on ordinarthur_os.jobs
|
||||
for all using (auth.role() = 'service_role')
|
||||
with check (auth.role() = 'service_role');
|
||||
|
||||
create table if not exists ordinarthur_os.job_search_criteria (
|
||||
id uuid primary key default gen_random_uuid(),
|
||||
name text,
|
||||
titles text[] not null default '{}',
|
||||
locations text[] not null default '{}',
|
||||
stack text[] not null default '{}',
|
||||
remote_types text[] not null default '{}',
|
||||
salary_min int,
|
||||
active boolean not null default true,
|
||||
created_at timestamptz not null default now(),
|
||||
updated_at timestamptz not null default now()
|
||||
);
|
||||
create index if not exists job_criteria_active_idx on ordinarthur_os.job_search_criteria(active);
|
||||
|
||||
alter table ordinarthur_os.job_search_criteria enable row level security;
|
||||
drop policy if exists job_criteria_service_role on ordinarthur_os.job_search_criteria;
|
||||
create policy job_criteria_service_role on ordinarthur_os.job_search_criteria
|
||||
for all using (auth.role() = 'service_role')
|
||||
with check (auth.role() = 'service_role');
|
||||
20
packages/db/migrations/meta/_journal.json
Normal file
20
packages/db/migrations/meta/_journal.json
Normal file
@ -0,0 +1,20 @@
|
||||
{
|
||||
"version": "7",
|
||||
"dialect": "postgresql",
|
||||
"entries": [
|
||||
{
|
||||
"idx": 0,
|
||||
"version": "7",
|
||||
"when": 1744848000000,
|
||||
"tag": "0000_init",
|
||||
"breakpoints": true
|
||||
},
|
||||
{
|
||||
"idx": 1,
|
||||
"version": "7",
|
||||
"when": 1744848060000,
|
||||
"tag": "0001_jobs",
|
||||
"breakpoints": true
|
||||
}
|
||||
]
|
||||
}
|
||||
@ -2,7 +2,27 @@
|
||||
"name": "@ordinarthur-os/db",
|
||||
"version": "0.0.0",
|
||||
"private": true,
|
||||
"type": "module",
|
||||
"main": "./src/index.ts",
|
||||
"types": "./src/index.ts",
|
||||
"exports": {
|
||||
".": "./src/index.ts",
|
||||
"./schema": "./src/schema/index.ts"
|
||||
},
|
||||
"scripts": {
|
||||
"migrate:print": "ls migrations"
|
||||
"generate": "drizzle-kit generate",
|
||||
"migrate": "tsx src/migrate.ts",
|
||||
"studio": "drizzle-kit studio",
|
||||
"typecheck": "tsc --noEmit"
|
||||
},
|
||||
"dependencies": {
|
||||
"drizzle-orm": "^0.36.4",
|
||||
"postgres": "^3.4.5"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@types/node": "^20.16.10",
|
||||
"drizzle-kit": "^0.28.1",
|
||||
"tsx": "^4.19.2",
|
||||
"typescript": "^5.5.4"
|
||||
}
|
||||
}
|
||||
|
||||
31
packages/db/src/client.ts
Normal file
31
packages/db/src/client.ts
Normal file
@ -0,0 +1,31 @@
|
||||
import { drizzle, type PostgresJsDatabase } from "drizzle-orm/postgres-js";
|
||||
import postgres, { type Sql } from "postgres";
|
||||
import * as schema from "./schema";
|
||||
|
||||
export type Db = PostgresJsDatabase<typeof schema>;
|
||||
|
||||
export interface DbHandle {
|
||||
db: Db;
|
||||
sql: Sql;
|
||||
close: () => Promise<void>;
|
||||
}
|
||||
|
||||
/**
|
||||
* Crée un client Drizzle connecté à Postgres.
|
||||
*
|
||||
* - `max` à 10 connexions (single-user, trafic très faible).
|
||||
* - `prepare: false` pour éviter les conflits avec les prepared statements côté
|
||||
* pool k3s (pas indispensable ici mais sans coût mesurable).
|
||||
*/
|
||||
export function createDb(connectionString: string): DbHandle {
|
||||
const sql = postgres(connectionString, {
|
||||
max: 10,
|
||||
prepare: false,
|
||||
});
|
||||
const db = drizzle(sql, { schema });
|
||||
return {
|
||||
db,
|
||||
sql,
|
||||
close: () => sql.end({ timeout: 5 }),
|
||||
};
|
||||
}
|
||||
3
packages/db/src/index.ts
Normal file
3
packages/db/src/index.ts
Normal file
@ -0,0 +1,3 @@
|
||||
export * as schema from "./schema";
|
||||
export { appSchema } from "./schema";
|
||||
export { createDb, type Db, type DbHandle } from "./client";
|
||||
29
packages/db/src/migrate.ts
Normal file
29
packages/db/src/migrate.ts
Normal file
@ -0,0 +1,29 @@
|
||||
/**
|
||||
* Exécute les migrations Drizzle pendantes.
|
||||
*
|
||||
* DATABASE_URL=postgres://... pnpm --filter @ordinarthur-os/db migrate
|
||||
*/
|
||||
import { migrate } from "drizzle-orm/postgres-js/migrator";
|
||||
import { createDb } from "./client";
|
||||
|
||||
async function main() {
|
||||
const url = process.env.DATABASE_URL;
|
||||
if (!url) {
|
||||
console.error("[migrate] DATABASE_URL manquant");
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
const { db, close } = createDb(url);
|
||||
try {
|
||||
console.log("[migrate] application des migrations…");
|
||||
await migrate(db, { migrationsFolder: new URL("../migrations", import.meta.url).pathname });
|
||||
console.log("[migrate] ✓ à jour");
|
||||
} finally {
|
||||
await close();
|
||||
}
|
||||
}
|
||||
|
||||
main().catch((err) => {
|
||||
console.error("[migrate] échec:", err);
|
||||
process.exit(1);
|
||||
});
|
||||
8
packages/db/src/schema/_schema.ts
Normal file
8
packages/db/src/schema/_schema.ts
Normal file
@ -0,0 +1,8 @@
|
||||
import { pgSchema } from "drizzle-orm/pg-core";
|
||||
|
||||
/**
|
||||
* Schéma Postgres dédié. Toutes les tables métier vivent ici.
|
||||
* L'extension pgcrypto (pour gen_random_uuid()) et le schéma lui-même sont
|
||||
* créés par la première migration SQL (cf. packages/db/migrations/0000_init.sql).
|
||||
*/
|
||||
export const appSchema = pgSchema("ordinarthur_os");
|
||||
2
packages/db/src/schema/index.ts
Normal file
2
packages/db/src/schema/index.ts
Normal file
@ -0,0 +1,2 @@
|
||||
export { appSchema } from "./_schema";
|
||||
export * from "./jobs";
|
||||
67
packages/db/src/schema/jobs.ts
Normal file
67
packages/db/src/schema/jobs.ts
Normal file
@ -0,0 +1,67 @@
|
||||
import { sql } from "drizzle-orm";
|
||||
import { boolean, index, integer, text, timestamp, uuid } from "drizzle-orm/pg-core";
|
||||
import { appSchema } from "./_schema";
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// jobs
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export const jobs = appSchema.table(
|
||||
"jobs",
|
||||
{
|
||||
id: uuid("id").primaryKey().default(sql`gen_random_uuid()`),
|
||||
source: text("source").notNull(), // 'Indeed','WeLoveDevs',...
|
||||
sourceUrl: text("source_url").notNull().unique(), // clé de dedup
|
||||
title: text("title").notNull(),
|
||||
company: text("company"),
|
||||
description: text("description"),
|
||||
location: text("location"),
|
||||
// 'remote' | 'hybrid' | 'onsite' — contraint côté applicatif (zod)
|
||||
remoteType: text("remote_type"),
|
||||
salaryMin: integer("salary_min"),
|
||||
salaryMax: integer("salary_max"),
|
||||
stack: text("stack").array().notNull().default(sql`'{}'::text[]`),
|
||||
applyUrl: text("apply_url"),
|
||||
firstSeenAt: timestamp("first_seen_at", { withTimezone: true }).notNull().defaultNow(),
|
||||
lastSeenAt: timestamp("last_seen_at", { withTimezone: true }).notNull().defaultNow(),
|
||||
archived: boolean("archived").notNull().default(false),
|
||||
starred: boolean("starred").notNull().default(false),
|
||||
appliedAt: timestamp("applied_at", { withTimezone: true }),
|
||||
notes: text("notes"),
|
||||
},
|
||||
(t) => ({
|
||||
lastSeenIdx: index("jobs_last_seen_idx").on(t.lastSeenAt.desc()),
|
||||
archivedIdx: index("jobs_archived_idx").on(t.archived),
|
||||
remoteTypeIdx: index("jobs_remote_type_idx").on(t.remoteType),
|
||||
stackGin: index("jobs_stack_gin").using("gin", t.stack),
|
||||
}),
|
||||
);
|
||||
|
||||
export type JobRow = typeof jobs.$inferSelect;
|
||||
export type JobInsert = typeof jobs.$inferInsert;
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// job_search_criteria
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export const jobSearchCriteria = appSchema.table(
|
||||
"job_search_criteria",
|
||||
{
|
||||
id: uuid("id").primaryKey().default(sql`gen_random_uuid()`),
|
||||
name: text("name"),
|
||||
titles: text("titles").array().notNull().default(sql`'{}'::text[]`),
|
||||
locations: text("locations").array().notNull().default(sql`'{}'::text[]`),
|
||||
stack: text("stack").array().notNull().default(sql`'{}'::text[]`),
|
||||
remoteTypes: text("remote_types").array().notNull().default(sql`'{}'::text[]`),
|
||||
salaryMin: integer("salary_min"),
|
||||
active: boolean("active").notNull().default(true),
|
||||
createdAt: timestamp("created_at", { withTimezone: true }).notNull().defaultNow(),
|
||||
updatedAt: timestamp("updated_at", { withTimezone: true }).notNull().defaultNow(),
|
||||
},
|
||||
(t) => ({
|
||||
activeIdx: index("job_criteria_active_idx").on(t.active),
|
||||
}),
|
||||
);
|
||||
|
||||
export type JobSearchCriteriaRow = typeof jobSearchCriteria.$inferSelect;
|
||||
export type JobSearchCriteriaInsert = typeof jobSearchCriteria.$inferInsert;
|
||||
12
packages/db/tsconfig.json
Normal file
12
packages/db/tsconfig.json
Normal file
@ -0,0 +1,12 @@
|
||||
{
|
||||
"extends": "../../tsconfig.base.json",
|
||||
"compilerOptions": {
|
||||
"module": "ESNext",
|
||||
"moduleResolution": "Bundler",
|
||||
"target": "ES2022",
|
||||
"outDir": "dist",
|
||||
"declaration": true,
|
||||
"noEmit": true
|
||||
},
|
||||
"include": ["src", "drizzle.config.ts"]
|
||||
}
|
||||
746
pnpm-lock.yaml
generated
746
pnpm-lock.yaml
generated
File diff suppressed because it is too large
Load Diff
Loading…
x
Reference in New Issue
Block a user