Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions .env.example
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,17 @@ NEXT_PUBLIC_BASE_URL=https://alpha.moss.land
# ALERT_WEBHOOK_URL=https://discord.com/api/webhooks/...
# HEALTH_ALERT_SUMMARY_HOUR=10

# ─── DB 백업 (scripts/backup-db.ts, 매일 03:00 KST) ──────────────────
# 스냅샷 위치. 기본값은 deploy.sh 의 배포 전 백업과 같은 ~/backups/alpha.
# BACKUP_DIR=/abs/path/backups
# 보관 개수 (기본 14). 매 실행이 DB 전체 사본 한 개를 추가한다.
# BACKUP_KEEP=14
# ★ 호스트 밖으로 미는 rsync 목적지. 이게 비어 있으면 원본과 사본이 같은
# 디스크에 있고, 호스트가 사라지면 둘 다 사라진다 — 백업이 아니라 실행
# 취소다. 설정해 두면 복사 실패가 /health 의 db_backup 을 빨갛게 만든다.
# BACKUP_REMOTE=user@backup-host:/srv/alpha-backups/
# BACKUP_RSYNC_BIN=rsync

# ─── (Optional) OAuth for community accounts ─────────────────────────
# SESSION_SECRET=
# KAKAO_OAUTH_CLIENT_ID=
Expand Down
33 changes: 25 additions & 8 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -1,12 +1,15 @@
# The three checks that gate a deploy, run on every PR and every push to main.
# The checks that gate a deploy, run on every PR and every push to main.
#
# Deliberately the SAME checks scripts/deploy.sh runs on the server before it
# swaps a release — typecheck, production build, and the health smoke test —
# so a red PR here is a deploy that would have been refused there, and a green
# one is a build the server will accept. Nothing here needs a secret: the smoke
# test builds its own throwaway SQLite schema, and none of the three call an
# external API. That matters because this repository is public and fork PRs
# run without secrets.
# Typecheck, tests, production build and the health smoke test are the SAME
# ones scripts/deploy.sh runs on the server before it swaps a release, so a
# red PR here is a deploy that would have been refused there, and a green one
# is a build the server will accept. The dependency audit is CI-only: a
# registry outage must not be able to block a deploy.
#
# Nothing here needs a secret — the smoke test builds its own throwaway SQLite
# schema, the tests use a temp file, and none of them call an external API
# (the audit talks to the registry, not to a service of ours). That matters
# because this repository is public and fork PRs run without secrets.
#
# `deploy.sh` reads this workflow's conclusion for the target SHA (see
# ci_conclusion there) and will not deploy a commit whose checks failed or are
Expand Down Expand Up @@ -60,9 +63,16 @@ jobs:
restore-keys: |
next-${{ runner.os }}-${{ hashFiles('pnpm-lock.yaml') }}-

# Covers scripts/ as well as the app. It did not until lib/script-env.ts
# removed the twenty copies of `process.env.NODE_ENV = …` that forced
# the directory out of tsconfig — so the cron scripts, where every
# production incident in this repo has started, were unchecked.
- name: Typecheck
run: pnpm exec tsc --noEmit

- name: Test
run: pnpm test

- name: Build
run: pnpm build
env:
Expand All @@ -79,3 +89,10 @@ jobs:
run: pnpm exec tsx scripts/check-health.ts
env:
MIC_DATA_PATH: ${{ runner.temp }}/mic-data

# Fails on a NEW high-severity advisory only; the ones already judged
# are listed, with reasons, in pnpm-workspace.yaml's auditConfig. The
# Next.js backlog this repo was carrying (30 advisories, 17 high, patch
# available) went unnoticed because nothing ever asked.
- name: Audit dependencies
run: pnpm audit --audit-level=high
78 changes: 71 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,18 @@ cp .env.example .env.local
pnpm dev # http://localhost:6900
```

The checks CI runs, and the ones `scripts/deploy.sh` re-runs on the server
before it swaps a release:

```bash
pnpm typecheck && pnpm test && pnpm build && pnpm tsx scripts/check-health.ts && pnpm audit:deps
```

`pnpm typecheck` covers `scripts/` as well as the app. `pnpm audit:deps` fails only
on a *new* high-severity advisory — the ones already assessed are listed with
their reasoning in `pnpm-workspace.yaml`, and that list is the thing to re-open
on every Next.js upgrade.

### Required env vars

| Var | Purpose | Default |
Expand Down Expand Up @@ -119,10 +131,26 @@ anything; `touch ~/alpha/.git/alpha-deploy-hold` pauses deploys (do this
tick); `--force` overrides the hold, the CI gate and the failure backoff. All
knobs are documented at the top of the script.

Because the poller deploys whatever reaches `main`, `main` should be
protected — PR required, at least one approval, no self-approval — before it
is switched on. Anyone with push access is otherwise one push away from
production.
Because the poller deploys whatever reaches `main`, what protects `main`
protects production. The `main` ruleset currently enforces: a PR (no direct
pushes), no deletion, no force-push, and the `checks` status check — so
nothing reaches production without CI green. It does **not** require a human
approval: `required_approving_review_count` is 0 and `require_last_push_approval`
is off, which means a merge is one click by whoever opened the PR.

Whether to close that gap depends on how many people can merge. With more than
one maintainer, require an approval and make it a real second pair of eyes:

```bash
gh api repos/MosslandOpenDevs/alpha/rulesets/20975561 | jq '{name,target,enforcement,conditions,rules:(.rules|map(if .type=="pull_request" then (.parameters.required_approving_review_count=1 | .parameters.require_last_push_approval=true) else . end))}' | gh api -X PUT repos/MosslandOpenDevs/alpha/rulesets/20975561 --input -
```

(Read-modify-write, so the other rules and the branch conditions survive; it
changes only the two approval fields.)

With a single maintainer that setting blocks every merge, including your own,
and the honest answer is that CI is the gate — say so here rather than
documenting a control nobody turned on.

Before restarting PM2 by hand, run the health smoke check:

Expand All @@ -148,9 +176,45 @@ When Alpha runs behind a reverse proxy or CDN, set `TRUSTED_PROXY_HOPS` to the n
The cron apps cover macro fetch, AI synthesis, daily brief, English brief
translation, persona ticks, persona replies, trackable call resolution,
why-moved article generation, entity connections, dynamic Q&A seeding,
IndexNow weekly ping, a weekly LLM-citation audit, and a health watchdog.
`ecosystem.config.cjs` is the list of record — `scripts/deploy.sh` reads the
app names from it rather than keeping its own copy.
IndexNow weekly ping, a weekly LLM-citation audit, a nightly verified DB
backup, and a health watchdog. `ecosystem.config.cjs` is the list of record —
`scripts/deploy.sh` reads the app names from it rather than keeping its own
copy.

### Backups and restore

`scripts/backup-db.ts` runs nightly at 03:00 KST. It snapshots the DB through
SQLite's own backup API (not `cp` — the DB is in WAL mode), opens the copy and
runs `PRAGMA integrity_check` on it, and, when `BACKUP_REMOTE` is set, rsyncs
it off the box. The result lands on `/health` as `db_backup`, so a backup that
stopped running, stopped verifying, or stopped leaving the host is visible
before you need it rather than after.

**Set `BACKUP_REMOTE`.** Without it the only copies of the DB are on the same
disk as the original, and `scripts/deploy.sh`'s pre-swap snapshot has the same
problem plus a recovery point of "whenever we last deployed". Community posts,
trackable calls and audit history cannot be regenerated from anywhere.

Run it once by hand after first deploying it — until it has run, `db_backup`
reads `fail` (correctly: no backup exists yet) and `?strict=1` answers 503:

```bash
pnpm tsx scripts/backup-db.ts
```

The drill that proves a snapshot is restorable — it reads the snapshot, never
production, so it is safe to run any time:

```bash
DB_PATH=<snapshot> pnpm tsx scripts/check-health.ts --live
```

The restore itself. The stale `-wal`/`-shm` beside the *original* must go
first, or SQLite replays them over the file you just restored:

```bash
pm2 stop all && rm -f "$DB_PATH-wal" "$DB_PATH-shm" && cp <snapshot> "$DB_PATH" && pm2 start all
```

## AI persona disclosure

Expand Down
10 changes: 10 additions & 0 deletions ecosystem.config.cjs
Original file line number Diff line number Diff line change
Expand Up @@ -161,6 +161,16 @@ module.exports = {
cronRestart: "0 4 * * *",
note: "매일 13:00 KST — call backfill + pending resolve",
}),
cronApp({
// The DB was only ever snapshotted by scripts/deploy.sh, immediately
// before a swap — so the recovery point was "the last deploy", and both
// copies lived on the same disk. This one runs daily, verifies what it
// wrote, and pushes it off the box when BACKUP_REMOTE is set.
name: "alpha-backup-cron",
script: "scripts/backup-db.ts --scheduled",
cronRestart: "0 18 * * *",
note: "매일 03:00 KST — DB 스냅샷 + 무결성 검사 + off-host 복사",
}),
cronApp({
// No --scheduled guard on purpose: this one is supposed to run at every
// registration. It only reads /api/health and speaks on a state change,
Expand Down
130 changes: 108 additions & 22 deletions lib/calls.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,9 @@
*
* 흐름:
* 1. 페르소나/사용자가 asset entity에 stance 글 작성
* 2. lib/calls.ts가 자동으로 call 레코드 생성
* 2. 글과 **같은 transaction 안에서** call 레코드 생성 (lib/persona-post.ts).
* 글만 남고 call 이 없는 상태는 /agents 가 지키지 못하는 약속이므로,
* reference price 는 글을 쓰기 전에 확보한다.
* - direction: agree → up, disagree → down, observe → skip
* - reference_price: 글 작성 시점 가격 (lib/prices.ts — 코인이면
* CoinGecko, 지수·원자재면 Yahoo)
Expand All @@ -15,7 +17,14 @@
* 4. handle별 적중률 누적 → /agents/[handle] 트랙레코드
*/

import {
createPost,
ensureCommunityTables,
type CreatePostArgs,
type Post,
} from "./community";
import { getDb } from "./db";
import { getAssetOrStub } from "./mic";
import { currentPrice, flatPctFor, isCallableAsset, marketFor, priceOn } from "./prices";

export type Direction = "up" | "down";
Expand Down Expand Up @@ -138,11 +147,8 @@ function nano(): string {
return crypto.randomUUID().replace(/-/g, "").slice(0, 16);
}

/**
* 새 글에 대해 call 레코드 생성 (이미 있으면 skip).
* Returns null if not callable (no asset, no stance, etc.)
*/
export async function createCallFromPost(post: {
/** The subset of a post a call is built from. */
export type CallSource = {
id: string;
ref_type: string;
ref_id: string | null;
Expand All @@ -151,39 +157,73 @@ export async function createCallFromPost(post: {
author_handle: string;
stance: string | null;
created_at: string;
}): Promise<TrackableCall | null> {
};

/** Create the table from outside, so a caller can do it before opening a
* transaction rather than running DDL inside one. */
export function ensureCallsTable(): void {
ensureTable();
}

/**
* Can this post carry a call, and in which direction? No network, no writes.
*
* Split out from the insert so a caller can ask the question *before* it
* spends money generating the post — lib/persona-post.ts fetches the
* reference price up front on the strength of this answer.
*/
export function callDirectionFor(post: CallSource): Direction | null {
// 자산 entity여야 함
if (post.ref_type !== "asset" || !post.ref_id) return null;
// 답글은 call 대상이 아니다 — 트랙레코드는 페이지에 대한 최초 판단만 센다.
// (백필 쿼리에도 같은 조건이 있지만, 여기서도 막아야 호출 경로가 늘어도 안전.)
if (post.parent_id) return null;
// stance가 있어야 함 (observe 제외)
if (!post.stance || post.stance === "observe" || post.stance === "neutral")
return null;
if (post.stance !== "agree" && post.stance !== "disagree") return null;
// 가격 출처가 없거나, 있어도 페그 자산이면 방향성 call 이 성립하지 않는다.
// (isCallableAsset 이 두 조건을 모두 본다.)
if (!isCallableAsset(post.ref_id)) return null;
return post.stance === "agree" ? "up" : "down";
}

/**
* Write the call for a post whose reference price is already in hand.
*
* Synchronous on purpose. better-sqlite3 transactions cannot await, and the
* only way a stance post and its call are guaranteed to exist together is to
* write them in one — see lib/persona-post.ts.
*
* Throws on an unusable price: by the time we are here the caller has claimed
* to have one, and silently dropping the call is the failure mode this split
* exists to remove.
*/
export function insertCallForPost(
post: CallSource,
referencePrice: number
): TrackableCall | null {
ensureTable();
const direction = callDirectionFor(post);
if (!direction) return null;
if (!Number.isFinite(referencePrice) || referencePrice <= 0) {
throw new Error(
`unusable reference price for ${post.ref_id}: ${referencePrice}`
);
}
// 이미 있으면 skip
const existing = getDb()
.prepare(`SELECT id FROM alpha_trackable_calls WHERE post_id = ?`)
.get(post.id);
if (existing) return null;

// 가격 출처가 없거나, 있어도 페그 자산이면 방향성 call 이 성립하지 않는다.
// (isCallableAsset 이 두 조건을 모두 본다.)
if (!isCallableAsset(post.ref_id)) return null;

const price = await currentPrice(post.ref_id);
if (price == null || !Number.isFinite(price) || price <= 0) return null;

const assetId = post.ref_id as string;
// getAssetOrStub, not getEntity: getAllEntities() reads the canonical store
// only, so an asset that lives as a stub (ethereum was one) resolves to null
// and the published call would carry the raw id — "ethereum" where the page
// says 이더리움.
const { getAssetOrStub } = await import("./mic");
const entity = getAssetOrStub(post.ref_id);
const assetLabel = entity?.label || post.ref_id;
const entity = getAssetOrStub(assetId);
const assetLabel = entity?.label || assetId;

const direction: Direction = post.stance === "agree" ? "up" : "down";
const price = referencePrice;
const refDate = new Date(post.created_at);
const targetDate = new Date(refDate.getTime() + DEFAULT_HORIZON_DAYS * 86400_000);

Expand All @@ -192,14 +232,14 @@ export async function createCallFromPost(post: {
post_id: post.id,
author_kind: post.author_kind === "agent" ? "agent" : "anonymous",
author_handle: post.author_handle,
asset_id: post.ref_id,
asset_id: assetId,
asset_label: assetLabel,
direction,
horizon_days: DEFAULT_HORIZON_DAYS,
reference_price: price,
reference_date: refDate.toISOString(),
target_date: targetDate.toISOString(),
flat_pct: flatPctFor(post.ref_id),
flat_pct: flatPctFor(assetId),
resolution_status: "pending",
resolution_price: null,
resolved_at: null,
Expand Down Expand Up @@ -239,6 +279,52 @@ export async function createCallFromPost(post: {
return call;
}

/**
* 글과 call 을 한 transaction 으로 쓴다 — 트랙레코드의 유일한 무결성 보장.
*
* A stance on a priceable asset IS a call: /agents publishes the record built
* from them, so a post that exists without its call is a promise the site
* cannot keep. Pass the reference price you already fetched (see
* lib/persona-post.ts, which fetches it before the model call) and both rows
* land together or neither does.
*
* `referencePrice` null means "this was never going to carry a call" — an
* unpriceable page, a dry stance — and the post is written on its own.
* insertCallForPost re-checks stance, asset and duplication, so the caller
* does not repeat those conditions and they cannot drift apart.
*/
export function createPostWithCall(
args: CreatePostArgs,
referencePrice: number | null
): Post {
ensureCommunityTables();
ensureTable();
return getDb().transaction(() => {
const post = createPost(args);
if (referencePrice != null) insertCallForPost(post, referencePrice);
return post;
})();
}

/**
* 새 글에 대해 call 레코드 생성 (이미 있으면 skip).
* Returns null if not callable (no asset, no stance, no price, …).
*
* The price is fetched here, so post and call cannot be written together.
* That is fine for the backfill path (scripts/track-calls.ts), which is
* recovering posts that already exist. A writer creating the post right now
* should pre-fetch the price and use insertCallForPost() inside its own
* transaction instead.
*/
export async function createCallFromPost(
post: CallSource
): Promise<TrackableCall | null> {
if (!callDirectionFor(post)) return null;
const price = await currentPrice(post.ref_id as string);
if (price == null || !Number.isFinite(price) || price <= 0) return null;
return insertCallForPost(post, price);
}

/** target_date 도달한 pending call resolve. */
export async function resolveCall(callId: string): Promise<TrackableCall | null> {
ensureTable();
Expand Down
Loading
Loading