Joysafeter v2 - #141
Open
yuzzjj wants to merge 958 commits into
Open
Joysafeter v2#141yuzzjj wants to merge 958 commits into
yuzzjj wants to merge 958 commits into
Conversation
GLei16
approved these changes
Jul 2, 2026
GLei16
previously approved these changes
Jul 2, 2026
…ct access _auth_via_api_key authenticated a key purely on its own not-revoked/not-expired state and never checked the creator's current standing; since the read path does not re-verify context, a key kept working for reads after its creator lost access to the project. It now confirms the creator is still an org member with access to the key's project (get_accessible_project) and raises AUTH_API_KEY_ACCESS_REVOKED otherwise, without rebuilding the key's capped identity. Registers the new code in the error catalog. Co-Authored-By: Claude-Opus-4.8[1m] <noreply@anthropic.com>
task_stream authorized a subscriber by org membership + project-in-org only, skipping the per-user project-access check the HTTP read path enforces, so an org member with no ProjectMember row could stream another project's task output over the websocket. The inline auth is extracted into a testable _authorize_task_stream helper that reuses get_accessible_project (org super-users org-wide, everyone else needs an explicit row); close codes and the streaming logic are unchanged. Co-Authored-By: Claude-Opus-4.8[1m] <noreply@anthropic.com>
The organization-members table had no uniqueness guarantee on (organization_id, user_id), so a race between two add_member calls could create duplicate rows; authorization then resolved the member with .limit(1) and no ordering, making a user's effective org role nondeterministic. Adds a unique constraint (model + migration that first de-dupes existing rows, keeping the highest-privilege one), and converts a lost-race IntegrityError in add_member/invite_member_by_email into ORGANIZATION_MEMBER_ALREADY_EXISTS (409). Co-Authored-By: Claude-Opus-4.8[1m] <noreply@anthropic.com>
…rrent insert grant_project_membership did an unguarded check-then-insert against the uniquely-constrained project-members table, so two concurrent grants for the same (project, user) surfaced the second insert's IntegrityError as a 500. On the standalone commit=True path it now rolls back, re-loads the winning row, applies the requested role, and commits (idempotent upsert); the commit=False path nested in a caller's transaction is unchanged. Co-Authored-By: Claude-Opus-4.8[1m] <noreply@anthropic.com>
… api-key, and skill Collapse the organization role to owner/admin/member (developer/viewer fold into member; org role now only answers "am I a super-user?"), keep the project capability vocabulary admin/editor/viewer as the single source of write/read/ admin via effective_project_capability, and reuse that vocabulary for api-key and skill-collaborator roles. Consolidate the scattered normalize/assignable-role logic (single ProjectRole.parse_strict; delete the dead OrgRole enum), fold the skill "publisher" tier into admin with the collaborator column moving off its bespoke PG enum, and lower the fail-toward-write grant defaults to viewer. Co-Authored-By: Claude-Opus-4.8[1m] <noreply@anthropic.com>
Fold org developer/viewer into member, reinterpret api-key roles in the project vocabulary (owner->admin, developer/member->editor), and move skill collaborators off the collaborator_role PG enum to varchar with publisher folded into admin. Co-Authored-By: Claude-Opus-4.8[1m] <noreply@anthropic.com>
Rename org-context developer/viewer seeds to member, map legacy ProjectMember seeds to editor, correct api-key capability-cap arguments to ProjectRole, drop a now-duplicate test, and fix stale VIEWER/developer comments left by the rename. Co-Authored-By: Claude-Opus-4.8[1m] <noreply@anthropic.com>
…y gating Collapse ManagedRole to owner/admin/member, drop the org-rank write notion (canWrite/canEdit/userPermissions) that stood in for project capability, gate the api-keys page purely on projectReadOnly with the project role selector, and rename the roleDeveloper label to roleMember. Co-Authored-By: Claude-Opus-4.8[1m] <noreply@anthropic.com>
Co-Authored-By: Claude-Opus-4.8[1m] <noreply@anthropic.com>
…asking gaps
Audit fixes for the joysafeter_api HTTP layer (multi-tenant, real pentest execution):
- CSRF: enforce signed double-submit verification on cookie-authenticated
mutations via a global CsrfProtectionMiddleware (the frontend already sends the
X-CSRF-Token header). Header/bearer-authenticated and safe requests are
unaffected; session-bootstrap endpoints stay exempt.
- session-stop: make POST /sessions/{id}/stop fail closed. Delegate to
TaskCancellationService (relay-before-DB, raises on an unconfirmed relay),
re-check remaining active tasks, and treat an already-terminal ValueError as an
idempotent stop instead of letting it surface as a 500.
- task idempotency: derive a short-window server-side Idempotency-Key when the
client omits the header, so an accidental double-submit does not fire the
tooling against a target twice.
- credential masking: default-deny. Mask every secret value except an allowlist
of display-safe config keys, and unconditionally redact vault tokens / oauth
secrets (removes the first-6-chars prefix leak); the update-preserving
predicate is kept in sync so echoing a masked value back does not corrupt it.
Co-Authored-By: Claude-Opus-4.8[1m] <noreply@anthropic.com>
… ACL Idempotent grant/upsert, revoke, list-with-users, and a role read helper, mirroring ProjectService member management. Stores the plain project-capability vocabulary string (admin/editor/viewer) and folds legacy publisher to admin. Co-Authored-By: Claude-Opus-4.8[1m] <noreply@anthropic.com>
…ield
Expose GET/POST/DELETE /skills/{id}/collaborators to grant, list, and revoke
per-skill collaborators. The management gate enforces org isolation (a cross-org
reference 404s before the super-user bypass), then requires skill-admin
capability (owner, an admin collaborator, or an org super-user of the skill's
own org); the skill owner cannot be modeled as a collaborator.
Also return the caller's effective capability (owner/admin/editor/viewer/none)
on the skill detail response via a non-raising compute_skill_capability helper
that mirrors the read gate's precedence, so the client can gate its UI.
Co-Authored-By: Claude-Opus-4.8[1m] <noreply@anthropic.com>
Add a capability-gated "Collaborators" tab on the skill detail view that lists, adds (via an org-user search picker), and removes collaborators with a role dropdown. Consumes the new caller `capability` field on SkillRecord and gates management on owner/admin via canManageSkillCollaborators. Includes en/zh strings and a colocated test. Co-Authored-By: Claude-Opus-4.8[1m] <noreply@anthropic.com>
… alive B1: include 'error' status in the stopped/destroy sweep so failure-ejected sandboxes (set to 'error' without a disconnected_at marker) are reclaimed to 'destroyed' instead of leaking their container, DB row, and Envoy listeners until the absolute hard timeout (or forever, if disabled). B2: touch_sandbox now also refreshes idle_since, not just last_used_at. A sandbox reused by the resolver keeps its 'idle' status (task attach never transitions it), so a stale idle_since let the idle reaper stop a sandbox mid-reuse (TOCTOU). Only the clean-idle criterion reads idle_since; the bridge-disconnect and hard-timeout criteria are unaffected. NOT cargo-verified: no Rust toolchain in the authoring environment. Run cargo check / cargo test before release. Co-Authored-By: Claude <noreply@anthropic.com>
… schedule triggers
- health: the unauthenticated GET /api/v1/health (and /ready) no longer exposes
orchestrator fleet topology (live/stale counts, heartbeat/expiry) or raw DB
exception text. The readiness body is reduced to up/down/degraded status
strings; 200/503 gating and "ok"-only-when-all-subsystems-ok semantics are
unchanged. /live stays a static liveness probe.
- schedules: POST /schedules/{id}/trigger now enforces the schedule owner's
per-user concurrency quota (enforce_user_quota=True), closing an unbounded,
quota-exempt task-spawn primitive. Run-as-owner semantics and the cron
auto-fire path (project-quota-only, by design) are unchanged.
Co-Authored-By: Claude-Opus-4.8[1m] <noreply@anthropic.com>
…mbing Delete six unreachable methods left over from the skill-service consolidation (import_skill_from_directory, _sync_skill_from_skill_md, get_skill_by_name, _detect_file_type, the sync rescan_skill superseded by rescan_skill_async, and SkillVersionService.get_latest_version) — ~220 lines with no callers. Drop the never-settable root_path parameter from create_skill/update_skill: no request schema exposes it, so it could only ever write NULL. The column is left in place (its removal needs a migration) but the dead plumbing is gone. Also remove the unused SkillCollaboratorService.get_collaborator_role helper (no production caller; check_skill_access uses its own lookup) and its test. Co-Authored-By: Claude-Opus-4.8[1m] <noreply@anthropic.com>
Add a Restore action to each row in the version-history tab: an explicit
confirm dialog (restoring replaces the current draft) that POSTs to
/skills/{id}/versions/restore/{version} and refreshes the skill, its files, and
the version list. This closes the gap where users could delete versions but had
no way to restore one.
Also reset the detail editor tab when the open skill changes, so a
capability-gated tab (Collaborators) can't keep rendering stale content after
switching to a skill the caller cannot manage; make types/managed the single
source for SkillCapability (roles.ts re-exports it); and delete the dead legacy
Skill interface in types.ts. en/zh strings included.
Co-Authored-By: Claude-Opus-4.8[1m] <noreply@anthropic.com>
Add a page test that selects a skill, opens the version-history tab, triggers
Restore on the latest version, confirms the dialog, and asserts the POST to
/skills/{id}/versions/restore/{version} — the restore UI shipped without
coverage in the prior commit.
Co-Authored-By: Claude-Opus-4.8[1m] <noreply@anthropic.com>
…ma layer P1-16 (task entry point): JoySafeterCreateTaskRequest.prompt/system_prompt had no max_length. The body-size middleware is only a coarse 64 MiB OOM guard; a single ~60 MiB prompt still reaches a DB row and is fanned out over Redis pub/sub to every SSE subscriber. Add a MAX_PROMPT_CHARS (1M) per-field cap so an oversized field is rejected with a clean 422 well below the body cap. Co-Authored-By: Claude-Opus-4.8[1m] <noreply@anthropic.com>
…y-size cap
P1-13a: _validate_state returned {} (a truthy non-reject value) when Redis was
unavailable or the lookup raised, silently disabling the OAuth login-CSRF nonce
check during an outage. Both fail-open branches now return None (the reject
sentinel) so an unverifiable state aborts the login.
P1-12: add an outermost pure-ASGI RequestBodySizeLimitMiddleware that rejects
oversized request bodies with 413 before they are buffered — via declared
Content-Length (no allocation) and via a streamed byte-counter (covers
chunked/understated length). Adds PayloadTooLargeError, the REQUEST_BODY_TOO_LARGE
catalog entry, the 413 status mapping, and the max_request_body_bytes setting
(64 MiB, above the 50 MiB upload cap).
Co-Authored-By: Claude-Opus-4.8[1m] <noreply@anthropic.com>
P1-16 (session entry point): _validate_message_content concatenates a user.message into the text that becomes an agent task prompt via the internal service path, bypassing JoySafeterCreateTaskRequest's schema cap. Bound the concatenated text at MAX_PROMPT_CHARS and reject an oversized message with a structured 422 (SESSION_CONTENT_TOO_LARGE), matching the task-prompt cap. Co-Authored-By: Claude-Opus-4.8[1m] <noreply@anthropic.com>
Promote the module-private _skill_org_id helper to a public resolve_skill_org_id and reuse it for the two inline select(Project.org_id) queries in the skill routes (_skill_superuser_scope, _load_manageable_skill), collapsing three copies of the org-resolution logic (and its no-project guard) into one. Update the two service tests that monkeypatch the resolver by string path to target the new name. Co-Authored-By: Claude-Opus-4.8[1m] <noreply@anthropic.com>
Extract a RoleSelect component in the collaborators panel to replace two identical role-picker Select blocks (row + add-form), and type canManageSkillCollaborators against the SkillCapability union instead of a raw string. Co-Authored-By: Claude-Opus-4.8[1m] <noreply@anthropic.com>
Envoy process hang (not crash) would leave sandbox egress broken with no recovery. Added K8s health probes on the Envoy admin endpoint: - livenessProbe: GET /ready :9901, period 5s, threshold 3 → Envoy unresponsive for 15s → K8s kills and recreates pod - readinessProbe: GET /ready :9901, period 3s, threshold 2 → Envoy not yet connected to ADS / listeners not warm → not ready Envoy's /ready endpoint returns 200 only when: - All listeners are initialized - Workers are ready to accept connections This ensures DaemonSet pod replacement on both crash AND hang scenarios. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…initContainer Completes the K8s provider's Envoy egress management, making it feature- equivalent to the Docker provider for credential injection and network isolation. Changes to k8s.rs: - capabilities(): has_egress_management=true, NetworkIsolation::Envoy when Envoy is enabled. This activates the full credential-injection pipeline (setup_networking, LDS push, transparent proxy mode). - build_manifest(): adds hostPath volume (/data/joysafeter/envoy-sockets → /sockets) so sandbox pod shares the per-node Envoy DaemonSet socket directory. Adds busybox initContainer that creates /sockets/<sandbox_id>/ with 777 perms (orchestrator can't mkdir on a remote node). - setup_networking(): delegates to EnvoyManager.setup_for_sandbox() to push the LDS listener (same as Docker provider). Skips prepare_socket_dir (handled by initContainer). - teardown_networking(): delegates to EnvoyManager.teardown_for_sandbox(). - on_startup(): initializes EnvoyManager LDS state + recovers from DB. No health monitor spawn (K8s livenessProbe on DaemonSet handles it). - Holds EnvoyManager + DeltaXdsServer (like DockerProvider), exposes xds_service() for gRPC ADS registration. Changes to main.rs: - K8s provider now extracts xds_service for gRPC server ADS registration. Result: K8s sandboxes now get full Envoy egress isolation — per-sandbox UDS socket, credential injection, DFP shared clusters, transparent mode, all via the same Envoy DaemonSet + hostPath mechanism described in the deployment YAML. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…isolation Fixes several issues that would cause K8s deployments to fail or have security gaps: 1. prepare_socket_dir skip (EnvoyConfig.skip_socket_dir_prep): In K8s mode, orchestrator cannot mkdir on a remote node. The flag makes prepare_socket_dir a no-op; the sandbox pod's initContainer creates the dir locally on the same node as the DaemonSet Envoy. 2. init_xds_only() — skip bootstrap write in K8s: Envoy DaemonSet uses embedded bootstrap (not orchestrator-written). K8s on_startup now calls init_xds_only() which only resets in-memory LDS state, avoiding write errors on non-existent config_dir. 3. initContainer image: envoy image (not busybox:latest): Uses the already-pulled envoy image (has sh) instead of an unversioned external dependency. No extra image pull needed. 4. subPath isolation for socket hostPath mount: Runner container mounts /sockets/<sandbox_id> (subPath) instead of the full /sockets/ directory. Sandbox can only see its own socket, not other sandboxes'. Combined with proxy_auth_token route matching, this is defense-in-depth. Docker provider unchanged (skip_socket_dir_prep=false, uses full init()). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… idle timeout) Added environment variables that have no sensible code-default for K8s deployments or are required for multi-instance coordination: - JOYSAFETER_LLM_EGRESS_ALLOWED_HOSTS: required for credential injection to know which upstream hosts are LLM gateways - JOYSAFETER_EVENT_STREAM_KEY/GROUP: shared Redis stream coordinates between orchestrator and worker - JOYSAFETER_REDIS_QUEUE_PREFIX: namespace isolation in shared Redis - JOYSAFETER_SANDBOX_IDLE_TIMEOUT: pod lifecycle management Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
deploy/k8s/env-reference.md documents how .env.example maps to K8s: - Layer 1: Secret (DATABASE_URL, REDIS_URL, vault key, etc.) - Layer 2: YAML env (all orchestrator non-sensitive config) - Layer 3: Defaults (don't set, code provides sensible values) - Not applicable (Docker Compose-only vars explained) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
settings.py builds the database URL by string-formatting POSTGRES_* env vars directly into the connection string. Special characters in POSTGRES_PASSWORD (@, #, !, etc.) broke the URL structure — asyncpg parsed the password's @ as the host delimiter, resulting in 'Name or service not known' (garbage hostname). Added urllib.parse.quote(safe='') on both user and password before interpolation, matching the DATABASE_URL convention that passwords must be percent-encoded. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Both Python (settings.py) and Rust (config.rs) now auto-encode user/password when building the database URL from POSTGRES_* env vars. Special chars in POSTGRES_PASSWORD (@, #, !) no longer break either. Users only need to set POSTGRES_PASSWORD with the raw password — no manual URL encoding needed. Both services read the same POSTGRES_* vars and produce the same result. If DATABASE_URL is set directly, it's used as-is (user must encode it themselves). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Same pattern as POSTGRES_*: users can now set REDIS_HOST, REDIS_PASSWORD,
REDIS_PORT, REDIS_DB, REDIS_SCHEME instead of manually encoding a
REDIS_URL. Both Python and Rust auto-encode the password.
Priority:
1. REDIS_URL (if set, used as-is — user must encode)
2. REDIS_HOST + REDIS_PASSWORD + REDIS_PORT + REDIS_DB (auto-encoded)
Python (settings.py):
- Added effective_redis_url property + model_post_init to populate
redis_url from REDIS_* when REDIS_URL is absent.
Rust (config.rs):
- Added build_redis_url() with same logic as build_database_url().
.env only needs:
REDIS_HOST=redis-host
REDIS_PASSWORD=raw password with @#! etc
REDIS_PORT=6379
REDIS_DB=0
No manual URL encoding needed for either service.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
URL-encoded passwords contain % chars (e.g. %40 for @). Python's configparser treats % as interpolation syntax (%(name)s), causing 'invalid interpolation syntax' when alembic sets the URL option. Fix: replace % with %% before passing to config.set_main_option(). configparser renders %% back to % at read time. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ed it
Interactive chat (POST /sessions/{id}/events with type=user.message)
already persists the user.message event before calling
create_and_dispatch(). The submit path was emitting a second one,
causing duplicate user messages in the conversation timeline.
Added emit_user_message parameter (default True) to create_and_dispatch.
Set to False for the two interactive-chat callers in sessions.py;
API create-task, cron/trigger, and follow-up callers keep the default
True so their prompts still appear in the timeline.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…r status/list) Implements the Orchard-inspired PodWatcher pattern: a single K8s Watch stream keeps a local HashMap of all sandbox pods synchronized in real-time. status() and list_active() now read from this cache instead of calling the K8s API server. Before: 200 sandboxes × controller sweep every 15s = 800+ API calls/min After: 1 persistent Watch stream, 0 API calls for status/list Implementation: - sandbox/pod_watcher.rs: PodWatcher struct with background Watch loop. Handles Init/InitDone/Apply/Delete events to keep cache fresh. Automatic re-list on Watch errors with 5s backoff. - k8s.rs: K8sProvider now holds a PodWatcher, initialized in new(). status() and list_active() delegate to cache reads (Arc<RwLock>). Docker provider: unaffected (uses bollard Docker API, no K8s Watch). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Resolve conflicts preferring internal changes while excluding .jdos build configs from origin.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.