Skip to content

CONTRIBUTING.md

Latest commit

 

History

History
141 lines (111 loc) · 7.48 KB

CONTRIBUTING.md

File metadata and controls

141 lines (111 loc) · 7.48 KB
Raw

Contributing to actor-runtime

Building and running from source

docker build -t actor-runtime .
mkdir -p data
docker run --rm -p 3333:3333 -p 3000:3000 \
  -v /var/run/docker.sock:/var/run/docker.sock \
  -v "$(pwd)/data:/data" \
  actor-runtime

./data on the host becomes the runtime's /data, so every storage, build and run record lands there for inspection. To drive a locally built image with the CLI, point apify runtime install at it. With Podman, build with podman build and mount Podman's socket as described in README.md's "Running with Podman instead of Docker". podman images lists the images the runtime builds as actor-runtime/<actor>:<buildId>, under the registry prefix Podman adds (docker.io/ or localhost/).

To point the CLI at it without apify runtime connect:

export APIFY_CLIENT_BASE_URL=http://localhost:3333
export APIFY_CONSOLE_URL=http://localhost:3000

The sample Actors (samples/actor_*) crawl the live web, so running them needs network access.

Specification

The behavioural spec lives in requirements/*.md: system.md, api.md, storage.md, actor-driver.md, cli.md, console.md and test.md. unsupported.md lists the platform behaviour the runtime deliberately leaves out.

User-facing documentation lives in skills/actor-runtime/SKILL.md, which ships inside the image. When a change alters what the runtime does for its users, update it in the same commit. Contribution conventions for requirements, code comments and pull requests are in CLAUDE.MD.

Development

pnpm install
pnpm run build     # tsc
pnpm test          # unit + integration (no Docker needed)
pnpm run test:e2e  # full CLI-driven dev loop against a built image (requires Docker, or Podman with CONTAINER_CLI=podman; the browser-view case pulls the ~2 GB Playwright base image)
pnpm run dev       # run the server directly against ./data with tsx

pnpm run dev sets ACTOR_RUNTIME_DATA_DIR=./data inline in the script (DEFAULT_DATA_DIR otherwise falls back to the container path /data - see src/config.ts); this only works as written on a POSIX shell (Linux/macOS). On Windows, set the env var separately before running tsx src/index.ts (e.g. in PowerShell: $env:ACTOR_RUNTIME_DATA_DIR="./data"; tsx src/index.ts), or use a cross-platform env-setter like cross-env if you add it as a dependency.

Bumping the pinned Crawlee v4 version

@crawlee/core and @crawlee/fs-storage are pinned to the exact version the npm v4 dist-tag resolves to (both must move in lockstep - @crawlee/fs-storage pins its own native addon, @crawlee/fs-storage-native). To bump:

pnpm view @crawlee/core dist-tags.v4
pnpm view @crawlee/fs-storage dist-tags.v4   # should match
# update both versions in package.json, then:
pnpm install
pnpm run build && pnpm test

While bumping, check whether the pnpm.overrides pin on @crawlee/fs-storage-native in package.json is still needed: it forces the first release with linux-arm64 bindings (0.1.5-beta.19, API-identical to the 0.1.5-beta.18 that released @crawlee/fs-storage versions still depend on). Once the bumped @crawlee/fs-storage depends on >= 0.1.5-beta.19 on its own, delete the override.

Releasing

A stable release is cut by the Release workflow (.github/workflows/release.yml): Actions -> Release -> Run workflow on master. With the default auto release type the version follows the Conventional Commits since the last release (while on 0.x, feat bumps the patch version and a breaking change the minor one); patch, minor, major force the bump and custom takes the version from custom_version. Once the CI checks on the commit have passed, the workflow:

  1. moves the "not yet released" section of CHANGELOG.md under the new version and bumps package.json, in one commit pushed to master,
  2. tags that commit v<version> and publishes a GitHub release with the version's release notes,
  3. builds that commit and publishes the image as v<version> and latest (plus the master tags, see below).

Between releases, every feat, fix, perf, refactor or style commit on master regenerates the "not yet released" section (.github/workflows/changelog.yml), so CHANGELOG.md always lists what the next release will contain. Both are generated by git-cliff from the commit messages - PR titles, as PRs are squash-merged - so those have to be conventional commits to show up. docs, test, chore and ci commits are left out. Do not edit CHANGELOG.md by hand: the next run regenerates it.

Both workflows push to master as the Apify service account, with the APIFY_SERVICE_ACCOUNT_GITHUB_TOKEN secret synced from the org's secret manager.

Publishing the image

Images go to apify/actor-runtime on Docker Hub by default; the target repository is a workflow input, so a one-off build can be pushed elsewhere.

Besides releases, the Build Docker image workflow (.github/workflows/build-image.yml) can be run by hand to publish any branch: Actions -> Build Docker image -> Run workflow, pick the branch in Use workflow from, and run it. That is the only branch to choose - the workflow always builds the branch it was dispatched from. Everything else is optional: an extra tag, whether to also move :latest, and which platforms to build. Such builds create no GitHub release and leave CHANGELOG.md alone.

It pushes one multi-arch manifest per tag - linux/amd64 and linux/arm64 by default - so the same tag serves x86_64 and Apple Silicon. Every run publishes <branch>-<short-sha> (immutable) and <branch> (moving), with / in a branch name slugified to -. It pushes as the Apify service account, using the same two repository secrets as apify-actor-docker: APIFY_SERVICE_ACCOUNT_DOCKERHUB_USERNAME and APIFY_SERVICE_ACCOUNT_DOCKERHUB_TOKEN. They are synced into this repository's Actions secrets from the org's secret manager, so they are managed there rather than added by hand.

Deleting published tags

Per-branch and per-commit tags pile up on Docker Hub as branches come and go. The Delete Docker image tags workflow (.github/workflows/delete-image-tags.yml) removes them: Actions -> Delete Docker image tags -> Run workflow, then give it the tags to delete - comma- or newline-separated, either exact tag names or shell-style globs matched against the repository's current tags (claude-*, master-*, * for everything deletable). The repository it deletes from is hardcoded to apify/actor-runtime, unlike the build workflow's target: it deletes, so it can only ever reach the one repository it is written for.

master, main and latest are never deleted: a glob covering one of them skips it and says so, so the tags users pull cannot be removed from here. Every other tag in the repository can be.

Runs are a dry run by default - they list what would go and delete nothing. Uncheck dry_run to delete for real. Either way the run summary lists the tags. It authenticates with the same two service-account secrets as the build workflow; the token needs delete permission on the repository, or each delete comes back 403.

Deleting a tag only removes that tag. The manifest and layers stay until Docker Hub's own garbage collection reclaims them, and any other tag pointing at the same digest keeps working - so deleting master-<sha> does not break master when both point at the same build.