Skip to content
sfcomputePublic

About

No description, website, or topics provided.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Latest commit

 

History

67 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

vast

A strongly-typed, async Rust client for the VAST Data Management System (VMS) REST API.

The vast crate (repo: vast-rs) is a peer library to vastpy, VAST's official Python SDK. API types are intended to be generated directly from your cluster's OpenAPI specification so they are always in sync with your installed VMS version.


Features

  • Async-first — built on Tokio and reqwest
  • Strongly typed — serde models for clusters, nodes, users, volumes, and more; generated from the VMS OpenAPI spec
  • Two auth strategies — long-lived API token or username/password (JWT obtained and cached automatically)
  • Secret-safe — credentials are stored as SecretString and never appear in logs or Debug output
  • Ergonomic builder API — VastClient::builder().address(...).token(...).build()?
  • Cheap to clone — VastClient is Arc-backed and shares a single connection pool
  • Graceful forward-compatibility — unknown fields from newer API versions are captured in an extra map instead of failing deserialization

Installation

Add to your Cargo.toml:

[dependencies]
vast = { git = "https://github.com/sfcompute/vast-rs" }
tokio = { version = "1", features = ["full"] }

Rust 1.85 or later is required (Rust 2024 edition).


Quick Start

use vast::VastClient;

#[tokio::main]
async fn main() -> vast::Result<()> {
    let client = VastClient::builder()
        .address("vms.example.com")
        .token("your-api-token")
        .build()?;

    let clusters = client.clusters().list().await?;
    for c in &clusters {
        println!("{}: {} ({})", c.id, c.name, c.sw_version);
    }

    Ok(())
}

Authenticate with username and password

The client exchanges credentials for a JWT automatically and caches it for the session:

// Cluster admin account
let client = VastClient::builder()
    .address("vms.example.com")
    .credentials("admin", "secret")
    .build()?;

// Tenant admin account — tenant name is required, otherwise the VMS returns 401
let client = VastClient::builder()
    .address("vms.example.com")
    .credentials("alice", "secret")
    .tenant("acme")
    .build()?;

Load configuration from environment variables

// Reads VMS_ADDRESS + VMS_TOKEN (or VMS_USER + VMS_PASSWORD) from the environment
let client = vast::VastClient::from_env()?;

Environment Variables

Variable Description
VMS_ADDRESS Hostname or IP of the VMS (e.g. vms.example.com)
VMS_TOKEN API token — takes precedence over user/password
VMS_USER Username for credential-based auth
VMS_PASSWORD Password for credential-based auth
VMS_TENANT Tenant name — required for tenant admin accounts, omit for cluster admins
VMS_API_VERSION REST API version to address, as the path segment it becomes (e.g. v7): every request goes to /api/v7/…. Unset or empty addresses the unversioned /api/… routes. Equivalent to .api_version("v7") on the builder. See API version.
VMS_CA_CERT_FILE Path to a PEM CA certificate (or bundle) to trust in addition to the public roots. Equivalent to .ca_certificate(bytes) on the builder. See TLS.
VMS_DANGER_ACCEPT_INVALID_CERTS Set to 1 / true / yes / on to disable TLS certificate validation. Development / self-signed VMS deployments only. Equivalent to .danger_accept_invalid_certs(true) on the builder.

TLS

The client validates the VMS certificate against the Mozilla root set compiled into the binary (reqwest's rustls-tls). It does not read the host's certificate directory and does not honor SSL_CERT_FILE, so a VMS whose certificate comes from a private CA — including VAST's factory CA — needs that CA passed in explicitly. Installing it on the machine or in the container image has no effect.

let client = VastClient::builder()
    .address("vms.example.com")
    .token("tok")
    .ca_certificate(std::fs::read("/etc/vast-ca/ca.crt")?)
    .build()?;

Or set VMS_CA_CERT_FILE=/etc/vast-ca/ca.crt and use from_env().

The argument is a PEM certificate or a bundle of several, added to the public roots rather than replacing them. Repeated .ca_certificate() calls accumulate, so endpoints anchored to different roots can both be trusted.

Failures are reported at build(), as Error::Config, and name the file: an unreadable VMS_CA_CERT_FILE path, a PEM holding no -----BEGIN CERTIFICATE----- block (an unfilled placeholder, an empty ConfigMap key), or a block whose body is not a certificate. None of these fall back to the public roots — that would turn a config mistake into a confusing handshake failure on the first request.

.danger_accept_invalid_certs(true) disables validation altogether and takes precedence, so a CA supplied alongside it is unused; the client logs a warning when both are set.


Usage

Clusters

// List all clusters
let clusters = client.clusters().list().await?;

// Get a single cluster
let cluster = client.clusters().get(1).await?;

Nodes

use vast::api::ListNodesParams;

// List all nodes
let nodes = client.nodes().list().await?;

// Filter by cluster
let params = ListNodesParams { cluster_id: Some(1), ..Default::default() };
let nodes = client.nodes().list_with_params(&params).await?;

Volumes

use vast::api::{CreateVolume, UpdateVolume};

// List volumes
let volumes = client.volumes().list().await?;

// Create a volume
let vol = client.volumes().create(&CreateVolume {
    name: "data".into(),
    path: "/data".into(),
    quota: Some(10 * 1024 * 1024 * 1024), // 10 GiB
}).await?;

// Update a volume
client.volumes().update(vol.id, &UpdateVolume {
    quota: Some(20 * 1024 * 1024 * 1024),
    ..Default::default()
}).await?;

// Delete a volume
client.volumes().delete(vol.id).await?;

Users

use vast::api::{CreateUser, ListUsersParams, UpdateUser};

let users = client.users().list().await?;

// Look up a user by name
let named = client.users().list_with_params(&ListUsersParams {
    name: Some("alice".into()),
    ..Default::default()
}).await?;

let user = client.users().create(&CreateUser {
    name: "alice".into(),
    uid: Some(1001),
    email: Some("alice@example.com".into()),
    enabled: Some(true),
}).await?;

client.users().delete(user.id).await?;

Views

use vast::api::{CreateView, ListViewsParams, UpdateView};

let views = client.views().list().await?;

// Look up a view by path, or by the S3 bucket it exports
let by_path = client.views().list_with_params(&ListViewsParams {
    path: Some("/data".into()),
    ..Default::default()
}).await?;
let by_bucket = client.views().list_with_params(&ListViewsParams {
    bucket: Some("my-bucket".into()),
    ..Default::default()
}).await?;

let view = client.views().create(&CreateView {
    name: "nfs-export".into(),
    path: "/data".into(),
    policy_id: 1,
    protocols: vec!["NFS".into()],
    create_dir: Some(true),
    alias: None,
    bucket: None,
    allow_anonymous_access: None,
    s3_versioning: None,
    s3_locks: None,
    s3_locks_retention_mode: None,
}).await?;

// `force` is the endpoint's optional `force` query parameter; `None` omits it.
client.views().delete(view.id, None).await?;
client.views().delete(view.id, Some(true)).await?;

View Policies

let policies = client.view_policies().list().await?;
let policy = client.view_policies().get(1).await?;

Quotas

use vast::api::{CreateQuota, ListQuotasParams, UpdateQuota};

let quotas = client.quotas().list().await?;

// Look up the quota on a specific path
let on_path = client.quotas().list_with_params(&ListQuotasParams {
    path: Some("/data".into()),
    ..Default::default()
}).await?;

let quota = client.quotas().create(&CreateQuota {
    name: "team-limit".into(),
    path: "/data".into(),
    hard_limit: Some(10 * 1024 * 1024 * 1024), // 10 GiB
    soft_limit: Some(8 * 1024 * 1024 * 1024),
    hard_limit_inodes: None,
    soft_limit_inodes: None,
}).await?;

client.quotas().update(quota.id, &UpdateQuota {
    hard_limit: Some(20 * 1024 * 1024 * 1024),
    ..Default::default()
}).await?;

VIP Pools

let pools = client.vip_pools().list().await?;

Snapshots

use vast::api::CreateSnapshot;

let snaps = client.snapshots().list().await?;

let snap = client.snapshots().create(&CreateSnapshot {
    name: "daily-backup".into(),
    path: "/data".into(),
}).await?;

S3 Policies

use vast::api::{CreateS3Policy, ListS3PoliciesParams};

let policies = client.s3_policies().list().await?;

// Look up a policy by name
let named = client.s3_policies().list_with_params(&ListS3PoliciesParams {
    name: Some("read-only".into()),
    ..Default::default()
}).await?;

Filtering lists

Resources whose endpoints support server-side filtering expose a List<Resource>Params struct and three methods taking it in place of PageParams: list_with_params (auto-paginates), list_paged_with_params (one page), and iter_with_params (lazy stream). Every params struct derives Default, and None fields are omitted from the query string — so Default::default() filters nothing, and filters are re-sent on each page request as pagination advances.

Resource Params struct Filters
Nodes ListNodesParams cluster_id, state
Users ListUsersParams name
Volumes ListVolumesParams path, is_snapshot
Views ListViewsParams path, bucket
Quotas ListQuotasParams path
S3 Policies ListS3PoliciesParams name

Filters are additive on the VMS side — setting two narrows the result set to items matching both.

Error handling

use vast::Error;

match client.clusters().get(999).await {
    Ok(cluster) => println!("Found: {}", cluster.name),
    Err(e) if e.is_not_found() => println!("Cluster not found"),
    Err(e) if e.is_unauthorized() => println!("Check your credentials"),
    Err(e) => return Err(e),
}

Builder Options

use std::time::Duration;

let client = VastClient::builder()
    .address("vms.example.com")                  // required — scheme added automatically
    .token("tok")                                 // or .credentials("user", "pass")
    .tenant("acme")                               // required for tenant admin accounts
    .api_version("v7")                            // omit for the unversioned /api/ routes
    .timeout(Duration::from_secs(60))             // default: 30s
    .ca_certificate(ca_pem)                       // trust a private CA — see TLS
    .danger_accept_invalid_certs(true)            // skip validation entirely; dev only
    .max_attempts(3)                              // GET retries (default: 3, set 1 to disable)
    .retry_backoff(Duration::from_secs(1))        // exponential backoff base (default: 1s)
    .build()?;

API version

By default the client addresses the unversioned routes — https://<address>/api/clusters/ — which a VMS resolves to its own current API version. .api_version("v7") (or VMS_API_VERSION=v7) inserts the version as a path segment instead, so every request goes to https://<address>/api/v7/clusters/. The credential exchange moves with it: /api/v7/token/.

Which versions a cluster serves depends on its software release, so the version belongs in per-deployment configuration rather than in code. It is validated at build() — a value that is not a v-and-digits segment is an Error::Config, rather than a path the VMS 404s on for every call. Put it here and not in .address(): an address carrying its own version segment is rejected, so there is only ever one place the version comes from.

Retries

GET requests are retried automatically on transient failures — transport-level errors (network blips, timeouts) and retryable HTTP statuses (5xx, 429). POST / PATCH / DELETE are sent at most once regardless of the retry budget, since they may be non-idempotent.

The default budget is 3 attempts with exponential backoff (base * 2^(attempt-1), so 1s then 2s with the default base). Set .max_attempts(1) to disable retries entirely; .retry_backoff(Duration::from_millis(100)) to tighten the loop for latency-sensitive workloads.

Streaming with iter()

For very large collections, iter() exposes a lazy async iterator that fetches one page at a time. Memory stays bounded to a single page rather than the whole Vec<T>:

let mut iter = client.quotas().iter();
while let Some(quota) = iter.next().await {
    let quota = quota?;
    process(&quota);
}

Filter params work the same way via iter_with_params on resources that support them (e.g. client.nodes().iter_with_params(&ListNodesParams { cluster_id: Some(7), ..Default::default() })). Retries are applied per page fetch, so a transient blip mid-iteration recovers automatically.


Spec Refresh

The VMS exposes a Swagger 2.0 spec at /api/?format=openapi without authentication. scripts/generate.sh downloads it to api-spec/vast-openapi.json as a reference. Types in src/api/ are hand-written from this spec — the API surface is small and changes additively between VAST versions.

# Refresh the local spec copy
./scripts/generate.sh -a vms.example.com

# Non-standard port or self-signed certificate
./scripts/generate.sh -a vms.example.com:8443 -k

After running, diff api-spec/vast-openapi.json against the previous version to spot new or changed endpoints, then update src/api/ accordingly.


API Coverage

Resource list filters get create update delete
Clusters ✓ ✓
Nodes ✓ ✓ ✓
Users ✓ ✓ ✓ ✓ ✓ ✓
Volumes ✓ ✓ ✓ ✓ ✓ ✓
Views ✓ ✓ ✓ ✓ ✓ ✓
View Policies ✓ ✓ ✓ ✓ ✓
Quotas ✓ ✓ ✓ ✓ ✓ ✓
VIP Pools ✓ ✓ ✓ ✓ ✓
Snapshots ✓ ✓ ✓ ✓ ✓
Protection Policies ✓ ✓ ✓ ✓ ✓
S3 Policies ✓ ✓ ✓ ✓ ✓ ✓

Running Tests

Unit tests use wiremock to mock the VMS HTTP API — no cluster needed:

cargo test

The mock client (see Mock client for consumer tests below) lives behind the mock feature. To exercise its own test suite, run:

cargo test --features mock

Integration tests against a real cluster live in tests/integration.rs and are gated behind the integration feature so the default cargo test doesn't touch the network. They're designed to be non-destructive — every resource they create uses a vast-rs-test- name prefix and is cleaned up at the end of each test. Set the environment variables and run:

VMS_ADDRESS=vms.example.com VMS_TOKEN=<token> cargo test --features integration

Mock client for consumer tests

If you're building an application or library on top of vast, you can use vast::mock::MockVastClient to unit-test code that takes a &VastClient — no live cluster, no wiremock boilerplate. Enable the mock feature in your dev-dependencies:

[dev-dependencies]
vast = { version = "0.1", features = ["mock"] }
tokio = { version = "1", features = ["full"] }
serde_json = "1"

Then stub VMS responses with json! literals and exercise your code:

use serde_json::json;
use vast::{VastClient, mock::MockVastClient};

async fn count_online_clusters(client: &VastClient) -> usize {
    client.clusters().list().await.unwrap()
        .iter()
        .filter(|c| c.state == "ONLINE")
        .count()
}

#[tokio::test]
async fn counts_only_online() {
    let mock = MockVastClient::start().await;
    mock.stub_get("clusters/", json!([
        { "id": 1, "name": "a", "state": "ONLINE" },
        { "id": 2, "name": "b", "state": "OFFLINE" },
        { "id": 3, "name": "c", "state": "ONLINE" },
    ])).await;

    assert_eq!(count_online_clusters(&mock.client()).await, 2);
}

Helpers on MockVastClient:

Helper What it does
stub_get/post/patch/delete(path, ...) Stub the common verbs with sensible default statuses
stub_error(method, path, status, msg) Stub a {"detail": msg} error response
stub_with(method, path, status, body, times) Stub with a call-count expectation (u64 or range); enforce via verify()
server() Underlying wiremock::MockServer for advanced matching
reset() / verify() Clear stubs / assert expectations were met

Paths may be written as "clusters/", "/clusters/", or "/api/clusters/" — they all match the same endpoint.


Contributing

Contributions are welcome. A few guidelines:

  • New API resources — add the resource model and Create/Update structs in src/api.rs, register the handle via the crud! macro (or hand-code it if the endpoint has bespoke methods like delete_folder), then expose it from VastClient with a small accessor method. Follow the pattern of Volumes / Tenants.
  • Tests — every new method should have at least one wiremock-backed unit test in tests/client_tests.rs.
  • No unwrap() in library code — propagate errors with ?.
  • Secrets discipline — any new credential type must use SecretString (or equivalent) so it cannot leak into logs.

To run the full check suite before opening a PR:

cargo fmt --check
cargo clippy -- -D warnings
cargo test

License

Licensed under either of:

at your option.


Related Projects

  • vastpy — VAST's official Python SDK
  • vast-admin-mcp — MCP server for VAST administration
  • vastdb_sdk — Python SDK for VAST Database (columnar data, separate from VMS)

About

No description, website, or topics provided.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Used by

Contributors

Languages