📚 Read the full documentation at grafana.github.io/nanogit

Overview

nanogit is a lightweight, cloud-native Git implementation designed for applications that need efficient Git operations over HTTPS without the complexity and resource overhead of traditional Git implementations.

Why we built nanogit in the first place

nanogit was created by Grafana to power Git Sync — the Observability as Code feature that lets you store Grafana dashboards and folders as files in your own Git repository and manage them as code: auditable, reviewable, and reproducible, editing in the UI and opening pull requests without leaving Grafana.

Git Sync needed a real Git engine embedded in Grafana's backend, and none of the obvious options fit:

• A GitHub API wasn't enough. The goal grew from "sync with GitHub" to "sync with any Git provider." That means speaking the standard Git Smart HTTP protocol directly, so GitLab, Bitbucket, and self-hosted servers work without provider-specific code.
• The git CLI and libgit2 assume a local .git working directory. Maintaining a checkout per tenant, and shelling out to a binary, is impractical and hard to reason about operationally in a shared, multitenant backend.
• go-git is mature, but its abstraction and overhead were too heavy for this use case. It is built around local-disk operations and full clones — stateful and memory-heavy once multiplied across thousands of tenants and large repositories.

So we built nanogit around the constraints Git Sync actually has:

• Provider-agnostic — talks the Git wire protocol over HTTPS, so any compliant provider works without bespoke integrations.
• Stateless — no clones, no .git — a defining goal was to avoid full clones and never persist a .git directory or per-tenant working tree. nanogit reads and writes objects directly over HTTPS, so there is no local repository state to store, clean up, or keep consistent across a horizontally scaled, multitenant backend.
• Lean and fast — parses only what Git Sync needs. Streaming packfiles, path filtering, shallow reads, and delta handling keep it fast and memory-efficient on Grafana-scale repositories, where an operation usually touches a subpath rather than the whole repo.
• Operational control — speaking the Git protocol directly lets Grafana control exactly how it talks to each provider: how many requests it makes and how large each response is (to stay within provider rate limits and byte budgets), what objects get cached and reused across operations, and how many round trips a sync costs. Hosted provider APIs and general-purpose clients don't expose that level of control.
• Safe and controllable — a focused, embeddable library with a minimal surface area is easier to secure and operate than a general-purpose tool, and it needs only token auth (no SSH key management).

nanogit is open source and usable on its own, but its design — and the performance numbers below — come directly from this workload: doing the Git plumbing behind Git Sync efficiently, safely, and for many tenants at scale.

Known usage

• grafana/grafana — nanogit is the Git engine behind Git Sync provisioning (apps/provisioning/pkg/repository/git). It resolves refs, reads and writes dashboards and folders, stages commits, and pushes to each tenant's repository over HTTPS.
• grafana/grafana-bench — nanogit is the default Git driver for fetching the test-suite repository before a benchmark run. grafana-bench pulls its k6/Playwright suites from Git; nanogit's lightweight, HTTP-only client with parallel object fetching and sparse/subpath checkout pulls large monorepos faster and with less overhead than the alternative go-git driver, keeping benchmark setup fast and repeatable. (go-git stays available via --git-driver gogit when SSH or full Git features are needed.)

Features

• HTTPS-only Git operations - Works with any Git service supporting Smart HTTP Protocol v2 (GitHub, GitLab, Bitbucket, etc.), eliminating the need for SSH key management in cloud environments

• Stateless architecture - No local .git directory dependency, making it perfect for serverless functions, containers, and microservices where persistent local state isn't available or desired

• Memory-optimized design - Streaming packfile operations and configurable writing modes minimize memory usage, crucial for bulk operations and memory-constrained environments

• Flexible storage architecture - Pluggable object storage and configurable writing modes allow optimization for different deployment patterns, from high-performance in-memory operations to memory-efficient disk-based processing

• Cloud-native authentication - Built-in support for Basic Auth and API tokens, designed for automated workflows and CI/CD systems without interactive authentication

• Essential Git operations - Focused on core functionality (read/write objects, commit operations, diffing) without the complexity of full Git implementations, reducing attack surface and resource requirements

• High performance - Significantly faster than traditional Git implementations for common cloud operations, with up to 300x speed improvements for certain scenarios

Non-Goals

The following features are explicitly not supported:

• git:// and Git-over-SSH protocols
• File protocol (local Git operations)
• Signature verification
• Git hooks
• Git configuration management
• Direct .git directory access
• "Dumb" servers
• Complex permissions (all objects use mode 0644)

Why Git Protocol v2 Only?

nanogit speaks only Git Smart HTTP Protocol v2. It does not implement the legacy v1 protocol (neither the original "smart" v1 negotiation nor the "dumb" HTTP protocol), and it will not silently fall back to it. This is a deliberate design decision, not a missing feature:

• Stateless by design — Protocol v2 replaces v1's stateful, multi-round want/have negotiation with a command-oriented request model that completes in a single stateless HTTP round trip. That maps directly onto nanogit's stateless, serverless-friendly architecture. v1's negotiation assumes connection state that nanogit deliberately does not keep.
• Server-side ref filtering — v2's ls-refs command lets the client request only the references it needs (via ref-prefix). v1 dumps the entire ref advertisement on every info/refs request, which is wasteful for repositories with thousands of branches and tags — exactly the multitenant, large-repo case nanogit targets.
• Smaller surface area — Supporting both protocols would double the negotiation and parsing code paths. Minimal surface area is a core design principle: less code means fewer bugs and a smaller attack surface.
• Broad (not universal) provider support — Protocol v2 has been available since Git 2.18 (2018) and the default fetch protocol since Git 2.26 (2020). GitHub, GitLab, and Bitbucket all support it. The notable exception is Azure DevOps / Azure Repos, which only speaks protocol v1 — nanogit cannot talk to it, and there is no fallback. For nanogit's target audience (cloud-native services talking to modern hosted Git), the trade-off is worth it: v1 support would add complexity to serve a shrinking set of servers.

When a server only speaks v1, nanogit check reports it as incompatible and every operation fails fast rather than silently degrading. Always run it against a new provider before integrating. See the Server Compatibility guide for how to verify support.

Why nanogit?

While go-git is a mature Git implementation, nanogit is designed for cloud-native, multitenant environments requiring minimal, stateless operations.

Feature	nanogit	go-git
Protocol	HTTPS-only	All protocols
Storage	Stateless, configurable object storage + writing modes	Local disk operations
Cloning	Path filtering with glob patterns, shallow clones	Full repository clones
Scope	Essential operations only	Full Git functionality
Use Case	Cloud services, multitenant	General purpose
Resource Usage	Minimal footprint	Full Git features

Choose nanogit for lightweight cloud services requiring stateless operations and minimal resources. Use go-git when you need full Git functionality, local operations, or advanced features.

This are some of the performance differences between nanogit and go-git in some of the measured scenarios:

Scenario	Speed	Memory Usage
CreateFile (XL repo)	306x faster	186x less
UpdateFile (XL repo)	291x faster	178x less
DeleteFile (XL repo)	302x faster	175x less
BulkCreateFiles (1000 files, medium repo)	607x faster	11x less
CompareCommits (XL repo)	60x faster	96x less
GetFlatTree (XL repo)	258x faster	160x less

For detailed performance metrics, see the latest performance report and performance analysis.

Getting Started

Prerequisites

• Go 1.25 or later.
• Git (for development)

Installation

Go Library

Install the latest version:

go get github.com/grafana/nanogit@latest

Or install a specific version:

go get github.com/grafana/nanogit@v0.x.x # Replace v0.x.x with the latest released version

See all available versions on the releases page.

Command-Line Interface

nanogit provides a CLI for terminal-based Git operations. Download pre-built binaries from the releases page or install using Go.

Download pre-built binary (recommended):

# Linux (amd64)
wget https://github.com/grafana/nanogit/releases/latest/download/nanogit_Linux_x86_64.tar.gz
tar -xzf nanogit_Linux_x86_64.tar.gz
sudo mv nanogit /usr/local/bin/

# macOS (Apple Silicon)
wget https://github.com/grafana/nanogit/releases/latest/download/nanogit_Darwin_arm64.tar.gz
tar -xzf nanogit_Darwin_arm64.tar.gz
sudo mv nanogit /usr/local/bin/

# macOS (Intel)
wget https://github.com/grafana/nanogit/releases/latest/download/nanogit_Darwin_x86_64.tar.gz
tar -xzf nanogit_Darwin_x86_64.tar.gz
sudo mv nanogit /usr/local/bin/

# Windows (PowerShell)
Invoke-WebRequest -Uri "https://github.com/grafana/nanogit/releases/latest/download/nanogit_Windows_x86_64.zip" -OutFile "nanogit.zip"
Expand-Archive nanogit.zip
Move-Item nanogit\nanogit.exe C:\Windows\System32\

Or install using Go:

go install github.com/grafana/nanogit/cli/cmd/nanogit@latest

See the CLI documentation for more details.

Usage

// Create client with authentication
client, err := nanogit.NewHTTPClient(
    "https://github.com/user/repo.git",
    options.WithBasicAuth("username", "token"),
)

// Get main branch and create staged writer
ref, err := client.GetRef(ctx, "refs/heads/main")
writer, err := client.NewStagedWriter(ctx, ref)

// Create and update files
writer.CreateBlob(ctx, "docs/new-feature.md", []byte("# New Feature"))
writer.UpdateBlob(ctx, "README.md", []byte("Updated content"))

// Commit changes with proper author/committer info
author := nanogit.Author{
    Name:  "John Doe",
    Email: "john@example.com",
    Time:  time.Now(),
}
committer := nanogit.Committer{
    Name:  "Deploy Bot",
    Email: "deploy@example.com",
    Time:  time.Now(),
}

commit, err := writer.Commit(ctx, "Add feature and update docs", author, committer)
writer.Push(ctx)

Cloning Repositories with Path Filtering

nanogit provides efficient cloning with flexible path filtering, ideal for CI environments where only specific directories are needed:

// First, get the commit hash for the branch you want to clone
ref, err := client.GetRef(ctx, "main")
if err != nil {
    return err
}

// Clone specific directories only (perfect for CI with no caching)
result, err := client.Clone(ctx, nanogit.CloneOptions{
    Path:         "/tmp/my-repo",        // Local filesystem path (required)
    Hash:         ref.Hash,              // Commit hash (required)
    IncludePaths: []string{"src/**", "docs/**"}, // Include only these paths
    ExcludePaths: []string{"*.tmp", "node_modules/**"}, // Exclude these paths
})
if err != nil {
    return err
}

// result.Commit contains the commit information
// result.FlatTree contains filtered file tree
// Files are automatically written to result.Path
fmt.Printf("Cloned %d of %d files to %s\n",
    result.FilteredFiles, result.TotalFiles, result.Path)

Key clone features:

• Path filtering: Use glob patterns to include/exclude specific files and directories
• Filesystem output: Automatically writes filtered files to specified local path
• Shallow clones: Fetch only the latest commit to minimize bandwidth
• Branch isolation: Clone only specific branches to reduce transfer time
• CI optimized: Perfect for build environments with no persistent storage
• Performance tuning: Configurable batch fetching and concurrency for optimal performance

Performance Optimization Options

The Clone operation supports two key performance optimization options to significantly improve cloning speed:

// Clone with performance optimizations
result, err := client.Clone(ctx, nanogit.CloneOptions{
    Path:         "/tmp/my-repo",
    Hash:         ref.Hash,
    IncludePaths: []string{"pkg/api/**"},
    BatchSize:    50,      // Fetch 50 blobs per network request
    Concurrency:  8,       // Use 8 concurrent workers
})

BatchSize - Controls how many blobs to fetch in a single network request:

• Value 0 or 1: Fetches blobs individually (backward compatible, default behavior)
• Values > 1: Enables batch fetching, reducing network round trips by 50-70%
• Automatically falls back to individual fetching if a blob is missing from a batch response
• Recommended for repositories with many files to minimize network overhead
• Recommended value: 20-100 depending on average blob size and network conditions

Concurrency - Controls parallel blob fetching:

• Value 0 or 1: Sequential fetching (backward compatible, default behavior)
• Values > 1: Enables concurrent fetching using worker pools
• Works with both batch fetching (fetches multiple batches in parallel) and individual fetching
• Recommended value: 4-10 depending on network conditions and server capacity
• Can improve performance by 2-3x on high-latency networks

Performance Impact: Combined optimization (BatchSize=50, Concurrency=8) can achieve 5-10x speedup compared to default sequential fetching, making it ideal for CI/CD environments and large repository operations.

Configurable Writing Modes

nanogit provides flexible writing modes to optimize memory usage during write operations:

// Auto mode (default) - smart memory/disk switching
writer, err := client.NewStagedWriter(ctx, ref)

// Memory mode - maximum performance
writer, err := client.NewStagedWriter(ctx, ref, nanogit.WithMemoryStorage())

// Disk mode - minimal memory usage for bulk operations
writer, err := client.NewStagedWriter(ctx, ref, nanogit.WithDiskStorage())

For detailed information about writing modes, performance characteristics, and use cases, see Storage Architecture Documentation.

Retry Mechanism

nanogit includes a pluggable retry mechanism, making operations more robust against transient network errors and server issues. The retry mechanism follows the same pattern as storage options, using context-based injection.

Basic Usage

By default, no retries are performed (backward compatible). To enable retries, inject a retrier into the context:

import "github.com/grafana/nanogit/retry"

// Create a retrier with default settings (3 attempts, exponential backoff)
retrier := retry.NewExponentialBackoffRetrier()
ctx = retry.ToContext(ctx, retrier)

// All HTTP operations will now use retry logic
client, err := nanogit.NewHTTPClient(repo, options...)
ref, err := client.GetRef(ctx, "main")

Built-in Retrier

The ExponentialBackoffRetrier provides configurable exponential backoff retry logic:

// Customize retry behavior
retrier := retry.NewExponentialBackoffRetrier().
    WithMaxAttempts(5).                    // Retry up to 5 times
    WithInitialDelay(200 * time.Millisecond). // Start with 200ms delay
    WithMaxDelay(10 * time.Second).        // Cap at 10 seconds
    WithMultiplier(2.0).                   // Double delay each retry
    WithJitter()                          // Add random jitter

ctx = retry.ToContext(ctx, retrier)

Custom Retrier

You can implement your own retry logic by implementing the Retrier interface:

type MyRetrier struct {
    // Your custom fields
}

func (r *MyRetrier) ShouldRetry(ctx context.Context, err error, attempt int) bool {
    // Your retry logic
    return true
}

func (r *MyRetrier) Wait(ctx context.Context, attempt int) error {
    // Your backoff logic
    return nil
}

func (r *MyRetrier) MaxAttempts() int {
    return 3
}

// Use your custom retrier
ctx = retry.ToContext(ctx, &MyRetrier{})

What Gets Retried

The retry mechanism automatically retries on:

• Network timeout errors
• 5xx server errors: Server unavailable errors (for GET requests only)
• Temporary errors: Any error marked as temporary

The retry mechanism does not retry on:

• 4xx client errors: Bad requests, authentication failures, etc.
• Context cancellation: When the context is cancelled or deadline exceeded
• POST request 5xx errors: POST requests cannot retry 5xx errors because the request body (io.Reader) is consumed when the request is sent and cannot be re-read

Retry Behavior by Request Type

• GET requests (SmartInfo): Retry on network errors and 5xx status codes
• POST requests (UploadPack, ReceivePack): Retry only on network errors (before response is received)

This limitation exists because POST request bodies are consumed during the HTTP request and cannot be re-read for retries.

Storage Architecture

nanogit features a flexible two-layer storage architecture that separates concerns and allows independent optimization:

1. Writing modes: Control temporary storage during packfile creation (memory/disk/auto)
2. Object storage: Handle long-term caching and retrieval of Git objects (pluggable backends)

Object Storage and Caching

nanogit provides context-based object storage with pluggable backends. The default in-memory implementation is optimized for stateless operations, but you can implement custom backends for persistent caching:

// Custom storage example
ctx = storage.ToContext(ctx, myRedisStorage)
client, err := nanogit.NewHTTPClient(repo, options...)

This enables sharing Git object cache across multiple repositories, persistent caching across service restarts, and optimization for specific deployment patterns.

For detailed information about storage architecture, writing modes, and custom implementations, see Storage Architecture Documentation.

Testing

Unit Testing with Mocks

nanogit includes generated mocks for easy unit testing. The mocks are generated using counterfeiter and provide comprehensive test doubles for both the Client and StagedWriter interfaces.

For detailed testing examples and instructions, see CONTRIBUTING.md. You can also find complete working examples in mocks/example_test.go.

Integration Testing with gittest

The gittest package provides utilities for testing Git operations with a containerized Gitea server:

go get github.com/grafana/nanogit/gittest@latest

See gittest README for API documentation and examples.

Contributing

We welcome contributions! Please see our Contributing Guide for details on how to submit pull requests, report issues, and set up your development environment.

Code of Conduct

This project follows the Grafana Code of Conduct. By participating, you are expected to uphold this code.

License

This project is licensed under the Apache License 2.0 - see the LICENSE file for details.

Project Status

This project is currently in active development. While it's open source, it's important to note that it was initially created as part of a hackathon. We're working to make it production-ready, but please use it with appropriate caution.

Documentation

Comprehensive documentation is available at grafana.github.io/nanogit:

• Getting Started - Installation and quick start guide
• Architecture - Design principles, storage backend, performance
• API Reference (GoDoc) - Complete API documentation
• Changelog - Version history and release notes

Resources

Want to learn how Git works? The following resources are useful:

• Git on the Server - The Protocols
• Git Protocol v2
• Pack Protocol
• Git HTTP Backend
• HTTP Protocol
• Git Protocol HTTP
• Git Protocol v2
• Git Protocol Pack
• Git Protocol Common

Security

If you find a security vulnerability, please report it to us according to our security policy.

Support

• GitHub Issues: Create an issue
• Community: Grafana Community Forums

Acknowledgments

• The Grafana team for their support and guidance
• The open source community for their valuable feedback and contributions