ghoten

Ghoten

Name origin: Ghoten blends GhitHub and OpenTofu, with a nod to Goten from Dragon Ball Z.

An OpenTofu fork with a native ORAS backend for OCI registry state storage.

Use an OCI-compatible registry — such as GitHub Container Registry (GHCR) — as the durable store for your infrastructure state, without external HTTP backends or third-party wrappers.

Why Ghoten?

OpenTofu and Terraform have not added new built-in remote state backends in years, preferring the generic HTTP backend for custom implementations. If you already operate an OCI registry for container images, forcing state through an extra HTTP proxy adds moving parts, latency, and operational burden.

Ghoten adds a first-class oras backend that speaks the OCI distribution protocol directly. State snapshots, locks, and version history are stored as OCI artifacts inside a repository you control, reusing existing authentication, authorization, and operational tooling.

Upstream: Ghoten tracks OpenTofu and carries only the additions required for the ORAS backend. All standard OpenTofu functionality is preserved.

Key Features

OpenTofu Core

• Infrastructure as Code — Describe infrastructure with a high-level configuration syntax. Version, share, and reuse blueprints like any other code.
• Execution Plans — Preview exactly what will change before applying, avoiding surprises.
• Resource Graph — Parallel creation and modification of non-dependent resources for efficient operations.
• Change Automation — Apply complex changesets with minimal human interaction.

ORAS Backend

• Native OCI storage — State, locks, and versions stored as OCI manifests and layers via ORAS.
• Atomic locking with generation semantics — Prevents concurrent lock holders through post-write verification.
• Stale lock cleanup (TTL) — Crashed-process locks are automatically cleared during the next acquisition.
• State versioning with async retention — Historical snapshots with non-blocking background cleanup.
• Gzip compression — Optional state compression before pushing to the registry.
• Rate limiting — Token-bucket rate limiter to stay within registry quotas.
• Retry with exponential backoff — Automatic retries for transient errors (429, 502–504, timeouts, DNS failures).
• GHCR deletion fallback — Falls back to the GitHub Packages API when OCI DELETE returns HTTP 405.
• Docker credential helpers — Reuses Docker config / credential store with caching and singleflight deduplication.
• OpenTelemetry tracing — All OCI HTTP calls are instrumented via otelhttp.
• Client-side encryption — Combine with OpenTofu state encryption for end-to-end protection.

Installation

Build from Source

git clone https://github.com/vmvarela/ghoten.git
cd ghoten
make build        # produces ./ghoten binary

Docker

# Full image (Alpine, includes git/bash/openssh)
docker pull ghcr.io/vmvarela/ghoten:<version>

# Minimal image (scratch, binary only)
docker pull ghcr.io/vmvarela/ghoten:<version>-minimal

GitHub Releases

Pre-built binaries for 17 platform combinations (linux, darwin, windows, freebsd, netbsd, openbsd, solaris) are available on the Releases page.

ORAS Backend

Security Considerations

State snapshots frequently contain sensitive data (provider credentials, resource attributes, and sometimes plaintext secrets). Treat the OCI repository with the same care as any other credentials store:

• Enable client-side encryption — OpenTofu supports state encryption which encrypts state before it leaves your machine. This is the recommended approach.
• Keep the repository private and issue least-privilege tokens.
• Prefer registries that provide encryption at rest and audit trails.
• Be deliberate about retention/versioning; a long history increases exposure.

Note: The implementation has primarily been exercised against GitHub Container Registry (ghcr.io). Other registries should work but haven't yet been validated.

Configuration Parameters

How State Is Stored (Tags)

Tags act as stable references inside the OCI repository:

workspaceTag equals the workspace name when it is a valid OCI tag. Otherwise the backend uses a stable ws-<sha256> form and persists the real workspace name in OCI annotations.

Wire Format

State objects use ORAS manifest v1.1 packing:

• Manifest artifactType: application/vnd.terraform.state.v1
• Layer media type:
  • application/vnd.terraform.statefile.v1 (uncompressed)
  • application/vnd.terraform.statefile.v1+gzip (gzip)
• Annotations:
  • org.terraform.workspace — workspace name
  • org.terraform.state.updated_at — RFC 3339 timestamp

Lock objects:

• Manifest artifactType: application/vnd.terraform.lock.v1
• Annotations:
  • org.terraform.workspace — workspace name
  • org.terraform.lock.id — lock ID
  • org.terraform.lock.info — JSON-encoded lock metadata
  • org.terraform.lock.generation — JSON with generation, lease_expiry, and holder_id

Reads are strict: unexpected artifact types or media types raise errors instead of silently proceeding.

Locking

The lock lives at locked-<ws> tags and carries metadata in annotations. Unlocking deletes that manifest via OCI DELETE. Registries that respond with HTTP 405 trigger a fallback: the lock reference is retagged to an unlocked-<ws> placeholder.

Generation semantics prevent concurrent lock holders:

1. Read the current generation before any modifications.
2. Clear stale locks if lock_ttl > 0 and the existing lock has expired.
3. Write a new lock manifest with generation = currentGen + 1.
4. Post-write verification: re-read the lock and confirm the generation matches. A mismatch means another process won the race.

Stale Lock Cleanup (TTL)

When lock_ttl > 0, each lock includes a lease_expiry timestamp. During lock acquisition, if the existing lock's lease has expired, it is automatically cleared before the new lock attempt proceeds. This handles the common case where a crashed process left a lock behind — the next ghoten apply clears it automatically.

Versioning & Retention

When max_versions > 0, every successful state write gets an additional state-<ws>-v<N> tag. Version numbers are computed by scanning existing tags and picking max + 1.

Retention cleanup runs asynchronously in a background goroutine (30-second timeout, limited to 3 concurrent cleanups) so that ghoten apply returns immediately without waiting for old version pruning.

Authentication

Credentials are discovered in this order:

1. Docker credential helpers — Docker config / credential store, with caching (5-minute TTL for successful lookups, 10-second TTL for errors) and singleflight deduplication.
2. CLI host credentials — OpenTofu/Terraform CLI login tokens (ghoten login).

If Docker credentials are missing or fail, the backend falls back to CLI tokens automatically.

GHCR Deletion Fallback

GitHub Container Registry (ghcr.io) returns HTTP 405 for OCI DELETE operations. The backend automatically falls back to the GitHub Packages REST API to delete manifests. The token used for registry access must include delete:packages when retention is enabled or manual lock cleanup is needed.

Usage

Minimal

terraform {
  backend "oras" {
    repository = "ghcr.io/acme/infra-state"
  }
}

Production-Ready

terraform {
  backend "oras" {
    repository  = "ghcr.io/myorg/infra-state"
    compression = "gzip"

    # Lock reliability
    lock_ttl = 300   # 5 minutes

    # State versioning
    max_versions = 10
  }

  # Client-side state encryption (OpenTofu feature)
  encryption {
    key_provider "pbkdf2" "main" {
      passphrase = var.state_passphrase
    }

    method "aes_gcm" "main" {
      key_provider = key_provider.pbkdf2.main
    }

    state {
      method = method.aes_gcm.main
    }

    plan {
      method = method.aes_gcm.main
    }
  }
}

variable "state_passphrase" {
  type      = string
  sensitive = true
}

Tip: For production, consider using a KMS-backed key provider (AWS KMS, GCP KMS, etc.) instead of PBKDF2 with a passphrase.

GitHub Actions CI

- name: Ghoten Init
  env:
    TF_BACKEND_ORAS_REPOSITORY: ghcr.io/${{ github.repository_owner }}/tf-state
  run: |
    echo "${{ secrets.GITHUB_TOKEN }}" | docker login ghcr.io -u ${{ github.actor }} --password-stdin
    ghoten init

The token must be able to read/write the repository. If you enable version retention or rely on GHCR deletion fallbacks, it must also include delete:packages.

Testing

# Full test suite
go test ./...

# ORAS backend only (offline, uses in-memory fake OCI registry)
go test ./internal/backend/remote-state/oras/...

Troubleshooting

"unauthorized" or "denied" errors

• Confirm docker login <registry> works with the same credentials.
• For GHCR, ensure the token grants read:packages + write:packages (and delete:packages if retention is enabled).

Lock stuck after a crashed run

Option 1: Automatic cleanup with TTL (recommended)

lock_ttl = 300  # 5 minutes

The next ghoten apply will automatically clear any lock older than the TTL.

Option 2: Manual cleanup

Delete the locked-<workspace> tag from the registry UI or API.

Version deletion on GHCR

• GHCR does not support OCI DELETE; the backend automatically falls back to the GitHub Packages API.
• Ensure the token has delete:packages permission.
• If the API call also fails (e.g., insufficient permissions), old versions accumulate but writes still succeed.

Debug mode

TF_LOG=DEBUG ghoten plan

Limitations

• Only GHCR is exercised in automated tests; other registries may exhibit different quirks.
• Registries that refuse OCI DELETE degrade lock/unlock behavior (GHCR has a dedicated fallback via GitHub Packages API).
• insecure = true disables TLS verification — use only in controlled environments.

Getting Help & Contributing

• Open a GitHub issue
• Read the Contributing Guide
• Review the Code of Conduct
• For upstream OpenTofu questions, see OpenTofu Discussions

Security

Please report security vulnerabilities through GitHub Security Advisories. See SECURITY.md for details.

License

Mozilla Public License 2.0