github

package module
v0.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Apr 5, 2026 License: Apache-2.0 Imports: 17 Imported by: 0

README

Github Backend for Afero

Build codecov Go Report Card

A Go afero.Fs backed by a GitHub repository. Read, write, delete, and rename files in a repo using standard Go file-IO semantics; under the hood it calls the GitHub Contents API and raw.githubusercontent.com for byte ranges.

Features

  • afero.Fs-compatible: Drop-in replacement for any code using github.com/spf13/afero.Fs
  • Multiple auth modes: Personal Access Token, GitHub App (private key + installation ID), or bring your own go-github client
  • Cached reads: TTL-based in-memory caching (default 30s) reduces redundant API calls
  • Deferred writes: All modifications are buffered; a single GitHub API call per Close() means one commit per file flush
  • Partial reads via RangeReader: Fetch byte ranges efficiently using HTTP Range requests against raw.githubusercontent.com without downloading the entire file
  • WalkDir helper: Tree traversal with fs.SkipDir / fs.SkipAll support mimics standard fs.WalkDir semantics

Installation

go get github.com/kdihalas/github

Requires Go 1.25.0 or later.

Quick Start

package main

import (
	"context"
	"fmt"
	"log"
	"os"

	"github.com/kdihalas/github"
)

func main() {
	client, err := github.NewClient(
		context.Background(),
		github.WithGithubToken(os.Getenv("GITHUB_TOKEN")),
	)
	if err != nil {
		log.Fatalf("Failed to create GitHub client: %v", err)
	}

	fs := github.NewFsFromClient(client, "owner", "repo", "main")

	// Create and write a file
	f, err := fs.Create("config/app.json")
	if err != nil {
		log.Fatal("create:", err)
	}
	_, err = f.WriteString(`{"env": "production"}`)
	if err != nil {
		log.Fatal("write:", err)
	}
	// Single GitHub API call happens here
	if err = f.Close(); err != nil {
		log.Fatal("close:", err)
	}
	fmt.Println("✓ wrote config/app.json")
}

Authentication

Personal Access Token
client, err := github.NewClient(
	context.Background(),
	github.WithGithubToken(os.Getenv("GITHUB_TOKEN")),
)
GitHub App (Private Key + Installation ID)
privateKey := []byte(`-----BEGIN RSA PRIVATE KEY-----
...
-----END RSA PRIVATE KEY-----`)

client, err := github.NewClient(
	context.Background(),
	github.WithGithubApplication("your-app-id", 12345, privateKey),
)
Bring Your Own go-github Client
import gh "github.com/google/go-github/v84/github"

myGithubClient := gh.NewClient(httpClient)
client, err := github.NewClient(
	context.Background(),
	github.WithGithubClient(myGithubClient),
)

Usage Patterns

Reading a File
f, err := fs.Open("path/to/file.txt")
if err != nil {
	log.Fatal(err)
}
defer f.Close()

content, err := io.ReadAll(f)
Listing a Directory
dir, err := fs.Open("path/to/dir")
if err != nil {
	log.Fatal(err)
}
defer dir.Close()

ghDir := dir.(*github.File)
entries, err := ghDir.ReaddirAll()
if err != nil {
	log.Fatal(err)
}
for _, entry := range entries {
	fmt.Printf("%s (dir=%v)\n", entry.Name(), entry.IsDir())
}
Walking a Tree
err := afero.Walk(fs, ".", func(path string, fi os.FileInfo, err error) error {
	if err != nil {
		return err
	}
	fmt.Printf("%s (dir=%v, size=%d)\n", path, fi.IsDir(), fi.Size())
	return nil
})
Partial Reads with RangeReader
f, err := fs.Open("large-file.bin")
if err != nil {
	log.Fatal(err)
}
defer f.Close()

ghFile := f.(*github.File)
rc, err := ghFile.RangeReader(1024, 512) // 512 bytes starting at offset 1024
if err != nil {
	log.Fatal(err)
}
defer rc.Close()

chunk := make([]byte, 512)
n, _ := io.ReadFull(rc, chunk)
fmt.Printf("Read %d bytes\n", n)
Configuring TTL and Commit Author
fs := github.NewFsFromClient(
	client,
	"owner", "repo", "main",
	github.WithCacheTTL(60 * time.Second),        // Cache for 60s instead of 30s
	github.WithAPITimeout(30 * time.Second),      // Per-request timeout
	github.WithCommitAuthor("Bot", "bot@example.com"), // Author on commits
)

How It Works

GitHub has no directories. When you call Mkdir, the library creates a .gitkeep placeholder file. Stat returns directory metadata by checking whether the GitHub Contents API returns a single file or a slice of items.

Writes are deferred. Calls to Write only touch an in-memory buffer. The actual GitHub API call (via the Contents API) happens when you call Close() or Sync(). Each flush produces exactly one commit.

Reads are cached. A three-layer cache minimizes API calls:

  • memFs (in-memory file content): Stores fetched blobs
  • shaCache (SHA lookup): Maps paths to their blob SHAs (needed for updates and deletes)
  • ttlCache (TTL gating): Tracks when each path was last fetched; entries older than the TTL are re-fetched

Default cache TTL is 30 seconds; configure with WithCacheTTL.

Range requests. RangeReader uses HTTP Range requests against raw.githubusercontent.com for efficient partial reads. If the requested range is already in the in-memory buffer, it's served directly without a network call.

Limitations

  • No real directories: GitHub stores only files. Mkdir creates a .gitkeep placeholder; empty directories cannot be stored.
  • No permission metadata: Chmod, Chown, and Chtimes return ErrNotSupported. GitHub does not track file permissions or timestamps.
  • One commit per Close: Every file flush produces a separate commit. Bulk updates result in N commits, not 1.
  • Size limit: Files larger than GitHub's Contents API size limit (~100 MB) are not supported.
  • No atomic transactions: Individual file operations are atomic, but sequences of operations (e.g., "write file A then file B") are not transactional.

Error Sentinels

The library exports error sentinels for type-safe error handling:

  • ErrNotExist — file or directory does not exist
  • ErrExist — file or directory already exists
  • ErrNotSupported — operation not supported by GitHub (e.g., Chmod)
  • ErrNotImplemented — operation not (yet) implemented
  • ErrAlreadyOpened — file is already open for reading/writing
  • ErrInvalidSeek — invalid seek offset

Errors are wrapped in *os.PathError at method boundaries for standard Go error handling.

Dependencies

  • github.com/google/go-github/v84 — GitHub API client
  • github.com/jferrl/go-githubauth — GitHub authentication helpers
  • github.com/spf13/afero — Abstract filesystem interface
  • golang.org/x/oauth2 — OAuth2 token management

License

Copyright (c) 2026 Kostas Dihalas

Licensed under the Apache License, Version 2.0. See LICENSE for details.

Documentation

Index

Constants

This section is empty.

Variables

View Source 
var ErrAlreadyOpened = errors.New("already opened")

ErrAlreadyOpened is returned when the file is already opened

View Source 
var ErrExist = errors.New("already exists")

ErrExist is returned when a file or dir already exists

View Source 
var ErrInvalidSeek = errors.New("invalid seek offset")

ErrInvalidSeek is returned when the seek operation is not doable

View Source 
var ErrNotExist = errors.New("does not exist")

ErrNotExist is returned when a file or dir does not exist

View Source 
var ErrNotImplemented = errors.New("not implemented")

ErrNotImplemented is returned when this operation is not (yet) implemented

View Source 
var ErrNotSupported = errors.New("git fs doesn't support this operation")

ErrNotSupported is returned when this operations is not supported by Git FS

Functions

This section is empty.

Types

type Client 

type Client struct {
	// contains filtered or unexported fields
}

Client is a wrapper around the github.Client that provides additional functionality and configuration options.

func NewClient 

func NewClient(ctx context.Context, options ...ClientOptions) (*Client, error)

NewClient creates a new Client instance with the provided context and options.

type ClientOptions 

type ClientOptions func(*Client) error

ClientOptions is a function type that defines the signature for configuration options that can be applied to the Client struct.

func WithGithubApplication 

func WithGithubApplication(clientID string, installationID int64, privateKey []byte) ClientOptions

WithGithubApplication is a ClientOption that configures the Client to use GitHub App authentication.

func WithGithubClient 

func WithGithubClient(githubClient *gh.Client) ClientOptions

WithGithubClient is a ClientOption that sets the github.Client directly on the Client struct.

func WithGithubToken 

func WithGithubToken(token string) ClientOptions

WithGithubToken is a ClientOption that configures the Client to use a personal access token for authentication with GitHub.

type File 

type File struct {
	// contains filtered or unexported fields
}

File represents a file in the GitHub repository. It contains a reference to the Fs it belongs to and the name of the file.

func NewFile 

func NewFile(fs *Fs, path, name string) *File

NewFile creates a new File instance with the provided Fs and file name.

func (*File) Close 

func (f *File) Close() error

Close closes the file, flushing any dirty content to GitHub if necessary. It returns an error if the file is already closed or if the flush operation fails.

func (*File) Name 

func (f *File) Name() string

Name returns the name of the file.

func (*File) RangeReader 

func (f *File) RangeReader(offset, length int64) (io.ReadCloser, error)

RangeReader returns an io.ReadCloser that reads a specific byte range from the file. It uses HTTP Range requests against the raw content endpoint to efficiently fetch only the requested portion of the file. It returns an error if the file is closed, is a directory, is not readable, or if the range parameters are invalid.

func (*File) Read 

func (f *File) Read(p []byte) (n int, err error)

Read implements the io.Reader interface for the File. It reads data from the file's reader and returns the number of bytes read and any error encountered. It returns an error if the file is closed, is a directory, or is not readable.

func (*File) ReadAt 

func (f *File) ReadAt(p []byte, off int64) (n int, err error)

ReadAt implements the io.ReaderAt interface for the File. It reads data from the file's reader at the specified offset and returns the number of bytes read and any error encountered.

func (*File) Readdir 

func (f *File) Readdir(count int) ([]os.FileInfo, error)

Readdir reads the contents of the directory and returns a slice of os.FileInfo for the entries. It maintains an internal offset to allow for multiple calls to Readdir, returning subsequent entries until the end of the directory is reached.

func (*File) ReaddirAll 

func (f *File) ReaddirAll() ([]os.FileInfo, error)

ReaddirAll returns all the directory entries. It resets the internal offset to allow for subsequent calls to return the full listing again.

func (*File) Readdirnames 

func (f *File) Readdirnames(count int) ([]string, error)

Readdirnames returns a slice of the names of the directory entries. It uses Readdir to get the entries and extracts their names.

func (*File) Seek 

func (f *File) Seek(offset int64, whence int) (int64, error)

Seek implements the io.Seeker interface for the File. It changes the read/write position in the file based on the offset and whence parameters. It returns the new position and any error encountered. It returns an error if the file is closed, is a directory, or if the whence parameter is invalid.

func (*File) Stat 

func (f *File) Stat() (os.FileInfo, error)

Stat returns the FileInfo structure describing the file. It returns an error if the file is closed.

func (*File) Sync 

func (f *File) Sync() error

Sync flushes the file's content to GitHub if it has been modified (dirty). It returns an error if the file is closed or if the flush operation fails.

func (*File) Truncate 

func (f *File) Truncate(size int64) error

Truncate changes the size of the file to the specified size. If the file is extended, the new bytes are zero-filled. It marks the file as dirty if it was modified. It returns an error if the file is closed or if the file is not writable.

func (*File) Write 

func (f *File) Write(p []byte) (int, error)

Write implements the io.Writer interface for the File. It writes data to the file's buffer and returns the number of bytes written and any error encountered. It marks the file as dirty if it was modified. It returns an error if the file is closed, is a directory, or is not writable.

func (*File) WriteAt 

func (f *File) WriteAt(p []byte, off int64) (int, error)

WriteAt implements the io.WriterAt interface for the File. It writes data to the file's buffer at the specified offset and returns the number of bytes written and any error encountered. It marks the file as dirty if it was modified. It returns an error if the file is closed, is a directory, or is not writable.

func (*File) WriteString 

func (f *File) WriteString(s string) (n int, err error)

WriteString writes the string s to the file. It returns the number of bytes written and any error encountered. It marks the file as dirty if it was modified. It returns an error if the file is closed, is a directory, or is not writable.

type Fs 

type Fs struct {
	// contains filtered or unexported fields
}

Fs is an FS object backed by a GitHub repository. It provides methods to interact with the repository's file system.

func NewFsFromClient 

func NewFsFromClient(client *Client, owner, repo, branch string, opts ...Option) *Fs

NewFsFromClient creates a new Fs instance using the provided Client and repository name.

func (*Fs) Branch 

func (fs *Fs) Branch() string

Branch returns the branch name, or an empty string if not set.

func (*Fs) Chmod 

func (fs *Fs) Chmod(name string, mode os.FileMode) error

func (*Fs) Chown 

func (fs *Fs) Chown(name string, uid, gid int) error

func (*Fs) Chtimes 

func (fs *Fs) Chtimes(name string, atime, mtime time.Time) error

func (*Fs) Create 

func (fs *Fs) Create(name string) (afero.File, error)

Create creates a new file with the specified name. It is a wrapper around OpenFile with appropriate flags for creating a new file.

func (*Fs) Mkdir 

func (fs *Fs) Mkdir(name string, perm os.FileMode) error

Mkdir creates a new directory with the specified name. Since GitHub doesn't have real directories, this is implemented by creating a placeholder file (".gitkeep) in the target directory.

func (*Fs) MkdirAll 

func (fs *Fs) MkdirAll(path string, perm os.FileMode) error

MkdirAll creates a directory and all necessary parents. It iteratively checks each level of the path and creates directories as needed.

func (*Fs) Name 

func (*Fs) Name() string

Name returns the name of the file system, which is "github" in this case.

func (*Fs) Open 

func (fs *Fs) Open(name string) (afero.File, error)

Open opens the named file for reading. It is a wrapper around OpenFile with appropriate flags for read-only access.

func (*Fs) OpenFile 

func (fs *Fs) OpenFile(name string, flag int, perm os.FileMode) (afero.File, error)

OpenFile opens the named file with specified flags and permissions. It handles various scenarios such as reading from cache, fetching from GitHub, and preparing files for writing.

func (*Fs) Owner 

func (fs *Fs) Owner() string

Owner returns the repository owner, or an empty string if not set.

func (*Fs) Remove 

func (fs *Fs) Remove(name string) error

Remove deletes the specified file from the GitHub repository. It first ensures that the file exists and retrieves its SHA, then it constructs a commit message and options for the GitHub API call to delete the file. If the deletion is successful, it evicts the file from the cache.

func (*Fs) RemoveAll 

func (fs *Fs) RemoveAll(path string) error

RemoveAll deletes the specified file or directory and all its contents from the GitHub repository. If the path is a directory, it recursively deletes all child files and directories before deleting the directory itself. If the path does not exist, it returns nil.

func (*Fs) Rename 

func (fs *Fs) Rename(oldname, newname string) error

Rename renames (moves) a file from oldname to newname. It reads the content of the source file, writes it to the destination path, and then deletes the source file. If any step fails, it returns an appropriate error.

func (*Fs) Repo 

func (fs *Fs) Repo() string

Repo returns the repository name, or an empty string if not set.

func (*Fs) Stat 

func (fs *Fs) Stat(name string) (os.FileInfo, error)

Stat retrieves the FileInfo for the specified path. It first checks the cache for a valid entry, and if not found or invalid, it queries GitHub. It handles both file and directory responses from GitHub, populating the SHA cache as needed. If the path does not exist, it returns an os.PathError with ErrNotExist.

type GitHubFileInfo 

type GitHubFileInfo struct {
	// contains filtered or unexported fields
}

GitHubFileInfo implements os.FileInfo.

func (*GitHubFileInfo) IsDir 

func (fi *GitHubFileInfo) IsDir() bool

func (*GitHubFileInfo) ModTime 

func (fi *GitHubFileInfo) ModTime() time.Time

func (*GitHubFileInfo) Mode 

func (fi *GitHubFileInfo) Mode() os.FileMode

func (*GitHubFileInfo) Name 

func (fi *GitHubFileInfo) Name() string

func (*GitHubFileInfo) Size 

func (fi *GitHubFileInfo) Size() int64

func (*GitHubFileInfo) Sys 

func (fi *GitHubFileInfo) Sys() any

type Option 

type Option func(*Fs)

Option is a functional option for GitHubFs.

func WithAPITimeout 

func WithAPITimeout(d time.Duration) Option

WithAPITimeout overrides the per-request timeout (15 s).

func WithCacheTTL 

func WithCacheTTL(d time.Duration) Option

WithCacheTTL overrides the default cache TTL (5 min).

func WithCommitAuthor 

func WithCommitAuthor(name, email string) Option

WithCommitAuthor sets the author stamped on every commit.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.