store

package
v0.10.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Aug 21, 2026 License: MIT Imports: 30 Imported by: 0

Documentation

Overview

Package store provides abstractions for file storage operations.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func ParseGitHubRepo added in v0.10.0

func ParseGitHubRepo(rawURL string) (owner, repo string, err error)

ParseGitHubRepo extracts the owner and repository name from a GitHub URL. It accepts https://, ssh:// and scp-style (git@github.com:owner/repo) forms, with or without a ".git" suffix and with or without embedded credentials.

Types

type FileInfo

type FileInfo struct {
	Path    string
	IsDir   bool
	Size    int64
	ModTime time.Time
}

FileInfo represents file metadata.

type GitHubStore added in v0.10.0

type GitHubStore struct {
	// contains filtered or unexported fields
}

GitHubStore implements Store on top of the GitHub Git Data API.

It holds no working tree and writes nothing to the local filesystem: reads descend the tree one directory at a time and writes are buffered until the transaction commits, which becomes a blob/tree/commit/ref sequence.

func NewGitHubStore added in v0.10.0

func NewGitHubStore(cfg *RemoteConfig, opts ...GitHubStoreOption) (*GitHubStore, error)

NewGitHubStore validates the configuration and builds a GitHub API store. It fails immediately when NTN_GIT_URL is not a github.com repository URL or NTN_GIT_PASS is missing.

func (*GitHubStore) BeginTx added in v0.10.0

func (s *GitHubStore) BeginTx(_ context.Context) (Transaction, error)

BeginTx starts a buffered transaction.

func (*GitHubStore) Branch added in v0.10.0

func (s *GitHubStore) Branch() string

Branch returns the branch this store reads and writes.

func (*GitHubStore) CheckWholeTreeSupported added in v0.10.0

func (s *GitHubStore) CheckWholeTreeSupported(command string) error

CheckWholeTreeSupported rejects commands that need to walk the whole repository. Those need the truncation-prone recursive listing this backend deliberately avoids, so they fail fast instead of half-working.

func (*GitHubStore) Exists added in v0.10.0

func (s *GitHubStore) Exists(ctx context.Context, path string) (bool, error)

Exists reports whether a path resolves to an object, pending writes included.

func (*GitHubStore) IsRemoteEnabled added in v0.10.0

func (s *GitHubStore) IsRemoteEnabled() bool

IsRemoteEnabled always returns true: the GitHub backend has no local mode.

func (*GitHubStore) List added in v0.10.0

func (s *GitHubStore) List(ctx context.Context, dir string) ([]FileInfo, error)

List lists one directory with a single non-recursive tree fetch, merged with the pending writes and deletions of any open transaction.

func (*GitHubStore) Lock added in v0.10.0

func (s *GitHubStore) Lock()

Lock acquires the store's write lock for external coordination.

func (*GitHubStore) Pull added in v0.10.0

func (s *GitHubStore) Pull(ctx context.Context) error

Pull refreshes the cached branch HEAD and reports the rate-limit budget.

func (*GitHubStore) Push added in v0.10.0

func (s *GitHubStore) Push(ctx context.Context) error

Push is a no-op: a committed transaction is already on the remote.

func (*GitHubStore) Read added in v0.10.0

func (s *GitHubStore) Read(ctx context.Context, path string) ([]byte, error)

Read reads a file, preferring any pending write from an open transaction.

func (*GitHubStore) RemoteConfig added in v0.10.0

func (s *GitHubStore) RemoteConfig() *RemoteConfig

RemoteConfig returns the remote configuration backing this store.

func (*GitHubStore) TestConnection added in v0.10.0

func (s *GitHubStore) TestConnection(ctx context.Context) error

TestConnection verifies credentials and repository access.

func (*GitHubStore) Unlock added in v0.10.0

func (s *GitHubStore) Unlock()

Unlock releases the store's write lock.

type GitHubStoreOption added in v0.10.0

type GitHubStoreOption func(*gitHubStoreOptions)

GitHubStoreOption configures a GitHubStore.

func WithGitHubBranch added in v0.10.0

func WithGitHubBranch(branch string) GitHubStoreOption

WithGitHubBranch overrides the branch taken from the remote configuration. The queue store uses it to target NTN_QUEUE_BRANCH.

func WithGitHubStoreLogger added in v0.10.0

func WithGitHubStoreLogger(logger *slog.Logger) GitHubStoreOption

WithGitHubStoreLogger sets the logger for the store and its API client.

type LocalStore

type LocalStore struct {
	// contains filtered or unexported fields
}

LocalStore implements Store using local filesystem and git.

func NewLocalStore

func NewLocalStore(path string, opts ...LocalStoreOption) (*LocalStore, error)

NewLocalStore creates a new local store at the given path.

func (*LocalStore) BeginTx

func (s *LocalStore) BeginTx(_ context.Context) (Transaction, error)

BeginTx starts a new transaction.

func (*LocalStore) Exists

func (s *LocalStore) Exists(ctx context.Context, path string) (bool, error)

Exists checks if a file exists.

func (*LocalStore) FS

func (s *LocalStore) FS() fs.FS

FS returns an fs.FS view of the store.

func (*LocalStore) IsRemoteEnabled

func (s *LocalStore) IsRemoteEnabled() bool

IsRemoteEnabled returns true if remote git operations are configured.

func (*LocalStore) List

func (s *LocalStore) List(ctx context.Context, dir string) ([]FileInfo, error)

List lists files in a directory.

func (*LocalStore) Lock

func (s *LocalStore) Lock()

Lock acquires the store's write lock for external coordination.

func (*LocalStore) Pull

func (s *LocalStore) Pull(ctx context.Context) error

Pull fetches and merges changes from the remote repository.

func (*LocalStore) Push

func (s *LocalStore) Push(ctx context.Context) error

Push pushes local commits to the remote repository. If a non-fast-forward error occurs, it will attempt to pull first and retry the push.

func (*LocalStore) Read

func (s *LocalStore) Read(ctx context.Context, path string) ([]byte, error)

Read reads a file from the store.

func (*LocalStore) RemoteConfig

func (s *LocalStore) RemoteConfig() *RemoteConfig

RemoteConfig returns the remote configuration.

func (*LocalStore) TestConnection

func (s *LocalStore) TestConnection(ctx context.Context) error

TestConnection tests the connection to the remote repository.

func (*LocalStore) Unlock

func (s *LocalStore) Unlock()

Unlock releases the store's write lock.

type LocalStoreOption

type LocalStoreOption func(*LocalStore)

LocalStoreOption configures LocalStore.

func WithCreateBranchIfMissing added in v0.8.0

func WithCreateBranchIfMissing() LocalStoreOption

WithCreateBranchIfMissing makes the store initialize a fresh local branch (pushed lazily on first commit) when the configured branch does not yet exist on the remote, instead of failing to clone. Used for the queue branch.

func WithLogger

func WithLogger(l *slog.Logger) LocalStoreOption

WithLogger sets a custom logger for the store.

func WithRemoteConfig

func WithRemoteConfig(cfg *RemoteConfig) LocalStoreOption

WithRemoteConfig sets the remote git configuration.

type ReadFSProvider

type ReadFSProvider interface {
	FS() fs.FS
}

ReadFSProvider returns an fs.FS view for read-only consumers.

type RemoteConfig

type RemoteConfig struct {
	Storage      StorageMode   // Storage mode: "local", "remote", or auto-detect (NTN_STORAGE)
	URL          string        // Remote git repository URL (NTN_GIT_URL)
	Password     string        // Password/token for HTTPS auth (NTN_GIT_PASS)
	Branch       string        // Target branch (NTN_GIT_BRANCH)
	QueueBranch  string        // Separate branch for the queue (NTN_QUEUE_BRANCH), empty = disabled
	User         string        // Commit author name (NTN_GIT_USER)
	Email        string        // Commit author email (NTN_GIT_EMAIL)
	Commit       bool          // Enable automatic git commit (NTN_COMMIT)
	CommitPeriod time.Duration // Periodic commit interval during sync (NTN_COMMIT_PERIOD)
	Push         *bool         // Push to remote after commits (NTN_PUSH), nil means auto-detect
	Depth        int           // Shallow clone/fetch depth (NTN_GIT_DEPTH), 0 = full history
}

RemoteConfig holds configuration for remote git operations.

func LoadRemoteConfigFromEnv

func LoadRemoteConfigFromEnv() *RemoteConfig

LoadRemoteConfigFromEnv loads remote configuration from environment variables.

func (*RemoteConfig) EffectiveStorageMode

func (c *RemoteConfig) EffectiveStorageMode() StorageMode

EffectiveStorageMode returns the effective storage mode after auto-detection. If Storage is set explicitly, it returns that value. Otherwise, it returns "remote" if URL is configured, or "local" if not.

func (*RemoteConfig) GetAuth

func (c *RemoteConfig) GetAuth() (transport.AuthMethod, error)

GetAuth returns the appropriate authentication method for the remote URL.

func (*RemoteConfig) GetCommitPeriod

func (c *RemoteConfig) GetCommitPeriod() time.Duration

GetCommitPeriod returns the periodic commit interval.

func (*RemoteConfig) GitHubRepo added in v0.10.0

func (c *RemoteConfig) GitHubRepo() (owner, repo string, err error)

GitHubRepo parses NTN_GIT_URL into a GitHub owner and repository name and validates that a token is available. It is the startup validation for NTN_STORAGE=github.

func (*RemoteConfig) HasQueueBranch added in v0.8.0

func (c *RemoteConfig) HasQueueBranch() bool

HasQueueBranch returns true if a separate queue branch is configured.

func (*RemoteConfig) IsCommitEnabled

func (c *RemoteConfig) IsCommitEnabled() bool

IsCommitEnabled returns true if automatic commits are enabled.

func (*RemoteConfig) IsEnabled

func (c *RemoteConfig) IsEnabled() bool

IsEnabled returns true if remote operations should be used. This checks both the storage mode and whether a URL is configured.

func (*RemoteConfig) IsGitHubAPI added in v0.10.0

func (c *RemoteConfig) IsGitHubAPI() bool

IsGitHubAPI returns true when the GitHub Git Data API backend is selected.

func (*RemoteConfig) IsPushEnabled

func (c *RemoteConfig) IsPushEnabled() bool

IsPushEnabled returns true if push to remote is enabled. When NTN_PUSH is not explicitly set, defaults to true if NTN_GIT_URL is set.

func (*RemoteConfig) IsSSH

func (c *RemoteConfig) IsSSH() bool

IsSSH returns true if the URL is an SSH URL.

func (*RemoteConfig) TestConnection

func (c *RemoteConfig) TestConnection(ctx context.Context) error

TestConnection tests the connection to the remote repository.

type RemoteStore added in v0.10.0

type RemoteStore interface {
	Store

	// Pull refreshes the local view from the remote. It is a no-op when
	// remote operations are not enabled.
	Pull(ctx context.Context) error

	// IsRemoteEnabled reports whether remote operations are configured.
	IsRemoteEnabled() bool

	// RemoteConfig returns the remote configuration backing this store.
	RemoteConfig() *RemoteConfig
}

RemoteStore is a Store that can synchronize with a remote repository. Every concrete store in this package implements it, which is what lets callers avoid type-switching on concrete store types.

type SplitStore added in v0.7.0

type SplitStore struct {
	// contains filtered or unexported fields
}

SplitStore routes operations by path prefix. Paths under ".notion-sync/queue" go to queueStore (the queue branch), everything else — including content, ".notion-sync/ids", and ".notion-sync/state.json" — goes to contentStore (the main branch).

func NewSplitStore added in v0.7.0

func NewSplitStore(contentStore, queueStore RemoteStore) *SplitStore

NewSplitStore creates a new SplitStore that routes queue operations to a separate store. Both stores are held as interface values so any backend (local git or GitHub API) can back either branch.

func (*SplitStore) BeginTx added in v0.7.0

func (s *SplitStore) BeginTx(ctx context.Context) (Transaction, error)

BeginTx starts a split transaction that routes writes to the correct store.

func (*SplitStore) CheckWholeTreeSupported added in v0.10.0

func (s *SplitStore) CheckWholeTreeSupported(command string) error

CheckWholeTreeSupported delegates to the content store when it advertises the capability, since whole-tree operations always run against content.

func (*SplitStore) ContentStore added in v0.7.0

func (s *SplitStore) ContentStore() RemoteStore

ContentStore returns the underlying content store.

func (*SplitStore) Exists added in v0.7.0

func (s *SplitStore) Exists(ctx context.Context, path string) (bool, error)

Exists checks if a file exists in the appropriate store.

func (*SplitStore) IsRemoteEnabled added in v0.7.0

func (s *SplitStore) IsRemoteEnabled() bool

IsRemoteEnabled returns true if the content store has remote operations configured.

func (*SplitStore) List added in v0.7.0

func (s *SplitStore) List(ctx context.Context, dir string) ([]FileInfo, error)

List lists files in the appropriate store.

func (*SplitStore) Lock added in v0.7.0

func (s *SplitStore) Lock()

Lock acquires locks on both stores (content first).

func (*SplitStore) Pull added in v0.7.0

func (s *SplitStore) Pull(ctx context.Context) error

Pull fetches and merges changes from remote for both stores.

func (*SplitStore) Push added in v0.7.0

func (s *SplitStore) Push(ctx context.Context) error

Push pushes both stores to their respective remotes.

func (*SplitStore) QueueStore added in v0.8.0

func (s *SplitStore) QueueStore() RemoteStore

QueueStore returns the underlying queue store.

func (*SplitStore) Read added in v0.7.0

func (s *SplitStore) Read(ctx context.Context, path string) ([]byte, error)

Read reads a file from the appropriate store.

func (*SplitStore) RemoteConfig added in v0.7.0

func (s *SplitStore) RemoteConfig() *RemoteConfig

RemoteConfig returns the content store's remote configuration.

func (*SplitStore) Unlock added in v0.7.0

func (s *SplitStore) Unlock()

Unlock releases locks on both stores (queue first to avoid deadlock).

type StorageMode

type StorageMode string

StorageMode defines the storage mode for git operations.

const (
	// StorageModeAuto automatically detects the storage mode based on configuration.
	StorageModeAuto StorageMode = ""
	// StorageModeLocal uses local-only storage (no remote operations).
	StorageModeLocal StorageMode = "local"
	// StorageModeRemote uses remote storage (pull/push enabled).
	StorageModeRemote StorageMode = "remote"
	// StorageModeGitHub talks to the GitHub Git Data API directly, with no
	// clone and no working tree. Opt-in only, see docs/github-storage.md.
	StorageModeGitHub StorageMode = "github"
)

type Store

type Store interface {
	// Read operations
	Read(ctx context.Context, path string) ([]byte, error)
	Exists(ctx context.Context, path string) (bool, error)
	List(ctx context.Context, dir string) ([]FileInfo, error)

	// Transaction management - all writes go through transactions
	BeginTx(ctx context.Context) (Transaction, error)

	// Remote operations
	Push(ctx context.Context) error

	// Concurrency control for external coordination (e.g., sync worker)
	Lock()
	Unlock()
}

Store abstracts file storage with transactional write operations.

type Transaction

type Transaction interface {
	// Write operations - applied immediately to filesystem
	Write(ctx context.Context, path string, content []byte) error
	WriteStream(ctx context.Context, path string, reader io.Reader) (int64, error)
	Delete(ctx context.Context, path string) error
	Mkdir(ctx context.Context, path string) error

	// Commit creates a git commit with all changes made in this transaction.
	// After commit, the transaction can continue to be used for more changes.
	Commit(ctx context.Context, message string) error

	// Rollback reverts all uncommitted changes and closes the transaction.
	Rollback(ctx context.Context) error
}

Transaction groups multiple write operations. All writes are applied immediately to the filesystem. Commit creates a git commit with all changes. Rollback reverts uncommitted changes.

type WholeTreeChecker added in v0.10.0

type WholeTreeChecker interface {
	// CheckWholeTreeSupported returns a non-nil error naming the storage mode
	// when the named command needs a whole-tree walk the store cannot provide.
	CheckWholeTreeSupported(command string) error
}

WholeTreeChecker is implemented by stores that cannot serve operations requiring a walk of the entire repository tree. Stores that do not implement it are assumed to support every operation.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL