jsonl

package
v0.4.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 3, 2026 License: MIT Imports: 15 Imported by: 0

Documentation

Overview

Package jsonl is the portable JSONL implementation of the durable Storage contract: one main commit log and per-document and per-task sidecar logs in one directory, accessed through a file system capability.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type FileSystem

type FileSystem interface {
	AbsolutePath(ctx context.Context, path string) (string, error)
	JoinPath(ctx context.Context, parts []string) (string, error)
	ReadBinaryFile(ctx context.Context, path string) ([]byte, error)
	WriteFile(ctx context.Context, path string, content any) error
	AppendFile(ctx context.Context, path string, content any) error
	TruncateFile(ctx context.Context, path string, size int64) error
	FlushFile(ctx context.Context, path string) error
	RenameFile(ctx context.Context, sourcePath, destinationPath string) error
	ListDir(ctx context.Context, path string) ([]env.FileInfo, error)
	CreateDir(ctx context.Context, path string, options *env.CreateDirOptions) error
	Remove(ctx context.Context, path string, options *env.RemoveOptions) error
}

FileSystem is the part of env.FileSystem that JSONL storage uses. Every env.FileSystem satisfies it.

type JsonlCorruptionError

type JsonlCorruptionError struct {
	Message string
	Cause   error
}

JsonlCorruptionError reports complete persisted data that cannot be recovered.

func (*JsonlCorruptionError) Error

func (e *JsonlCorruptionError) Error() string

func (*JsonlCorruptionError) Unwrap

func (e *JsonlCorruptionError) Unwrap() error

Unwrap returns the cause.

type JsonlStorage

type JsonlStorage struct {
	// contains filtered or unexported fields
}

JsonlStorage is the portable JSONL implementation of durable.Storage. An in-memory MemoryStorage holds the recovered state; every commit is published to the files before it is applied there.

func Open

func Open(ctx context.Context, directory string, fs FileSystem, options JsonlStorageOptions) (*JsonlStorage, error)

Open opens or creates a JSONL storage directory using the supplied file system and recovers its state.

func (*JsonlStorage) Close

func (jsonl *JsonlStorage) Close(ctx context.Context) error

Close releases the storage; later operations fail. Repeated calls do nothing.

func (*JsonlStorage) Commit

func (jsonl *JsonlStorage) Commit(ctx context.Context, writes []durable.StorageWrite) (durable.Seq, error)

Commit publishes every sidecar record, then the main marker, then applies the batch in memory and reclaims sidecars whose content the batch superseded.

func (*JsonlStorage) Conversation

Conversation looks up one conversation by exact ID.

func (*JsonlStorage) Document

Document materializes one specific incarnation by ID at the selected point.

func (*JsonlStorage) Entry

func (jsonl *JsonlStorage) Entry(ctx context.Context, id durable.EntryId) (*durable.EntryAt, error)

Entry looks up one global entry and the sequence of the commit that persisted it.

func (*JsonlStorage) FindDocument

func (jsonl *JsonlStorage) FindDocument(
	ctx context.Context, address durable.DocumentAddress, at durable.DocumentPoint,
) (*durable.DocumentRecord, error)

FindDocument resolves the incarnation occupying one exact logical address at the selected point.

func (*JsonlStorage) FindLatestHeadMarker

func (jsonl *JsonlStorage) FindLatestHeadMarker(
	ctx context.Context, conversationId durable.ConversationId, atOrBeforeEntryId *durable.EntryId,
) (*durable.EntryRecord, error)

FindLatestHeadMarker returns the newest visible entry with a head at or below the optional inclusive cutoff.

func (*JsonlStorage) MintId

func (jsonl *JsonlStorage) MintId() (int64, error)

MintId returns a fresh candidate from the Session-global numeric ID namespace.

func (*JsonlStorage) ScanConversations

func (jsonl *JsonlStorage) ScanConversations(
	ctx context.Context, query durable.ConversationQuery, limit int, cursor durable.Cursor,
) (durable.Page[durable.ConversationRecord, durable.Cursor], error)

ScanConversations scans conversations in ascending ID order.

func (*JsonlStorage) ScanDocuments

func (jsonl *JsonlStorage) ScanDocuments(
	ctx context.Context, query durable.DocumentQuery, limit int, cursor durable.Cursor,
) (durable.Page[durable.DocumentRecord, durable.Cursor], error)

ScanDocuments scans incarnations alive in one exact scope at the selected point.

func (*JsonlStorage) ScanEntries

func (jsonl *JsonlStorage) ScanEntries(
	ctx context.Context, query durable.EntryQuery, limit int, cursor durable.Cursor,
) (durable.Page[durable.EntryRecord, durable.Cursor], error)

ScanEntries scans the inclusive visible range newest-first.

func (*JsonlStorage) ScanSubmissions

func (jsonl *JsonlStorage) ScanSubmissions(
	ctx context.Context, query durable.SubmissionQuery, limit int, cursor durable.Cursor,
) (durable.Page[durable.SubmissionRecord, durable.Cursor], error)

ScanSubmissions scans submissions matching every supplied filter in ascending ID order.

func (*JsonlStorage) ScanTasks

func (jsonl *JsonlStorage) ScanTasks(
	ctx context.Context, query durable.TaskQuery, limit int, cursor durable.Cursor,
) (durable.Page[storedTask, durable.Cursor], error)

ScanTasks scans task records matching every supplied filter.

func (*JsonlStorage) Submission

Submission looks up the latest complete record for one admitted submission.

func (*JsonlStorage) SubmissionByRequest

func (jsonl *JsonlStorage) SubmissionByRequest(
	ctx context.Context, conversationId durable.ConversationId, requestId string,
) (*durable.SubmissionRecord, error)

SubmissionByRequest finds a submission by its conversation-scoped host deduplication key.

func (*JsonlStorage) Task

func (jsonl *JsonlStorage) Task(ctx context.Context, id durable.TaskId) (*storedTask, error)

Task looks up the latest complete record for one task.

func (*JsonlStorage) VisibleEntry

func (jsonl *JsonlStorage) VisibleEntry(
	ctx context.Context, conversationId durable.ConversationId, id durable.EntryId,
) (*durable.EntryAt, error)

VisibleEntry looks up one entry only when it is visible through the requested conversation's ancestry.

type JsonlStorageOptions

type JsonlStorageOptions struct {
	// Fsync flushes every affected sidecar before appending the main marker. Defaults to false.
	Fsync bool
}

JsonlStorageOptions configures JsonlStorage.

type JsonlStoragePoisonedError

type JsonlStoragePoisonedError struct {
	Cause error
}

JsonlStoragePoisonedError reports storage whose files may hold a partial publication; it must be reopened.

func (*JsonlStoragePoisonedError) Error

func (e *JsonlStoragePoisonedError) Error() string

func (*JsonlStoragePoisonedError) Unwrap

func (e *JsonlStoragePoisonedError) Unwrap() error

Unwrap returns the cause.

Directories

Path Synopsis
Package node opens JSONL durable storage on the local file system, the Go counterpart of Pi's Node JSONL adapter.
Package node opens JSONL durable storage on the local file system, the Go counterpart of Pi's Node JSONL adapter.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL