provider

type BatchSink ¶

type BatchSink struct {
	// contains filtered or unexported fields
}

BatchSink is the Sink handed to a provider for one unit. It owns every record from acceptance until the destination has persisted it: a record is admitted only after its bytes are charged to the pool, a batch flushes as soon as the next record would exceed either limit, and a failed write latches the sink, discards every queued batch (returning its bytes) and cancels every producer of the unit. Nothing here reaches the database except through the destination, which in production is storage's UnitWriter. It is safe for concurrent use by a provider's workers; writes to the destination are serialized. The owner must call Discard when the unit is finished, whatever the outcome; RunUnit does.

func NewBatchSink(ctx context.Context, dst Sink, limits Limits, pool *Pool, cancel context.CancelCauseFunc) (*BatchSink, error)

func (s *BatchSink) Bytes() uint64

func (s *BatchSink) Degradations() []Degradation

func (s *BatchSink) Discard()

func (s *BatchSink) Flush(ctx context.Context) error

func (s *BatchSink) PutAliases(ctx context.Context, aliases []model.NativeAlias) error

func (s *BatchSink) PutKeyedNodes(ctx context.Context, facts []model.NodeFact, keys [][]string) error

func (s *BatchSink) PutKeyedRelations(ctx context.Context, facts []model.RelationFact, keys [][]string) error

func (s *BatchSink) PutNodes(ctx context.Context, facts []model.NodeFact) error

func (s *BatchSink) PutRelations(ctx context.Context, facts []model.RelationFact) error

func (s *BatchSink) PutSearchUnits(ctx context.Context, docs []model.SearchUnit) error

func (s *BatchSink) Records() uint64

func (s *BatchSink) Reserve(ctx context.Context, n int64) (release func(), err error)

type Degradation ¶

type Degradation struct {
	// Limit is the configuration key whose user-set value was exceeded.
	Limit string
	// Bound is the value the user set.
	Bound int64
	// Count is how many records and reservations exceeded it.
	Count uint64
	// Largest is the largest size observed over the bound.
	Largest int64
}

Degradation is the sink's one degradation surface: a bound a unit's records exceeded, counted, so that exceeding it is reported rather than failing the unit. It is the sink's equivalent of the dropped-and-counted rows the bundled providers already publish, and RunUnit folds it into the unit's capability rows as bounded details the generation stores.

type DeltaSink ¶

type DeltaSink interface {
	Sink
	PutKeyedNodes(ctx context.Context, facts []model.NodeFact, keys [][]string) error
	PutKeyedRelations(ctx context.Context, facts []model.RelationFact, keys [][]string) error
}

DeltaSink is the optional extension of Sink for a provider that imports incrementally (Section 11.4). Its destination records, alongside each fact, every one of the producer's own id-independent keys — the keys the provider's next run classifies as changed, unchanged or removed. Storage needs them because a removed fact cannot be named any other way: a fact identity is derived from the resolved entities, which an edit changes, and a cross-file edge can disappear while every file holding its evidence is untouched.

keys is either empty, meaning the batch carries no keys, or parallel to facts: keys[i] is every key backing facts[i] — at least one, each a lowercase hex digest, sorted and without duplicates. One fact is backed by several keys whenever the producer's keys are finer than the identities they resolve to, which is ordinary: a canonical edge is published once and derived from N occurrences, each with its own key. The destination drops the fact when any of them is replaced, which is the condition the producer re-emits the whole fact on.

A destination that does not implement DeltaSink cannot serve a keyed batch; the sink refuses it rather than dropping the keys, because a unit written without them silently cannot be delta-refreshed.

type Detection ¶

type Detection struct {
	Available bool `json:"available"`
	// DiagnosticCode is the Section 22 code explaining an unavailable result,
	// for example CTX_TRUST_REQUIRED for an unapproved executable or
	// CTX_PROVIDER_UNAVAILABLE for a tool that is not installed.
	DiagnosticCode string `json:"diagnostic_code,omitempty"`
	// Capabilities are the declared capabilities this provider can actually
	// offer here; a subset of its descriptor's list.
	Capabilities []string `json:"capabilities,omitempty"`
	// InputPaths are the root-relative inputs detection recognized, bounded by
	// MaxDetectionInputs.
	InputPaths []string `json:"input_paths,omitempty"`
	// ObservedVersion is the exact tool version detection observed for an
	// external analyzer (for example the output of a trusted `--version`
	// probe). It is empty for bundled providers. The coordinator folds it into
	// UnitSpec.ProviderVersion so units are keyed by the tool that actually
	// produced them; Descriptor() itself stays static.
	ObservedVersion string `json:"observed_version,omitempty"`
	// Details carry the machine-readable particulars of a detection that
	// DiagnosticCode cannot hold, as bounded key/value pairs. A provider that
	// covers several languages through several payloads is available as soon as
	// one of them resolves, and then has no other place to say that another
	// language's indexer is absent and why: it plans no unit for that language,
	// so no capability row is ever produced for it either. One pair per
	// affected input -- `details["rust-analyzer"] = "CTX_TOOL_OFFLINE"` --
	// keeps that typed rather than silent.
	Details map[string]string `json:"details,omitempty"`
	// Reason is the one bounded phrase that says, in the product's own words,
	// why a detection is unavailable. A diagnostic code alone names the
	// category and not the finding -- CTX_PROVIDER_UNAVAILABLE on a
	// repository whose projects simply sit in subdirectories reads as "this
	// provider is broken or absent" -- and Details deliberately do not
	// survive an unavailable detection, so this is where the finding goes.
	// It is empty for an available detection, which has its rows to speak
	// with.
	Reason string `json:"reason,omitempty"`
}

Detection is a provider's answer to "can you run against this workspace". Available false with a DiagnosticCode is the honest optional-absence outcome of Section 11.1: a missing or unapproved tool is unavailable, which is distinct from a failure. Detection must not walk the whole repository into memory; it inspects declared inputs through the confined Root.

func (d Detection) Validate(desc model.ProviderDescriptor) error

func (d Detection) WithDetail(key, value string) Detection

type Limits ¶

type Limits struct {
	BatchRecords   int
	BatchBytes     int64
	MaxRecordBytes int64
}

Limits are the per-batch bounds of Section 11.1: a batch is capped by both records and retained bytes, and flushes at whichever is reached first. MaxRecordBytes is resources.max_provider_record_bytes, and zero is unlimited: no shipped value refuses a record for being large. A record over a limit the USER set is admitted and REPORTED -- the sink counts it as a degradation the unit's capability rows carry -- because a size key names what the answer should tell the caller about, not what it should refuse. Nothing here is what bounds memory: the batch reservations and the pool are.

func (l Limits) Validate() error

type Pool ¶

type Pool struct {
	// contains filtered or unexported fields
}

Pool is the retained-byte reservation shared by every sink of one indexing run (index.queue_bytes). Queued records and outstanding decode reservations are both charged against it, so the bytes providers hold before persistence are bounded across concurrent units, not only per batch.

The pool also tracks its live sinks. An acquirer that cannot be charged first asks every live sink, its own included, to persist what it has queued, so one sink that has stopped producing cannot pin the pool with a half-filled batch while another starves.

func NewPool(capacity int64) (*Pool, error)

func (p *Pool) Used() int64

type Provider ¶

type Provider interface {
	Descriptor() model.ProviderDescriptor
	Detect(context.Context, workspace.Root, workspace.Policy) (Detection, error)
	IndexUnit(context.Context, UnitRequest, Sink) (model.ProviderResult, error)
}

Provider is one fact producer. Descriptor is static identity and scheduling contract; Detect inspects the workspace without indexing it; IndexUnit produces exactly the assigned unit through the sink.

Detect takes the confined workspace root and traversal policy rather than the sketch's `workspace.View`, which does not exist: Root is the only safe way to open a repository file and Policy is what decides eligibility.

type Registry ¶

type Registry struct {
	// contains filtered or unexported fields
}

Registry is the validated provider DAG of Section 13.1. It is built once by the composition owner from the complete provider list; construction rejects a duplicate ID, a dependency on an unknown provider and a dependency cycle, because a coordinator that scheduled such a graph could report units as ready whose dependencies can never complete.

func NewRegistry(providers ...Provider) (*Registry, error)

func (r *Registry) Lookup(id string) (Provider, bool)

func (r *Registry) Providers() []Provider

func (r *Registry) Select(ctx context.Context, root workspace.Root, policy workspace.Policy,
	enablement func(providerID string) config.Enablement) (Selection, error)

type Resolver ¶

type Resolver interface {
	Resolve(context.Context, model.NodeCandidate) (model.Resolution, error)
}

Resolver reconciles a provider candidate to canonical identity. The result depends only on the candidate and the set of sealed dependency units, never on scheduling (Section 9.4).

type Selection ¶

type Selection struct {
	Active []Provider
	States []model.CapabilityState
	// Detections holds the validated Detection of every Active provider, by
	// provider ID. The planner needs two things only a Detection carries --
	// ObservedVersion, folded into UnitSpec.ProviderVersion, and InputPaths,
	// which scope the SCIP profile units -- and must not re-run detection
	// against the live checkout when the plan is about the pinned snapshot.
	Detections map[string]Detection
}

Selection is the outcome of detection over one workspace. Active holds the providers that will run, in dependency order. States holds every capability row detection publishes, at ScopeWorkspace and one per declared capability: `unavailable` for an absent optional provider, `failed` for an enabled provider that could not be detected, and `partial` for a provider that *will* run but whose detection named something inside it that cannot (Detection.Details). The categories are kept apart because an optional tool that is absent must not make a healthy base generation falsely fail, while an enabled requested capability that fails makes the generation degraded (Sections 11.1, 13.3); a partial row is the middle case -- the provider runs, and the coordinator publishes the reasons the rest of it will not.

A provider the configuration disabled publishes NO row at all. `unavailable` is the record of work that was wanted and not attempted, each row carrying the reason it was not; nobody asked a disabled provider for anything, so it has nothing to report per capability, and a report full of `unavailable` rows for it reads as a degraded index rather than a configured one. That a provider is off is said once, by the coordinator, on the result and in the log -- not once per capability.

type Sink ¶

type Sink interface {
	PutNodes(context.Context, []model.NodeFact) error
	PutRelations(context.Context, []model.RelationFact) error
	PutAliases(context.Context, []model.NativeAlias) error
	PutSearchUnits(context.Context, []model.SearchUnit) error
}

Sink receives a unit's facts. Ownership of a handed-off slice transfers to the sink: the provider must not retain or mutate it afterwards. A call blocks while the sink's retained-byte reservation is exhausted and returns promptly on cancellation. A single record over a user-set resources.max_provider_record_bytes is admitted and counted as a degradation the unit's capability rows report; it never fails the unit. What bounds the heap is the batch reservation and the shared pool, not that key.

Hand a node fact to the sink before any relation, alias or search document that references its identity. Batch flush timing is not under the provider's control: a batch may be persisted at any call, including by another unit relieving pool pressure, and storage requires a relation's endpoints and an alias's target to be registered identities when the row is written.

type UnitOutput ¶

type UnitOutput interface {
	Sink
	Seal(context.Context) error
	Fail(context.Context) error
	Abandon(context.Context) error
}

UnitOutput is the storage side of one building unit: the writer the sink drains into, sealing after whole-unit validation, and discarding on failure. Seal is the only path to generation membership; Fail deletes every row the unit wrote so unsealed output can never be queried (Section 11.1). Abandon is the cancel-path discard: it gives up the unit without the cascading delete, leaving the rows for storage's own collection, which is what keeps an interrupt from waiting on a rollback nobody is going to read.

func StoreUnit(store *sqlite.Store, w *sqlite.UnitWriter) UnitOutput

type UnitRequest ¶

type UnitRequest struct {
	Binding  model.Binding
	Unit     model.UnitSpec
	Run      model.ProviderRunID
	Content  model.SnapshotView
	Resolver Resolver
}

UnitRequest is one assigned unit of work. Binding is the staging generation the unit is built for; Unit is the immutable identity storage has already opened; Content is the pinned snapshot view every byte must be read from; Resolver reconciles candidates against the unit's completed dependencies.

Run is not in the Section 11.1 sketch. It is carried because every Evidence row must name the producing run (storage rejects evidence for another run) and ProviderResult.RunID must echo it; without it a provider could not build a single valid fact.