tableregister

package
v1.128.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Sep 1, 2026 License: Apache-2.0 Imports: 24 Imported by: 0

Documentation

Overview

Package tableregister makes a file already in object storage readable as a table, without copying it and without giving anything a write tool.

A managed resource and a portal asset are both stored as one object under a per-object directory. Trino's Hive connector reads CSV from an external location, so "make this file queryable" is a CREATE TABLE naming that directory: no ingestion, no copy, and the table tracks whatever the object holds now. One registrar serves both kinds because a registration says the same thing about either.

What the registrar does NOT do is decide who may register. Every entry point -- the resources REST API, the asset REST API, the manage_table tool -- resolves its own caller and hands one in; the registrar applies the persona connection boundary to that caller and refuses on anything it cannot establish.

Index

Constants

View Source
const (
	// KindResource is a managed resource: a file a person uploaded.
	KindResource = "resource"
	// KindAsset is a portal asset: a file the platform wrote, typically a
	// trino_export or a script's output.
	KindAsset = "asset"
)

Source kinds a registration can be built from. They are the two things a person can put a file into the platform through.

View Source
const (
	// DefaultListLimit is the page size a caller who names none gets.
	DefaultListLimit = 50
	// MaxListLimit is the largest page the store will build, whatever was
	// asked for.
	MaxListLimit = 200
)

The bounds a listing page is served within.

View Source
const DefaultMaxBytes = 100 << 20

DefaultMaxBytes bounds the object the registrar reads to find a header row.

Neither S3 adapter has a range read, so learning the first line costs a full GetObject. The bound matches the managed-resource upload cap, which is the largest object either surface can have put there.

Variables

View Source
var (
	// ErrNotFound is returned for a registration id that does not exist.
	ErrNotFound = errors.New("registration not found")

	// ErrNoScratchTarget means the connection has no scratch: block, so
	// nothing can be registered on it.
	ErrNoScratchTarget = errors.New("this connection has no scratch catalog and schema configured, so a table cannot be registered on it")

	// ErrConnectionReadOnly means the connection names a scratch target but
	// will not run the statement that creates the table.
	//
	// It is the sibling of ErrNoScratchTarget and is answered the same way: a
	// fact about the connection, knowable before the request, that no retry
	// and no different table name changes. Without it the refusal arrived from
	// the Trino interceptor as an unclassified error, which the HTTP surface
	// could only report as a 500 "the registration could not be completed" --
	// a configuration fact rendered as a platform outage, with the one word
	// that explains it ("read-only") dropped on the way out.
	ErrConnectionReadOnly = errors.New("this connection is read-only, so a table cannot be created on it; ask an administrator for a connection that accepts writes")

	// ErrConnectionDenied means the caller's persona is not granted the
	// connection. It is the same boundary a tool call meets.
	ErrConnectionDenied = errors.New("your persona is not granted this connection")

	// ErrNotCSV means the source object is not a CSV, which is the only format
	// a registration can be built from.
	ErrNotCSV = errors.New("only a CSV file can be registered as a table")

	// ErrEmptyHeader means the object had no header row to take columns from.
	ErrEmptyHeader = tablecsv.ErrEmptyHeader

	// ErrNoIdentity means the call carried no identity to register under.
	// Registration records who made it and decides replacement on that, so an
	// anonymous registration would be one nobody owns and anyone could take
	// over.
	ErrNoIdentity = errors.New("registering a table needs a signed-in identity")

	// ErrBadReference means the reference a caller passed is not one this
	// platform issues, or names something that is not a stored file. It is
	// separate from ErrNoSuchFile because the caller can see the difference
	// themselves: the string they sent is malformed, so telling them so
	// discloses nothing they did not already have.
	ErrBadReference = errors.New("not a reference to a stored file")

	// ErrNoSuchFile is the one answer to a reference that resolves to a record
	// this caller may not register: one that does not exist, one that was
	// deleted, and one that exists but belongs to somebody else are answered
	// identically, so the surface never confirms the existence of a record the
	// caller cannot act on. It is what `fetch` does with a reference outside
	// the caller's reach, held to here for the same reason.
	ErrNoSuchFile = errors.New("that reference names no stored file you can register")

	// ErrNeedsRepair marks the refusal of a file that cannot be read as a
	// table the way it is stored but could be if a corrected version of it
	// were saved first. Both surfaces match on it to offer that correction,
	// rather than leaving a person with a refusal and nothing to do about it.
	ErrNeedsRepair = errors.New("this file needs correcting before it can be read as a table")

	// ErrRefused marks a refusal the caller can act on -- a name already
	// taken, a sibling object in the way -- as opposed to a failure of the
	// platform. Every such refusal wraps it, so a surface can answer with a
	// status that says "your request was understood and declined" rather than
	// reporting a store outage as a conflict, or the reverse.
	ErrRefused = errors.New("registration refused")
)

Errors the registrar returns. Every surface renders these, so the wording a person sees comes from one place.

View Source
var ErrNameTaken = errors.New("that table name was registered by someone else while this registration was being made")

ErrNameTaken is returned when the unique index on the table name rejects an insert. The registrar checks for a holder before it writes; this is the race between that check and this write, and it must not surface as a bare constraint violation.

View Source
var ErrUnavailable = errors.New("table registration is not available on this deployment")

ErrUnavailable means the deployment has no registration mechanism wired.

Functions

func BuildDDL

func BuildDDL(r Registration, replacing bool) []string

BuildDDL returns the statements that make a registration, in the order they must run.

CREATE SCHEMA comes first and is IF NOT EXISTS: the scratch schema is the target of every registration on a connection and the first one to arrive has to make it. DROP TABLE is issued only when replacing a registration the caller is entitled to replace -- an unconditional drop would let a name collision quietly take out somebody else's table, which is why the decision is made before this is called rather than here.

Every column is VARCHAR because Hive CSV admits nothing else; skipping the header line is what keeps the column names out of the rows.

func DirectoryOf

func DirectoryOf(key string) string

DirectoryOf returns the directory portion of an object key, with its trailing slash. A key with no directory yields the empty string, which no registration can be built on.

func LocationURI

func LocationURI(bucket, dir string) string

LocationURI renders the external location a directory in a bucket is addressed by.

func ParseReference

func ParseReference(reference string) (kind, id string, err error)

ParseReference resolves the canonical reference an agent already holds -- the string `search` emits on a hit and `fetch` dereferences -- into the kind and id a registration is built over.

It is what makes one action serve every kind of stored file. The platform has exactly one vocabulary for naming a record across tools, so a registration keyed by that vocabulary needs no per-kind argument and no second tool; the kind travels inside the reference.

Only the two stored-file kinds resolve. Any other well-formed reference (a knowledge page, a dataset, a memory record) parses and is then refused by name, because naming what was passed tells the caller what to pass instead.

func PrefixedTableName

func PrefixedTableName(persona, slug string) string

PrefixedTableName is the name a registration takes: the persona, then the slug.

The scratch schema is one shared workspace -- everyone granted the connection sees every table in it, and resource and asset permissions are not carried into Trino. The prefix is not a boundary and does not pretend to be one; it is what keeps two people who both registered "vendors" from colliding on the name, and what tells a reader of the schema whose working table they are looking at.

func QuoteIdentifier

func QuoteIdentifier(name string) string

QuoteIdentifier renders a name as a Trino delimited identifier.

Every identifier the registrar puts in a statement goes through this, including ones it derived itself: a column name comes from the first line of a file somebody uploaded, and the DDL is assembled as text because Trino has no parameter binding for identifiers. Doubling the quote is what closes that.

func QuoteLiteral

func QuoteLiteral(value string) string

QuoteLiteral renders a value as a Trino string literal, for the table properties -- the external location -- that are values rather than names.

func SampleJoinSQL

func SampleJoinSQL(r Registration) string

SampleJoinSQL renders a statement showing how the registered table is used: a SELECT over it, with the CAST that joining it to a typed warehouse column requires. Every column is VARCHAR, so a reader who writes the obvious join gets a type error and no explanation of why; this is the explanation.

func Sentences added in v1.126.5

func Sentences(outcomes []FollowOutcome) []string

Sentences renders every outcome, in order, as the lines a write's result carries.

func SlugifyTableName

func SlugifyTableName(raw string) string

SlugifyTableName turns a filename or a person's suggestion into a table name Trino accepts unquoted: lowercase, alphanumeric and underscore, never leading with a digit.

Types

type AuditLogger

type AuditLogger interface {
	Log(ctx context.Context, event audit.Event) error
}

AuditLogger is the write half of audit.Logger. The registrar records events and never reads them, so it names only what it uses; audit.Logger satisfies it.

type Caller

type Caller struct {
	UserID  string
	Email   string
	Persona string
	Roles   []string
	IsAdmin bool
	// OnBehalfOf is the address of the person an unattended caller acts for,
	// empty for a person acting as themselves. A managed-script run
	// authenticates as a principal that owns no stored file, so the authority
	// checks over the file being registered read this to reach what the run's
	// author reaches (#1419, #1487). The portal path never sets it: a browser
	// request is always somebody acting as themselves.
	OnBehalfOf string
}

Caller is who is asking. Persona drives the connection boundary and the table-name prefix; IsAdmin lifts the ownership check on replacing a registration.

type Column

type Column = tablecsv.Column

Column is one column a registered table declares; see tablecsv.Column.

func ReadHeaderColumns

func ReadHeaderColumns(content []byte) ([]Column, error)

ReadHeaderColumns parses the first line of a CSV and returns the columns a table registered over it declares.

A blank column name is filled in positionally rather than refused: a trailing comma or an unnamed index column is ordinary in an exported CSV, and a table that refuses to exist over it helps nobody. A duplicate name is suffixed for the same reason -- the file is what it is, and the column still has to be addressable.

type ConnectionScope

type ConnectionScope interface {
	AllowConnection(persona, connection string) bool
}

ConnectionScope is the persona connection boundary, the same predicate the authorizer applies to a tool call.

type Deps

type Deps struct {
	Store Store
	Trino Executor
	// Objects reads a source's bytes, keyed by source kind. It is per kind
	// because the two kinds do not have to share an object store: a
	// deployment names the portal's S3 connection and the managed-resources
	// one separately, and reading a resource through the portal's client
	// would look in the wrong bucket on any deployment that split them.
	Objects map[string]ObjectReader
	// Revisers save a corrected copy of a source as a new version of itself,
	// keyed by source kind the way Objects is and for the same reason: the two
	// kinds keep their version trails in different places. A kind with no
	// entry can be registered and refused, but not corrected.
	Revisers map[string]Reviser
	Scope    ConnectionScope
	Audit    AuditLogger
	NewID    func() (string, error)
	MaxBytes int64
}

Deps is what a Registrar needs.

type Executor

type Executor interface {
	Exec(ctx context.Context, connection, sql string) error
	ScratchTarget(connection string) (trino.ScratchConfig, bool)
	// AcceptsWrites reports whether Exec would be allowed to run write SQL on
	// this connection. It is on the port rather than discovered by assertion
	// because the picker MUST ask it: a connection that carries a scratch
	// target but refuses writes was offered and then refused its DDL.
	AcceptsWrites(connection string) bool
	// TableExists reports whether the catalog on a connection still holds a
	// table. It is what a write that ran DROP TABLE asks about every OTHER
	// registration on the connection afterwards (#1546): an object store
	// whose prefix listing does not stop at a directory boundary lets one
	// table's drop take a name-prefix sibling's metadata with it, and the
	// registration row would otherwise go on describing a table that is
	// not there.
	TableExists(ctx context.Context, connection, catalog, schema, table string) (bool, error)
}

Executor runs a statement on a named Trino connection and reports the registration target that connection writes into. The Trino toolkit satisfies it.

type Filter added in v1.126.0

type Filter struct {
	// AllConnections lifts the connection boundary, which is what an
	// administrator gets.
	//
	// It is a flag rather than a nil Connections slice because the two states
	// it separates are opposites and both are reachable: a persona granted no
	// connection reaches nothing, and an administrator reaches everything.
	// Reading one as the other would either hide every table from an operator
	// or show every table to a persona that may query none of them.
	AllConnections bool
	// Connections is what the caller may see when AllConnections is not set.
	Connections []string
	// SourceKind limits the listing to KindResource or KindAsset. Empty spans
	// both, which is the point of the listing.
	SourceKind string
	// Query matches the qualified name -- catalog, schema and table -- without
	// regard to case.
	Query string
	// Limit and Offset page the result. A Limit at or below zero takes
	// DefaultListLimit, and one above MaxListLimit takes that: the caller of a
	// listing does not get to ask for the whole table.
	Limit  int
	Offset int
}

Filter narrows a cross-source listing.

func (Filter) EffectiveLimit added in v1.126.0

func (f Filter) EffectiveLimit() int

EffectiveLimit is the page size a filter resolves to.

type FollowOutcome added in v1.126.5

type FollowOutcome struct {
	RegistrationID string `json:"registration_id"`
	// Table is the qualified name a query writes, and Connection where it
	// lives; together they name the table the way every surface does.
	Table      string `json:"table"`
	Connection string `json:"connection"`
	// Followed means the table now reads the version the write produced.
	Followed bool `json:"followed"`
	// Version is the version the write produced, which a followed table now
	// reads and a table that did not follow is behind.
	Version int `json:"version"`
	// Pinned means the registration was made without follow, so it stays on
	// the version it was registered over by design.
	Pinned bool `json:"pinned,omitempty"`
	// Reason is why a following registration was not moved. Empty when it
	// followed or is pinned.
	Reason string `json:"reason,omitempty"`
	// ColumnsChanged means the new version's header differs from the one the
	// table declared, so the table was rebuilt with the new columns.
	ColumnsChanged bool `json:"columns_changed,omitempty"`
	// Missing means the table this registration names no longer exists on
	// its connection: a write that ran DROP TABLE on the connection found it
	// gone afterwards (#1546). Reason says which write. The registration is
	// kept, with the reason recorded on it, so the listing says so too.
	Missing bool `json:"missing,omitempty"`
	// Repaired is what a correction of the new version changed, when the
	// version arrived with a defect a reader cannot see past and a
	// registration carrying the repair choice had it corrected (#1577).
	//
	// It is carried by one outcome, not by every table over the file: the
	// corrected version is a fact about the file, and the follow that read
	// the new head is the one that saved it.
	Repaired string `json:"repaired,omitempty"`
}

FollowOutcome is what a write did to one registration over the file it changed: the table followed, is pinned and now behind, or follows and could not be moved.

func (FollowOutcome) Sentence added in v1.126.5

func (o FollowOutcome) Sentence() string

Sentence renders the outcome as the write reports it: the table, what happened to it, what to do when something is left to do, and -- on the one outcome that carries it -- what a correction of the new version changed.

type Locator added in v1.126.5

type Locator func(ctx context.Context, kind, id string) (Source, bool)

Locator resolves a source by kind and id without deciding authority over it. It serves the follow a write triggers (#1536), where the caller's authority over the file was settled by the write itself and the follow acts for the registrant. A kind the deployment does not store answers ok=false.

type Lookup

type Lookup struct {
	// contains filtered or unexported fields
}

Lookup adapts a Registrar to the discovery layer's TableLookup: it answers, for a page of search hits or one fetched record, which of them are readable as query-engine tables.

It lives here rather than in pkg/knowledge because staleness needs both halves -- the registration's recorded location and the record's current head key -- and only this side knows what a registration is.

func NewLookup

func NewLookup(reg *Registrar) *Lookup

NewLookup adapts a Registrar for discovery. A nil or unwired Registrar yields a lookup that finds nothing, which is a deployment with no registration mechanism rather than an error every search has to carry.

func (*Lookup) TablesFor

func (l *Lookup) TablesFor(
	ctx context.Context, subjects []knowledge.TableSubject,
) (map[string]*knowledge.HitTable, error)

TablesFor returns the table reference for every subject that has one.

A subject with several registrations -- the same file registered on two connections -- yields the first by the store's ordering, which is the most recent. A hit carries one table reference because it is a pointer to where the data can be queried, not an inventory of every place it was registered.

type ObjectEntry

type ObjectEntry struct {
	Key  string
	Size int64
}

ObjectEntry names one object in a directory listing.

type ObjectReader

type ObjectReader interface {
	GetObject(ctx context.Context, bucket, key string) (body []byte, contentType string, err error)
	ListDirectory(ctx context.Context, bucket, prefix string) (entries []ObjectEntry, truncated bool, err error)
}

ObjectReader reads the source object and lists what sits beside it.

type Record

type Record struct {
	ID          string
	Name        string
	Bucket      string
	Key         string
	ContentType string
}

Record is what a caller already holds about a stored file, in the terms both kinds share. It is the argument SourceFromResource and SourceFromAssetRecord take, so neither depends on the portal or resource types.

type Registrar

type Registrar struct {
	// contains filtered or unexported fields
}

Registrar registers and unregisters tables over stored objects.

func New

func New(deps Deps) *Registrar

New creates a Registrar. A nil Store or Trino executor makes every call report that registration is unavailable rather than panicking, which is the state of a deployment with no database or no Trino toolkit.

func (*Registrar) Available

func (r *Registrar) Available() bool

Available reports whether registration is wired at all, so a surface can hide the action rather than offering one that always refuses.

func (*Registrar) BySource

func (r *Registrar) BySource(ctx context.Context, kind, sourceID string) ([]Registration, error)

BySource returns every registration over one source.

func (*Registrar) FollowSource added in v1.126.5

func (r *Registrar) FollowSource(ctx context.Context, src Source, version int) []FollowOutcome

FollowSource moves every following registration of a source onto the head the write just produced, and reports what happened to each registration over the file, pinned ones included.

It never fails the write. The file changed; that is the caller's write and it succeeded. A follow whose DDL fails leaves the registration where it was, records why, and says so in the outcome, so the registration is behind the file exactly as a pinned one is, with the reason attached.

version is the revision or version number the write produced, which is what the outcome names.

func (*Registrar) ForSources

func (r *Registrar) ForSources(ctx context.Context, kind string, ids []string) (map[string][]Registration, error)

ForSources returns the registrations of many sources at once.

func (*Registrar) List added in v1.126.0

func (r *Registrar) List(ctx context.Context, f Filter) ([]Registration, int, error)

List returns a page of registrations across every source, with the total the filter matched.

The connection boundary is the filter's, not this method's: the caller of a listing is the surface that enumerated what this person reaches, and pushing it into the query is what keeps the count and the page in agreement.

func (*Registrar) Register

func (r *Registrar) Register(ctx context.Context, caller Caller, src Source, req Request) (*Result, error)

Register makes the source's directory readable as a table and records it.

The order is deliberate: everything that can refuse does so before any statement runs, so a refused registration leaves nothing behind in Trino. The record is written last, because a row naming a table that was never created is worse than a table with no row -- the first is a lie a search hit repeats, the second is an object in a scratch schema.

func (*Registrar) Unregister

func (r *Registrar) Unregister(ctx context.Context, caller Caller, id, source string) error

Unregister drops a registered table and forgets it.

Dropping a Hive external table removes the metastore entry and leaves the objects, so unregistering never touches the file the person uploaded. The row goes even when the DROP fails: the alternative is a record of a table nobody can remove through the platform, and the DROP is IF EXISTS so a table already gone is not an error in the first place.

func (*Registrar) UnregisterAllForSource

func (r *Registrar) UnregisterAllForSource(ctx context.Context, kind, sourceID string)

UnregisterAllForSource drops every table registered over a source. It is what a resource or asset delete calls: the file is going, and a table over where it used to be would return nothing and explain nothing.

It is best-effort by design. The delete that triggered it has its own reasons to succeed, and failing it because a scratch table could not be dropped would make an unrelated Trino outage look like a broken delete.

func (*Registrar) Visible added in v1.126.0

func (r *Registrar) Visible(ctx context.Context, caller Caller, id string) (*Registration, error)

Visible reads one registration for a caller who may see it.

A registration on a connection the caller's persona is not granted is answered as ErrNotFound rather than as a denial: the caller cannot query the table, cannot act on it, and telling them it exists discloses a table name somebody else registered in a schema they have no reach into. An administrator is unrestricted, as everywhere else.

type Registration

type Registration struct {
	ID           string    `json:"id"`
	SourceKind   string    `json:"source_kind"`
	SourceID     string    `json:"source_id"`
	Connection   string    `json:"connection"`
	Catalog      string    `json:"catalog"`
	Schema       string    `json:"schema"`
	Table        string    `json:"table"`
	Location     string    `json:"location"`
	Columns      []Column  `json:"columns"`
	RegisteredBy string    `json:"registered_by"`
	RegisteredAt time.Time `json:"registered_at"`
	// Follow means the registration is moved to the source's new head
	// directory when a revision or version is written (#1536). It is what a
	// registration gets unless the caller pins it: a person or an agent that
	// replaces a file expects the table over it to read the new contents.
	// Pinned -- follow off -- is the choice for a report that must keep
	// returning the rows it was registered over until somebody decides
	// otherwise.
	Follow bool `json:"follow"`
	// Repair means the registration corrects its file: a version of the source
	// carrying a defect a reader cannot see past, of a kind the platform can
	// correct, is saved corrected as the file's next version and the table is
	// moved onto that version (#1577). Off -- the default, and what a
	// registration made without asking gets -- such a version leaves the
	// registration where it was with the reason recorded on it, because nobody
	// asked for the file to be rewritten.
	//
	// It is on the record because the follow has nothing else to act on: the
	// choice was made once, at a registration, and the versions it applies to
	// arrive afterwards. It does nothing for a registration that is not
	// following, which never meets a new version at all.
	Repair bool `json:"repair"`
	// FollowError is why the last follow did not move the registration, and is
	// empty while it is where the file is. It is kept on the record because a
	// follow never fails the write that triggered it, so the listing has to be
	// able to say what is behind and why without the log of that write.
	FollowError string `json:"follow_error,omitempty"`
}

Registration records that a source object's directory is readable as a table on a connection.

func (Registration) IsStale

func (r Registration) IsStale(bucket, currentHeadKey string) bool

IsStale reports whether the registration still points at the source's current content.

A resource revision and an asset version both write a new object under a new directory and move the head key to it. The table keeps serving the directory it was registered against, which is the revision that was current then -- correct SQL over the wrong bytes, and nothing about the table says so. This compares the recorded location against the directory of the head key the source carries now; re-registering targets the current head.

An overwrite in place is not staleness: replacing the object at the same key changes what the table returns on the next query, with no re-registration, which is what makes a repeating vendor drop a re-upload rather than a chore.

func (Registration) QualifiedName

func (r Registration) QualifiedName() string

QualifiedName is the table as a query names it.

type RepairReport added in v1.125.3

type RepairReport struct {
	tablecsv.NormalizeReport
	Version int `json:"version"`
	// Followed is what writing the corrected version did to every OTHER
	// registration over the file (#1536): a following one was moved onto it,
	// a pinned one is now behind it. The registration the correction was made
	// for is not among them, being current by construction.
	Followed []FollowOutcome `json:"followed,omitempty"`
}

RepairReport is what saving a corrected version of a file changed, which version it was saved as, and what that version did to the other tables registered over the file.

func RepairOf added in v1.125.3

func RepairOf(err error) *RepairReport

RepairOf returns the correction an error carries, or nil when it carries none. A surface that replaces a platform failure's text with its own uses it to keep the part about the file.

func (*RepairReport) Summary added in v1.125.3

func (r *RepairReport) Summary() string

Summary renders the correction as the sentence a person whose file changed is told, in both surfaces, followed by what it did to the other tables over the file.

type Request

type Request struct {
	Connection string
	// TableName is the caller's choice. Empty takes a slug of the source's
	// filename. Either way it is slugified and persona-prefixed.
	TableName string
	// Source names where the registration comes from: "portal" for a REST or
	// UI action, "mcp" for a tool call. It is recorded on the audit event.
	Source string
	// Repair asks the platform to save a corrected version of the file when it
	// cannot be read as a table the way it is stored, and to register that
	// version. Without it such a file is refused and the refusal says what is
	// wrong with it, because correcting somebody's file is not something to do
	// on the way to something they asked for.
	Repair bool
	// Follow moves the registration to the file's new head whenever a
	// revision or version of it is written (#1536). Off, the registration is
	// pinned to the directory it was registered over, and a later write
	// leaves it behind the file and says so. The surfaces default it to on;
	// the registrar takes the resolved choice.
	Follow bool
}

Request is one registration.

type Result added in v1.125.3

type Result struct {
	Registration
	Source Source `json:"-"`
	// Correction is what saving a corrected version changed, or nil when the
	// file was registered exactly as it was stored.
	//
	// It is not called Repair. The embedded Registration carries a Repair of
	// its own -- the standing choice this registration was made with (#1577) --
	// and two fields under one name at two depths is a shadow in Go and a
	// dropped field in JSON: encoding/json keeps the shallower tag and emits
	// neither the report nor the choice.
	Correction *RepairReport `json:"correction,omitempty"`
	// Siblings is what a replacing registration found about the OTHER
	// registrations on the connection after its DROP ran (#1546): one outcome
	// per table that no longer exists. Empty when every other table is still
	// there, and always empty for a registration that replaced nothing.
	Siblings []FollowOutcome `json:"siblings,omitempty"`
}

Result is one completed registration: the record, the source it was built over, and what a correction of that source changed.

Source is not always the source that was handed in. A file that had to be corrected first is registered over the version the correction wrote, and a surface reporting on the registration -- whether it is stale, which object it reads -- has to be looking at that version rather than the one it asked about.

type Revised added in v1.125.3

type Revised struct {
	Bucket  string
	Key     string
	Version int
}

Revised is where a correction landed: the object the source's head now points at, and the number the version trail gave it.

type Reviser added in v1.125.3

type Reviser interface {
	Revise(ctx context.Context, src Source, caller Caller, content []byte, summary string) (Revised, error)
}

Reviser saves corrected content as a new version of the source it came from, through the version mechanism that kind already has, and reports where the new head sits.

It is a port rather than a copy of either version writer because the two kinds keep their trails in different tables: a managed resource records a revision, a portal asset records a version, and both move the head to a fresh per-version directory in one transaction. That directory rule is what the registrar needs and is the only thing it asks of either.

The original object is never modified. A correction is the version on top of it, so the file a person uploaded stays what they uploaded and the change is revertible from the panel every other version is.

summary says why the file changed, in the terms the person who uploaded it would use. Both kinds record it on the version they write, so a reader of either history sees the reason beside the version without having to find the registration that made it.

type Source

type Source struct {
	Kind        string
	ID          string
	Name        string
	Bucket      string
	HeadKey     string
	ContentType string
}

Source is the object a registration is built over: where it lives, what it is, and who it belongs to. Each surface builds one from its own record, so the registrar never learns what a resource or an asset is.

func SourceFromAssetRecord

func SourceFromAssetRecord(rec Record) Source

SourceFromAssetRecord builds a portal-asset source.

The kind is what separates it from SourceFromResource, and it is not a detail: it selects which object store the file is read from and which rows a delete sweeps.

func SourceFromResource

func SourceFromResource(rec Record) Source

SourceFromResource builds a managed-resource source: its head key is the current revision, and its directory holds only that file.

type SourceRef added in v1.126.0

type SourceRef struct {
	// Name is what the source is called, for a reader who is looking at a
	// table name and does not recognize it.
	Name string
	// Bucket and HeadKey are where the source's content sits NOW, which is the
	// half IsStale needs and the registration does not carry.
	Bucket  string
	HeadKey string
	// CanModify is authority over the source, the half of the unregister rule
	// the registration cannot answer. It is resolved here rather than at the
	// point of the action because a listing has to decide whether to offer the
	// action at all, and offering one that is then refused is the same defect
	// as refusing one that was never offered.
	CanModify bool
}

SourceRef is what a cross-source listing has to say about the record a registration was built over: what it is called, where its content sits now, and whether this caller may act on it.

It exists because the listing spans sources and the per-source reads do not. A registration alone cannot answer either question a reader of the list has -- which file is this, and is the table still reading its current contents -- and resolving that one row at a time is what the per-source panels already do (#1472).

type Sources added in v1.126.0

type Sources func(ctx context.Context, kind string, ids []string, caller Caller) map[string]SourceRef

Sources resolves the sources a page of registrations names, one kind at a time, for one caller.

It is the bulk form of Subject and is separate from it for two reasons. A listing reads a page of sources at once, so a per-id resolver would cost one store read per row. And a listing shows a registration the caller may see by its connection, which is not the same set as the sources they may change: authority is a field on the answer here rather than the answer itself.

An id absent from the returned map names a source that is gone.

type Store

type Store interface {
	Insert(ctx context.Context, r Registration) error
	Get(ctx context.Context, id string) (*Registration, error)
	// ByName returns the registration holding a name on a connection, or nil
	// when the name is free.
	ByName(ctx context.Context, connection, catalog, schema, table string) (*Registration, error)
	// BySource returns every registration of one resource or asset.
	BySource(ctx context.Context, kind, sourceID string) ([]Registration, error)
	// ForSources returns the registrations of many sources of one kind, keyed
	// by source id. It is the read a list view and a search result set use, so
	// a page of hits costs one query rather than one per hit.
	ForSources(ctx context.Context, kind string, sourceIDs []string) (map[string][]Registration, error)
	// List returns a page of registrations across every source, newest first,
	// with the total the filter matched.
	//
	// It is the only read here that is not keyed by one source or one name,
	// which is why nothing could answer "what is registered on this platform"
	// before it: the scratch schema is shared, so a reader could query a table
	// through Trino that no surface would list (#1472).
	List(ctx context.Context, f Filter) ([]Registration, int, error)
	// Relocate moves a registration onto a new directory with the columns the
	// file there declares, and clears the failure of any earlier follow. It
	// is the store half of a follow (#1536): the table was already moved by
	// the DDL, and the row has to say what the table now reads.
	Relocate(ctx context.Context, id, location string, columns []Column) error
	// RecordFollowFailure keeps why a follow did not move a registration, so
	// a listing reports it behind the file with the reason.
	RecordFollowFailure(ctx context.Context, id, reason string) error
	Delete(ctx context.Context, id string) error
}

Store persists registrations.

Insert is expected to fail when the name is already claimed; the registrar turns that into a refusal naming the holder rather than silently replacing a table someone else registered.

func NewPostgresStore

func NewPostgresStore(db *sql.DB) Store

NewPostgresStore creates a registration store backed by PostgreSQL.

type Subject

type Subject func(ctx context.Context, id string, caller Caller) (Source, bool)

Subject resolves the record an id names into what the registrar needs, and decides whether this caller may act on it. Returning ok=false means the caller may not act on the record at all, which every surface answers as a not-found so none of them reveals a record the caller cannot reach.

One resolver per kind serves both surfaces: the REST routes convert their authenticated portal user into a Caller and the tool reads one from the platform context, so the authorization rule for a kind is written once and cannot drift between the two doors.

type ToolAdapter

type ToolAdapter struct {
	// contains filtered or unexported fields
}

ToolAdapter satisfies the asset toolkit's TableRegistrar over a Registrar, keyed by the canonical reference a caller already holds rather than by the id of one kind of record.

It exists because the tool's contract carries no caller: the acting identity belongs on the PlatformContext the middleware chain put in the request's context, and reading it here rather than taking it as an argument is what keeps a tool call from presenting an identity the rest of the platform would refuse.

func NewToolAdapter

func NewToolAdapter(reg *Registrar, adminRoles []string, subjects map[string]Subject, locate Locator) *ToolAdapter

NewToolAdapter adapts a Registrar for the table tool. A nil or unwired Registrar, or one with no kind to resolve, yields nil, which the toolkit renders as "this deployment cannot register tables" rather than as a failure.

func (*ToolAdapter) DropAssetTables

func (a *ToolAdapter) DropAssetTables(ctx context.Context, assetID string)

DropAssetTables removes every table registered over a deleted asset.

func (*ToolAdapter) FollowAssetTables added in v1.126.5

func (a *ToolAdapter) FollowAssetTables(ctx context.Context, assetID string, version int) []string

FollowAssetTables moves the following registrations over an asset onto the version a write just produced, and reports every registration over it.

func (*ToolAdapter) FollowResourceTables added in v1.126.5

func (a *ToolAdapter) FollowResourceTables(ctx context.Context, resourceID string, version int) []string

FollowResourceTables is FollowAssetTables for a managed resource whose content was just replaced.

func (*ToolAdapter) Register

func (a *ToolAdapter) Register(
	ctx context.Context, reference, connection, tableName string, opts portaltoolkit.RegisterOptions,
) (*portaltoolkit.TableRegistration, error)

Register registers a table over the current content of the file a reference names.

func (*ToolAdapter) Tables

func (a *ToolAdapter) Tables(ctx context.Context, reference string) ([]portaltoolkit.TableRegistration, error)

Tables reports what is registered over the file a reference names.

func (*ToolAdapter) Unregister

func (a *ToolAdapter) Unregister(ctx context.Context, registrationID string) error

Unregister drops a registered table.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL