Documentation
¶
Overview ¶
Package resource provides the data layer for human-uploaded reference material (samples, playbooks, templates, references). Resources are scoped to global, persona, or user visibility and stored as blobs in S3 with metadata in PostgreSQL.
Index ¶
- Constants
- Variables
- func BuildRevisionS3Key(scope Scope, scopeID, resourceID, revisionID, filename string) string
- func BuildS3Key(scope Scope, scopeID, resourceID, filename string) string
- func BuildURI(scheme string, scope Scope, scopeID, path, filename string) string
- func CanAccessResource(c Claims, r *Resource) bool
- func CanModifyResource(c Claims, r *Resource) bool
- func CanMoveToLibrary(c Claims, scope Scope, scopeID string) bool
- func CanReadResource(c Claims, r *Resource) bool
- func CanSeeLibrary(c Claims, lib ScopeFilter) bool
- func CanWriteScope(c Claims, scope Scope, scopeID string) bool
- func GenerateID() (string, error)
- func IndexText(r Resource, contentText string) string
- func IsFolderMoveRefused(err error) bool
- func IsInvalidPath(err error) bool
- func IsInvalidScope(err error) bool
- func IsMoveConflict(err error) bool
- func IsNotFound(err error) bool
- func IsObjectNotFound(err error) bool
- func MoveResource(ctx context.Context, deps Deps, claims *Claims, res *Resource, to Destination) (string, error)
- func NormalizeMaxVersions(configured int) int
- func PathSegments(p string) []string
- func PathUnder(p, prefix string) bool
- func PersonAddress(c Claims) string
- func PersonaAdminRoles(roles []string) []string
- func RelocatedURI(scheme string, r *Resource, scope Scope, scopeID, path string) string
- func RepointPath(p, from, to string) string
- func ReviseContent(ctx context.Context, deps Deps, res *Resource, claims *Claims, ...) (*Resource, *Version, error)
- func SanitizeFilename(name string) (string, error)
- func StampsLastRead(surface string) bool
- func URIFilename(scheme string, r *Resource) string
- func URIInLibrary(scheme string, scope Scope, scopeID, tail string) string
- func ValidateDescription(desc string) error
- func ValidateDisplayName(name string) error
- func ValidateMIMEType(mt string) error
- func ValidatePath(p string) error
- func ValidateScope(scope Scope, scopeID string) error
- func ValidateTags(tags []string) error
- type Claims
- type ClaimsExtractor
- type Deps
- type Destination
- type Filter
- type Folder
- type FolderMove
- type FolderMoveEntry
- type FolderRename
- type Handler
- type Move
- type MoveEvent
- type MoveRecorder
- type NewResource
- type ParsedURI
- type ReadEvent
- type ReadRecorder
- type ReadTracker
- type Resource
- type Revision
- type RevisionUpload
- type S3Client
- type Scope
- type ScopeFilter
- type ScoredResource
- type SearchQuery
- type Searcher
- type Sort
- type Store
- type ThumbnailCapture
- type Update
- type Usage
- type UsageReader
- type Version
- type VersionStore
Constants ¶
const ( // MaxPathSegments bounds how deep a folder chain may go. MaxPathSegments = 8 // MaxPathLen bounds the whole path, separators included. MaxPathLen = 200 // MaxPathSegmentLen is the longest a single folder name may be. It is the // rule the flat category carried, kept unchanged so every path that was // legal before the tree existed is still legal. MaxPathSegmentLen = 31 )
Folder-path limits (#1529). A path is the slash-separated folder chain a resource is filed under inside its library.
const ( // DefaultListLimit is used when no limit is specified in a list query. DefaultListLimit = 100 // MaxListLimit caps a client-supplied page size so a single list request // cannot pull an unbounded window. MaxListLimit = 200 )
const ( ThumbnailVariantLight = "light" ThumbnailVariantDark = "dark" )
ThumbnailVariantLight and ThumbnailVariantDark name the two captures a resource can carry. A content type that brings its own colors stores only the light one and serves it in both modes.
const ( // SurfaceMCPRead is an MCP resources/read call. SurfaceMCPRead = "mcp_read" // SurfaceFetch is a search `fetch` of an mcp:resource:<id> reference. SurfaceFetch = "fetch" // SurfaceDownload is a REST content download (the portal's Download // button, and any direct API client). SurfaceDownload = "rest_download" // SurfacePreview is the portal rendering a resource's own bytes as part of // showing the library: an image tile in a folder grid, where the object // itself stands in for the thumbnail a resource does not have (#1471). // // It is a door of its own because it is not somebody using the file. A // library of photographs would otherwise mark every image read on every // page view, which is exactly the signal the never-read flag and the // last-read ordering exist to give a curator. A preview is audited like any // other read — the bytes did reach an identified caller — and is the one // surface that does not stamp the durable last-read column. // // The caller declares it, so it says why a read happened rather than // controlling whether it is recorded: a client that asks for a preview // still produces an audit row under its own identity. SurfacePreview = "portal_preview" )
Read surfaces a resource's content can be served through. Recorded on every read event so a curator can tell material an agent actually pulls from material a human downloads once and forgets.
const ( MaxUploadBytes = 100 << 20 // 100 MB MaxDescriptionLen = 2000 MaxDisplayNameLen = 200 MaxTagsPerResource = 20 MaxTagLen = 50 )
Validation limits.
const DefaultMaxVersions = 10
DefaultMaxVersions is the number of content revisions a resource keeps, counting the current head. A revision past the cap prunes the oldest version rows and their blobs. Bounded by default because a resource blob is up to MaxUploadBytes (100 MB) and an unbounded trail turns every edit into permanent storage growth.
const (
// DefaultSearchLimit is the top-K returned when the caller specifies none.
DefaultSearchLimit = 20
)
Search result limits, mirroring the asset/prompt/memory ranked surfaces so every ranked surface clamps the same way.
const DefaultURIScheme = "mcp"
DefaultURIScheme is used when no scheme is configured.
const MaxContentIndexBytes = 32 << 10
MaxContentIndexBytes bounds how much of a resource's content is extracted into content_text for search. The prefix has to be large enough to cover the part of a document that describes it (a data dictionary's column list, a runbook's procedure) while keeping the embedded text within one provider call and the row small enough that listing resources stays cheap.
const MaxContentReadBytes = 8 << 20
MaxContentReadBytes bounds the object size the index consumer will pull from blob storage to extract that prefix. The blob API has no range read, so extracting 32 KiB means holding the whole object in memory; an upload may be up to MaxUploadBytes (100 MB) and several index workers run concurrently. A resource larger than this is indexed on its metadata alone rather than risking the sweep's memory on a file whose first 32 KiB is all that would have been kept anyway.
const MaxFolderMoveResources = 500
MaxFolderMoveResources bounds how many resources one folder move may rewrite.
The move is one transaction and one audit event per resource, both of which are paid inside the request. The cap is a refusal rather than a truncation: a rename that moved the first five hundred files and left the rest behind would be the half-renamed folder the transaction exists to prevent, reported as a success.
const MaxInlineContentBytes = 1 << 20
MaxInlineContentBytes is the size at or below which a text resource's content is returned inline rather than as a pointer to the blob. It is shared by the MCP resources/read path and the search `fetch` reference so an agent gets the same answer for the same file whichever door it comes through.
const MaxMultipartMemory = 10 << 20
MaxMultipartMemory is the max memory for multipart form parsing (10 MB).
const MaxThumbnailSourceBytes = 1 << 20 // 1 MB
MaxThumbnailSourceBytes is the largest resource a capture is attempted from.
Capture renders the document a second time and rasterizes it on the main thread, so its cost tracks the file. It is the same cap the asset queue applies, and the outliers it excludes are exactly the ones that stall the tab doing the work.
const ( // MaxThumbnailUploadBytes bounds a capture upload. A tile is 400x300 PNG; // anything approaching this is not one. MaxThumbnailUploadBytes = 2 << 20 // 2 MB )
const MinMaxVersions = 2
MinMaxVersions is the floor a configured retention cap is raised to. One version is the head itself, so a cap below 2 keeps no history at all and would make the version panel a list of one row that changes under the reader.
Variables ¶
var DeniedExtensions = map[string]bool{ ".exe": true, ".sh": true, ".bat": true, ".cmd": true, ".ps1": true, ".msi": true, ".com": true, ".scr": true, }
DeniedExtensions lists file extensions that are blocked for upload.
var DeniedMIMETypes = map[string]bool{ "application/x-executable": true, "application/x-msdos-program": true, "application/x-msdownload": true, "application/x-sh": true, "application/x-shellscript": true, "application/x-bat": true, "application/x-msi": true, contenttype.XHTML: true, }
DeniedMIMETypes lists MIME types that are blocked for upload.
A resource is human-uploaded reference material — report templates, brand files, sample documents, CAD exports — so this stays a denylist. An allowlist here would refuse the long tail the library exists to hold, and it would buy almost nothing: blobserve serves every stored byte under a sandbox CSP and hands the scriptable document families to the browser as attachments, and DeniedExtensions already refuses the executable extensions.
var ErrFolderEmpty = errors.New("no resources are filed under that folder")
ErrFolderEmpty is returned when no resource in the library lies under the folder being moved. Folders are derived from the paths in use (#1529), so a folder with nothing under it does not exist and cannot be renamed.
var ErrForbidden = errors.New("forbidden")
ErrForbidden signals that authentication succeeded at the credential level but the request is refused for a policy reason the client can recover from without re-authenticating — specifically a CSRF-token failure on a cookie-authenticated mutation. A ClaimsExtractor returns it so the handler responds 403 (not 401), matching the admin/portal surfaces and preventing the SPA from force-logging-out the user on a recoverable CSRF error.
var ErrMoveForbidden = errors.New("insufficient permissions for target library")
ErrMoveForbidden is returned when the caller may modify the resource but may not file it in the library they named.
var ErrURIConflict = errors.New("a resource already occupies that URI")
ErrURIConflict is returned by Move when the target library already holds a resource at the URI the moved resource would take. The caller names the collision from its own read; this is the store's report of the constraint the database enforced, which is what closes the gap between that read and the write.
Functions ¶
func BuildRevisionS3Key ¶ added in v1.115.0
BuildRevisionS3Key constructs the S3 object key for a revision's blob. Each revision gets its own key under a v/<revisionID>/ segment so prior versions remain independently readable and pruning one never touches another's bytes.
The key is keyed by an opaque revision id rather than the version number because the number is assigned by the database inside the insert transaction, after the blob has already been written; deriving the key from the number would require knowing it first, which is exactly the race the in-transaction assignment exists to avoid. Version 1 of a resource uploaded before versioning keeps the flat key BuildS3Key produced at create time.
func BuildS3Key ¶
BuildS3Key constructs the S3 object key for a resource blob.
func BuildURI ¶
BuildURI constructs the canonical resource URI from its components: the library prefix, the folder path inside it, and the filename.
func CanAccessResource ¶ added in v1.115.0
CanAccessResource checks whether the caller may see a specific resource at all: it is inside their visible scopes, OR they hold write authority over the scope it lives in (a platform admin, or that persona's admin).
The second clause is what separates this from CanReadResource. VisibleScopes is membership-based and grants an admin no cross-persona read, so a platform admin who uploads a persona-scoped resource — which CanWriteScope explicitly permits — was then refused GET, PATCH and DELETE on it: they could create material they could neither manage nor remove. Use this as the visibility gate on a resource the caller names by id; CanReadResource remains the membership rule for enumeration and for content served into an agent's session.
It deliberately checks CanWriteScope rather than CanModifyResource: the latter also grants the original uploader, and that grant is not re-derived from current authority. An admin who uploaded into another user's scope and then lost their admin role would otherwise keep reading, editing, and deleting that user's private file forever, because the uploader_sub on the row never changes. Every legitimate uploader whose authority came from their own scope (a user uploading to their own user scope) is already covered by CanReadResource.
The one narrow exception is an unattended caller acting for the person who uploaded the file INTO THEIR OWN LIBRARY (uploadedBy). That is the same grant CanReadResource already gives the person, reached by the only identifier such a caller has, and it carries none of the decay the warning above is about -- see uploadedBy for why the scope is part of the test.
func CanModifyResource ¶
CanModifyResource checks whether the caller can update or delete a resource. The caller must be the original uploader OR have write permission for the scope.
A managed-script run is the uploader when its version author is: the run authenticates as a principal with no uploaded file of its own, so matching on uploader_sub alone refuses it the very files its author uploaded. The match is on the recorded uploader address, which is the same rule the asset toolkit's ownership check applies to a run (#1419).
func CanMoveToLibrary ¶ added in v1.126.3
CanMoveToLibrary reports whether the caller may file a resource into the named library. The caller must separately be allowed to modify the resource itself (CanModifyResource); this answers only the destination half.
It is deliberately looser than CanWriteScope on one arm and identical on every other. Uploading into a persona library takes that persona's admin role, and that stays: putting new material in front of a persona's members is the persona administrator's call. Moving a file you already own into a persona you BELONG to is a different act -- the file exists, you own it, and you are one of the people who will read it -- so membership is enough.
Widening CanWriteScope itself would have been the smaller diff and the wrong one: it is what the upload route checks and what the Resources page derives its Upload control and its scope tabs from, so every member of every persona would have gained an upload door nobody asked to open.
Global stays admin-only through CanWriteScope, and so does another person's library.
func CanReadResource ¶
CanReadResource checks whether the caller can read a specific resource.
func CanSeeLibrary ¶ added in v1.127.0
func CanSeeLibrary(c Claims, lib ScopeFilter) bool
CanSeeLibrary reports whether the caller may read the named library at all.
It is the listing rule plus write authority, which is the same pair CanAccessResource applies to one resource: membership is what makes a library visible, and an administrator who may write a library they are not a member of must still be able to read and reorganize it.
A platform administrator who uploads into a persona they do not belong to is the case this exists for. CanWriteScope permits the upload and CanAccessResource permits every later read of the file by id, so a library rule built on membership alone left that administrator holding material they could create and could not find.
func CanWriteScope ¶
CanWriteScope checks whether the caller has write permission for the given scope.
func GenerateID ¶
GenerateID returns a cryptographically random 32-character hex string.
func IndexText ¶ added in v1.115.0
IndexText composes the text a resource is embedded and lexically indexed on: its display name, description, folder path, filename, tags, and contentText, the bounded text prefix the index consumer extracted from the uploaded file. The indexjobs resource consumer and the request-path search MUST agree on this composition so a stored embedding lives in the same space as the query; it is defined once here for both. Empty fields are skipped so a sparse resource does not pad the text with blank lines. The lexical arm's resource_fts (migration 000091) composes the same corpus from the same columns.
func IsFolderMoveRefused ¶ added in v1.126.5
IsFolderMoveRefused reports whether a folder move was refused for a reason the caller can state and act on, as opposed to having failed.
func IsInvalidPath ¶ added in v1.126.5
IsInvalidPath reports whether an error names an unusable folder path, so a caller several layers above ValidatePath answers 400 rather than treating it as a failure of its own.
func IsInvalidScope ¶ added in v1.126.3
IsInvalidScope reports whether an error names an unusable scope/scope_id pair, so a caller several layers above ValidateScope can answer 400 rather than treating it as a failure of its own.
func IsMoveConflict ¶ added in v1.126.3
IsMoveConflict reports whether an error from MoveResource means another resource already answers at the destination URI.
func IsNotFound ¶ added in v1.111.0
IsNotFound reports whether an error from a Store read means the resource does not exist, as opposed to the read having failed.
The Postgres store surfaces a missing row as a wrapped sql.ErrNoRows rather than as (nil, nil), so a caller that must distinguish "deleted" from "the database is down" cannot do it by nil-checking the result. Getting that distinction wrong is not cosmetic: a prompt attachment whose resource was deleted has to degrade to a flagged broken link, while a failed read has to fail closed.
func IsObjectNotFound ¶ added in v1.115.0
IsObjectNotFound reports whether a blob-store GetObject error indicates the object does not exist (an orphaned resource: the metadata row survived its content), as opposed to a transient or permission failure that a retry might resolve.
The mcp-s3 client wraps the underlying AWS/SeaweedFS error without a typed not-found, so detection is by the standard S3 not-found signatures present in the wrapped message. It lives here because both readers of resource blobs depend on the distinction and must draw it identically: the resources/read middleware self-heals a confirmed orphan by pruning the row, and the search index consumer clears a confirmed-orphan's indexed content instead of leaving stale text behind. A per-caller copy of this heuristic would let the two diverge on exactly the case that matters.
func MoveResource ¶ added in v1.126.3
func MoveResource(ctx context.Context, deps Deps, claims *Claims, res *Resource, to Destination) (string, error)
MoveResource refiles an existing resource: in another library, in another folder of the one it is in, or both at once.
The caller has already established that claims may modify res (CanModifyResource); this checks the destination half and performs the write. It returns the resource's new URI, or ("", nil) when the resource already lives at that address -- refiling a file where it already is is not an error, and refusing it would make an idempotent PATCH fail.
The comparison that decides "already there" includes the URI the destination composes, not just the library and the path columns. A row whose stored URI disagreed with its folder -- which every category edit before #1528 produced -- is at the right destination by both columns and still answers at the wrong address, and saving the folder it already shows is how a person repairs it.
The blob is not copied and the reference rows are not touched. An asset or a prompt that declared this resource keeps rendering it: both key on the resource id, and the serve-time rewrite matches the URI string recorded on the reference row, which is what the author wrote and stays what they wrote.
func NormalizeMaxVersions ¶ added in v1.115.0
NormalizeMaxVersions clamps a configured retention cap: a non-positive value selects the default, and anything below MinMaxVersions is raised to it.
func PathSegments ¶ added in v1.126.5
PathSegments splits a path into its folder names. An empty path has none, which is the library root.
func PathUnder ¶ added in v1.126.5
PathUnder reports whether p is prefix or lies beneath it. An empty prefix is the library root, which everything is under.
The separator is part of the comparison: without it "data-archive" would read as being under "data", and a folder rename would drag in a sibling whose name merely starts with the same letters.
func PersonAddress ¶ added in v1.126.0
PersonAddress is the address of the person these claims speak for.
For anyone acting as themselves that is their own. For an unattended caller it is the person it acts for, and its own Email is deliberately not consulted: a managed-script run carries its OWNER's address for accountability while presenting its version AUTHOR's roles, and after a transfer those are different people. Treating the owner's address as a scope the run may reach would combine one person's authority with another's ownership, which is a pairing neither of them has (#1419).
It is exported because it is also what a write records as its author: a row a script produced has to name the person whose authority ran, or a scheduled refresh reads in the version history as having been made by whoever happens to own the script.
func PersonaAdminRoles ¶ added in v1.111.0
PersonaAdminRoles extracts the persona names a role set grants admin authority over, tolerating any role prefix.
func RelocatedURI ¶ added in v1.126.5
RelocatedURI is the URI a resource takes when it is refiled: under the target library's prefix, at the target folder path, keeping its own filename.
The filename is read off the stored URI rather than off the row's filename column, so a resource whose stored address was minted under an older scheme keeps answering at the address its citations use for everything except the half being changed. A stored URI that will not parse, or whose tail carries no filename, falls back to the row's own filename, which is what the URI would have been had it been minted now: that is the only answer available for an address the mover never chose, and it is better than refusing the move over it.
func RepointPath ¶ added in v1.126.5
RepointPath rewrites p's from-prefix to to, which is what renaming or moving a folder does to every resource beneath it. A path that is not under from is returned unchanged, so a caller that hands it the whole library gets back only the affected rows changed.
func ReviseContent ¶ added in v1.125.3
func ReviseContent( ctx context.Context, deps Deps, res *Resource, claims *Claims, up RevisionUpload, ) (*Resource, *Version, error)
ReviseContent writes the bytes to a fresh per-revision key, records the revision (which moves the head), prunes beyond the retention cap, and returns the updated resource. A failure after the blob is written removes it, so a failed revision leaves neither a dangling object nor a moved head.
It is exported because the revision trail is where a corrected copy of a resource belongs, and the surface that corrects one is not this handler: a registration that has to rewrite a CSV before it can be read as a table writes the corrected bytes through here (#1441), so a revision made on somebody's behalf is the same kind of revision as one they uploaded, in the same trail, with the same retention.
func SanitizeFilename ¶
SanitizeFilename normalizes a filename for storage: lowercase, no spaces, no path separators or shell metacharacters, preserves extension.
func StampsLastRead ¶ added in v1.126.0
StampsLastRead reports whether a read through this surface should update the resource's durable last-read column. Every surface but the portal's own preview does; see SurfacePreview for why that one does not.
func URIFilename ¶ added in v1.126.5
URIFilename is the last segment of a resource's stored URI, falling back to its filename column when the stored URI does not parse into one.
func URIInLibrary ¶ added in v1.126.3
URIInLibrary constructs a resource URI naming a library and a tail within it, where tail is the "path/filename" BuildURI composes.
It is separate from BuildURI because the two halves of the tail move independently: refiling a resource in another library keeps its folder path, and refiling it in another folder keeps its library. Each rewrite composes the half it changes with the half it does not and passes the result through here (see RelocatedURI).
func ValidateDescription ¶
ValidateDescription checks description length and content.
func ValidateDisplayName ¶
ValidateDisplayName checks display name length and content.
func ValidateMIMEType ¶
ValidateMIMEType checks that the MIME type is not on the deny list. The type is normalized first (parameters stripped, aliases collapsed), so a denied family cannot be smuggled in under a spelling the map does not list.
func ValidatePath ¶ added in v1.126.5
ValidatePath checks a resource's folder path.
Each segment keeps the rule the flat category was validated against, which is what makes every pre-tree row a legal one-segment path and every pre-tree URI unchanged. The rest bounds the tree: a depth, a total length, and a refusal of the three shapes that would make a path mean something other than a folder chain -- an empty segment, a leading or trailing slash, and a relative segment.
Each refusal names the rule it broke rather than restating the whole grammar, because the person reading it typed one path and needs to know which part of it is the problem.
func ValidateScope ¶
ValidateScope checks scope and scope_id consistency.
func ValidateTags ¶
ValidateTags checks tag count, length, and format.
Types ¶
type Claims ¶
type Claims struct {
Sub string // Keycloak subject (user ID)
Email string // user email
Personas []string // persona names the user belongs to
Roles []string // raw roles from auth (may have prefix, e.g., "dp_admin")
IsAdmin bool // resolved by the caller from persona config
AdminOfPersonas []string // persona names this user can admin (resolved by caller from role patterns)
// OnBehalfOf is the address of the person an unattended caller acts for,
// carried from PlatformContext.OnBehalfOfEmail. A managed-script run
// authenticates as script:<name>, a principal that owns nothing a person
// owns, so claims built on the principal alone refuse a run the very files
// its own author can edit -- and file a resource it creates in a library
// nobody can see. Every rule below that turns on "is this you?" reads this
// too, so a run reaches what its author reaches and nothing else (#1419,
// #1487).
//
// Empty for every human caller, which is what keeps this inert everywhere
// else. An empty value must never match an empty owner or an empty scope
// id: absence of an identity is not a shared identity.
OnBehalfOf string
}
Claims represents the identity information needed for resource permission checks.
func BuildClaims ¶ added in v1.111.0
BuildClaims assembles permission claims from an authenticated caller's identity. It owns the roles-to-persona-admin mapping so every surface that must apply the resource read rule — the resources middleware, prompt attachment serving (#1013), the attachment REST handler — derives claims identically instead of each reimplementing it.
persona is the caller's single resolved persona; pass "" when none is resolved. It is separate from roles because persona membership is resolved from roles by the platform's persona registry, which this package does not see.
func (Claims) ActingFor ¶ added in v1.126.0
ActingFor returns the claims with the address of the person an unattended caller acts for. It is a step after BuildClaims rather than another parameter on it because only the surfaces an unattended caller reaches have an address to supply, and a surface that has none says nothing.
An empty address is a no-op, so a surface can pass whatever its context carries without asking whether the caller is one.
type ClaimsExtractor ¶
ClaimsExtractor extracts resource Claims from an HTTP request. Provided by the platform auth middleware.
type Deps ¶
type Deps struct {
Store Store
S3Client S3Client
S3Bucket string
URIScheme string // defaults to "mcp" if empty
OnCreate func(*Resource) // called after successful create to register with MCP
OnDelete func(string) // called after successful delete with URI to unregister
// OnDeleteID is called after a successful delete with the resource's ID,
// so a consumer keyed on the record rather than on its MCP URI can clean
// up after it -- the tables registered over its file (#1327). Separate
// from OnDelete because that one exists to unregister an MCP resource and
// is keyed on the URI it was registered under.
OnDeleteID func(context.Context, string)
// OnRevised is called after a revision moves the resource's head -- a
// replacement or a restore -- with the version number it was recorded as,
// so the tables registered over the file follow it (#1536). It returns
// what happened to each table, which the response carries; the revision
// is never failed by it. Nil on a deployment that cannot register tables.
OnRevised func(ctx context.Context, id string, version int) []string
// Versions records content revisions. Absent on a deployment whose store
// does not implement VersionStore, which disables the revision and version
// routes (503) and leaves create, metadata edits, and reads unaffected.
Versions VersionStore
// MaxVersions is the retention cap; non-positive selects DefaultMaxVersions.
MaxVersions int
// ReadRecorder audits served content reads. Absent when audit is disabled,
// which silences read events without affecting the reads themselves.
ReadRecorder ReadRecorder
// Usage supplies audit-derived read counts for the detail read. Absent when
// audit is disabled, which leaves the usage field off the response.
Usage UsageReader
// MoveRecorder audits a resource refiled into another library. Absent when
// audit is disabled, which silences the event without affecting the move.
MoveRecorder MoveRecorder
// Producers records what wrote each resource (#1569): the script, session
// or person behind the create and behind every content revision since.
// Absent on a deployment with no database, which records nothing and
// leaves the writes themselves unaffected.
Producers producedby.Store
}
Deps holds the dependencies for the resource HTTP handler.
type Destination ¶ added in v1.126.5
Destination is where a resource is being refiled: the library, and the folder path inside it. Both halves travel together because they are the two halves of one URI (#1528).
type Filter ¶
type Filter struct {
Scopes []ScopeFilter // visibility scopes (derived from claims)
// Path narrows the listing to one folder and everything beneath it. Empty
// is the whole library. It is a prefix rather than an equality so opening a
// folder reports what the folder holds, subfolders included, which is what
// makes a count under a folder mean anything.
Path string
Tag string // optional tag filter
Query string // optional text search in display_name/description
Sort Sort // ordering; empty selects SortUpdated
Limit int
Offset int
// AllScopes lists every library in the deployment, whatever Scopes holds.
//
// It is the one predicate a set of (scope, scope_id) pairs cannot express:
// the user libraries are keyed by subject and address, and the platform
// keeps no roster of them to enumerate, so "every library" has to be said
// rather than listed. Only a platform administrator's unnarrowed listing
// sets it (ListScopes), and nothing else in the package reads Scopes when
// it is set.
AllScopes bool
}
Filter specifies criteria for listing resources.
type Folder ¶ added in v1.127.0
type Folder struct {
// Path is the folder's full slash-separated path inside its library.
Path string `json:"path" example:"data/media-manager"`
// Count is the resources filed at this path and beneath it.
Count int `json:"count" example:"12"`
}
Folder is one folder of a library and how much it holds.
A folder is not a stored row: it exists because a resource is filed under it and stops existing when the last one leaves. Count is everything beneath it at every depth, which is what makes the number on a folder mean something to somebody deciding whether to open it.
type FolderMove ¶ added in v1.126.5
type FolderMove struct {
From string `json:"from"`
To string `json:"to"`
// Moved is one entry per resource rewritten, in listing order.
Moved []FolderMoveEntry `json:"moved"`
}
FolderMove is what one folder move did: the resources it rewrote, and the address each of them now answers at.
func MoveFolder ¶ added in v1.126.5
func MoveFolder(ctx context.Context, deps Deps, claims *Claims, move FolderRename) (*FolderMove, error)
MoveFolder renames a folder, or nests it under another one, by rewriting the path prefix of every resource beneath it.
The whole subtree moves in one transaction and every refusal leaves it untouched, because a half-renamed folder is not a state anyone should be able to observe (#1529). Each resource records the address it vacated in the alias table, so a citation written against the old address keeps resolving -- the same machinery a single move uses, applied once per resource.
The caller must be able to modify every resource under the folder. A folder is not a thing with permissions of its own -- it exists because resources are filed under it -- so the authority to move it is the authority over what is in it, and a subtree holding one file the caller may not touch is refused whole rather than moved in part.
type FolderMoveEntry ¶ added in v1.126.5
type FolderMoveEntry struct {
ID string `json:"id"`
Path string `json:"path"`
URI string `json:"uri"`
FromURI string `json:"from_uri"`
}
FolderMoveEntry is one resource carried by a folder move.
type FolderRename ¶ added in v1.126.5
type FolderRename struct {
Library ScopeFilter
From string
To string
}
FolderRename names a folder in one library and where it is going.
The library is part of it rather than derived from the folder, because a path is only unique inside one: "data/weekly" names a different folder in every library that has one.
type Handler ¶
type Handler struct {
// contains filtered or unexported fields
}
Handler provides HTTP endpoints for resource CRUD.
func NewHandler ¶
func NewHandler(deps Deps, extractFn ClaimsExtractor, authMiddle func(http.Handler) http.Handler) *Handler
NewHandler creates a resource handler with auth middleware.
type Move ¶ added in v1.126.3
type Move struct {
// ID is the resource being refiled. It is on the value rather than a
// separate argument because a folder rename hands the store a batch.
ID string
Scope Scope
ScopeID string
Path string
URI string
FromURI string
}
Move is a resource's new home: the resource, the library it is filed in, the folder path it takes inside it, the URI those two compose, and the URI it is leaving.
FromURI is carried rather than re-read inside the store so the alias the move records is the address the caller checked its permissions and its collision against, not whatever the row happened to say a moment later.
type MoveEvent ¶ added in v1.126.3
type MoveEvent struct {
ResourceID string
DisplayName string
FromScope Scope
FromScopeID string
FromPath string
FromURI string
ToScope Scope
ToScopeID string
ToPath string
ToURI string
UserID string
UserEmail string
}
MoveEvent describes one resource refiled, as the audit trail records it: who moved what, out of which library and folder and into which.
Both URIs are carried because the move rewrites the address. An event naming only the resource id would answer "who published this to everyone" but not "what is the address a knowledge page written last year still cites", and the second question is the one an alias exists to answer.
A relocation that changes only the folder is the same event with the two library halves equal, and it is one event even when the library changes in the same request: the resource took one new address, so the trail records one move to it (#1528).
type MoveRecorder ¶ added in v1.126.3
MoveRecorder records a completed move.
Implementations are best-effort and must not fail the move: the resource has already been refiled by the time this is called, and refusing to report a completed write because its audit row would not persist would be a lie about what happened. Recording is off entirely when audit is disabled, which is why every call site tolerates a nil recorder.
type NewResource ¶ added in v1.126.0
type NewResource struct {
Scope Scope
ScopeID string
Path string
Filename string
DisplayName string
Description string
Tags []string
// Data is the content, and MIMEType the type it is stored under.
Data []byte
MIMEType string
// DeclaredMIMEType is what the caller said the bytes were, kept so a
// detection that replaced it is recorded. Empty when nothing was declared.
DeclaredMIMEType string
}
NewResource is a resource about to exist: where it is filed, what it is called, and the bytes it starts life with.
Every field is already validated by the time it reaches CreateResource. The caller owns validation because the two surfaces read their input from different places -- a multipart form, a tool call -- and reporting a bad folder path as a form error or as a tool result is theirs to decide.
type ReadEvent ¶ added in v1.115.0
type ReadEvent struct {
ResourceID string
URI string
// Surface is one of the Surface* constants: which door served the content.
Surface string
// Version is the revision served, or 0 when the head was served without
// naming a version.
Version int
UserID string
UserEmail string
Persona string
SessionID string
RequestID string
}
ReadEvent describes one served read of a resource's content.
type ReadRecorder ¶ added in v1.115.0
ReadRecorder records a served read. Every surface that hands a resource's bytes to a caller calls it, so "who read this, through which door" has one answer rather than one per surface.
Implementations are best-effort and must not fail the read: a recorder that cannot write must swallow the failure, because refusing to serve a file because its audit row would not persist trades a working read for a lost log line. Recording is off entirely when audit is disabled, which is why every call site tolerates a nil recorder.
type ReadTracker ¶ added in v1.115.0
ReadTracker stamps the durable last-read time on a resource row. It is separate from the audit rollup because it answers a different question: the rollup is bounded by audit retention and cannot be sorted on in SQL, while this column survives retention and is what the admin table orders by.
type Resource ¶
type Resource struct {
ID string `json:"id" example:"res_01HK7R9F"`
Scope Scope `json:"scope" example:"persona"`
ScopeID string `json:"scope_id,omitempty" example:"data-engineer"` // persona name or user sub; empty for global
// Path is the slash-separated folder path this resource is filed under
// inside its library, and the tail of its URI ahead of the filename. A
// one-segment path is what every resource carried before folders (#1529).
Path string `json:"path" example:"runbooks/etl"`
Filename string `json:"filename" example:"etl-runbook.md"`
DisplayName string `json:"display_name" example:"ETL Runbook"`
Description string `json:"description" example:"Step-by-step procedures for ETL pipeline operations"`
MIMEType string `json:"mime_type" example:"text/markdown"`
SizeBytes int64 `json:"size_bytes" example:"34000"`
S3Key string `json:"s3_key" example:"resources/res_01HK7R9F/etl-runbook.md"`
URI string `json:"uri" example:"mcp://persona/data-engineer/runbooks/etl-runbook.md"`
Tags []string `json:"tags"`
UploaderSub string `json:"uploader_sub" example:"550e8400-e29b-41d4-a716-446655440000"`
UploaderEmail string `json:"uploader_email" example:"marcus.johnson@example.com"`
CreatedAt time.Time `json:"created_at"`
UpdatedAt time.Time `json:"updated_at"`
// LastReadAt is when the resource's content was last served through any
// surface, stamped by the read recorder. NULL means never read since the
// deployment began auditing reads (#1014). It is the durable answer, unlike
// the audit-derived Usage.LastReadAt, which is bounded by audit retention.
LastReadAt *time.Time `json:"last_read_at,omitempty"`
// Usage is the audit-derived read activity of this resource. It is not a
// stored column: the detail read fills it from the audit rollup, and it is
// absent everywhere the rollup was not consulted.
Usage *Usage `json:"usage,omitempty"`
// ThumbnailS3Key and ThumbnailDarkS3Key are the captured PNGs stored beside
// the resource's own object, empty until one is taken (#1554). The library
// used to draw the original file scaled down instead, which meant a
// non-image had no tile at all and an image cost its full size to show.
ThumbnailS3Key string `json:"thumbnail_s3_key,omitempty"`
ThumbnailDarkS3Key string `json:"thumbnail_dark_s3_key,omitempty"`
// ThumbnailCapturedAt and ThumbnailDarkCapturedAt are when each capture was
// taken. A capture older than the resource's UpdatedAt is behind the file it
// came from, which is what the pending list is built on; see migration
// 000134 for why this is a timestamp rather than a version.
ThumbnailCapturedAt *time.Time `json:"thumbnail_captured_at,omitempty"`
ThumbnailDarkCapturedAt *time.Time `json:"thumbnail_dark_captured_at,omitempty"`
}
Resource represents a human-uploaded reference material entry.
func CreateResource ¶ added in v1.126.0
func CreateResource(ctx context.Context, deps Deps, claims *Claims, in NewResource) (*Resource, error)
CreateResource stores the blob, inserts the metadata row, and records the content as version 1 so the trail starts at the upload rather than at the first revision. A metadata failure removes the blob, so a failed create leaves nothing behind.
It is exported for the same reason ReviseContent is: creating a managed resource is not the browser's alone. An agent, and therefore a scheduled script, creates one through here (#1487), so a resource written on the platform's own initiative is the same record as one a person uploaded -- same URI, same version trail, same retention.
type Revision ¶ added in v1.115.0
type Revision struct {
ResourceID string
MIMEType string
SizeBytes int64
S3Key string
UploaderSub string
UploaderEmail string
// RestoredFrom names the version a restore re-promoted, or nil for a fresh
// upload.
RestoredFrom *int
// ChangeSummary says why the content changed. Empty for an upload; set by a
// caller revising on somebody's behalf, which is the case the version panel
// cannot otherwise explain.
ChangeSummary string
}
Revision is a new content revision to record: the blob that was just written and who wrote it. The version number is not an input — the store assigns it — and the resource's id, uri, filename, scope and metadata are untouched, which is the whole point of revising in place instead of delete-plus-re-upload.
type RevisionUpload ¶ added in v1.125.3
type RevisionUpload struct {
Data []byte
MIMEType string
// RestoredFrom names the version a restore re-promoted, nil otherwise.
RestoredFrom *int
// ChangeSummary says why the content changed, for a revision written on the
// uploader's behalf. Empty for an upload the uploader picked themselves.
ChangeSummary string
}
RevisionUpload is the content a revision writes: the bytes, the type they are stored under, and the version a restore re-promoted (nil for fresh content).
type S3Client ¶
type S3Client interface {
PutObject(ctx context.Context, bucket, key string, data []byte, contentType string) error
GetObject(ctx context.Context, bucket, key string) (body []byte, contentType string, err error)
DeleteObject(ctx context.Context, bucket, key string) error
}
S3Client abstracts blob storage operations for resources.
type ScopeFilter ¶
ScopeFilter identifies a single scope+id pair for visibility filtering.
func ListScopes ¶ added in v1.127.0
func ListScopes(c Claims, scopeParam, scopeIDParam string) (scopes []ScopeFilter, all bool)
ListScopes is the visibility predicate one listing request runs under: the scopes it may read, and whether it is unrestricted.
An unnarrowed listing by a platform administrator is every library in the deployment. That is what the control asking for it has always been labeled, and it is the only reading under which an administrator's own uploads into a persona library appear in a listing at all. Every other caller's unnarrowed listing is their visible scopes, unchanged.
A narrowed listing accepts one library the caller may see or write, which is the rule a folder move already runs under (CanSeeLibrary). Naming a library the caller may not reach narrows to nothing rather than widening: the empty set is a listing with no rows, which is what the store returns for it.
A scope named without an id is a kind rather than a library -- "persona", with no persona -- so it can only narrow the caller's own memberships. The portal never sends one; a hand-written request that does gets the membership answer rather than a widened one.
func VisibleScopes ¶
func VisibleScopes(c Claims) []ScopeFilter
VisibleScopes returns the set of (scope, scope_id) tuples the caller is allowed to see. Always derived from claims, never from request input.
type ScoredResource ¶ added in v1.115.0
ScoredResource pairs a resource with its relevance score in [0,1].
type SearchQuery ¶ added in v1.115.0
type SearchQuery struct {
Embedding []float32 // query vector; nil selects lexical-only ranking
QueryText string // raw query text for the lexical arm
Scopes []ScopeFilter // caller's visible scopes; mandatory
Limit int // max results; clamped into [1, maxSearchLimit]
}
SearchQuery describes a relevance ranking request over managed resources. Visibility is applied in SQL before ranking: Scopes is the caller's visible (scope, scope_id) set as VisibleScopes computes it, so a resource the caller could not list is never ranked. An empty Scopes returns nothing rather than searching unscoped. A nil Embedding selects lexical-only ranking (the graceful-degradation path when no embedding provider is configured); a non-nil Embedding selects hybrid ranking.
func (SearchQuery) EffectiveLimit ¶ added in v1.115.0
func (q SearchQuery) EffectiveLimit() int
EffectiveLimit clamps the requested limit into the search bounds.
type Searcher ¶ added in v1.115.0
type Searcher interface {
Search(ctx context.Context, q SearchQuery) ([]ScoredResource, error)
}
Searcher ranks the resources visible to a caller by relevance to a query. It is a capability separate from Store: only a backing store that can rank (the PostgreSQL store with pgvector) implements it, so the feature degrades to absent rather than forcing every Store implementation to carry a ranking query.
type Store ¶
type Store interface {
Insert(ctx context.Context, r Resource) error
Get(ctx context.Context, id string) (*Resource, error)
// GetByIDs returns the resources among ids that exist, keyed by id. An id
// with no row is simply absent: the caller is reading a set of references
// and a missing one is an answer, not a failure.
//
// It is the read a listing over records that POINT AT resources needs, so
// a page costs one query rather than one per row.
GetByIDs(ctx context.Context, ids []string) (map[string]*Resource, error)
GetByURI(ctx context.Context, uri string) (*Resource, error)
List(ctx context.Context, filter Filter) ([]Resource, int, error)
// Folders returns every folder holding a resource the filter admits, with
// the exact number filed under each at every depth.
//
// It exists because a folder is derived from the paths in use rather than
// stored, and the portal used to derive it in the browser from the paged
// listing: drawing four folder names cost a fetch of the whole library, the
// counts read "25+" until the last page arrived, and the root offered a
// Load-more control over rows it never displayed (#1555). One grouped query
// answers it exactly and in one round trip.
Folders(ctx context.Context, filter Filter) ([]Folder, error)
// SetThumbnail records a capture: the object it was stored under and the
// moment it was taken.
//
// It is not an Update. Update bumps updated_at and drops the stored
// embedding, and a capture must do neither: bumping the timestamp would
// mark the capture that just landed as older than the row it came from,
// which is the definition of pending, so every capture would queue itself
// again forever (#1554).
SetThumbnail(ctx context.Context, id string, t ThumbnailCapture) error
// ClearThumbnail forgets a capture, which is how a wrong tile is asked to
// be taken again.
ClearThumbnail(ctx context.Context, id, variant string) error
// PendingThumbnails lists resources whose capture is missing or older than
// the file it came from, most recently changed first, capped at limit.
PendingThumbnails(ctx context.Context, filter Filter, limit int) ([]Resource, error)
// Tags returns the distinct tags carried by the resources the filter
// admits, so the tag facet offers what the library holds rather than what
// one page of it happened to carry.
Tags(ctx context.Context, filter Filter) ([]string, error)
Update(ctx context.Context, id string, u Update) error
// Move refiles resources, rewriting the four columns that say where each one
// lives -- scope, scope_id, path and uri -- and recording the URI each used
// to answer to so an already-written citation keeps resolving.
//
// It is separate from Update because it is not a metadata edit: it changes
// who can see the file, it changes the resource's address, and the address
// is UNIQUE, so it is the one write on this table that another resource can
// refuse. A caller must be prepared for ErrURIConflict.
//
// It takes a batch because renaming a folder is one relocation per resource
// beneath it and a half-renamed folder is not a state anyone should be able
// to observe (#1529). Every element commits or none does, which is also what
// makes the batch refusable as a whole.
//
// The blob is not touched. The S3 key embeds the scope only because
// BuildS3Key composed it at creation; nothing re-derives it on read, so the
// object stays where it is and the row keeps pointing at it.
Move(ctx context.Context, moves []Move) error
Delete(ctx context.Context, id string) error
}
Store persists and queries resource metadata.
func NewPostgresStore ¶
func NewPostgresStore(db *sql.DB, opts ...indexjobs.StoreOption) Store
NewPostgresStore creates a resource store backed by PostgreSQL. Pass indexjobs.WithProducer to have resource writes enqueue their own index job; without it, resources are indexed on the reconciler's next sweep.
Every notify fires after the write commits, never before: a job claimed while the row still holds its pre-write text (or its previous blob) would have the worker stamp that snapshot as current.
type ThumbnailCapture ¶ added in v1.127.0
ThumbnailCapture is one stored capture: which of the two it is, the object it was written to, and when it was taken.
type Update ¶
type Update struct {
DisplayName *string `json:"display_name,omitempty"`
Description *string `json:"description,omitempty"`
Tags []string `json:"tags,omitempty"`
// Path refiles the resource in another folder of its library. Like Scope it
// is not metadata: the folder path is half of the resource's URI, so an edit
// to it rewrites the address and records the one it vacated (#1528).
Path *string `json:"path,omitempty"`
// Scope names the library to move the resource into. Nil leaves it where it
// is, which is what every request that is not a move sends.
Scope *Scope `json:"scope,omitempty"`
// ScopeID is the persona name or the user's sub or address, and is empty for
// the global library. It is read only when Scope is set: a scope id on its
// own names no library.
ScopeID *string `json:"scope_id,omitempty"`
// ThumbnailS3Key and ThumbnailCapturedAt record a capture (#1554). They are
// written by the capture route alone and are not part of the metadata edit
// the PATCH route accepts: a person editing a description is not saying
// anything about the image.
ThumbnailS3Key *string `json:"-"`
ThumbnailCapturedAt *time.Time `json:"-"`
// ThumbnailVariant names which of the two a capture is for. Empty is the
// light one.
ThumbnailVariant string `json:"-"`
}
Update holds mutable fields for a PATCH operation.
Scope and ScopeID are the move (#1502) and are not metadata: they change who can see the file and they change its canonical URI. They travel on the same request because the permission check a move needs is the one the update route already runs -- the caller must be able to modify the resource before the destination is even considered -- and splitting them would have meant a second route re-deriving CanModifyResource.
func (Update) Fields ¶ added in v1.126.3
Fields reports whether the update carries any metadata edit. A request that is only a move has none, and applying an empty Update would still bump updated_at and drop the stored embedding for nothing.
func (Update) Relocates ¶ added in v1.126.5
Relocates reports whether the update moves the resource: into another library, into another folder, or both. The two travel together because they are the two halves of one URI, and an edit touching either rewrites the address once rather than twice (#1528).
type Usage ¶ added in v1.115.0
type Usage struct {
// Reads30d and Reads90d count audited reads in the trailing 30 and 90
// days. Both are bounded by the audit retention window: a deployment
// keeping 30 days of audit reports the same number twice.
Reads30d int64 `json:"reads_30d" example:"42"`
Reads90d int64 `json:"reads_90d" example:"117"`
// BySurface30d breaks the 30-day count down by Surface* value.
BySurface30d map[string]int64 `json:"by_surface_30d,omitempty"`
// LastReadAt is the most recent audited read within the retention window.
// The durable answer lives on the resource row (Resource.LastReadAt), which
// outlives retention; this field is what the rollup itself saw.
LastReadAt *time.Time `json:"last_read_at,omitempty"`
}
Usage is the audit-derived read activity of a single resource.
type UsageReader ¶ added in v1.115.0
type UsageReader interface {
ResourceUsage(ctx context.Context, resourceIDs []string) (map[string]Usage, error)
}
UsageReader aggregates read events into per-resource usage. The Postgres audit store implements it, the same way it implements prompt usage (#1009): the audit log is already the durable record of who read what, so usage is a rollup of it rather than a second set of counters to keep in sync.
type Version ¶ added in v1.115.0
type Version struct {
ResourceID string `json:"resource_id" example:"a1b2c3d4e5f67890a1b2c3d4e5f67890"`
Version int `json:"version" example:"3"`
MIMEType string `json:"mime_type" example:"text/markdown"`
SizeBytes int64 `json:"size_bytes" example:"34000"`
S3Key string `json:"s3_key" example:"resources/persona/data-engineer/a1b2/v3/etl-runbook.md"`
UploaderSub string `json:"uploader_sub" example:"550e8400-e29b-41d4-a716-446655440000"`
UploaderEmail string `json:"uploader_email" example:"marcus.johnson@example.com"`
// RestoredFrom names the version this revision re-promoted, or nil when the
// revision was a fresh upload. A restore is recorded as a new head revision
// rather than a rewind so the trail stays append-only.
RestoredFrom *int `json:"restored_from,omitempty" example:"1"`
// ChangeSummary says why the content changed, for a revision written on the
// uploader's behalf rather than picked by them. Empty for an upload, where
// the uploader is the answer. It is what the version panel shows beside the
// revision, so a reader of the history sees the reason without having to
// find the operation that caused it.
ChangeSummary string `json:"change_summary,omitempty" example:"put 3 rows back onto one line"`
CreatedAt time.Time `json:"created_at"`
}
Version is one recorded revision of a resource's content.
It carries only what varies per revision. Filename is absent on purpose: the canonical URI embeds the filename and revision keeps the URI stable, so every version of a resource shares the resource's filename.
type VersionStore ¶ added in v1.115.0
type VersionStore interface {
// AddRevision records a revision and points the resource head at its blob,
// in one transaction. The version number is assigned inside that
// transaction as one past the highest recorded, so two concurrent revisions
// cannot claim the same number, and the stored row is returned.
AddRevision(ctx context.Context, rev Revision) (*Version, error)
// ListVersions returns every recorded revision of a resource, newest first.
ListVersions(ctx context.Context, resourceID string) ([]Version, error)
// GetVersion returns one recorded revision. A missing revision surfaces as
// a wrapped sql.ErrNoRows (see IsNotFound).
GetVersion(ctx context.Context, resourceID string, version int) (*Version, error)
// PruneVersions deletes the oldest revisions beyond the newest keep and
// returns the deleted rows so the caller can remove their blobs. It never
// deletes a row whose S3 key is currently referenced by the resource head,
// so pruning can never orphan the live content.
PruneVersions(ctx context.Context, resourceID string, keep int) ([]Version, error)
}
VersionStore persists the content-revision trail of a resource and moves the head to a new revision. The Postgres resource store implements it; it is a separate interface from Store so a deployment or test that only needs metadata CRUD is not forced to model the version trail, and so callers can type-assert for the capability.
The trail and the head move together — a version row whose bytes no resource points at, or a head pointing at bytes no version row records, is a broken state — so AddRevision owns both writes rather than leaving the caller to sequence them.