Affected by 
Documentation
¶
Overview ¶
Package storageurl converts internal storage references into HTTP(S) URLs an external client can load directly.
WeKnora persists files behind two internal reference forms: the stable `resource://<handle>` application identity and legacy/canonical provider paths (`local://…`, `minio://…`, `storage://<backend-id>/cos://…`). Neither is fetchable by a browser or a third-party app, which must otherwise call the authenticated `/files` proxy for every image.
This package is the single implementation of the "give me a loadable link" translation. It is used by the IM channels (which have no way to attach WeKnora credentials to an image fetch) and, opt-in, by the HTTP API so integrators receive ready-to-render URLs.
Index ¶
- Constants
- Variables
- func BuildFileServiceForProvider(tenant *types.Tenant, provider string, defaultSvc interfaces.FileService) interfaces.FileService
- func FindIncompleteMarkdownImage(s string) int
- func FindIncompleteRef(s string) int
- func HandleModeForced(ctx context.Context) bool
- func HoldbackCutoff(chunk string) int
- func IsHTTPURL(s string) bool
- func LocalStorageBaseDir() string
- func Rewrite(ctx context.Context, content string, r Resolver, logPrefix string) string
- func WithForcedHandleMode(ctx context.Context) context.Context
- type FileServiceResolver
- type Held
- type Mode
- type Resolver
- type Rewriter
- func (w *Rewriter) CopyData(ctx context.Context, data map[string]interface{}) map[string]interface{}
- func (w *Rewriter) CopyReferences(ctx context.Context, refs []*types.SearchResult) []*types.SearchResult
- func (w *Rewriter) CopyValue(ctx context.Context, value interface{}) interface{}
- func (w *Rewriter) Enabled() bool
- func (w *Rewriter) Ref(ctx context.Context, ref string) string
- func (w *Rewriter) RewriteMessages(ctx context.Context, messages []*types.Message)
- func (w *Rewriter) RewriteMessagesResponse(ctx context.Context, messages []*types.Message) []*types.Message
- func (w *Rewriter) String(ctx context.Context, content string) string
- type StreamRewriter
Constants ¶
const ( // QueryParam is the request query parameter that selects the mode per call. QueryParam = "resource_urls" // EnvVar sets the deployment-wide default mode for requests that omit // QueryParam. It sits alongside APP_EXTERNAL_URL, which is what actually // makes `resource://` handles resolvable to a public `/r/<token>` URL. EnvVar = "RESOURCE_URL_MODE" )
Variables ¶
var ErrPublicModeForbidden = errors.New(
"resource_urls=public is not available for a knowledge-base-restricted API key")
ErrPublicModeForbidden is returned when the caller asked for ModePublic but its credentials must not receive anonymous, time-limited file URLs. Callers map it to 403 rather than 400: the request is well-formed, the scope is not.
var Pattern = types.StorageReferencePattern
Pattern matches every internal storage reference form: `resource://` handles, legacy `provider://` paths, and canonical `storage://<backend-id>/provider://` paths. The trailing character class stops at Markdown/HTML delimiters so a reference inside `` or `src="…"` is matched without them.
Functions ¶
func BuildFileServiceForProvider ¶
func BuildFileServiceForProvider( tenant *types.Tenant, provider string, defaultSvc interfaces.FileService, ) interfaces.FileService
BuildFileServiceForProvider selects the FileService for a storage provider. The reference's own scheme wins over the tenant DefaultProvider. It falls back to the process-wide default FileService (STORAGE_TYPE / env) when tenant config is missing — mirrors ImageMultimodalService.resolveFileServiceForPayload (issue #1282).
func FindIncompleteMarkdownImage ¶
FindIncompleteMarkdownImage returns the byte offset of an unclosed ` when the whole chunk can be emitted.
func IsHTTPURL ¶
IsHTTPURL reports whether s is an http(s) URL — the only form an external client can fetch; any provider scheme (oss://, local://, …) is not. Scheme match is case-insensitive per RFC 3986 §3.1: a backend may emit an operator-configured host (e.g. OBS_PROXY_DOMAIN) with an uppercase scheme.
func LocalStorageBaseDir ¶
func LocalStorageBaseDir() string
LocalStorageBaseDir is the on-disk root used when a reference resolves to the local provider.
func Rewrite ¶
Rewrite is the one-shot form of Rewriter.String for callers that translate a single self-contained message.
func WithForcedHandleMode ¶
WithForcedHandleMode pins a request to ModeHandle. Anonymous surfaces use it: an embed visitor is authorized only by the channel's session handle, so it must keep fetching images through the channel-scoped `/embed/…/files` proxy instead of receiving a shareable, credential-free URL.
Types ¶
type FileServiceResolver ¶
type FileServiceResolver struct {
// contains filtered or unexported fields
}
FileServiceResolver resolves and caches one FileService per storage provider. The cache is scoped to a single request or outbound message so a long answer with many images does not re-create an SDK client per reference.
Not safe for concurrent use; Rewriter drives it from one goroutine at a time.
func NewFileServiceResolver ¶
func NewFileServiceResolver( tenant *types.Tenant, defaultSvc interfaces.FileService, storageResolvers ...interfaces.StorageBackendResolver, ) *FileServiceResolver
NewFileServiceResolver builds a resolver for tenant. defaultSvc is the process-wide FileService used for `resource://` handles and as the fallback when tenant storage config is missing.
func (*FileServiceResolver) ResolveFileService ¶
func (r *FileServiceResolver) ResolveFileService(filePath string) interfaces.FileService
ResolveFileService implements Resolver.
func (*FileServiceResolver) WithContext ¶
func (r *FileServiceResolver) WithContext(ctx context.Context) *FileServiceResolver
WithContext sets the context used for backend lookups and their logs.
type Held ¶
type Held struct {
Content string
Meta interface{}
}
Held is one released tail: the rewritten content and the metadata carried by the last Push that contributed to it.
type Mode ¶
type Mode string
Mode selects how stored files are referenced in an API response.
const ( // ModeHandle returns the internal `resource://<handle>` reference. Clients // must fetch the bytes through the authenticated `/files` proxy. This is the // default and the only mode that never depends on external reachability. ModeHandle Mode = "handle" // ModePublic returns a time-limited HTTP(S) URL the client can load // directly, so third-party apps can render images without a second // authenticated call. References that cannot be turned into an HTTP URL are // left as handles. ModePublic Mode = "public" )
func DefaultMode ¶
DefaultMode returns the deployment-wide default from EnvVar. An unset or unparseable value yields ModeHandle, so a typo degrades to the safe default rather than failing every request.
func ParseMode ¶
ParseMode validates a mode supplied by a client or by configuration. An empty value yields ModeHandle so an unset parameter or setting keeps the default.
func ResolveMode ¶
ResolveMode combines the per-request query value with the deployment default, then applies the two limits that no caller may opt out of.
An explicit query value wins over the deployment default; an invalid one is returned as an error so integrators find their typo instead of silently receiving handles.
Limits:
- A ctx pinned by WithForcedHandleMode (anonymous embed traffic) is silently downgraded rather than rejected, so an embed client that happens to send the parameter keeps working — it just never gets a public URL.
- A knowledge-base-restricted API key is rejected with ErrPublicModeForbidden. Such a key is already denied the `/files` proxy, so handing it anonymous file URLs would widen its scope from "chunk text" to "file bytes" through the back door.
type Resolver ¶
type Resolver interface {
// ResolveFileService returns nil when no backend can serve ref.
ResolveFileService(ref string) interfaces.FileService
}
Resolver maps one storage reference to the FileService that owns it. A reference carries its own provider scheme and optional storage backend id, so a single answer may span several backends.
type Rewriter ¶
type Rewriter struct {
// contains filtered or unexported fields
}
Rewriter replaces storage references with loadable HTTP URLs.
Resolutions are memoised for the Rewriter's lifetime. Create one per request or per outbound message: a `resource://` handle costs one access-grant row per resolution, and a streamed answer repeats the same image across many chunks.
Safe for concurrent use: mu covers both the memo and the resolver, which is itself single-threaded. Resolution therefore serialises across goroutines, which is what we want — two chunks naming the same image must not each pay for a signature.
Logging policy:
- A successful rewrite logs the source reference at INFO. The signed URL is DEBUG-only so log aggregation cannot hand out anonymously readable links; operators can raise the log level when verifying reachability.
- Failure or no-op logs at WARN. The no-op case usually means APP_EXTERNAL_URL is unset, the most common cause of "image broken in the IM channel / in my app" reports.
func NewRequestRewriter ¶
func NewRequestRewriter( ctx context.Context, mode Mode, defaultSvc interfaces.FileService, storageResolver interfaces.StorageBackendResolver, ) *Rewriter
NewRequestRewriter builds the Rewriter for one API request or response stream.
ModeHandle yields a disabled Rewriter, so the default path resolves nothing and costs no access-grant rows. The tenant is taken from ctx because a reference may live on a tenant-configured storage backend rather than the process-wide default.
No extra authorization gate applies here, unlike the `/files` proxy which rejects KB-restricted API keys. That gate exists because `/files` takes an arbitrary caller-supplied path it cannot bind to a KB allow-list. Here the references come from a response the caller is already authorized to receive, so the server — not the client — chooses which resources get a URL.
func NewRewriter ¶
NewRewriter returns a Rewriter that resolves references through r. logPrefix tags log lines with the calling surface (for example "IM" or "API"). A nil resolver yields a Rewriter that leaves content unchanged.
func (*Rewriter) CopyData ¶
func (w *Rewriter) CopyData(ctx context.Context, data map[string]interface{}) map[string]interface{}
CopyData returns a rewritten copy of an SSE metadata map, or data itself when it holds no storage reference. Agent tool results put renderable Markdown into this map, and its shape is tool-defined, so every string leaf is rewritten.
func (*Rewriter) CopyReferences ¶
func (w *Rewriter) CopyReferences(ctx context.Context, refs []*types.SearchResult) []*types.SearchResult
CopyReferences returns rewritten copies of retrieval results, covering both the chunk text and the structured image_info payload.
Copies rather than in-place edits because an SSE references payload shares its *SearchResult pointers with the stream replay buffer and the assistant message being persisted; rewriting those in place would corrupt both.
func (*Rewriter) CopyValue ¶
CopyValue is CopyData for a decoded JSON value of any shape (object, array or scalar). Unchanged values are returned as-is.
func (*Rewriter) Ref ¶
Ref rewrites a value that is a bare storage reference rather than prose containing one — for example `MessageImage.URL`.
func (*Rewriter) RewriteMessages ¶
RewriteMessages replaces storage references in a message history response so clients receive loadable image URLs. Messages are mutated in place.
func (*Rewriter) RewriteMessagesResponse ¶
func (w *Rewriter) RewriteMessagesResponse(ctx context.Context, messages []*types.Message) []*types.Message
RewriteMessagesResponse returns rewritten message copies so callers can keep the originals untouched (for example objects returned from a service cache).
func (*Rewriter) String ¶
String replaces every storage reference in content with an HTTP URL. References that are already HTTP, that no backend claims, or that resolve to a non-HTTP location are left untouched so the caller degrades to the authenticated file proxy rather than emitting an unfetchable URL.
type StreamRewriter ¶
type StreamRewriter struct {
// contains filtered or unexported fields
}
StreamRewriter rewrites storage references in a stream of content deltas.
Each logical stream is identified by a key (WeKnora uses the SSE event id, the same key clients accumulate on). Push returns only the prefix that is safe to emit now, retaining any tail that may be an incomplete reference until the next Push for that key.
Safe for concurrent use.
func NewStreamRewriter ¶
func NewStreamRewriter(rewriter *Rewriter) *StreamRewriter
NewStreamRewriter wraps rewriter with per-stream holdback state.
func (*StreamRewriter) Enabled ¶
func (s *StreamRewriter) Enabled() bool
Enabled reports whether this StreamRewriter can rewrite anything.
func (*StreamRewriter) FlushAll ¶
func (s *StreamRewriter) FlushAll(ctx context.Context) map[string]Held
FlushAll releases every held tail, rewritten, keyed by stream. Callers use it when a stream ends without a terminal chunk (for example a client disconnect) so buffered content is not silently dropped.
func (*StreamRewriter) Push ¶
func (s *StreamRewriter) Push(ctx context.Context, key, chunk string, flush bool, meta interface{}) string
Push feeds the next chunk of the stream identified by key and returns the rewritten content that is ready to emit. Set flush on the stream's terminal chunk to release any held tail. meta is retained opaquely alongside the tail and handed back by FlushAll so a late release can carry the same metadata as the event it was cut from.
func (*StreamRewriter) Rewriter ¶
func (s *StreamRewriter) Rewriter() *Rewriter
Rewriter exposes the underlying Rewriter for stream fields that arrive whole (references, metadata) and therefore need no holdback.
Source Files