Documentation
¶
Index ¶
- Constants
- Variables
- func Conflictf(format string, args ...any) error
- func HasOperatorDetail(err error) bool
- func InvalidInputf(format string, args ...any) error
- func NotFoundf(format string, args ...any) error
- func ResolvePublicMessage(err error) (string, bool)
- func WithOperatorDetail(err error, detail error) error
- func WrapPublicf(err error, format string, args ...any) error
- type AttributeMetadata
- type AttributeStorageLocation
- type BatchConfig
- type BatchOperation
- type BatchResult
- type CascadeAction
- type CascadeRule
- type CompositeCondition
- type Condition
- type Config
- type ConfigError
- type CrossSchemaRequest
- type CursorQueryResult
- type DataRecord
- type DatabaseConfig
- type DuckDBConfig
- type EntityBatchCreator
- type EntityBatchOperator
- type EntityConfig
- type EntityIdentifier
- type EntityManager
- type EntityOperation
- type EntityReader
- type EntityUpdate
- type EntityWriter
- type ExecutionMerge
- type ExecutionPlan
- type ExecutionRouting
- type ExecutionSource
- type FederatedQueryRequest
- type FilterField
- type FilterType
- type JSONSchema
- type KvCondition
- type LoggingConfig
- type Logic
- type MainColumn
- type MainColumnBinding
- type MainColumnEncoding
- type MainColumnType
- type ManifestSchemaMismatchError
- type MetricsConfig
- type NoParquetPathsError
- type OperationError
- type OperationType
- type Option
- func WithDatabase(db DatabaseConfig) Option
- func WithDuckDB(d DuckDBConfig) Option
- func WithEntity(e EntityConfig) Option
- func WithLogging(l LoggingConfig) Option
- func WithMetrics(m MetricsConfig) Option
- func WithPerformance(p PerformanceConfig) Option
- func WithQuery(q QueryConfig) Option
- func WithReference(r ReferenceConfig) Option
- func WithSchemaRegistry(sr SchemaRegistry) Option
- func WithTransaction(t TransactionConfig) Option
- type OrderBy
- type ParquetSetInconsistentError
- type PartialResultInfo
- type PerformanceConfig
- type PropertySchema
- type PublicError
- type QueryConfig
- type QueryRequest
- type QueryResult
- type Reference
- type ReferenceConfig
- type ReferenceType
- type RelationSchema
- type RequiredPolicy
- type RoutingPolicy
- type RoutingStrategy
- type SchemaAttributeCache
- type SchemaRegistry
- type SortOrder
- type TableNames
- type TransactionConfig
- type ValueType
Constants ¶
const PartialReasonCorruptParquetExcluded = "corrupt_parquet_excluded"
PartialReasonCorruptParquetExcluded reports a #251 partial read: one or more verification-confirmed corrupt parquet objects were excluded and the page was answered from the readable remainder plus the hot tier.
Variables ¶
var ErrConflict = errors.New("conflict")
ErrConflict is returned when an operation would violate a uniqueness constraint or produce a duplicate record.
var ErrInvalidInput = errors.New("invalid input")
ErrInvalidInput is returned when caller-supplied input fails validation (missing required fields, unsupported values, etc.).
var ErrManifestSchemaMismatch = errors.New("manifest schema id does not match the schema being read")
ErrManifestSchemaMismatch marks a manifest object whose recorded schema ID disagrees with the schema being read. A manifest addresses one schema by path convention alone; nothing downstream re-checks it — the parquet scan does not filter rows by schema (files are per-schema by path) and the projection stamps whatever it scans as the requested schema. So a path collision between two schemas would not merely under-read, it would serve another schema's rows under this schema's identity. Not degradable: cross-schema contamination is the opposite of a partial answer.
var ErrNoParquetPaths = errors.New("no parquet paths resolved")
ErrNoParquetPaths marks a DuckDB-routed federated read whose parquet path set resolved empty: no per-request render hint, and either no configured parquet source or a source whose manifest lists no files while the fallback glob is disabled. Every query that reaches the DuckDB engine wants warm and/or cold data (hot-only requests short-circuit to Postgres first), so an empty set cannot be answered honestly. Not degradable — a Postgres-only fallback would be silently short precisely where the cold tier was requested (#299).
var ErrNotFound = errors.New("not found")
ErrNotFound is returned when a requested entity does not exist.
var ErrParquetSetInconsistent = errors.New("parquet set inconsistent with manifest")
ErrParquetSetInconsistent marks a federated read whose manifest lists parquet objects that do not exist in storage. The manifest is the authoritative record of the schema's cold/warm tier, so a listed-but-absent object means that tier has lost data. Not degradable: surfacing it even under AllowPartialDegradedMode is the whole point — degrading here would return exactly the silently short answer this classification exists to make loud (#187 scenario 2).
It lives here rather than in internal/federated for the same reason as the two errors above, plus one specific to #301: internal/httpapi must classify it to redact its object keys, and cannot import internal/federated without pulling DuckDB CGO into a pure-Go test build.
Functions ¶
func Conflictf ¶ added in v0.2.0
Conflictf builds a client error classified by ErrConflict. Error() renders "<message>: conflict".
func HasOperatorDetail ¶ added in v0.2.0
HasOperatorDetail reports whether err withholds detail from its published message. internal/httpapi keys a log level on it: a disclosed 4xx whose chain still holds operator text makes the log line the only copy of that text.
func InvalidInputf ¶ added in v0.2.0
InvalidInputf builds a client error classified by ErrInvalidInput whose formatted message is published verbatim. Error() renders "<message>: invalid input".
func NotFoundf ¶ added in v0.2.0
NotFoundf builds a client error classified by ErrNotFound. Error() renders "<message>: not found".
func ResolvePublicMessage ¶ added in v0.2.0
ResolvePublicMessage returns the message err deliberately published, if any. It is the canonical resolution used by the HTTP boundary, by WrapPublicf/WithOperatorDetail to qualify their input, and by the decorators' own PublicMessage delegation — one traversal, so the publication a decorator carries is exactly the one the boundary would emit.
The walk is preorder, left-first (errors.As order, so wrap prefixes accumulate outermost-wins), and each node is matched the way errors.As matches: direct implementation or the node's As(any) bool protocol (#363 review, P2). A node qualifies only when a client sentinel is reachable from that node's OWN subtree (#362/#363 reviews, P1): two gates searching the whole tree independently let a mixed tree borrow — sentinel evidence from one branch, PublicMessage() from a foreign sibling — and decorating such a tree must not manufacture the same borrow one level up. Non-qualifying nodes are stepped over, not terminal, so a real carrier behind a foreign publisher still resolves. An empty publication is treated as no publication and the walk continues past it.
func WithOperatorDetail ¶ added in v0.2.0
WithOperatorDetail attaches operator-only detail to a client error. The detail joins the chain (errors.Is/As still see it) and its text reaches Error(), so logs keep it — but it never reaches PublicMessage(). Returns err unchanged when detail is nil or err carries no PublicError, and nil for a nil err.
func WrapPublicf ¶ added in v0.2.0
WrapPublicf adds caller-actionable context to err, prefixing BOTH its operator message and its published message: Error() is identical to fmt.Errorf("<prefix>: %w", err) either way. When err carries no PublicError the result IS exactly that plain wrap — an operator error can never gain a publication by being wrapped. Returns nil for a nil err.
Use it only where the wrap adds identification the caller can act on (a batch index, a caller-supplied name). A layer that adds operator context should stay a plain fmt.Errorf so its prefix stays out of the body.
Types ¶
type AttributeMetadata ¶ added in v0.0.3
type AttributeMetadata struct {
AttributeName string `json:"attr_name"` // attr_name, JSON Path
AttributeID int16 `json:"attr_id"` // attr_id
ValueType ValueType `json:"value_type"` // 'text', 'numeric', 'date', 'bool'
// ItemsType is the element value type when ValueType is ValueTypeList.
// Empty means text (see EffectiveItemsType).
ItemsType ValueType `json:"items_type,omitempty"`
RequiredPolicy RequiredPolicy `json:"required_policy,omitempty"`
// Required is kept for backward compatibility with older metadata readers.
// New code should use RequiredPolicy.
Required bool `json:"required,omitempty"`
ColumnBinding *MainColumnBinding `json:"column_binding,omitempty"`
// Retired marks an attribute removed from the schema whose entry stays in
// the attributes file as the attributeID ledger (#342). Retired attributes
// are excluded from every active cache — reads skip their EAV rows (#294),
// writes preserve them — but their (id, name, valueType) still guards
// against rebinding the freed id to a different attribute.
Retired bool `json:"retired,omitempty"`
}
AttributeMetadata stores cached metadata from the attributes table.
func (AttributeMetadata) EffectiveItemsType ¶ added in v0.2.0
func (m AttributeMetadata) EffectiveItemsType() ValueType
EffectiveItemsType returns the element type of a list attribute, defaulting to text when items_type is not declared.
func (AttributeMetadata) EffectiveRequiredPolicy ¶ added in v0.0.27
func (m AttributeMetadata) EffectiveRequiredPolicy() RequiredPolicy
EffectiveRequiredPolicy returns the policy used at runtime. If RequiredPolicy is unset, it falls back to legacy Required semantics.
func (AttributeMetadata) IsInsideArray ¶ added in v0.0.3
func (m AttributeMetadata) IsInsideArray() bool
IsInsideArray infers if the attribute is inside an array based on its name.
func (AttributeMetadata) Location ¶ added in v0.0.3
func (m AttributeMetadata) Location() AttributeStorageLocation
Location returns where the attribute is stored. If ColumnBinding is nil, the attribute is in EAV; otherwise in main table.
type AttributeStorageLocation ¶ added in v0.0.3
type AttributeStorageLocation string
AttributeStorageLocation enumerates where the attribute physically resides.
const ( AttributeStorageLocationUnknown AttributeStorageLocation = "" AttributeStorageLocationMain AttributeStorageLocation = "main" AttributeStorageLocationEAV AttributeStorageLocation = "eav" )
type BatchConfig ¶ added in v0.0.2
type BatchConfig struct {
EnableDynamicSizing bool `json:"enableDynamicSizing"`
EnableParallelProcessing bool `json:"enableParallelProcessing"`
EnableBatchStreaming bool `json:"enableBatchStreaming"`
ParallelThreshold int `json:"parallelThreshold"`
StreamingThreshold int `json:"streamingThreshold"`
MaxParallelWorkers int `json:"maxParallelWorkers"`
StreamingChunkSize int `json:"streamingChunkSize"`
StreamingDelay int `json:"streamingDelay"` // milliseconds
MaxComplexityPerBatch int `json:"maxComplexityPerBatch"`
AttributeComplexityScore int `json:"attributeComplexityScore"`
OptimalChunkSize int `json:"optimalChunkSize"`
}
BatchConfig contains batch processing settings
type BatchOperation ¶
type BatchOperation struct {
Operations []EntityOperation `json:"operations"`
Atomic bool `json:"atomic"` // Request all-or-nothing execution; may be rejected when unsupported.
}
BatchOperation represents batch entity operations
type BatchResult ¶
type BatchResult struct {
Successful []*DataRecord `json:"successful"`
Failed []OperationError `json:"failed"`
TotalCount int `json:"totalCount"`
Duration int64 `json:"duration"` // microseconds
}
BatchResult represents results from batch operations
type CascadeAction ¶ added in v0.0.2
type CascadeAction string
CascadeAction defines the type of cascade action
const ( CascadeActionDelete CascadeAction = "delete" CascadeActionUpdate CascadeAction = "update" CascadeActionNullify CascadeAction = "nullify" CascadeActionRestrict CascadeAction = "restrict" )
type CascadeRule ¶ added in v0.0.2
type CascadeRule struct {
SourceSchema string `json:"sourceSchema"`
TargetSchema string `json:"targetSchema"`
Action CascadeAction `json:"action"`
MaxDepth int `json:"maxDepth,omitempty"`
}
CascadeRule defines cascade behavior for specific schema relationships
type CompositeCondition ¶
--- 3. Composite Condition (Non-Leaf Node) ---
func (*CompositeCondition) IsLeaf ¶
func (c *CompositeCondition) IsLeaf() bool
func (*CompositeCondition) UnmarshalJSON ¶
func (c *CompositeCondition) UnmarshalJSON(data []byte) error
UnmarshalJSON customizes decoding so that nested conditions are turned into the appropriate concrete condition implementations.
type Config ¶ added in v0.0.2
type Config struct {
Database DatabaseConfig `json:"database"`
Query QueryConfig `json:"query"`
Entity EntityConfig `json:"entity"`
Transaction TransactionConfig `json:"transaction"`
Performance PerformanceConfig `json:"performance"`
Logging LoggingConfig `json:"logging"`
Metrics MetricsConfig `json:"metrics"`
Reference ReferenceConfig `json:"reference"`
DuckDB DuckDBConfig `json:"duckdb"`
SchemaRegistry SchemaRegistry `json:"-"` // Custom schema registry implementation (optional)
}
Config consolidates settings from both modules
func DefaultConfig ¶ added in v0.0.2
func DefaultConfig(schemaRegistry SchemaRegistry) *Config
DefaultConfig returns a default configuration
func NewConfig ¶ added in v0.0.25
NewConfig creates a Config starting from the defaults produced by DefaultConfig(nil) and then applies each provided Option in order. The schema registry can be set via WithSchemaRegistry.
Example:
cfg := forma.NewConfig(
forma.WithDatabase(forma.DatabaseConfig{Host: "db.example.com", Port: 5432, MaxConnections: 50}),
forma.WithDuckDB(forma.DuckDBConfig{Enabled: true, DBPath: ":memory:"}),
)
type ConfigError ¶ added in v0.0.2
ConfigError represents a configuration validation error
func (*ConfigError) Error ¶ added in v0.0.2
func (e *ConfigError) Error() string
type CrossSchemaRequest ¶
type CrossSchemaRequest struct {
SchemaNames []string `json:"schema_names" validate:"required"`
SearchTerm string `json:"search_term" validate:"required"`
Page int `json:"page" validate:"min=1"`
ItemsPerPage int `json:"items_per_page" validate:"min=1,max=100"`
Condition Condition `json:"-"` // Custom unmarshal, can be CompositeCondition or KvCondition
Attrs []string `json:"attrs,omitempty"` // Attributes to return (field projection)
}
CrossSchemaRequest represents a cross-schema search request
func (CrossSchemaRequest) MarshalJSON ¶
func (r CrossSchemaRequest) MarshalJSON() ([]byte, error)
MarshalJSON implements custom JSON marshaling for CrossSchemaRequest.
func (*CrossSchemaRequest) UnmarshalJSON ¶
func (r *CrossSchemaRequest) UnmarshalJSON(data []byte) error
UnmarshalJSON implements custom JSON unmarshaling for CrossSchemaRequest. It allows the Condition field to be either a CompositeCondition or KvCondition.
type CursorQueryResult ¶
type CursorQueryResult struct {
Data []*DataRecord `json:"data"`
NextCursor string `json:"next_cursor,omitempty"`
HasMore bool `json:"has_more"`
ExecutionTime time.Duration `json:"execution_time"`
}
CursorQueryResult represents cursor-based pagination results.
type DataRecord ¶
type DatabaseConfig ¶ added in v0.0.2
type DatabaseConfig struct {
Host string `json:"host"`
Port int `json:"port"`
Database string `json:"database"`
Username string `json:"username"`
Password string `json:"password"`
SSLMode string `json:"sslMode"`
Schema string `json:"schema"`
MaxConnections int `json:"maxConnections"`
MaxIdleConns int `json:"maxIdleConns"`
ConnMaxLifetime time.Duration `json:"connMaxLifetime"`
ConnMaxIdleTime time.Duration `json:"connMaxIdleTime"`
Timeout time.Duration `json:"timeout"`
TableNames TableNames `json:"tableNames"`
}
DatabaseConfig contains database connection settings
type DuckDBConfig ¶ added in v0.0.23
type DuckDBConfig struct {
Enabled bool `json:"enabled"`
DBPath string `json:"dbPath"` // path to local DuckDB file (or ":memory:")
MemoryLimitMB int `json:"memoryLimitMB"` // memory limit for DuckDB in MB
EnableS3 bool `json:"enableS3"` // enable S3/http file system
S3Endpoint string `json:"s3Endpoint"` // custom S3 endpoint (for MinIO)
S3AccessKey string `json:"s3AccessKey"`
S3SecretKey string `json:"s3SecretKey"`
S3Region string `json:"s3Region"`
EnableParquet bool `json:"enableParquet"` // enable parquet extension
Extensions []string `json:"extensions"` // additional extensions to load
MaxConnections int `json:"maxConnections"`
QueryTimeout time.Duration `json:"queryTimeout"` // per-query timeout for DuckDB access
MaxParallelism int `json:"maxParallelism"` // max threads/pragmas for DuckDB
// Deprecated: ignored failure-rate threshold; use
// CircuitBreakerFailureThreshold instead.
CircuitBreakerThreshold float64 `json:"circuitBreakerThreshold"`
// CircuitBreakerFailureThreshold is the number of consecutive failures that
// opens the DuckDB circuit breaker. Zero means use the built-in default.
CircuitBreakerFailureThreshold int `json:"circuitBreakerFailureThreshold"`
// CircuitBreakerWindow is the sliding window for counting DuckDB failures.
// Zero means use the built-in default.
CircuitBreakerWindow time.Duration `json:"circuitBreakerWindow"`
// CircuitBreakerOpenDuration is how long the DuckDB breaker stays open after
// tripping. Zero means use the built-in default.
CircuitBreakerOpenDuration time.Duration `json:"circuitBreakerOpenDuration"`
// FlushVisibilityGraceMs is the #252 clock-skew margin for the federated
// dirty barrier. Each query anchors its cutoff at the instant it resolved
// its parquet path set: rows marked flushed after that instant (their
// delta may be missing from the resolved set) stay hot-readable, minus
// this margin to absorb CDC-host vs query-host clock skew. Zero (the
// default) is the exact anchor — the steady state is never widened; a
// positive value hot-serves rows flushed up to that long before the
// query; a negative value disables the widening (the pre-#252 barrier).
FlushVisibilityGraceMs int64 `json:"flushVisibilityGraceMs"`
// S3Bucket is the bucket holding the lakehouse parquet files and the
// manifests that index them. Required whenever ManifestTemplate is set.
S3Bucket string `json:"s3Bucket"`
// S3DataPrefix mirrors the CDC write side's S3Prefix (the prefix under
// which delta/base parquet files are written). It is used *only* for the
// legacy glob fallback covering schemas that have never been flushed and
// therefore have no manifest object yet. Empty disables that fallback: a
// schema without a manifest then resolves to zero parquet paths, and a
// DuckDB-routed read of it fails fast with ErrNoParquetPaths — a
// non-degradable read-path error naming the schema, distinct from the
// transient failures AllowPartialDegradedMode absorbs (#299). Leave this
// set unless every schema is known to be manifested.
S3DataPrefix string `json:"s3DataPrefix"`
// ManifestPrefix is the root prefix for manifest objects in S3. It must
// match the CDC/compaction write side's ManifestPrefix.
ManifestPrefix string `json:"manifestPrefix"`
// ManifestTemplate is the manifest path template (e.g.
// "manifest/{{.SchemaID}}.json") and doubles as the enable gate for
// manifest-driven parquet resolution. It must be identical to the write
// side's template: a writer-on/reader-off mismatch makes the reader miss
// the cold tier entirely and lose those rows silently — no error is
// raised, the result set is merely short. Empty (the default) keeps the
// pre-existing glob-based read path.
ManifestTemplate string `json:"manifestTemplate"`
// AllowCallerParquetPaths opts the deployment into honoring the
// per-request federated.s3_parquet_path_template field (#456). It defaults
// to false: the field is a caller-controlled scan target, so on every
// existing deployment a request carrying it is rejected. When true, a
// caller template is still honored only for paths inside the configured
// S3Bucket (see the engine's path resolver) — the flag is necessary, not
// sufficient.
AllowCallerParquetPaths bool `json:"allowCallerParquetPaths"`
Routing RoutingPolicy `json:"routing"` // routing policy for federated queries
}
DuckDBConfig contains DuckDB connection and S3 settings for federated queries
func (DuckDBConfig) ManifestReadEnabled ¶ added in v0.2.0
func (d DuckDBConfig) ManifestReadEnabled() bool
ManifestReadEnabled reports whether manifest-driven parquet path resolution is configured. ManifestTemplate is the single enable gate.
The TrimSpace here is defense-in-depth, not normalization: a padded template is rejected outright by ValidateManifestRead (byte parity with the untrimmed write side is the contract), so no validated config ever reaches this method with surrounding whitespace. The trim only guards direct programmatic misuse — a caller that builds a DuckDBConfig by hand and skips validation — where reading a whitespace-only template as "manifest reads are on" would be strictly worse.
func (DuckDBConfig) ValidateCallerParquetPaths ¶ added in v0.2.0
func (d DuckDBConfig) ValidateCallerParquetPaths() error
ValidateCallerParquetPaths rejects the #456 caller-path opt-in when no bucket is configured to scope hints against. Honoring federated.s3_parquet_path_template requires a non-empty S3Bucket: validateHintPathScope would otherwise reject every such request at run time, turning an operator misconfiguration into invalid caller input. Like ValidateManifestRead, validation is independent of Enabled — whether the setting takes effect is the factory's gate, but an incoherent combination is always a configuration error. Both cmd/server and the factory call this beside ValidateManifestRead so the failure lands at startup, before any I/O, rather than on the first hint-bearing request.
func (DuckDBConfig) ValidateManifestRead ¶ added in v0.2.0
func (d DuckDBConfig) ValidateManifestRead() error
ValidateManifestRead validates the manifest read surface. It returns a *ConfigError naming the offending field, or nil when the combination is coherent. Validation is independent of Enabled: whether the settings take effect is the factory's gate, but an incoherent combination is always a configuration error.
type EntityBatchCreator ¶ added in v0.0.24
type EntityBatchCreator interface {
BatchCreate(ctx context.Context, req *BatchOperation) (*BatchResult, error)
}
type EntityBatchOperator ¶ added in v0.0.24
type EntityBatchOperator interface {
BatchCreate(ctx context.Context, req *BatchOperation) (*BatchResult, error)
BatchUpdate(ctx context.Context, req *BatchOperation) (*BatchResult, error)
BatchDelete(ctx context.Context, req *BatchOperation) (*BatchResult, error)
}
type EntityConfig ¶ added in v0.0.2
type EntityConfig struct {
EnableReferenceValidation bool `json:"enableReferenceValidation"`
EnableCascadeDelete bool `json:"enableCascadeDelete"`
BatchSize int `json:"batchSize"`
CacheEnabled bool `json:"cacheEnabled"`
CacheTTL time.Duration `json:"cacheTTL"`
MaxEntitySize int `json:"maxEntitySize"`
EnableVersioning bool `json:"enableVersioning"`
SchemaDirectory string `json:"schemaDirectory"`
// ValidateUpdatesStrict makes update payloads that violate the entity's
// JSON Schema fail with 4xx instead of only being logged (#314).
//
// Default false: rows written before schema enforcement may already violate
// their schema, and rejecting on update would make them un-updatable — a
// caller touching one unrelated field would be refused for a pre-existing
// violation elsewhere. Creates are always enforced; they have no legacy data.
//
// Whether it is safe to flip this yet is answered by the #317 aggregate:
// the "report-only schema validation violations reached a milestone" Warn
// line (per schema, at the 1st and every 100th accepted violation) and the
// entity_report_only_validation_violation_total telemetry counter. While
// those lines keep appearing for a schema, its rows are not yet repaired.
// Their absence is not proof of the converse: the signal fires only when a
// violating row is written, and only every 100th time thereafter within one
// process (strictly, per EntityManager — the shipped wiring builds exactly
// one), so a schema whose bad rows are never updated stays silent.
// Confirm with an e2e pass over real data before flipping
// (docs/error-handling.md).
ValidateUpdatesStrict bool `json:"validateUpdatesStrict"`
}
EntityConfig contains entity management settings
type EntityIdentifier ¶
type EntityIdentifier struct {
SchemaName string `json:"schemaName"`
RowID uuid.UUID `json:"rowId"`
}
EntityIdentifier identifies an entity for operations
type EntityManager ¶
type EntityManager interface {
EntityWriter
EntityReader
EntityBatchOperator
Close() error
}
EntityManager provides comprehensive entity and query operations.
Close releases resources the manager owns — for factory-built managers that is the embedded DuckDB client (#302). It is safe for concurrent use: teardown runs exactly once and every call returns the same result, so a failed close stays observable to late callers. Managers constructed without owned resources return nil. Embedders must Close the manager when done with it or the DuckDB instance lives until process exit.
type EntityOperation ¶
type EntityOperation struct {
EntityIdentifier
Type OperationType `json:"type"`
Data map[string]any `json:"data,omitempty"`
Updates map[string]any `json:"updates,omitempty"`
}
EntityOperation represents CRUD operations
type EntityReader ¶ added in v0.0.24
type EntityReader interface {
Get(ctx context.Context, req *QueryRequest) (*DataRecord, error)
Query(ctx context.Context, req *QueryRequest) (*QueryResult, error)
CrossSchemaSearch(ctx context.Context, req *CrossSchemaRequest) (*QueryResult, error)
}
type EntityUpdate ¶
type EntityUpdate struct {
EntityIdentifier
Updates any `json:"updates"`
}
EntityUpdate represents an update operation
type EntityWriter ¶ added in v0.0.24
type EntityWriter interface {
Create(ctx context.Context, req *EntityOperation) (*DataRecord, error)
Update(ctx context.Context, req *EntityOperation) (*DataRecord, error)
Delete(ctx context.Context, req *EntityOperation) error
}
type ExecutionMerge ¶ added in v0.2.0
type ExecutionMerge struct {
Strategy string `json:"strategy,omitempty"`
PreferHot bool `json:"prefer_hot,omitempty"`
DedupKeys []string `json:"dedup_keys,omitempty"`
DurationMs int64 `json:"duration_ms,omitempty"`
}
ExecutionMerge describes the tier-merge strategy the plan applied.
type ExecutionPlan ¶ added in v0.2.0
type ExecutionPlan struct {
Routing ExecutionRouting `json:"routing"`
Sources []ExecutionSource `json:"sources,omitempty"`
Merge *ExecutionMerge `json:"merge,omitempty"`
Timings map[string]int64 `json:"timings,omitempty"`
}
ExecutionPlan is the JSON-serializable projection of the engine's internal execution plan surfaced on QueryResult. It carries only safe routing/tier metadata an external caller needs to understand the route.
SECURITY: the internal plan's SQL, bind params, and free-text notes are deliberately NOT projected here. Since #306, the database password is redacted at the source (plan SQL and failure notes carry password=***REDACTED***), but the rendered DuckDB SQL still embeds host/user/dbname and table internals; Params carry query arguments, and notes carry storage keys and engine internals. Exposing them on the HTTP API would leak internals to any caller of advanced_query. Only enum/numeric/static-string fields are surfaced. It also imports nothing so it can live in the public API.
type ExecutionRouting ¶ added in v0.2.0
type ExecutionRouting struct {
UsedDuckDB bool `json:"used_duckdb"`
Tiers []string `json:"tiers,omitempty"`
Reason string `json:"reason,omitempty"`
}
ExecutionRouting reports the routing decision the engine committed to.
type ExecutionSource ¶ added in v0.2.0
type ExecutionSource struct {
Tier string `json:"tier"`
Engine string `json:"engine,omitempty"`
RowEstimate int64 `json:"row_estimate,omitempty"`
ActualRows int64 `json:"actual_rows,omitempty"`
PredicatePushdown bool `json:"predicate_pushdown,omitempty"`
DurationMs int64 `json:"duration_ms,omitempty"`
Reason string `json:"reason,omitempty"`
}
ExecutionSource describes one physical data source (tier) the plan read from. It intentionally omits the raw SQL and bind params (see ExecutionPlan).
type FederatedQueryRequest ¶ added in v0.1.0
type FederatedQueryRequest struct {
Enabled bool `json:"enabled,omitempty"`
PreferredTiers []string `json:"preferred_tiers,omitempty"`
PreferHot bool `json:"prefer_hot,omitempty"`
UseMainAsAnchor bool `json:"use_main_as_anchor,omitempty"`
S3ParquetPathTemplate string `json:"s3_parquet_path_template,omitempty"`
AllowPartialDegradedMode bool `json:"allow_partial_degraded_mode,omitempty"`
IncludeExecutionPlan bool `json:"include_execution_plan,omitempty"`
ConsistencyMode string `json:"consistency_mode,omitempty"`
}
FederatedQueryRequest carries optional hints for routing QueryRequest through the federated repository path while preserving the normal API surface for callers that do not need DuckDB/S3-backed reads.
type FilterField ¶
type FilterField string
const ( FilterFieldAttributeName FilterField = "attr_name" FilterFieldValueText FilterField = "value_text" FilterFieldValueNumeric FilterField = "value_numeric" FilterFieldRowID FilterField = "row_id" FilterFieldSchemaName FilterField = "schema_name" )
type FilterType ¶
type FilterType string
FilterType defines supported filter operations
const ( FilterEquals FilterType = "equals" FilterNotEquals FilterType = "not_equals" FilterStartsWith FilterType = "starts_with" FilterContains FilterType = "contains" FilterGreaterThan FilterType = "gt" FilterLessThan FilterType = "lt" FilterGreaterEq FilterType = "gte" FilterLessEq FilterType = "lte" FilterIn FilterType = "in" FilterNotIn FilterType = "not_in" )
type JSONSchema ¶ added in v0.0.15
type JSONSchema struct {
ID int16 `json:"id"`
Name string `json:"name"`
Version int `json:"version"`
Schema string `json:"schema"`
Properties map[string]*PropertySchema `json:"properties"`
Required []string `json:"required"`
CreatedAt int64 `json:"created_at"`
}
JSONSchema represents a schema definition.
type KvCondition ¶
--- 4. KvCondition (Leaf Node) ---
func (*KvCondition) IsLeaf ¶
func (kv *KvCondition) IsLeaf() bool
func (*KvCondition) UnmarshalJSON ¶
func (kv *KvCondition) UnmarshalJSON(data []byte) error
UnmarshalJSON ensures short-hand keys are present.
type LoggingConfig ¶ added in v0.0.2
type LoggingConfig struct {
Level string `json:"level"`
Format string `json:"format"`
EnableStructured bool `json:"enableStructured"`
EnablePerformance bool `json:"enablePerformance"`
EnableQueryLogging bool `json:"enableQueryLogging"`
LogSlowQueries bool `json:"logSlowQueries"`
SlowQueryThreshold time.Duration `json:"slowQueryThreshold"`
MaxLogSize int `json:"maxLogSize"`
LogRotation bool `json:"logRotation"`
SanitizeParameters bool `json:"sanitizeParameters"`
LogQueries bool `json:"logQueries"`
LogErrors bool `json:"logErrors"`
LogSecurityEvents bool `json:"logSecurityEvents"`
LogPerformanceWarnings bool `json:"logPerformanceWarnings"`
LogAllOperations bool `json:"logAllOperations"`
EnableDetailedLogging bool `json:"enableDetailedLogging"`
}
LoggingConfig contains logging settings
type MainColumn ¶ added in v0.0.3
type MainColumn string
MainColumn represents column names in the main entity table.
const ( MainColumnText01 MainColumn = "text_01" MainColumnText02 MainColumn = "text_02" MainColumnText03 MainColumn = "text_03" MainColumnText04 MainColumn = "text_04" MainColumnText05 MainColumn = "text_05" MainColumnText06 MainColumn = "text_06" MainColumnText07 MainColumn = "text_07" MainColumnText08 MainColumn = "text_08" MainColumnText09 MainColumn = "text_09" MainColumnText10 MainColumn = "text_10" MainColumnSmallint01 MainColumn = "smallint_01" MainColumnSmallint02 MainColumn = "smallint_02" MainColumnInteger01 MainColumn = "integer_01" MainColumnInteger02 MainColumn = "integer_02" MainColumnInteger03 MainColumn = "integer_03" MainColumnBigint01 MainColumn = "bigint_01" MainColumnBigint02 MainColumn = "bigint_02" MainColumnBigint03 MainColumn = "bigint_03" MainColumnBigint04 MainColumn = "bigint_04" MainColumnBigint05 MainColumn = "bigint_05" MainColumnDouble01 MainColumn = "double_01" MainColumnDouble02 MainColumn = "double_02" MainColumnDouble03 MainColumn = "double_03" MainColumnDouble04 MainColumn = "double_04" MainColumnDouble05 MainColumn = "double_05" MainColumnUUID01 MainColumn = "uuid_01" MainColumnUUID02 MainColumn = "uuid_02" MainColumnCreatedAt MainColumn = "ltbase_created_at" MainColumnUpdatedAt MainColumn = "ltbase_updated_at" MainColumnDeletedAt MainColumn = "ltbase_deleted_at" MainColumnCreatedBy MainColumn = "ltbase_created_by" MainColumnUpdatedBy MainColumn = "ltbase_updated_by" MainColumnDeletedBy MainColumn = "ltbase_deleted_by" MainColumnSchemaID MainColumn = "ltbase_schema_id" MainColumnRowID MainColumn = "ltbase_row_id" )
type MainColumnBinding ¶ added in v0.0.3
type MainColumnBinding struct {
ColumnName MainColumn `json:"col_name"`
Encoding MainColumnEncoding `json:"encoding,omitempty"`
}
MainColumnBinding describes how a schema attribute maps into a hot attribute column.
func (*MainColumnBinding) ColumnType ¶ added in v0.0.3
func (m *MainColumnBinding) ColumnType() MainColumnType
ColumnType derives the column type from the column name prefix.
type MainColumnEncoding ¶ added in v0.0.3
type MainColumnEncoding string
MainColumnEncoding represents special encoding for main column values.
const ( MainColumnEncodingDefault MainColumnEncoding = "default" MainColumnEncodingBoolText MainColumnEncoding = "bool_text" // "1"/"0" string in a text column MainColumnEncodingUnixMs MainColumnEncoding = "unix_ms" MainColumnEncodingBoolInt MainColumnEncoding = "bool_smallint" MainColumnEncodingISO8601 MainColumnEncoding = "iso8601" )
type MainColumnType ¶ added in v0.0.3
type MainColumnType string
MainColumnType represents the data type of a main column.
const ( MainColumnTypeText MainColumnType = "text" MainColumnTypeSmallint MainColumnType = "smallint" MainColumnTypeInteger MainColumnType = "integer" MainColumnTypeBigint MainColumnType = "bigint" MainColumnTypeDouble MainColumnType = "double" MainColumnTypeUUID MainColumnType = "uuid" )
type ManifestSchemaMismatchError ¶ added in v0.2.0
type ManifestSchemaMismatchError struct {
RequestedSchemaID int16
ManifestSchemaID int16
Path string
}
ManifestSchemaMismatchError names both schema IDs and the manifest object, so an operator can see which template or object is misaddressed.
func (*ManifestSchemaMismatchError) Error ¶ added in v0.2.0
func (e *ManifestSchemaMismatchError) Error() string
func (*ManifestSchemaMismatchError) Unwrap ¶ added in v0.2.0
func (e *ManifestSchemaMismatchError) Unwrap() error
type MetricsConfig ¶ added in v0.0.2
type MetricsConfig struct {
Enabled bool `json:"enabled"`
Provider string `json:"provider"` // prometheus, statsd, etc.
Endpoint string `json:"endpoint"`
CollectionInterval time.Duration `json:"collectionInterval"`
EnableHistograms bool `json:"enableHistograms"`
EnableCounters bool `json:"enableCounters"`
EnableGauges bool `json:"enableGauges"`
Namespace string `json:"namespace"`
Labels map[string]string `json:"labels"`
MaxSamples int `json:"maxSamples"`
EnableOperationMetrics bool `json:"enableOperationMetrics"`
EnableTransactionMetrics bool `json:"enableTransactionMetrics"`
EnablePatternMetrics bool `json:"enablePatternMetrics"`
}
MetricsConfig contains metrics collection settings
type NoParquetPathsError ¶ added in v0.2.0
type NoParquetPathsError struct {
SchemaID int16
// SourceConfigured reports whether a parquet source was consulted and
// returned nothing, as opposed to no source existing to consult.
SourceConfigured bool
}
NoParquetPathsError names the schema whose path set came back empty and which resolution level was in play, because the two states have different remedies: a consulted-but-empty source needs its manifest repaired (or the fallback prefix set), while an absent source needs configuring at all.
func (*NoParquetPathsError) Error ¶ added in v0.2.0
func (e *NoParquetPathsError) Error() string
func (*NoParquetPathsError) Unwrap ¶ added in v0.2.0
func (e *NoParquetPathsError) Unwrap() error
type OperationError ¶
type OperationError struct {
Operation EntityOperation `json:"operation"`
Error string `json:"error"`
Code string `json:"code"`
Details map[string]any `json:"details,omitempty"`
}
OperationError represents an error for a specific operation
type OperationType ¶
type OperationType string
OperationType represents CRUD operations
const ( OperationCreate OperationType = "create" OperationRead OperationType = "read" OperationUpdate OperationType = "update" OperationDelete OperationType = "delete" OperationQuery OperationType = "query" )
type Option ¶ added in v0.0.25
type Option func(*Config)
Option is a functional option that mutates a Config. Use the With* constructors below to build option values, then pass them to NewConfig to obtain a fully-configured Config without touching the struct fields directly.
func WithDatabase ¶ added in v0.0.25
func WithDatabase(db DatabaseConfig) Option
WithDatabase replaces the DatabaseConfig section.
func WithDuckDB ¶ added in v0.0.25
func WithDuckDB(d DuckDBConfig) Option
WithDuckDB replaces the DuckDBConfig section.
func WithEntity ¶ added in v0.0.25
func WithEntity(e EntityConfig) Option
WithEntity replaces the EntityConfig section.
func WithLogging ¶ added in v0.0.25
func WithLogging(l LoggingConfig) Option
WithLogging replaces the LoggingConfig section.
func WithMetrics ¶ added in v0.0.25
func WithMetrics(m MetricsConfig) Option
WithMetrics replaces the MetricsConfig section.
func WithPerformance ¶ added in v0.0.25
func WithPerformance(p PerformanceConfig) Option
WithPerformance replaces the PerformanceConfig section.
func WithQuery ¶ added in v0.0.25
func WithQuery(q QueryConfig) Option
WithQuery replaces the QueryConfig section.
func WithReference ¶ added in v0.0.25
func WithReference(r ReferenceConfig) Option
WithReference replaces the ReferenceConfig section.
func WithSchemaRegistry ¶ added in v0.0.25
func WithSchemaRegistry(sr SchemaRegistry) Option
WithSchemaRegistry sets the schema registry on the config.
func WithTransaction ¶ added in v0.0.25
func WithTransaction(t TransactionConfig) Option
WithTransaction replaces the TransactionConfig section.
type ParquetSetInconsistentError ¶ added in v0.2.0
ParquetSetInconsistentError carries the schema and the missing object keys so the message names the offending state, per the read-path error style.
MissingKeys holds bucket-relative S3 object keys — operator detail that must not cross a public transport. internal/httpapi redacts it from response bodies (#301). That redaction is gated on sentinel evidence, not on the status: this error wraps no client sentinel, so it is redacted whatever status it is classified as — including a 4xx, since DuckDB renders a missing object as "404 (Not Found)". Any new transport owes the same treatment.
func (*ParquetSetInconsistentError) Error ¶ added in v0.2.0
func (e *ParquetSetInconsistentError) Error() string
func (*ParquetSetInconsistentError) Unwrap ¶ added in v0.2.0
func (e *ParquetSetInconsistentError) Unwrap() error
type PartialResultInfo ¶ added in v0.2.0
type PartialResultInfo struct {
Reason string `json:"reason"`
ExcludedObjectCount int `json:"excluded_object_count,omitempty"`
}
PartialResultInfo marks a QueryResult whose page was answered from a deliberately reduced data surface. It is the sanctioned HTTP-visible partial signal (#348): the #251 corrupt-parquet exclusion note lives in internal plan Notes, which never cross the HTTP boundary (#301/#306), so without this field an API consumer sees fewer rows and a smaller total_records with no explanation. Reason is a closed enum, not free text, and the excluded objects are identified only by count — storage keys stay internal. It is deliberately NOT Routing.Reason: the route does not change on a partial read.
type PerformanceConfig ¶ added in v0.0.2
type PerformanceConfig struct {
EnableMonitoring bool `json:"enableMonitoring"`
SlowQueryThreshold time.Duration `json:"slowQueryThreshold"`
SlowOperationThreshold time.Duration `json:"slowOperationThreshold"`
MetricsCollectionInterval time.Duration `json:"metricsCollectionInterval"`
BatchSize int `json:"batchSize"`
MaxBatchSize int `json:"maxBatchSize"`
Batch BatchConfig `json:"batch"`
// Unified monitoring settings
MaxMetricsHistory int `json:"maxMetricsHistory"`
MaxAlertsHistory int `json:"maxAlertsHistory"`
MaxRecommendations int `json:"maxRecommendations"`
EnableAlerting bool `json:"enableAlerting"`
EnableRecommendations bool `json:"enableRecommendations"`
AlertingInterval time.Duration `json:"alertingInterval"`
RecommendationInterval time.Duration `json:"recommendationInterval"`
// Memory monitoring
EnableMemoryMonitoring bool `json:"enableMemoryMonitoring"`
MemoryThreshold int64 `json:"memoryThreshold"`
// Correlation tracking
EnableCorrelationTracking bool `json:"enableCorrelationTracking"`
CorrelationTTL time.Duration `json:"correlationTTL"`
}
PerformanceConfig contains performance monitoring settings
type PropertySchema ¶ added in v0.0.15
type PropertySchema struct {
Name string `json:"name"`
Type string `json:"type"` // "string", "integer", "number", "boolean", "array", "object", "null"
Format string `json:"format,omitempty"`
Items *PropertySchema `json:"items,omitempty"`
Properties map[string]*PropertySchema `json:"properties,omitempty"`
Required bool `json:"required"`
Default any `json:"default,omitempty"`
Enum []any `json:"enum,omitempty"`
Minimum *float64 `json:"minimum,omitempty"`
Maximum *float64 `json:"maximum,omitempty"`
MinLength *int `json:"minLength,omitempty"`
MaxLength *int `json:"maxLength,omitempty"`
Pattern string `json:"pattern,omitempty"`
Relation *RelationSchema `json:"x-relation,omitempty"`
LTBaseType string `json:"x-ltbase-type,omitempty"` // "virtual" for virtual fields that are populated dynamically
LTBaseNote string `json:"x-ltbase-note-prop,omitempty"` // Reference to note field: "${note_id}", "${owner_id}", "${note_data}"
}
PropertySchema defines the schema for a single property.
type PublicError ¶ added in v0.2.0
PublicError is implemented by an error that carries a message deliberately published to API clients.
type QueryConfig ¶ added in v0.0.2
type QueryConfig struct {
DefaultTimeout time.Duration `json:"defaultTimeout"`
MaxRows int `json:"maxRows"`
DefaultPageSize int `json:"defaultPageSize"`
MaxPageSize int `json:"maxPageSize"`
EnableQueryPlan bool `json:"enableQueryPlan"`
EnableOptimization bool `json:"enableOptimization"`
CacheQueryPlans bool `json:"cacheQueryPlans"`
QueryPlanCacheTTL time.Duration `json:"queryPlanCacheTTL"`
}
QueryConfig contains query execution settings
type QueryRequest ¶
type QueryRequest struct {
SchemaName string `json:"schema_name" validate:"required"`
Page int `json:"page" validate:"min=1"`
ItemsPerPage int `json:"items_per_page" validate:"min=1,max=100"`
Condition Condition `json:"-"` // Custom unmarshal, can be CompositeCondition or KvCondition
SortBy []string `json:"sort_by,omitempty"`
SortOrder SortOrder `json:"sort_order,omitempty"`
// Sort carries per-key sort directions (#240). Mutually exclusive with
// SortBy/SortOrder; an entry's empty SortOrder defaults to asc.
Sort []OrderBy `json:"sort,omitempty"`
RowID *uuid.UUID `json:"row_id,omitempty"` // For entity-specific operations
Attrs []string `json:"attrs,omitempty"` // Attributes to return (field projection)
Federated *FederatedQueryRequest `json:"federated,omitempty"`
}
QueryRequest represents a pagination query request.
func (QueryRequest) MarshalJSON ¶
func (r QueryRequest) MarshalJSON() ([]byte, error)
MarshalJSON implements custom JSON marshaling for QueryRequest.
func (*QueryRequest) UnmarshalJSON ¶
func (r *QueryRequest) UnmarshalJSON(data []byte) error
UnmarshalJSON implements custom JSON unmarshaling for QueryRequest. It allows the Condition field to be either a CompositeCondition or KvCondition.
type QueryResult ¶
type QueryResult struct {
Data []*DataRecord `json:"data"`
TotalRecords int `json:"total_records"`
TotalPages int `json:"total_pages"`
CurrentPage int `json:"current_page"`
ItemsPerPage int `json:"items_per_page"`
HasNext bool `json:"has_next"`
HasPrevious bool `json:"has_previous"`
ExecutionTime time.Duration `json:"execution_time"`
// ExecutionPlan is populated only for federated requests that set
// federated.include_execution_plan; it reports the route the engine
// actually took (DuckDB vs Postgres-only) and per-tier sources so callers
// can distinguish federated reads from hot-path reads without guessing.
ExecutionPlan *ExecutionPlan `json:"execution_plan,omitempty"`
// Partial marks a response answered from an incomplete data surface
// (#348); currently the only reason is the #251 corrupt-parquet
// exclusion. Nil/omitted for complete answers. Unlike ExecutionPlan it
// does not require federated.include_execution_plan.
Partial *PartialResultInfo `json:"partial,omitempty"`
}
QueryResult represents paginated query results.
type Reference ¶
type Reference struct {
SourceSchemaName string `json:"sourceSchemaName"`
SourceRowID uuid.UUID `json:"sourceRowId"`
SourceFieldName string `json:"sourceFieldName"`
TargetSchemaName string `json:"targetSchemaName"`
TargetRowID uuid.UUID `json:"targetRowId"`
ReferenceType ReferenceType `json:"referenceType"`
}
Reference represents a reference from one entity to another
type ReferenceConfig ¶ added in v0.0.2
type ReferenceConfig struct {
ValidateOnCreate bool `json:"validateOnCreate"`
ValidateOnUpdate bool `json:"validateOnUpdate"`
CheckIntegrity bool `json:"checkIntegrity"`
CascadeDelete bool `json:"cascadeDelete"`
CascadeUpdate bool `json:"cascadeUpdate"`
MaxCascadeDepth int `json:"maxCascadeDepth"`
CascadeRules map[string]CascadeRule `json:"cascadeRules,omitempty"`
EnableCaching bool `json:"enableCaching"`
CacheTTL time.Duration `json:"cacheTTL"`
MaxCacheSize int `json:"maxCacheSize"`
BatchSize int `json:"batchSize"`
}
ReferenceConfig contains reference management settings
type ReferenceType ¶
type ReferenceType string
ReferenceType represents the type of reference
const ( ReferenceTypeSingle ReferenceType = "single" ReferenceTypeArray ReferenceType = "array" ReferenceTypeNested ReferenceType = "nested" )
type RelationSchema ¶ added in v0.0.15
type RelationSchema struct {
Target string `json:"target"` // Target schema name
Type string `json:"type"` // "reference" for foreign key relationships
KeyProperty string `json:"key_property"` // child-side foreign key attribute
}
RelationSchema defines reference relationships between objects.
type RequiredPolicy ¶ added in v0.0.27
type RequiredPolicy string
RequiredPolicy defines when an attribute is required.
const ( RequiredPolicyOptional RequiredPolicy = "optional" RequiredPolicyAlways RequiredPolicy = "required_always" RequiredPolicyIfParentPresent RequiredPolicy = "required_if_parent_present" )
type RoutingPolicy ¶ added in v0.0.23
type RoutingPolicy struct {
Strategy RoutingStrategy `json:"strategy"` // "freshness-first", "cost-first", "hybrid"
HotTTL time.Duration `json:"hotTTL"` // TTL to consider data "hot"
MaxDuckDBScanRows int `json:"maxDuckDBScanRows"` // threshold for preferring cold scans
AllowS3Fallback bool `json:"allowS3Fallback"` // allow falling back to S3/DuckDB when PG not used
}
RoutingPolicy defines federated query routing behavior
type RoutingStrategy ¶ added in v0.0.25
type RoutingStrategy string
RoutingStrategy specifies the federated query routing algorithm.
const ( // RoutingStrategyFreshnessFirst prefers the hot (PostgreSQL) tier for // queries that explicitly request fresh data (PreferHot flag). RoutingStrategyFreshnessFirst RoutingStrategy = "freshness-first" // RoutingStrategyCostFirst routes large scans to DuckDB to reduce // PostgreSQL load; small scans stay on the hot tier. RoutingStrategyCostFirst RoutingStrategy = "cost-first" // RoutingStrategyHybrid uses DuckDB by default but short-circuits to // PostgreSQL when the result set is expected to be small or hot data is // explicitly preferred. RoutingStrategyHybrid RoutingStrategy = "hybrid" )
type SchemaAttributeCache ¶ added in v0.0.3
type SchemaAttributeCache map[string]AttributeMetadata
SchemaAttributeCache is a mapping of attr_name -> metadata. Strongly recommended to populate per schema_id at application startup.
type SchemaRegistry ¶ added in v0.0.3
type SchemaRegistry interface {
// GetSchemaAttributeCacheByName retrieves schema ID and attribute cache by schema name
GetSchemaAttributeCacheByName(name string) (int16, SchemaAttributeCache, error)
// GetSchemaAttributeCacheByID retrieves schema name and attribute cache by schema ID
GetSchemaAttributeCacheByID(id int16) (string, SchemaAttributeCache, error)
GetSchemaByName(name string) (int16, JSONSchema, error)
// GetSchemaAttributeCacheByID retrieves schema name and attribute cache by schema ID
GetSchemaByID(id int16) (string, JSONSchema, error)
ListSchemas() []string
}
SchemaRegistry provides schema lookup operations. Implementations can load schemas from files, databases, or other sources.
type TableNames ¶ added in v0.0.2
type TableNames struct {
SchemaRegistry string `json:"schemaRegistry"`
EntityMain string `json:"entityMain"`
EAVData string `json:"eavData"`
ChangeLog string `json:"changeLog"`
}
TableNames generates the table names for a specific client and project
type TransactionConfig ¶ added in v0.0.2
type TransactionConfig struct {
DefaultTimeout time.Duration `json:"defaultTimeout"`
MaxTimeout time.Duration `json:"maxTimeout"`
MaxRetryAttempts int `json:"maxRetryAttempts"`
RetryAttempts int `json:"retryAttempts"`
RetryDelay time.Duration `json:"retryDelay"`
IsolationLevel string `json:"isolationLevel"`
EnableDeadlockDetection bool `json:"enableDeadlockDetection"`
DeadlockCheckInterval time.Duration `json:"deadlockCheckInterval"`
DeadlockMaxWaitTime time.Duration `json:"deadlockMaxWaitTime"`
SlowTransactionThreshold time.Duration `json:"slowTransactionThreshold"`
MinSuccessRate float64 `json:"minSuccessRate"`
MaxAverageDuration time.Duration `json:"maxAverageDuration"`
MaxConnectionPoolUsage float64 `json:"maxConnectionPoolUsage"`
}
TransactionConfig contains transaction settings
type ValueType ¶ added in v0.0.3
type ValueType string
ValueType represents supported attribute value types.
const ( ValueTypeText ValueType = "text" ValueTypeSmallInt ValueType = "smallint" ValueTypeInteger ValueType = "integer" ValueTypeBigInt ValueType = "bigint" ValueTypeNumeric ValueType = "numeric" // double precision ValueTypeDate ValueType = "date" // for JSON attributes with format `date` ValueTypeDateTime ValueType = "datetime" // for JSON attributes with format `date-time` ValueTypeUUID ValueType = "uuid" ValueTypeBool ValueType = "bool" ValueTypeList ValueType = "list" // array type stored as DuckDB LIST in parquet )
Source Files
¶
Directories
¶
| Path | Synopsis |
|---|---|
|
cmd
|
|
|
benchmark
command
|
|
|
lambda
command
Package main provides the AWS Lambda entry point for the Forma API server.
|
Package main provides the AWS Lambda entry point for the Forma API server. |
|
sample
command
|
|
|
server
command
|
|
|
tools
command
|
|
|
duckdbinit
Package duckdbinit builds and applies the session-scoped initialization (INSTALL/LOAD/SET/PRAGMA) that every pooled DuckDB connection must run on open.
|
Package duckdbinit builds and applies the session-scoped initialization (INSTALL/LOAD/SET/PRAGMA) that every pooled DuckDB connection must run on open. |
|
e2e_harness/federated
Package federated provides custom assertions for E2E testing.
|
Package federated provides custom assertions for E2E testing. |
|
e2e_harness/production
Package production is the reusable E2E test harness that exercises the REAL production stack end to end (#173, epic #172):
|
Package production is the reusable E2E test harness that exercises the REAL production stack end to end (#173, epic #172): |
|
parquetcheck
Package parquetcheck defines the parquet export schema invariant shared by every Forma parquet consumer: the three system columns each generation carries regardless of attribute evolution, with the exact DuckDB types both exporters emit (delta flush and init/compaction base).
|
Package parquetcheck defines the parquet export schema invariant shared by every Forma parquet consumer: the three system columns each generation carries regardless of attribute evolution, with the exact DuckDB types both exporters emit (delta flush and init/compaction base). |
|
pgdsn
Package pgdsn builds libpq keyword/value connection strings.
|
Package pgdsn builds libpq keyword/value connection strings. |
|
queryplan
Package queryplan provides the plan-cache primitives for #142: a stable query-shape fingerprint, a composite cache key, and a concurrency-safe cache for compiled planning artifacts.
|
Package queryplan provides the plan-cache primitives for #142: a stable query-shape fingerprint, a composite cache key, and a concurrency-safe cache for compiled planning artifacts. |
|
reconcile
Package reconcile diffs a schema's S3 parquet objects against its manifest (issue #203).
|
Package reconcile diffs a schema's S3 parquet objects against its manifest (issue #203). |
|
redact
Package redact removes credential material from strings before they leave the process — into a log sink, or into an HTTP response body.
|
Package redact removes credential material from strings before they leave the process — into a log sink, or into an HTTP response body. |
|
schemavalidate
Package schemavalidate resolves entity JSON Schemas once and validates write payloads against them.
|
Package schemavalidate resolves entity JSON Schemas once and validates write payloads against them. |
|
sqlgen/sqlgentest
Package sqlgentest provides shared helpers for tests that pin the postgres_scan contract on both sides: the runtime template (internal/sqlgen) and the executable §5 sketch in docs/federated-query/design.md (internal/federated).
|
Package sqlgentest provides shared helpers for tests that pin the postgres_scan contract on both sides: the runtime template (internal/sqlgen) and the executable §5 sketch in docs/federated-query/design.md (internal/federated). |