Documentation
¶
Overview ¶
Package recordexport converts the Base OpenAPI record matrix into a stable, typed row model that output formats can share. The matrix is intentionally parsed only once at the package boundary; exporters never depend on loose map keys or parallel arrays.
Index ¶
- Constants
- func WriteManifest(w io.Writer, manifest Manifest) error
- func WriteNDJSON(w io.Writer, dataset Dataset) error
- type Column
- type ColumnManifest
- type ColumnStats
- type Dataset
- type IgnoredField
- type Manifest
- type ManifestOptions
- type MatrixError
- type MinimalManifest
- type Page
- type Record
- type SchemaChangedError
- type ValueKind
Constants ¶
const ( ManifestVersion = "v1" FormatNDJSON = "ndjson" )
const RecordIDColumnName = "record_id"
Variables ¶
This section is empty.
Functions ¶
func WriteManifest ¶
WriteManifest writes deterministic indented JSON without HTML escaping.
Types ¶
type Column ¶
Column is the format-neutral schema used by all record exporters. FieldID and FieldType are empty only for the synthetic record_id system column.
func (Column) PhysicalType ¶
PhysicalType renders the compact, engine-neutral type used in manifests.
type ColumnManifest ¶
type ColumnManifest struct {
FieldID string `json:"field_id,omitempty"`
FieldType string `json:"field_type,omitempty"`
PhysicalType string `json:"physical_type"`
Stats ColumnStats `json:"stats"`
Example any `json:"example,omitempty"`
ExampleTruncated bool `json:"example_truncated,omitempty"`
Hint string `json:"hint,omitempty"`
}
ColumnManifest is the user-facing description of one actual data column.
type ColumnStats ¶
type ColumnStats struct {
NullCount *int `json:"null_count,omitempty"`
EmptyCount *int `json:"empty_count,omitempty"`
TrueCount *int `json:"true_count,omitempty"`
Min any `json:"min,omitempty"`
Max any `json:"max,omitempty"`
Avg *float64 `json:"avg,omitempty"`
MaxLength *int `json:"max_length,omitempty"`
AvgLength *float64 `json:"avg_length,omitempty"`
}
ColumnStats contains only the metrics that are meaningful for a column's Base field type. All metrics describe the records in this export.
type Dataset ¶
Dataset is the stable tabular model consumed by NDJSON now and by future JSON-array or Parquet exporters. SourceColumns retains the complete OpenAPI schema for cross-page consistency checks, while Columns contains the actual exported columns (including the synthetic record_id column).
func (*Dataset) AppendPage ¶
AppendPage appends rows only after proving that the complete source schema and timezone still match the first page.
type IgnoredField ¶
type IgnoredField struct {
ID string `json:"id"`
Name string `json:"name"`
Reason string `json:"reason"`
}
IgnoredField mirrors the structured OpenAPI read warning.
type Manifest ¶
type Manifest struct {
ManifestVersion string `json:"manifest_version"`
Format string `json:"format"`
BaseToken string `json:"base_token"`
TableID string `json:"table_id"`
Rev *int64 `json:"rev,omitempty"`
Timezone string `json:"timezone"`
QueryContext map[string]any `json:"query_context,omitempty"`
Offset int `json:"offset,omitempty"`
RequestedLimit int `json:"requested_limit,omitempty"`
RecordsCount int `json:"records_count"`
PageCount int `json:"page_count"`
HasMore bool `json:"has_more"`
NextOffset *int `json:"next_offset,omitempty"`
RecordFile string `json:"record_file"`
RecordFileSizeBytes int64 `json:"record_file_size_bytes"`
ManifestFile string `json:"manifest_file"`
Columns map[string]ColumnManifest `json:"columns"`
IgnoredFields []IgnoredField `json:"ignored_fields,omitempty"`
RecordNotFound []string `json:"record_not_found,omitempty"`
}
Manifest describes one exported artifact and the exact query boundary that produced it. Format is explicit so the same structure can describe future JSON-array and Parquet outputs.
func BuildManifest ¶
func BuildManifest(dataset Dataset, opts ManifestOptions) Manifest
func (Manifest) Minimal ¶
func (m Manifest) Minimal() MinimalManifest
type ManifestOptions ¶
type ManifestOptions struct {
BaseToken string
TableID string
Rev *int64
QueryContext map[string]any
Offset int
RequestedLimit int
PageCount int
HasMore bool
RecordFile string
RecordFileSizeBytes int64
ManifestFile string
IgnoredFields []IgnoredField
RecordNotFound []string
}
ManifestOptions carries query and file details that are outside Dataset.
type MatrixError ¶
type MatrixError struct {
Reason string
}
MatrixError means the OpenAPI response does not satisfy its parallel-array contract. The command boundary wraps it as a typed invalid-response error.
func (*MatrixError) Error ¶
func (e *MatrixError) Error() string
type MinimalManifest ¶
type MinimalManifest struct {
RecordFile string `json:"record_file"`
RecordFileSizeBytes int64 `json:"record_file_size_bytes"`
ManifestFile string `json:"manifest_file"`
RecordsCount int `json:"records_count"`
HasMore bool `json:"has_more"`
}
MinimalManifest is the stable low-token stdout result.
type Page ¶
type Page struct {
Dataset Dataset
Rev *int64
HasMore bool
IgnoredFields []IgnoredField
QueryContext map[string]any
RecordNotFound []string
}
Page contains one parsed matrix page and its query metadata.
type Record ¶
type Record struct {
Values []any
}
Record stores decoded JSON values in the same order as Dataset.Columns. Object cells remain map[string]any and are not converted into Go structs.
type SchemaChangedError ¶
type SchemaChangedError struct {
Reason string
}
SchemaChangedError protects a multi-page export from mixing schemas.
func (*SchemaChangedError) Error ¶
func (e *SchemaChangedError) Error() string
Source Files