spanvalue

package module
v0.4.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: May 28, 2026 License: MIT Imports: 16 Imported by: 1

README

spanvalue

Go Reference

Helpers for working with Cloud Spanner’s spanner.GenericColumnValue and related client types: format values to text (literals, JSON, CLI-style output) and construct values from Go types.

Package Role
github.com/apstndb/spanvalue Format spanner.GenericColumnValue and *spanner.Row using FormatConfig and presets such as LiteralFormatConfig, JSONFormatConfig, SpannerCLICompatibleFormatConfig.
github.com/apstndb/spanvalue/gcvctor Build spanner.GenericColumnValue (scalars, ARRAY, STRUCT, typed nulls). Types are often composed with github.com/apstndb/spantype/typector.
github.com/apstndb/spanvalue/writer Stream Spanner rows to CSV, TSV, JSONL, or SQL INSERT statements using spanvalue formatters.

Identifier quoting helpers

QuoteIdentifier and QuoteQualifiedIdentifier are conservative quoting helpers. They always quote for the selected dialect, escape embedded quote characters, and do not attempt a minimal "quote only when necessary" strategy.

  • DATABASE_DIALECT_UNSPECIFIED follows the Spanner default and uses GoogleSQL quoting.
  • QuoteQualifiedIdentifier quotes each dotted path segment independently.
  • The helpers do not validate empty identifiers or empty path segments; callers that reject those shapes must do so before calling them.
quotedTable := spanvalue.QuoteQualifiedIdentifier(
    databasepb.DatabaseDialect_GOOGLE_STANDARD_SQL,
    "analytics.daily_metrics",
)
quotedColumn := spanvalue.QuoteIdentifier(
    databasepb.DatabaseDialect_GOOGLE_STANDARD_SQL,
    "select",
)
// quotedTable == "`analytics`.`daily_metrics`"
// quotedColumn == "`select`"

Adoption snippets

Use the small helper APIs directly when replacing ad hoc downstream formatting code:

jsonLine, err := spanvalue.FormatRowJSONObjectFromColumns(
    spanvalue.JSONFormatConfig(),
    columnNames,
    gcvs,
    spanvalue.IndexedUnnamedFieldNamer,
)
w := writer.NewSQLInsertWriter(out, "analytics.daily_metrics")
if err := w.WriteValues(columnNames, gcvs); err != nil {
    return err
}
return w.Flush()

Streaming row exports

The writer package accepts *spanner.Row values directly through WriteRow. Use writer.Writer when an adapter only needs row streaming. Use writer.FlushWriter when an adapter owns both row streaming and finalization. DelimitedWriter and JSONLWriter preserve explicit duplicate column names. Empty column names are the only names passed to UnnamedFieldNamer, and generated names avoid collisions with existing explicit names. Set UnnamedFieldNamer to nil when callers need empty names to remain empty.

Call Flush after the final row when using writer.FlushWriter; see the Writer, FlushWriter, and Flusher godoc for the interface lifecycle contract.

Constructors accept options when setup should be explicit:

w := writer.NewDelimitedWriter(
	out,
	'\t',
	writer.WithMetadata(meta),
	writer.WithFormatter(cfg),
	writer.WithHeader(true),
	writer.WithUnnamedFieldNamer(nil),
)

When metadata is known after construction but before rows are streamed, call Prepare(metadata) on the concrete writer. For non-streaming paths, use writer.RowData, writer.FormatDelimitedRow, or writer.FormatJSONLRow directly. Pass the JSON field-name policy explicitly, for example:

line, err := writer.FormatJSONLRow(
	spanvalue.JSONFormatConfig(),
	row,
	spanvalue.IndexedUnnamedFieldNamer,
)

CSV output:

func writeCSV(out io.Writer, rows []*spanner.Row) error {
	w := writer.NewCSVWriter(out)
	for _, row := range rows {
		if err := w.WriteRow(row); err != nil {
			return err
		}
	}
	return w.Flush()
}

TSV output uses the same CSV-style writer with a tab delimiter. NewCSVWriter is a thin helper for NewDelimitedWriter(out, writer.Comma). Pass writer.Comma when using the generic delimited constructor for CSV output. Delimiters must be non-zero valid runes other than ", \r, \n, or utf8.RuneError.

func writeTSV(out io.Writer, rows []*spanner.Row) error {
	w := writer.NewDelimitedWriter(out, '\t')
	for _, row := range rows {
		if err := w.WriteRow(row); err != nil {
			return err
		}
	}
	return w.Flush()
}

JSONL output:

func writeJSONL(out io.Writer, rows []*spanner.Row) error {
	w := writer.NewJSONLWriter(out)
	for _, row := range rows {
		if err := w.WriteRow(row); err != nil {
			return err
		}
	}
	return w.Flush()
}

SQL INSERT output uses Spanner GoogleSQL quoting. Use writer.WithSQLInsertKind for INSERT OR IGNORE or INSERT OR UPDATE; see INSERT DML syntax.

func writeInserts(out io.Writer, table string, rows []*spanner.Row) error {
	w := writer.NewSQLInsertWriter(out, table)
	for _, row := range rows {
		if err := w.WriteRow(row); err != nil {
			return err
		}
	}
	return w.Flush()
}

Integration tests that exercise the Spanner client with PostgreSQL dialect (TypeAnnotation on query params and row metadata) are maintained in github.com/apstndb/spanpg (integration/pgtypeannotation), not in this repository.

Documentation

Overview

Package spanvalue formats Cloud Spanner data from the Go client (cloud.google.com/go/spanner): cloud.google.com/go/spanner.GenericColumnValue values for individual columns and `*spanner.Row` values for full rows (cloud.google.com/go/spanner.Row), into strings for SQL literals, JSON, Spanner CLI–compatible text, and related styles.

Primary API

Configure output with FormatConfig. Use the constructors LiteralFormatConfig, SimpleFormatConfig, SpannerCLICompatibleFormatConfig, and JSONFormatConfig to pick a preset. Customize a preset with FormatConfig.Clone. FormatConfig.FormatColumn runs FormatComplexFunc plugins first, then built-in ARRAY, STRUCT, and scalar formatting. Convenience entry points include FormatRowLiteral, FormatColumnLiteral, FormatRowJSONObject, and FormatRowSpannerCLICompatible. Identifier quoting helpers are QuoteIdentifier and QuoteQualifiedIdentifier.

Advanced extension API

Lower-level callbacks and plugin types are intended for custom output formats: FormatArrayFunc, FormatStructFieldFunc, FormatStructParenFunc, FormatComplexFunc, ErrFallthrough, FormatStruct, FormatTupleStruct, TypedStructFormat, and JSONObjectStructFormat. Copy a preset with FormatConfig.Clone before changing callbacks. Convenience formatters such as FormatRowSpannerCLICompatible use internal singleton configs; call FormatConfig.Clone on a preset from LiteralFormatConfig, SimpleFormatConfig, or the other constructors before changing callbacks.

To build cloud.google.com/go/spanner.GenericColumnValue values from Go types, see github.com/apstndb/spanvalue/gcvctor. For streaming row export, see github.com/apstndb/spanvalue/writer.

Index

Examples

Constants

This section is empty.

Variables

View Source
var (
	ErrNilRow                     = errors.New("nil row")
	ErrNilStructField             = errors.New("nil struct field descriptor")
	ErrUnknownType                = errors.New("unknown type")
	ErrMismatchedFields           = errors.New("mismatched struct value/field count")
	ErrUnexpectedComplexValueKind = errors.New("unexpected complex value kind")
	ErrEmptyTypeFQN               = errors.New("empty type FQN")
)
View Source
var ErrFallthrough = errors.New("fallthrough")

Functions

func ColumnNames added in v0.1.10

func ColumnNames(fields []*sppb.StructType_Field, namer UnnamedFieldNamer) ([]string, error)

ColumnNames returns the names of the provided fields. Unnamed fields are kept as empty strings unless a non-nil namer is provided, in which case the namer is used to generate names for unnamed fields. If a non-nil UnnamedFieldNamer returns an empty string or repeatedly returns colliding names such that a unique column name cannot be chosen, ColumnNames returns a non-nil error describing the contract violation.

func FormatBracketStruct

func FormatBracketStruct(typ *sppb.Type, toplevel bool, fieldStrings []string) (string, error)

func FormatColumnLiteral

func FormatColumnLiteral(value spanner.GenericColumnValue) (string, error)

func FormatColumnSpannerCLICompatible

func FormatColumnSpannerCLICompatible(value spanner.GenericColumnValue) (string, error)

func FormatCompactArray added in v0.1.9

func FormatCompactArray(_ *sppb.Type, _ bool, elemStrings []string) (string, error)

FormatCompactArray formats array elements without spaces between separators. Output: [elem1,elem2,elem3]

func FormatEnumAsCast

func FormatEnumAsCast(formatter Formatter, value spanner.GenericColumnValue, toplevel bool) (string, error)

func FormatJSONSimpleValue added in v0.1.9

func FormatJSONSimpleValue(formatter Formatter, value spanner.GenericColumnValue, _ bool) (string, error)

FormatJSONSimpleValue is a FormatComplexFunc that formats non-ARRAY, non-STRUCT types as valid JSON values. It returns ErrFallthrough for ARRAY and STRUCT so that the built-in handlers format them.

For most types, structpb.Value.MarshalJSON() produces the correct JSON representation (BOOL→true/false, FLOAT→number, STRING→"quoted", NULL→null, NaN/Inf→"NaN"/"Infinity"). Only INT64, ENUM, and JSON (including PostgreSQL PG_JSONB-annotated JSON) columns need special handling:

  • INT64: Spanner encodes as StringValue("42"), MarshalJSON() would produce "42" (quoted), but we want 42 (unquoted number).
  • ENUM: Spanner stores proto enum values as INT64; same handling as INT64.
  • JSON / PG_JSONB: Spanner encodes as StringValue('{"key":"value"}'), MarshalJSON() would produce escaped quoted string, but we want the raw JSON value passed through.

func FormatNullableSpannerCLICompatible

func FormatNullableSpannerCLICompatible(value NullableValue) (string, error)

func FormatOptionallyTypedArray

func FormatOptionallyTypedArray(typ *sppb.Type, toplevel bool, elemStrings []string) (string, error)

func FormatProtoAsCast

func FormatProtoAsCast(formatter Formatter, value spanner.GenericColumnValue, toplevel bool) (string, error)

func FormatRowColumns added in v0.1.10

func FormatRowColumns(fc *FormatConfig, columnNames []string, values []spanner.GenericColumnValue) ([]string, error)

FormatRowColumns formats a row represented as column names plus GCV values. The column names are validated for shape compatibility, but the formatted cell values come from the GCVs themselves.

func FormatRowJSONObject added in v0.1.9

func FormatRowJSONObject(fc *FormatConfig, row *spanner.Row, namer UnnamedFieldNamer) (string, error)

FormatRowJSONObject formats a spanner.Row as a single JSON object string using the given FormatConfig for value formatting and column names as keys. The FormatConfig must produce standalone JSON values per column (e.g., JSONFormatConfig()). Using a non-JSON config produces syntactically invalid output. Empty column names (e.g., from expressions without aliases like SELECT 1+1) are assigned names by the provided namer function. If namer is nil, empty names are kept as empty-string JSON keys. Returns an error if the namer returns an empty name, or if repeated name collisions prevent choosing a unique name for a field. Output: {"col1":val1,"col2":val2,...}

func FormatRowJSONObjectFromColumns added in v0.1.10

func FormatRowJSONObjectFromColumns(fc *FormatConfig, columnNames []string, values []spanner.GenericColumnValue, namer UnnamedFieldNamer) (string, error)

FormatRowJSONObjectFromColumns formats a row represented as column names plus GCV values into a JSON object string. The provided FormatConfig must emit standalone JSON values per column (for example, as configured by JSONFormatConfig()), otherwise the assembled object may be syntactically invalid JSON.

func FormatRowLiteral

func FormatRowLiteral(value *spanner.Row) ([]string, error)

func FormatRowSpannerCLICompatible

func FormatRowSpannerCLICompatible(row *spanner.Row) ([]string, error)

func FormatSimpleStructField

func FormatSimpleStructField(fc *FormatConfig, field *sppb.StructType_Field, value *structpb.Value) (string, error)

func FormatTupleStruct

func FormatTupleStruct(typ *sppb.Type, toplevel bool, fieldStrings []string) (string, error)

func FormatTypelessStructField

func FormatTypelessStructField(fc *FormatConfig, field *sppb.StructType_Field, value *structpb.Value) (string, error)

func FormatUntypedArray

func FormatUntypedArray(_ *sppb.Type, _ bool, elemStrings []string) (string, error)

func IndexedUnnamedFieldNamer added in v0.1.9

func IndexedUnnamedFieldNamer(index int) string

IndexedUnnamedFieldNamer produces names like "_0", "_1", etc. The underscore prefix minimizes collision with user-defined names. Suitable for row columns (e.g., SELECT 1+1 produces "_0").

func IsNull added in v0.1.10

func IsNull(gcv spanner.GenericColumnValue) bool

IsNull reports whether gcv represents a NULL value. A nil gcv.Value is treated as NULL.

func QuoteIdentifier added in v0.3.0

func QuoteIdentifier(dialect databasepb.DatabaseDialect, name string) string

QuoteIdentifier quotes a single identifier for dialect. It does not validate the identifier; for example, an empty string becomes an empty quoted identifier. DATABASE_DIALECT_UNSPECIFIED follows the Spanner default and uses GoogleSQL quoting.

func QuoteQualifiedIdentifier added in v0.3.0

func QuoteQualifiedIdentifier(dialect databasepb.DatabaseDialect, name string) string

QuoteQualifiedIdentifier quotes each segment of a dotted identifier path for dialect. It does not validate the path; callers that reject empty segments must do so before calling it.

Types

type FormatArrayFunc

type FormatArrayFunc func(typ *sppb.Type, toplevel bool, elemStrings []string) (string, error)

type FormatComplexFunc

type FormatComplexFunc = func(formatter Formatter, value spanner.GenericColumnValue, toplevel bool) (string, error)

FormatComplexFunc is a function to format spanner.GenericColumnValue. If it returns ErrFallthrough, value will pass through to next step.

type FormatConfig

type FormatConfig struct {
	NullString           string
	FormatArray          FormatArrayFunc
	FormatStruct         FormatStruct
	FormatComplexPlugins []FormatComplexFunc
	FormatNullable       FormatNullableFunc
}

FormatConfig controls how Spanner values are formatted. Preset constructors such as LiteralFormatConfig return a fresh instance with non-nil callbacks for the value kinds they support. FormatConfig.FormatColumn calls FormatArray, FormatStruct, and FormatNullable directly; a nil callback panics when that branch runs unless a FormatComplexFunc plugin handles the value first.

Nil field behavior:

  • FormatArray: required for non-NULL ARRAY values. NULL ARRAY values use FormatConfig.GetNullString before FormatArray is called.
  • FormatStruct.FormatStructField and FormatStruct.FormatStructParen: required for non-NULL STRUCT values. NULL STRUCT values use FormatConfig.GetNullString before struct callbacks run.
  • FormatComplexPlugins: nil or empty means no plugins run.
  • FormatNullable: required for non-NULL scalars reached through FormatConfig.FormatColumn. NULL scalars use FormatConfig.GetNullString before FormatNullable is called.

Use FormatConfig.Clone to customize a preset without mutating shared instances.

func JSONFormatConfig added in v0.1.9

func JSONFormatConfig() *FormatConfig

JSONFormatConfig returns a new FormatConfig that produces valid JSON value strings for each Spanner value. Each call returns a fresh instance that the caller may customize.

Each formatted string is a standalone JSON value:

  • NULL → null
  • BOOL → true / false
  • INT64 → 42 (unquoted number)
  • FLOAT32/FLOAT64 → 3.14 (NaN/Inf as quoted strings)
  • ENUM → 42 (unquoted number, Spanner stores proto enum values as INT64)
  • STRING, BYTES, TIMESTAMP, DATE, NUMERIC, PROTO, INTERVAL, UUID → "quoted string"
  • JSON column → raw JSON value (passed through)
  • ARRAY → [elem1,elem2,...]
  • STRUCT → {"field1":val1,"field2":val2,...}

func LiteralFormatConfig

func LiteralFormatConfig() *FormatConfig

LiteralFormatConfig returns a new FormatConfig that produces parseable SQL literal expressions with type annotations.

func SimpleFormatConfig

func SimpleFormatConfig() *FormatConfig

SimpleFormatConfig returns a new FormatConfig that produces human-readable output using client library conventions.

func SpannerCLICompatibleFormatConfig

func SpannerCLICompatibleFormatConfig() *FormatConfig

SpannerCLICompatibleFormatConfig returns a new FormatConfig that matches the output format of spanner-cli.

Example (TupleStruct)
package main

import (
	"fmt"

	"cloud.google.com/go/spanner"

	"github.com/apstndb/spanvalue"
	"github.com/apstndb/spanvalue/gcvctor"
)

func main() {
	fc := spanvalue.SpannerCLICompatibleFormatConfig().Clone()
	fc.FormatStruct.FormatStructParen = spanvalue.FormatTupleStruct

	structElem, err := gcvctor.StructValueOf(
		[]string{"id", "region"},
		[]spanner.GenericColumnValue{gcvctor.Int64Value(1), gcvctor.StringValue("east")},
	)
	if err != nil {
		panic(err)
	}
	arrayOfStruct, err := gcvctor.ArrayValue(structElem)
	if err != nil {
		panic(err)
	}

	out, err := fc.FormatToplevelColumn(arrayOfStruct)
	if err != nil {
		panic(err)
	}
	fmt.Println(out)
}
Output:
[(1, east)]

func (*FormatConfig) Clone added in v0.3.2

func (fc *FormatConfig) Clone() *FormatConfig

Clone returns a shallow copy of fc with a copied FormatComplexPlugins slice. The returned config is independent for field assignment and plugin list mutation; callback values themselves are shared with the source. Clone returns nil when fc is nil.

func (*FormatConfig) FormatColumn

func (fc *FormatConfig) FormatColumn(value spanner.GenericColumnValue, toplevel bool) (string, error)

func (*FormatConfig) FormatRow

func (fc *FormatConfig) FormatRow(row *spanner.Row) ([]string, error)

func (*FormatConfig) FormatToplevelColumn

func (fc *FormatConfig) FormatToplevelColumn(value spanner.GenericColumnValue) (string, error)

func (*FormatConfig) GetNullString added in v0.1.10

func (fc *FormatConfig) GetNullString() string

type FormatNullableFunc

type FormatNullableFunc = func(value NullableValue) (string, error)

type FormatStruct

type FormatStruct struct {
	FormatStructField FormatStructFieldFunc
	FormatStructParen FormatStructParenFunc
}

func TypedStructFormat added in v0.4.0

func TypedStructFormat() FormatStruct

TypedStructFormat returns a FormatStruct that formats STRUCT values with typed field names in literal style.

type FormatStructFieldFunc

type FormatStructFieldFunc func(fc *FormatConfig, field *sppb.StructType_Field, value *structpb.Value) (string, error)

type FormatStructParenFunc

type FormatStructParenFunc func(typ *sppb.Type, toplevel bool, fieldStrings []string) (string, error)

func JSONObjectStructFormat added in v0.4.0

func JSONObjectStructFormat(namer UnnamedFieldNamer) FormatStructParenFunc

JSONObjectStructFormat returns a FormatStructParenFunc that formats struct fields as a JSON object. When namer is nil, unnamed struct fields produce empty-string keys, matching Spanner's own representation.

func NewJSONObjectStructFormatter added in v0.1.9

func NewJSONObjectStructFormatter(namer UnnamedFieldNamer) FormatStructParenFunc

NewJSONObjectStructFormatter creates a FormatStructParenFunc that formats struct fields as a JSON object with field names as keys. Unnamed fields are assigned names by the provided namer function. If namer is nil, unnamed fields keep empty-string keys (which produces duplicate keys — valid per RFC 8259 but may cause issues with parsers that deduplicate keys). Returns an error if the namer returns an empty name, or if repeated name collisions prevent choosing a unique name for a field. Output: {"field1":val1,"field2":val2,...}

type Formatter

type Formatter interface {
	FormatColumn(value spanner.GenericColumnValue, toplevel bool) (string, error)
	GetNullString() string
}

type NullBytes

type NullBytes []byte

func (NullBytes) IsNull

func (n NullBytes) IsNull() bool

func (NullBytes) String

func (n NullBytes) String() string

type NullableValue

type NullableValue interface {
	spanner.NullableValue
	fmt.Stringer
}

type UnnamedFieldNamer added in v0.1.9

type UnnamedFieldNamer func(index int) string

UnnamedFieldNamer generates a name for an unnamed field or column. The index argument is a monotonically increasing counter (not necessarily the field's positional index) that may skip values due to collision avoidance. It must return distinct non-empty names for distinct indices. Functions that accept UnnamedFieldNamer (such as NewJSONObjectStructFormatter and FormatRowJSONObject) return an error if the namer violates this contract. Pass nil instead of a namer to keep unnamed fields as empty-string keys.

Directories

Path Synopsis
dbsqlrows module
Package gcvctor constructs cloud.google.com/go/spanner.GenericColumnValue values from Go values and explicit cloud.google.com/go/spanner/apiv1/spannerpb.Type metadata, using github.com/apstndb/spantype/typector for type shapes.
Package gcvctor constructs cloud.google.com/go/spanner.GenericColumnValue values from Go values and explicit cloud.google.com/go/spanner/apiv1/spannerpb.Type metadata, using github.com/apstndb/spantype/typector for type shapes.
Package writer provides small streaming helpers for exporting Spanner rows using spanvalue formatters.
Package writer provides small streaming helpers for exporting Spanner rows using spanvalue formatters.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL