spanenc

package module
v0.2.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jun 10, 2026 License: MIT Imports: 18 Imported by: 0

README

spanenc

Go Reference

Convert plain Go values into Cloud Spanner GenericColumnValue (GCV) values and derive column names and Spanner types from Go structs, following the Cloud Spanner Go client library's own encoding semantics: the spanner struct tag rules and the Go type coverage of statement parameters and mutations.

The client library keeps its encoding internal (encodeValue, structToMutationParams, and the internal fields cache). This package mirrors those semantics on top of github.com/apstndb/spanvalue/gcvctor constructors, so the results compose with the spanvalue formatting and writer stack.

Status: experimental. The API may change while encodeValue parity is proven against client library releases. The mirrored behavior currently tracks cloud.google.com/go/spanner v1.91.0.

Motivation

  • googleapis/google-cloud-go#13800 asks for a StructColumns() helper that derives Read columns from the same struct used with Row.ToStruct. StructColumns provides it with the exact field listing the client uses.
  • The client offers no public Go value → *structpb.Value / *sppb.Type encoding, but GCV-level tooling (exporters, REPLs, test fixtures, proto-level API callers) needs one that behaves identically to the client.
  • Mutation constructors come in three flavors (Update, UpdateMap, UpdateStruct); only the struct flavor understands tags, and it cannot mask columns. MutationColumnsAndValues / MutationMap extract tag-derived cols/vals (plain Go values, encoded later by the client itself) so the other two flavors get tag support and per-column masking.

API overview

Function Input Output
ValueOf Go value spanner.GenericColumnValue
TypeFor[T] / TypeFromGoType Go type *sppb.Type
StructColumns[T] / StructColumnsFromGoType struct type []string column names
RowTypeFor[T] / RowTypeFromGoType struct type *sppb.StructType row type
StructColumnsAndValues struct value []string, []GCV
MutationColumnsAndValues struct value []string, []any (for spanner.Insert/Update/Replace...)
MutationMap struct value map[string]any (for spanner.InsertMap/UpdateMap...)
ValuesFromSlice[T] homogeneous slice *sppb.Type (element), []*structpb.Value
ArrayValueFromSlice[T] homogeneous slice ARRAY GCV (nil slice → typed NULL ARRAY)

Slice helpers enforce homogeneity through the static element type: interface element types (which could hold heterogeneous values) are rejected before any element is examined.

Examples

Derive Read columns from a tagged struct (the issue #13800 use case):

type Singer struct {
    SingerID  int64 `spanner:"SingerId"`
    FirstName string
    Internal  string `spanner:"-"`
}

columns, _ := spanenc.StructColumns[Singer]() // [SingerId FirstName]
iter := client.Single().Read(ctx, "Singers", spanner.AllKeys(), columns)

Mask mutation columns by name:

cols, vals, _ := spanenc.MutationColumnsAndValues(singer)
// filter cols/vals pairs, then:
m := spanner.Update("Singers", maskedCols, maskedVals)

Stream Go structs through github.com/apstndb/spanvalue/writer (CSV/TSV/JSONL/SQL INSERT exporters for GCV rows):

names, values, _ := spanenc.StructColumnsAndValues(singer)
w, _ := writer.NewCSVWriter(os.Stdout, writer.WithColumnNames(names))
_ = w.WriteValues(names, values)
_ = w.Flush()

Semantics notes

Following the client, there are two different struct field listings:

  • Row-shaped (StructColumns, RowTypeFor, StructColumnsAndValues, MutationColumnsAndValues, MutationMap): the mutation/ToStruct listing — exported fields, embedded struct fields flattened with Go's shadowing rules, spanner:"-" skipped, declaration order. Read-only fields (spanner:"->" / spanner:"Name;readonly", since spanner v1.86.0) are included in the read-shaped helpers and excluded from MutationColumnsAndValues / MutationMap, matching the client's mutation constructors.
  • STRUCT-typed values (ValueOf on a struct, TypeFor): the encodeStruct listing — declaration order, embedded fields rejected, spanner:"" producing an unnamed STRUCT field.

Deliberate divergences from the client (strictness so malformed GCVs never enter the spanvalue stack) are documented in the package documentation: untyped nil and nil struct pointers return errors, NUMERIC precision is always validated, and non-finite floats / JSON use gcvctor's canonical wire forms.

Tracking upstream

This package is, by design, a behavioral mirror of googleapis/google-cloud-go's spanner package:

  • Struct-field listing reuses github.com/apstndb/structfields, an exported fork of cloud.google.com/go/internal/fields (Apache License 2.0) maintained as a separate module so that this module contains no upstream-derived code.
  • ValueOf / TypeFromGoType mirror the behavior of encodeValue and getDecodableSpannerType with independent code; new client-supported Go types and Spanner types must be added here when upstream adds them.

The mirrored-semantics version is independent of the go.mod requirement: go.mod declares only the minimum cloud.google.com/go/spanner providing the APIs this module compiles against (currently the v1.84.1 floor inherited from spanvalue), so downstream modules keep control of the client version under MVS. CI additionally tests against the latest spanner release to catch drift early.

When re-auditing against a newer client release, update the tracked version in the package documentation — and bump go.mod only if newly used APIs require it.

License

MIT. The Apache-2.0 fork of upstream code lives separately in structfields.

Documentation

Overview

Package spanenc converts plain Go values into cloud.google.com/go/spanner.GenericColumnValue (GCV) values and derives column names and Spanner types from Go structs, following the Cloud Spanner Go client library's own encoding semantics — the `spanner` struct tag rules and the Go type coverage of statement parameters and mutations.

The client library keeps its encoding internal (encodeValue, structToMutationParams, and the internal fields cache); this package mirrors those semantics on top of github.com/apstndb/spanvalue/gcvctor constructors so the results compose with the spanvalue formatting and writer stack. The mirrored behavior tracks cloud.google.com/go/spanner v1.91.0.

API overview

Struct field listings

Following the client, there are two different struct field listings:

  • Row-shaped helpers (StructColumns, RowTypeFor, StructColumnsAndValues, MutationColumnsAndValues, MutationMap) use the mutation/ToStruct listing: exported fields, embedded struct fields flattened with Go's shadowing rules, `spanner:"-"` skipped, declaration order. Tags split on ";" with the column name first; `spanner:"->"` or a `readonly` part marks the field read-only (since spanner v1.86.0). Read-only fields stay in the read-shaped listings (StructColumns, RowTypeFor, StructColumnsAndValues) and are excluded from the write-shaped ones (MutationColumnsAndValues, MutationMap), mirroring structToMutationParams.
  • STRUCT-typed values (ValueOf on a struct, TypeFor) use the encodeStruct listing: declaration order, embedded fields rejected with ErrEmbeddedStructField, and `spanner:""` producing an unnamed field. encodeStruct reads the raw tag, so tag options leak into STRUCT field names verbatim (`spanner:"Name;readonly"` yields a field literally named "Name;readonly"); this mirrors the client.

Divergences from the client library

This package is strict where the client is lenient, so malformed GCVs never enter the spanvalue stack:

The package is experimental: the API may change while encodeValue parity is being proven against client library releases.

Index

Examples

Constants

This section is empty.

Variables

View Source
var (
	// ErrUnsupportedType is returned when a Go value or type has no Cloud Spanner
	// client library encoding. It corresponds to the client's
	// "client doesn't support Go type" errors from encodeValue.
	ErrUnsupportedType = errors.New("spanenc: unsupported Go type")

	// ErrUntypedNil is returned by [ValueOf] for an untyped nil input.
	// This is a deliberate divergence from the client library, which encodes
	// untyped nil as a NULL value WITHOUT type information; spanvalue/gcvctor
	// consumers require a well-formed Type in every GenericColumnValue.
	// Use a typed nil (for example (*int64)(nil)) or
	// [github.com/apstndb/spanvalue/gcvctor.NullOf] instead.
	ErrUntypedNil = errors.New("spanenc: untyped nil value")

	// ErrNotStruct is returned by struct-shaped helpers ([StructColumns],
	// [RowTypeFor], [StructColumnsAndValues], and variants) when the input is
	// not a Go struct or pointer to struct. It mirrors the client's
	// errNotStruct from structToMutationParams.
	ErrNotStruct = errors.New("spanenc: not a Go struct type")

	// ErrNilStructPointer is returned by [StructColumnsAndValues] for a nil
	// pointer to struct. This is a deliberate divergence from the client's
	// structToMutationParams, which silently returns empty columns and values;
	// silently producing an empty row is a footgun for export use cases.
	ErrNilStructPointer = errors.New("spanenc: nil struct pointer")

	// ErrEmbeddedStructField is returned when a Go struct with embedded
	// (anonymous) fields is encoded as a Spanner STRUCT value. It mirrors the
	// client's errUnsupportedEmbeddedStructFields in encodeStruct. Note that
	// the row-shaped helpers ([StructColumns], [RowTypeFor],
	// [StructColumnsAndValues]) flatten embedded fields instead, mirroring the
	// client's mutation/ToStruct field listing.
	ErrEmbeddedStructField = errors.New("spanenc: embedded struct fields are not supported in STRUCT values")

	// ErrTypeNotInferable is returned by [TypeFor], [TypeFromGoType], and the
	// slice helpers when the Spanner type cannot be derived from the Go type
	// alone: types implementing [cloud.google.com/go/spanner.Encoder],
	// [cloud.google.com/go/spanner.GenericColumnValue],
	// [cloud.google.com/go/spanner.NullProtoMessage],
	// [cloud.google.com/go/spanner.NullProtoEnum], and interface types. The
	// client library never needs a type-only path because it always encodes
	// concrete values.
	ErrTypeNotInferable = errors.New("spanenc: Spanner type is not inferable from the Go type alone")

	// ErrInvalidSource is returned when a value cannot represent a typed NULL
	// and is invalid, mirroring the client's errNotValidSrc: NullProtoMessage
	// and NullProtoEnum with Valid == false.
	ErrInvalidSource = errors.New("spanenc: invalid (NULL) source value")

	// ErrNumericOutOfRange is returned when a NUMERIC input exceeds the
	// precision or scale supported by Cloud Spanner, mirroring the client's
	// validateNumeric under the default NumericError loss-of-precision
	// handling.
	ErrNumericOutOfRange = errors.New("spanenc: NUMERIC value exceeds supported precision or scale")
)

Functions

func ArrayValueFromSlice

func ArrayValueFromSlice[T any](vs []T) (spanner.GenericColumnValue, error)

ArrayValueFromSlice converts a homogeneous slice into an ARRAY spanner.GenericColumnValue with the element type inferred statically from T (see ValuesFromSlice for the homogeneity rules). Following the client library's slice handling, a nil slice becomes a typed NULL ARRAY and an empty slice an empty ARRAY.

func MutationColumnsAndValues

func MutationColumnsAndValues(v any) ([]string, []any, error)

MutationColumnsAndValues extracts column names and plain Go field values from a struct (or non-nil pointer to struct), mirroring the client's structToMutationParams: read-only fields (`spanner:"->"` or `spanner:"Name;readonly"`, since spanner v1.86.0) are excluded from the results. The results fit the cols/vals form of mutation constructors such as spanner.Insert, spanner.Update, and spanner.Replace, so callers can mask columns by name before building the mutation — something the *Struct constructors cannot do.

Values are returned as-is (no GCV conversion); the client library encodes them when the mutation is applied, so this helper accepts whatever InsertStruct accepts. A nil pointer returns ErrNilStructPointer where the client would silently produce an empty mutation.

Example

ExampleMutationColumnsAndValues masks columns by name before building a plain cols/vals mutation, which the *Struct mutation constructors cannot express.

package main

import (
	"fmt"
	"slices"

	"cloud.google.com/go/spanner"

	"github.com/apstndb/spanenc"
)

func main() {
	type Singer struct {
		SingerID  int64 `spanner:"SingerId"`
		FirstName string
		LastName  string
	}

	cols, vals, err := spanenc.MutationColumnsAndValues(Singer{SingerID: 1, FirstName: "Marc", LastName: "Richards"})
	if err != nil {
		fmt.Println(err)
		return
	}
	// Update only SingerId and LastName, masking FirstName.
	var mcols []string
	var mvals []any
	for i, c := range cols {
		if !slices.Contains([]string{"SingerId", "LastName"}, c) {
			continue
		}
		mcols = append(mcols, c)
		mvals = append(mvals, vals[i])
	}
	_ = spanner.Update("Singers", mcols, mvals)
	fmt.Println(mcols)
	fmt.Println(mvals)
}
Output:
[SingerId LastName]
[1 Richards]

func MutationMap

func MutationMap(v any) (map[string]any, error)

MutationMap extracts a column-name-to-Go-value map from a struct (or non-nil pointer to struct) for the *Map mutation constructors (spanner.InsertMap, spanner.UpdateMap, spanner.ReplaceMap, spanner.InsertOrUpdateMap). Read-only fields are excluded like MutationColumnsAndValues. Masking a column is a map delete away.

Duplicate column names (possible with explicit duplicate `spanner` tags) return an error rather than silently dropping a value.

func RowTypeFor

func RowTypeFor[T any]() (*sppb.StructType, error)

RowTypeFor returns the sppb.StructType describing a row of T, pairing StructColumns names (read-only fields included) with statically inferred field types (see TypeFromGoType). It suits writer metadata such as github.com/apstndb/spanvalue/writer's WithRowType, or the row_type of a ResultSetMetadata.

Note that this row-shaped view flattens embedded struct fields, while a STRUCT-typed value of T (TypeFor, ValueOf) rejects them; the client library has the same split between mutations/ToStruct and STRUCT parameters.

func RowTypeFromGoType

func RowTypeFromGoType(t reflect.Type) (*sppb.StructType, error)

RowTypeFromGoType is RowTypeFor for a reflect.Type.

func StructColumns

func StructColumns[T any]() ([]string, error)

StructColumns returns the column names derived from T's fields and `spanner` tags, in declaration order, with the same field listing the client library uses for cloud.google.com/go/spanner.Row.ToStruct: exported fields only, embedded struct fields flattened, `spanner:"-"` skipped. Read-only fields (`spanner:"->"` or `spanner:"Name;readonly"`) are included, because they are readable; the write-shaped MutationColumnsAndValues and MutationMap exclude them like the client's mutation constructors do.

It is the spanvalue-side answer to the StructColumns helper requested in https://github.com/googleapis/google-cloud-go/issues/13800, typically used to build the columns argument of Read calls from the same struct passed to ToStruct. T may be a struct or pointer-to-struct type.

Example

ExampleStructColumns derives Read columns from the same struct used with Row.ToStruct, the use case of https://github.com/googleapis/google-cloud-go/issues/13800.

package main

import (
	"fmt"

	"github.com/apstndb/spanenc"
)

func main() {
	type Singer struct {
		SingerID  int64 `spanner:"SingerId"`
		FirstName string
		LastName  string
		Internal  string `spanner:"-"`
	}

	columns, err := spanenc.StructColumns[Singer]()
	if err != nil {
		fmt.Println(err)
		return
	}
	fmt.Println(columns)
	// client.Single().Read(ctx, "Singers", spanner.AllKeys(), columns) ...
	

func StructColumnsAndValues

func StructColumnsAndValues(v any) ([]string, []spanner.GenericColumnValue, error)

StructColumnsAndValues converts a struct (or non-nil pointer to struct) into parallel column-name and spanner.GenericColumnValue slices using the ToStruct-shaped field listing (see StructColumns; read-only fields included) and ValueOf for each field. The result feeds GCV-level consumers such as github.com/apstndb/spanvalue/writer's WriteValues, and stays aligned with RowTypeFor.

For the client library's own mutation constructors, use MutationColumnsAndValues or MutationMap instead, which keep plain Go values, exclude read-only fields, and let the client encode the values.

Example

ExampleStructColumnsAndValues streams Go structs to CSV through spanvalue/writer.

package main

import (
	"fmt"
	"os"

	"github.com/apstndb/spanvalue/writer"

	"github.com/apstndb/spanenc"
)

func main() {
	type Singer struct {
		SingerID  int64 `spanner:"SingerId"`
		FirstName string
	}

	names, values, err := spanenc.StructColumnsAndValues(Singer{SingerID: 1, FirstName: "Marc"})
	if err != nil {
		fmt.Println(err)
		return
	}
	w, err := writer.NewCSVWriter(os.Stdout, writer.WithColumnNames(names))
	if err != nil {
		fmt.Println(err)
		return
	}
	if err := w.WriteValues(names, values); err != nil {
		fmt.Println(err)
		return
	}
	if err := w.Flush(); err != nil {
		fmt.Println(err)
		return
	}
}
Output:
SingerId,FirstName
1,Marc

func StructColumnsFromGoType

func StructColumnsFromGoType(t reflect.Type) ([]string, error)

StructColumnsFromGoType is StructColumns for a reflect.Type.

func TypeFor

func TypeFor[T any]() (*sppb.Type, error)

TypeFor returns the sppb.Type that ValueOf would produce for a non-NULL value of Go type T, following the Cloud Spanner client library encoding semantics. See TypeFromGoType.

func TypeFromGoType

func TypeFromGoType(t reflect.Type) (*sppb.Type, error)

TypeFromGoType returns the sppb.Type that ValueOf produces for values of the given Go type, following the Cloud Spanner client library encoding semantics (encodeValue in cloud.google.com/go/spanner).

It returns ErrTypeNotInferable for Go types whose Spanner type depends on the value rather than the type: interface types, cloud.google.com/go/spanner.Encoder implementations, cloud.google.com/go/spanner.GenericColumnValue, cloud.google.com/go/spanner.NullProtoMessage, and cloud.google.com/go/spanner.NullProtoEnum. It returns ErrUnsupportedType for Go types the client library cannot encode.

func ValueOf

func ValueOf(v any) (spanner.GenericColumnValue, error)

ValueOf converts a Go value into a spanner.GenericColumnValue, following the Cloud Spanner client library encoding semantics (encodeValue in cloud.google.com/go/spanner): the Go types accepted for statement parameters and mutation values, including the typed NULL rules (nil pointers and nil slices become typed NULLs), spanner.Null* wrappers, spanner.Encoder implementations, protobuf messages and enums, named variants of base types, Go structs as STRUCT values, and spanner.CommitTimestamp.

Deliberate divergences from the client library are listed in the package documentation; most notably an untyped nil returns ErrUntypedNil instead of a NULL without type information.

Example

ExampleValueOf encodes Go values with the client library's parameter semantics and formats them with spanvalue.

package main

import (
	"fmt"

	"github.com/apstndb/spanvalue"

	"github.com/apstndb/spanenc"
)

func main() {
	for _, v := range []any{
		"foo",
		(*int64)(nil),
		[]string{"a", "b"},
	} {
		gcv, err := spanenc.ValueOf(v)
		if err != nil {
			fmt.Println(err)
			return
		}
		s, err := spanvalue.FormatColumnLiteral(gcv)
		if err != nil {
			fmt.Println(err)
			return
		}
		fmt.Println(s)
	}
}
Output:
"foo"
NULL
["a", "b"]

func ValuesFromSlice

func ValuesFromSlice[T any](vs []T) (*sppb.Type, []*structpb.Value, error)

ValuesFromSlice converts a homogeneous slice into the shared element sppb.Type and one wire structpb.Value per element. Homogeneity is enforced through the static element type: T must have a type-inferable Spanner type (see TypeFromGoType), so interface element types — which could hold heterogeneous values — are rejected with ErrTypeNotInferable before any element is examined. Each encoded element is additionally verified against the inferred type.

A nil slice returns the element type with a nil values slice. To express a typed NULL ARRAY versus an empty ARRAY at the GCV level, use ArrayValueFromSlice.

Types

This section is empty.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL