README
¶
dbsqlrows
Experimental — APIs may change before a stable release.
Package documentation: pkg.go.dev/github.com/apstndb/spanvalue/dbsqlrows
go test ./dbsqlrows/...
# or from repo root: make check
Optional nested module gospanner/ (QueryExport, DefaultExecOptions).
Documentation
¶
Overview ¶
Package dbsqlrows is experimental: APIs may change before a stable release.
It streams database/sql query results into github.com/apstndb/spanvalue/writer using the GenericColumnValue slice export path, or into custom sinks via SQLRowsHooks.
Callers that use a Spanner database/sql driver (for example github.com/googleapis/go-sql-spanner) configure driver-specific options themselves; this package only iterates *sql.Rows once they are open.
Naming ¶
The name combines db (standard library database/sql) and sqlrows (*sql.Rows as input). For the native-client export path, use github.com/apstndb/spanvalue/writer ([writer.WriteRowIterator] on *cloud.google.com/go/spanner.RowIterator).
writer vs dbsqlrows ¶
| Path | Iterator | Row shape | spanvalue entry | |------|----------|-----------|-----------------| | Native client | *spanner.RowIterator | *spanner.Row | [writer.WriteRowIterator] | | database/sql driver | *sql.Rows | []spanner.GenericColumnValue | [writer.DelimitedWriter.WriteGCVs] | | dbsqlrows | *sql.Rows (caller-owned) | []spanner.GenericColumnValue | RunRowsAtData / WriteRowsAtData |
dbsqlrows does not convert GCV slices to *spanner.Row for [writer.Writer.WriteRow].
Module layout ¶
Package path github.com/apstndb/spanvalue/dbsqlrows is part of the single github.com/apstndb/spanvalue module. The package does not import go-sql-spanner (or any database/sql driver). Optional one-shot helpers live in nested module github.com/apstndb/spanvalue/dbsqlrows/gospanner.
Goals ¶
- Own the *sql.Rows loop: metadata pseudo-row → data rows → optional stats pseudo-row.
- Delegate csv/jsonl formatting to [writer.WriteGCVs] / GCVStreamWriter.Flush.
- Expose SQLRowsHooks for custom sinks (table layout, drain-only) parallel to [writer.RowIteratorHooks].
- Keep database/sql drivers out of spanvalue go.mod.
Non-goals ¶
- Native *spanner.RowIterator export ([writer.WriteRowIterator]).
- String → GCV parsing, PostgreSQL table cells, or built-in ASCII table layout.
- Batch orchestration, SQL INSERT export, or owning db.QueryContext / driver ExecOptions.
API overview ¶
| Entry point | When to use | |-------------|-------------| | WriteRows | Open *sql.Rows at metadata pseudo-row; csv/jsonl via GCVStreamWriter | | RunRows / RunRowsAtData | Custom sinks via SQLRowsHooks | | ReadMetadataAndAdvanceToData | Metadata-first apps; advances cursor to data rows | | WriteRowsAtData | RunRowsAtData + SQLRowsHooksFromGCVWriter |
Symmetry with writer:
| writer | dbsqlrows | |--------|-----------| | [writer.RunRowIterator] | RunRows / RunRowsAtData | | [writer.RowIteratorHooks] | SQLRowsHooks | | [writer.RowIteratorHooksFromWriter] | SQLRowsHooksFromGCVWriter | | [writer.RowIteratorResult] | SQLRowsResult |
SQLRowsResult carries Metadata when known on error paths (partial-result contract matching [writer.RowIteratorResult]). Stats are not consumed unless SQLRowsConfig.ReadResultSetStats is true; the iterator then advances with NextResultSet for multi-statement batches.
An empty SQLRowsHooks value advances past data rows without per-row decode when WriteDataRow is nil (EXPLAIN / drain before stats). When WriteDataRow is set, the GCV slice passed to it is reused each row — copy or format before returning if the sink retains row data.
go-sql-spanner integration ¶
Option A (driver-agnostic): configure ExecOptions at query time, then pass open *sql.Rows to WriteRows or RunRows:
opts := spannerdriver.ExecOptions{
DecodeOption: spannerdriver.DecodeOptionProto,
ReturnResultSetMetadata: true,
ReturnResultSetStats: false,
}
rows, err := db.QueryContext(ctx, q, opts)
// ...
result, err := dbsqlrows.WriteRows(rows, w, dbsqlrows.SQLRowsConfig{})
Option B: nested module github.com/apstndb/spanvalue/dbsqlrows/gospanner provides DefaultExecOptions and QueryExport for one-shot query → csv/jsonl export when the app already depends on go-sql-spanner. It is a thin reference integration (ExecOptions + QueryContext + WriteRows); root go.mod still has no go-sql-spanner. Interactive shells, metadata-first batches, EXPLAIN, and per-query driver options (QueryMode, DirectExecuteQuery) should use Option A with app-owned ExecOptions instead — validated by [spannersh](https://github.com/apstndb/spannersh).
Stats: driver vs export ¶
A common REPL pattern: set ReturnResultSetStats true on the driver at QueryContext, keep SQLRowsConfig.ReadResultSetStats false during csv/jsonl/table export, then read the stats pseudo-row in application code after render. dbsqlrows leaves the cursor on the data result set until export completes or ReadResultSetStats is enabled.
Application patterns ¶
Metadata-first batch: ReadMetadataAndAdvanceToData, render (table via RunRowsAtData + hooks), then read stats from rows or set ReadResultSetStats.
Table sink: RunRowsAtData with SQLRowsHooks.WithPrepareMetadata, SQLRowsHooks.WithWriteDataRow, and SQLRowsHooks.WithFinish — apps own layout libraries and cell formatting.
csv/jsonl: WriteRowsAtData with [writer.DelimitedGCVExportOptions] or [writer.JSONLGCVExportOptions] at writer construction.
EXPLAIN / drain: RunRowsAtData with NewSQLRowsHooks and SQLRowsConfig.WithReadResultSetStats(true).
Index ¶
- Variables
- func ReadMetadataAndAdvanceToData(rows *sql.Rows) (*sppb.ResultSetMetadata, bool, error)
- type GCVStreamWriter
- type SQLRowsConfig
- type SQLRowsHooks
- type SQLRowsResult
- func RunRows(rows *sql.Rows, hooks SQLRowsHooks, cfg SQLRowsConfig) (*SQLRowsResult, error)
- func RunRowsAtData(rows *sql.Rows, metadata *sppb.ResultSetMetadata, hooks SQLRowsHooks, ...) (*SQLRowsResult, error)
- func WriteRows(rows *sql.Rows, w GCVStreamWriter, cfg SQLRowsConfig) (*SQLRowsResult, error)
- func WriteRowsAtData(rows *sql.Rows, metadata *sppb.ResultSetMetadata, w GCVStreamWriter, ...) (*SQLRowsResult, error)
Constants ¶
This section is empty.
Variables ¶
var ( // ErrNilRows reports that a dbsqlrows entry point was called with a nil *sql.Rows // (for example [WriteRows], [RunRows], [RunRowsAtData], [WriteRowsAtData], or // [ReadMetadataAndAdvanceToData]). ErrNilRows = errors.New("nil sql.Rows") // ErrNilWriter reports that an export entry point was called with a nil [GCVStreamWriter] // (for example [WriteRows] or [WriteRowsAtData]). ErrNilWriter = errors.New("nil GCV stream writer") // ErrNilMetadata reports that a data-phase entry point was called with nil metadata // (for example [WriteRowsAtData] or [RunRowsAtData]). ErrNilMetadata = errors.New("nil result set metadata") // ErrMissingMetadataRow reports that the iterator produced no metadata // pseudo-row when WriteRows expected one. ErrMissingMetadataRow = errors.New("missing result set metadata row") // ErrMissingDataResultSet reports that NextResultSet did not advance to the // data rows result set after the metadata pseudo-row. ErrMissingDataResultSet = errors.New("missing data rows result set after metadata") // ErrMissingStatsRow reports that the stats result set had no stats pseudo-row. ErrMissingStatsRow = errors.New("missing result set stats row") )
Functions ¶
func ReadMetadataAndAdvanceToData ¶
ReadMetadataAndAdvanceToData reads the metadata pseudo-row from rows and advances to the data result set. Use before WriteRowsAtData or custom rendering when metadata is consumed outside export.
If there is no metadata row, returns ok=false and err=rows.Err() (nil on clean EOF).
Types ¶
type GCVStreamWriter ¶
type GCVStreamWriter interface {
WriteGCVs([]spanner.GenericColumnValue) error
Flush() error
}
GCVStreamWriter is the subset of github.com/apstndb/spanvalue/writer types that dbsqlrows drives. Built-in writers also implement PrepareRowType or Prepare for metadata registration; SQLRowsHooksFromGCVWriter calls those when present after reading the metadata pseudo-row.
type SQLRowsConfig ¶
type SQLRowsConfig struct {
// ReadResultSetStats, when true, advances past data rows to read the stats
// pseudo-row into [SQLRowsResult.Stats]. For [WriteRowsAtData] this field is
// consulted directly (default false). For [WriteRows] the same field applies.
ReadResultSetStats bool
}
SQLRowsConfig configures a SQL rows streaming run.
func (SQLRowsConfig) WithReadResultSetStats ¶
func (cfg SQLRowsConfig) WithReadResultSetStats(read bool) SQLRowsConfig
WithReadResultSetStats returns a copy of cfg with ReadResultSetStats set.
type SQLRowsHooks ¶
type SQLRowsHooks struct {
PrepareMetadata func(*sppb.ResultSetMetadata) error
WriteDataRow func([]spanner.GenericColumnValue) error
Finish func(*SQLRowsResult) error
}
SQLRowsHooks drives RunRows and RunRowsAtData. Nil function fields are skipped.
An empty hooks value (from NewSQLRowsHooks) still advances past data rows and increments SQLRowsResult.RowsRead while WriteDataRow is nil (no per-row decode). Use that to drain rows before reading stats (for example EXPLAIN with SQLRowsConfig.ReadResultSetStats).
PrepareMetadata runs once after metadata is known and before data rows are scanned. WriteDataRow runs per data row when set. The []spanner.GenericColumnValue argument is reused across calls: valid only for the duration of WriteDataRow; copy or format synchronously before returning if the sink retains row data. Finish runs only after all rows and optional stats consumption succeed; it is not called when PrepareMetadata or WriteDataRow returns an error. The returned SQLRowsResult still carries Metadata and RowsRead at the abort point (same partial-result contract as [writer.RowIteratorHooks] and [writer.RunRowIterator]).
func NewSQLRowsHooks ¶
func NewSQLRowsHooks() SQLRowsHooks
NewSQLRowsHooks returns an empty hooks value for custom decoration or SQLRowsHooksFromGCVWriter.
func SQLRowsHooksFromGCVWriter ¶
func SQLRowsHooksFromGCVWriter(w GCVStreamWriter) SQLRowsHooks
SQLRowsHooksFromGCVWriter returns hooks that register metadata via GCVStreamWriter PrepareRowType or Prepare when implemented, write each row with GCVStreamWriter.WriteGCVs, and call GCVStreamWriter.Flush in Finish. Finish (and thus Flush) runs only after all rows and optional stats consumption succeed; it is skipped when PrepareMetadata, WriteDataRow, or the stats phase returns an error. A nil writer returns empty hooks.
func (SQLRowsHooks) WithFinish ¶
func (h SQLRowsHooks) WithFinish(fn func(*SQLRowsResult) error) SQLRowsHooks
WithFinish sets Finish and returns h.
func (SQLRowsHooks) WithPrepareMetadata ¶
func (h SQLRowsHooks) WithPrepareMetadata(fn func(*sppb.ResultSetMetadata) error) SQLRowsHooks
WithPrepareMetadata sets PrepareMetadata and returns h.
func (SQLRowsHooks) WithWriteDataRow ¶
func (h SQLRowsHooks) WithWriteDataRow(fn func([]spanner.GenericColumnValue) error) SQLRowsHooks
WithWriteDataRow sets WriteDataRow and returns h. The slice passed to fn is reused on each row; do not retain it after fn returns.
type SQLRowsResult ¶
type SQLRowsResult struct {
Metadata *sppb.ResultSetMetadata
Stats *sppb.ResultSetStats
RowsRead int
}
SQLRowsResult holds metadata and stats surfaced from driver pseudo result sets, analogous to [writer.RowIteratorResult] for native iterators. On error paths after metadata is known, Metadata and RowsRead reflect progress at the abort point (same partial-result contract as writer row-iterator helpers).
func RunRows ¶
func RunRows(rows *sql.Rows, hooks SQLRowsHooks, cfg SQLRowsConfig) (*SQLRowsResult, error)
RunRows streams an open *sql.Rows positioned at the metadata pseudo-row using hooks. See WriteRows for driver conventions, ownership, and stats behavior.
func RunRowsAtData ¶
func RunRowsAtData( rows *sql.Rows, metadata *sppb.ResultSetMetadata, hooks SQLRowsHooks, cfg SQLRowsConfig, ) (*SQLRowsResult, error)
RunRowsAtData streams rows already positioned on the data result set using hooks. metadata must be non-nil. See WriteRowsAtData for stats and partial-result semantics.
func WriteRows ¶
func WriteRows(rows *sql.Rows, w GCVStreamWriter, cfg SQLRowsConfig) (*SQLRowsResult, error)
WriteRows streams an open *sql.Rows positioned at the metadata pseudo-row into w. The caller must open rows with a driver that returns proto-decoded GCV columns and a leading metadata pseudo result set (see README). The caller retains ownership of rows and must Close it and check sql.Rows.Err when appropriate.
On success WriteRows calls GCVStreamWriter.Flush and returns its error explicitly (do not defer Flush at the call site). Data rows are scanned into []spanner.GenericColumnValue.
When ReadResultSetStats is false (the default), rows remain on the data result set after export so the caller can advance to stats separately.
For custom sinks (for example ASCII table rendering), use RunRows with SQLRowsHooks instead.
func WriteRowsAtData ¶
func WriteRowsAtData( rows *sql.Rows, metadata *sppb.ResultSetMetadata, w GCVStreamWriter, cfg SQLRowsConfig, ) (*SQLRowsResult, error)
WriteRowsAtData streams rows already positioned on the data result set into w. metadata must be non-nil (typically from ReadMetadataAndAdvanceToData or an earlier statement in a batch). The writer is prepared from metadata when it implements PrepareRowType or Prepare.
Stats are not consumed unless cfg.ReadResultSetStats is true, so callers can render first and read stats from rows afterward (spannersh execution summary).
For custom sinks, use RunRowsAtData with SQLRowsHooks.
Source Files