experiments

func AppendToDefaultStream ¶

func AppendToDefaultStream(w io.Writer, projectID, datasetID, tableID string) error

appendToDefaultStream demonstrates using the managedwriter package to write some example data to a default stream.

func AppendToPendingStream ¶

func AppendToPendingStream(w io.Writer, projectID, datasetID, tableID string) error

func BuildAppendRowsRequest ¶

func BuildAppendRowsRequest(data [][]byte) *storagepb.AppendRowsRequest

func GoTypeToArrowType ¶

func GoTypeToArrowType(goType reflect.Type) arrow.DataType

func NormalizeDescriptor ¶

func NormalizeDescriptor(in protoreflect.MessageDescriptor) (*descriptorpb.DescriptorProto, error)

NormalizeDescriptor builds a self-contained DescriptorProto suitable for communicating schema information with the BigQuery Storage write API. It's primarily used for cases where users are interested in sending data using a predefined protocol buffer message.

The storage API accepts a single DescriptorProto for decoding message data. In many cases, a message is comprised of multiple independent messages, from the same .proto file or from multiple sources. Rather than being forced to communicate all these messages independently, what this method does is rewrite the DescriptorProto to inline all messages as nested submessages. As the backend only cares about the types and not the namespaces when decoding, this is sufficient for the needs of the API's representation.

In addition to nesting messages, this method also handles some encapsulation of enum types to avoid possible conflicts due to ambiguities, and clears oneof indices as oneof isn't a concept that maps into BigQuery schemas.

To enable proto3 usage, this function will also rewrite proto3 descriptors into equivalent proto2 form. Such rewrites include setting the appropriate default values for proto3 fields.

func StorageSchemaToProto2Descriptor ¶

func StorageSchemaToProto2Descriptor(inSchema *storagepb.TableSchema, scope string) (protoreflect.Descriptor, error)

StorageSchemaToProto2Descriptor builds a protoreflect.Descriptor for a given table schema using proto2 syntax.

func StorageSchemaToProto3Descriptor ¶

func StorageSchemaToProto3Descriptor(inSchema *storagepb.TableSchema, scope string) (protoreflect.Descriptor, error)

StorageSchemaToProto3Descriptor builds a protoreflect.Descriptor for a given table schema using proto3 syntax.

NOTE: Currently the write API doesn't yet support proto3 behaviors (default value, wrapper types, etc), but this is provided for completeness.

func UniqueBQName ¶

func UniqueBQName(prefix string) (string, error)

UniqueBQName returns a more unique name for a BigQuery resource.

func UniqueBucketName ¶

func UniqueBucketName(prefix, projectID string) (string, error)

UniqueBucketName returns a more unique name cloud storage bucket.