datasets

package
v0.1.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 30, 2025 License: MIT Imports: 23 Imported by: 1

Documentation

Overview

Package datasets provides a client for interacting with Tilebox Datasets.

Documentation: https://docs.tilebox.com/datasets

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func As

func As[T proto.Message](seq iter.Seq2[[]byte, error]) iter.Seq2[T, error]

As converts a sequence of bytes into a sequence of proto.Message.

func Collect

func Collect[K any](seq iter.Seq2[K, error]) ([]K, error)

Collect converts any sequence into a slice.

It returns an error if any of the elements in the sequence has a non-nil error.

func CollectAs

func CollectAs[T proto.Message](seq iter.Seq2[[]byte, error]) ([]T, error)

CollectAs converts a sequence of bytes into a slice of proto.Message.

Types

type Client

type Client struct {
	Datasets    DatasetClient
	Collections CollectionClient
	Datapoints  DatapointClient
}

Client is a Tilebox Datasets client.

func NewClient

func NewClient(options ...ClientOption) *Client

NewClient creates a new Tilebox Datasets client.

By default, the returned Client is configured with:

  • "https://api.tilebox.com" as the URL
  • environment variable TILEBOX_API_KEY as the API key
  • a grpc.RetryHTTPClient HTTP client
  • the global tracer provider

The passed options are used to override these default values and configure the returned Client appropriately.

type ClientOption

type ClientOption func(*clientConfig)

ClientOption is an interface for configuring a client. Using such options helpers is a quite common pattern in Go, as it allows for optional parameters in constructors. This concrete implementation here is inspired by how libraries such as axiom-go and connect do their configuration.

func WithAPIKey

func WithAPIKey(apiKey string) ClientOption

WithAPIKey sets the API key to use for the client.

Defaults to no API key.

func WithConnectClientOptions

func WithConnectClientOptions(options ...connect.ClientOption) ClientOption

WithConnectClientOptions sets additional options for the connect.HTTPClient.

func WithDisableTracing

func WithDisableTracing() ClientOption

WithDisableTracing disables OpenTelemetry tracing for the client.

func WithHTTPClient

func WithHTTPClient(httpClient connect.HTTPClient) ClientOption

WithHTTPClient sets the connect.HTTPClient to use for the client.

Defaults to grpc.RetryHTTPClient.

func WithURL

func WithURL(url string) ClientOption

WithURL sets the URL of the Tilebox Datasets service.

Defaults to "https://api.tilebox.com".

type Collection

type Collection struct {
	// ID is the unique identifier of the collection.
	ID uuid.UUID
	// Name is the name of the collection.
	Name string
	// Availability is the time interval for which data is available.
	Availability *query.TimeInterval
	// Count is the number of datapoints in the collection.
	Count uint64
}

Collection represents a Tilebox Dataset collection.

Documentation: https://docs.tilebox.com/datasets/concepts/collections

func (Collection) String

func (c Collection) String() string

type CollectionClient

type CollectionClient interface {
	// Create creates a new collection in the dataset with the given name.
	Create(ctx context.Context, datasetID uuid.UUID, name string) (*Collection, error)

	// Get returns a collection by its name.
	Get(ctx context.Context, datasetID uuid.UUID, name string) (*Collection, error)

	// GetOrCreate returns a collection by its name, creating it if it does not exist.
	GetOrCreate(ctx context.Context, datasetID uuid.UUID, name string) (*Collection, error)

	// Delete deletes a collection by its id.
	Delete(ctx context.Context, datasetID uuid.UUID, collectionID uuid.UUID) error

	// List returns a list of all available collections in the dataset.
	List(ctx context.Context, datasetID uuid.UUID) ([]*Collection, error)
}

type CollectionService

type CollectionService interface {
	CreateCollection(ctx context.Context, datasetID uuid.UUID, collectionName string) (*datasetsv1.CollectionInfo, error)
	GetCollectionByName(ctx context.Context, datasetID uuid.UUID, collectionName string) (*datasetsv1.CollectionInfo, error)
	DeleteCollection(ctx context.Context, datasetID uuid.UUID, collectionID uuid.UUID) error
	ListCollections(ctx context.Context, datasetID uuid.UUID) (*datasetsv1.CollectionInfos, error)
}

type DataAccessService

type DataAccessService interface {
	Query(ctx context.Context, collectionIDs []uuid.UUID, filters *datasetsv1.QueryFilters, page *tileboxv1.Pagination, skipData bool) (*datasetsv1.QueryResultPage, error)
	QueryByID(ctx context.Context, collectionIDs []uuid.UUID, datapointID uuid.UUID, skipData bool) (*datasetsv1.Any, error)
}

type DataIngestionService

type DataIngestionService interface {
	Ingest(ctx context.Context, collectionID uuid.UUID, datapoints [][]byte, allowExisting bool) (*datasetsv1.IngestResponse, error)
	Delete(ctx context.Context, collectionID uuid.UUID, datapointIDs []uuid.UUID) (*datasetsv1.DeleteResponse, error)
}

type DatapointClient

type DatapointClient interface {
	// GetInto get a single datapoint by its ID from one or more collections of the same dataset into a proto.Message.
	//
	// Options:
	//   - WithSkipData: can be used to skip the actual data, only returning required datapoint fields. (Optional)
	//
	// Example usage:
	//
	//	var datapoint v1.Sentinel1Sar
	//	err = client.Datapoints.GetInto(ctx, collectionIDs, datapointID, &datapoint)
	GetInto(ctx context.Context, collectionIDs []uuid.UUID, datapointID uuid.UUID, datapoint proto.Message, options ...QueryOption) error

	// Query datapoints from one or more collections of the same dataset.
	//
	// Options:
	//   - WithTemporalExtent: specifies the time or data point interval for which data should be loaded. (Required)
	//   - WithSpatialExtent: specifies the spatial extent for which data should be loaded. (Optional)
	//   - WithSkipData: can be used to skip the actual data when loading datapoints, only returning required datapoint fields. (Optional)
	//
	// The datapoints are lazily loaded and returned as a sequence of bytes.
	// The output sequence can be transformed into a proto.Message using CollectAs / As.
	//
	// Example usage:
	//
	//	for datapointBytes, err := range client.Datapoints.Query(ctx, collectionIDs, WithTemporalExtent(timeInterval), WithSpatialExtent(geometry)) {
	//	  if err != nil {
	//	    // handle error
	//	  }
	//	  datapoint := &v1.Sentinel1Sar{}
	//	  err = proto.Unmarshal(datapointBytes, datapoint)
	//	  if err != nil {
	//	    // handle unmarshal error
	//	  }
	//	  // do something with the datapoint
	//	}
	//
	// Documentation: https://docs.tilebox.com/datasets/query
	Query(ctx context.Context, collectionIDs []uuid.UUID, options ...QueryOption) iter.Seq2[[]byte, error]

	// QueryInto queries datapoints from one or more collections of the same dataset into a slice of datapoints of a
	// compatible proto.Message type.
	//
	// QueryInto is a convenience function for Query, when no manual pagination or custom iteration is required.
	//
	// Example usage:
	//
	//	var datapoints []*v1.Sentinel1Sar
	//	err := client.Datapoints.QueryInto(ctx, collectionIDs, &datapoints, WithTemporalExtent(timeInterval))
	QueryInto(ctx context.Context, collectionIDs []uuid.UUID, datapoints any, options ...QueryOption) error

	// Ingest datapoints into a collection.
	//
	// data is a list of datapoints to ingest that should be created using Datapoints.
	//
	// allowExisting specifies whether to allow existing datapoints as part of the request. If true, datapoints that already
	// exist will be ignored, and the number of such existing datapoints will be returned in the response. If false, any
	// datapoints that already exist will result in an error. Setting this to true is useful for achieving idempotency (e.g.
	// allowing re-ingestion of datapoints that have already been ingested in the past).
	Ingest(ctx context.Context, collectionID uuid.UUID, datapoints any, allowExisting bool) (*IngestResponse, error)

	// Delete datapoints from a collection.
	//
	// The datapoints are identified by their IDs.
	//
	// Returns the number of deleted datapoints.
	Delete(ctx context.Context, collectionID uuid.UUID, datapoints any) (int64, error)

	// DeleteIDs deletes datapoints from a collection by their IDs.
	//
	// Returns the number of deleted datapoints.
	DeleteIDs(ctx context.Context, collectionID uuid.UUID, datapointIDs []uuid.UUID) (int64, error)
}

type Dataset

type Dataset struct {
	// ID is the unique identifier of the dataset.
	ID uuid.UUID
	// Type is the type of the dataset.
	Type *datasetsv1.AnnotatedType
	// Name is the name of the dataset.
	Name string
	// Description is a short description of the dataset.
	Description string
	// Slug is the unique slug of the dataset.
	Slug string
}

Dataset represents a Tilebox Dataset.

Documentation: https://docs.tilebox.com/datasets/concepts/datasets

func (Dataset) String

func (d Dataset) String() string

type DatasetClient

type DatasetClient interface {
	// Get returns a dataset by its slug, e.g. "open_data.copernicus.sentinel1_sar".
	Get(ctx context.Context, slug string) (*Dataset, error)

	// List returns a list of all available datasets.
	List(ctx context.Context) ([]*Dataset, error)
}

type DatasetService

type DatasetService interface {
	GetDataset(ctx context.Context, slug string) (*datasetsv1.Dataset, error)
	ListDatasets(ctx context.Context) (*datasetsv1.ListDatasetsResponse, error)
}

type IngestResponse

type IngestResponse struct {
	// NumCreated is the number of datapoints that were created.
	NumCreated int64
	// NumExisting is the number of datapoints that were ignored because they already existed.
	NumExisting int64
	// DatapointIDs is the list of all the datapoints IDs in the same order as the datapoints in the request.
	DatapointIDs []uuid.UUID
}

IngestResponse contains the response from the Ingest method.

type QueryOption

type QueryOption func(*queryOptions)

QueryOption is an interface for configuring a Query request.

func WithSkipData

func WithSkipData() QueryOption

WithSkipData skips the data when querying datapoints. It is an optional flag for omitting the actual datapoint data from the response. If set, only the required datapoint fields will be returned.

Defaults to false.

func WithSpatialExtent

func WithSpatialExtent(geometry orb.Geometry) QueryOption

WithSpatialExtent specifies the geographical extent in which to query data. Optional, if not specified the query will return all results found globally. Convenience wrapper for WithSpatialExtentFilter applying a default spatial filter mode and coordinate system.

func WithSpatialExtentFilter added in v0.1.0

func WithSpatialExtentFilter(spatialExtent query.SpatialExtent) QueryOption

WithSpatialExtentFilter specifies a geographical extent as well as an intersection mode and coordinate system to use for spatial filtering when querying data.

func WithTemporalExtent

func WithTemporalExtent(temporalExtent query.TemporalExtent) QueryOption

WithTemporalExtent specifies the time interval for which data should be queried. Right now, a temporal extent is required for every query.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL