object_store

Overview ¶

Command object_store writes and scans a Lance dataset on any object store. The destination and credentials come from the environment:

LANCE_URI              dataset URI, e.g. s3://bucket/path/data.lance,
                       az://container/path/data.lance,
                       gs://bucket/path/data.lance (or a local path)
LANCE_STORAGE_OPTIONS  comma-separated key=value storage options passed
                       to the object store, e.g.
                       "access_key_id=...,secret_access_key=...,endpoint=http://localhost:8333,region=us-east-1,allow_http=true"

Example against the SeaweedFS emulator from `make object-store-up`:

LANCE_URI=s3://lance-test/example/data.lance \
LANCE_STORAGE_OPTIONS="access_key_id=lance-access-key,secret_access_key=lance-secret-key,endpoint=http://localhost:8333,region=us-east-1,allow_http=true,virtual_hosted_style_request=false" \
go run ./examples/object_store