examples/

directory
v0.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 17, 2026 License: Apache-2.0

README

lance-go examples

Small runnable programs demonstrating the lance-go API. You need the native library first. From a checkout of this repo, build it from source:

make rust   # from the repository root

Use make rust, not make artifacts / scripts/download-artifacts.sh, to run the in-repo examples: prebuilt release artifacts only match published releases, so on a development checkout they can lack this branch's new exports, and the download script overwrites the checked-in include/lance_go.h, which then fails make header-check. The prebuilt- artifacts path (make artifacts) is for external consumers of the published lance-go module — see the main README.

Then run any example with go run ./examples/<name>.

  • write_scan: write a dataset to the local filesystem, scan it back, and run a filtered scan.
  • vector_search: build an IVF_PQ vector index and run nearest-neighbor searches, including a prefiltered search.
  • fts: build an inverted (full-text search) index and query it with MatchQuery and PhraseQuery.
  • versioning: append new versions, time-travel with checkout, and manage tags.
  • object_store: write/scan a dataset on any object store. Takes LANCE_URI and LANCE_STORAGE_OPTIONS from the environment (works with the emulators from make object-store-up).
  • sql_query: run SQL queries (filter/project and aggregate) over a dataset with Dataset.SQL.
  • take_blobs: write a Lance blob column (LargeBinary tagged lance-encoding:blob=true) and read blob bytes back with TakeBlobs: whole blobs, sub-ranges, and cursor seeks.
  • distributed_index: distributed write (two workers WriteFragments → batch commit) and distributed index build (per-fragment segments → merge → commit), then a vector search over the committed index.
  • plugincache: the cache building blocks: a CacheBackend (external index cache on a Session) and an ObjectStoreCache (byte-range cache), both as swappable Go interfaces. See docs/caching.md.
  • observability: attach OpenTelemetry providers via WithWriteObservability/WithObservability and print the emitted spans and metrics with the stdout exporters. Swap the exporter for OTLP to ship the same telemetry to Datadog with no change to lance-go.
  • schema_evolution: evolve a dataset's schema with AddColumnsSQL, AlterColumns (rename + cast), DropColumns, and Merge, printing the schema after each step.
  • maintenance: create a dataset with several small appends (many fragments), then compact them with CompactFiles and reclaim old versions with CleanupOldVersions, printing fragment/version counts before and after.

Records written through lance.Write must be allocated with a C-backed Arrow allocator (lance.Allocator()), because their buffers are exported across the Arrow C Data Interface. All examples do this.

Directories

Path Synopsis
Command distributed_index demonstrates the v0.2 distributed write and distributed index-build workflow:
 Command distributed_index demonstrates the v0.2 distributed write and distributed index-build workflow:
Command fts builds an inverted (full-text search) index over a small document corpus and runs match and phrase queries against it.
 Command fts builds an inverted (full-text search) index over a small document corpus and runs match and phrase queries against it.
Command maintenance demonstrates dataset maintenance: creating a dataset with several small appends (which leaves many small fragments), compacting them into fewer, larger fragments with CompactFiles, then reclaiming the storage held by old versions with CleanupOldVersions.
 Command maintenance demonstrates dataset maintenance: creating a dataset with several small appends (which leaves many small fragments), compacting them into fewer, larger fragments with CompactFiles, then reclaiming the storage held by old versions with CleanupOldVersions.
Command object_store writes and scans a Lance dataset on any object store.
 Command object_store writes and scans a Lance dataset on any object store.
Command observability demonstrates lance-go's OpenTelemetry instrumentation.
 Command observability demonstrates lance-go's OpenTelemetry instrumentation.
Command plugincache demonstrates the v0.2 cache building blocks: a Go lance.CacheBackend (the external index-cache store attached to a Session) and a lance.ObjectStoreCache (a byte-range cache for immutable file reads).
 Command plugincache demonstrates the v0.2 cache building blocks: a Go lance.CacheBackend (the external index-cache store attached to a Session) and a lance.ObjectStoreCache (a byte-range cache for immutable file reads).
Command schema_evolution walks through Lance's schema evolution operations: adding a computed column with AddColumnsSQL, renaming and casting columns with AlterColumns, dropping a column with DropColumns, and joining in new columns with Merge.
 Command schema_evolution walks through Lance's schema evolution operations: adding a computed column with AddColumnsSQL, renaming and casting columns with AlterColumns, dropping a column with DropColumns, and joining in new columns with Merge.
Command sql_query writes a small dataset and runs SQL queries over it with Dataset.SQL (DataFusion under the hood).
 Command sql_query writes a small dataset and runs SQL queries over it with Dataset.SQL (DataFusion under the hood).
Command take_blobs writes a dataset with a Lance blob column (a LargeBinary column tagged lance-encoding:blob=true), then reads blob bytes back with Dataset.TakeBlobs (whole blobs, sub-ranges, and cursor seeks) without materializing them into a scan.
 Command take_blobs writes a dataset with a Lance blob column (a LargeBinary column tagged lance-encoding:blob=true), then reads blob bytes back with Dataset.TakeBlobs (whole blobs, sub-ranges, and cursor seeks) without materializing them into a scan.
Command vector_search builds an IVF_PQ vector index over a small dataset and runs nearest-neighbor searches, including a prefiltered one.
 Command vector_search builds an IVF_PQ vector index over a small dataset and runs nearest-neighbor searches, including a prefiltered one.
Command versioning demonstrates Lance's versioning features: every write commits a new version, old versions stay readable (time travel), and tags give versions stable names.
 Command versioning demonstrates Lance's versioning features: every write commits a new version, old versions stay readable (time travel), and tags give versions stable names.
Command write_scan writes a Lance dataset to the local filesystem, scans it back in full, and runs a filtered + projected scan.
 Command write_scan writes a Lance dataset to the local filesystem, scans it back in full, and runs a filtered + projected scan.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.