db

package
v0.0.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 31, 2024 License: Apache-2.0 Imports: 7 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func CombineQueryWithContext

func CombineQueryWithContext(query string, retrievedDocs []Document) string

CombineQueryWithContext combines the user's query with the relevant retrieved documents' content

func ConvertEmbeddingToPGVector

func ConvertEmbeddingToPGVector(embedding []float32) string

ConvertEmbeddingToPGVector converts a slice of float32 values representing an embedding into a string format compatible with PostgreSQL's vector type. The resulting string is a comma-separated list of values enclosed in curly braces, with each value formatted to 6 decimal places of precision.

func ConvertMetadata

func ConvertMetadata(metadata map[string]string) map[string]interface{}

ConvertMetadata converts a map of string keys and string values to a map of string keys and interface{} values. This is useful when working with metadata that needs to be stored in a more flexible format.

Types

type Document

type Document struct {
	ID       string
	Metadata map[string]interface{}
}

Document represents a single document in the vector database. It contains a unique identifier and associated metadata.

type PGVector

type PGVector struct {
	// contains filtered or unexported fields
}

PGVector represents a connection to a PostgreSQL database with pgvector extension. It provides methods for storing and querying vector embeddings.

func NewPGVector

func NewPGVector(connString string) (*PGVector, error)

NewPGVector creates a new PGVector instance with a connection to the PostgreSQL database.

Parameters:

  • connString: A string containing the connection details for the PostgreSQL database.

Returns:

  • A pointer to a new PGVector instance.
  • An error if the connection fails, nil otherwise.

func (*PGVector) Close

func (pg *PGVector) Close()

Close closes the PostgreSQL connection pool.

func (*PGVector) InsertDocument

func (pg *PGVector) InsertDocument(ctx context.Context, content string, embedding []float32) error

InsertDocument inserts a document into the PGVector store, implementing the VectorDatabase interface.

func (*PGVector) QueryRelevantDocuments

func (pg *PGVector) QueryRelevantDocuments(ctx context.Context, embedding []float32, backend string) ([]Document, error)

QueryRelevantDocuments retrieves the most relevant documents from the database based on the given embedding. It uses cosine similarity to find the closest matches and returns a slice of Document structs.

Parameters:

  • ctx: The context for the database query.
  • embedding: A slice of float32 values representing the query embedding.

Returns:

  • A slice of Document structs containing the most relevant documents.
  • An error if the query fails or if there's an issue scanning the results.

func (*PGVector) SaveEmbeddings

func (pg *PGVector) SaveEmbeddings(ctx context.Context, docID string, embedding []float32, metadata map[string]interface{}) error

SaveEmbedding stores a document embedding and associated metadata in the database.

Parameters:

  • ctx: The context for the database operation.
  • docID: A unique identifier for the document.
  • embedding: A slice of float32 values representing the document's embedding.
  • metadata: A map of additional information associated with the document.

Returns:

  • An error if the saving operation fails, nil otherwise.

SaveEmbeddings stores a document embedding and associated metadata in the PostgreSQL database, implementing the VectorDatabase interface.

type QdrantVector

type QdrantVector struct {
	// contains filtered or unexported fields
}

QdrantVector represents a connection to Qdrant.

func NewQdrantVector

func NewQdrantVector(address string, port int) (*QdrantVector, error)

NewQdrantVector initializes a connection to Qdrant.

Parameters:

  • address: The Qdrant server address (e.g., "localhost").
  • port: The port Qdrant is running on (e.g., 6333).

Returns:

  • A pointer to a new QdrantVector instance.
  • An error if the connection fails, nil otherwise.

func (*QdrantVector) Close

func (qv *QdrantVector) Close()

Close closes the Qdrant client connection.

func (*QdrantVector) CreateCollection

func (qv *QdrantVector) CreateCollection(ctx context.Context, collectionName string, vectorSize uint64, distance string) error

CreateCollection creates a new collection in Qdrant

func (*QdrantVector) InsertDocument

func (qv *QdrantVector) InsertDocument(ctx context.Context, content string, embedding []float32, collection string) error

InsertDocument inserts a document into the Qdrant vector store.

Parameters:

  • ctx: Context for the operation.
  • vectorDB: A QdrantVector instance.
  • content: The document content to be inserted.
  • embedding: The embedding vector for the document.

Returns:

  • An error if the operation fails, nil otherwise.

QdrantVector should implement the InsertDocument method as defined in VectorDatabase

func (*QdrantVector) QueryRelevantDocuments

func (qv *QdrantVector) QueryRelevantDocuments(
	ctx context.Context, embedding []float32, collection string, queryOpts ...QueryOpt,
) ([]Document, error)

QueryRelevantDocuments retrieves the most relevant documents based on a given embedding.

Parameters:

  • ctx: The context for the query.
  • embedding: The query embedding.
  • limit: The number of documents to return.
  • collection: The collection name to query.

Returns:

  • A slice of QDrantDocument structs containing the most relevant documents.
  • An error if the query fails.

func (*QdrantVector) SaveEmbeddings

func (qv *QdrantVector) SaveEmbeddings(ctx context.Context, docID string, embedding []float32, metadata map[string]interface{}, collection string) error

SaveEmbedding stores an embedding and metadata in Qdrant.

Parameters:

  • ctx: Context for the operation.
  • docID: A unique identifier for the document.
  • embedding: A slice of float32 values representing the document's embedding.
  • metadata: A map of additional information associated with the document.

Returns:

  • An error if the saving operation fails, nil otherwise.

SaveEmbeddings stores an embedding and metadata in Qdrant, implementing the VectorDatabase interface.

type QueryOpt

type QueryOpt func(*qdrant.QueryPoints)

QueryOpt represents an option for a query. This is the type that should be returned from query options functions.

func WithLimit

func WithLimit(limit uint64) QueryOpt

WithLimit sets the limit of the number of documents to return in a query.

func WithScoreThreshold

func WithScoreThreshold(threshold float32) QueryOpt

WithScoreThreshold sets the score threshold for a query. The higher the threshold, the more relevant the results.

type VectorDatabase

type VectorDatabase interface {
	InsertDocument(ctx context.Context, content string, embedding []float32) error
	QueryRelevantDocuments(ctx context.Context, embedding []float32, backend string) ([]Document, error)
	SaveEmbeddings(ctx context.Context, docID string, embedding []float32, metadata map[string]interface{}) error
}

VectorDatabase is the interface that both QdrantVector and PGVector implement

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL