aillm

package
v1.2.15 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 27, 2025 License: Apache-2.0 Imports: 34 Imported by: 0

Documentation

Overview

Copyright (c) 2025 Reza Arani

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at 

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Copyright (c) 2025 Reza Arani

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at 

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Copyright (c) 2025 Reza Arani

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at 

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Copyright (c) 2025 Reza Arani

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at 

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Copyright (c) 2025 Reza Arani

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at 

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Copyright (c) 2025 Reza Arani

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at 

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Copyright (c) 2025 Reza Arani

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at 

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Copyright (c) 2025 Reza Arani

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at 

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Copyright (c) 2025 Reza Arani

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at 

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Copyright (c) 2025 Reza Arani

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at 

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Copyright (c) 2025 Reza Arani

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at 

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Copyright (c) 2025 Reza Arani

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at 

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Copyright (c) 2025 Reza Arani

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at 

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Copyright (c) 2025 Reza Arani

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at 

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Index

Constants

View Source 
const (
	NotDefinedSearch  = 0 // Default search - will use system default
	SimilaritySearch  = 1 // Cosine similarity search only
	KNearestNeighbors = 2 // K-Nearest Neighbors search only
	NoSearch          = 3 // No search performed
	HybridSearch      = 4 // Hybrid search (vector + lexical search with RRF)
	LexicalSearch     = 5 // Lexical/keyword search only
	SemanticSearch    = 6 // Enhanced semantic search (auto-selects best algorithm)

)

Variables

This section is empty.

Functions

This section is empty.

Types

type AillmTools added in v1.0.4 

type AillmTools struct {
	Handlers map[string]func(interface{}) (string, error)
	Tools    []llms.Tool
}

LangchainGo tools plus handlers

type ChatCompletionRequest added in v1.0.3 

type ChatCompletionRequest struct {
	Model            string            `json:"model"`
	Messages         []Message         `json:"messages"`
	Temperature      float64           `json:"temperature"`
	MaxTokens        int               `json:"max_completion_tokens"`
	TopP             float64           `json:"top_p"`
	FrequencyPenalty float64           `json:"frequency_penalty"`
	PresencePenalty  float64           `json:"presence_penalty"`
	ResponseFormat   map[string]string `json:"response_format"`
}

Structures needed for the request

type ChatCompletionResponse added in v1.0.3 

type ChatCompletionResponse struct {
	ID      string   `json:"id"`
	Object  string   `json:"object"`
	Created int64    `json:"created"`
	Model   string   `json:"model"`
	Choices []Choice `json:"choices"`
	Usage   Usage    `json:"usage"`
}

Structures needed for the response

type Choice added in v1.0.3 

type Choice struct {
	Index        int          `json:"index"`
	Message      ReplyMessage `json:"message"`
	FinishReason string       `json:"finish_reason"`
}

type EmbeddingClient 

type EmbeddingClient interface {
	// NewEmbedder initializes and returns an embedding model instance.
	NewEmbedder() (embeddings.Embedder, error)
	// contains filtered or unexported methods
}

EmbeddingClient defines an interface to abstract embedding model clients.

This interface provides methods to initialize and retrieve an embedding model, ensuring a standard contract for different embedding providers such as Ollama and OpenAI.

Methods:

  • NewEmbedder(): Initializes and returns an embedding model instance, or an error if the operation fails.
  • initialized(): Checks if the embedding model has been initialized and is ready for use.

type EmbeddingConfig 

type EmbeddingConfig struct {
	ChunkSize    int // Size of each text chunk for embedding
	ChunkOverlap int // Number of overlapping characters between chunks
}

EmbeddingConfig holds the configuration settings for text chunking during embedding operations.

Fields:

  • ChunkSize: The size of each chunk to be created when splitting text for embedding purposes.
  • ChunkOverlap: The number of overlapping characters between consecutive chunks to maintain context.

type HybridSearchConfig added in v1.2.10 

type HybridSearchConfig struct {
	VectorWeight    float64 // Weight for vector similarity (0.0 to 1.0)
	LexicalWeight   float64 // Weight for lexical search (0.0 to 1.0)
	MinVectorScore  float32 // Minimum vector similarity score
	MinLexicalScore float32 // Minimum lexical relevance score
	UseRRF          bool    // Use Reciprocal Rank Fusion instead of weighted scoring
	RRFConstant     float64 // Constant for RRF calculation (default 60)
	MaxResults      int     // Maximum number of results to return
}

HybridSearchConfig contains configuration for hybrid search

func DefaultHybridSearchConfig added in v1.2.10 

func DefaultHybridSearchConfig() HybridSearchConfig

DefaultHybridSearchConfig returns default configuration for hybrid search

type HybridSearchResult added in v1.2.10 

type HybridSearchResult struct {
	Document     schema.Document
	VectorScore  float64
	LexicalScore float64
	HybridScore  float64
	SearchType   string
}

HybridSearchResult represents a result from hybrid search with combined scores

type LLMAction 

type LLMAction struct {
	Action    interface{} `json:"action"`
	TimeStamp time.Time   `json:"timestamp"`
}

Each action should be a timestamp for benchmarking or output management

type LLMCallOption 

type LLMCallOption func(*LLMCallOptions)

LLMCallOption is a function that configures a LLMCallOptions.

type LLMCallOptions 

type LLMCallOptions struct {
	// StreamingFunc is a function to be called for each chunk of a streaming response.
	// Return an error to stop streaming early.
	StreamingFunc  func(ctx context.Context, chunk []byte) error `json:"-"`
	ActionCallFunc func(action LLMAction)                        `json:"-"`
	Prefix         string
	Index          string
	Language       string
	SessionID      string
	ExtraContext   string
	ExactPrompt    string

	LimitGeneralEmbedding bool
	CotextCleanup         bool

	PersistentMemory     bool
	MaxTokens            int
	LanguageChannel      chan<- string
	ToolCallChannel      chan<- ToolCallStep
	ForceLanguage        bool
	AllowHallucinate     bool
	Tools                AillmTools
	ForceLLMToAnswerLong bool
	UseLLMToSplitText    bool
	IncludeDate          bool
	RagReferences        bool
	SearchAlgorithm      int
	// contains filtered or unexported fields
}

type LLMClient 

type LLMClient interface {
	// NewLLMClient initializes and returns an LLM model instance.
	NewLLMClient() (llms.Model, error)
	GetConfig() LLMConfig
}

LLMClient defines an interface for creating a new LLM (Large Language Model) client instance.

Methods:

  • NewLLMClient(): Creates and returns an instance of an LLM model, or returns an error if the initialization fails.
  • GetConfig(): returns configuration of current instance of LLM model.

type LLMConfig 

type LLMConfig struct {
	Apiurl   string // API endpoint for the LLM service
	AiModel  string // Name of the AI model to be used
	APIToken string // API key required for authorization (e.g., for OpenAI or OVHCloud)
}

LLMConfig struct holds configuration details for the embedding and AI model service.

This struct is used to store the necessary configuration parameters needed to interact with LLM (Large Language Model) services, including API endpoint URLs, model names, and authentication credentials.

Fields:

  • Apiurl: The API endpoint URL of the LLM service for sending requests.
  • AiModel: The specific AI model to be used for embedding or inference operations.
  • APIToken: Authentication token required to access the API (e.g., for OpenAI services).

type LLMContainer 

type LLMContainer struct {
	Embedder                            EmbeddingClient // Embedding client to handle text processing
	EmbeddingConfig                     EmbeddingConfig // Configuration for text chunking
	LLMClient                           LLMClient       // AI model client for generating responses
	VisionClient                        LLMClient       // AI model client for image vision responses
	MemoryManager                       *MemoryManager  // Session-based memory management
	LLMModelLanguageDetectionCapability bool            // Language detection capability flag

	AnswerLanguage          string           // Default answer language - will be ignored if  LLMModelLanguageDetectionCapability = true
	RedisClient             RedisClient      // Redis client for caching and retrieval
	SearchAlgorithm         int              // Semantic search algorithm Cosine Similarity or The k-nearest neighbors
	Temperature             float64          // Controls randomness of model output
	TopP                    float64          // Probability threshold for response diversity
	ScoreThreshold          float32          // Threshold for RAG-based responses
	RagRowCount             int              // Number of RAG rows to retrieve for context
	AllowHallucinate        bool             // Enables/disables AI-generated responses when data is
	FallbackLanguage        string           // Default language fallback
	NoRagErrorMessage       string           // Message shown when RAG results are empty
	NotRelatedAnswer        string           // Predefined response for unrelated queries
	Character               string           // AI assistant's character/personality settings
	Transcriber             Transcriber      // Responsible for processing and transcribing content
	PersistentMemoryManager PersistentMemory // Advanced Memory manager controller
	ShowWarnings            bool             // Mute warnings
	// contains filtered or unexported fields
}

LLMContainer serves as the main struct that manages LLM operations, embedding configurations, and data storage.

It acts as a container for managing various components required for interacting with an AI model, embedding data, and handling queries and responses.

Fields:

  • Embedder: The embedding client responsible for processing and storing text embeddings.
  • EmbeddingConfig: Configuration settings for text chunking operations.
  • LLMClient: The LLM client that provides access to the AI model for generating responses.
  • MemoryManager: A memory management component that stores session-related data.
  • LLMModelLanguageDetectionCapability: A boolean indicating if the model supports automatic language detection.
  • AnswerLanguage: The preferred language for responses from the model.
  • RedisClient: Redis client for caching embeddings and retrieval operations.
  • Temperature: Controls the randomness of the AI's responses (lower values = more deterministic).
  • TopP: Probability threshold for response generation (higher values = more diverse responses).
  • ScoreThreshold: The similarity threshold for retrieval-augmented generation (RAG).
  • RagRowCount: The number of RAG rows to retrieve and analyze for context.
  • AllowHallucinate: Determines if the model can generate responses without relevant data (true/false).
  • FallbackLanguage: The default language to use if the primary language is unavailable.
  • NoRagErrorMessage: The error message to display if no relevant data is found during retrieval.
  • NotRelatedAnswer: A predefined response when the model cannot find relevant information.
  • Character: A personality trait or characteristic assigned to the AI assistant (e.g., formal, friendly).
  • Transcriber: Component responsible for converting speech or text inputs into usable data.

func (*LLMContainer) AskLLM 

func (llm *LLMContainer) AskLLM(Query string, options ...LLMCallOption) (LLMResult, error)

AskLLM processes a user query and retrieves an AI-generated response using Retrieval-Augmented Generation (RAG).

This function supports multi-step processes:

  • Retrieves session memory to provide context for the query.
  • Uses a semantic search algorithm (Cosine Similarity or K-Nearest Neighbors) to fetch relevant documents.
  • Constructs the query for the LLM based on user input and past interactions.
  • Calls the LLM to generate a response, optionally streaming results to a callback function.
  • Updates session memory with the query if relevant documents are found.

Parameters:

  • Query: The user's input query.
  • options: A variadic parameter to specify additional options like session ID, language, and streaming functions.

Returns:

  • LLMResult: Struct containing the AI-generated response, retrieved documents, session memory, and logged actions.
  • error: An error if the query fails or if essential components are missing.

func (*LLMContainer) CleanEmbeddings added in v1.1.6 

func (llm *LLMContainer) CleanEmbeddings(Confirm, prefix string) error

cleanEmbeddings cleans the embeddings from the Redis database.

Parameters:

  • Confirm: The confirmation string to clean the embeddings.
  • prefix: The prefix of the embeddings to clean.
  • index: The index of the embeddings to clean.

Returns:

  • error: An error if the cleaning fails.

func (*LLMContainer) CosineSimilarity 

func (llm *LLMContainer) CosineSimilarity(prefix, Query string, rowCount int, ScoreThreshold float32) ([]schema.Document, error)

CosineSimilarity searches for similar documents in the vector store using cosine similarity.

This function initializes the embedding model, sets up a Redis-based vector store, and performs a similarity search based on the provided query.

Parameters:

  • prefix: A string prefix used to organize and identify related vector entries.
  • Query: The query string to search for similar documents.
  • rowCount: The number of results to retrieve from the vector store.
  • ScoreThreshold: The minimum similarity score threshold for the results.

Returns:

  • interface{}: The search results containing the most similar documents.
  • error: An error if the search fails or the embedding model is missing.

func (*LLMContainer) DescribeImage 

func (llm *LLMContainer) DescribeImage(encodedImage, query string, options ...LLMCallOption) (ChatCompletionResponse, error)

func (*LLMContainer) DescribeImageFromFile 

func (llm *LLMContainer) DescribeImageFromFile(imagePath, query string, options ...LLMCallOption) (ChatCompletionResponse, error)

DescribeImageFromFile reads an image file, encodes it to Base64, and sends it along with a text query to an AI vision model for description.

This function reads an image file from the specified path, converts it to a Base64-encoded string, and calls the `DescribeImage` function to obtain a textual description of the image.

Parameters:

  • imagePath: The file path of the image to be processed.
  • query: A text query to describe the image (e.g., "Describe this image.").
  • options: Variadic LLMCallOption parameters for additional configuration (if applicable).

Returns:

  • string: The textual description generated by the AI model.
  • error: An error if the file cannot be read, encoding fails, or the AI request encounters an issue.

Example Usage: 

description, err := llm.DescribeImageFromFile("sample.jpg", "Describe this image.")
if err != nil {
    log.Fatalf("Error: %v", err)
}
fmt.Println("Image Description:", description)

Notes:

  • The function reads the image as a binary file using `os.ReadFile()`.
  • It converts the image to a Base64-encoded string before calling `DescribeImage()`.
  • Requires a valid API token and properly configured `llm.VisionClient` settings.
  • If the image file is too large, encoding may consume significant memory.

func (LLMContainer) EmbeddFile 

func (llm LLMContainer) EmbeddFile(Index, Title, fileName string, tc TranscribeConfig, options ...LLMCallOption) (LLMEmbeddingObject, error)

EmbeddFile processes and embeds the content of a given file into the LLM system.

Parameters:

  • ObjectId: A unique identifier for the embedding object.
  • Title: The Title of the document being embedded. Also it will be used for raw data for a better Context
  • Index: The Index of the document being embedded. Also it will be used for the raw file redis key
  • fileName: The path to the file to be embedded.
  • tc: Configuration for transcription, such as language settings.

Returns:

  • LLMEmbeddingObject: The embedded object containing the processed content.
  • error: An error if any issues occur during processing.

func (*LLMContainer) EmbeddText 

func (llm *LLMContainer) EmbeddText(Index string, Contents LLMEmbeddingContent, options ...LLMCallOption) (LLMEmbeddingObject, error)

EmbeddText embeds provided text content into the LLM system and stores the data in Redis.

Parameters:

  • ObjectId: Unique identifier for the embedding object.
  • Index: Index associated with the content being embedded.
  • Contents: A map containing language-specific content to be embedded.

Returns:

  • LLMEmbeddingObject: The resulting embedding object after processing and storage.
  • error: An error if any issues occur during embedding or Redis operations.

func (LLMContainer) EmbeddURL 

func (llm LLMContainer) EmbeddURL(Index, url string, tc TranscribeConfig, options ...LLMCallOption) (LLMEmbeddingObject, error)

EmbeddURL processes and embeds content from a given URL into the LLM system.

Parameters:

  • ObjectId: A unique identifier for the embedding object.
  • Index: The Index associated with the content being embedded.
  • url: The web URL from which content will be transcribed and embedded.
  • tc: Configuration for transcription, including language and extraction options.

Returns:

  • LLMEmbeddingObject: The embedded object containing the processed content.
  • error: An error if any issues occur during the transcription or embedding process.

func (*LLMContainer) FindKNN 

func (llm *LLMContainer) FindKNN(prefix, searchQuery string, rowCount int, ScoreThreshold float32) ([]schema.Document, error)

FindKNN performs a K-Nearest Neighbors (KNN) search on the stored vector embeddings.

This function retrieves the most relevant documents based on the provided query, using the KNN algorithm to rank them according to their proximity in the vector space.

Parameters:

  • prefix: A string prefix used to identify relevant vector entries.
  • searchQuery: The query string to find the nearest neighbors for.
  • rowCount: The number of closest neighbors to retrieve.
  • ScoreThreshold: The minimum similarity score for considering results.

Returns:

  • []schema.Document: The retrieved relevant documents.
  • error: An error if the search fails or the embedding model is missing.

func (*LLMContainer) GetQueryLanguage added in v1.0.6 

func (llm *LLMContainer) GetQueryLanguage(Query, sessionId string, languageChannel chan<- string) (string, TokenUsage, error)

GetQueryLanguage detects user query language with optimized client reuse

func (*LLMContainer) GetRagIndexs added in v1.0.6 

func (llm *LLMContainer) GetRagIndexs(docs []schema.Document, options ...LLMCallOption) ([]string, error)

GetRagIndexs retrieves the Redis index values for the given documents.

Parameters:

  • docs: A slice of schema.Document objects containing the documents to search for.
  • options: Additional options for the search operation.

Returns:

  • []string: A slice of Redis index values.
  • error: An error if the operation fails.

func (*LLMContainer) HybridSearch added in v1.2.10 

func (llm *LLMContainer) HybridSearch(prefix, searchQuery string, rowCount int, ScoreThreshold float32, config *HybridSearchConfig) ([]schema.Document, error)

HybridSearch performs a hybrid search combining vector similarity and lexical search for improved accuracy.

This function combines the power of semantic vector search with traditional keyword-based search to provide more accurate and comprehensive results. It uses techniques like Reciprocal Rank Fusion (RRF) to merge results from both search methods.

Parameters:

  • prefix: A string prefix used to identify relevant vector entries.
  • searchQuery: The query string to search for.
  • rowCount: The number of results to retrieve.
  • ScoreThreshold: The minimum similarity score threshold for results.
  • config: Configuration for hybrid search behavior.

Returns:

  • []schema.Document: The retrieved relevant documents with hybrid scores.
  • error: An error if the search fails or required components are missing.

func (*LLMContainer) Init 

func (llm *LLMContainer) Init() error

Init initializes the LLMContainer by configuring memory management, embedding settings, transcriber configurations, and connecting to the Redis database.

This function sets default parameters for temperature, token threshold, RAG configurations, fallback responses, and initializes essential components like the memory manager and transcriber.

Returns:

  • error: An error if Redis configuration is missing or if the Redis connection fails.

func (*LLMContainer) InitEmbedding 

func (llm *LLMContainer) InitEmbedding() error

InitEmbedding initializes the embedding model based on the type of embedding provider.

This function checks the type of the configured embedding provider (Ollama or OpenAI) and initializes it with the appropriate configuration settings.

Returns:

  • error: Returns an error if initialization fails or if the provider is unsupported.

func (*LLMContainer) IsQuerySafe added in v1.2.9 

func (llm *LLMContainer) IsQuerySafe(Query string, debug bool) (bool, TokenUsage, string, error)

func (*LLMContainer) ListEmbeddings 

func (llm *LLMContainer) ListEmbeddings(KeyID string, offset, limit int) (map[string]interface{}, error)

List retrieves multiple embedding objects from Redis with pagination support.

Parameters:

  • rdb: Redis client instance for database operations.
  • KeyID: The prefix used to filter stored keys in Redis.
  • offset: The starting position for retrieval.
  • limit: The number of items to retrieve.

Returns:

  • map[string]interface{}: A map containing retrieved objects and total count.
  • error: An error if the operation fails.

func (*LLMContainer) RemoveEmbedding 

func (llm *LLMContainer) RemoveEmbedding(Index string, options ...LLMCallOption) error

RemoveEmbedding deletes an embedding object and its associated keys from Redis.

Parameters:

  • ObjectId: The unique identifier for the embedding object to be removed.
  • Index: The Index of the embedding object.

Returns:

  • error: An error if deletion fails.

func (*LLMContainer) RemoveEmbeddingSubKey 

func (llm *LLMContainer) RemoveEmbeddingSubKey(Index, rawDocID string, options ...LLMCallOption) error

func (*LLMContainer) SearchAll 

func (llm *LLMContainer) SearchAll(language string) LLMCallOption

SearchAll specifies the scope of search,

Parameters:

  • language: scope of general search in specific language

Returns:

  • LLMCallOption: An option that sets the embedding prefix.

func (*LLMContainer) SemanticSearch added in v1.2.10 

func (llm *LLMContainer) SemanticSearch(prefix, searchQuery string, rowCount int, ScoreThreshold float32) ([]schema.Document, error)

SemanticSearch performs enhanced semantic search using the best available algorithm

This function automatically selects the most appropriate search algorithm based on the system configuration and query characteristics.

Parameters:

  • prefix: A string prefix used to identify relevant vector entries.
  • searchQuery: The query string to search for.
  • rowCount: The number of results to retrieve.
  • ScoreThreshold: The minimum similarity score threshold for results.

Returns:

  • []schema.Document: The retrieved relevant documents.
  • error: An error if the search fails.

func (*LLMContainer) WithActionCallFunc 

func (llm *LLMContainer) WithActionCallFunc(ActionCallFunc func(action LLMAction)) LLMCallOption

WithActionCallFunc specifies a callback function to log custom actions during LLM query processing.

Parameters:

  • ActionCallFunc: A function defining custom actions to be logged during query processing.

Returns:

  • LLMCallOption: An option to set the custom action callback function.

func (*LLMContainer) WithAllowHallucinate added in v1.0.8 

func (llm *LLMContainer) WithAllowHallucinate(AllowHallucinate bool) LLMCallOption

WithAllowHallucinate allows model to hallucinate

Parameters:

  • AllowHallucinate: A boolean value to update property

Returns:

  • LLMCallOption: An option that sets the query language.

func (*LLMContainer) WithAsyncMemorySummarization added in v1.2.11 

func (llm *LLMContainer) WithAsyncMemorySummarization(asyncMemorySummarization bool) LLMCallOption

WithAsyncMemorySummarization specifies the async memory summarization.

Parameters:

  • asyncMemorySummarization: boolean value of async memory summarization

Returns:

  • LLMCallOption: An option that sets the query language.

func (*LLMContainer) WithCharacter 

func (llm *LLMContainer) WithCharacter(character string) LLMCallOption

WithPersistentMemory enhances the memory by using vector search to create more efficient prompts for conversation memory

Parameters:

  • usePersistentMemory: A boolean value to update property

Returns:

  • LLMCallOption: An option that sets the embedding prefix.

func (*LLMContainer) WithCotextCleanup 

func (llm *LLMContainer) WithCotextCleanup(cleanup bool) LLMCallOption

WithCotextCleanup Cleanup retrieved context, specially html codes to make a clear context

Parameters:

  • cleanup: A boolean value to update property

Returns:

  • LLMCallOption: An option that sets the embedding prefix.

func (*LLMContainer) WithCustomModel added in v1.2.10 

func (llm *LLMContainer) WithCustomModel(customModel string) LLMCallOption

WithCustomModel specifies the custom mode of response.

Parameters:

  • maxTokens: integer value of Maximum allowed tokens

Returns:

  • LLMCallOption: An option that sets the query language.

func (*LLMContainer) WithDebug added in v1.2.10 

func (llm *LLMContainer) WithDebug(debug bool) LLMCallOption

WithDebug enables debug mode

func (*LLMContainer) WithEmbeddingIndex 

func (llm *LLMContainer) WithEmbeddingIndex(Index string) LLMCallOption

func (*LLMContainer) WithEmbeddingPrefix 

func (llm *LLMContainer) WithEmbeddingPrefix(Prefix string) LLMCallOption

WithEmbeddingPrefix specifies a prefix for identifying related embeddings.

Parameters:

  • Prefix: A string prefix used to group or identify embeddings in the store.

Returns:

  • LLMCallOption: An option that sets the embedding prefix.

func (*LLMContainer) WithExactPrompt 

func (llm *LLMContainer) WithExactPrompt(ExactPrompt string) LLMCallOption

WithExtraExactPromot queries LLM with exact promot.

Parameters:

  • ExactPromot: string, prompt

Returns:

  • LLMCallOption: An option that sets the embedding prefix.

func (*LLMContainer) WithExtraContext 

func (llm *LLMContainer) WithExtraContext(ExtraContext string) LLMCallOption

WithExtraExtraContext specifies a extra context for search

Parameters:

  • ExtraContext: Extra provided text to provide LLM.

Returns:

  • LLMCallOption: An option that sets the embedding prefix.

func (*LLMContainer) WithForceLLMToAnswerLong added in v1.1.4 

func (llm *LLMContainer) WithForceLLMToAnswerLong(forceLong bool) LLMCallOption

WithForceLLMToAnswerLong forces LLM to answer long

Parameters:

  • forceLong: A boolean value to update property

Returns:

  • LLMCallOption: An option that sets the query language.

func (*LLMContainer) WithForcedLanguage added in v1.0.6 

func (llm *LLMContainer) WithForcedLanguage(forceUserLanguage bool) LLMCallOption

WithForcedLanguage Forces ExactPrompt output language

Parameters:

  • Language: The language in which the LLM should generate responses.

Returns:

  • LLMCallOption: An option that sets the query language.

func (*LLMContainer) WithHybridSearch added in v1.2.10 

func (llm *LLMContainer) WithHybridSearch() LLMCallOption

WithHybridSearch enables hybrid search combining vector similarity and lexical search

func (*LLMContainer) WithIgnoreSecurityCheck added in v1.2.9 

func (llm *LLMContainer) WithIgnoreSecurityCheck(ignoreSecurityCheck bool) LLMCallOption

WithIgnoreSecurityCheck ignores security check

Parameters:

  • IgnoreSecurityCheck: A boolean value to update property

Returns:

  • LLMCallOption: An option that sets the query language.

func (*LLMContainer) WithIncludeDate added in v1.2.1 

func (llm *LLMContainer) WithIncludeDate(IncludeDate bool) LLMCallOption

WithIncludeDate include date in prompt

Parameters:

  • IncludeDate: A boolean value to update property

Returns:

  • LLMCallOption: An option that sets the query language.

func (*LLMContainer) WithKNNSearch added in v1.2.10 

func (llm *LLMContainer) WithKNNSearch() LLMCallOption

WithKNNSearch enables K-Nearest Neighbors search

func (*LLMContainer) WithLLMSpliter added in v1.1.6 

func (llm *LLMContainer) WithLLMSpliter(UseLLMToSplitText bool) LLMCallOption

WithLLMSpliter uses LLM to split text

Parameters:

  • UseLLMToSplitText: A boolean value to update property

Returns:

  • LLMCallOption: An option that sets the query language.

func (*LLMContainer) WithLanguage 

func (llm *LLMContainer) WithLanguage(Language string) LLMCallOption

WithLanguage specifies the language to use for the query response.

Parameters:

  • Language: The language in which the LLM should generate responses.

Returns:

  • LLMCallOption: An option that sets the query language.

func (*LLMContainer) WithLanguageChannel added in v1.0.4 

func (llm *LLMContainer) WithLanguageChannel(userChannel chan string) LLMCallOption

WithLanguageChannel returns user language and send it to main thread

Parameters:

  • userChannel: AILLM will send language to selected channel for post processing.

Returns:

  • LLMCallOption: An option that sets the query language.

func (*LLMContainer) WithLexicalSearch added in v1.2.10 

func (llm *LLMContainer) WithLexicalSearch() LLMCallOption

WithLexicalSearch enables lexical/keyword search only

func (*LLMContainer) WithLimitGeneralEmbedding 

func (llm *LLMContainer) WithLimitGeneralEmbedding(denied bool) LLMCallOption

WithEmbeddingPrefix specifies a prefix for identifying related embeddings.

Parameters:

  • Prefix: A string prefix used to group or identify embeddings in the store.

Returns:

  • LLMCallOption: An option that sets the embedding prefix.

func (*LLMContainer) WithMaxTokens added in v1.0.3 

func (llm *LLMContainer) WithMaxTokens(maxTokens int) LLMCallOption

WithMaxTokens specifies the maximum tokens of response.

Parameters:

  • maxTokens: integer value of Maximum allowed tokens

Returns:

  • LLMCallOption: An option that sets the query language.

func (*LLMContainer) WithMaxToolCallIteration added in v1.2.15 

func (llm *LLMContainer) WithMaxToolCallIteration(maxToolCallIteration int) LLMCallOption

func (*LLMContainer) WithMaxWords added in v1.2.10 

func (llm *LLMContainer) WithMaxWords(maxWords int) LLMCallOption

WithMaxWords specifies the maximum words of response.

Parameters:

  • maxTokens: integer value of Maximum allowed tokens

Returns:

  • LLMCallOption: An option that sets the query language.

func (*LLMContainer) WithPersistentMemory 

func (llm *LLMContainer) WithPersistentMemory(usePersistentMemory bool) LLMCallOption

WithPersistentMemory enhances the memory by using vector search to create more efficient prompts for conversation memory

Parameters:

  • usePersistentMemory: A boolean value to update property

Returns:

  • LLMCallOption: An option that sets the embedding prefix.

func (*LLMContainer) WithRagReferences added in v1.2.1 

func (llm *LLMContainer) WithRagReferences(RagReferences bool) LLMCallOption

WithIncludeDate include date in prompt

Parameters:

  • IncludeDate: A boolean value to update property

Returns:

  • LLMCallOption: An option that sets the query language.

func (*LLMContainer) WithSearchAlgorithm added in v1.2.8 

func (llm *LLMContainer) WithSearchAlgorithm(SearchAlgorithm int) LLMCallOption

WithSearchAlgorithm specifies the search algorithm to use for the query.

Parameters:

  • SearchAlgorithm: The search algorithm to use for the query.

Returns:

  • LLMCallOption: An option that sets the search algorithm.

func (*LLMContainer) WithSemanticSearch added in v1.2.10 

func (llm *LLMContainer) WithSemanticSearch() LLMCallOption

WithSemanticSearch enables enhanced semantic search (auto-selects best algorithm)

func (*LLMContainer) WithSessionID 

func (llm *LLMContainer) WithSessionID(SessionID string) LLMCallOption

WithSessionID specifies the session ID for tracking user chat history and interactions.

Parameters:

  • SessionID: A unique identifier for the user's session.

Returns:

  • LLMCallOption: An option that sets the session ID.

func (*LLMContainer) WithSimilaritySearch added in v1.2.10 

func (llm *LLMContainer) WithSimilaritySearch() LLMCallOption

WithSimilaritySearch enables cosine similarity search (default)

func (*LLMContainer) WithStreamingFunc 

func (llm *LLMContainer) WithStreamingFunc(streamingFunc func(ctx context.Context, chunk []byte) error) LLMCallOption

WithStreamingFunc specifies a callback function for handling streaming output during query processing.

Parameters:

  • streamingFunc: A function to process streaming chunks of output from the LLM. Returning an error stops the streaming process.

Returns:

  • LLMCallOption: An option to set the streaming function.

func (*LLMContainer) WithToolCallChannel added in v1.2.15 

func (llm *LLMContainer) WithToolCallChannel(toolCallChannel chan ToolCallStep) LLMCallOption

WithToolCallChannel sends each tool calling step to a channel for external monitoring

Parameters:

  • toolCallChannel: AILLM will send tool call information to this channel during iteration.

Returns:

  • LLMCallOption: An option that sets the tool call channel.

func (*LLMContainer) WithTools added in v1.0.4 

func (llm *LLMContainer) WithTools(tools AillmTools) LLMCallOption

WithTools specifies the tools to use for the query.

Parameters:

  • tools: A list of tools to use for the query.

Returns:

  • LLMCallOption: An option that sets the tools.

Experimental - Just works with OpenAI

type LLMEmbeddingContent 

type LLMEmbeddingContent struct {
	Text        string   `json:"Text" redis:"Text"`
	Title       string   `json:"Title" redis:"Title"`
	Language    string   `json:"Language" redis:"Language"`
	Id          string   `json:"Id" redis:"Id"`
	Keys        []string `json:"Keys" redis:"Keys"`
	GeneralKeys []string `json:"GeneralKeys" redis:"GeneralKeys"`
	Keywords    []string `json:"Keywords" redis:"Keywords"`
	Sources     string   `json:"Sources" redis:"Sources"`
}

LLMEmbeddingContent represents a single piece of text content that is embedded and stored in the system.

This struct holds the necessary information for managing and identifying embedded text, including the raw text, a title, its source, and associated keys.

Fields:

  • Text: The raw text content that is embedded.
  • Title: A descriptive title for the embedded content.
  • Index:
  • Source: The origin of the content, such as a file name, URL, or other identifier.
  • Keys: A slice of strings representing the Redis keys associated with this content.

type LLMEmbeddingObject 

type LLMEmbeddingObject struct {
	EmbeddingPrefix string                         `json:"EmbeddingPrefix" redis:"EmbeddingPrefix"`
	Index           string                         `json:"Index" redis:"Index"`
	Contents        map[string]LLMEmbeddingContent `json:"Contents" redis:"Contents"`
}

LLMEmbeddingObject represents a collection of embedded text contents grouped under a specific object ID.

This struct serves as a container for multiple pieces of embedded text content, organized by language or context. It provides a way to store and manage embeddings for a specific use case or document.

Fields:

  • EmbeddingPrefix: A unique prefix or identifier for the embedding object (e.g., "ObjectId").
  • Index: An Index for the embedding object, providing a future access.
  • Contents: A map of language-specific content, where the key is the language code (e.g., "en", "pt") and the value is an LLMEmbeddingContent struct containing the associated content details.

type LLMResult 

type LLMResult struct {
	Prompt          []llms.MessageContent
	RagDocs         []schema.Document
	Response        *llms.ContentResponse
	Memory          []MemoryData
	MemorySummary   string
	Actions         []LLMAction
	LLMReferences   []string
	TokenReport     TokenReport
	FailedToRespond bool
	Warning         string
}

LLMResult represents the result of an LLM query, including the generated response, retrieved documents, and logged actions.

This struct captures the outcome of a query processed by the LLMContainer, storing essential components such as the query prompt, RAG (Retrieval-Augmented Generation) documents, memory updates, and actions logged during the query lifecycle.

Fields:

  • Prompt: A slice of MessageContent representing the constructed query prompt sent to the LLM.
  • RagDocs: An interface containing the retrieved documents (e.g., schema.Document) used in the RAG process.
  • Response: A pointer to the ContentResponse generated by the LLM, containing the AI's output and metadata.
  • Memory: A slice of strings representing session-based memory for context-aware interactions.
  • Actions: A slice of LLMAction structs, each representing a logged action or milestone during the query lifecycle.
  • FailedToRespond: A boolean indicating if the LLM failed to respond.

type LLMTextEmbedding 

type LLMTextEmbedding struct {
	ChunkSize         int
	ChunkOverlap      int
	Text              string
	EmbeddedDocuments []schema.Document
	// contains filtered or unexported fields
}

LLMTextEmbedding is a struct designed to handle text processing and splitting operations.

Fields:

  • ChunkSize: The maximum size of each text chunk after splitting.
  • ChunkOverlap: The number of overlapping characters between consecutive chunks to ensure context retention.
  • Text: The original text content to be processed and split into chunks.
  • EmbeddedDocuments: A slice of schema.Document representing the resulting chunks after processing.

func (*LLMTextEmbedding) SplitText 

func (emb *LLMTextEmbedding) SplitText() ([]schema.Document, error)

SplitText method to split the text into smaller chunks.

This function takes the text stored in the LLMTextEmbedding struct and splits it into smaller chunks based on the specified chunk size and overlap settings. The resulting chunks are stored as schema.Document objects.

Returns:

  • []schema.Document: A slice containing the split document chunks.
  • error: An error if the text splitting process encounters any issues.

func (*LLMTextEmbedding) SplitTextWithLLM added in v1.1.6 

func (emb *LLMTextEmbedding) SplitTextWithLLM() (docs []schema.Document, keywords []string, inconsistentChunks map[int]string, err error)

type Memory 

type Memory struct {
	Questions       []MemoryData // Stores user queries during the session
	MemoryStartTime time.Time    // Timestamp when the session started
	Summary         string       // Summary of the session
}

Memory structure to store user memory session data.

This struct keeps track of a user's questions and the session start time.

Fields:

  • Questions: A slice of strings representing the list of user queries in the session.
  • MemoryStartTime: A timestamp indicating when the session started.

type MemoryData 

type MemoryData struct {
	Question string
	Answer   string
	Keys     []string
	Summary  string
}

Memory structure to store user memory question data.

This struct keeps track of a user's questions and the session data in Redis.

Fields:

  • Questions: A string representing the user query.
  • Answer: A string representing the LLM response to the query.
  • Keys: A slice of strings that keeps keys of Redis vector data related to this question.

type MemoryManager 

type MemoryManager struct {
	// contains filtered or unexported fields
}

MemoryManager manages session memories with a time-to-live (TTL) mechanism.

This struct is responsible for storing user sessions, providing thread-safe access to session data, and cleaning up expired sessions periodically.

Fields:

  • memoryMap: A map storing session ID as the key and Memory struct as the value.
  • mu: A mutex to ensure thread-safe operations on the memory map.
  • ttl: The time-to-live (TTL) duration after which sessions will be removed automatically.

func NewMemoryManager 

func NewMemoryManager(ttlMinutes int) *MemoryManager

NewMemoryManager creates and initializes a new MemoryManager with a specified TTL (Time-To-Live).

This function initializes the session memory map and sets up a background process to periodically remove expired sessions.

Parameters:

  • ttlMinutes: The time-to-live duration in minutes before session data expires.

Returns:

  • *MemoryManager: A pointer to the newly created MemoryManager instance.

func (*MemoryManager) AddMemory 

func (m *MemoryManager) AddMemory(sessionID string, questions []MemoryData)

AddMemory adds or updates a session's memory in the memory map.

This function stores user queries within a session and ensures thread-safe access using a mutex lock to avoid concurrent read/write issues.

Parameters:

  • sessionID: The unique identifier for the user's session.
  • questions: A slice of strings containing user queries.

func (*MemoryManager) DeleteMemory 

func (m *MemoryManager) DeleteMemory(sessionID string)

DeleteMemory removes a user's session memory from the memory map.

This function ensures safe deletion using a mutex lock to prevent data races.

Parameters:

  • sessionID: The unique identifier for the session to be deleted.

func (*MemoryManager) GetMemory 

func (m *MemoryManager) GetMemory(sessionID string) (Memory, bool)

GetMemory retrieves stored session memory for a given session ID.

The function safely reads from the session map and returns the stored memory if it exists.

Parameters:

  • sessionID: The unique identifier for the user's session.

Returns:

  • Memory: The stored session data containing questions and timestamp.
  • bool: A boolean indicating whether the session ID exists in the memory map.

type Message added in v1.0.3 

type Message struct {
	Role    string      `json:"role"`
	Content interface{} `json:"content"`
}

type OllamaController 

type OllamaController struct {
	Config        LLMConfig   // Configuration for the Ollama LLM service
	LLMController *ollama.LLM // Instance of the Ollama LLM client
}

OllamaController struct to manage the Ollama embedding and language model service.

This struct implements the EmbeddingClient interface and acts as a wrapper around the Ollama LLM (Large Language Model), handling initialization and interactions.

Fields:

  • Config: Configuration details such as API URL and model name.
  • LLMController: Instance of the Ollama LLM client for handling AI operations.

func (*OllamaController) GetConfig 

func (oc *OllamaController) GetConfig() LLMConfig

func (*OllamaController) NewEmbedder 

func (oc *OllamaController) NewEmbedder() (embeddings.Embedder, error)

NewEmbedder initializes and returns an Ollama embedding model instance.

This function implements the EmbeddingClient interface to create and return an embedding model using the current LLMController instance.

Returns:

  • embeddings.Embedder: The initialized embedding model instance.
  • error: An error if the initialization fails.

func (*OllamaController) NewLLMClient 

func (oc *OllamaController) NewLLMClient() (llms.Model, error)

NewLLMClient initializes and returns a new instance of the Ollama LLM client.

This function sets up the Ollama model based on the provided API URL and model name in the configuration.

Returns:

  • llms.Model: The initialized LLM model instance.
  • error: An error if the initialization fails.

type OpenAIController 

type OpenAIController struct {
	Config        LLMConfig
	LLMController *openai.LLM
}

OpenAIController struct to manage OpenAI embedding and language model services.

This struct implements the EmbeddingClient interface and acts as a wrapper around the OpenAI LLM (Large Language Model), handling initialization and interactions.

Fields:

  • Config: Configuration details such as API URL, model name, and API token.
  • LLMController: Instance of the OpenAI LLM client for handling AI operations.

func (*OpenAIController) GetConfig 

func (oc *OpenAIController) GetConfig() LLMConfig

func (*OpenAIController) NewEmbedder 

func (oc *OpenAIController) NewEmbedder() (embeddings.Embedder, error)

NewEmbedder initializes and returns an OpenAI embedding model instance.

This function implements the EmbeddingClient interface to create and return an embedding model using the current LLMController instance.

Returns:

  • embeddings.Embedder: The initialized embedding model instance.
  • error: An error if the initialization fails.

func (*OpenAIController) NewLLMClient 

func (oc *OpenAIController) NewLLMClient() (llms.Model, error)

NewLLMClient initializes and returns a new instance of the OpenAI LLM client.

This function sets up the OpenAI model based on the provided API token, API base URL, and the selected AI model from the configuration.

Returns:

  • llms.Model: The initialized LLM model instance.
  • error: An error if the initialization fails.

type PersistentMemory 

type PersistentMemory struct {
	MemoryPrefix          string        //prefix for redis storage
	MemoryTTL             time.Duration // auto delete memory question TTL
	MemorySearchThreshold float32       //Memory vector search Threshold
	HistoryItemCount      int           // More queries = more tokens. adjus it carefully.
	// contains filtered or unexported fields
}

PersistentMemory structure to store user memory session data in a persistent storage (Redis) for future retrival or vector search.

This struct keeps track of a user's questions parameters.

Fields:

  • Questions: A slice of strings representing the list of user queries in the session.
  • MemoryStartTime: A timestamp indicating when the session started.

func (*PersistentMemory) AddMemory 

func (pm *PersistentMemory) AddMemory(sessionID string, query MemoryData) (TokenUsage, error)

Returns:

  • error: An error if the embedding process fails.

func (*PersistentMemory) DeleteMemory 

func (pm *PersistentMemory) DeleteMemory(sessionID string) error

DeleteMemory removes a user's session memory from the memory map.

Parameters:

  • sessionID: The unique identifier for the session to be deleted.

func (*PersistentMemory) GetMemory 

func (pm *PersistentMemory) GetMemory(sessionID string, query string) (MemoryData, Memory, string, []schema.Document, error)

GetMemory retrieves stored session memory for a given session ID.

The function safely reads from the session map and returns the stored memory if it exists.

Parameters:

  • sessionID: The unique identifier for the user's session.
  • query: Will be used for Vector search in user query history to find previous related questions

Returns:

  • MemoryData: Last asked question.
  • string: generated prompt for memory context.
  • error: An error if the memory retrival process fails.

type RedisClient 

type RedisClient struct {
	Host     string // Redis server address
	Password string // Redis authentication password (if applicable)
	// contains filtered or unexported fields
}

RedisClient manages the connection details for a Redis database instance used for storing embeddings.

Fields:

  • Host: The address of the Redis server (e.g., "localhost:6379").
  • Password: The password for connecting to the Redis server (if authentication is required).
  • redisClient: The Redis client instance used for executing operations.

type ReplyMessage added in v1.0.3 

type ReplyMessage struct {
	Role    string `json:"role"`
	Content string `json:"content"`
}

type TokenReport added in v1.2.1 

type TokenReport struct {
	CompletionTokens          TokenUsage
	TextChunkingTokens        TokenUsage
	LanguageDetectionTokens   TokenUsage
	MemorySummarizationTokens TokenUsage
	SecurityCheckTokens       TokenUsage
}

TokenReport represents a comprehensive report of token usage across different components of the LLM process.

This struct captures the number of input and output tokens used in a given context.

Fields:

  • TotalTokens: The total number of tokens used in the entire process.
  • PromptTokens: The number of tokens used in the prompt.
  • CompletionTokens: The number of tokens used in the completion.
  • TextChunkingTokens: The number of tokens used in the text chunking.
  • LanguageDetectionTokens: The number of tokens used in the language detection.
  • MemorySummarizationTokens: The number of tokens used in the memory summarization.

type TokenUsage added in v1.2.1 

type TokenUsage struct {
	InputTokens  int
	OutputTokens int
}

TokenUsage represents the usage of tokens in a specific context.

This struct captures the number of input and output tokens used in a given context.

Fields:

  • InputTokens: The number of input tokens consumed.
  • OutputTokens: The number of output tokens generated.

type ToolCallStep added in v1.2.15 

type ToolCallStep struct {
	EventType     string    `json:"event_type"`     // Type: "prompt", "llm_response", "tool_call"
	Iteration     int       `json:"iteration"`      // Current iteration number (0-based)
	Content       string    `json:"content"`        // Prompt text or LLM response content
	ToolName      string    `json:"tool_name"`      // Name of the tool being called (for tool_call events)
	ToolArguments string    `json:"tool_arguments"` // Arguments passed to the tool (JSON string, for tool_call events)
	ToolResult    string    `json:"tool_result"`    // Result returned from the tool (for tool_call events)
	Timestamp     time.Time `json:"timestamp"`      // When this step occurred
}

ToolCallStep represents a single step in the tool calling iteration It can represent different event types: user prompt, LLM response, or tool execution

type TranscribeConfig 

type TranscribeConfig struct {
	TikaLanguage        string        //PDF ONLY, OCR language code (refer to Tesseract OCR languages) can be found @ https://github.com/tesseract-ocr/tessdata/
	Language            string        // Target language for transcription
	OCROnly             bool          // Perform OCR only, ignoring non-image text
	ExtractInlineImages bool          // Enable extraction of text from inline images
	MaxTimeout          time.Duration // Maximum processing time before timeout
}

type Transcriber 

type Transcriber struct {
	MaxPageLimit uint   // Maximum number of pages allowed for processing
	TikaURL      string // URL of the Apache Tika service for text extraction

	TempFolder string // Path to the temporary folder for storing transcribed files
	// contains filtered or unexported fields
}

Transcriber handles document transcription by extracting text from various file formats.

This struct provides configurations and options for processing documents through the Tika service, managing temporary files, and setting processing limits.

Fields:

  • MaxPageLimit: The maximum number of pages to process in a document (JUST PDF Documents).
  • TikaURL: The URL of the Apache Tika server used for text extraction.
  • initialized: A boolean indicating if the transcriber has been initialized successfully.
  • TempFolder: The folder where temporary files will be stored during processing (Downloading / Transcribing).
  • folderSep: The file path separator used for compatibility across operating systems.

func (*Transcriber) TranscribeURL added in v1.0.6 

func (Ts *Transcriber) TranscribeURL(inputURL string, tc TranscribeConfig) (string, int, error)

transcribeURL downloads and processes content from a given URL.

This function downloads the content from the URL, detects the MIME type, and extracts text based on file type (PDF, HTML, etc.).

Parameters:

  • inputURL: The URL of the document to be transcribed.
  • tc: Transcription configuration settings.

Returns:

  • string: Extracted text content.
  • int: Number of pages processed (if applicable).
  • error: An error if the transcription fails.

type Usage added in v1.0.3 

type Usage struct {
	PromptTokens     int `json:"prompt_tokens"`
	CompletionTokens int `json:"completion_tokens"`
	TotalTokens      int `json:"total_tokens"`
}

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.