aillm

Golang RAG/LLM framework with Memory and Transcriber - All-in-One Platform

This project is an easy to use RAG (Retrieval-Augmented Generation) system that combines an LLM client with memory, text embedding, and document transcription in a single easy-to-configure platform framework over LangChain Go.

It offers seamless compatibility with OLLAMA and OpenAI, making it an excellent choice for chatbot applications.

Features

1. LLM Client - Sync and Stream with High Compatibility

• Supports both synchronous and streaming interactions with LLMs.

• Fully compatible with OLLAMA and OpenAI, ensuring smooth chatbot integration.

1. Text Embedding & Similarity Search

• Efficient document embedding and retrieval using Cosine Similarity and K-Nearest Neighbors (KNN).

• Powered by Redis Vector Store for fast lookups.

• Advanced Hybrid Search combining vector and lexical search for improved accuracy.

• Multiple Search Algorithms including semantic, lexical, and hybrid search with configurable weights.

1. Document Transcription and Image description generation

• Internet content downloader
• Processes files such as PDFs, Word, Excel, and HTML using Apache Tika.
• Extracts structured data, including OCR-based transcription for scanned documents.
• Ability to call multimodal models for describing images.

1. Multilingual Support

• Process and analyze content in multiple languages with AI-powered detection and response customization.

1. Memory Management

• Leverages Redis and efficient text processing pipelines to scale with large datasets.
• Memory Summarization for condensing long conversation histories.
• Vector search for retrieving relevant past interactions.

1. Tool Calling & Function Integration

• Smart Tool Execution with automatic vector search bypass when tools are available.
• Intelligent Fallback to RAG/vector search when no tools can handle the query.
• Security-First Approach maintaining all security checks during tool operations.
• Performance Optimized reducing unnecessary computation and token usage during tool calls.

Configuration

Configure environment variables in .env file:

APITOKEN=your-openai-api-token //not mandatory for Ollama
REDIS_HOST=localhost:6379
REDIS_PASSWORD=your_redis_password
TIKA_URL=http://localhost:9998/tika //not mandatory for text, html or multimodal model usage and should be used just for PDF,MS-Word, MS-Excel & ...

Usage

Refer to the examples folder in the repository for more details. Below is a simple usage example:

package main

import (
	"context"
	"fmt"
	"log"

	aillm "github.com/RezaArani/aillm/controller"
)

func main() {
	log.Println("Testing aillm framework:")
	// locally hosted ollama
	llmclient := &aillm.OllamaController{
		Config: aillm.LLMConfig{
			Apiurl:  "http://127.0.0.1:11434",
			AiModel: "llama3.1",
		},
	}
	// Create an LLM instance with OllamaClient
	llm := aillm.LLMContainer{
		Embedder:  llmclient,
		LLMClient: llmclient,
		RedisClient: aillm.RedisClient{
			Host: "localhost:6379",
		},
	}
	
	llm.Init()
	// asking question without context
	askKLLM(llm, "What is SemMapas?")
	// let's embed some data
	log.Println("Embedding:")
	embedd(llm)
	
	// Time for asking some questions
	askKLLM(llm, "What is SemMapas?")
	askKLLM(llm, "Where did it launched?")
	// Now removing embedded data and asking the same question, result should be I'm unable to provide a specific location regarding the launch of SemMapas as I don't have sufficient information on this topic.
	llm.RemoveEmbedding("")
	// Asking the same question again
	
}

func askKLLM(llm aillm.LLMContainer,query string) {
	log.Println("LLM Reply to " + query + ":")
	_, err := llm.AskLLM(query, llm.WithStreamingFunc(print))
 	if err != nil {
		log.Fatal(err)
	}
}
 
func embedd(llm aillm.LLMContainer) {
	// Text Embedding
	contents := aillm.LLMEmbeddingContent{
		Text: enRawText,
	}
	llm.EmbeddText("", contents)
}

func print(ctx context.Context, chunk []byte) error {
	fmt.Print(string(chunk))
	return nil
}

const enRawText = `Welcome to SemMapas, your strategic partner in enhancing local engagement and tourism development. Designed specifically for businesses and municipalities, SemMapas offers a powerful platform to connect with residents and visitors alike, driving growth and prosperity in your community.
With SemMapas, you can effortlessly map out venues, highlight points of interest, and provide real-time updates to ensure smooth navigation for attendees. Our user-friendly interface and customizable options make it easy to tailor the experience to your specific event or business requirements.
Our platform goes beyond traditional mapping services, offering a comprehensive suite of features tailored to meet the diverse needs of event organizers and businesses alike. From tourism guides to event navigation, SemMapas empowers you to create immersive experiences that captivate your audience and enhance their journey.
Our project has been launched since 2023 in Portugal.
`

Advanced Search Usage

The framework now supports multiple search algorithms for improved accuracy:

// Traditional vector search
result, err := llm.AskLLM("machine learning", llm.WithSimilaritySearch())

// Hybrid search for better accuracy (recommended)
result, err := llm.AskLLM("machine learning", llm.WithHybridSearch())

// Lexical search for exact keyword matching
result, err := llm.AskLLM("machine learning", llm.WithLexicalSearch())

// Semantic search (auto-optimized)
result, err := llm.AskLLM("machine learning", llm.WithSemanticSearch())

Tool Calling Example

The framework supports intelligent tool calling with automatic fallback to vector search:

package main

import (
	"context"
	"fmt"
	"log"

	aillm "github.com/RezaArani/aillm/controller"
	"github.com/tmc/langchaingo/tools"
)

func main() {
	// Initialize LLM client
	llmclient := &aillm.OpenAIController{
		Config: aillm.LLMConfig{
			Apiurl:   "https://api.openai.com/v1",
			AiModel:  "gpt-4",
			APIToken: "your-openai-api-token",
		},
	}
	
	llm := aillm.LLMContainer{
		LLMClient: llmclient,
		Embedder:  llmclient,
		RedisClient: aillm.RedisClient{
			Host: "localhost:6379",
		},
	}
	
	llm.Init()
	
	// Define tools
	weatherTool := tools.Tool{
		Name:        "get_weather",
		Description: "Get current weather for a location",
		// ... tool configuration
	}
	
	// Tools will be used if applicable, otherwise falls back to RAG
	result, err := llm.AskLLM(
		"What's the weather like in Tokyo?",
		llm.WithTools(aillm.ToolsConfig{
			Tools:    []tools.Tool{weatherTool},
			Handlers: map[string]func(interface{}) (string, error){
				"get_weather": handleWeatherRequest,
			},
		}),
	)
	
	if err != nil {
		log.Fatal(err)
	}
	
	fmt.Println("Response:", result.Response.Choices[0].Content)
}

func handleWeatherRequest(params interface{}) (string, error) {
	// Tool implementation
	return "Sunny, 25°C in Tokyo", nil
}

Image Description Example

package main

import (
	"fmt"
	"log"

	aillm "github.com/RezaArani/aillm/controller"
)

func main() {
	// Initialize with a vision-capable model
	llmclient := &aillm.OpenAIController{
		Config: aillm.LLMConfig{
			Apiurl:   "https://api.openai.com/v1",
			AiModel:  "gpt-4-vision-preview",
			APIToken: "your-openai-api-token",
		},
	}
	
	llm := aillm.LLMContainer{
		VisionClient: llmclient,
	}
	
	llm.Init()
	
	// Describe an image from a file
	response, err := llm.DescribeImageFromFile("path/to/your/image.jpg", "What can you see in this image?")
	if err != nil {
		log.Fatalf("Error: %v", err)
	}
	
	fmt.Println("Image Description:", response.Choices[0].Message.Content)
}

TODO

• Implement parallelism to optimize processing efficiency.
• Enhance the chatbot integration by supporting additional LLM models.
• Dockerized version with Reset Service and Websocket web server.

📢 Recent Updates in v1.2.14

• Smart Tool Calling Optimization - 🚀 Revolutionary tool calling system that bypasses vector search when tools are available, reducing unnecessary computation and token usage
• Intelligent Fallback System - 🔄 Automatic fallback to RAG/vector search when no tools can handle the query, ensuring queries are always answered
• Security-First Tool Integration - 🔒 All security checks are preserved during tool operations, maintaining system integrity
• Performance Optimized Tool Workflow - ⚡ Eliminated redundant vector searches during tool calling, significantly improving response times and reducing costs
• Enhanced Memory Management - 🧠 Tool interactions are now properly tracked and stored in conversation memory
• Backward Compatibility - ✅ All existing functionality preserved with seamless tool integration

📢 Recent Updates in v1.2.13

• Fixed Redis FT.SEARCH Special Characters - 🔧 Resolved critical issue with special characters like colons (:) and slashes (/) in search queries that caused "Syntax error at offset" errors
• Enhanced Query Escaping - 🛡️ Proper escaping of special Redis FT.SEARCH characters including @, (), [], {}, *, +, ?, |, ^, $, -, =, ~, :, ;, !, #, %, &, ', ", \ and /
• Improved Search Reliability - ✅ URLs, email addresses, and other text with special characters now work correctly with Redis FT.SEARCH
• Better Error Handling - 🚀 Added safety checks to prevent empty queries and improved error messaging

📢 Recent Updates in v1.2.10

• Advanced Hybrid Search - 🚀 Revolutionary hybrid search system combining vector similarity and lexical search with Reciprocal Rank Fusion (RRF) algorithm for up to 17% improved accuracy
• Multiple Search Algorithms - 🔍 New search options: Hybrid Search, Lexical Search, and Semantic Search with automatic algorithm selection
• Enhanced Search Performance - ⚡ Up to 28% improvement for keyword queries and 5% for semantic queries over traditional vector-only search
• Configurable Search Weights - ⚖️ Customizable weights for vector vs lexical search results with advanced scoring methods
• Comprehensive Search Examples - 📚 New example demonstrating all search algorithms with performance comparisons
• Backward Compatibility - ✅ All existing code continues to work unchanged with gradual migration support

📢 Recent Updates in v.1.2.8

• Guard prompt - 🔐 A lightweight guard‑prompt toolkit that prepends a security layer to block jailbreaks, prompt injection, unauthorized code execution, and data leaks before processing user input.

📢 Updates in v1.2.7

• Llama-4-Maverick - Llama-4-Maverick compatibility
• Memory Summarization - Conversations are now automatically summarized for more efficient context management
• Vision Support Improvements - Enhanced image description capabilities with better error handling and mime type detection
• User Memory Management - Improved persistent memory with Redis TTL and vector similarity search for related conversations
• Performance Optimizations - Reduced token usage and improved response times

License

This project is licensed under the Apache License, Version 2.0.

Contact

For inquiries or support, reach out via:

• 📧 Email: reza.arani@gmail.com
• 🌐 Website: semmapas.com
• 🐦 Twitter: @RezaArani2

🙌 Special Thanks to OVHCloud

A huge thanks to OVHCloud for their fantastic AI Endpoints, which power our model with high-performance and scalable infrastructure. Their support has been instrumental in making AI-driven image description fast, efficient, and reliable.

🔗 Learn more about OVHCloud AI Endpoints: OVHCloud AI 🚀