Documentation
¶
Overview ¶
Copyright 2024 Stacklok, Inc
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
Copyright 2024 Stacklok, Inc ¶
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
Copyright 2024 Stacklok, Inc ¶
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Backend ¶
type Backend interface {
Generate(ctx context.Context, prompt *Prompt) (string, error)
Embed(ctx context.Context, input string) ([]float32, error)
}
Backend defines the interface for interacting with various LLM backends.
type OllamaBackend ¶
OllamaBackend represents a backend for interacting with the Ollama API.
func NewOllamaBackend ¶
func NewOllamaBackend(baseURL, model string, timeout time.Duration) *OllamaBackend
NewOllamaBackend creates a new OllamaBackend instance.
func (*OllamaBackend) Embed ¶
Embed generates embeddings for the given input text using the Ollama API.
func (*OllamaBackend) Generate ¶
Generate produces a response from the Ollama API based on the given structured prompt.
Parameters:
- ctx: The context for the API request, which can be used for cancellation.
- prompt: A structured prompt containing messages and parameters.
Returns:
- A string containing the generated response from the Ollama model.
- An error if the API request fails or if there's an issue processing the response.
type OllamaEmbeddingResponse ¶
type OllamaEmbeddingResponse struct {
Embedding []float32 `json:"embedding"`
}
OllamaEmbeddingResponse represents the response from the Ollama API for embeddings.
type OpenAIBackend ¶
OpenAIBackend represents a backend for interacting with the OpenAI API. It contains configuration details and methods for making API requests.
func NewOpenAIBackend ¶
func NewOpenAIBackend(apiKey, model string, timeout time.Duration) *OpenAIBackend
NewOpenAIBackend creates and returns a new OpenAIBackend instance with a custom timeout.
Parameters:
- apiKey: The API key for authenticating with the OpenAI API.
- model: The name of the OpenAI model to use for generating responses.
- timeout: The duration for the HTTP client timeout. If zero, the default timeout is used.
Returns:
- A pointer to a new OpenAIBackend instance configured with the provided API key, model, and timeout.
func (*OpenAIBackend) Embed ¶
Embed generates an embedding vector for the given text using the OpenAI API.
Parameters:
- ctx: The context for the API request, which can be used for cancellation.
- text: The input text to be embedded.
Returns:
- A slice of float32 values representing the embedding vector.
- An error if the API request fails or if there's an issue processing the response.
func (*OpenAIBackend) Generate ¶
Generate sends a structured prompt to the OpenAI API and returns the generated response.
Parameters:
- ctx: The context for the API request, which can be used for cancellation.
- prompt: A structured prompt containing messages and parameters.
Returns:
- A string containing the generated response from the OpenAI model.
- An error if the API request fails or if there's an issue processing the response.
type OpenAIEmbeddingResponse ¶
type OpenAIEmbeddingResponse struct {
Object string `json:"object"`
Data []struct {
Object string `json:"object"`
Embedding []float32 `json:"embedding"`
Index int `json:"index"`
} `json:"data"`
Model string `json:"model"`
Usage struct {
PromptTokens int `json:"prompt_tokens"`
TotalTokens int `json:"total_tokens"`
} `json:"usage"`
}
OpenAIEmbeddingResponse represents the structure of the response received from the OpenAI API for an embedding request. It contains the generated embeddings, usage statistics, and other metadata related to the API call.
type OpenAIResponse ¶
type OpenAIResponse struct {
ID string `json:"id"`
Object string `json:"object"`
Created int64 `json:"created"`
Model string `json:"model"`
Choices []struct {
Index int `json:"index"`
Message struct {
Role string `json:"role"`
Content string `json:"content"`
} `json:"message"`
FinishReason string `json:"finish_reason"`
} `json:"choices"`
Usage struct {
PromptTokens int `json:"prompt_tokens"`
CompletionTokens int `json:"completion_tokens"`
TotalTokens int `json:"total_tokens"`
} `json:"usage"`
}
OpenAIResponse represents the structure of the response received from the OpenAI API for a chat completion request. It contains information about the generated text, usage statistics, and other metadata related to the API call.
type Parameters ¶
type Parameters struct {
MaxTokens int `json:"max_tokens"`
Temperature float64 `json:"temperature"`
TopP float64 `json:"top_p"`
FrequencyPenalty float64 `json:"frequency_penalty"`
PresencePenalty float64 `json:"presence_penalty"`
}
Parameters defines generation settings for LLM completions.
type Prompt ¶
type Prompt struct {
Messages []Message `json:"messages"`
Parameters Parameters `json:"parameters"`
}
Prompt represents a structured prompt with role-based messages and parameters.
func (*Prompt) AddMessage ¶
AddMessage adds a message with a specific role to the prompt.
func (*Prompt) SetParameters ¶
func (p *Prompt) SetParameters(params Parameters) *Prompt
SetParameters sets the generation parameters for the prompt.
type Response ¶
type Response struct {
Model string `json:"model"`
CreatedAt string `json:"created_at"`
Response string `json:"response"`
Done bool `json:"done"`
DoneReason string `json:"done_reason"`
Context []int `json:"context"`
TotalDuration int64 `json:"total_duration"`
LoadDuration int64 `json:"load_duration"`
PromptEvalCount int `json:"prompt_eval_count"`
PromptEvalDuration int64 `json:"prompt_eval_duration"`
EvalCount int `json:"eval_count"`
EvalDuration int64 `json:"eval_duration"`
}
Response represents the structure of the response received from the Ollama API.
Source Files