Documentation
¶
Index ¶
Constants ¶
const ( DefaultModel = openai.GPT3Dot5Turbo DefaultMaxTokens = 512 )
DefaultModel is a constant that represents the default OpenAI model used by the Service type.
Variables ¶
var DefaultMinification = [...]nodes.MinifyOptions{ nodes.MinifyUnexported, { FuncBody: true, Exported: true, }, nodes.MinifyExported, nodes.MinifyAll, }
DefaultMinification is an array of MinifyOptions used as the default steps for the Minify function. The Minify function takes a byte slice of code and a maximum number of tokens as input, and returns a Minification struct, a slice of Minification structs, and an error. The Minification struct contains the input code, the minified code, the number of tokens in the minified code, and the MinifyOptions used to minify the code. The MinifyOptions struct contains the maximum number of tokens, a model string, a prepend string, and a slice of MinifyOptions(https://pkg.go.dev/github.com/modernice/jotbot/internal/nodes#MinifyOptions).
Functions ¶
func Minify ¶
func Minify(code []byte, maxTokens int) (Minification, []Minification, error)
Minify function in the openai package minifies a given byte slice of code to a specified maximum number of tokens. It returns a Minification struct which contains the input code, the minified code, the number of tokens in the minified code, and the MinifyOptions used to minify the code. The function can also return a slice of Minification structs and an error. The MinifyOptions struct contains the maximum number of tokens, a model string, a prepend string, and a slice of MinifyOptions.
Types ¶
type Minification ¶
type Minification struct {
Input []byte
Minified []byte
// Tokens is the number of tokens in the minified code.
Tokens []uint
Options nodes.MinifyOptions
}
Minification package provides functions and options to minify code using tiktoken-go/tokenizer library. The Minify function takes a byte slice of code and a maximum number of tokens as input, and returns a Minification struct, a slice of Minification structs, and an error. The Minification struct contains the input code, the minified code, the number of tokens in the minified code, and the MinifyOptions used to minify the code. The MinifyOptions struct contains the maximum number of tokens, a model string, a prepend string, and a slice of MinifyOptions.
type MinifyOptions ¶
type MinifyOptions struct {
MaxTokens int
Model string
Prepend string
Force bool
Steps []nodes.MinifyOptions
}
MinifyOptions represents the options for the minification process. It contains the maximum number of tokens, a model string, a prepend string, a boolean to force minification even if it exceeds the maximum number of tokens, and a slice of MinifyOptions that defines the steps to be taken during minification.
func (MinifyOptions) Minify ¶
func (opts MinifyOptions) Minify(code []byte) (Minification, []Minification, error)
MinifyOptions.Minify takes a byte slice of code and returns a Minification struct, a slice of Minification structs, and an error. The Minification struct contains the input code, the minified code, the number of tokens in the minified code, and the MinifyOptions used to minify the code. The slice of Minification structs contains each step of the minification process. If the total number of tokens in the minified code is less than or equal to MaxTokens, no further minification steps are performed and only one Minification struct is returned. Otherwise, multiple Minification structs are returned for each step in opts.Steps until either MaxTokens is reached or all steps have been taken.
type Option ¶
type Option func(*Service)
Option represents an option that can be passed to the New or NewFrom function to configure a Service. It is a function that takes a pointer to a Service and modifies it. The available options are: WithLogger, WithClient, Model, MaxTokens, and MinifyWith.
func MaxTokens ¶
MaxTokens represents the maximum number of tokens that can be used by the OpenAI API to generate documentation. It is a constant set to 512 by default, but can be changed using the MaxTokens option when creating a new Service.
func MinifyWith ¶
func MinifyWith(steps []nodes.MinifyOptions, force bool) Option
MinifyWith minifies code with the given MinifyOptions steps and force flag. It returns an Option that sets the Service's minifySteps and forceMinify fields to the given values.
func Model ¶
Model represents an OpenAI language model used to generate documentation for code. It is a type of Service and can be set as an option when creating a new Service. By default, the GPT-3.5 Turbo model is used. Other available models include GPT-4, GPT-4 32B, and GPT-3 Text Davinci. The maximum number of tokens that can be generated by the model is set based on the selected model or a default value.
func WithClient ¶
WithClient sets the OpenAI client for the Service type. It returns an Option that can be passed to New or NewFrom when creating a new Service.
func WithLogger ¶
WithLogger sets the logger for the Service type. It returns an Option that can be passed to New or NewFrom to configure a new Service.
type Service ¶
type Service struct {
// contains filtered or unexported fields
}
Service is a type that provides the ability to generate documentation for Go code using OpenAI's GPT-3 model. It implements the generate.Service interface and can be customized using various options such as the OpenAI model to use, maximum number of tokens to generate, and minification options. To use Service, create a new instance with an API key and any desired options, then call the GenerateDoc method with a generate.Context containing the file name and identifier for which documentation is requested. The method returns the generated documentation in GoDoc format.
func New ¶
New creates a new instance of Service that uses the OpenAI API to generate documentation for Go code. It takes an API key as its first argument and optional Option values to configure the Service.
func NewFrom ¶
NewFrom returns a new Service with the given Options applied. The returned Service has default values for fields that are not set in the Options.
func (*Service) GenerateDoc ¶
GenerateDoc generates a concise documentation for a given function or type identified by the provided identifier in GoDoc format. The function takes a generate.Context as input, reads the file from the context and minifies the code using MinifyOptions. The prompt is then created using the file, identifier and longIdentifier. The createCompletion function is called to generate documentation from the prompt using OpenAI's GPT models. The resulting text is normalized and returned as a string along with an error, if any.
type SourceTooLarge ¶
SourceTooLarge is a type that represents an error when the provided source code is too large to be minified to a specified number of tokens. The error message includes the maximum number of tokens and the number of tokens in the resulting minified code.
func (*SourceTooLarge) Error ¶
func (err *SourceTooLarge) Error() string
Error SourceTooLarge is a struct that represents an error returned by the Minify function when the input source code is too large to be minified to the specified maximum number of tokens. It contains two fields: MaxTokens and MinifiedTokens, which are the maximum number of tokens and the actual number of tokens in the minified code, respectively. The Error method of this type returns a string that describes the error.
Source Files