Documentation
¶
Overview ¶
Package postprocessors is basically the same as package transformers, but used at a different stage of the RAG pipeline
Index ¶
Constants ¶
View Source
const CohereRerankPostprocessorName = "cohere_rerank"
View Source
const ContentFilterPostprocessorName = "content_filter"
View Source
const ContentSubstringFilterPostprocessorName = "content_substring_filter"
View Source
const SimilarityPostprocessorName = "similarity"
Variables ¶
View Source
var PostprocessorMap = map[string]Postprocessor{ transformers.ExtraMetadataName: NewTransformerWrapper(&transformers.ExtraMetadata{}), transformers.KeywordExtractorName: NewTransformerWrapper(&transformers.KeywordExtractor{}), transformers.FilterMarkdownDocsNoContentName: NewTransformerWrapper(&transformers.FilterMarkdownDocsNoContent{}), SimilarityPostprocessorName: &SimilarityPostprocessor{}, ContentSubstringFilterPostprocessorName: &ContentSubstringFilterPostprocessor{}, ContentFilterPostprocessorName: &ContentFilterPostprocessor{}, CohereRerankPostprocessorName: &CohereRerankPostprocessor{}, }
Functions ¶
This section is empty.
Types ¶
type CohereRerankPostprocessor ¶ added in v0.1.7
type CohereRerankPostprocessor struct {
ApiKey string `json:"apiKey" yaml:"apiKey"`
Model string
TopN int
}
func (*CohereRerankPostprocessor) Name ¶ added in v0.1.9
func (c *CohereRerankPostprocessor) Name() string
func (*CohereRerankPostprocessor) Transform ¶ added in v0.1.7
func (c *CohereRerankPostprocessor) Transform(ctx context.Context, response *types.RetrievalResponse) error
type ContentFilterPostprocessor ¶ added in v0.1.7
type ContentFilterPostprocessor struct {
Question string // Question about the content, that can be answered with yes or no
Include bool // Whether to include or exclude the documents for which the answer is yes
LLM llm.LLM
}
func (*ContentFilterPostprocessor) Name ¶ added in v0.1.9
func (c *ContentFilterPostprocessor) Name() string
func (*ContentFilterPostprocessor) Transform ¶ added in v0.1.7
func (c *ContentFilterPostprocessor) Transform(ctx context.Context, response *types.RetrievalResponse) error
type ContentSubstringFilterPostprocessor ¶ added in v0.1.7
func (*ContentSubstringFilterPostprocessor) Name ¶ added in v0.1.9
func (c *ContentSubstringFilterPostprocessor) Name() string
func (*ContentSubstringFilterPostprocessor) Transform ¶ added in v0.1.7
func (c *ContentSubstringFilterPostprocessor) Transform(ctx context.Context, response *types.RetrievalResponse) error
type Postprocessor ¶
type Postprocessor interface {
Transform(ctx context.Context, response *types.RetrievalResponse) error
Name() string
}
Postprocessor is similar to types.DocumentTransformer, but can take into account the retrieval query
func GetPostprocessor ¶
func GetPostprocessor(name string) (Postprocessor, error)
type SimilarityPostprocessor ¶ added in v0.1.7
type SimilarityPostprocessor struct {
Threshold float32
}
func (*SimilarityPostprocessor) Name ¶ added in v0.1.9
func (s *SimilarityPostprocessor) Name() string
func (*SimilarityPostprocessor) Transform ¶ added in v0.1.7
func (s *SimilarityPostprocessor) Transform(ctx context.Context, response *types.RetrievalResponse) error
type TransformerWrapper ¶ added in v0.1.7
type TransformerWrapper struct {
types.DocumentTransformer
}
func NewTransformerWrapper ¶ added in v0.1.7
func NewTransformerWrapper(transformer types.DocumentTransformer) *TransformerWrapper
func (*TransformerWrapper) Name ¶ added in v0.1.9
func (t *TransformerWrapper) Name() string
func (*TransformerWrapper) Transform ¶ added in v0.1.7
func (t *TransformerWrapper) Transform(ctx context.Context, response *types.RetrievalResponse) error
Click to show internal directories.
Click to hide internal directories.