v1alpha1

package
v0.9.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 5, 2026 License: Apache-2.0 Imports: 8 Imported by: 0

Documentation

Overview

Package v1alpha1 contains API Schema definitions for the operator v1alpha1 API group +kubebuilder:object:generate=true +groupName=operator.dataverse.redhat.com

Index

Constants

View Source 
const (
	SuccessfullyReconciled = "SuccessfullyReconciled"
	ReconcileFailed        = "ReconcileFailed"
)
View Source 
const (
	TypeS3                             UnstructuredDataType = "s3"
	TypeGoogleDrive                    UnstructuredDataType = "googleDrive"
	ChunkingStrategyRecursiveCharacter ChunkingStrategy     = "recursiveCharacterTextSplitter"
	ChunkingStrategyMarkdown           ChunkingStrategy     = "markdownTextSplitter"
	ChunkingStrategyToken              ChunkingStrategy     = "tokenTextSplitter"

	UnstructuredDataPipelineCondition = "UnstructuredDataPipelineReady"
)
View Source 
const (
	ChunksGeneratorCondition = "ChunksGeneratorReady"
)
View Source 
const (
	ConfigCondition = "ConfigReady"
)
View Source 
const (
	DestinationSyncerCondition = "DestinationSyncerReady"
)
View Source 
const (
	DocumentProcessorCondition = "DocumentProcessorReady"
)
View Source 
const (
	SourceCrawlerCondition = "SourceCrawlerReady"
)
View Source 
const (
	VectorEmbeddingGenerationConditionType = "VectorEmbeddingGenerationReady"
)

Variables

View Source 
var (
	// GroupVersion is group version used to register these objects
	GroupVersion = schema.GroupVersion{Group: "operator.dataverse.redhat.com", Version: "v1alpha1"}

	// SchemeBuilder is used to add go types to the GroupVersionKind scheme
	SchemeBuilder = &scheme.Builder{GroupVersion: GroupVersion}

	// AddToScheme adds the types in this group-version to the given scheme.
	AddToScheme = SchemeBuilder.AddToScheme
)

Functions

func ConditionTypeForStage added in v0.8.0 

func ConditionTypeForStage(t StageType) string

ConditionTypeForStage maps a StageType to its corresponding CR condition string.

func ValidateStages added in v0.8.0 

func ValidateStages(stages []PipelineStage) error

ValidateStages checks that all dependency references exist and the DAG has no cycles.

Types

type ChunkingStrategy 

type ChunkingStrategy string

type ChunksGenerator 

type ChunksGenerator struct {
	metav1.TypeMeta   `json:",inline"`
	metav1.ObjectMeta `json:"metadata,omitempty"`

	Spec   ChunksGeneratorSpec   `json:"spec,omitempty"`
	Status ChunksGeneratorStatus `json:"status,omitempty"`
}

ChunksGenerator is the Schema for the chunksgenerators API

func (*ChunksGenerator) DeepCopy 

func (in *ChunksGenerator) DeepCopy() *ChunksGenerator

DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new ChunksGenerator.

func (*ChunksGenerator) DeepCopyInto 

func (in *ChunksGenerator) DeepCopyInto(out *ChunksGenerator)

DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.

func (*ChunksGenerator) DeepCopyObject 

func (in *ChunksGenerator) DeepCopyObject() runtime.Object

DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object.

func (*ChunksGenerator) GetFilesProcessed added in v0.8.0 

func (c *ChunksGenerator) GetFilesProcessed() int64

func (*ChunksGenerator) SetWaiting 

func (c *ChunksGenerator) SetWaiting()

func (*ChunksGenerator) UpdateStatus 

func (c *ChunksGenerator) UpdateStatus(message string, err error)

type ChunksGeneratorConfig 

type ChunksGeneratorConfig struct {
	Strategy                         ChunkingStrategy                 `json:"strategy"`
	RecursiveCharacterSplitterConfig RecursiveCharacterSplitterConfig `json:"recursiveCharacterSplitterConfig,omitempty"`
	MarkdownSplitterConfig           MarkdownSplitterConfig           `json:"markdownSplitterConfig,omitempty"`
	TokenSplitterConfig              TokenSplitterConfig              `json:"tokenSplitterConfig,omitempty"`
}

func (*ChunksGeneratorConfig) DeepCopy 

func (in *ChunksGeneratorConfig) DeepCopy() *ChunksGeneratorConfig

DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new ChunksGeneratorConfig.

func (*ChunksGeneratorConfig) DeepCopyInto 

func (in *ChunksGeneratorConfig) DeepCopyInto(out *ChunksGeneratorConfig)

DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.

type ChunksGeneratorList 

type ChunksGeneratorList struct {
	metav1.TypeMeta `json:",inline"`
	metav1.ListMeta `json:"metadata,omitempty"`
	Items           []ChunksGenerator `json:"items"`
}

ChunksGeneratorList contains a list of ChunksGenerator

func (*ChunksGeneratorList) DeepCopy 

func (in *ChunksGeneratorList) DeepCopy() *ChunksGeneratorList

DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new ChunksGeneratorList.

func (*ChunksGeneratorList) DeepCopyInto 

func (in *ChunksGeneratorList) DeepCopyInto(out *ChunksGeneratorList)

DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.

func (*ChunksGeneratorList) DeepCopyObject 

func (in *ChunksGeneratorList) DeepCopyObject() runtime.Object

DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object.

type ChunksGeneratorSpec 

type ChunksGeneratorSpec struct {
	StageName             string                `json:"stageName,omitempty"`
	DependsOn             []StageDependency     `json:"dependsOn,omitempty"`
	ChunksGeneratorConfig ChunksGeneratorConfig `json:"config,omitempty"`
	// Deprecated: use StageName and DependsOn instead.
	// +optional
	DataProduct string `json:"dataProduct,omitempty"`
}

ChunksGeneratorSpec defines the desired state of ChunksGenerator

func (*ChunksGeneratorSpec) DeepCopy 

func (in *ChunksGeneratorSpec) DeepCopy() *ChunksGeneratorSpec

DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new ChunksGeneratorSpec.

func (*ChunksGeneratorSpec) DeepCopyInto 

func (in *ChunksGeneratorSpec) DeepCopyInto(out *ChunksGeneratorSpec)

DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.

type ChunksGeneratorStatus 

type ChunksGeneratorStatus struct {
	LastAppliedGeneration int64              `json:"lastAppliedGeneration,omitempty"`
	Conditions            []metav1.Condition `json:"conditions,omitempty"`
	FilesProcessed        int64              `json:"filesProcessed,omitempty"`
}

ChunksGeneratorStatus defines the observed state of ChunksGenerator

func (*ChunksGeneratorStatus) DeepCopy 

func (in *ChunksGeneratorStatus) DeepCopy() *ChunksGeneratorStatus

DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new ChunksGeneratorStatus.

func (*ChunksGeneratorStatus) DeepCopyInto 

func (in *ChunksGeneratorStatus) DeepCopyInto(out *ChunksGeneratorStatus)

DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.

type ControllerConfig 

type ControllerConfig struct {
	metav1.TypeMeta   `json:",inline"`
	metav1.ObjectMeta `json:"metadata,omitempty"`

	Spec   ControllerConfigSpec   `json:"spec,omitempty"`
	Status ControllerConfigStatus `json:"status,omitempty"`
}

ControllerConfig is the Schema for the controllerconfigs API.

func (*ControllerConfig) DeepCopy 

func (in *ControllerConfig) DeepCopy() *ControllerConfig

DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new ControllerConfig.

func (*ControllerConfig) DeepCopyInto 

func (in *ControllerConfig) DeepCopyInto(out *ControllerConfig)

DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.

func (*ControllerConfig) DeepCopyObject 

func (in *ControllerConfig) DeepCopyObject() runtime.Object

DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object.

func (*ControllerConfig) IsHealthy 

func (c *ControllerConfig) IsHealthy() bool

func (*ControllerConfig) SetWaiting 

func (c *ControllerConfig) SetWaiting()

func (*ControllerConfig) UpdateStatus 

func (c *ControllerConfig) UpdateStatus(err error)

type ControllerConfigList 

type ControllerConfigList struct {
	metav1.TypeMeta `json:",inline"`
	metav1.ListMeta `json:"metadata,omitempty"`
	Items           []ControllerConfig `json:"items"`
}

ControllerConfigList contains a list of ControllerConfig.

func (*ControllerConfigList) DeepCopy 

func (in *ControllerConfigList) DeepCopy() *ControllerConfigList

DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new ControllerConfigList.

func (*ControllerConfigList) DeepCopyInto 

func (in *ControllerConfigList) DeepCopyInto(out *ControllerConfigList)

DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.

func (*ControllerConfigList) DeepCopyObject 

func (in *ControllerConfigList) DeepCopyObject() runtime.Object

DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object.

type ControllerConfigSpec 

type ControllerConfigSpec struct {
	SecretRef                   string `json:"secretRef,omitempty"`
	MaxConcurrentDoclingTasks   int    `json:"maxConcurrentDoclingTasks,omitempty"`
	MaxConcurrentLangchainTasks int    `json:"maxConcurrentLangchainTasks,omitempty"`
	DoclingServeURL             string `json:"doclingServeURL,omitempty"`
	CacheDirectory              string `json:"cacheDirectory,omitempty"`
	DataStorageBucket           string `json:"dataStorageBucket,omitempty"`
	// +kubebuilder:validation:Minimum=1
	UnstructuredDataPipelineResyncInterval *int `json:"unstructuredDataPipelineResyncInterval,omitempty"`
	// GDriveConfig holds operator-level Google Drive crawling settings.
	// +optional
	GoogleDriveConfig *GoogleDriveControllerConfig `json:"googleDriveConfig,omitempty"`
	// LDAPConfig holds LDAP configuration for user/group identity resolution.
	// +optional
	LDAPConfig *LDAPConfig `json:"ldapConfig,omitempty"`

	// Deprecated: use SecretRef instead.
	// +optional
	UnstructuredSecret string `json:"unstructuredSecret,omitempty"`
	// Deprecated: fields are now top-level on ControllerConfigSpec.
	// +optional
	SnowflakeConfig *SnowflakeConfig `json:"snowflakeConfig,omitempty"`
	// Deprecated: fields are now top-level on ControllerConfigSpec.
	// +optional
	UnstructuredDataProcessingConfig *UnstructuredDataProcessingConfigSpec `json:"unstructuredDataProcessingConfig,omitempty"`
}

ControllerConfigSpec defines the desired state of ControllerConfig.

func (*ControllerConfigSpec) DeepCopy 

func (in *ControllerConfigSpec) DeepCopy() *ControllerConfigSpec

DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new ControllerConfigSpec.

func (*ControllerConfigSpec) DeepCopyInto 

func (in *ControllerConfigSpec) DeepCopyInto(out *ControllerConfigSpec)

DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.

type ControllerConfigStatus 

type ControllerConfigStatus struct {
	Conditions            []metav1.Condition `json:"conditions,omitempty"`
	LastAppliedGeneration int64              `json:"lastAppliedGeneration,omitempty"`
}

ControllerConfigStatus defines the observed state of ControllerConfig.

func (*ControllerConfigStatus) DeepCopy 

func (in *ControllerConfigStatus) DeepCopy() *ControllerConfigStatus

DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new ControllerConfigStatus.

func (*ControllerConfigStatus) DeepCopyInto 

func (in *ControllerConfigStatus) DeepCopyInto(out *ControllerConfigStatus)

DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.

type DestinationSyncer added in v0.8.0 

type DestinationSyncer struct {
	metav1.TypeMeta   `json:",inline"`
	metav1.ObjectMeta `json:"metadata,omitempty"`

	Spec   DestinationSyncerSpec   `json:"spec,omitempty"`
	Status DestinationSyncerStatus `json:"status,omitempty"`
}

DestinationSyncer is the Schema for the destinationsyncers API.

func (*DestinationSyncer) DeepCopy added in v0.8.0 

func (in *DestinationSyncer) DeepCopy() *DestinationSyncer

DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DestinationSyncer.

func (*DestinationSyncer) DeepCopyInto added in v0.8.0 

func (in *DestinationSyncer) DeepCopyInto(out *DestinationSyncer)

DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.

func (*DestinationSyncer) DeepCopyObject added in v0.8.0 

func (in *DestinationSyncer) DeepCopyObject() runtime.Object

DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object.

func (*DestinationSyncer) GetFilesProcessed added in v0.8.0 

func (d *DestinationSyncer) GetFilesProcessed() int64

func (*DestinationSyncer) SetWaiting added in v0.8.0 

func (d *DestinationSyncer) SetWaiting()

func (*DestinationSyncer) UpdateStatus added in v0.8.0 

func (d *DestinationSyncer) UpdateStatus(message string, err error)

type DestinationSyncerConfig added in v0.8.0 

type DestinationSyncerConfig struct {
	Type                UnstructuredDataType `json:"type,omitempty"`
	S3DestinationConfig S3Config             `json:"s3DestinationConfig,omitempty"`
}

DestinationSyncerConfig configures where to write processed data.

func (*DestinationSyncerConfig) DeepCopy added in v0.8.0 

func (in *DestinationSyncerConfig) DeepCopy() *DestinationSyncerConfig

DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DestinationSyncerConfig.

func (*DestinationSyncerConfig) DeepCopyInto added in v0.8.0 

func (in *DestinationSyncerConfig) DeepCopyInto(out *DestinationSyncerConfig)

DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.

type DestinationSyncerList added in v0.8.0 

type DestinationSyncerList struct {
	metav1.TypeMeta `json:",inline"`
	metav1.ListMeta `json:"metadata,omitempty"`
	Items           []DestinationSyncer `json:"items"`
}

DestinationSyncerList contains a list of DestinationSyncer.

func (*DestinationSyncerList) DeepCopy added in v0.8.0 

func (in *DestinationSyncerList) DeepCopy() *DestinationSyncerList

DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DestinationSyncerList.

func (*DestinationSyncerList) DeepCopyInto added in v0.8.0 

func (in *DestinationSyncerList) DeepCopyInto(out *DestinationSyncerList)

DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.

func (*DestinationSyncerList) DeepCopyObject added in v0.8.0 

func (in *DestinationSyncerList) DeepCopyObject() runtime.Object

DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object.

type DestinationSyncerSpec added in v0.8.0 

type DestinationSyncerSpec struct {
	StageName               string                  `json:"stageName,omitempty"`
	SecretRef               string                  `json:"secretRef,omitempty"`
	DependsOn               []StageDependency       `json:"dependsOn,omitempty"`
	DestinationSyncerConfig DestinationSyncerConfig `json:"destinationSyncerConfig,omitempty"`
}

DestinationSyncerSpec defines the desired state of DestinationSyncer.

func (*DestinationSyncerSpec) DeepCopy added in v0.8.0 

func (in *DestinationSyncerSpec) DeepCopy() *DestinationSyncerSpec

DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DestinationSyncerSpec.

func (*DestinationSyncerSpec) DeepCopyInto added in v0.8.0 

func (in *DestinationSyncerSpec) DeepCopyInto(out *DestinationSyncerSpec)

DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.

type DestinationSyncerStatus added in v0.8.0 

type DestinationSyncerStatus struct {
	LastAppliedGeneration int64              `json:"lastAppliedGeneration,omitempty"`
	Conditions            []metav1.Condition `json:"conditions,omitempty"`
	FilesProcessed        int64              `json:"filesProcessed,omitempty"`
}

DestinationSyncerStatus defines the observed state of DestinationSyncer.

func (*DestinationSyncerStatus) DeepCopy added in v0.8.0 

func (in *DestinationSyncerStatus) DeepCopy() *DestinationSyncerStatus

DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DestinationSyncerStatus.

func (*DestinationSyncerStatus) DeepCopyInto added in v0.8.0 

func (in *DestinationSyncerStatus) DeepCopyInto(out *DestinationSyncerStatus)

DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.

type DoclingConfig 

type DoclingConfig struct {
	FromFormats     []string `json:"from_formats,omitempty"`
	ToFormats       []string `json:"to_formats,omitempty"`
	ImageExportMode string   `json:"image_export_mode,omitempty"`
	DoOCR           bool     `json:"do_ocr,omitempty"`
	ForceOCR        bool     `json:"force_ocr,omitempty"`
	OCREngine       string   `json:"ocr_engine,omitempty"`
	OCRLang         []string `json:"ocr_lang,omitempty"`
	PDFBackend      string   `json:"pdf_backend,omitempty"`
	TableMode       string   `json:"table_mode,omitempty"`
	AbortOnError    bool     `json:"abort_on_error,omitempty"`
}

func (*DoclingConfig) DeepCopy 

func (in *DoclingConfig) DeepCopy() *DoclingConfig

DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DoclingConfig.

func (*DoclingConfig) DeepCopyInto 

func (in *DoclingConfig) DeepCopyInto(out *DoclingConfig)

DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.

type DocumentProcessor 

type DocumentProcessor struct {
	metav1.TypeMeta   `json:",inline"`
	metav1.ObjectMeta `json:"metadata,omitempty"`

	Spec   DocumentProcessorSpec   `json:"spec,omitempty"`
	Status DocumentProcessorStatus `json:"status,omitempty"`
}

DocumentProcessor is the Schema for the documentprocessors API

func (*DocumentProcessor) AddOrUpdateJob 

func (d *DocumentProcessor) AddOrUpdateJob(newJob Job)

func (*DocumentProcessor) AddPermanentlyFailingFile 

func (d *DocumentProcessor) AddPermanentlyFailingFile(filePath string)

func (*DocumentProcessor) DeepCopy 

func (in *DocumentProcessor) DeepCopy() *DocumentProcessor

DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DocumentProcessor.

func (*DocumentProcessor) DeepCopyInto 

func (in *DocumentProcessor) DeepCopyInto(out *DocumentProcessor)

DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.

func (*DocumentProcessor) DeepCopyObject 

func (in *DocumentProcessor) DeepCopyObject() runtime.Object

DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object.

func (*DocumentProcessor) DeleteJobByFilePath 

func (d *DocumentProcessor) DeleteJobByFilePath(filePath string)

func (*DocumentProcessor) GetFilesProcessed added in v0.8.0 

func (d *DocumentProcessor) GetFilesProcessed() int64

func (*DocumentProcessor) GetJobByFilePath 

func (d *DocumentProcessor) GetJobByFilePath(filePath string) *Job

func (*DocumentProcessor) IsFilePermanentlyFailing 

func (d *DocumentProcessor) IsFilePermanentlyFailing(filePath string) bool

func (*DocumentProcessor) SetWaiting 

func (d *DocumentProcessor) SetWaiting()

func (*DocumentProcessor) UpdateStatus 

func (d *DocumentProcessor) UpdateStatus(message string, err error)

type DocumentProcessorConfig 

type DocumentProcessorConfig struct {
	Type          string        `json:"type,omitempty"`
	DoclingConfig DoclingConfig `json:"doclingConfig,omitempty"`
}

func (*DocumentProcessorConfig) DeepCopy 

func (in *DocumentProcessorConfig) DeepCopy() *DocumentProcessorConfig

DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DocumentProcessorConfig.

func (*DocumentProcessorConfig) DeepCopyInto 

func (in *DocumentProcessorConfig) DeepCopyInto(out *DocumentProcessorConfig)

DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.

type DocumentProcessorList 

type DocumentProcessorList struct {
	metav1.TypeMeta `json:",inline"`
	metav1.ListMeta `json:"metadata,omitempty"`
	Items           []DocumentProcessor `json:"items"`
}

DocumentProcessorList contains a list of DocumentProcessor

func (*DocumentProcessorList) DeepCopy 

func (in *DocumentProcessorList) DeepCopy() *DocumentProcessorList

DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DocumentProcessorList.

func (*DocumentProcessorList) DeepCopyInto 

func (in *DocumentProcessorList) DeepCopyInto(out *DocumentProcessorList)

DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.

func (*DocumentProcessorList) DeepCopyObject 

func (in *DocumentProcessorList) DeepCopyObject() runtime.Object

DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object.

type DocumentProcessorSpec 

type DocumentProcessorSpec struct {
	StageName               string                  `json:"stageName,omitempty"`
	DependsOn               []StageDependency       `json:"dependsOn,omitempty"`
	DocumentProcessorConfig DocumentProcessorConfig `json:"config,omitempty"`
	// Deprecated: use StageName and DependsOn instead.
	// +optional
	DataProduct string `json:"dataProduct,omitempty"`
}

DocumentProcessorSpec defines the desired state of DocumentProcessor

func (*DocumentProcessorSpec) DeepCopy 

func (in *DocumentProcessorSpec) DeepCopy() *DocumentProcessorSpec

DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DocumentProcessorSpec.

func (*DocumentProcessorSpec) DeepCopyInto 

func (in *DocumentProcessorSpec) DeepCopyInto(out *DocumentProcessorSpec)

DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.

type DocumentProcessorStatus 

type DocumentProcessorStatus struct {
	LastAppliedGeneration   int64              `json:"lastAppliedGeneration,omitempty"`
	Conditions              []metav1.Condition `json:"conditions,omitempty"`
	Jobs                    []Job              `json:"jobs,omitempty"`
	PermanentlyFailingFiles []string           `json:"permanentlyFailingFiles,omitempty"`
	FilesProcessed          int64              `json:"filesProcessed,omitempty"`
}

DocumentProcessorStatus defines the observed state of DocumentProcessor

func (*DocumentProcessorStatus) DeepCopy 

func (in *DocumentProcessorStatus) DeepCopy() *DocumentProcessorStatus

DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DocumentProcessorStatus.

func (*DocumentProcessorStatus) DeepCopyInto 

func (in *DocumentProcessorStatus) DeepCopyInto(out *DocumentProcessorStatus)

DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.

type EmbeddingProvider added in v0.2.0 

type EmbeddingProvider string

type GoogleDriveConfig added in v0.8.0 

type GoogleDriveConfig struct {
	// FolderIDs is the list of Google Drive folder IDs to crawl recursively.
	// +kubebuilder:validation:MinItems=1
	Folders []GoogleDriveFolders `json:"folders"`
	// SkipFolderNames is an optional list of folder names to skip during crawling.
	// +optional
	SkipFolders []SkipFolders `json:"skipFolders,omitempty"`
}

GDriveConfig configures Google Drive folder crawling at the pipeline level. Controller-level settings (maxRetries, concurrency, LDAP) are in ControllerConfig.

func (*GoogleDriveConfig) DeepCopy added in v0.8.0 

func (in *GoogleDriveConfig) DeepCopy() *GoogleDriveConfig

DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new GoogleDriveConfig.

func (*GoogleDriveConfig) DeepCopyInto added in v0.8.0 

func (in *GoogleDriveConfig) DeepCopyInto(out *GoogleDriveConfig)

DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.

type GoogleDriveControllerConfig added in v0.8.0 

type GoogleDriveControllerConfig struct {
	// MaxRetries is the maximum number of retries for Google API calls. Defaults to 3.
	// +optional
	MaxRetries int `json:"maxRetries,omitempty"`
	// ConcurrentFolders is the maximum number of folders to crawl concurrently. Defaults to 5.
	// +optional
	ConcurrentFolders int `json:"concurrentFolders,omitempty"`
	// ConcurrentDownloads is the maximum number of files to download concurrently. Defaults to 10.
	// +optional
	ConcurrentDownloads int `json:"concurrentDownloads,omitempty"`
}

GDriveControllerConfig holds operator-level settings for Google Drive crawling.

func (*GoogleDriveControllerConfig) DeepCopy added in v0.8.0 

func (in *GoogleDriveControllerConfig) DeepCopy() *GoogleDriveControllerConfig

DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new GoogleDriveControllerConfig.

func (*GoogleDriveControllerConfig) DeepCopyInto added in v0.8.0 

func (in *GoogleDriveControllerConfig) DeepCopyInto(out *GoogleDriveControllerConfig)

DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.

type GoogleDriveFolders added in v0.8.0 

type GoogleDriveFolders struct {
	URL string `json:"url"`
}

func (*GoogleDriveFolders) DeepCopy added in v0.8.0 

func (in *GoogleDriveFolders) DeepCopy() *GoogleDriveFolders

DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new GoogleDriveFolders.

func (*GoogleDriveFolders) DeepCopyInto added in v0.8.0 

func (in *GoogleDriveFolders) DeepCopyInto(out *GoogleDriveFolders)

DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.

type Job 

type Job struct {
	FilePath          string        `json:"filePath,omitempty"`
	FileIdentifier    string        `json:"fileIdentifier,omitempty"`
	DocumentConverter string        `json:"documentConverter,omitempty"`
	DoclingConfig     DoclingConfig `json:"doclingConfig,omitempty"`
	TaskID            string        `json:"taskID,omitempty"`
	Status            string        `json:"status,omitempty"`
	CreatedOn         string        `json:"createdOn,omitempty"`
	Attempts          int           `json:"attempts,omitempty"`
}

func (*Job) DeepCopy 

func (in *Job) DeepCopy() *Job

DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new Job.

func (*Job) DeepCopyInto 

func (in *Job) DeepCopyInto(out *Job)

DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.

type LDAPConfig added in v0.8.0 

type LDAPConfig struct {
	// Server is the LDAP server URL (e.g., "ldap://ldap.example.com:389").
	Server string `json:"server"`
	// GroupDN is the base DN for group searches.
	// +optional
	GroupDN string `json:"groupDN,omitempty"`
	// UserDN is the DN template for user lookups (must contain %s for the user ID).
	// +optional
	UserDN string `json:"userDN,omitempty"`
	// BaseUserDN is the base DN for user searches.
	// +optional
	BaseUserDN string `json:"baseUserDN,omitempty"`
	// UserSearchFilter is the LDAP filter for user searches (e.g., "(objectClass=person)").
	// +optional
	UserSearchFilter string `json:"userSearchFilter,omitempty"`
	// EmailAttribute is the LDAP attribute containing user email addresses (e.g., "mail").
	// +optional
	EmailAttribute string `json:"emailAttribute,omitempty"`
	// Attributes is the list of LDAP attributes to retrieve.
	// +optional
	Attributes []string `json:"attributes,omitempty"`
}

LDAPConfig holds configuration for connecting to an LDAP server for identity resolution.

func (*LDAPConfig) DeepCopy added in v0.8.0 

func (in *LDAPConfig) DeepCopy() *LDAPConfig

DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new LDAPConfig.

func (*LDAPConfig) DeepCopyInto added in v0.8.0 

func (in *LDAPConfig) DeepCopyInto(out *LDAPConfig)

DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.

type MarkdownSplitterConfig 

type MarkdownSplitterConfig struct {
	ChunkSize        int  `json:"chunkSize,omitempty"`
	ChunkOverlap     int  `json:"chunkOverlap,omitempty"`
	CodeBlocks       bool `json:"codeBlocks,omitempty"`
	ReferenceLinks   bool `json:"referenceLinks,omitempty"`
	HeadingHierarchy bool `json:"headingHierarchy,omitempty"`
	JoinTableRows    bool `json:"joinTableRows,omitempty"`
}

func (*MarkdownSplitterConfig) DeepCopy 

func (in *MarkdownSplitterConfig) DeepCopy() *MarkdownSplitterConfig

DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new MarkdownSplitterConfig.

func (*MarkdownSplitterConfig) DeepCopyInto 

func (in *MarkdownSplitterConfig) DeepCopyInto(out *MarkdownSplitterConfig)

DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.

type NomicEmbedTextV15Config added in v0.2.0 

type NomicEmbedTextV15Config struct {
	EncodingFormat string `json:"encodingformat,omitempty"`
}

func (*NomicEmbedTextV15Config) DeepCopy added in v0.2.0 

func (in *NomicEmbedTextV15Config) DeepCopy() *NomicEmbedTextV15Config

DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new NomicEmbedTextV15Config.

func (*NomicEmbedTextV15Config) DeepCopyInto added in v0.2.0 

func (in *NomicEmbedTextV15Config) DeepCopyInto(out *NomicEmbedTextV15Config)

DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.

type PipelineStage added in v0.8.0 

type PipelineStage struct {
	// +kubebuilder:validation:Required
	// +kubebuilder:validation:Pattern=`^[a-z0-9]([-a-z0-9]*[a-z0-9])?$`
	Name string `json:"name"`

	// +kubebuilder:validation:Required
	Type StageType `json:"type"`

	// +optional
	DependsOn []StageDependency `json:"dependsOn,omitempty"`

	// +optional
	SourceCrawlerConfig *SourceCrawlerConfig `json:"sourceCrawlerConfig,omitempty"`
	// +optional
	DocumentProcessorConfig *DocumentProcessorConfig `json:"documentProcessorConfig,omitempty"`
	// +optional
	ChunksGeneratorConfig *ChunksGeneratorConfig `json:"chunksGeneratorConfig,omitempty"`
	// +optional
	VectorEmbeddingsGeneratorConfig *VectorEmbeddingsGeneratorConfig `json:"vectorEmbeddingsGeneratorConfig,omitempty"`
	// +optional
	DestinationSyncerConfig *DestinationSyncerConfig `json:"destinationSyncerConfig,omitempty"`

	// +optional
	QueryConfig *QueryConfig `json:"queryConfig,omitempty"`
}

PipelineStage defines a single step in the pipeline DAG. +kubebuilder:validation:XValidation:rule="self.type == 'SourceCrawler' ? has(self.sourceCrawlerConfig) : true",message="sourceCrawlerConfig is required when type is SourceCrawler" +kubebuilder:validation:XValidation:rule="self.type == 'DocumentProcessor' ? has(self.documentProcessorConfig) : true",message="documentProcessorConfig is required when type is DocumentProcessor" +kubebuilder:validation:XValidation:rule="self.type == 'ChunksGenerator' ? has(self.chunksGeneratorConfig) : true",message="chunksGeneratorConfig is required when type is ChunksGenerator" +kubebuilder:validation:XValidation:rule="self.type == 'VectorEmbeddingsGenerator' ? has(self.vectorEmbeddingsGeneratorConfig) : true",message="vectorEmbeddingsGeneratorConfig is required when type is VectorEmbeddingsGenerator" +kubebuilder:validation:XValidation:rule="self.type == 'DestinationSyncer' ? has(self.destinationSyncerConfig) : true",message="destinationSyncerConfig is required when type is DestinationSyncer"

func (*PipelineStage) DeepCopy added in v0.8.0 

func (in *PipelineStage) DeepCopy() *PipelineStage

DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new PipelineStage.

func (*PipelineStage) DeepCopyInto added in v0.8.0 

func (in *PipelineStage) DeepCopyInto(out *PipelineStage)

DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.

type QueryConfig added in v0.8.0 

type QueryConfig struct {
	// +kubebuilder:validation:Required
	Type QueryEndpointType `json:"type"`
	// +optional
	Snowflake *SnowflakeQueryConfig `json:"snowflake,omitempty"`
}

QueryConfig describes where a stage's output data can be queried. This is purely informational metadata for the MCP server — the pipeline controller does not act on it or create the query endpoint. +kubebuilder:validation:XValidation:rule="self.type == 'snowflake' ? has(self.snowflake) : true",message="snowflake is required when type is snowflake"

func (*QueryConfig) DeepCopy added in v0.8.0 

func (in *QueryConfig) DeepCopy() *QueryConfig

DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new QueryConfig.

func (*QueryConfig) DeepCopyInto added in v0.8.0 

func (in *QueryConfig) DeepCopyInto(out *QueryConfig)

DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.

type QueryEndpointType added in v0.8.0 

type QueryEndpointType string

QueryEndpointType identifies the type of query endpoint. +kubebuilder:validation:Enum=snowflake 

const (
	QueryEndpointTypeSnowflake QueryEndpointType = "snowflake"
)

type RecursiveCharacterSplitterConfig 

type RecursiveCharacterSplitterConfig struct {
	Separators    []string `json:"separators,omitempty"`
	ChunkSize     int      `json:"chunkSize,omitempty"`
	ChunkOverlap  int      `json:"chunkOverlap,omitempty"`
	KeepSeparator bool     `json:"keepSeparator,omitempty"`
}

func (*RecursiveCharacterSplitterConfig) DeepCopy 

func (in *RecursiveCharacterSplitterConfig) DeepCopy() *RecursiveCharacterSplitterConfig

DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new RecursiveCharacterSplitterConfig.

func (*RecursiveCharacterSplitterConfig) DeepCopyInto 

func (in *RecursiveCharacterSplitterConfig) DeepCopyInto(out *RecursiveCharacterSplitterConfig)

DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.

type S3Config 

type S3Config struct {
	Bucket      string `json:"bucket"`
	Prefix      string `json:"prefix,omitempty"`
	SQSQueueURL string `json:"sqsQueueURL,omitempty"`
}

S3Config configures an S3 bucket and optional prefix.

func (*S3Config) DeepCopy 

func (in *S3Config) DeepCopy() *S3Config

DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new S3Config.

func (*S3Config) DeepCopyInto 

func (in *S3Config) DeepCopyInto(out *S3Config)

DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.

type SkipFolders added in v0.8.0 

type SkipFolders struct {
	Pattern string `json:"pattern"`
}

func (*SkipFolders) DeepCopy added in v0.8.0 

func (in *SkipFolders) DeepCopy() *SkipFolders

DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new SkipFolders.

func (*SkipFolders) DeepCopyInto added in v0.8.0 

func (in *SkipFolders) DeepCopyInto(out *SkipFolders)

DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.

type SnowflakeConfig deprecated 

type SnowflakeConfig struct {
	Name      string `json:"name,omitempty"`
	Account   string `json:"account,omitempty"`
	User      string `json:"user,omitempty"`
	Role      string `json:"role,omitempty"`
	Region    string `json:"region,omitempty"`
	Warehouse string `json:"warehouse,omitempty"`
}

Deprecated: SnowflakeConfig is no longer used.

func (*SnowflakeConfig) DeepCopy 

func (in *SnowflakeConfig) DeepCopy() *SnowflakeConfig

DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new SnowflakeConfig.

func (*SnowflakeConfig) DeepCopyInto 

func (in *SnowflakeConfig) DeepCopyInto(out *SnowflakeConfig)

DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.

type SnowflakeQueryConfig added in v0.8.0 

type SnowflakeQueryConfig struct {
	Account  string `json:"account"`
	Database string `json:"database"`
	Schema   string `json:"schema"`
	Table    string `json:"table"`
}

SnowflakeQueryConfig contains the connection details for a Snowflake query endpoint.

func (*SnowflakeQueryConfig) DeepCopy added in v0.8.0 

func (in *SnowflakeQueryConfig) DeepCopy() *SnowflakeQueryConfig

DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new SnowflakeQueryConfig.

func (*SnowflakeQueryConfig) DeepCopyInto added in v0.8.0 

func (in *SnowflakeQueryConfig) DeepCopyInto(out *SnowflakeQueryConfig)

DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.

type SourceCrawler added in v0.8.0 

type SourceCrawler struct {
	metav1.TypeMeta   `json:",inline"`
	metav1.ObjectMeta `json:"metadata,omitempty"`

	Spec   SourceCrawlerSpec   `json:"spec,omitempty"`
	Status SourceCrawlerStatus `json:"status,omitempty"`
}

SourceCrawler is the Schema for the sourcecrawlers API.

func (*SourceCrawler) DeepCopy added in v0.8.0 

func (in *SourceCrawler) DeepCopy() *SourceCrawler

DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new SourceCrawler.

func (*SourceCrawler) DeepCopyInto added in v0.8.0 

func (in *SourceCrawler) DeepCopyInto(out *SourceCrawler)

DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.

func (*SourceCrawler) DeepCopyObject added in v0.8.0 

func (in *SourceCrawler) DeepCopyObject() runtime.Object

DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object.

func (*SourceCrawler) GetFilesProcessed added in v0.8.0 

func (c *SourceCrawler) GetFilesProcessed() int64

func (*SourceCrawler) SetWaiting added in v0.8.0 

func (c *SourceCrawler) SetWaiting()

func (*SourceCrawler) UpdateStatus added in v0.8.0 

func (c *SourceCrawler) UpdateStatus(message string, err error)

type SourceCrawlerConfig added in v0.8.0 

type SourceCrawlerConfig struct {
	Type              UnstructuredDataType `json:"type,omitempty"`
	S3Config          S3Config             `json:"s3Config,omitempty"`
	GoogleDriveConfig *GoogleDriveConfig   `json:"googleDriveConfig,omitempty"`
}

SourceCrawlerConfig configures where to read unstructured data from.

func (*SourceCrawlerConfig) DeepCopy added in v0.8.0 

func (in *SourceCrawlerConfig) DeepCopy() *SourceCrawlerConfig

DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new SourceCrawlerConfig.

func (*SourceCrawlerConfig) DeepCopyInto added in v0.8.0 

func (in *SourceCrawlerConfig) DeepCopyInto(out *SourceCrawlerConfig)

DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.

type SourceCrawlerList added in v0.8.0 

type SourceCrawlerList struct {
	metav1.TypeMeta `json:",inline"`
	metav1.ListMeta `json:"metadata,omitempty"`
	Items           []SourceCrawler `json:"items"`
}

SourceCrawlerList contains a list of SourceCrawler.

func (*SourceCrawlerList) DeepCopy added in v0.8.0 

func (in *SourceCrawlerList) DeepCopy() *SourceCrawlerList

DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new SourceCrawlerList.

func (*SourceCrawlerList) DeepCopyInto added in v0.8.0 

func (in *SourceCrawlerList) DeepCopyInto(out *SourceCrawlerList)

DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.

func (*SourceCrawlerList) DeepCopyObject added in v0.8.0 

func (in *SourceCrawlerList) DeepCopyObject() runtime.Object

DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object.

type SourceCrawlerSpec added in v0.8.0 

type SourceCrawlerSpec struct {
	StageName           string              `json:"stageName,omitempty"`
	SecretRef           string              `json:"secretRef,omitempty"`
	DependsOn           []StageDependency   `json:"dependsOn,omitempty"`
	SourceCrawlerConfig SourceCrawlerConfig `json:"sourceCrawlerConfig,omitempty"`
}

SourceCrawlerSpec defines the desired state of SourceCrawler.

func (*SourceCrawlerSpec) DeepCopy added in v0.8.0 

func (in *SourceCrawlerSpec) DeepCopy() *SourceCrawlerSpec

DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new SourceCrawlerSpec.

func (*SourceCrawlerSpec) DeepCopyInto added in v0.8.0 

func (in *SourceCrawlerSpec) DeepCopyInto(out *SourceCrawlerSpec)

DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.

type SourceCrawlerStatus added in v0.8.0 

type SourceCrawlerStatus struct {
	LastAppliedGeneration int64              `json:"lastAppliedGeneration,omitempty"`
	Conditions            []metav1.Condition `json:"conditions,omitempty"`
	FilesProcessed        int64              `json:"filesProcessed,omitempty"`
}

SourceCrawlerStatus defines the observed state of SourceCrawler.

func (*SourceCrawlerStatus) DeepCopy added in v0.8.0 

func (in *SourceCrawlerStatus) DeepCopy() *SourceCrawlerStatus

DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new SourceCrawlerStatus.

func (*SourceCrawlerStatus) DeepCopyInto added in v0.8.0 

func (in *SourceCrawlerStatus) DeepCopyInto(out *SourceCrawlerStatus)

DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.

type StageCreationStatus added in v0.8.0 

type StageCreationStatus struct {
	Name    string `json:"name"`
	Created bool   `json:"created"`
}

StageCreationStatus tracks whether a child CR has been created for a stage.

func (*StageCreationStatus) DeepCopy added in v0.8.0 

func (in *StageCreationStatus) DeepCopy() *StageCreationStatus

DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new StageCreationStatus.

func (*StageCreationStatus) DeepCopyInto added in v0.8.0 

func (in *StageCreationStatus) DeepCopyInto(out *StageCreationStatus)

DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.

type StageDependency added in v0.8.0 

type StageDependency struct {
	Name string `json:"name"`
}

StageDependency identifies an upstream stage by name.

func (*StageDependency) DeepCopy added in v0.8.0 

func (in *StageDependency) DeepCopy() *StageDependency

DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new StageDependency.

func (*StageDependency) DeepCopyInto added in v0.8.0 

func (in *StageDependency) DeepCopyInto(out *StageDependency)

DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.

type StageMapping added in v0.8.0 

type StageMapping struct {
	Type       StageType
	Object     client.Object
	ObjectList client.ObjectList
}

+kubebuilder:object:generate=false

func ListStages added in v0.8.0 

func ListStages() []StageMapping

type StageType added in v0.8.0 

type StageType string

StageType is the discriminator for the stage union config. +kubebuilder:validation:Enum=SourceCrawler;DocumentProcessor;ChunksGenerator;VectorEmbeddingsGenerator;DestinationSyncer 

const (
	StageTypeSourceCrawler             StageType = "SourceCrawler"
	StageTypeDocumentProcessor         StageType = "DocumentProcessor"
	StageTypeChunksGenerator           StageType = "ChunksGenerator"
	StageTypeVectorEmbeddingsGenerator StageType = "VectorEmbeddingsGenerator"
	StageTypeDestinationSyncer         StageType = "DestinationSyncer"
)

type TokenSplitterConfig 

type TokenSplitterConfig struct {
	ChunkSize         int      `json:"chunkSize,omitempty"`
	ChunkOverlap      int      `json:"chunkOverlap,omitempty"`
	ModelName         string   `json:"modelName,omitempty"`
	EncodingName      string   `json:"encodingName,omitempty"`
	AllowedSpecial    []string `json:"allowedSpecial,omitempty"`
	DisallowedSpecial []string `json:"disallowedSpecial,omitempty"`
}

func (*TokenSplitterConfig) DeepCopy 

func (in *TokenSplitterConfig) DeepCopy() *TokenSplitterConfig

DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new TokenSplitterConfig.

func (*TokenSplitterConfig) DeepCopyInto 

func (in *TokenSplitterConfig) DeepCopyInto(out *TokenSplitterConfig)

DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.

type UnstructuredDataPipeline added in v0.8.0 

type UnstructuredDataPipeline struct {
	metav1.TypeMeta   `json:",inline"`
	metav1.ObjectMeta `json:"metadata,omitempty"`

	Spec   UnstructuredDataPipelineSpec   `json:"spec,omitempty"`
	Status UnstructuredDataPipelineStatus `json:"status,omitempty"`
}

UnstructuredDataPipeline is the Schema for the unstructureddatapipelines API

func (*UnstructuredDataPipeline) DeepCopy added in v0.8.0 

func (in *UnstructuredDataPipeline) DeepCopy() *UnstructuredDataPipeline

DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new UnstructuredDataPipeline.

func (*UnstructuredDataPipeline) DeepCopyInto added in v0.8.0 

func (in *UnstructuredDataPipeline) DeepCopyInto(out *UnstructuredDataPipeline)

DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.

func (*UnstructuredDataPipeline) DeepCopyObject added in v0.8.0 

func (in *UnstructuredDataPipeline) DeepCopyObject() runtime.Object

DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object.

func (*UnstructuredDataPipeline) SetWaiting added in v0.8.0 

func (u *UnstructuredDataPipeline) SetWaiting()

func (*UnstructuredDataPipeline) UpdateStatus added in v0.8.0 

func (u *UnstructuredDataPipeline) UpdateStatus(message string, err error)

type UnstructuredDataPipelineList added in v0.8.0 

type UnstructuredDataPipelineList struct {
	metav1.TypeMeta `json:",inline"`
	metav1.ListMeta `json:"metadata,omitempty"`
	Items           []UnstructuredDataPipeline `json:"items"`
}

UnstructuredDataPipelineList contains a list of UnstructuredDataPipeline

func (*UnstructuredDataPipelineList) DeepCopy added in v0.8.0 

func (in *UnstructuredDataPipelineList) DeepCopy() *UnstructuredDataPipelineList

DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new UnstructuredDataPipelineList.

func (*UnstructuredDataPipelineList) DeepCopyInto added in v0.8.0 

func (in *UnstructuredDataPipelineList) DeepCopyInto(out *UnstructuredDataPipelineList)

DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.

func (*UnstructuredDataPipelineList) DeepCopyObject added in v0.8.0 

func (in *UnstructuredDataPipelineList) DeepCopyObject() runtime.Object

DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object.

type UnstructuredDataPipelineSpec added in v0.8.0 

type UnstructuredDataPipelineSpec struct {
	// Description is a human-readable summary of what the pipeline does.
	// +kubebuilder:validation:Required
	// +kubebuilder:validation:MinLength=1
	Description string `json:"description"`
	// +optional
	SecretRef string `json:"secretRef,omitempty"`
	// +kubebuilder:validation:MinItems=1
	// +listType=map
	// +listMapKey=name
	Stages []PipelineStage `json:"stages"`
}

UnstructuredDataPipelineSpec defines the desired state of UnstructuredDataPipeline

func (*UnstructuredDataPipelineSpec) DeepCopy added in v0.8.0 

func (in *UnstructuredDataPipelineSpec) DeepCopy() *UnstructuredDataPipelineSpec

DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new UnstructuredDataPipelineSpec.

func (*UnstructuredDataPipelineSpec) DeepCopyInto added in v0.8.0 

func (in *UnstructuredDataPipelineSpec) DeepCopyInto(out *UnstructuredDataPipelineSpec)

DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.

type UnstructuredDataPipelineStatus added in v0.8.0 

type UnstructuredDataPipelineStatus struct {
	LastAppliedGeneration int64                 `json:"lastAppliedGeneration,omitempty"`
	Conditions            []metav1.Condition    `json:"conditions,omitempty"`
	Stages                []StageCreationStatus `json:"stages,omitempty"`
}

UnstructuredDataPipelineStatus defines the observed state of UnstructuredDataPipeline

func (*UnstructuredDataPipelineStatus) DeepCopy added in v0.8.0 

func (in *UnstructuredDataPipelineStatus) DeepCopy() *UnstructuredDataPipelineStatus

DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new UnstructuredDataPipelineStatus.

func (*UnstructuredDataPipelineStatus) DeepCopyInto added in v0.8.0 

func (in *UnstructuredDataPipelineStatus) DeepCopyInto(out *UnstructuredDataPipelineStatus)

DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.

type UnstructuredDataProcessingConfigSpec deprecated 

type UnstructuredDataProcessingConfigSpec struct {
	MaxConcurrentDoclingTasks   int    `json:"maxConcurrentDoclingTasks,omitempty"`
	MaxConcurrentLangchainTasks int    `json:"maxConcurrentLangchainTasks,omitempty"`
	IngestionBucket             string `json:"ingestionBucket,omitempty"`
	DoclingServeURL             string `json:"doclingServeURL,omitempty"`
	CacheDirectory              string `json:"cacheDirectory,omitempty"`
	DataStorageBucket           string `json:"dataStorageBucket,omitempty"`
}

Deprecated: fields are now top-level on ControllerConfigSpec.

func (*UnstructuredDataProcessingConfigSpec) DeepCopy 

func (in *UnstructuredDataProcessingConfigSpec) DeepCopy() *UnstructuredDataProcessingConfigSpec

DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new UnstructuredDataProcessingConfigSpec.

func (*UnstructuredDataProcessingConfigSpec) DeepCopyInto 

func (in *UnstructuredDataProcessingConfigSpec) DeepCopyInto(out *UnstructuredDataProcessingConfigSpec)

DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.

type UnstructuredDataType added in v0.2.0 

type UnstructuredDataType string

type VectorEmbeddingsGenerator added in v0.2.0 

type VectorEmbeddingsGenerator struct {
	metav1.TypeMeta   `json:",inline"`
	metav1.ObjectMeta `json:"metadata,omitempty"`

	Spec   VectorEmbeddingsGeneratorSpec   `json:"spec,omitempty"`
	Status VectorEmbeddingsGeneratorStatus `json:"status,omitempty"`
}

VectorEmbeddingsGenerator is the Schema for the vectorembeddingsgenerators API.

func (*VectorEmbeddingsGenerator) DeepCopy added in v0.2.0 

func (in *VectorEmbeddingsGenerator) DeepCopy() *VectorEmbeddingsGenerator

DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new VectorEmbeddingsGenerator.

func (*VectorEmbeddingsGenerator) DeepCopyInto added in v0.2.0 

func (in *VectorEmbeddingsGenerator) DeepCopyInto(out *VectorEmbeddingsGenerator)

DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.

func (*VectorEmbeddingsGenerator) DeepCopyObject added in v0.2.0 

func (in *VectorEmbeddingsGenerator) DeepCopyObject() runtime.Object

DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object.

func (*VectorEmbeddingsGenerator) GetFilesProcessed added in v0.8.0 

func (c *VectorEmbeddingsGenerator) GetFilesProcessed() int64

func (*VectorEmbeddingsGenerator) SetWaiting added in v0.2.0 

func (c *VectorEmbeddingsGenerator) SetWaiting()

func (*VectorEmbeddingsGenerator) UpdateStatus added in v0.2.0 

func (c *VectorEmbeddingsGenerator) UpdateStatus(message string, err error)

type VectorEmbeddingsGeneratorConfig added in v0.2.0 

type VectorEmbeddingsGeneratorConfig struct {
	ModelName string `json:"modelName,omitempty"`
	// +kubebuilder:validation:Minimum=1
	// +optional
	BatchSize               int                     `json:"batchSize,omitempty"`
	NomicEmbedTextV15Config NomicEmbedTextV15Config `json:"nomicEmbedTextV15Config,omitempty"`
}

func (*VectorEmbeddingsGeneratorConfig) DeepCopy added in v0.2.0 

func (in *VectorEmbeddingsGeneratorConfig) DeepCopy() *VectorEmbeddingsGeneratorConfig

DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new VectorEmbeddingsGeneratorConfig.

func (*VectorEmbeddingsGeneratorConfig) DeepCopyInto added in v0.2.0 

func (in *VectorEmbeddingsGeneratorConfig) DeepCopyInto(out *VectorEmbeddingsGeneratorConfig)

DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.

type VectorEmbeddingsGeneratorList added in v0.2.0 

type VectorEmbeddingsGeneratorList struct {
	metav1.TypeMeta `json:",inline"`
	metav1.ListMeta `json:"metadata,omitempty"`
	Items           []VectorEmbeddingsGenerator `json:"items"`
}

VectorEmbeddingsGeneratorList contains a list of VectorEmbeddingsGenerator.

func (*VectorEmbeddingsGeneratorList) DeepCopy added in v0.2.0 

func (in *VectorEmbeddingsGeneratorList) DeepCopy() *VectorEmbeddingsGeneratorList

DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new VectorEmbeddingsGeneratorList.

func (*VectorEmbeddingsGeneratorList) DeepCopyInto added in v0.2.0 

func (in *VectorEmbeddingsGeneratorList) DeepCopyInto(out *VectorEmbeddingsGeneratorList)

DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.

func (*VectorEmbeddingsGeneratorList) DeepCopyObject added in v0.2.0 

func (in *VectorEmbeddingsGeneratorList) DeepCopyObject() runtime.Object

DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object.

type VectorEmbeddingsGeneratorSpec added in v0.2.0 

type VectorEmbeddingsGeneratorSpec struct {
	StageName                       string                          `json:"stageName,omitempty"`
	DependsOn                       []StageDependency               `json:"dependsOn,omitempty"`
	VectorEmbeddingsGeneratorConfig VectorEmbeddingsGeneratorConfig `json:"embeddingGeneratorConfig,omitempty"`
	// Deprecated: use StageName and DependsOn instead.
	// +optional
	DataProduct string `json:"dataProduct,omitempty"`
}

VectorEmbeddingsGeneratorSpec defines the desired state of VectorEmbeddingsGenerator.

func (*VectorEmbeddingsGeneratorSpec) DeepCopy added in v0.2.0 

func (in *VectorEmbeddingsGeneratorSpec) DeepCopy() *VectorEmbeddingsGeneratorSpec

DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new VectorEmbeddingsGeneratorSpec.

func (*VectorEmbeddingsGeneratorSpec) DeepCopyInto added in v0.2.0 

func (in *VectorEmbeddingsGeneratorSpec) DeepCopyInto(out *VectorEmbeddingsGeneratorSpec)

DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.

type VectorEmbeddingsGeneratorStatus added in v0.2.0 

type VectorEmbeddingsGeneratorStatus struct {
	LastAppliedGeneration int64              `json:"lastAppliedGeneration,omitempty"`
	Conditions            []metav1.Condition `json:"conditions,omitempty"`
	FilesProcessed        int64              `json:"filesProcessed,omitempty"`
}

VectorEmbeddingsGeneratorStatus defines the observed state of VectorEmbeddingsGenerator.

func (*VectorEmbeddingsGeneratorStatus) DeepCopy added in v0.2.0 

func (in *VectorEmbeddingsGeneratorStatus) DeepCopy() *VectorEmbeddingsGeneratorStatus

DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new VectorEmbeddingsGeneratorStatus.

func (*VectorEmbeddingsGeneratorStatus) DeepCopyInto added in v0.2.0 

func (in *VectorEmbeddingsGeneratorStatus) DeepCopyInto(out *VectorEmbeddingsGeneratorStatus)

DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.