Documentation
¶
Index ¶
- type APIError
- type Chunk
- type ChunkOptions
- type ChunkParser
- type ConversionResult
- type ConvertOptions
- type ConvertParser
- type Converter
- type DoclingClient
- func (c *DoclingClient) ChunkHybrid(ctx context.Context, filename string, r io.Reader, opts ChunkOptions) ([]Chunk, error)
- func (c *DoclingClient) ChunkHybridBytes(ctx context.Context, data []byte, filename string, opts ChunkOptions) ([]Chunk, error)
- func (c *DoclingClient) ChunkHybridFile(ctx context.Context, filePath string, opts ChunkOptions) ([]Chunk, error)
- func (c *DoclingClient) ConvertBytes(ctx context.Context, data []byte, filename string) (*ConversionResult, error)
- func (c *DoclingClient) ConvertBytesWithOptions(ctx context.Context, data []byte, filename string, opts ConvertOptions) (*ConversionResult, error)
- func (c *DoclingClient) ConvertFile(ctx context.Context, filePath string) (*ConversionResult, error)
- func (c *DoclingClient) ConvertFileWithOptions(ctx context.Context, filePath string, opts ConvertOptions) (*ConversionResult, error)
- func (c *DoclingClient) ConvertURL(ctx context.Context, docURL string) (*ConversionResult, error)
- func (c *DoclingClient) IsAvailable(ctx context.Context) bool
- type ExtractedImage
- type GenericClient
- func (g *GenericClient) ChunkHybridBytes(ctx context.Context, data []byte, filename string, opts ChunkOptions) ([]Chunk, error)
- func (g *GenericClient) ChunkHybridFile(ctx context.Context, filePath string, opts ChunkOptions) ([]Chunk, error)
- func (g *GenericClient) ConvertBytes(ctx context.Context, data []byte, filename string) (*ConversionResult, error)
- func (g *GenericClient) ConvertFile(ctx context.Context, filePath string) (*ConversionResult, error)
- func (g *GenericClient) ConvertURL(context.Context, string) (*ConversionResult, error)
- func (g *GenericClient) IsAvailable(ctx context.Context) bool
- type GenericConfig
- type HealthChecker
- type HybridChunker
- type Option
- func WithAPIKey(key string) Option
- func WithHTTPClient(client *http.Client) Option
- func WithHealthCacheTTL(ttl time.Duration) Option
- func WithMaxResponseBytes(n int64) Option
- func WithRetry(maxAttempts int, baseDelay, maxDelay time.Duration) Option
- func WithTenantID(id string) Option
- func WithTimeout(d time.Duration) Option
- func WithUserAgent(userAgent string) Option
- type RetryConfig
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Chunk ¶
type Chunk struct {
Filename string `json:"filename"`
ChunkIndex int `json:"chunk_index"`
Text string `json:"text"`
RawText string `json:"raw_text"`
NumTokens *int `json:"num_tokens"`
Headings []string `json:"headings"`
Captions []string `json:"captions"`
PageNumbers []int `json:"page_numbers"`
DocItems []string `json:"doc_items"`
Metadata map[string]any `json:"metadata"`
}
Chunk is one docling-serve document-aware chunk.
type ChunkOptions ¶
type ChunkOptions struct {
MaxTokens int
Tokenizer string
MergePeers *bool
UseMarkdownTables *bool
UseMarkdownImages *bool
ImagePlaceholder string
IncludeRawText *bool
ConvertOptions ConvertOptions
}
ChunkOptions tunes docling-serve's hybrid chunker.
type ChunkParser ¶
ChunkParser translates a server's raw chunk-endpoint response body into seshat's own []Chunk shape. Client's own parseChunkResponse is one example.
type ConversionResult ¶
type ConversionResult struct {
// Markdown is the full document rendered as Markdown.
Markdown string
// Images are the pictures extracted from the document (inline figures, diagrams).
// Each image carries a base64-encoded data URI so the agent can view them.
Images []ExtractedImage
// PageCount is the number of pages (0 if unknown).
PageCount int
}
ConversionResult is what we get back from docling-serve for a single file.
type ConvertOptions ¶
type ConvertOptions struct {
FromFormats []string
ToFormats []string
DoOCR *bool
ForceOCR *bool
DoTableStructure *bool
TableMode string
PDFBackend string
Pipeline string
OCRLang []string
ImageExportMode string
IncludePageImages *bool
IncludeImages *bool
AbortOnError *bool
DocumentTimeoutSec *float64
ImagesScale *float64
}
ConvertOptions tunes docling-serve conversion. Zero values keep server defaults, which is the safest behavior across docling-serve versions.
type ConvertParser ¶
type ConvertParser func(rawBody []byte) (*ConversionResult, error)
ConvertParser translates a server's raw convert-endpoint response body into seshat's own ConversionResult shape. Client's own parseResponse (docling-serve's JSON shape) is one example; a different server gets its own parser doing the equivalent field-by-field translation.
type Converter ¶
type Converter interface {
HealthChecker
ConvertFile(ctx context.Context, filePath string) (*ConversionResult, error)
ConvertBytes(ctx context.Context, data []byte, filename string) (*ConversionResult, error)
ConvertURL(ctx context.Context, docURL string) (*ConversionResult, error)
}
Converter is anything that can convert documents to markdown. *DoclingClient (this package) is the default implementation, backed by docling-serve - every consumer in internal/tools and internal/pdfsmart depends on this interface rather than *DoclingClient directly, so a host application can supply its own implementation (e.g. one backed by a different service) via pkg/sdk.ClientConfig.DocumentConverter instead of being forced onto docling-serve's wire format.
type DoclingClient ¶
type DoclingClient struct {
// contains filtered or unexported fields
}
DoclingClient calls a running docling-serve instance to convert documents to markdown.
func NewDoclingClient ¶
func NewDoclingClient(baseURL string, opts ...Option) *DoclingClient
NewDoclingClient creates a client pointing at a docling-serve base URL. baseURL is typically "http://localhost:5001". Defaults to a 120s per-request timeout - pass WithTimeout to override it.
func (*DoclingClient) ChunkHybrid ¶
func (c *DoclingClient) ChunkHybrid(ctx context.Context, filename string, r io.Reader, opts ChunkOptions) ([]Chunk, error)
ChunkHybrid sends a document to docling-serve's hybrid chunk endpoint.
func (*DoclingClient) ChunkHybridBytes ¶
func (c *DoclingClient) ChunkHybridBytes(ctx context.Context, data []byte, filename string, opts ChunkOptions) ([]Chunk, error)
ChunkHybridBytes chunks an in-memory document without writing it to disk.
func (*DoclingClient) ChunkHybridFile ¶
func (c *DoclingClient) ChunkHybridFile(ctx context.Context, filePath string, opts ChunkOptions) ([]Chunk, error)
ChunkHybridFile sends filePath to docling-serve's hybrid chunk endpoint.
func (*DoclingClient) ConvertBytes ¶
func (c *DoclingClient) ConvertBytes(ctx context.Context, data []byte, filename string) (*ConversionResult, error)
ConvertBytes converts an in-memory document without writing it to disk first. filename is used only to hint the MIME type to docling (e.g. "report.pdf").
func (*DoclingClient) ConvertBytesWithOptions ¶
func (c *DoclingClient) ConvertBytesWithOptions(ctx context.Context, data []byte, filename string, opts ConvertOptions) (*ConversionResult, error)
ConvertBytesWithOptions converts an in-memory document with conversion options.
func (*DoclingClient) ConvertFile ¶
func (c *DoclingClient) ConvertFile(ctx context.Context, filePath string) (*ConversionResult, error)
ConvertFile sends filePath to docling-serve and returns the markdown + images.
func (*DoclingClient) ConvertFileWithOptions ¶
func (c *DoclingClient) ConvertFileWithOptions(ctx context.Context, filePath string, opts ConvertOptions) (*ConversionResult, error)
ConvertFileWithOptions sends filePath to docling-serve with conversion options.
func (*DoclingClient) ConvertURL ¶
func (c *DoclingClient) ConvertURL(ctx context.Context, docURL string) (*ConversionResult, error)
ConvertURL fetches and converts a remote document (e.g. an arXiv PDF URL).
func (*DoclingClient) IsAvailable ¶
func (c *DoclingClient) IsAvailable(ctx context.Context) bool
IsAvailable does a cheap health check against the running service.
type ExtractedImage ¶
type ExtractedImage struct {
// Filename is the suggested on-disk name (e.g. "image_0001.png").
Filename string
// MimeType is the image MIME type (e.g. "image/png").
MimeType string
// Base64 is the raw base64-encoded image bytes (no data-URI prefix).
Base64 string
}
ExtractedImage is one picture found inside the converted document.
type GenericClient ¶
type GenericClient struct {
// contains filtered or unexported fields
}
GenericClient is a configurable client for any document-intelligence HTTP server that accepts a multipart file upload and returns JSON - the shape docling-serve, seshat-intelligence, and similar services all share, even though each names its own endpoints and JSON fields differently.
Unlike Client (docling-serve-specific: hardcoded paths, a fixed JSON shape, docling-serve's own conversion/chunking option names), every protocol detail here is supplied by the caller at construction time via GenericConfig: paths, the multipart file field name, and functions translating that server's own JSON into the same ConversionResult/Chunk types Client produces. The rest of seshat (internal/rag.HybridDocumentChunker, internal/pdfsmart, the file-read/convert tools) depends only on the Converter/HybridChunker interfaces, so it works unchanged regardless of which concrete backend a host configures.
A concrete backend's own adapter is typically small: paths + a parser function mapping its response fields onto ConversionResult/Chunk, then NewGenericClient. It doesn't need to live in this package, or even this repo - GenericClient (and the ConvertParser/ChunkParser it's built from) are exported specifically so a private downstream repo (e.g. seshat-backend, for its own seshat-intelligence deployment) can build one without needing a change here first.
func NewGenericClient ¶
func NewGenericClient(cfg GenericConfig) (*GenericClient, error)
NewGenericClient validates cfg and builds a GenericClient. Returns an error rather than panicking or silently producing an unusable client, since a misconfigured backend adapter (e.g. forgetting ParseConvert) would otherwise only fail at first use, far from the actual mistake.
func (*GenericClient) ChunkHybridBytes ¶
func (g *GenericClient) ChunkHybridBytes(ctx context.Context, data []byte, filename string, opts ChunkOptions) ([]Chunk, error)
ChunkHybridBytes satisfies HybridChunker. The options go to the server as the form fields GenericConfig.ChunkFields gives them; without that mapping the file is sent alone, and the server chunks as it likes.
func (*GenericClient) ChunkHybridFile ¶
func (g *GenericClient) ChunkHybridFile(ctx context.Context, filePath string, opts ChunkOptions) ([]Chunk, error)
ChunkHybridFile mirrors Client's own convenience method of the same name (see ChunkHybridBytes's doc comment for why opts is unused here).
func (*GenericClient) ConvertBytes ¶
func (g *GenericClient) ConvertBytes(ctx context.Context, data []byte, filename string) (*ConversionResult, error)
func (*GenericClient) ConvertFile ¶
func (g *GenericClient) ConvertFile(ctx context.Context, filePath string) (*ConversionResult, error)
func (*GenericClient) ConvertURL ¶
func (g *GenericClient) ConvertURL(context.Context, string) (*ConversionResult, error)
ConvertURL is not supported by GenericClient - a URL-based conversion endpoint's request shape (JSON body vs. docling-serve's own {"sources":[...]}) is specific enough per server that it isn't worth generalizing here without a second real backend that needs it. Returns a clear error rather than silently failing a caller that assumed every Converter supports it.
func (*GenericClient) IsAvailable ¶
func (g *GenericClient) IsAvailable(ctx context.Context) bool
type GenericConfig ¶
type GenericConfig struct {
// BaseURL is the server's base URL, e.g. "http://localhost:5100" for a
// local seshat-intelligence instance. Required.
BaseURL string
// FileField is the multipart form field name the server expects the
// uploaded file under. docling-serve uses "files"; seshat-intelligence
// uses "file" (FastAPI's UploadFile parameter is named `file` in its
// route signature, and FastAPI binds the multipart field name to the
// parameter name). Required.
FileField string
// ConvertPath is the endpoint ConvertFile/ConvertBytes POST the file
// to, e.g. "/v1/documents" for seshat-intelligence (vs docling-serve's
// "/v1/convert/file"). Leave empty (with ParseConvert nil) to leave
// conversion unsupported - ConvertFile/ConvertBytes/ConvertURL then
// return a clear error instead of attempting a request.
ConvertPath string
ParseConvert ConvertParser
// ChunkPath is the endpoint ChunkHybridBytes/ChunkHybridFile POST the
// file to, e.g. "/v1/documents/chunks" for seshat-intelligence (vs
// docling-serve's "/v1/chunk/hybrid/file"). Leave empty (with
// ParseChunk nil) to leave chunking unsupported.
ChunkPath string
ParseChunk ChunkParser
// ChunkFields maps the ChunkOptions of a call (the size the chunks should have, for one) onto the form fields the
// server's chunk endpoint takes, by name. docling-serve has its own mapping (see ChunkOptions); an arbitrary
// server has its own names, or none. Leave nil to send the file alone.
ChunkFields func(ChunkOptions) map[string]string
// HealthPath is the endpoint IsAvailable GETs. Defaults to "/health" -
// true for both docling-serve and seshat-intelligence today, but
// overridable for a server that names it differently.
HealthPath string
// The remaining fields mirror Client's own Options (see their doc
// comments in client.go) - a plain config struct here instead of the
// same functional-options pattern, since a concrete backend's adapter
// typically builds one GenericConfig value up front rather than
// composing it incrementally.
HTTPClient *http.Client
Timeout time.Duration
APIKey string
TenantID string
UserAgent string
MaxResponseBytes int64
Retry RetryConfig
// HealthCacheTTL caches IsAvailable results for this duration, mirroring
// WithHealthCacheTTL. nil (the zero value - easy to get by accident by
// simply not setting this field) keeps Client's own default (5s), not
// "disabled" - a pointer, not a plain time.Duration, specifically so an
// unset field can't be silently misread as an explicit request to
// disable caching. Pass a pointer to 0 to actually disable it.
HealthCacheTTL *time.Duration
}
GenericConfig configures a GenericClient. BaseURL and FileField are always required; ConvertPath/ParseConvert and ChunkPath/ParseChunk are each optional as a pair - configure whichever capabilities the target server actually offers (see NewGenericClient's validation).
type HealthChecker ¶
HealthChecker reports whether a document-intelligence backend is reachable. Embedded by both interfaces below rather than duplicated on each.
type HybridChunker ¶
type HybridChunker interface {
HealthChecker
ChunkHybridBytes(ctx context.Context, data []byte, filename string, opts ChunkOptions) ([]Chunk, error)
}
HybridChunker is anything that can produce document-aware hybrid chunks. *DoclingClient (this package) is the default implementation, backed by docling-serve's hybrid chunk endpoint - internal/rag.HybridDocumentChunker depends on this interface rather than *DoclingClient directly, so a caller assembling its own rag.Service can supply a different backend.
type Option ¶
type Option func(*DoclingClient)
Option configures a DoclingClient at construction time.
func WithAPIKey ¶
WithAPIKey sets the X-Api-Key header sent to docling-serve.
func WithHTTPClient ¶
WithHTTPClient replaces the HTTP client used for docling-serve calls.
func WithHealthCacheTTL ¶
WithHealthCacheTTL caches IsAvailable results for the given duration.
func WithMaxResponseBytes ¶
WithMaxResponseBytes adjusts the response body safety limit.
func WithTenantID ¶
WithTenantID sets the X-Tenant-Id header sent to docling-serve.
func WithTimeout ¶
WithTimeout overrides the default 120s per-request HTTP timeout. Pass a larger budget for deployments that expect large or scan-heavy documents (see defaultTimeout's doc comment for why the default is often too short), or a smaller one for latency-sensitive callers that would rather fail fast than wait.
func WithUserAgent ¶
WithUserAgent sets the User-Agent header sent to docling-serve.