utils

package
v0.9.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 29, 2026 License: Apache-2.0, MIT Imports: 37 Imported by: 0

Documentation

Index

Examples

Constants

View Source
const EncPrefix = "enc:v1:"

EncPrefix marks a string as AES-256-GCM encrypted

View Source
const MySQLSSRFNetwork = "tcp-weknora-ssrf"

MySQLSSRFNetwork is the mysql driver network name registered by RegisterMySQLSSRFDialer. DSNs must set cfg.Net to this value so every database/sql open uses SSRFSafeDialContext at the final TCP sink.

Variables

View Source
var AllowedStdioCommands = map[string]bool{
	"uvx": true,
	"npx": true,
}

AllowedStdioCommands defines the whitelist of allowed commands for MCP stdio transport These are the standard MCP server launchers that are considered safe

View Source
var DangerousArgPatterns = []*regexp.Regexp{
	regexp.MustCompile(`(?i)^-c$`),
	regexp.MustCompile(`(?i)^--command$`),
	regexp.MustCompile(`(?i)^-e$`),
	regexp.MustCompile(`(?i)^--eval$`),
	regexp.MustCompile(`(?i)[;&|]`),
	regexp.MustCompile(`(?i)\$\(`),
	regexp.MustCompile("(?i)`"),
	regexp.MustCompile(`(?i)>\s*[/~]`),
	regexp.MustCompile(`(?i)<\s*[/~]`),
	regexp.MustCompile(`(?i)^/bin/`),
	regexp.MustCompile(`(?i)^/usr/bin/`),
	regexp.MustCompile(`(?i)^/sbin/`),
	regexp.MustCompile(`(?i)^/usr/sbin/`),
	regexp.MustCompile(`(?i)^\.\./`),
	regexp.MustCompile(`(?i)/\.\./`),
	regexp.MustCompile(`(?i)^(bash|sh|zsh|ksh|csh|tcsh|fish|dash)$`),
	regexp.MustCompile(`(?i)^(curl|wget|nc|netcat|ncat)$`),
	regexp.MustCompile(`(?i)^(rm|dd|mkfs|fdisk)$`),
}

DangerousArgPatterns contains patterns that indicate potentially dangerous arguments

View Source
var DangerousEnvVarPatterns = []*regexp.Regexp{
	regexp.MustCompile(`(?i)^LD_PRELOAD$`),
	regexp.MustCompile(`(?i)^LD_LIBRARY_PATH$`),
	regexp.MustCompile(`(?i)^DYLD_`),
	regexp.MustCompile(`(?i)^PATH$`),
	regexp.MustCompile(`(?i)^PYTHONPATH$`),
	regexp.MustCompile(`(?i)^NODE_OPTIONS$`),
	regexp.MustCompile(`(?i)^BASH_ENV$`),
	regexp.MustCompile(`(?i)^ENV$`),
	regexp.MustCompile(`(?i)^SHELL$`),
}

DangerousEnvVarPatterns contains patterns for dangerous environment variable names or values

View Source
var ErrEncryptedDataMissingKey = errors.New("encrypted data found but SYSTEM_AES_KEY is not set or has wrong length")

ErrEncryptedDataMissingKey is returned by DecryptStoredSecret when the value carries the enc:v1: prefix but no AES key is available to decrypt it. This signals an operator misconfiguration (typically a rotated or unset SYSTEM_AES_KEY) and must propagate so the system fails loudly instead of silently using ciphertext as a credential.

View Source
var ErrSSRFHostNotWhitelisted = errors.New("host is not in the SSRF whitelist")

ErrSSRFHostNotWhitelisted is what whitelist-only mode returns for a host outside SSRF_WHITELIST. It is raised before the name is resolved.

View Source
var ErrSSRFRedirectBlocked = fmt.Errorf("redirect blocked: target URL failed SSRF validation")

ErrSSRFRedirectBlocked is returned when a redirect target is blocked due to SSRF protection

Functions

func ApplyCustomHeaders

func ApplyCustomHeaders(req *http.Request, headers map[string]string)

ApplyCustomHeaders 将用户自定义的 header 写入 http.Request。 保留 header(Authorization、api-key、Content-Type 等)会被跳过以避免破坏鉴权/签名。 其它 header 会直接覆盖同名条目,允许用户替换默认值(例如 Accept)。

func CheckDialWhitelistOnly

func CheckDialWhitelistOnly(host string) error

CheckDialWhitelistOnly is CheckSSRFWhitelistOnly for a dial address, and it judges hostnames only. An IP literal arriving at a dialer is either a name something else already resolved — gRPC's own resolver runs before the dialer it was given, so a whitelisted service name reaches this point as an address — or a literal the URL layer already refused. Refusing it again here would break clients whose whitelisted host was resolved for them, while adding nothing the URL layer does not already do.

func CheckRunningTaskStatus

func CheckRunningTaskStatus(ctx context.Context, redisClient *redis.Client, runningKey, progressKey string) (map[string]interface{}, error)

CheckRunningTaskStatus 检查指定running task的状态

func CheckSSRFWhitelistOnly

func CheckSSRFWhitelistOnly(host string) error

CheckSSRFWhitelistOnly refuses a host that whitelist-only mode does not admit. Paths that still hold the URL run it before resolving a name, so a host outside the whitelist never produces a DNS query; with the mode off it admits everything and the usual checks apply. An IP literal is judged here too: the URL layer is where "no direct IP" belongs.

func CleanMarkdown

func CleanMarkdown(input string) string

CleanMarkdown 清理 Markdown 内容

func CleanupStaleRunningTasks

func CleanupStaleRunningTasks(ctx context.Context, redisClient *redis.Client, keyPrefix string, maxAge time.Duration) (int, error)

CleanupStaleRunningTasks 清理可能残留的running task keys 这是一个调试和维护工具,可以用来清理因异常情况导致的残留running keys

func CompactImageDataURLForLog

func CompactImageDataURLForLog(raw string) string

CompactImageDataURLForLog shortens large image data URLs for log output.

func DecryptAESGCM

func DecryptAESGCM(encrypted string, key []byte) (string, error)

DecryptAESGCM decrypts an AES-256-GCM encrypted string. If the string lacks the enc:v1: prefix, it's treated as legacy plaintext and returned as-is.

func DecryptStoredSecret

func DecryptStoredSecret(encrypted string) (string, error)

DecryptStoredSecret decrypts a value loaded from the database with strict error propagation. Use this in hot-path code that is about to actually USE the credential (e.g. building an upstream HTTP request) — a rotated or missing key must surface as a loud error rather than silently degrading to "no credential", because the user expects the configured upstream call to either work or fail with a clear "auth misconfigured" message.

Behaviour:

  • empty input -> empty output, no error.
  • no enc:v1: prefix -> returned as-is (legacy plaintext column), no error.
  • has enc:v1: prefix and SYSTEM_AES_KEY is missing or wrong length -> returns ErrEncryptedDataMissingKey.
  • has enc:v1: prefix and key is set -> decrypts, returns any decryption error verbatim (e.g. base64 decode failure, GCM auth tag mismatch from a rotated key).

For GORM Scan paths that load a whole row (where erroring out hides the rest of the resource from the user) use DecryptStoredSecretLenient instead — it lets the row load with the secret blanked out so the UI can render "credential not configured" and the user can re-enter it, without crashing list endpoints when the operator rotates or removes SYSTEM_AES_KEY.

func DecryptStoredSecretLenient

func DecryptStoredSecretLenient(encrypted string) (plaintext string, ok bool)

DecryptStoredSecretLenient is the load-path counterpart for use inside GORM Scan / AfterFind hooks. It prefers graceful degradation over failing the whole row load.

Returns:

  • (plaintext, true) for empty input, legacy plaintext, or successful decrypt
  • ("", false) when the value has the enc:v1: prefix but cannot be decrypted (SYSTEM_AES_KEY missing, rotated, or ciphertext corrupted). The bool=false case is the operator's signal: log a warning and treat the field as unconfigured. The row still loads so the rest of the resource displays normally; the UI shows "credential not configured", and any upstream call attempted with this empty value fails with a clear "missing api key" rather than sending ciphertext as the key.

Rationale: a single line of broken ciphertext used to crash entire list endpoints (e.g. ListModels returning "" because one row failed Scan), hiding all other models from the user. Lenient load + loud per-row log is strictly more recoverable than fail-fast Scan.

func DownloadBytes

func DownloadBytes(url string) ([]byte, error)

DownloadBytes fetches the content at the given HTTP(S) URL and returns the raw bytes. It reuses a package-level http.Client with a 60-second timeout. Responses exceeding the configured MAX_FILE_SIZE_MB limit are rejected.

func EncryptAESGCM

func EncryptAESGCM(plaintext string, key []byte) (string, error)

EncryptAESGCM encrypts plaintext with AES-256-GCM. Returns the original string if empty, already encrypted, or key is nil.

func EscapeHTML

func EscapeHTML(input string) string

EscapeHTML 转义 HTML 特殊字符

func ExtractShellAssignments

func ExtractShellAssignments(command string) map[string]string

ExtractShellAssignments returns NAME=value pairs from a model-built command.

func FormatSSRFError

func FormatSSRFError(label, rawURL string, err error) string

FormatSSRFError takes the error returned by ValidateURLForSSRF and wraps it with operator guidance — specifically how to add a host to the SSRF allow-list. Without this hint, users hit "Base URL 未通过安全校验" with no idea how to recover (the allowlist is configured server-side, not in the UI). The hint references SSRF_WHITELIST_EXTRA rather than SSRF_WHITELIST because the latter is the project's baseline list and EXTRA is the operator's append-only escape hatch.

`label` is a short noun describing the URL field that failed, e.g. "Base URL" or "VLM Base URL". The function returns an empty string for a nil err so callers can use it inline without guarding.

func GenerateSchema

func GenerateSchema[T any]() json.RawMessage

GenerateSchema generates JSON schema for type T and returns it as a map This is optimized to avoid unnecessary serialization/deserialization

func GenerateTaskID

func GenerateTaskID(taskType string, tenantID uint64, businessID ...string) string

GenerateTaskID generates a unique task ID with multiple collision-resistant elements. The format is: <taskType>_<tenantID>_<timestamp>_<uuid>_<businessID>

Parameters:

  • taskType: Type of task (e.g., "faq_import", "kb_clone")
  • tenantID: Tenant ID for multi-tenancy isolation
  • businessID: Optional business-specific ID (e.g., knowledge base ID)

Returns a task ID like: "faq_import_12345_1704628851692_a1b2c3d4_kb789"

func GenerateTaskIDWithPrefix

func GenerateTaskIDWithPrefix(prefix string, tenantID uint64, businessID ...string) string

GenerateTaskIDWithPrefix generates a task ID with a custom prefix. This is useful when you want more control over the task ID format.

func GetAESKey

func GetAESKey() []byte

GetAESKey reads the 32-byte AES key from SYSTEM_AES_KEY env. Returns nil if not set or not exactly 32 bytes.

func GetContentTypeByExt

func GetContentTypeByExt(ext string) string

GetContentTypeByExt returns the content type based on file extension

func GetMaxFileSize

func GetMaxFileSize() int64

GetMaxFileSize returns the maximum file upload size in bytes. Default is 50MB, can be configured via MAX_FILE_SIZE_MB environment variable.

MAX_FILE_SIZE_MB is intentionally a deploy-time-only knob (NOT a runtime system_setting). The effective upload limit is gated by three other layers that all read this env at startup and cache the value:

  • frontend nginx client_max_body_size (envsubst into nginx.conf)
  • docreader gRPC max_send/recv_message_length
  • frontend client-side check via window.__RUNTIME_CONFIG__

Surfacing a SystemAdmin UI knob whose effect is silently capped by any of the above would mislead operators ("I raised it to 200MB but nginx still returns 413"). Until all four layers can be reconfigured in lockstep without container restarts, every call site must read the env directly via this helper.

func GetMaxFileSizeMB

func GetMaxFileSizeMB() int64

GetMaxFileSizeMB returns the maximum file upload size in MB. Same caveat as GetMaxFileSize — handlers should prefer SystemSettingService.GetInt.

func GetMaxSkillBundleSize

func GetMaxSkillBundleSize() int64

GetMaxSkillBundleSize is the compressed zip / source-download cap for skills. Knowledge uploads stay on GetMaxFileSize: packages such as ppt-master exceed 50MB, but raising the document limit would also inflate docreader gRPC messages. A GitHub zipball is the whole repository archive, not the SKILL.md subtree, so a huge monorepo can still be refused here even when the skill itself is small.

func GetMaxSkillBundleSizeMB

func GetMaxSkillBundleSizeMB() int64

GetMaxSkillBundleSizeMB returns the skill zip cap in MB. MAX_SKILL_BUNDLE_SIZE_MB defaults to 256, is never below MAX_FILE_SIZE_MB, and is capped at 512.

func InjectAndConditions

func InjectAndConditions(sql, filter string) string

InjectAndConditions combines parsed expressions, preserving literal and alias boundaries. An invalid statement or filter returns no executable SQL.

func IsActiveBrowserContentExt

func IsActiveBrowserContentExt(ext string) bool

IsActiveBrowserContentExt reports whether a file extension can execute script or otherwise participate as active same-origin browser content when served inline. User-uploaded files with these extensions must be downloaded instead of previewed inline.

func IsPublicIP

func IsPublicIP(ip net.IP) bool

IsPublicIP returns true if the IP is safe for outbound fetch (not private, loopback, link-local, etc.). Used for DNS pinning: after resolving a hostname we pick the first public IP and pin all requests to it.

func IsReservedHeader

func IsReservedHeader(key string) bool

IsReservedHeader 判断某个 header key 是否为保留 header,保留 header 不允许被自定义头覆盖。

func IsSSRFWhitelisted

func IsSSRFWhitelisted(hostname string) bool

IsSSRFWhitelisted checks whether the given hostname (or IP string) is covered by the SSRF_WHITELIST environment variable.

func IsSupportedImportExtension

func IsSupportedImportExtension(ext string) bool

IsSupportedImportExtension reports whether a bare extension can be imported.

func IsSystemProxy

func IsSystemProxy(host string) bool

IsSystemProxy 判断是否为系统代理

func IsValidImageURL

func IsValidImageURL(url string) bool

IsValidImageURL 验证图片 URL 是否安全

func IsValidURL

func IsValidURL(url string) bool

IsValidURL 验证 URL 是否安全

func MaskCommandAssignments

func MaskCommandAssignments(command string) string

MaskCommandAssignments replaces the value of every NAME=value assignment with a placeholder. A command is logged at Info, and passing a credential inline is a documented way to hand a skill its key, so the raw string must never reach the log.

func NewSSRFSafeHTTPClient

func NewSSRFSafeHTTPClient(config SSRFSafeHTTPClientConfig) *http.Client

NewSSRFSafeHTTPClient creates an HTTP client that validates redirect targets against SSRF protections. This prevents SSRF attacks via HTTP redirects where an attacker's server redirects to internal services. Each call builds a dedicated transport; callers that create many short-lived clients against the same upstream should share one NewSSRFSafeTransport via NewSSRFSafeHTTPClientWithTransport instead.

func NewSSRFSafeHTTPClientWithTransport

func NewSSRFSafeHTTPClientWithTransport(
	config SSRFSafeHTTPClientConfig, transport http.RoundTripper,
) *http.Client

NewSSRFSafeHTTPClientWithTransport wraps a caller-supplied transport in an *http.Client carrying the given timeout and the SSRF-aware redirect policy. Pass a transport from NewSSRFSafeTransport (optionally shared across clients) to reuse a single connection pool while keeping per-client timeouts.

func NewSSRFSafeTransport

func NewSSRFSafeTransport(config SSRFSafeHTTPClientConfig) *http.Transport

NewSSRFSafeTransport builds an *http.Transport whose connections are guarded by SSRFSafeDialContext. The transport carries no per-request timeout and no redirect policy — those live on the *http.Client — so a single transport can be shared across many clients to pool keep-alive connections globally.

func NormalizeImportExtension

func NormalizeImportExtension(ext string) string

NormalizeImportExtension lowercases an extension and strips surrounding whitespace and one leading dot, so callers can pass "xlsx", ".XLSX" or a raw user-supplied file_type.

func OutboundDialTimeout

func OutboundDialTimeout() time.Duration

OutboundDialTimeout returns the current per-connection dial timeout.

func ParseTaskID

func ParseTaskID(taskID string) (taskType string, tenantID uint64, timestamp int64, uuidPart string, businessID string, err error)

ParseTaskID parses a task ID generated by GenerateTaskID and returns its components. Returns taskType, tenantID, timestamp, uuid, businessID, and error.

Task types may contain underscores (e.g. "faq_import", "kb_clone"), so the parser locates the tenant/timestamp pair rather than assuming parts[0] is the full task type.

func ParseTenantIDFromStoragePath

func ParseTenantIDFromStoragePath(filePath string) uint64

ParseTenantIDFromStoragePath extracts the tenant ID from a provider:// storage path. Storage paths follow the convention: {scheme}://.../{tenantID}/... Returns 0 if the path does not contain a valid tenant ID.

NOTE: For cloud providers whose paths embed numeric bucket or region names before the tenant segment, the first numeric segment may not be the tenant. Callers that have an authoritative resource-owner tenant ID available should pass it directly to SignFileURL instead of relying on this parser.

func RegisterMySQLSSRFDialer

func RegisterMySQLSSRFDialer()

RegisterMySQLSSRFDialer installs a go-sql-driver/mysql dialer that routes connections through SSRFSafeDialContext. Safe to call from multiple packages; registration happens at most once per process.

func ResetSSRFOutboundValidationCacheForTest

func ResetSSRFOutboundValidationCacheForTest()

ResetSSRFOutboundValidationCacheForTest clears the outbound validation cache. NOT for production use.

func ResetSSRFWhitelistForTest

func ResetSSRFWhitelistForTest()

ResetSSRFWhitelistForTest resets the whitelist singleton so tests in any package can re-read the SSRF_WHITELIST environment variable after changing it. Exported (rather than unexported) because callers exist outside internal/utils — notably internal/infrastructure/web_search/searxng_test.go, whose tests would otherwise see whatever whitelist an alphabetically- earlier test in the same binary (e.g. proxy_test.go's TestValidateProxyURL) cached via the first sync.Once.Do(). NOT for production use — the ForTest suffix is the contract.

func SSRFOutboundValidationMissesForTest

func SSRFOutboundValidationMissesForTest() uint64

SSRFOutboundValidationMissesForTest returns how many uncached outbound validations have run since the last ResetSSRFOutboundValidationCacheForTest.

func SSRFSafeDialContext

func SSRFSafeDialContext(ctx context.Context, network, addr string) (net.Conn, error)

SSRFSafeDialContext is a custom dial function that validates the resolved IP addresses before establishing a connection. This provides an additional layer of SSRF protection against DNS rebinding attacks during the connection phase.

func SSRFSafeGRPCDialer

func SSRFSafeGRPCDialer(ctx context.Context, addr string) (net.Conn, error)

SSRFSafeGRPCDialer is compatible with grpc.WithContextDialer and pins DNS answers the same way as SSRFSafeDialContext.

func SSRFWhitelistOnlyEnabled

func SSRFWhitelistOnlyEnabled() bool

SSRFWhitelistOnlyEnabled reports whether SSRF_DNS_WHITELIST_ONLY turns the whitelist into the whole egress policy: a host outside it is refused before any DNS query, and a hostname counts as whitelisted by name only, never by what it resolves to (#3378).

The value is parsed with strconv.ParseBool, so 1/t/T/TRUE/true/True enable it and 0/f/F/FALSE/false/False disable it. A non-empty value that parses as neither enables it: silently leaving an egress lockdown off because an operator wrote "yes" is the one failure mode this control must not have.

func SafeContentTypeByFilename

func SafeContentTypeByFilename(filename string) (string, bool)

SafeContentTypeByFilename returns a browser-safe content type for serving user-controlled files and whether that file may be displayed inline.

func SafeFileName

func SafeFileName(fileName string) (string, error)

SafeFileName 校验并返回安全的“仅文件名”部分,防止路径遍历。 仅保留最后一个路径成分,禁止 ".."、空名或仅含点,用于 SaveBytes 等场景。

func SafeJoinUnderBase

func SafeJoinUnderBase(baseDir, relPath string) (string, error)

SafeJoinUnderBase 把调用方提供的相对后缀拼到 baseDir 下,并返回仍落在 baseDir 内的绝对路径。首尾分隔符会被去掉,避免 "/etc" 一类输入覆盖根目录; ".." 经 path.Clean 后若仍指向父级则拒绝。空后缀表示 baseDir 本身。

func SafeObjectKey

func SafeObjectKey(objectKey string) error

SafeObjectKey 校验对象存储的 key(如 COS/MinIO objectName),禁止包含 ".." 等路径遍历

func SafePathUnderBase

func SafePathUnderBase(baseDir, filePath string) (string, error)

SafePathUnderBase 校验 filePath 是否落在 baseDir 下,防止路径遍历(如 ../../)。 返回规范化的绝对路径;若路径逃逸出 baseDir 则返回错误。

func SanitizeForDisplay

func SanitizeForDisplay(input string) string

SanitizeForDisplay 为显示清理内容

func SanitizeForLog

func SanitizeForLog(input string) string

SanitizeForLog 清理日志输入,防止日志注入攻击 日志注入攻击是指攻击者通过在输入中插入换行符和其他控制字符, 伪造日志条目,可能导致日志分析工具误判或隐藏恶意活动

func SanitizeForLogArray

func SanitizeForLogArray(input []string) []string

SanitizeForLogArray 清理日志输入数组,防止日志注入攻击

func SanitizeHTML

func SanitizeHTML(input string) string

SanitizeHTML 清理 HTML 内容,防止 XSS 攻击

func SanitizeStorageConnectivityError

func SanitizeStorageConnectivityError(err error) string

SanitizeStorageConnectivityError converts a raw storage connectivity error into a safe, user-facing message. It deliberately avoids echoing the raw driver/network error so responses never leak internal hostnames, IPs, ports or TLS/certificate details. Callers that need the full error must log it server-side instead of returning it to the client.

func SetOutboundDialTimeout

func SetOutboundDialTimeout(d time.Duration)

SetOutboundDialTimeout overrides the per-connection dial timeout used by SSRFSafeDialContext. It is called once at startup from config.LoadConfig and is safe to call concurrently. Non-positive values are ignored.

func SetSSRFWhitelistFromRaw

func SetSSRFWhitelistFromRaw(raw string)

SetSSRFWhitelistFromRaw atomically replaces the active SSRF whitelist with the parse of `raw` (comma-separated entries, same syntax as the SSRF_WHITELIST env var). The new whitelist takes effect for every subsequent ValidateURLForSSRF call across all goroutines without additional synchronisation.

Called by SystemSettingService at preload, after each Update, and after each pubsub-driven peer change. Empty `raw` clears the whitelist (only built-in private-IP rejection remains in effect).

Note: this replaces the ENV-only fallback completely. If you want SSRF_WHITELIST_EXTRA to keep being merged, the caller must do the merge before calling this — see service.systemSettingService. applySSRFWhitelist for the canonical merge logic.

func SignFileURL

func SignFileURL(baseURL, filePath string, tenantID uint64, ttl time.Duration) (string, error)

SignFileURL generates a presigned HTTP URL for accessing a storage file. baseURL is the external URL of the WeKnora instance (e.g. "https://weknora.example.com"). filePath is the provider:// storage path (e.g. "local://1/abc/img.png"). tenantID identifies the tenant that owns the file. ttl is how long the URL remains valid (0 uses the default presignDefaultTTL).

Returns ("", error) if the signing key is not configured.

func SignOIDCState

func SignOIDCState(payload *OIDCStatePayload) (string, error)

SignOIDCState returns a tamper-evident state token: base64url(payload).base64url(hmac).

func SupportedImportExtensions

func SupportedImportExtensions() []string

SupportedImportExtensions returns a sorted copy of the supported extensions.

func SystemHMACKey

func SystemHMACKey() []byte

SystemHMACKey returns the deployment-wide HMAC key derived from SYSTEM_SIGNING_KEY (falling back to SYSTEM_AES_KEY for existing deployments), or nil when unset, too short, or a known example value. Callers must treat nil as "this deployment cannot sign", not as an empty key.

func TaskTenantID

func TaskTenantID(taskID string) (uint64, error)

TaskTenantID extracts the tenant ID embedded in a GenerateTaskID task ID.

func ToJSON

func ToJSON(v interface{}) string

ToJSON converts a value to a JSON string

func ValidateInput

func ValidateInput(input string) (string, bool)

ValidateInput 验证用户输入

func ValidateInputSyntax

func ValidateInputSyntax(input string) (string, bool)

ValidateInputSyntax checks the input properties shared by all user-query entry points. It deliberately does not apply the XSS pattern list because some callers, such as AgentQA, accept frontend code as conversation text.

func ValidateKBScopedStoragePath

func ValidateKBScopedStoragePath(filePath string, tenantID uint64) error

ValidateKBScopedStoragePath is used by GET /knowledge-bases/:id/files. It requires the path to belong to the KB owner tenant and to live under the exports/ namespace used for embedded images (SaveBytes / multimodal output). This prevents borrowers with shared-KB read access from using the proxy to fetch arbitrary owner-tenant objects such as raw knowledge uploads.

func ValidateSQL

func ValidateSQL(sql string, opts ...SQLValidationOption) (*SQLParseResult, *SQLValidationResult)

ValidateSQL validates a SQL statement with the given options

Example
// Example 1: Validate table names
sql1 := "SELECT * FROM users WHERE age > 18"
_, validation1 := ValidateSQL(sql1, WithAllowedTables("users", "orders"))
fmt.Printf("Example 1 - Valid: %v\n", validation1.Valid)

// Example 2: Detect SQL injection
sql2 := "SELECT * FROM users WHERE id = 1 OR 1=1"
_, validation2 := ValidateSQL(sql2, WithInjectionRiskCheck())
fmt.Printf("Example 2 - Valid: %v\n", validation2.Valid)
if !validation2.Valid {
	fmt.Printf("Error: %s\n", validation2.Errors[0].Message)
}

// Example 3: Combined validation
sql3 := "SELECT * FROM products WHERE 1=1"
_, validation3 := ValidateSQL(sql3,
	WithAllowedTables("users", "orders"),
	WithInjectionRiskCheck(),
)
fmt.Printf("Example 3 - Valid: %v, Error count: %d\n", validation3.Valid, len(validation3.Errors))
Output:
Example 1 - Valid: true
Example 2 - Valid: false
Error: Potential SQL injection risk detected
Example 3 - Valid: false, Error count: 2

func ValidateSSRFWhitelistEntries

func ValidateSSRFWhitelistEntries(entries []string) error

ValidateSSRFWhitelistEntries returns nil when every entry in `entries` would be accepted by parseSSRFWhitelistRaw, or an error describing the first malformed entry. Used by the system_settings Update path to give the UI a clear 400 instead of silently dropping bad input at parse-time.

Validation rules mirror parseSSRFWhitelistRaw exactly:

  • "<a>/<b>" must be a valid CIDR
  • "*.<domain>" must have a non-empty domain after the prefix
  • mid-string "*" is not supported
  • everything else is treated as an exact host or literal IP (we don't pre-resolve DNS here; that's a runtime concern)

func ValidateStdioArgs

func ValidateStdioArgs(args []string) error

ValidateStdioArgs validates the arguments for MCP stdio transport Returns an error if any argument contains dangerous patterns

func ValidateStdioCommand

func ValidateStdioCommand(command string) error

ValidateStdioCommand validates the command for MCP stdio transport Returns an error if the command is not in the whitelist or contains dangerous patterns

func ValidateStdioConfig

func ValidateStdioConfig(command string, args []string, envVars map[string]string) error

ValidateStdioConfig performs comprehensive validation of stdio configuration This should be called before creating or executing any stdio-based MCP client

func ValidateStdioEnvVars

func ValidateStdioEnvVars(envVars map[string]string) error

ValidateStdioEnvVars validates environment variables for MCP stdio transport Returns an error if any env var name or value is dangerous

func ValidateStoragePathTenant

func ValidateStoragePathTenant(filePath string, tenantID uint64) error

ValidateStoragePathTenant ensures the tenant segment embedded in a provider:// storage path matches the authenticated caller's tenant. Cross-tenant access for arbitrary tenant paths uses /api/v1/files/presigned with an HMAC bound to the resource owner; KB-scoped shared rendering uses ValidateKBScopedStoragePath.

func ValidateTaskID

func ValidateTaskID(taskID string) error

ValidateTaskID reports whether taskID is safe to embed in file names and Redis keys: non-empty, bounded, and only [A-Za-z0-9_-] (no separators or "..").

func ValidateURLForSSRF

func ValidateURLForSSRF(rawURL string) error

ValidateURLForSSRF is the centralised entry-point that all handlers should call to validate a user-supplied URL. It first checks the SSRF_WHITELIST; whitelisted hosts skip the full isSSRFSafeURL check.

rawURL may be a full URL ("https://example.com/v1") or a bare host/host:port (for cases like ReconnectDocReader). If a scheme is missing the function prepends "https://" before parsing so that net/url can extract the host.

Returns nil when the URL is safe, or an error describing the problem.

func VerifyFileURLSig

func VerifyFileURLSig(filePath string, tenantID uint64, expiresStr, sig string) bool

VerifyFileURLSig checks the HMAC signature and expiry of a presigned URL. Returns true only if the signature is valid and the URL has not expired.

Types

type OIDCStatePayload

type OIDCStatePayload struct {
	Nonce       string `json:"nonce"`
	RedirectURI string `json:"redirect_uri,omitempty"`
	IssuedAt    int64  `json:"iat"`
}

OIDCStatePayload is the signed OIDC authorization state carried in the redirect URL and validated on callback.

func VerifyOIDCState

func VerifyOIDCState(raw string) (*OIDCStatePayload, error)

VerifyOIDCState validates the HMAC and freshness of a state token.

type SQLParseResult

type SQLParseResult struct {
	IsSelect     bool     `json:"is_select"`             // Whether the SQL is a SELECT statement
	TableNames   []string `json:"table_names"`           // List of table names in FROM clause
	SelectFields []string `json:"select_fields"`         // List of fields in SELECT clause
	WhereFields  []string `json:"where_fields"`          // List of fields in WHERE clause
	WhereClause  string   `json:"where_clause"`          // Complete WHERE clause text
	OriginalSQL  string   `json:"original_sql"`          // Original SQL statement
	ParseError   string   `json:"parse_error,omitempty"` // Error message if parsing failed
}

SQLParseResult represents the parsed components of a SELECT SQL statement

func ParseSQL

func ParseSQL(sql string) *SQLParseResult

ParseSQL parses a SQL statement using pg_query_go and extracts table names, select fields, and where fields This uses the PostgreSQL parser for accurate SQL parsing

Example
sql := "SELECT id, name, email FROM users WHERE age > 18 AND status = 'active'"
result := ParseSQL(sql)

fmt.Printf("Is SELECT: %v\n", result.IsSelect)
fmt.Printf("Tables: %v\n", result.TableNames)
fmt.Printf("SELECT fields: %v\n", result.SelectFields)
fmt.Printf("WHERE fields: %v\n", result.WhereFields)
fmt.Printf("WHERE clause: %s\n", result.WhereClause)
Output:
Is SELECT: true
Tables: [users]
SELECT fields: [id name email]
WHERE fields: [age status]
WHERE clause: age > 18 AND status = 'active'

type SQLValidationError

type SQLValidationError struct {
	Type    string `json:"type"`    // Error type: "table_not_allowed", "sql_injection_risk", etc.
	Message string `json:"message"` // Error message
	Details string `json:"details"` // Additional details
}

SQLValidationError represents a validation error

type SQLValidationOption

type SQLValidationOption func(*sqlValidator)

SQLValidationOption is a function that configures SQL validation

func WithAllowedFunctions

func WithAllowedFunctions(functions ...string) SQLValidationOption

WithAllowedFunctions creates a validation option that checks if functions are in the allowed list

func WithAllowedTables

func WithAllowedTables(tables ...string) SQLValidationOption

WithAllowedTables creates a validation option that checks if table names are in the allowed list

func WithChunkEnabledFilter

func WithChunkEnabledFilter() SQLValidationOption

WithChunkEnabledFilter excludes disabled chunks from model-visible SQL queries. It is intentionally opt-in because administrative queries may need to inspect disabled rows.

func WithDefaultSafeFunctions

func WithDefaultSafeFunctions() SQLValidationOption

WithDefaultSafeFunctions enables a default set of safe SQL functions

func WithHiddenKBFilter

func WithHiddenKBFilter() SQLValidationOption

WithHiddenKBFilter excludes internal/temporary knowledge bases (is_temporary = true) from query results. These are system-managed KBs like __chat_history__ that should not be visible to end users.

func WithInjectionRiskCheck

func WithInjectionRiskCheck() SQLValidationOption

WithInjectionRiskCheck creates a validation option that checks for SQL injection risks

func WithInputValidation

func WithInputValidation(minLen, maxLen int) SQLValidationOption

WithInputValidation enables basic input validation (length, null bytes, etc.)

func WithNoCTEs

func WithNoCTEs() SQLValidationOption

WithNoCTEs blocks Common Table Expressions (WITH clause)

func WithNoDangerousFunctions

func WithNoDangerousFunctions() SQLValidationOption

WithNoDangerousFunctions blocks dangerous PostgreSQL functions

func WithNoSchemaAccess

func WithNoSchemaAccess() SQLValidationOption

WithNoSchemaAccess blocks schema-qualified access (except public schema)

func WithNoSubqueries

func WithNoSubqueries() SQLValidationOption

WithNoSubqueries blocks all subqueries

func WithNoSystemColumns

func WithNoSystemColumns() SQLValidationOption

WithNoSystemColumns blocks access to PostgreSQL system columns

func WithSearchScopeFilter

func WithSearchScopeFilter(kbIDs []string, knowledgeIDs []string) SQLValidationOption

WithSearchScopeFilter restricts queries to the specified knowledge bases and (optionally) specific knowledge documents. For the knowledge_bases table it filters by id; for knowledges it filters by knowledge_base_id (and id when knowledgeIDs is non-empty); for chunks it filters by knowledge_base_id (and knowledge_id when knowledgeIDs is non-empty).

func WithSearchScopes

func WithSearchScopes(scopes []SearchScope) SQLValidationOption

WithSearchScopes restricts queries using structured OR scopes. Each scope can represent a full KB, specific documents, or a tag-constrained KB. Nonzero TenantID values grant access to that owner within that scope, so scopes must come from server-side authorization, never directly from untrusted input.

func WithSecurityDefaults

func WithSecurityDefaults(tenantID uint64) SQLValidationOption

WithSecurityDefaults applies a comprehensive set of security validations

func WithSelectOnly

func WithSelectOnly() SQLValidationOption

WithSelectOnly ensures only SELECT statements are allowed

func WithSingleStatement

func WithSingleStatement() SQLValidationOption

WithSingleStatement ensures only single statement is allowed (no multiple statements)

func WithSoftDeleteFilter

func WithSoftDeleteFilter(tables ...string) SQLValidationOption

WithSoftDeleteFilter enables automatic deleted_at IS NULL injection.

func WithTenantIsolation

func WithTenantIsolation(tenantID uint64, tables ...string) SQLValidationOption

WithTenantIsolation enables automatic tenant_id injection for multi-tenant security

type SQLValidationResult

type SQLValidationResult struct {
	Valid  bool                 `json:"valid"`  // Whether the SQL passed validation
	Errors []SQLValidationError `json:"errors"` // List of validation errors
}

SQLValidationResult represents the result of SQL validation

func ValidateAndSecureSQL

func ValidateAndSecureSQL(sql string, opts ...SQLValidationOption) (string, *SQLValidationResult, error)

ValidateAndSecureSQL validates SQL and returns a secured version with tenant isolation This is a convenience function that combines validation and SQL rewriting

type SSRFSafeHTTPClientConfig

type SSRFSafeHTTPClientConfig struct {
	SameOriginRedirectsOnly bool
	Timeout                 time.Duration
	MaxRedirects            int
	DisableKeepAlives       bool
	DisableCompression      bool
}

SSRFSafeHTTPClientConfig contains configuration for the SSRF-safe HTTP client

func DefaultSSRFSafeHTTPClientConfig

func DefaultSSRFSafeHTTPClientConfig() SSRFSafeHTTPClientConfig

DefaultSSRFSafeHTTPClientConfig returns the default configuration

type SSRFValidatingRoundTripper

type SSRFValidatingRoundTripper struct {
	Base http.RoundTripper
}

SSRFValidatingRoundTripper enforces the URL policy for every outbound request, including URLs discovered at runtime by SDKs (for example OAuth metadata) that never passed through an application handler. Dial-time checks remain necessary to pin DNS answers and cover transports that cannot accept this wrapper directly.

func (*SSRFValidatingRoundTripper) RoundTrip

func (t *SSRFValidatingRoundTripper) RoundTrip(req *http.Request) (*http.Response, error)

type SearchScope

type SearchScope struct {
	// TenantID is the authorized owner of this KB. Zero uses the caller tenant
	// when tenant isolation is enabled. Scopes must be authorized by the caller.
	TenantID        uint64
	KnowledgeBaseID string
	KnowledgeIDs    []string
	TagIDs          []string
}

SearchScope describes one allowed knowledge scope for SQL query injection. Empty KnowledgeIDs and TagIDs means the whole KB is in scope.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL