store

package
v0.0.0-...-cd5cb31 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 15, 2026 License: MIT Imports: 19 Imported by: 0

Documentation

Overview

Package store persists Factor configuration and asynchronous recalc jobs.

Index

Constants

View Source
const (
	SubmitAccepted  = "accepted"
	SubmitDuplicate = "duplicate"
	SubmitRejected  = "rejected"
)

参数提交结果。

View Source
const (
	RecalcStatusAccepted  = "accepted"
	RecalcStatusRunning   = "running"
	RecalcStatusSucceeded = "succeeded"
	RecalcStatusFailed    = "failed"
	RecalcStatusCancelled = "cancelled"
)

Variables

View Source
var ErrConflict = errors.New("factor store conflict")

Functions

func UDFBindingsOf

func UDFBindingsOf(members []domain.MemberDef) map[string]string

UDFBindingsOf 汇总成员定义引用的自定义算子版本(名称 → 版本哈希);编译失败的定义不贡献绑定。

func UDFFromVersion

func UDFFromVersion(v UdfVersion) (factorexpr.UDF, error)

UDFFromVersion 把存储的版本转换成 factorexpr 的自定义算子定义。

func WithUDFPins

func WithUDFPins(ctx context.Context, pins map[string]string) context.Context

WithUDFPins 返回带固定绑定(名称 → 版本哈希)的上下文:读取定义时,pins 里的名称按固定版本解析而不是当前启用版本。 补算作业用它:首次领取时固化的绑定在续跑、重领、重启后都不变。

Types

type EngineHeartbeat

type EngineHeartbeat struct {
	EngineID string
	BootID   string
	At       time.Time
}

EngineHeartbeat 是最近一次被接受的引擎心跳。

type Options

type Options struct {
	Path            string
	MaxIdleConns    int
	MaxOpenConns    int
	ConnMaxLifetime time.Duration
	ConnMaxIdleTime time.Duration
}

Options configures the Factor SQLite store.

type ParamSubmission

type ParamSubmission struct {
	MemberKey string
	Snapshot  domain.ParamSnapshot
}

ParamSubmission 是一个可变成员的一份参数提交(完整快照:默认值加按标的覆盖)。

type ParamSubmitOutcome

type ParamSubmitOutcome struct {
	MemberKey string
	Result    string
	Reason    string
}

ParamSubmitOutcome 是一个成员的提交结果;Result 为 rejected 时 Reason 说明原因。

type RecalcJob

type RecalcJob struct {
	JobID           string   `json:"job_id"`
	RequestID       string   `json:"request_id"`
	TaskID          string   `json:"task_id"`
	Members         []string `json:"members"`
	Subjects        []string `json:"subjects"`
	MembersOmitted  bool     `json:"-"`
	SubjectsOmitted bool     `json:"-"`
	StartTime       int64    `json:"start_time"`
	EndTime         int64    `json:"end_time"`
	Status          string   `json:"status"`
	ProgressTime    int64    `json:"progress_time"`
	Error           string   `json:"error"`
	// DefaultFilled 是按默认参数新补的期次(成员键 → 闭区间 [起始秒, 结束秒] 列表,相邻期次已合并):
	// 原行不存在、或原行的 __params 里没有这个成员,只能用默认参数算出来,与当时的实时结果不保证一致。
	DefaultFilled  map[string][]TimeRange `json:"default_filled,omitempty"`
	EngineID       string                 `json:"engine_id"`
	LeaseToken     string                 `json:"-"`
	LeaseExpiresAt int64                  `json:"-"`
	CreatedAt      time.Time              `json:"created_at"`
	UpdatedAt      time.Time              `json:"updated_at"`
}

RecalcJob is the durable state needed to accept, resume, inspect, or cancel one asynchronous factor-task recalculation.

type RecalcJobFilter

type RecalcJobFilter struct {
	TaskID   string
	Statuses []string
}

RecalcJobFilter narrows ListRecalcJobs; zero fields match everything.

type Store

type Store struct {
	// contains filtered or unexported fields
}

Store owns the Factor SQLite connection.

func Open

func Open(opts *Options) (*Store, error)

Open opens the SQLite store and creates its schema on a fresh database.

func (*Store) AddMember

func (s *Store) AddMember(ctx context.Context, taskID, memberKey, factorID, kind string, params map[string]float64) (domain.FactorTaskMember, error)

AddMember attaches a definition to a factor task as a disabled member under member key. params 是成员的固定参数, 或可变成员的默认参数(完整参数,之后不变)。成员键与类型由调用方按公共命名包推导。

func (*Store) ApplySchema

func (s *Store) ApplySchema(sql string) error

ApplySchema executes the embedded schema on a new database and rejects a database whose Factor tables do not match the current schema. 已有库缺少后来新增的表时由 CREATE TABLE IF NOT EXISTS 补齐;已有的表必须与当前结构一致。

func (*Store) BindTask

func (s *Store) BindTask(ctx context.Context, taskID, instanceID string) (domain.FactorTask, error)

BindTask 把任务与策略实例永久关联:同一个实例重复调用直接成功;任务已关联别的实例时返回 ErrConflict。 一个实例可以关联多个任务(例如现货、永续各一个)。关联写入后不能修改或清空。

func (*Store) ClearParamSubmissions

func (s *Store) ClearParamSubmissions(ctx context.Context) error

ClearParamSubmissions 清掉全部可变成员最近一次被接受的提交(数据重置用);成员本身和配置不变,也不更新 c_mtime。

func (*Store) Close

func (s *Store) Close() error

Close releases the underlying SQL connection.

func (*Store) CountMembers

func (s *Store) CountMembers(ctx context.Context, taskID string) (int, error)

CountMembers returns how many members a set has in any status.

func (*Store) CreateFactor

func (s *Store) CreateFactor(ctx context.Context, def domain.FactorDef) error

CreateFactor creates a standalone definition.

func (*Store) CreateFactors

func (s *Store) CreateFactors(ctx context.Context, defs []domain.FactorDef) error

CreateFactors inserts one catalog import atomically.

func (*Store) CreateRecalcJob

func (s *Store) CreateRecalcJob(ctx context.Context, job RecalcJob) (RecalcJob, error)

CreateRecalcJob inserts a job or returns the existing job for the same request id, making request retries idempotent.

func (*Store) CreateTask

func (s *Store) CreateTask(ctx context.Context, task domain.FactorTask) error

CreateTask 创建一个因子任务。任务 ID、结果数据集或(空间内的)名称重复按 ErrConflict 返回。 新任务没有关联实例:BoundInstanceID 只能由 BindTask 写入。

func (*Store) DeleteBoundTask

func (s *Store) DeleteBoundTask(ctx context.Context, taskID, instanceID string) error

DeleteBoundTask 删除关联到 instanceID 的任务记录。任务不存在返回 gorm.ErrRecordNotFound, 还有成员或关联的不是这个实例时返回 ErrConflict。

func (*Store) DeleteCategory

func (s *Store) DeleteCategory(ctx context.Context, categoryID string) error

DeleteCategory removes a category that no definition uses; the default category cannot be removed.

func (*Store) DeleteFactor

func (s *Store) DeleteFactor(ctx context.Context, factorID string) error

DeleteFactor removes one definition that no set member references.

func (*Store) DeleteTask

func (s *Store) DeleteTask(ctx context.Context, taskID string) error

DeleteTask 删除没有成员、也没有关联实例的因子任务。SQLite 的外键在成员还引用它时拒绝删除,按 ErrConflict 返回。

func (*Store) EnabledUDFEnv

func (s *Store) EnabledUDFEnv(ctx context.Context) (factorexpr.UDFs, error)

EnabledUDFEnv 返回全部已启用版本的解析环境(函数参考与控制台补全用)。

func (*Store) EnsureDefaultCategory

func (s *Store) EnsureDefaultCategory(ctx context.Context) error

EnsureDefaultCategory 补回默认分类“未分类”(建表脚本也会写入;重置清空分类后由这里立刻补回)。

func (*Store) FindRecalcJobByRequestID

func (s *Store) FindRecalcJobByRequestID(ctx context.Context, requestID string) (RecalcJob, bool, error)

FindRecalcJobByRequestID returns the accepted job for an idempotency key.

func (*Store) GetFactor

func (s *Store) GetFactor(ctx context.Context, factorID string) (domain.FactorDef, error)

GetFactor returns one definition by id.

func (*Store) GetMember

func (s *Store) GetMember(ctx context.Context, taskID, memberKey string) (domain.FactorTaskMember, error)

GetMember returns one membership row.

func (*Store) GetRecalcJob

func (s *Store) GetRecalcJob(ctx context.Context, jobID string) (RecalcJob, error)

GetRecalcJob returns one job by id.

func (*Store) GetTask

func (s *Store) GetTask(ctx context.Context, taskID string) (domain.FactorTask, error)

GetTask returns a factor task by id.

func (*Store) GetUdf

func (s *Store) GetUdf(ctx context.Context, name string) (UdfName, []UdfVersion, error)

GetUdf 返回名称及其全部版本(不含 wasm,按创建时间从新到旧)。

func (*Store) GetUdfVersion

func (s *Store) GetUdfVersion(ctx context.Context, hash string, withWasm bool) (UdfVersion, error)

GetUdfVersion 返回一个版本;withWasm 为真时同时读出 wasm 字节。

func (*Store) JobBindings

func (s *Store) JobBindings(ctx context.Context, jobID string) (bindings map[string]string, found bool, err error)

JobBindings 返回补算作业首次领取时固化的绑定(名称 → 版本哈希);还没有固化时 found 为假。

func (*Store) LastEngineHeartbeat

func (s *Store) LastEngineHeartbeat(ctx context.Context) (EngineHeartbeat, bool, error)

LastEngineHeartbeat 返回最近一次心跳;ok 为 false 表示从未收到。

func (*Store) ListCategories

func (s *Store) ListCategories(ctx context.Context) ([]domain.FactorCategory, error)

ListCategories returns every factor category by sort order and id.

func (*Store) ListFactors

func (s *Store) ListFactors(ctx context.Context) ([]domain.FactorDef, error)

ListFactors returns every definition in factor id order.

func (*Store) ListMemberParamStates

func (s *Store) ListMemberParamStates(ctx context.Context, taskID string) ([]domain.FactorTaskMember, error)

ListMemberParamStates 返回任务里各可变成员最近一次被接受的提交(没有提交过的成员也在内,LastSubmittedPeriod 为 0), 按成员键排序。

func (*Store) ListMembers

func (s *Store) ListMembers(ctx context.Context, taskID, status string) ([]domain.TaskMember, error)

ListMembers returns one set's members joined with their definitions, sorted by member key. An empty status returns members in every status.

func (*Store) ListRecalcJobs

func (s *Store) ListRecalcJobs(ctx context.Context, filter RecalcJobFilter) ([]RecalcJob, error)

ListRecalcJobs returns jobs newest first, optionally restricted to one factor task and statuses.

func (*Store) ListTasks

func (s *Store) ListTasks(ctx context.Context) ([]domain.FactorTask, error)

ListTasks returns all factor tasks in stable id order.

func (*Store) ListUdfs

func (s *Store) ListUdfs(ctx context.Context) ([]UdfName, error)

ListUdfs 按名称返回全部自定义算子。

func (*Store) ListUsages

func (s *Store) ListUsages(ctx context.Context, factorIDs ...string) (map[string][]domain.FactorUsage, error)

ListUsages maps each definition to the set members that reference it. With no factor ids it returns the usages of every definition.

func (*Store) Ping

func (s *Store) Ping(ctx context.Context) error

Ping verifies that the database is available.

func (*Store) PullRecalcJob

func (s *Store) PullRecalcJob(ctx context.Context, engineID string, now time.Time, ttl time.Duration, skipTasks map[string]bool, maxRunning int) (RecalcJob, bool, error)

PullRecalcJob atomically leases the least recently touched pullable job to engineID (按 c_mtime 排序:归还会更新 c_mtime,被门禁挡住而归还的作业排到后面,不会每轮都抢在别的作业前)。 Jobs of tasks in skipTasks are left for a later pull. The returned job carries the new lease token; found is false when no job is pullable. maxRunning 是同时运行(租约未过期)的补算作业上限,达到上限时不再发出新任务;0 表示不限制。 补算和回填会整窗口读取源数据,并发多了会打满 Storage(2026-10-14 的 1h 因子任务回填),上限与领取在同一个事务里判断。

func (*Store) PutUdfVersion

func (s *Store) PutUdfVersion(ctx context.Context, v UdfVersion) (created bool, err error)

PutUdfVersion 保存一个版本(未启用):名称不存在时一并创建;同一哈希重复上传只更新说明与示例(c_manifest), 其余字段(wasm、来源、报告)不变。created 报告是否新增了版本。

func (*Store) RemoveMember

func (s *Store) RemoveMember(ctx context.Context, taskID, memberKey string) error

RemoveMember detaches a disabled member. Enabled members must be disabled first.

func (*Store) ReportRecalcProgress

func (s *Store) ReportRecalcProgress(ctx context.Context, jobID, leaseToken string, progress int64, status, errText string, defaultFilled map[string][]TimeRange, now time.Time, ttl time.Duration) (RecalcJob, error)

ReportRecalcProgress records a chunk reported by the engine holding the lease and renews the lease. A cancelled or finished job is returned as is so the engine can stop; a stale token or a progress regression is ErrConflict. defaultFilled 是引擎到目前为止累计的「按默认参数新补的期次」;nil 表示这次没有变化,非 nil 时整体替换。 status 为 accepted 表示引擎归还作业(被门禁挡住、引擎停止):作业回到待领取,进度保持管理端已记下的值, 下一次领取立刻能拿到,不用等租约过期;旧的租约令牌随之失效。

func (*Store) ResolveUDFs

func (s *Store) ResolveUDFs(ctx context.Context, names []string, overrides map[string]string) (map[string]factorexpr.UDF, error)

ResolveUDFs 按名称解析自定义算子(固定绑定优先,否则取当前启用版本)。没有启用版本的名称带空哈希返回, 编译时据此报“没有启用的版本”;不存在的名称不在结果里。overrides 是请求级覆盖(名称 → 版本哈希),优先于固定绑定与启用版本。

func (*Store) SQLDB

func (s *Store) SQLDB() (*sql.DB, error)

SQLDB 返回底层数据库连接(重置时备份与清表用,由生命周期的重置实现调用)。

func (*Store) SaveEngineHeartbeat

func (s *Store) SaveEngineHeartbeat(ctx context.Context, heartbeat EngineHeartbeat) error

SaveEngineHeartbeat 覆盖保存最近一次心跳(单行表)。

func (*Store) SaveJobBindings

func (s *Store) SaveJobBindings(ctx context.Context, jobID string, bindings map[string]string) (map[string]string, error)

SaveJobBindings 固化补算作业的绑定;已经固化过的不覆盖(重新领取、续跑都用第一次的),返回生效的绑定。

func (*Store) SetActiveUdf

func (s *Store) SetActiveUdf(ctx context.Context, name, hash string) error

SetActiveUdf 切换名称启用的版本(hash 为空表示清空):版本必须属于这个名称;已被因子定义引用的名称不能清空。 在事务里校验并修改,调用方持有全局 UDF 锁。

func (*Store) SetFactorCategory

func (s *Store) SetFactorCategory(ctx context.Context, factorID, categoryID string) error

SetFactorCategory changes only a definition's category; it is metadata and allowed while members are enabled.

func (*Store) SetMemberStatus

func (s *Store) SetMemberStatus(ctx context.Context, taskID, memberKey, expectedStatus, status string) error

SetMemberStatus changes a member's status only if it still matches expectedStatus.

func (*Store) SetTaskStatus

func (s *Store) SetTaskStatus(ctx context.Context, taskID, expectedStatus, status string) error

SetTaskStatus changes a set's lifecycle status only if it still matches expectedStatus. The single conditional update makes competing lifecycle transitions explicit instead of silently overwriting one another.

func (*Store) SubmitMemberParams

func (s *Store) SubmitMemberParams(ctx context.Context, taskID string, period int64, requestID string, submissions []ParamSubmission,
	check func(memberKey string, merged domain.ParamSnapshot) error) ([]ParamSubmitOutcome, error)

SubmitMemberParams 在一个事务里处理一次提交(每个成员一份)。对每个成员:

  1. 目标期不能倒退:小于该成员最近一次被接受的目标期则拒绝;
  2. 同一目标期下,同一个 requestID 重复提交按幂等成功,不同 requestID 拒绝;
  3. 按标的合并(domain.MergeSnapshots)后,交给 check 校验合并结果;通过就写入 c_last_submitted_*。

提交只写运行状态列,不更新 c_mtime,也就不改变目录指纹。成员不存在、或不是可变成员时该成员被拒绝,不影响其他成员。

func (*Store) UdfReferences

func (s *Store) UdfReferences(ctx context.Context, name string) ([]string, error)

UdfReferences 返回引用某个自定义算子名称的因子定义 ID(按定义文本扫描,只会多、不会漏)。

func (*Store) UpdateFactor

func (s *Store) UpdateFactor(ctx context.Context, def domain.FactorDef) error

UpdateFactor rewrites a definition while no enabled member references it. The NOT EXISTS guard runs in the same statement as the update, so a member enabled concurrently cannot slip past the check.

func (*Store) UpdateRecalcJob

func (s *Store) UpdateRecalcJob(ctx context.Context, jobID string, mutate func(*RecalcJob) error) (RecalcJob, error)

UpdateRecalcJob atomically applies a mutation, retaining monotonic progress and preventing cancelled jobs from being restarted.

func (*Store) UpsertCategory

func (s *Store) UpsertCategory(ctx context.Context, category domain.FactorCategory) (domain.FactorCategory, error)

UpsertCategory creates a category or renames and reorders an existing one.

func (*Store) WithTx

func (s *Store) WithTx(ctx context.Context, fn func(*gorm.DB) error) error

WithTx runs fn inside a Factor SQLite transaction.

type TimeRange

type TimeRange = domain.TimeRange

TimeRange 是 Unix 秒的闭区间。

type UdfName

type UdfName struct {
	Name       string    `json:"name"`
	ActiveHash string    `json:"active_hash"`
	CreatedAt  time.Time `json:"created_at"`
	UpdatedAt  time.Time `json:"updated_at"`
}

UdfName 是一个自定义算子名称:ActiveHash 是当前启用的版本,空串表示还没有启用。

type UdfVersion

type UdfVersion struct {
	Hash       string    `json:"hash"`
	Name       string    `json:"name"`
	Manifest   string    `json:"manifest"` // 包内 manifest.json(语义字段加说明、示例、来源)
	SourceRef  string    `json:"source_ref"`
	SDKVersion string    `json:"sdk_version"`
	Toolchain  string    `json:"toolchain"`
	Report     string    `json:"report"`
	CreatedAt  time.Time `json:"created_at"`
	Wasm       []byte    `json:"-"`
}

UdfVersion 是自定义算子的一个不可变版本(身份是 wasm 字节的 sha256)。Wasm 只在明确要求时才读出。

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL