Documentation
¶
Overview ¶
Package staging implements the stage_locally path delivery: copying job inputs to worker-local scratch before a task runs and outputs back afterward. sqi never copies bytes itself; it invokes an operator-configured sync command per path. The scratch destination paths are returned as path-map rules so the other deliveries advertise them.
Index ¶
- type Stager
- func (s *Stager) Cleanup(scratchDir string)
- func (s *Stager) Configured() bool
- func (s *Stager) StageIn(ctx context.Context, jobID, attemptID string, entries []protocol.StageEntry) ([]protocol.PathMapRule, string, error)
- func (s *Stager) StageOut(ctx context.Context, scratchDir string, entries []protocol.StageEntry) error
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Stager ¶
type Stager struct {
// contains filtered or unexported fields
}
Stager copies staged paths via an external sync command, or the built-in copy when unconfigured/defaults are enabled.
func New ¶
New returns a Stager. scratchBase/syncCommand come from worker config; defaults enables the built-in copy + TEMP scratch when staging is otherwise unconfigured.
func (*Stager) Configured ¶
Configured reports whether staging can proceed: explicitly configured (scratch + shell command), or the built-in copy is available (defaults on, or the `builtin` sentinel was set explicitly).
func (*Stager) StageIn ¶
func (s *Stager) StageIn(ctx context.Context, jobID, attemptID string, entries []protocol.StageEntry) ([]protocol.PathMapRule, string, error)
StageIn prepares a per-attempt scratch directory for every staged entry and returns one path-map rule per entry (original path -> scratch path) plus the scratch directory. IN/INOUT inputs are copied into scratch via the sync command; OUT entries only get their scratch destination created (the task writes the output there, and Stager.StageOut copies it back afterward). Returning a rule for OUT entries too is what lets the other deliveries redirect the task's OUTPUT paths into scratch — without it the task writes to the real path and copy-out fails on a missing scratch file. On any failure the partial scratch directory is removed.
func (*Stager) StageOut ¶
func (s *Stager) StageOut(ctx context.Context, scratchDir string, entries []protocol.StageEntry) error
StageOut copies every OUT/INOUT entry from scratch back to its original path. It iterates the full entries slice with its original index so the per-index subdirectory (<scratchDir>/<i>/<basename>) matches what copyInEntries created.
Source Files