Documentation
¶
Overview ¶
Concrete SpaceReader against the OpenCloud/Reva CS3 gateway (phase-2 deliverable 2). The read path — auth type, token header, ListStorageSpaces, download flow, and the space-membership source — was validated in Spike 3 (phase-0-findings.md); this file promotes those findings into a testable client.
The gateway dependency is injected behind GatewayClient so unit tests can substitute a fake without a live gateway or gRPC.
Package cs3 wraps the OpenCloud CS3 gateway so the rest of the service reads Spaces and file streams through a single, testable interface.
The unattended worker authenticates with an OpenCloud service account (CS3 auth type "serviceaccounts"; see decisions.md #11 and phase-0-findings.md Spike 3). The concrete client is implemented in Phase 2; this file defines the boundary interface only.
Space membership as an authority level, not a yes/no (remediation R3).
A CS3 space carries its grants in the Opaque map. The shape is pinned against OpenCloud 7.3.0 by TestIntegration_SpaceGrantsShape:
opaque["grants"] = {"<principal>": <provider.ResourcePermissions>, ...}
opaque["groups"] = {"<principal>": {}, ...} // which principals are groups
opaque["grants_expirations"] = {"<principal>": {"seconds": <unix>}, ...}
The grant value is a *permission set*, not a role name — reva does not serialize the display role. Roles are therefore derived from capabilities: whoever may change grants is a manager, whoever may write is an editor, whoever may read is a viewer. Deriving from capabilities rather than matching an exact permission set keeps the mapping stable when reva adds a permission.
Index ¶
- Constants
- Variables
- func CreateProjectSpace(ctx context.Context, gw SpaceProvisioner, auth Authenticator, name string) (string, error)
- func ParseDataServerOrigin(raw string) (*url.URL, error)
- type Authenticator
- type CachedAuth
- type Client
- func (c *Client) Delete(ctx context.Context, space Space, relPath string) error
- func (c *Client) ListDir(ctx context.Context, space Space, relDir string) ([]Entry, error)
- func (c *Client) ListSpaces(ctx context.Context) ([]Space, error)
- func (c *Client) MakeDir(ctx context.Context, space Space, relDir string) error
- func (c *Client) OpenFile(ctx context.Context, space Space, relPath string, offset int64) (io.ReadCloser, error)
- func (c *Client) Upload(ctx context.Context, space Space, relPath string, size int64, ...) error
- type ClientOption
- type Entry
- type Env
- type GatewayClient
- type Member
- type ResourceID
- type Role
- type ServiceAccountAuth
- type Space
- type SpaceProvisioner
- type SpaceReader
- type SpaceWriter
- type StaticTokenAuth
Constants ¶
const ( // TokenHeader is reva's access-token gRPC metadata key (pkg/ctx.TokenHeader). // The same header name is used on the data-gateway HTTP request. TokenHeader = "x-access-token" // TransferHeader carries the per-download transfer token returned by // InitiateFileDownload (phase-0-findings.md Spike 3, step 7). TransferHeader = "X-Reva-Transfer" )
const DefaultNotReadyWait = 2 * time.Minute
DefaultNotReadyWait is how long a read refused as "too early" is retried before it fails. It matches DefaultStallTimeout: both are how long OpenCloud may go without delivering a file before the service stops waiting for it.
const DefaultStallTimeout = 2 * time.Minute
DefaultStallTimeout is how long a transfer may make no progress at all before it is treated as failed. Generous: a busy storage backend may take a while to answer, and a false positive costs a whole run.
Variables ¶
var ErrAlreadyExists = errors.New("cs3: resource already exists")
ErrAlreadyExists is returned when a restore target path is already taken. A restore must never overwrite live data, so callers surface this rather than clobbering (decisions.md #3).
var ErrNotFound = errors.New("cs3: resource not found")
ErrNotFound is returned when a space-relative path does not exist.
var ErrNotReady = errors.New("cs3: file is still being processed by OpenCloud")
ErrNotReady is returned when OpenCloud still refused to hand out a file because it has not finished processing it, after the retry window ran out.
var ErrTransferStalled = errors.New("cs3: transfer stalled")
ErrTransferStalled is returned when a transfer made no progress for longer than the stall timeout.
Functions ¶
func CreateProjectSpace ¶
func CreateProjectSpace(ctx context.Context, gw SpaceProvisioner, auth Authenticator, name string) (string, error)
CreateProjectSpace creates a project Space owned by the authenticated principal and returns its id.
The caller must be the service account: whoever creates the Space ends up holding the grant that cannot be removed, so creating it as anyone else produces a Space the service will refuse to use.
Types ¶
type Authenticator ¶
type Authenticator interface {
// Token returns a reva access token to place in the x-access-token header.
Token(ctx context.Context) (string, error)
}
Authenticator obtains a reva access token for the worker credential. The default implementation authenticates a service account (decisions.md #11); a per-user forwarding implementation carries the caller's own bearer token.
type CachedAuth ¶
type CachedAuth struct {
// contains filtered or unexported fields
}
CachedAuth reuses an Authenticator's token until it is about to expire.
func NewCachedAuth ¶
func NewCachedAuth(inner Authenticator) *CachedAuth
NewCachedAuth wraps an authenticator with token reuse.
func (*CachedAuth) Invalidate ¶
func (a *CachedAuth) Invalidate()
Invalidate drops the cached token. It is called when reva rejects one: a token the server will not accept must not be presented again until it happens to expire.
type Client ¶
type Client struct {
// contains filtered or unexported fields
}
Client is the concrete SpaceReader. Construct with NewClient.
func NewClient ¶
func NewClient(gw GatewayClient, auth Authenticator, opts ...ClientOption) *Client
NewClient builds a Client from an injected gateway and authenticator.
func (*Client) ListDir ¶
ListDir returns the direct children of one space-relative directory via ListContainer. relDir is space-relative with no leading "./"; the empty string denotes the space root.
func (*Client) ListSpaces ¶
ListSpaces returns the storage spaces the worker credential may back up (phase-2 deliverable 2). Membership/roles are read from the space's Opaque grants map (decisions.md #7; phase-0-findings.md Spike 3) so later phases can authorize shared-space access. Pagination, if any, is handled here.
func (*Client) OpenFile ¶
func (c *Client) OpenFile(ctx context.Context, space Space, relPath string, offset int64) (io.ReadCloser, error)
OpenFile streams one space-relative file starting at offset. The CS3 reference is built from the space root plus relative path — a bare file id fails (phase-0-findings.md Spike 3, "Critical gotcha").
InitiateFileDownload yields a data-gateway URL plus a transfer token; the bytes are then fetched over HTTP carrying both the worker access token and the transfer token (Spike 3, steps 6–7). A non-zero offset is requested with a Range header and, if the gateway ignores it, satisfied by discarding the leading bytes so callers always observe the requested position.
A file OpenCloud has not finished processing is retried for a bounded period before ErrNotReady is returned (notready.go).
func (*Client) Upload ¶
func (c *Client) Upload(ctx context.Context, space Space, relPath string, size int64, modTime time.Time, r io.Reader) error
Upload writes one file into a Space.
The size must be known up front: reva announces the content length to the storage driver before any byte moves, and a restore always knows it from the snapshot metadata.
type ClientOption ¶
type ClientOption func(*Client)
ClientOption configures a Client.
func WithDataServerOrigin ¶
func WithDataServerOrigin(origin *url.URL) ClientOption
WithDataServerOrigin sends every file transfer the gateway routes to its data server directly (no transfer token) to origin (scheme and host) instead, keeping the gateway's path. Nil leaves the gateway's URL untouched.
func WithHTTPClient ¶
func WithHTTPClient(h *http.Client) ClientOption
WithHTTPClient sets the HTTP client used to stream file bytes from the reva data gateway. Injected so deployments can supply their own TLS/proxy config and tests can point at an httptest server.
func WithNotReadyWait ¶
func WithNotReadyWait(d time.Duration) ClientOption
WithNotReadyWait sets how long a read of a file OpenCloud has not finished processing is retried. Zero or less uses DefaultNotReadyWait.
func WithStallTimeout ¶
func WithStallTimeout(d time.Duration) ClientOption
WithStallTimeout sets how long a file transfer may make no progress before it is failed. Zero or less uses DefaultStallTimeout.
type Entry ¶
type Entry struct {
// Path is the space-relative, slash-separated path with no leading "./" or
// "/", e.g. "photos/img.jpg". The empty string denotes the space root.
Path string
// IsDir reports whether the entry is a directory.
IsDir bool
// Size is the file size in bytes (0 for directories).
Size int64
// MTimeUnix is the modification time (seconds); mtime is in scope
// (decisions.md #4).
MTimeUnix int64
}
Entry is a single file or directory encountered while walking a Space.
type Env ¶
type Env struct {
GatewayAddr string `` /* 233-byte string literal not displayed */
DataServerURL string `` /* 170-byte string literal not displayed */
ServiceAccountID config.Secret `env:"OC_SERVICE_ACCOUNT_ID" doc:"Id of the OpenCloud service account the service acts as."`
ServiceAccountSecret config.Secret `` /* 153-byte string literal not displayed */
}
Env is how this service reaches OpenCloud's CS3 gateway, as the environment declares it (internal/config).
func (Env) DataServerOrigin ¶
DataServerOrigin is CS3_DATA_SERVER_URL parsed, or nil when it is unset.
type GatewayClient ¶
type GatewayClient interface {
Authenticate(ctx context.Context, in *gateway.AuthenticateRequest, opts ...grpc.CallOption) (*gateway.AuthenticateResponse, error)
ListStorageSpaces(ctx context.Context, in *provider.ListStorageSpacesRequest, opts ...grpc.CallOption) (*provider.ListStorageSpacesResponse, error)
ListContainer(ctx context.Context, in *provider.ListContainerRequest, opts ...grpc.CallOption) (*provider.ListContainerResponse, error)
Stat(ctx context.Context, in *provider.StatRequest, opts ...grpc.CallOption) (*provider.StatResponse, error)
InitiateFileDownload(ctx context.Context, in *provider.InitiateFileDownloadRequest, opts ...grpc.CallOption) (*gateway.InitiateFileDownloadResponse, error)
// The write path (restore Path B).
CreateContainer(ctx context.Context, in *provider.CreateContainerRequest, opts ...grpc.CallOption) (*provider.CreateContainerResponse, error)
InitiateFileUpload(ctx context.Context, in *provider.InitiateFileUploadRequest, opts ...grpc.CallOption) (*gateway.InitiateFileUploadResponse, error)
// Delete backs the service's own state Space only; no restore path uses it.
Delete(ctx context.Context, in *provider.DeleteRequest, opts ...grpc.CallOption) (*provider.DeleteResponse, error)
}
GatewayClient is the subset of the CS3 gateway API the reader uses. The generated gateway.GatewayAPIClient satisfies it; tests provide a fake.
type Member ¶
type Member struct {
// Role is the authority derived from the grant's permission set.
Role Role
// ExpiresAt is when the grant lapses; the zero time means "never".
ExpiresAt time.Time
// Group reports whether the principal is a group rather than a user.
Group bool
}
Member is one principal's grant on a Space.
type ResourceID ¶
ResourceID mirrors the CS3 provider.ResourceId triple. The space root's resource id is required verbatim to build the space-relative references the data path needs; OpenCloud space ids are composite, so reconstructing the triple from the space id alone is not reliable (phase-0-findings.md Spike 3).
func (ResourceID) Zero ¶
func (r ResourceID) Zero() bool
Zero reports whether the resource id carries no addressing information.
type Role ¶
type Role int
Role is a principal's authority on a Space, ordered so that a numerically greater role subsumes a lesser one. Handlers compare with >=.
const ( // RoleNone means no grant at all: the principal is not a member. RoleNone Role = iota // RoleViewer may read the Space. RoleViewer // RoleEditor may write into the Space. RoleEditor // RoleManager may change the Space's grants. RoleManager // RoleOwner is the Space's owner. Only personal Spaces have a user owner; // a project Space is owned by itself, so this never matches a caller there. RoleOwner )
type ServiceAccountAuth ¶
type ServiceAccountAuth struct {
Gateway GatewayClient
ClientID string
Secret string
}
ServiceAccountAuth authenticates the unattended worker with an OpenCloud service account (CS3 auth type "serviceaccounts"; decisions.md #11). It mints a token per call and holds none, which keeps it stateless and trivially testable; wrap it in CachedAuth (as the service does) so an operation that touches a thousand files does not mint a thousand tokens.
type Space ¶
type Space struct {
// ID is the CS3 space (storage) ID.
ID string
// Name is the human-readable space name.
Name string
// Type is the space type ("personal", "project", ...).
Type string
// Owner is the owning user's opaque ID (empty for some project spaces).
Owner string
// Root is the space root's resource id, used to build space-relative
// references for the walk and download paths.
Root ResourceID
// Members maps principal ID -> grant for shared spaces, derived from the
// space's Opaque grants/groups/grants_expirations maps (see role.go).
// A principal may be a user or a group. Empty for a personal space.
Members map[string]Member
}
Space identifies an OpenCloud storage space, the unit of backup, keying, and scheduling (decisions.md #6).
func (Space) GroupGrants ¶
GroupGrants reports whether the Space has any group grant. Callers use it to avoid resolving the caller's groups — an upstream call — for the common case of a Space granted to users only.
func (Space) RoleFor ¶
RoleFor returns the caller's effective role on the Space: the highest of the Space ownership, the caller's own non-expired grant, and any non-expired grant held by a group the caller belongs to. RoleNone means "not a member" and must be answered exactly like "no such Space" so membership is not enumerable.
groups are the caller's group ids; pass nil when they are unknown or when the Space has no group grants (see GroupGrants).
type SpaceProvisioner ¶
type SpaceProvisioner interface {
CreateStorageSpace(ctx context.Context, in *provider.CreateStorageSpaceRequest, opts ...grpc.CallOption) (*provider.CreateStorageSpaceResponse, error)
}
SpaceProvisioner is the single gateway call provisioning needs.
Deliberately not folded into GatewayClient: that interface is the service's run path, implemented by fakes throughout the test suite, and none of them will ever create a Space. R9's lesson was that an interface method nothing calls costs every fake that has to implement it.
type SpaceReader ¶
type SpaceReader interface {
// ListSpaces returns the spaces the worker credential may back up.
ListSpaces(ctx context.Context) ([]Space, error)
// ListDir returns the direct children of one space-relative directory.
// The empty path denotes the space root. Returned entries carry paths
// relative to the space root.
ListDir(ctx context.Context, space Space, relDir string) ([]Entry, error)
// OpenFile streams the bytes of one space-relative file starting at offset.
// The caller closes the returned reader.
OpenFile(ctx context.Context, space Space, relPath string, offset int64) (io.ReadCloser, error)
}
SpaceReader is the read-only boundary onto OpenCloud. Implementations are expected to be safe for use by a single worker run.
InitiateFileDownload/OpenFile must be given a space-relative reference built from the space root plus relative path; a bare file id fails (phase-0-findings Spike 3, "Critical gotcha").
type SpaceWriter ¶
type SpaceWriter interface {
// MakeDir creates one space-relative directory. It is idempotent: an
// existing directory is not an error.
MakeDir(ctx context.Context, space Space, relDir string) error
// Upload writes size bytes from r to a space-relative path, creating the
// file. modTime is preserved where the server supports it.
Upload(ctx context.Context, space Space, relPath string, size int64, modTime time.Time, r io.Reader) error
// Delete removes one space-relative path, returning ErrNotFound when it
// does not exist.
//
// A restore never calls this — restores only ever create (decisions.md #3).
// It exists for the service's own state, which lives in its own Space and
// has to be able to drop a released lock or an expired job record.
Delete(ctx context.Context, space Space, relPath string) error
}
SpaceWriter is the write boundary onto OpenCloud, used by restores.
type StaticTokenAuth ¶
type StaticTokenAuth struct{ Value string }
StaticTokenAuth forwards a pre-obtained token (e.g. the caller's own bearer token for user-scoped reads). It performs no RPC.