storage

package
v0.48.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 18, 2026 License: Apache-2.0 Imports: 35 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

View Source 
var ErrEventRouteNotTransactional = errors.New("storage: event must route to a transactional transport")

ErrEventRouteNotTransactional indicates the framework's event bus is not configured to deliver a storage domain event through a transactional transport. Storage publishes with event.WithTx; without such a route the first publish would fail at runtime, so the module fails fast at start-up instead. The wrapped formatted error names the offending event type and points operators at the configuration that must be set.

View Source 
var ErrUnsupportedStorageProvider = errors.New("unsupported storage provider")

ErrUnsupportedStorageProvider is returned by NewService when the configured provider does not match any of the known backends.

View Source 
var Module = fx.Module(
	"vef:storage",
	fx.Provide(
		fx.Annotate(
			NewService,
			fx.OnStart(func(ctx context.Context, service storage.Service) error {
				if initializer, ok := service.(contract.Initializer); ok {
					if err := initializer.Init(ctx); err != nil {
						return fmt.Errorf("failed to initialize storage service: %w", err)
					}
				}

				return nil
			}),
		),
		fx.Annotate(
			NewResource,
			fx.ResultTags(`group:"vef:api:resources"`),
		),
		fx.Annotate(
			NewFileResource,
			fx.ResultTags(`group:"vef:api:resources"`),
		),
		fx.Annotate(
			NewProxyMiddleware,
			fx.ResultTags(`group:"vef:app:middlewares"`),
		),

		newDefaultFileACL,

		newDefaultURLKeyMapper,
	),

	migration.Module,
	store.Module,
	worker.Module,

	fx.Invoke(verifyEventRouting),
)

Functions

func NewFileResource added in v0.45.0 

func NewFileResource(registry storage.FileRegistry, acl storage.FileACL) api.Resource

func NewProxyMiddleware 

func NewProxyMiddleware(params ProxyMiddlewareParams) app.Middleware

func NewResource 

func NewResource(
	db orm.DB,
	service storage.Service,
	claimStore store.ClaimStore,
	partStore store.UploadPartStore,
	deleteQueue store.DeleteQueue,
	cfg *config.StorageConfig,
) api.Resource

func NewService 

func NewService(cfg *config.StorageConfig, appCfg *config.AppConfig) (storage.Service, error)

Types

type AbortUploadParams added in v0.23.0 

type AbortUploadParams struct {
	api.P

	ClaimID string `json:"claimId" validate:"required"`
}

type CompleteUploadParams added in v0.23.0 

type CompleteUploadParams struct {
	api.P

	ClaimID string `json:"claimId" validate:"required"`
}

type CompleteUploadResult added in v0.23.0 

type CompleteUploadResult struct {
	storage.ObjectInfo

	OriginalFilename string `json:"originalFilename"`
}

CompleteUploadResult bundles the backend ObjectInfo with the framework-tracked OriginalFilename. Returning a wrapper rather than the bare ObjectInfo keeps backend abstractions clean of framework concepts while still giving callers a single response shape that covers everything they need to render the upload.

type FileResource added in v0.45.0 

type FileResource struct {
	api.Resource
	// contains filtered or unexported fields
}

FileResource is the read side of the durable upload registry: it turns the object keys a business model stores into the metadata needed to render them — original filename above all.

Kept apart from the upload protocol resource on purpose: this is a query surface over recorded state, not a step in the chunked-upload state machine.

func (*FileResource) Resolve added in v0.45.0 

func (r *FileResource) Resolve(ctx fiber.Ctx, principal *security.Principal, params ResolveParams) error

Resolve returns what the framework recorded about each supplied object key. Keys the caller may not read, and keys with no record, are omitted from the response rather than reported — the caller is rendering a list and must degrade to showing the bare key, and distinguishing "not yours" from "never existed" would leak the existence of other tenants' files.

Authorization mirrors the download proxy exactly: pub/ keys are open, everything else goes through FileACL.CanRead. That is the same barrier the bytes themselves sit behind, so this endpoint cannot become the weaker door — an application that has not supplied a FileACL cannot serve its private files at all, and equally cannot name them here.

type InitUploadParams added in v0.23.0 

type InitUploadParams struct {
	api.P

	Filename    string `json:"filename"    validate:"required,max=255"`
	Size        int64  `json:"size"        validate:"required,min=1"`
	ContentType string `json:"contentType" validate:"max=127"`
	Public      bool   `json:"public"`
}

InitUploadParams declares an upload intent. Every upload goes through the chunked protocol — small files simply end up with PartCount=1. Size is required so the framework can compute the part plan and validate against the configured upload cap before opening any backend session. ContentType is persisted onto the final object; Public controls the key prefix.

type InitUploadResult added in v0.23.0 

type InitUploadResult struct {
	Key              string    `json:"key"`
	ClaimID          string    `json:"claimId"`
	OriginalFilename string    `json:"originalFilename"`
	PartSize         int64     `json:"partSize"`
	PartCount        int       `json:"partCount"`
	ExpiresAt        time.Time `json:"expiresAt"`
}

InitUploadResult tells the client how to deliver the parts. The client uploads each part via the upload_part action (multipart/form-data proxied through the framework) and finalizes with complete_upload. OriginalFilename is the client-supplied filename echoed back; the framework persists it on the claim row, not in backend user-metadata, so callers can rely on it independent of the storage backend.

The backend's multipart UploadID is intentionally NOT exposed to the client: every client-facing action (upload_part / list_parts / complete_upload / abort_upload) routes by ClaimID only. The framework loads the UploadID from the claim row internally.

type ListPartsParams added in v0.24.0 

type ListPartsParams struct {
	api.P

	ClaimID string `json:"claimId" validate:"required"`
}

ListPartsParams identifies which in-flight session to inspect.

type ListPartsResult added in v0.24.0 

type ListPartsResult struct {
	Parts []ListedPart `json:"parts"`
}

ListPartsResult enumerates the parts the backend has already accepted for an active claim. The list is ordered by part number ascending.

type ListedPart added in v0.24.0 

type ListedPart struct {
	PartNumber int   `json:"partNumber"`
	Size       int64 `json:"size"`
}

ListedPart describes a single successfully uploaded part. ETag is intentionally omitted: clients do not reconstruct the parts list — complete_upload assembles it from the database on the server side.

type ProxyMiddleware 

type ProxyMiddleware struct {
	// contains filtered or unexported fields
}

func (*ProxyMiddleware) Apply 

func (p *ProxyMiddleware) Apply(router fiber.Router)

func (*ProxyMiddleware) Name 

func (*ProxyMiddleware) Name() string

func (*ProxyMiddleware) Order 

func (*ProxyMiddleware) Order() int

type ProxyMiddlewareParams added in v0.47.2 

type ProxyMiddlewareParams struct {
	fx.In

	Service  storage.Service
	ACL      storage.FileACL
	Registry storage.FileRegistry
	Auth     security.AuthManager
	Security *config.SecurityConfig
}

ProxyMiddlewareParams contains the dependencies of the download proxy.

type ResolveParams added in v0.45.0 

type ResolveParams struct {
	api.P

	// A file list long enough to exceed the cap is a paging problem on
	// the client, and an unbounded IN (...) is not something an
	// authenticated caller should be able to ask for.
	Keys []string `json:"keys" validate:"required,min=1,max=200,dive,required"`
}

type ResolveResult added in v0.45.0 

type ResolveResult struct {
	Files []ResolvedFile `json:"files"`
}

type ResolvedFile added in v0.45.0 

type ResolvedFile struct {
	Key              string             `json:"key"`
	OriginalFilename string             `json:"originalFilename"`
	ContentType      string             `json:"contentType"`
	Size             int64              `json:"size"`
	Status           storage.FileStatus `json:"status"`
	UploadedAt       timex.DateTime     `json:"uploadedAt"`
	UploadedBy       string             `json:"uploadedBy"`
}

ResolvedFile is the client-facing projection of a registry record. Deliberately narrower than storage.FileRecord: lifecycle bookkeeping (claimed/deleted timestamps, delete reason) is operational data, not something a form field needs to render a filename.

type Resource 

type Resource struct {
	api.Resource
	// contains filtered or unexported fields
}

func (*Resource) AbortUpload added in v0.23.0 

func (r *Resource) AbortUpload(ctx fiber.Ctx, principal *security.Principal, params AbortUploadParams) error

AbortUpload cancels an in-flight upload. The claim row is the arbitration token: the handler deletes it under a status='pending' predicate, and only the transaction that wins that compare-and-set schedules the backend cleanup. A client canceling while its own complete_upload retry is in flight therefore either aborts the session or loses cleanly to the completion — it can never reap an object the completion has already finalized and recorded.

Backend cleanup (multipart abort + object delete) is delegated to the durable delete queue rather than performed inline. That is what makes the abort crash-safe: the queue row commits with the claim delete, so a process death between the two can no longer strand object bytes that nothing remembers. It also means abort_upload cannot fail on a momentarily unreachable backend — the delete worker owns the retry, backoff, and dead-lettering for that.

func (*Resource) CompleteUpload added in v0.23.0 

func (r *Resource) CompleteUpload(ctx fiber.Ctx, principal *security.Principal, params CompleteUploadParams) error

CompleteUpload finalizes a chunked upload. The handler reads the recorded parts from the database (clients never replay ETags themselves), verifies the part count matches the original plan, instructs the backend to assemble the object, then atomically marks the claim 'uploaded' and clears its part rows.

Idempotency: a retried complete_upload that arrives after the backend session is already closed surfaces ErrUploadSessionNotFound; the handler then re-stats the object to confirm it exists and commits the same MarkUploaded + DeleteByClaim transaction so the claim still ends in a consumable state.

func (*Resource) InitUpload added in v0.23.0 

func (r *Resource) InitUpload(ctx fiber.Ctx, principal *security.Principal, params InitUploadParams) error

InitUpload opens an upload session. Every upload — including small files that end up with a single part — flows through the same init → upload_part → complete protocol; this keeps the client and server logic uniform regardless of file size.

The flow:

  1. validate the declared size against the configured cap;
  2. compute the part plan from the backend's authoritative PartSize;
  3. INSERT the claim (status='pending') first so a backend failure leaves no orphan multipart session;
  4. open the backend multipart session and bind its UploadID to the claim row (best-effort cleanup on failure).

If the backend does not implement Multipart, init_upload is rejected outright — there is no fallback path in the unified protocol.

func (*Resource) ListParts added in v0.24.0 

func (r *Resource) ListParts(ctx fiber.Ctx, principal *security.Principal, params ListPartsParams) error

ListParts reports which parts of an in-flight chunked upload have already been accepted by the backend. Clients use the result to drive resumable uploads — skipping parts the server has confirmed and uploading only the remainder. The handler validates ownership and the claim's active state (pending + not expired) so the response is safe to act on without an additional round trip.

The list is sourced from the database, not the backend's native ListParts (S3 supports it, filesystem/memory do not). The database is the authoritative source: it carries the ETag complete_upload uses to assemble the object, so any part recorded here will be honored by complete_upload as-is.

func (*Resource) UploadPart added in v0.23.0 

func (r *Resource) UploadPart(ctx fiber.Ctx, principal *security.Principal, params UploadPartParams) error

UploadPart proxies a single multipart part through the framework to the backend. The handler validates ownership, the claim's pending status, and the part-number range before opening the backend stream; a successful PutPart is then mirrored to the upload_part table so complete_upload can drive the assemble step from the database (clients never round-trip ETags themselves).

type UploadPartParams added in v0.23.0 

type UploadPartParams struct {
	api.P

	File *multipart.FileHeader

	ClaimID    string `json:"claimId"    validate:"required"`
	PartNumber int    `json:"partNumber" validate:"required,min=1"`
}

UploadPartParams accepts multipart/form-data carrying a single part of an in-progress chunked upload. ClaimID and PartNumber identify which slot of which session the bytes belong to; the file payload is streamed through to the backend's PutPart.

type UploadPartResult added in v0.23.0 

type UploadPartResult struct {
	PartNumber int   `json:"partNumber"`
	Size       int64 `json:"size"`
}

UploadPartResult echoes the part position and recorded byte count. The backend ETag is intentionally NOT returned to the client: it is persisted server-side on the upload_part row so complete_upload can reconstruct the parts list without trusting client state.

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.