cliclient

package
v0.44.2 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Sep 25, 2026 License: MIT Imports: 27 Imported by: 0

Documentation

Overview

Package cliclient implements the HTTP client behind `filex client` — the subcommand family that talks to a REMOTE filex server over its public REST API. It deliberately reuses only the wire contracts (no server-side imports) so the same binary can act as a pure API consumer from any machine.

upload.go — recursive (tree) upload on top of the single-file Upload core in api.go. Only existing server verbs are used: `newfolder` to materialize directories (empty ones included) and the streaming multipart `upload` for files. Symlinks are never followed — they are reported and skipped, so a cyclic link can't wedge the walk. Named pipes, sockets and devices are reported and skipped too: opening a pipe nobody writes to never returns (issue #38).

Index

Constants

View Source 
const (
	TreeDir     = "dir"     // remote directory ensured (created or existing)
	TreeFile    = "file"    // file uploaded
	TreeSymlink = "symlink" // local symlink skipped
	TreeSpecial = "special" // local named pipe, socket or device skipped
	TreeErr     = "error"   // one item failed (walk continues)
)

Tree event kinds delivered to a TreeProgress callback.

View Source 
const DefaultStagedThreshold = 8 << 20

DefaultStagedThreshold is the size at which the CLI switches from the one-shot multipart POST to the resumable staged protocol. It matches the server's default chunk size, so "large" means the same thing on both ends.

Variables

View Source 
var ErrChangesUnsupported = errors.New("the server cannot report changes")

ErrChangesUnsupported means the server has no `action=changes` (anything before it answers 501): the caller has to walk the tree to find out.

View Source 
var ErrNoRange = errors.New("cliclient: the server ignored the requested range")

ErrNoRange means the server answered a ranged request with the whole object.

⚠ It is a real error, not a fallback: a caller that asked for bytes [1000,2000) and silently received [0,N) would write the WRONG BYTES into a file at the offset it asked about. Wrong data is worse than a refusal.

View Source 
var ErrPreconditionFailed = errors.New("the file changed on the server since it was listed")

ErrPreconditionFailed is a conditional upload the server refused because the target is no longer what the caller last saw (HTTP 412, code PRECONDITION_FAILED). Nothing was written.

Functions

func DefaultConfigPath 

func DefaultConfigPath() (string, error)

DefaultConfigPath returns the CLI config location: $FILEX_CLI_CONFIG when set (tests / unusual homes), otherwise ~/.filex/cli.yaml.

func DefaultResumeDir 

func DefaultResumeDir() (string, error)

DefaultResumeDir is where upload bookmarks live: $FILEX_UPLOAD_STATE when set (tests, unusual homes), otherwise ~/.filex/uploads.

func IsUnauthorized 

func IsUnauthorized(err error) bool

IsUnauthorized reports whether err is an APIError with status 401 — the CLI uses it to append the "run `filex client login`" hint.

func SaveFileConfig 

func SaveFileConfig(path string, cfg FileConfig) error

SaveFileConfig writes the CLI config with owner-only permissions. The explicit Chmod after the write covers a pre-existing file whose looser mode WriteFile would otherwise keep.

Types

type APIError 

type APIError struct {
	Status  int
	Message string
	Body    []byte
}

APIError is a non-2xx response mapped to an error. Body keeps the raw payload so callers can inspect extra fields (e.g. totp_required).

func (*APIError) Error 

func (e *APIError) Error() string

Error renders "HTTP <code>: <server message>".

type ChangeStream 

type ChangeStream struct {
	Client *Client
	// OnChange receives every change frame. Called from the stream's reader
	// goroutine; it must not block for long.
	OnChange func(TreeChange)
	// OnState receives every state transition, with a human-readable detail.
	OnState func(state StreamState, detail string)
	// OnConnected runs after every successful subscribe — the first one and
	// every reconnect. Changes made while the stream was down were not
	// announced to anybody; the caller answers this by asking what changed.
	OnConnected func()
	// OnUnauthorized is told when the server refuses the client's token
	// (HTTP 401 on the ticket). The stream stops for good: retrying a
	// revoked token only fills the server's log.
	OnUnauthorized func(error)

	// Tunables. Zero values mean the production defaults; tests shrink them.
	PingEvery        time.Duration // keep-alive + dead-connection detection
	AckTimeout       time.Duration // how long to wait for `watching`
	UnsupportedRetry time.Duration // re-probe a server without the stream
	MaxBackoff       time.Duration // ceiling of the reconnect backoff
	// contains filtered or unexported fields
}

ChangeStream follows the server's change stream for a set of watch roots.

func (*ChangeStream) Run 

func (s *ChangeStream) Run(ctx context.Context)

Run keeps the stream up until ctx ends: connect, subscribe, read, and on any failure back off and do it again.

func (*ChangeStream) SetRoots 

func (s *ChangeStream) SetRoots(roots []string)

SetRoots replaces the folders being watched. Safe to call at any time; a live connection re-subscribes at once.

func (*ChangeStream) State 

func (s *ChangeStream) State() StreamState

State reports the current state ("" before the first attempt).

type Client 

type Client struct {
	BaseURL string
	Token   string
	HTTP    *http.Client

	// StagedThreshold is the file size at or above which uploads use the
	// resumable staged protocol instead of one multipart POST. 0 means the
	// default (DefaultStagedThreshold); a negative value disables the staged
	// path, which is how a caller pins the old behaviour.
	StagedThreshold int64
	// ChunkSize is the part size asked for at `begin`. 0 lets the server
	// choose — and the server's answer is binding either way.
	ChunkSize int64
	// ResumeDir holds the bookmarks that let an interrupted upload continue
	// across process restarts. Empty disables persistence: uploads still
	// resume within a run, but a restart begins the file again.
	ResumeDir string
	// DownLimit and UpLimit cap download and upload bodies (bytes per
	// second), shared by every transfer of this client. nil = no limit.
	DownLimit *RateLimiter
	UpLimit   *RateLimiter
}

Client is a thin REST client for one filex server. Token may be either a session token minted by /api/auth/login or a durable API token — the server accepts both on the Authorization: Bearer header.

func New 

func New(conn Conn) *Client

New builds a Client from a resolved Conn. No global timeout is set — uploads/downloads stream arbitrarily large bodies; cancellation is the caller's context (Ctrl-C in the CLI).

func (*Client) Changes 

func (c *Client) Changes(ctx context.Context, remote, since string) (cursor string, changed bool, err error)

Changes asks whether anything under the folder remote changed since the cursor since (empty = never asked; always "changed"), and returns the cursor to send next time. One request, instead of listing the whole tree.

func (*Client) Download 

func (c *Client) Download(ctx context.Context, remote string, w io.Writer) (int64, error)

Download streams the remote file into w and returns the byte count.

⚠ Only a body that IS the file is ever copied: a 200, or a 206 answering this request's own `bytes=0-` with the whole object. Servers from v0.20 to v0.42 answered an unranged download of a big file on a slow storage with `202 {"state":"preparing",…}`, and the old check here — anything 2xx is the file — wrote that JSON to disk under the file's name. The sync engine then saw a locally edited file and uploaded the JSON over the real one: 45 files of 70–290 MB on one deployment, unrecoverable from version history.

`bytes=0-` is the request no server version answers with 202 (a Range comes from a client already committed to a body), so the CLI never waits on a server-side copy of a file it reads exactly once anyway. Any other 2xx is still refused, and nothing of it is written.

func (*Client) DownloadSized 

func (c *Client) DownloadSized(ctx context.Context, remote string, w io.Writer, want int64) (int64, error)

DownloadSized is Download for a caller that knows how big the file is — the sync engine does, from the listing it planned against. want < 0 means unknown. A body of any other length is a *SizeMismatchError, which on its own would have caught the 202 above: the listing said 151,983,227 bytes and the body was 88.

func (*Client) List 

func (c *Client) List(ctx context.Context, remote string) (*ListResult, error)

List returns the directory listing at remote (`adapter://rel`). An empty remote asks the server for its default view — useful only for discovering the Storages slice (adapter names).

func (*Client) ListStats 

func (c *Client) ListStats(ctx context.Context, remote string) ([]*Stat, error)

ListStats returns one directory's entries as stats.

func (*Client) Login 

func (c *Client) Login(ctx context.Context, email, password, totp string) (*LoginResponse, error)

Login exchanges email+password (and an optional TOTP code) for a session token. Call on a token-less Client.

func (*Client) Mkdir 

func (c *Client) Mkdir(ctx context.Context, remote string) ([]byte, error)

Mkdir creates the folder named by the last segment of remote inside its parent (server verb: newfolder).

func (*Client) Move 

func (c *Client) Move(ctx context.Context, src, dst string) (RemotePath, []byte, error)

Move implements Unix-mv semantics on top of the server's move/rename verbs. dst may be an existing directory (item moves into it, keeping its name) or a target path (rename, or move+rename across dirs — two wire calls, the manager API has no combined verb). Cross-adapter moves are rejected client-side; the server refuses them anyway.

func (*Client) ReadRange 

func (c *Client) ReadRange(ctx context.Context, remote string, off, length int64) ([]byte, error)

ReadRange fetches [off, off+length) of a remote file.

It returns the bytes actually read, which may be fewer at the end of the object. A server that answers 200 instead of 206 gets ErrNoRange rather than having its whole-object body treated as the window that was asked for.

func (*Client) Remove 

func (c *Client) Remove(ctx context.Context, remote string) ([]byte, error)

Remove sends the item to the server-side trash (server verb: delete — filex soft-deletes into `.filex-trash`, restorable from the panel).

func (*Client) Search 

func (c *Client) Search(ctx context.Context, query, scope string, storageID int64, limit int) (*SearchResult, error)

Search queries the server-side index. scope is "name", "content" or "all" ("" = server default, all). storageID 0 searches every storage.

func (*Client) Share 

func (c *Client) Share(ctx context.Context, remote string, pin bool, expiresDays int) (*ShareResult, error)

Share mints a public download link for remote. pin=true asks the server to generate an unlock PIN (returned once, in PIN); expiresDays > 0 sets the expiry that many days from now.

func (*Client) SizeOf 

func (c *Client) SizeOf(ctx context.Context, remote string) (int64, error)

SizeOf reads an object's length from a HEAD-shaped request.

It uses a one-byte ranged GET rather than HEAD: the manager endpoint answers HEAD, but a Content-Range header is the only place the TOTAL size appears in a partial response, and this is the same request the read path makes.

func (*Client) StatPath 

func (c *Client) StatPath(ctx context.Context, remote string) (*Stat, error)

StatPath returns metadata for one path.

⚠ It answers from the PARENT's listing rather than a per-file endpoint, because the manager API has no stat verb: `action=index` on a file 404s. The listing is what the server already computes, so this costs one request and stays correct when a driver reports a size the database has not caught up with.

func (*Client) Upload 

func (c *Client) Upload(ctx context.Context, localPath, remote string) (RemotePath, []byte, error)

Upload streams localPath to the server. remote may be a destination folder (`docs://reports/` — trailing slash or an existing dir) or a full target path (`docs://reports/renamed.pdf`); an existing remote folder wins, otherwise the last segment becomes the uploaded filename. Nothing is ever buffered in memory, and a large file is resumable — see uploadFile.

func (*Client) UploadTo 

func (c *Client) UploadTo(ctx context.Context, localPath string, destDir RemotePath, name, expect string) ([]byte, error)

UploadTo sends localPath to destDir/name with no destination probing and an optional overwrite precondition — the call `filex sync` makes.

expect is "" (no precondition), "none" (the target must not exist yet) or "<size>:<last_modified-ms>" (the target must still be exactly what a listing reported). A refusal comes back as ErrPreconditionFailed. An older server ignores the field and writes unconditionally — no worse than before.

⚠ No remoteIsDir probe, unlike Upload: the caller already knows the target is a file path, and the probe is a full listing round-trip in front of every file — on a server behind a CDN proxy (~0.35 s per request, measured) that was a third of the time a one-line edit took to leave the machine.

func (*Client) UploadTree 

func (c *Client) UploadTree(ctx context.Context, localDir, remote string, progress TreeProgress) (*TreeReport, error)

UploadTree uploads localDir recursively into remote. Empty local folders are created remotely too. Per-item failures accumulate in the report (a failed mkdir skips its subtree); only pre-flight problems (bad remote path, unreadable root) and context cancellation return a non-nil error. A plain file with -r degrades to a single upload, matching `cp -r` behavior.

type Conn 

type Conn struct {
	URL   string
	Token string
}

Conn is the resolved server coordinate a client command runs against.

func Resolve 

func Resolve(flagURL, flagToken, cfgPath string, getenv func(string) string) (Conn, error)

Resolve merges connection settings by precedence: 

--url/--token flags  >  FILEX_URL/FILEX_TOKEN env  >  config file.

getenv is injected so tests don't have to mutate the process env.

type FileConfig 

type FileConfig struct {
	URL   string `yaml:"url"`
	Token string `yaml:"token"`
}

FileConfig is the on-disk shape of ~/.filex/cli.yaml. It carries a bearer token, so SaveFileConfig always writes it owner-only (0600).

func LoadFileConfig 

func LoadFileConfig(path string) (FileConfig, error)

LoadFileConfig reads the CLI config. A missing file is not an error — it just yields the zero config so flags/env can still win.

type ListEntry 

type ListEntry struct {
	Path         string `json:"path"`
	Basename     string `json:"basename"`
	Type         string `json:"type"` // "file" | "dir"
	Extension    string `json:"extension"`
	Size         int64  `json:"size"`
	MimeType     string `json:"mime_type"`
	LastModified int64  `json:"last_modified"` // Unix millis; 0 = unknown
}

ListEntry is one row of a directory listing (server FileNode shape).

type ListResult 

type ListResult struct {
	Adapter  string      `json:"adapter"`
	Storages []string    `json:"storages"`
	Dirname  string      `json:"dirname"`
	ReadOnly bool        `json:"read_only"`
	Files    []ListEntry `json:"files"`
	Raw      []byte      `json:"-"`
}

ListResult is GET /api/files/manager?action=index.

type LoginResponse 

type LoginResponse struct {
	Token string `json:"token"`
	Raw   []byte `json:"-"`
}

LoginResponse is the subset of POST /api/auth/login the CLI needs.

type RateLimiter 

type RateLimiter struct {
	// contains filtered or unexported fields
}

RateLimiter caps a direction of traffic at a number of bytes per second.

One limiter is shared by every transfer of that direction, so the four parallel transfers of a sync run TOGETHER stay under the limit — which is the point: a nine-hour first sync saturated a server's ~18 Mbit line and everyone else using that server slowed down, and `--transfers` only changes how many files move at once, not how fast.

It paces the bodies, never the connection: HTTP/2 health pings and window updates share the connection, and a throttled connection would delay them past the client's own dead-connection timeout.

A token bucket holding one second's worth of bytes; a stream that runs ahead goes into debt and sleeps it off, so concurrent streams share the rate fairly. A nil *RateLimiter never waits.

func NewRateLimiter 

func NewRateLimiter(bytesPerSecond int64) *RateLimiter

NewRateLimiter returns a limiter for bytesPerSecond, or nil (no limit) for zero or less.

func (*RateLimiter) Reader 

func (l *RateLimiter) Reader(ctx context.Context, r io.Reader) io.Reader

Reader paces r through the limiter. A nil limiter returns r itself.

func (*RateLimiter) WaitN 

func (l *RateLimiter) WaitN(ctx context.Context, n int) error

WaitN takes n bytes' worth from the bucket, sleeping as long as that puts the bucket in debt. It returns early with the context's error.

type RemotePath 

type RemotePath struct {
	Adapter string
	Rel     string
}

RemotePath is a parsed `adapter://relative/path` reference. Adapter is the storage name on the server; Rel is the slash-separated relative path inside it ("" = storage root).

func ParseRemotePath 

func ParseRemotePath(raw string) (RemotePath, error)

ParseRemotePath validates and splits an `adapter://rel/path` argument. The adapter prefix is mandatory — a bare relative path is ambiguous when the server hosts several storages. `..` segments are rejected before they ever reach the wire.

func (RemotePath) Base 

func (p RemotePath) Base() string

Base returns the last path segment ("" at the storage root).

func (RemotePath) Dir 

func (p RemotePath) Dir() RemotePath

Dir returns the parent directory (the storage root stays the root).

func (RemotePath) IsRoot 

func (p RemotePath) IsRoot() bool

IsRoot reports whether the path points at the storage root.

func (RemotePath) Join 

func (p RemotePath) Join(name string) RemotePath

Join appends one segment to the path.

func (RemotePath) String 

func (p RemotePath) String() string

String renders the wire form `adapter://rel` (root: `adapter://`).

type SearchHit 

type SearchHit struct {
	ID        int64  `json:"id"`
	StorageID int64  `json:"storage_id"`
	Name      string `json:"name"`
	Path      string `json:"path"`
	Type      string `json:"type"`
	Size      int64  `json:"size"`
	Snippet   string `json:"snippet"`
	Matched   string `json:"matched"` // "name" | "content" | "both"
}

SearchHit is one result row from /api/files/search (node fields plus the v0.2 content-search additions).

type SearchResult 

type SearchResult struct {
	Results []SearchHit `json:"results"`
	Raw     []byte      `json:"-"`
}

SearchResult is the /api/files/search envelope.

type ShareResult 

type ShareResult struct {
	URL       string     `json:"url"`
	Token     string     `json:"token"`
	HasPin    bool       `json:"has_pin"`
	PIN       string     `json:"password_pin"` // only set when the server generated one
	ExpiresAt *time.Time `json:"expires_at"`
	Raw       []byte     `json:"-"`
}

ShareResult is the nested `share` object of POST /api/files/share.

type SizeMismatchError 

type SizeMismatchError struct {
	Remote string
	Want   int64 // what the listing said
	Got    int64 // what the server declared or sent
}

SizeMismatchError is a download whose body is not the length the caller's listing reported. When the server DECLARES the other length up front (Content-Length, or the total of a Content-Range) not one byte reaches the writer; a body that only turns out short or long at its end has reached it, and the caller — which writes to a temporary file — throws it away.

func (*SizeMismatchError) Error 

func (e *SizeMismatchError) Error() string

type Stat 

type Stat struct {
	Name    string
	Size    int64
	IsDir   bool
	ModTime time.Time
	Mime    string
}

Stat describes one remote entry.

type StreamState 

type StreamState string

StreamState is what the stream can honestly say about itself. 

const (
	// StreamLive: subscribed; changes arrive as they happen.
	StreamLive StreamState = "connected"
	// StreamPolling: the server cannot announce changes (older than the
	// `watch` subscription, or realtime is switched off). Only the interval
	// poll finds them.
	StreamPolling StreamState = "polling"
	// StreamOffline: the server could not be reached; retrying with backoff.
	StreamOffline StreamState = "offline"
)

type TreeChange 

type TreeChange struct {
	Root     string
	Dirs     []string
	Overflow bool
	Action   string
	Name     string
}

TreeChange is one `tree_change` frame: something changed at or below Root. Dirs are relative to Root ("" = Root itself). Overflow means too many folders changed to name them — treat the whole root as changed.

type TreeError 

type TreeError struct {
	Local string
	Err   error
}

TreeError is one failed item of a recursive upload.

type TreeEvent 

type TreeEvent struct {
	Kind   string     // TreeDir | TreeFile | TreeSymlink | TreeSpecial | TreeErr
	Local  string     // local path of the item
	Remote RemotePath // resolved remote target (zero for symlink/local errors)
	Err    error      // set when Kind == TreeErr
}

TreeEvent is one progress notification from UploadTree.

type TreeProgress 

type TreeProgress func(ev TreeEvent)

TreeProgress receives one call per processed item during UploadTree — the CLI streams these as they happen. May be nil.

type TreeReport 

type TreeReport struct {
	Files    int         // files uploaded successfully
	Dirs     int         // remote directories ensured (created or pre-existing)
	Symlinks []string    // local symlinks skipped (never followed)
	Special  []string    // local named pipes, sockets and devices skipped (never opened)
	Errors   []TreeError // items that failed (upload, mkdir, walk)
}

TreeReport summarizes a finished UploadTree run. Errors are per-item: the walk keeps going past a failed file, and a failed mkdir skips just that subtree.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.