Documentation
¶
Overview ¶
Package fileblock is the file-type blocking engine: an extension/MIME/ Content-Disposition blocklist (FileBlocker) and named file-type profiles (FileProfileStore, in fileprofile.go). It depends only on the shared seam (internal/obs for logging, internal/fileutil for atomic persistence) and the standard library — no dependency on the rest of Culvert (ADR-0002 leaf, unblocked by the ADR-0003 seam).
Index ¶
- Variables
- func BlockConn(dst interface{ ... }, host, urlPath, ext, source string)
- func BlockMessage(ext, source string) string
- func ExtractCDFilename(cd string) string
- func LogBlock(host, urlPath, ext, source string)
- func ValidateProfiles(profiles []FileExtProfile) error
- type ErrRevisionConflict
- type FileBlocker
- func (fb *FileBlocker) Add(ext string)
- func (fb *FileBlocker) CheckContentDisposition(cd string) string
- func (fb *FileBlocker) CheckContentType(contentType string) string
- func (fb *FileBlocker) CheckExt(ext string) string
- func (fb *FileBlocker) CheckPath(urlPath string) string
- func (fb *FileBlocker) ClearAll()
- func (fb *FileBlocker) Count() int
- func (fb *FileBlocker) List() []string
- func (fb *FileBlocker) Remove(ext string)
- func (fb *FileBlocker) ReplaceAll(exts []string)
- func (fb *FileBlocker) SetPath(p string)
- type FileExtProfile
- type FileProfileStore
- func (s *FileProfileStore) Create(name string, exts []string) (*FileExtProfile, error)
- func (s *FileProfileStore) CreateFenced(ifRevision, name string, exts []string) (*FileExtProfile, error)
- func (s *FileProfileStore) Delete(id string) error
- func (s *FileProfileStore) DeleteFenced(ifRevision, id string) error
- func (s *FileProfileStore) GetByID(id string) *FileExtProfile
- func (s *FileProfileStore) GetByName(name string) *FileExtProfile
- func (s *FileProfileStore) List() []*FileExtProfile
- func (s *FileProfileStore) Load(path string) error
- func (s *FileProfileStore) NameByID(id string) (string, bool)
- func (s *FileProfileStore) ReplaceAll(profiles []FileExtProfile) error
- func (s *FileProfileStore) Revision() string
- func (s *FileProfileStore) Save() error
- func (s *FileProfileStore) SetPath(p string)
- func (s *FileProfileStore) SnapshotWithRevision() (profiles []*FileExtProfile, revision string)
- func (s *FileProfileStore) Update(id, name string, exts []string) error
- func (s *FileProfileStore) UpdateFenced(ifRevision, id, name string, exts []string) error
Constants ¶
This section is empty.
Variables ¶
var DefaultBlockedExts = []string{
".exe", ".dll", ".bat", ".cmd", ".ps1",
".vbs", ".scr", ".msi", ".pif", ".com",
}
DefaultBlockedExts is loaded at startup when no config override is provided. Covers common Windows malware/script delivery formats.
Functions ¶
func BlockConn ¶
func BlockConn(dst interface {
Write([]byte) (int, error)
Close() error
}, host, urlPath, ext, source string)
BlockConn writes a synthetic HTTP/1.1 403 response to a raw connection and closes it. Retained for the raw-conn callers and its own test; the SSL-inspect path no longer uses it (it emits via the protocol-neutral blockResponder so the same detector serves HTTP/2, where closing the shared conn on a per-stream block would kill sibling streams and Connection: close is illegal per RFC 9113 §8.2.2). The force-close here prevents HTTP/1.1 pipelined-request bypass.
func BlockMessage ¶ added in v1.0.70
BlockMessage returns the plain-text body of a file-block 403 for the given extension and source label. Exposed so the SSL-inspect path can emit the block through its protocol-neutral responder (HTTP/1.1 or HTTP/2) without this package owning HTTP framing; the legacy raw-conn BlockConn uses it too.
func ExtractCDFilename ¶
ExtractCDFilename extracts the filename from a Content-Disposition header. Returns "" if no filename is found. Used by per-rule file profile checking when the download URL doesn't contain the file extension (e.g. SourceForge's /files/latest/download pattern).
func LogBlock ¶ added in v1.0.70
func LogBlock(host, urlPath, ext, source string)
LogBlock emits the FILE_BLOCKED tunnel observability line. Split out of BlockConn so the SSL-inspect path can log identically while emitting the wire block through its own responder (keeping this package framing-free).
func ValidateProfiles ¶ added in v1.0.218
func ValidateProfiles(profiles []FileExtProfile) error
ValidateProfiles is the canonical FileProfile SET validation seam (2D-C final §13–§14), applied at every trust/load boundary (disk Load, CP→DP ConfigSnapshot preflight, ReplaceAll): IDs are enforcement-authoritative, so a set with a missing or duplicate ID is ambiguous identity and must be refused — never repaired by minting fresh identity nondeterministically per load. Names are the interactive intent namespace and must be non-empty and case-insensitively unique. IDs are deliberately NOT required to be UUIDs: the seeded built-ins carry deterministic `builtin-*` IDs by design. Missing IDs are corruption, not legacy: FileExtProfile was born with the ID field, the built-in IDs, and a uuid-minting Create (§14 audit) — every version that ever persisted profiles wrote IDs.
Types ¶
type ErrRevisionConflict ¶ added in v1.0.218
type ErrRevisionConflict struct{ Current string }
ErrRevisionConflict is returned by a fenced mutation whose asserted revision no longer matches the committed profile set (another admin mutated it in between). Carries the CURRENT revision so the API can render a structured 409 the client uses to refresh.
func (*ErrRevisionConflict) Error ¶ added in v1.0.218
func (e *ErrRevisionConflict) Error() string
type FileBlocker ¶
type FileBlocker struct {
// contains filtered or unexported fields
}
FileBlocker holds the set of file extensions to block. Extensions are normalised to lowercase with a leading dot (e.g. ".exe"). All operations are safe for concurrent use. Persists to a JSON file when a path is configured via SetPath().
── The per-transaction gate is LOCK-FREE ─────────────────────────────────────
CheckPath runs on every plain-HTTP request (proxy.go preDispatchBlocked) and every SSL-inspected inner request; CheckContentType and CheckContentDisposition run on every plain-HTTP response (proxy_http.go blockedByResponseHeaders) and every inspected response (proxy_tunnel.go inspectFileBlocked). So one inspected transaction reached this store up to THREE times, and each probe took mu.RLock — an atomic read-modify-write on ONE process-wide word — purely to read a set that in steady state never changes (the shipped default is ten extensions seeded once at startup).
That is not a constant cost but a THROUGHPUT CEILING, the same shape already recorded for internal/threatfeed, security.go's IP filter, internal/connlimit and metrics.go's latency histogram. Measured on this machine (4-vCPU Xeon, Go 1.26, CheckPath against "/static/app.js", medians of n=15; BenchmarkCheckPathParallel{,Legacy}, whose ns/op is already wall-normalized across workers, so 1-core-ns ÷ 4-core-ns IS the scaling factor):
│ 1 core │ 4 workers │ scaling before │ 28.4 ns │ 105.6 ns │ 0.27x — four cores delivered a QUARTER of one after │ 19.7 ns │ 17.2 ns │ 1.15x — flat, and 6.2x the old four-core ceiling
Note the shape, not the constants: the "after" row is FLAT, not linear. The published view is a shared read-only cache line, so it does not scale a four-worker load on a 4-vCPU VM into 4x; what it removes is the DEGRADATION. The absolute numbers are load-sensitive by construction — a contended lock degrades further the busier the box is, so the "before" row understates the problem on the larger hardware the appliance ships to.
Reads now go through an immutable view published via atomic.Pointer. THE CONTRACT IS ONE LINE AND IT IS LOAD-BEARING: a map reachable from a published view is never mutated in place. The mu-guarded fields stay the AUTHORITATIVE write-side state — every mutator keeps its exact prior semantics and calls publishLocked() before releasing the lock. Adding a mutator without that call is a silent SECURITY failure, not a performance one (a newly-blocked extension that never blocks, a removed one that keeps blocking), so it is pinned per mutator by TestBlockerView_EveryMutatorRepublishes.
publishLocked COPIES rather than aliasing, so an Add loop is O(N) per call. That introduces no new complexity class: Add already calls save(), which marshals the WHOLE set and rewrites the file on every single call, so the bulk paths were quadratic in bytes-to-disk before this change and the map copy is strictly cheaper than the marshal it sits beside. ReplaceAll remains the batch entry point for bulk loads (CL-13).
func NewBlocker ¶
func NewBlocker() *FileBlocker
NewBlocker returns an empty, ready-to-use FileBlocker. package main holds the process-wide singleton (var fileBlocker = fileblock.NewBlocker()).
func (*FileBlocker) Add ¶
func (fb *FileBlocker) Add(ext string)
Add inserts a normalised extension into the block list and persists.
func (*FileBlocker) CheckContentDisposition ¶
func (fb *FileBlocker) CheckContentDisposition(cd string) string
CheckContentDisposition returns the blocked extension if the Content-Disposition response header carries a filename with a blocked extension (catches downloads that use a generic URL but declare the real file name in the header).
func (*FileBlocker) CheckContentType ¶
func (fb *FileBlocker) CheckContentType(contentType string) string
CheckContentType returns the blocked extension if the response Content-Type header matches a dangerous MIME type whose associated extension is in the block list. This prevents bypass by renaming files (e.g. malware.exe → malware.txt). The contentType parameter should be the raw Content-Type header value (e.g. "application/x-msdownload; charset=utf-8").
── Why the media type is decided before the header is parsed ─────────────────
This runs on EVERY proxied response that carries a Content-Type — the plain-HTTP forward path (proxy_http.go blockedByResponseHeaders) and the SSL-inspect path (proxy_tunnel.go inspectFileBlocked) alike. It consumed the full mime.ParseMediaType, which allocates a map[string]string for the header PARAMETERS and then walks the header to fill it — and this function discards that map. Every response paid it so that a ten-entry lookup could miss.
Measured on this machine (4-vCPU Xeon, Go 1.26, "text/html; charset=utf-8" — the shape almost all web traffic carries; medians of n=6):
before 281.7 ns/op 336 B/op 2 allocs/op after 35.6 ns/op 0 B/op 0 allocs/op
The trade-off is stated rather than papered over: the ten dangerous media types now pay the cheap split AND the parse, 347.2 -> 384.8 ns (+11%). They are the arm that is about to serve a block page and write two log lines, and ordinary traffic never reaches it, so the exchange is accepted deliberately. BenchmarkCheckContentType_Dangerous keeps it measurable.
The parse is now reached ONLY for a header whose media type is one of the ten dangerous types, i.e. never on ordinary traffic. The pre-filter is a pure NEGATIVE one: it can only return the empty (allow) answer early, and every BLOCK is still decided by the original body, unchanged, parse included. So the malformed-parameter case still declines to block exactly as before — ParseMediaType reports ErrInvalidMediaParameter for "application/x-msdownload; bogus" and this returns "", which a naive prefix split would have turned into a block. That would have been a tightening, but a tightening is still a behaviour change in a cost fix, so it is deliberately not taken here.
The equivalence is exact, not approximate. ParseMediaType computes its returned mediatype as strings.TrimSpace(strings.ToLower(base)) over base, _, _ := strings.Cut(v, ";") — reproduced verbatim below — and returns a non-nil error otherwise, in which case the pre-fix body returned "" as well. So "candidate is not a blocked MIME type" implies the pre-fix body returned "" for every input, including the malformed ones. Pinned against a verbatim copy of the pre-fix body by TestCheckContentType_MatchesPreFilterBehaviour and FuzzCheckContentType, so an upstream mime behaviour change fails CI rather than silently diverging.
func (*FileBlocker) CheckExt ¶
func (fb *FileBlocker) CheckExt(ext string) string
CheckExt checks if a specific extension is in the block list. Extension should include the dot (e.g., ".exe").
func (*FileBlocker) CheckPath ¶
func (fb *FileBlocker) CheckPath(urlPath string) string
CheckPath returns the blocked extension if urlPath ends with a blocked file extension, or empty string if the request is allowed. Pass r.URL.Path (not the full URL) to avoid matching query-string artefacts.
func (*FileBlocker) ClearAll ¶
func (fb *FileBlocker) ClearAll()
ClearAll removes all blocked extensions. Used by config import "replace" mode.
func (*FileBlocker) Count ¶
func (fb *FileBlocker) Count() int
Count returns the number of blocked extensions.
func (*FileBlocker) List ¶
func (fb *FileBlocker) List() []string
List returns a snapshot of the blocked extensions.
func (*FileBlocker) Remove ¶
func (fb *FileBlocker) Remove(ext string)
Remove deletes an extension from the block list and persists.
func (*FileBlocker) ReplaceAll ¶
func (fb *FileBlocker) ReplaceAll(exts []string)
ReplaceAll atomically replaces the in-memory extension set with the normalised contents of exts and persists once. Per-element semantics mirror Add exactly: lowercase + trim, leading-dot inserted if missing, empty / bare-dot entries skipped, duplicates collapsed by the set.
CL-13: used by applyConfigSnapshot's FileBlockExtensions branch to replace the prior ClearAll + per-extension Add loop, which triggered N+1 atomicWriteFile syscalls per snapshot apply (cap 10_000 extensions per maxSnapFileBlockExtensions). ReplaceAll triggers exactly one save() call regardless of N.
Save is invoked unconditionally — including when the new set is equal to the existing set. Skipping save on equal content would require deep-comparing maps under the lock and add a fast-path that could deviate from the long-term durability guarantee, which is out of CL-13 scope per the user brief.
func (*FileBlocker) SetPath ¶
func (fb *FileBlocker) SetPath(p string)
SetPath configures the persistence file and loads any previously saved extensions from it. If the file doesn't exist, the current in-memory state is kept (caller should load defaults before calling SetPath).
type FileExtProfile ¶
type FileExtProfile struct {
ID string `json:"id"`
Name string `json:"name"`
Extensions []string `json:"extensions"`
}
FileExtProfile is a named set of file extensions used for per-policy-rule blocking.
type FileProfileStore ¶
type FileProfileStore struct {
// contains filtered or unexported fields
}
FileProfileStore manages a persistent collection of file extension profiles. All operations are safe for concurrent use.
DURABILITY + PUBLICATION CONTRACT (2D-C.0B):
- Elements are IMMUTABLE after publication: every mutation builds a NEW slice with NEW element values (copy-on-write pointer swap) — never an in-place field write, which would race the lock-free p.Extensions reads on the enforcement path (FileProfileBlocked reads the returned pointer outside the store lock).
- Mutations are durable-or-nothing: the TARGET set is persisted FIRST (AtomicWrite) and the in-memory swap happens only after the write landed. A hard persistence failure returns the error with memory (and therefore restart truth) unchanged. fileutil.ErrReplacedNotSynced follows the landed-content doctrine: the file visibly contains the new set, so memory swaps forward and the sentinel is surfaced, never rolled back against landed content.
- The revision is a CONTENT-DERIVED semantic fingerprint (Revision) over the sorted (id, name, normalized extensions) tuples — restart-stable with no on-disk format migration, which is exactly what a browser optimistic fence needs. A fenced mutation compares the asserted revision INSIDE the same critical section that mutates and persists.
func (*FileProfileStore) Create ¶
func (s *FileProfileStore) Create(name string, exts []string) (*FileExtProfile, error)
Create adds a new profile, durable-or-nothing. ifRevision "" skips the optimistic fence. Returns an error if the name is already taken.
func (*FileProfileStore) CreateFenced ¶ added in v1.0.218
func (s *FileProfileStore) CreateFenced(ifRevision, name string, exts []string) (*FileExtProfile, error)
CreateFenced is Create with the v2 optimistic-revision fence: comparison, mutation, and durable publish share this one critical section.
func (*FileProfileStore) Delete ¶
func (s *FileProfileStore) Delete(id string) error
Delete removes a profile by ID, durable-or-nothing. ifRevision "" skips the fence (legacy callers).
func (*FileProfileStore) DeleteFenced ¶ added in v1.0.218
func (s *FileProfileStore) DeleteFenced(ifRevision, id string) error
DeleteFenced is Delete with the v2 optimistic-revision fence.
func (*FileProfileStore) GetByID ¶
func (s *FileProfileStore) GetByID(id string) *FileExtProfile
GetByID returns the profile with the given ID, or nil.
func (*FileProfileStore) GetByName ¶
func (s *FileProfileStore) GetByName(name string) *FileExtProfile
GetByName returns the profile with the given name (case-insensitive), or nil.
func (*FileProfileStore) List ¶
func (s *FileProfileStore) List() []*FileExtProfile
List returns a copy of all profiles.
func (*FileProfileStore) Load ¶
func (s *FileProfileStore) Load(path string) error
Load reads profiles from disk. If the file does not exist the built-in profiles are seeded and persisted so policy rules continue to work. A persisted set that fails ValidateProfiles is REFUSED — the store keeps its prior (empty at boot) contents rather than publishing ambiguous identity, and the ID-authoritative enforcement path fails closed on the rules that referenced it.
func (*FileProfileStore) NameByID ¶ added in v1.0.72
func (s *FileProfileStore) NameByID(id string) (string, bool)
NameByID returns the profile's name (a value copy, read under the lock) and whether it exists. Callers needing only the name must use this rather than GetByID().Name — GetByID returns the LIVE pointer, and reading .Name off it outside the lock races a concurrent Update (which mutates p.Name in place).
func (*FileProfileStore) ReplaceAll ¶
func (s *FileProfileStore) ReplaceAll(profiles []FileExtProfile) error
ReplaceAll atomically replaces all profiles in memory and persists the new set. FOLLOWER semantics by design (2D-C.0B §12 audit): the ONLY production caller is the CP→DP ConfigSnapshot apply, where the CONTROL PLANE is the source of truth — the in-memory swap is authoritative so the DP enforces the CP's profiles even on a wedged local disk, the persist failure is logged (and observed by the CHAOS-45 durable-write chokepoint), and the next CP sync re-converges. CANDIDATE VALIDITY is separate from that follower durability posture (2D-C final §15): a set failing ValidateProfiles is refused up front — ReplaceAll must never make ambiguous identity authoritative (the snapshot preflight rejects it earlier; this is the defense-in-depth at the store boundary). Local confirmed administrative mutations must NEVER use this path — they go through Create/Update/Delete, which are durable-or-nothing. Tests are the other callers (state seeding).
func (*FileProfileStore) Revision ¶ added in v1.0.218
func (s *FileProfileStore) Revision() string
Revision returns the content-derived semantic fingerprint of the committed profile set. Restart-stable: same profiles ⇒ same revision across restarts.
func (*FileProfileStore) Save ¶
func (s *FileProfileStore) Save() error
Save persists the current profile set to the configured path.
func (*FileProfileStore) SetPath ¶
func (s *FileProfileStore) SetPath(p string)
SetPath sets the persistence file path without reading from disk (use Load to also read existing profiles). Added for the package boundary: package main integration tests need to redirect persistence without the whitebox field access they relied on before the move.
func (*FileProfileStore) SnapshotWithRevision ¶ added in v1.0.218
func (s *FileProfileStore) SnapshotWithRevision() (profiles []*FileExtProfile, revision string)
SnapshotWithRevision returns the profile rows AND the revision that describes exactly them, captured under ONE read lock (coherent-read doctrine, 2D-A/2D-B): a management GET must never pair rows from one state with another state's fence token. The returned pointers are immutable published values (see the store contract).
func (*FileProfileStore) Update ¶
func (s *FileProfileStore) Update(id, name string, exts []string) error
Update replaces the name and/or extensions of an existing profile, durable-or-nothing (copy-on-write — the published element is never mutated in place). ifRevision "" skips the fence (legacy callers).
func (*FileProfileStore) UpdateFenced ¶ added in v1.0.218
func (s *FileProfileStore) UpdateFenced(ifRevision, id, name string, exts []string) error
UpdateFenced is Update with the v2 optimistic-revision fence.
Source Files