loading

package module
v0.26.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jun 7, 2026 License: Apache-2.0 Imports: 17 Imported by: 9

Documentation

Overview

Package loading provides tools to load a file from http or from a local file system.

Security

By default, the local loader reads any path the process can access, including absolute paths and "file://" URIs (for example "file:///etc/passwd"). Applications that pass untrusted input to LoadFromFileOrHTTP, JSONDoc (or to downstream consumers such as go-openapi/loads) must confine local loading to a trusted directory.

Use WithRoot to do so: it resolves every requested path relative to a chosen directory and rejects anything that escapes it, including via symlink. It is built on os.Root and is therefore safer than passing an os.DirFS to WithFS, which does not block symlink escapes.

Remote loading uses a standard net/http client. By default it follows redirects and performs no destination filtering — exactly like net/http.DefaultClient.

A caller-controlled URL may therefore reach internal services or cloud metadata endpoints (server-side request forgery).

This package does not, and should not, embed a network policy: when the URL may derive from untrusted input, supply a restricted client with WithHTTPClient whose transport rejects unwanted destinations at dial time — which also covers redirects and DNS rebinding. See the example on LoadFromFileOrHTTP.

Index

Examples

Constants

View Source
const (
	// ErrLoader is an error raised by the file loader utility
	ErrLoader loadingError = "loader error"
)

Variables

This section is empty.

Functions

func JSONDoc

func JSONDoc(path string, opts ...Option) (json.RawMessage, error)

JSONDoc loads a json document from either a file or a remote url.

func JSONMatcher

func JSONMatcher(path string) bool

JSONMatcher matches json for a file loader.

func LoadFromFileOrHTTP

func LoadFromFileOrHTTP(pth string, opts ...Option) ([]byte, error)

LoadFromFileOrHTTP loads the bytes from a file or a remote http server based on the path passed in.

Security: by default a local path is read with no confinement, so a caller-controlled path (including a "file://" URI or an absolute path) may read any file the process can access. When the path may derive from untrusted input, confine local loading with WithRoot.

Example (RestrictNetwork)

ExampleLoadFromFileOrHTTP_restrictNetwork shows how to confine remote spec loading so a caller-controlled URL cannot reach loopback, private, or link-local (cloud metadata) addresses.

The net.Dialer Control hook runs after DNS resolution and before connect, on every connection, so the check also covers HTTP redirects and DNS rebinding — neither of which a URL-string allowlist can defend against. Here a loopback test server stands in for an internal endpoint that the guard must refuse to reach.

// SPDX-FileCopyrightText: Copyright 2015-2025 go-swagger maintainers
// SPDX-License-Identifier: Apache-2.0

package main

import (
	"errors"
	"fmt"
	"net"
	"net/http"
	"net/http/httptest"
	"net/netip"
	"syscall"

	"github.com/go-openapi/swag/loading"
)

// errForbiddenAddr is returned by the dial guard when a destination is not allowed.
var errForbiddenAddr = errors.New("blocked dial to a forbidden address")

// ExampleLoadFromFileOrHTTP_restrictNetwork shows how to confine remote spec loading so a
// caller-controlled URL cannot reach loopback, private, or link-local (cloud metadata)
// addresses.
//
// The [net.Dialer] Control hook runs after DNS resolution and before connect, on every
// connection, so the check also covers HTTP redirects and DNS rebinding — neither of which
// a URL-string allowlist can defend against. Here a loopback test server stands in for an
// internal endpoint that the guard must refuse to reach.
func main() {
	control := func(_, address string, _ syscall.RawConn) error {
		host, _, err := net.SplitHostPort(address)
		if err != nil {
			return err
		}
		addr, err := netip.ParseAddr(host)
		if err != nil {
			return err
		}
		if a := addr.Unmap(); a.IsLoopback() || a.IsPrivate() || a.IsLinkLocalUnicast() || a.IsUnspecified() {
			return errForbiddenAddr
		}

		return nil
	}

	client := &http.Client{
		Transport: &http.Transport{
			DialContext: (&net.Dialer{Control: control}).DialContext,
		},
	}

	// An internal service the application must not let untrusted input reach.
	internal := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
		_, _ = w.Write([]byte("internal secret"))
	}))
	defer internal.Close()

	// internal.URL is a loopback address (the untrusted URL in a real attack).
	_, err := loading.LoadFromFileOrHTTP(internal.URL, loading.WithHTTPClient(client))
	fmt.Println("blocked:", errors.Is(err, errForbiddenAddr))

}
Output:
blocked: true

func LoadStrategy

func LoadStrategy(pth string, local, remote func(string) ([]byte, error), opts ...Option) func(string) ([]byte, error)

LoadStrategy returns a loader function for a given path or URI.

The load strategy returns the remote load for any path starting with `http`. So this works for any URI with a scheme `http` or `https`.

The fallback strategy is to call the local loader.

The local loader takes a local file system path (absolute or relative) as argument, or alternatively a `file://...` URI, **without host** (see also below for windows).

There are a few liberalities, initially intended to be tolerant regarding the URI syntax, especially on windows.

Before the local loader is called, the given path is transformed:

  • percent-encoded characters are unescaped
  • simple paths (e.g. `./folder/file`) are passed as-is
  • on windows, occurrences of `/` are replaced by `\`, so providing a relative path such a `folder/file` works too.

For paths provided as URIs with the "file" scheme, please note that:

  • `file://` is simply stripped. This means that the host part of the URI is not parsed at all. For example, `file:///folder/file" becomes "/folder/file`, but `file://localhost/folder/file` becomes `localhost/folder/file` on unix systems. Similarly, `file://./folder/file` yields `./folder/file`.
  • on windows, `file://...` can take a host so as to specify an UNC share location.

Reminder about windows-specifics: - `file://host/folder/file` becomes an UNC path like `\\host\folder\file` (no port specification is supported) - `file:///c:/folder/file` becomes `C:\folder\file` - `file://c:/folder/file` is tolerated (without leading `/`) and becomes `c:\folder\file`

func YAMLData

func YAMLData(path string, opts ...Option) (any, error)

YAMLData loads a yaml document from either http or a file.

func YAMLDoc

func YAMLDoc(path string, opts ...Option) (json.RawMessage, error)

YAMLDoc loads a yaml document from either http or a file and converts it to json.

func YAMLMatcher

func YAMLMatcher(path string) bool

YAMLMatcher matches yaml for a file loader.

Types

type Option

type Option func(*options)

Option provides options for loading a file over HTTP or from a file.

func WithBasicAuth

func WithBasicAuth(username, password string) Option

WithBasicAuth sets a basic authentication scheme for the remote file loader.

func WithCustomHeaders

func WithCustomHeaders(headers map[string]string) Option

WithCustomHeaders sets custom headers for the remote file loader.

func WithFS

func WithFS(filesystem fs.FS) Option

WithFS sets a file system for the local file loader.

If the provided file system is a fs.ReadFileFS, the ReadFile function is used. Otherwise, ReadFile is wrapped using fs.ReadFile.

By default, the file system is the one provided by the os package.

For example, this may be set to consume from an embedded file system, or a rooted FS.

WithFS and WithRoot are mutually exclusive: the last one applied wins.

Security note: a file system built from os.DirFS confines paths but does NOT protect against symlinks that escape the root. To load from a directory derived from untrusted input, prefer WithRoot, which is symlink-escape resistant.

func WithHTTPClient

func WithHTTPClient(client *http.Client) Option

WithHTTPClient overrides the default HTTP client used to fetch a remote file.

By default, http.DefaultClient is used.

func WithRoot added in v0.26.1

func WithRoot(dir string) Option

WithRoot confines local file loading to dir.

Every requested path is resolved relative to dir, and any path that would escape dir — whether through an absolute path, ".." traversal, or a symlink pointing outside dir — is rejected. This is built on os.Root and is therefore resistant to the symlink escapes that a plain os.DirFS does not prevent.

WithRoot is the recommended option when loading specs from a location derived from untrusted input. It applies to local loading only and has no effect on remote (http/https) loading. WithRoot and WithFS are mutually exclusive: the last one applied wins.

Note: os.Root confines path resolution but does not, by itself, protect against traversal of mount/bind boundaries, /proc special files, or device files. Point WithRoot at a directory that holds only the documents you intend to expose.

func WithTimeout

func WithTimeout(timeout time.Duration) Option

WithTimeout sets a timeout for the remote file loader.

The default timeout is 30s.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL