operation

package
v0.1.26 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Mar 24, 2021 License: MIT Imports: 32 Imported by: 5

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func FileServer

func FileServer(root FileSystem) http.Handler

FileServer returns a handler that serves HTTP http.Requests with the contents of the file system rooted at root.

To use the operating system's file system implementation, use http.Dir:

http.Handle("/", http.FileServer(http.Dir("/tmp")))

As a special case, the returned file server redirects any http.Request ending in "/index.html" to the same path, without the final "index.html".

func NewTransport

func NewTransport(dialTimeoutMs int) http.RoundTripper

func ServeContent

func ServeContent(w http.ResponseWriter, req *http.Request, name string, modtime time.Time, content io.ReadSeeker)

ServeContent replies to the http.Request using the content in the provided ReadSeeker. The main benefit of ServeContent over io.Copy is that it handles Range http.Requests properly, sets the MIME type, and handles If-Match, If-Unmodified-Since, If-None-Match, If-Modified-Since, and If-Range http.Requests.

If the response's Content-Type header is not set, ServeContent first tries to deduce the type from name's file extension and, if that fails, falls back to reading the first block of the content and passing it to DetectContentType. The name is otherwise unused; in particular it can be empty and is never sent in the response.

If modtime is not the zero time or Unix epoch, ServeContent includes it in a Last-Modified header in the response. If the http.Request includes an If-Modified-Since header, ServeContent uses modtime to decide whether the content needs to be sent at all.

The content's Seek method must work: ServeContent uses a seek to the end of the content to determine its size.

If the caller has set w's ETag header formatted per RFC 7232, section 2.3, ServeContent uses it to handle http.Requests using If-Match, If-None-Match, or If-Range.

Note that *os.File implements the io.ReadSeeker interface.

func ServeFile

func ServeFile(w http.ResponseWriter, r *http.Request, name string)

ServeFile replies to the http.Request with the contents of the named file or directory.

If the provided file or directory name is a relative path, it is interpreted relative to the current directory and may ascend to parent directories. If the provided name is constructed from user input, it should be sanitized before calling ServeFile.

As a precaution, ServeFile will reject http.Requests where r.URL.Path contains a ".." path element; this protects against callers who might unsafely use filepath.Join on r.URL.Path without sanitizing it and then use that filepath.Join result as the name argument.

As another special case, ServeFile redirects any http.Request where r.URL.Path ends in "/index.html" to the same path, without the final "index.html". To avoid such redirects either modify the path or use ServeContent.

Outside of those two special cases, ServeFile does not use r.URL.Path for selecting the file or directory to serve; only the file or directory provided in the name argument is used.

func SetCacheDirectoryAndLoad

func SetCacheDirectoryAndLoad(path string) error

func SetLogger

func SetLogger(logger kodocli.Ilog)

func StartServer

func StartServer(cfg *Config) (*http.Server, error)

func StartSimulateErrorServer

func StartSimulateErrorServer(_ *Config)

Types

type Config

type Config struct {
	IoHosts       []string `json:"io_hosts" toml:"io_hosts"`
	UcHosts       []string `json:"uc_hosts" toml:"uc_hosts"`
	UpHosts       []string `json:"up_hosts" toml:"up_hosts"`
	RsHosts       []string `json:"rs_hosts" toml:"rs_hosts"`
	RsfHosts      []string `json:"rsf_hosts" toml:"rsf_hosts"`
	Bucket        string   `json:"bucket" toml:"bucket"`
	Ak            string   `json:"ak" toml:"ak"`
	Sk            string   `json:"sk" toml:"sk"`
	PartSize      int64    `json:"part" toml:"part"`
	Addr          string   `json:"addr" toml:"addr"`
	Delete        bool     `json:"delete" toml:"delete"`
	UpConcurrency int      `json:"up_concurrency" toml:"up_concurrency"`

	DownPath string `json:"down_path" toml:"down_path"`
	Sim      bool   `json:"sim" toml:"sim"`

	Retry         int `json:"retry" toml:"retry"`
	PunishTimeS   int `json:"punish_time_s" toml:"punish_time_s"`
	DialTimeoutMs int `json:"dial_timeout_ms" toml:"dial_timeout_ms"`
}

func Load

func Load(file string) (*Config, error)

type Dir

type Dir string

A Dir implements FileSystem using the native file system restricted to a specific directory tree.

While the FileSystem.Open method takes '/'-separated paths, a Dir's string value is a filename on the native file system, not a URL, so it is separated by filepath.Separator, which isn't necessarily '/'.

Note that Dir could expose sensitive files and directories. Dir will follow symlinks pointing out of the directory tree, which can be especially dangerous if serving from a directory in which users are able to create arbitrary symlinks. Dir will also allow access to files and directories starting with a period, which could expose sensitive directories like .git or sensitive files like .htpasswd. To exclude files with a leading period, remove the files/directories from the server or create a custom FileSystem implementation.

An empty Dir is treated as ".".

func (Dir) Open

func (d Dir) Open(name string) (File, error)

Open implements FileSystem using os.Open, opening files for reading rooted and relative to the directory d.

type Downloader

type Downloader struct {
	// contains filtered or unexported fields
}

func NewDownloader

func NewDownloader(c *Config) *Downloader

func NewDownloaderV2

func NewDownloaderV2() *Downloader

func (*Downloader) DownloadBytes

func (d *Downloader) DownloadBytes(key string) ([]byte, error)

func (*Downloader) DownloadBytesWithContext

func (d *Downloader) DownloadBytesWithContext(ctx context.Context, key string) (data []byte, err error)

func (*Downloader) DownloadFile

func (d *Downloader) DownloadFile(key, path string) (*os.File, error)

func (*Downloader) DownloadFileWithContext

func (d *Downloader) DownloadFileWithContext(ctx context.Context, key, path string) (f *os.File, err error)

func (*Downloader) DownloadRangeBytes

func (d *Downloader) DownloadRangeBytes(key string, offset, size int64) (int64, []byte, error)

func (*Downloader) DownloadRangeBytesWithContext

func (d *Downloader) DownloadRangeBytesWithContext(ctx context.Context, key string, offset, size int64) (l int64, data []byte, err error)

func (*Downloader) DownloadReader

func (d *Downloader) DownloadReader(key string) (io.ReadCloser, error)

func (*Downloader) DownloadReaderWithContext

func (d *Downloader) DownloadReaderWithContext(ctx context.Context, key string) (r io.ReadCloser, err error)

type File

type File interface {
	io.Closer
	io.Reader
	io.Seeker
	Readdir(count int) ([]os.FileInfo, error)
	Stat() (os.FileInfo, error)
}

A File is returned by a FileSystem's Open method and can be served by the FileServer implementation.

The methods should behave the same as those on an *os.File.

type FileStat

type FileStat struct {
	Name string `json:"name"`
	Size int64  `json:"size"`
}

type FileSystem

type FileSystem interface {
	Open(name string) (File, error)
}

A FileSystem implements access to a collection of named files. The elements in a file path are separated by slash ('/', U+002F) characters, regardless of host operating system convention.

type HostSelector

type HostSelector struct {
	// contains filtered or unexported fields
}

func NewHostSelector

func NewHostSelector(hosts []string, update func() []string, updateDuration, punishDuration time.Duration,
	maxPunishedTimes, maxPunishedHostsPrecent int, shouldPunish func(error) bool) *HostSelector

func (*HostSelector) Punish

func (hostSelector *HostSelector) Punish(host string)

func (*HostSelector) PunishIfNeeded

func (hostSelector *HostSelector) PunishIfNeeded(host string, err error)

func (*HostSelector) Reward

func (hostSelector *HostSelector) Reward(host string)

func (*HostSelector) SelectHost

func (hostSelector *HostSelector) SelectHost() string

type Lister

type Lister struct {
	// contains filtered or unexported fields
}

func NewLister

func NewLister(c *Config) *Lister

func NewListerV2

func NewListerV2() *Lister

func (*Lister) Copy

func (l *Lister) Copy(fromKey, toKey string) error

func (*Lister) CopyWithContext

func (l *Lister) CopyWithContext(ctx context.Context, fromKey, toKey string) (err error)

func (*Lister) Delete

func (l *Lister) Delete(key string) error

func (*Lister) DeleteWithContext

func (l *Lister) DeleteWithContext(ctx context.Context, key string) (err error)

func (*Lister) ListPrefix

func (l *Lister) ListPrefix(prefix string) []string

func (*Lister) ListPrefixWithContext

func (l *Lister) ListPrefixWithContext(ctx context.Context, prefix string) (keys []string)

func (*Lister) ListPrefixWithMarker

func (l *Lister) ListPrefixWithMarker(prefix, marker string, limit int) ([]kodo.ListItem, string, error)

func (*Lister) ListPrefixWithMarkerAndContext

func (l *Lister) ListPrefixWithMarkerAndContext(ctx context.Context, prefix, marker string, limit int) (entries []kodo.ListItem, nextContinuousToken string, err error)

func (*Lister) ListStat

func (l *Lister) ListStat(paths []string) []*FileStat

func (*Lister) ListStatWithContext

func (l *Lister) ListStatWithContext(ctx context.Context, paths []string) (stats []*FileStat, anyError error)

func (*Lister) MoveTo

func (l *Lister) MoveTo(fromKey, toBucket, toKey string) error

func (*Lister) MoveToWithContext

func (l *Lister) MoveToWithContext(ctx context.Context, fromKey, toBucket, toKey string) (err error)

func (*Lister) Rename

func (l *Lister) Rename(fromKey, toKey string) error

func (*Lister) RenameWithContext

func (l *Lister) RenameWithContext(ctx context.Context, fromKey, toKey string) (err error)

func (*Lister) Stat

func (l *Lister) Stat(key string) (kodo.Entry, error)

func (*Lister) StatWithContext

func (l *Lister) StatWithContext(ctx context.Context, key string) (entry kodo.Entry, err error)

type Queryer

type Queryer struct {
	// contains filtered or unexported fields
}

func NewQueryer

func NewQueryer(c *Config) *Queryer

func (*Queryer) QueryIoHosts

func (queryer *Queryer) QueryIoHosts(https bool) (urls []string)

func (*Queryer) QueryRsHosts

func (queryer *Queryer) QueryRsHosts(https bool) (urls []string)

func (*Queryer) QueryRsfHosts

func (queryer *Queryer) QueryRsfHosts(https bool) (urls []string)

func (*Queryer) QueryUpHosts

func (queryer *Queryer) QueryUpHosts(https bool) (urls []string)

type Req

type Req struct {
	Path   string `json:"path"`
	Key    string `json:"key"`
	Delete *bool  `json:"del"`
}

type Uploader

type Uploader struct {
	// contains filtered or unexported fields
}

func NewUploader

func NewUploader(c *Config) *Uploader

func NewUploaderV2

func NewUploaderV2() *Uploader

func (*Uploader) Upload

func (p *Uploader) Upload(file string, key string) error

func (*Uploader) UploadData

func (p *Uploader) UploadData(data []byte, key string) error

func (*Uploader) UploadDataReader

func (p *Uploader) UploadDataReader(data io.ReaderAt, size int64, key string) error

func (*Uploader) UploadDataReaderAt

func (p *Uploader) UploadDataReaderAt(data io.ReaderAt, size int64, key string) error

func (*Uploader) UploadDataReaderAtWithContext

func (p *Uploader) UploadDataReaderAtWithContext(ctx context.Context, data io.ReaderAt, size int64, key string, ret interface{}) error

func (*Uploader) UploadDataReaderWithContext

func (p *Uploader) UploadDataReaderWithContext(ctx context.Context, data io.ReaderAt, size int64, key string, ret interface{}) error

func (*Uploader) UploadDataWithContext

func (p *Uploader) UploadDataWithContext(ctx context.Context, data []byte, key string, ret interface{}) error

func (*Uploader) UploadReader

func (p *Uploader) UploadReader(reader io.Reader, key string) error

func (*Uploader) UploadReaderWithContext

func (p *Uploader) UploadReaderWithContext(ctx context.Context, reader io.Reader, key string, ret interface{}) error

func (*Uploader) UploadWithContext

func (p *Uploader) UploadWithContext(ctx context.Context, file string, key string, ret interface{}) error

func (*Uploader) UploadWithDataChan

func (p *Uploader) UploadWithDataChan(key string, dataCh chan q.PartData, ret interface{}, initNotify func(suggestedPartSize int64)) error

func (*Uploader) UploadWithDataChanWithContext

func (p *Uploader) UploadWithDataChanWithContext(ctx context.Context, key string, dataCh chan q.PartData, ret interface{}, initNotify func(suggestedPartSize int64)) error

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL