README
¶
Pelican Command Line
)
)
)
The Pelican command line tool allows one to use a Pelican federation as a client and serve datasets through running a Pelican origin service.
For more information on Pelican, see the Pelican Platform page.
For documentation on using the Pelican Platform, see the Pelican Platform documentation page.
Testing and Usage
Run the following command to download a test file (/test/query1) from the configured federation:
$ pelican object get /test/query1 ./
Running an Pelican origin
To launch a Pelican origin, run the following:
pelican-server origin serve -f https://director.example.com -v /tmp/stash/:/test
Running an OSDF origin or client
The Open Science Data Federation (OSDF) is a well-known Pelican federation in support of the science and engineering communities in the US.
To launch an origin using OSDF defaults, rename the output binary from pelican to osdf. Then, run:
osdf origin serve -v /tmp/stash/:/test
Similarly, the osdf binary can be used to download from the OSDF:
$ osdf object get /osgconnect/public/dweitzel/blast/queries/query1 ./
To ease the transition of stashcp users to pelican, the tool can also be renamed or symlinked to stashcp:
$ stashcp /osgconnect/public/dweitzel/blast/queries/query1 ./
and it shares the same defaults and behavior as stashcp.
Building
Building is performed with the goreleaser tool. To build a snapshot (not release):
$ make goreleaser-config
$ goreleaser --clean --snapshot --config .goreleaser.generated.yml
The binaries will be located in ./dist directory.
Building for Development
To enable more rapid development you can include a dev goreleaser configuration. This allows you to only build the binaries that you will be using.
Example .goreleaser.dev.yml File
project_name: pelican
version: 2
release:
prerelease: true
before:
hooks:
- go mod tidy
- go generate ./...
- make web-build
builds:
- env:
- CGO_ENABLED=0
goos:
- linux
- darwin
goarch:
- "amd64"
- "arm64"
id: "pelican"
dir: ./cmd
binary: pelican
tags:
- forceposix
ldflags:
- -s -w -X github.com/pelicanplatform/pelican/version.commit={{.Commit}} -X github.com/pelicanplatform/pelican/version.date={{.Date}} -X github.com/pelicanplatform/pelican/version.builtBy=goreleaser -X github.com/pelicanplatform/pelican/version.version={{.Version}}
ignore:
- goos: windows
goarch: arm64
- goos: windows
goarch: ppc64le
- goos: darwin
goarch: ppc64le
Build using the dev file
make pelican-dev-build
Directories
¶
| Path | Synopsis |
|---|---|
|
Package backup_keys derives backup encryption keys from a server's issuer keys.
|
Package backup_keys derives backup encryption keys from a server's issuer keys. |
|
cmd
|
|
|
sample_metadata_server
command
Command sample_metadata_server is a reference receiver for the V2 origin's object-commit metadata webhook (see docs/v2-origin-posc-and-metadata.md).
|
Command sample_metadata_server is a reference receiver for the V2 origin's object-commit metadata webhook (see docs/v2-origin-posc-and-metadata.md). |
|
Package launcher_utils contains utility functions for the github.com/pelicanplatform/pelican/launcher package.
|
Package launcher_utils contains utility functions for the github.com/pelicanplatform/pelican/launcher package. |
|
persistent_cache.go implements PersistentCache, the main entry point for the local-cache subsystem.
|
persistent_cache.go implements PersistentCache, the main entry point for the local-cache subsystem. |
|
The LotMan library is used for managing storage in Pelican caches.
|
The LotMan library is used for managing storage in Pelican caches. |
|
issuer
Package issuer provides an embedded OAuth2/OIDC token issuer using the fosite framework.
|
Package issuer provides an embedded OAuth2/OIDC token issuer using the fosite framework. |
|
Package origin_serve provides checksum computation and caching for origin servers.
|
Package origin_serve provides checksum computation and caching for origin servers. |
|
Package pstore ("pelican store") uses the cache V2 block store as an origin's primary data store rather than as a cache.
|
Package pstore ("pelican store") uses the cache V2 block store as an origin's primary data store rather than as a cache. |
|
Package registry handles namespace registration in Pelican ecosystem.
|
Package registry handles namespace registration in Pelican ecosystem. |
|
Package server_structs shares structs and their methods used across multiple server packages (origin/cache/registry/director).
|
Package server_structs shares structs and their methods used across multiple server packages (origin/cache/registry/director). |
|
Package server_utils shares utility functions used across multiple server packages (origin, cache, registry, director).
|
Package server_utils shares utility functions used across multiple server packages (origin, cache, registry, director). |
|
Package ssh_posixv2 implements an SSH-based POSIXv2 backend for Pelican origins.
|
Package ssh_posixv2 implements an SSH-based POSIXv2 backend for Pelican origins. |
|
Package transfer implements the transfer server module which exposes the client_agent transfer APIs over HTTP with JWT-based authentication, credential management, and per-user job ownership.
|
Package transfer implements the transfer server module which exposes the client_agent transfer APIs over HTTP with JWT-based authentication, credential management, and per-user job ownership. |
|
Provide auto-generated information about the pelican version and build
|
Provide auto-generated information about the pelican version and build |
|
web_ui/middleware.go
|
web_ui/middleware.go |
|
**************************************************************
|
************************************************************** |