garf

package module
v0.5.6 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Dec 27, 2025 License: GPL-3.0 Imports: 12 Imported by: 0

README

garf

garf is a tool for mirroring artifacts from public repositories to JFrog Artifactory with structured path organization and metadata handling.

Overview

This CLI tool simplifies the artifact mirroring workflow by:

  • Downloading artifacts from sources like GitHub Releases
  • Extracting coordinates (org, repo, version, etc.) from URLs
  • Uploading to JFrog Artifactory with organized path structures
  • Supporting metadata through custom properties
  • Extracting files from single-file zip archives when needed

Quick Start

Prerequisites
  • Bazel for building and running
  • JFrog Artifactory instance credentials
Environment Setup
JFrog Artifactory
# Linux/macOS
export JFROG_URL="https://your-instance.jfrog.io/artifactory"
export JFROG_USER="username"
export JFROG_PASSWORD="password"

# Windows
set JFROG_URL=https://your-instance.jfrog.io/artifactory
set JFROG_USER=username
set JFROG_PASSWORD=password
Cloudsmith
# Linux/macOS
export REGISTRY_TYPE="cloudsmith"
export REGISTRY_URL="https://api.cloudsmith.io"
export REGISTRY_USER="your-username"
export REGISTRY_PASSWORD="your-api-key"

# Windows
set REGISTRY_TYPE=cloudsmith
set REGISTRY_URL=https://api.cloudsmith.io
set REGISTRY_USER=your-username
set REGISTRY_PASSWORD=your-api-key
Simple Examples
JFrog Artifactory
bazel run //:garf -- mirror \
  --source https://github.com/bazelbuild/bazel/releases/download/7.2.1/bazel-7.2.1-windows-x86_64.exe \
  --destination tools-local
Cloudsmith
bazel run //:garf -- mirror \
  --registry-type cloudsmith \
  --source https://github.com/bazelbuild/bazel/releases/download/7.2.1/bazel-7.2.1-windows-x86_64.exe \
  --destination owner/repo

Usage Examples

JFrog Artifactory Examples
Mirror GitHub Release Asset
bazel run //:garf -- mirror \
  --source https://github.com/bazelbuild/bazel/releases/download/7.2.1/bazel_nojdk-7.2.1-windows-x86_64.exe \
  --destination tools-local
Add Metadata Properties
bazel run //:garf -- mirror \
  --source https://github.com/bazelbuild/bazel/releases/download/7.2.1/bazel_nojdk-7.2.1-windows-x86_64.exe \
  --destination tools-local \
  --properties type=toolchain \
  --properties platform=windows \
  --properties version=7.2.1
Use Raw URL Path Structure
bazel run //:garf -- mirror \
  --source https://github.com/bazelbuild/bazel/releases/download/7.2.1/bazel_nojdk-7.2.1-windows-x86_64.exe \
  --destination tools-local \
  --raw
Upload From Local File
bazel run //:garf -- mirror \
  --source https://github.com/bazelbuild/bazel/releases/download/7.2.1/bazel_nojdk-7.2.1-windows-x86_64.exe \
  --from-file /path/to/bazel_nojdk-7.2.1-windows-x86_64.exe \
  --destination tools-local
Extract and Upload From Zip

For zip files containing a single file (like executables packaged as zip):

bazel run //:garf -- mirror \
  --source https://github.com/bazelbuild/bazel/releases/download/7.6.0/bazel_nojdk-7.6.0-windows-x86_64.zip \
  --destination tools-local \
  --unzip

This will extract the file (e.g., bazel_nojdk-7.6.0-windows-x86_64.exe) from the zip and upload it directly.

Cloudsmith Examples
Basic Mirror to Cloudsmith
bazel run //:garf -- mirror \
  --registry-type cloudsmith \
  --source https://github.com/bazelbuild/bazel/releases/download/7.2.1/bazel-7.2.1-linux-x86_64 \
  --destination myorg/tools
Mirror with Custom Description
bazel run //:garf -- mirror \
  --registry-type cloudsmith \
  --source https://github.com/docker/compose/releases/download/v2.21.0/docker-compose-linux-x86_64 \
  --destination myorg/docker-tools \
  --summary "Docker Compose v2.21.0 for Linux x86_64"
Upload From Local File to Cloudsmith
bazel run //:garf -- mirror \
  --registry-type cloudsmith \
  --source https://github.com/kubernetes/kubernetes/releases/download/v1.28.2/kubectl \
  --from-file /path/to/kubectl \
  --destination myorg/k8s-tools
Extract Zip and Upload to Cloudsmith
bazel run //:garf -- mirror \
  --registry-type cloudsmith \
  --source https://github.com/mikefarah/yq/releases/download/v4.35.2/yq_windows_amd64.zip \
  --destination myorg/tools \
  --unzip
Dry Run with Cloudsmith
bazel run //:garf -- mirror \
  --registry-type cloudsmith \
  --source https://github.com/hashicorp/terraform/releases/download/v1.5.7/terraform_1.5.7_linux_amd64.zip \
  --destination myorg/hashicorp-tools \
  --dry-run
JFrog-to-JFrog Mirroring with Source Path Stripping

For mirroring artifacts from one JFrog repository to another (e.g., staging to production), you can strip source path prefixes to avoid nested repository structures:

# Mirror from staging to production, stripping the staging prefix
bazel run //:garf -- mirror \
  --source https://artifactory.corp.net/staging/github.com/bazelbuild/bazel/releases/download/7.2.1/bazel-win.exe \
  --destination prod-repo \
  --source-path-strip "artifactory.corp.net/staging/"

This will:

  • Strip artifactory.corp.net/staging/ from the source URL
  • Process the remaining URL (github.com/bazelbuild/bazel/releases/download/7.2.1/bazel-win.exe) normally
  • Upload to prod-repo/github.com/bazelbuild/bazel/7.2.1/bazel-win.exe

Without source path stripping, the result would be: prod-repo/artifactory.corp.net/staging/github.com/bazelbuild/bazel/releases/download/7.2.1/bazel-win.exe

Common Use Cases

Strip staging repository prefix:

--source-path-strip "artifactory.corp.net/staging/"

Strip entire host:

--source-path-strip "artifactory.corp.net"

Strip custom path prefix:

--source-path-strip "my-company.jfrog.io/temp-repo/"
Dry Run Mode

The mirror command supports dry run modes to help with testing and validation without making actual changes to Artifactory. There are two dry run modes available:

Skip All Operations

This mode validates the command syntax and configuration without performing any actual operations:

bazel run //:garf -- mirror \
  --source https://github.com/bazelbuild/bazel/releases/download/7.6.0/bazel_nojdk-7.6.0-windows-x86_64.zip \
  --destination tools-local \
  --dry-run

This is useful for:

  • Testing command syntax
  • Validating configuration
  • Checking environment variables
  • Verifying artifact URLs
Skip Only Upload

This mode downloads and processes artifacts but skips uploading to Artifactory:

bazel run //:garf -- mirror \
  --source https://github.com/bazelbuild/bazel/releases/download/7.6.0/bazel_nojdk-7.6.0-windows-x86_64.zip \
  --destination tools-local \
  --dry-run \
  --dry-run-mode upload

This is useful for:

  • Local development without Artifactory access
  • Testing artifact processing without uploading
  • Validating download and processing logic
  • CI/CD environments where Artifactory is not available

Command Reference

Mirror Command
garf mirror [OPTIONS]
Options
Option Short Description
--destination -d Repository destination: JFrog repo name or Cloudsmith owner/repo (required)
--from-file -f Local file path to upload (URL still used for coordinates)
--properties Add properties to the artifact (can be used multiple times, JFrog only)
--raw Use the full URL path structure rather than parsed coordinates
--registry-type Registry type: 'jfrog' (default) or 'cloudsmith'
--source -s URL of the artifact to mirror (required)
--source-path-strip Strip source path prefixes for JFrog-to-JFrog mirroring
--summary Package summary/description (Cloudsmith only)
--unzip Extract and upload the content from zip files with a single file
--dry-run Perform a dry run without making actual changes
--dry-run-mode Dry run mode: 'all' (skip all operations), 'upload' (skip only upload)

Path Organization

Standard Path Structure (Default)
{destination}/{host}/{org}/{repo}/{version}/{artifact}

Example:

tools-local/github.com/bazelbuild/bazel/7.2.1/bazel_nojdk-7.2.1-windows-x86_64.exe
Raw Path Structure (with --raw flag)
{destination}/{host}/{full_url_path}

Example:

tools-local/github.com/bazelbuild/bazel/releases/download/7.2.1/bazel_nojdk-7.2.1-windows-x86_64.exe

Environment Variables

JFrog Artifactory
Variable Description
JFROG_URL URL of the JFrog Artifactory instance (required)
JFROG_USER Username for authentication (required)
JFROG_PASSWORD Password for authentication (required)
Cloudsmith
Variable Description
REGISTRY_TYPE Set to "cloudsmith" to use Cloudsmith (required)
REGISTRY_URL Cloudsmith API URL (default: https://api.cloudsmith.io)
REGISTRY_USER Cloudsmith username (required)
REGISTRY_PASSWORD Cloudsmith API key (required)

Building From Source

# Clone repository
git clone https://github.com/albertocavalcante/garf.git
cd garf

# Build the binary
bazel build //:garf

Documentation

Overview

Package garf provides a high-level API for mirroring artifacts from sources like GitHub to destinations like JFrog Artifactory.

This package offers a simple, stable API for Go programs that want to mirror artifacts programmatically. It handles authentication, URL processing, and artifact uploading automatically.

Basic usage: 

client, err := garf.NewClient(garf.Config{
	JFrogURL:      "https://mycompany.jfrog.io/artifactory",
	JFrogUser:     "username",
	JFrogPassword: "password",
})
if err != nil {
	log.Fatal(err)
}

result, err := client.Mirror(ctx, garf.MirrorRequest{
	Source:      "https://github.com/owner/repo/releases/download/v1.0.0/artifact.zip",
	Destination: "my-generic-repo",
	Properties:  map[string]string{"type": "binary", "platform": "linux"},
	Unzip:       true, // Extract single files from zip archives
})
if err != nil {
	log.Fatal(err)
}

For JFrog-to-JFrog mirroring with source path stripping: 

result, err := client.Mirror(ctx, garf.MirrorRequest{
	Source:          "https://artifactory.corp.net/staging/github.com/owner/repo/releases/download/v1.0.0/artifact.zip",
	Destination:     "prod-repo",
	SourcePathStrip: "artifactory.corp.net/staging/", // Strip staging prefix
	Properties:      map[string]string{"type": "binary"},
})
if err != nil {
	log.Fatal(err)
}

Index

Constants

View Source 
const (
	// DefaultTimeout is the default timeout for mirror operations.
	DefaultTimeout = 30 * time.Minute
	// DefaultConcurrent is the default number of concurrent operations.
	DefaultConcurrent = 4
	// UnknownArtifactName is returned when artifact name cannot be determined.
	UnknownArtifactName = "unknown"
)

Variables

This section is empty.

Functions

func ExtractArtifactName 

func ExtractArtifactName(urlStr string) string

ExtractArtifactName extracts the artifact name from a URL.

func ValidateConfig 

func ValidateConfig(config Config) error

ValidateConfig validates the client configuration.

Types

type Client 

type Client struct {
	Config Config
	// contains filtered or unexported fields
}

Client provides the main interface for mirroring artifacts.

func NewClient 

func NewClient(config Config) (*Client, error)

NewClient creates a new garf client with the provided configuration.

func (*Client) DetectSourceType added in v0.5.0 

func (c *Client) DetectSourceType(sourceURL, sourcePathStrip string) string

DetectSourceType determines the source type based on the URL. When SourcePathStrip is provided, it strips the prefix first to determine the actual source.

func (*Client) EnsureSourceAvailable added in v0.5.0 

func (c *Client) EnsureSourceAvailable(sourceType string) error

EnsureSourceAvailable ensures that the appropriate source is available in the mirror.

func (*Client) GetCachedDestinationsCount 

func (c *Client) GetCachedDestinationsCount() int

GetCachedDestinationsCount returns the number of cached destinations (for testing).

func (*Client) IsCachedDestination 

func (c *Client) IsCachedDestination(key string) bool

IsCachedDestination checks if a destination is cached (for testing).

func (*Client) Mirror 

func (c *Client) Mirror(ctx context.Context, request MirrorRequest) (*MirrorResult, error)

Mirror performs a single mirror operation.

func (*Client) ValidateRequest 

func (c *Client) ValidateRequest(request MirrorRequest) error

ValidateRequest validates a mirror request.

type Config 

type Config struct {
	// Generic registry credentials (PREFERRED - supports both JFrog and Cloudsmith)
	RegistryURL      string // Replaces JFrogURL but supports both
	RegistryUser     string // Replaces JFrogUser but supports both
	RegistryPassword string // Replaces JFrogPassword but supports both

	// Registry type (defaults to "jfrog" for backward compatibility)
	RegistryType string

	// Deprecated: JFrog-specific fields (kept for backward compatibility)
	// These will be mapped to Registry* fields if Registry* fields are empty
	JFrogURL      string
	JFrogUser     string
	JFrogPassword string

	// Optional: Custom logger (if nil, a default logger will be used)
	Logger *logrus.Logger

	// Optional: Request timeout (default: 30 minutes)
	Timeout time.Duration

	// Optional: Number of concurrent operations (default: 4)
	Concurrent int
}

Config holds the configuration for the garf client.

func (Config) String 

func (c Config) String() string

String implements fmt.Stringer to prevent accidental credential logging.

type MirrorRequest 

type MirrorRequest struct {
	// Source URL of the artifact to mirror
	Source string

	// Destination repository name in JFrog Artifactory
	Destination string

	// Optional: Properties to attach to the artifact
	Properties map[string]string

	// Optional: Whether to preserve the original URL structure (default: false)
	Raw bool

	// Optional: Whether to extract single files from zip archives (default: false)
	Unzip bool

	// Optional: Local file path to upload instead of downloading from source
	// The source URL is still used for coordinate extraction
	// TODO: This feature is not yet implemented
	FromFile string

	// Optional: Dry run mode - "all" skips everything, "upload" skips only upload
	DryRun     bool
	DryRunMode string

	// Optional: Source path prefix to strip from source URLs before processing.
	// This is useful for JFrog-to-JFrog mirroring where you want to remove the source
	// repository path. For example, setting this to "artifactory.corp.net/staging/"
	// will strip that prefix from source URLs before generating the destination path.
	// This enables clean mirroring from staging to production repositories.
	SourcePathStrip string
}

MirrorRequest represents a single mirror operation request.

type MirrorResult 

type MirrorResult struct {
	// Source URL that was mirrored
	Source string

	// Destination path where the artifact was stored
	DestinationPath string

	// Error if the operation failed (nil on success)
	Error error
}

MirrorResult represents the result of a mirror operation.

Source Files

View all Source files

Directories

Path Synopsis
cmd
garf command
Package main demonstrates how to use garf as a library.
 Package main demonstrates how to use garf as a library.
pkg
core
Package core provides core types and constants for the garf application.
 Package core provides core types and constants for the garf application.
core/config
Package config provides configuration management functionality.
 Package config provides configuration management functionality.
destinations
Package destinations provides implementations for various registry destinations.
 Package destinations provides implementations for various registry destinations.
io
io/archive
Package archive provides functionality for handling various archive formats.
 Package archive provides functionality for handling various archive formats.
io/progress
Package progress provides utilities for tracking progress of operations.
 Package progress provides utilities for tracking progress of operations.
mirror
Package mirror provides the core interfaces and types for artifact mirroring.
 Package mirror provides the core interfaces and types for artifact mirroring.
pointer
processor
Package processor provides functionality for processing artifacts.
 Package processor provides functionality for processing artifacts.
sources
Package sources provides implementations of the core.Source interface.
 Package sources provides implementations of the core.Source interface.
testutil
Package testutil provides shared testing utilities for the garf project.
 Package testutil provides shared testing utilities for the garf project.
urlprocessor
Package urlprocessor provides interfaces and implementations for processing URLs to create structured paths for artifacts.
 Package urlprocessor provides interfaces and implementations for processing URLs to create structured paths for artifacts.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.