godyl

godyl

Asset downloader

---

godyl helps with batch-downloading and installing statically compiled binaries from:

• GitHub releases
• URLs
• Go projects

As an alternative to above, custom commands can be used as well.

godyl will infer the platform and architecture from the system it is running on, and will attempt to download the appropriate binary.

This uses simple heuristics to select the correct binary to download, and will not work for all projects.

However, most properties can be overridden, with hints and skip used to help godyl make the correct decision.

[!WARNING] This repo is a work in progress! Needing both cleaning up and documenting.

[!CAUTION] This project serves as a learning exercise for Go and its surrounding ecosystem and tooling. As such, it might be of limited use for others.

[!NOTE] Tested on:

Linux: amd64, arm64

Windows: amd64

MacOS: amd64, arm64

for tools listed in tools.yml

Tool is inspired by task, dra and ansible

---

Table of Contents

• Installation
• Configuration
• Tools
  • Simple form
  • Full form
• Defaults
• Template overview
  • Variables
  • Allowed in
• Inference
  • Operating Systems
  • Architectures
  • Libraries
• Notes

---

Installation

From source

go install github.com/idelchi/godyl@latest

From installation script

curl -sSL https://raw.githubusercontent.com/idelchi/godyl/refs/heads/dev/install.sh | sh -s -- -d ~/.local/bin

Run

curl -sSL https://raw.githubusercontent.com/idelchi/godyl/refs/heads/dev/install.sh | sh -s -- -h

for all available options.

Run

curl -sSL https://raw.githubusercontent.com/idelchi/godyl/refs/heads/dev/scripts/tools.sh | sh -s -- -d ~/.local/bin

to install all tools listed in the tools.yml file.

Usage

godyl [command] [flags]

Available commands:

• install - Install tools from a YAML file
• download - Download and unpack individual tools
• dump - Display configuration information
• update - Update the godyl application

Install Command

Install tools defined in a YAML configuration file:

godyl install [tools.yml] --output ./bin

If no file is specified, godyl defaults to using tools.yml in the current directory.

You can dump out a default tools.yml file by running:

godyl dump tools > tools.yml

Download Command

Download a single tool:

godyl download idelchi/godyl --output ./bin

Download multiple tools:

godyl download idelchi/godyl idelchi/gogen idelchi/wslint

When using the download command, the mode will be set to extract by default.

Override os and arch to download a specific binary:

godyl download idelchi/godyl --os linux --arch amd64 --output ./bin

You can also download tools from direct URLs:

godyl download "https://github.com/idelchi/go-next-tag/releases/download/v0.0.1/go-next-tag_{{ .OS }}_{{ .ARCH }}.tar.gz" --output ./bin

Dump Command

Display various configuration settings and information:

godyl dump [config|defaults|env|platform|tools]

Subcommands:

• config - Display the current configuration settings
• defaults - Display the default configuration settings
• env - Display environment variables that affect the application
• platform - Display information about the current platform
• tools - Display information about available tools

Update Command

Update the godyl application to the latest version:

godyl update

[!NOTE] Set up a GitHub API token to avoid rate limiting when using github as a source type. See configuration for more information, or simply export GODYL_GITHUB_TOKEN=<token>

Configuration

The tools can be configured (in order of priority) by:

• flags
• environment variables
• .env file

Global Flags

The following global flags are available for all commands:

Flag	Environment Variable	Default	Description
--help, -h	GODYL_HELP	false	Show help message and exit
--version	GODYL_VERSION	false	Show version information and exit
--dry	GODYL_DRY	false	Run without making any changes (dry run)
--log	GODYL_LOG	info	Log level (debug, info, warn, error, silent)
--parallel, -j	GODYL_PARALLEL	0	Number of parallel downloads (0 is unlimited)
--no-verify-ssl, -k	GODYL_NO_VERIFY_SSL	false	Skip SSL verification
--env-file	GODYL_DOT_ENV	.env	Path to .env file
--defaults, -d	GODYL_DEFAULTS	defaults.yml	Path to defaults file
--show, -s	GODYL_SHOW	false	Show the configuration and exit

Tool-specific Flags

The following flags are available for tool-related commands (install and download):

Flag	Environment Variable	Default	Description
--output, -o	GODYL_OUTPUT	./bin	Output path for the downloaded tools
--tags, -t	GODYL_TAGS	["!native"]	Tags to filter tools by. Use ! to exclude
--source	GODYL_SOURCE	github	Source from which to install the tools
--strategy	GODYL_STRATEGY	none	Strategy to use for updating tools
--os	GODYL_OS	""	Operating system to use for downloading
--arch	GODYL_ARCH	""	Architecture to use for downloading
--github-token	GODYL_GITHUB_TOKEN	""	GitHub token for authentication

For the install command, the path to the file containing the tool installation instructions is provided as a positional argument, defaulting to tools.yml.

An example tools.yml is provided.

In general, settings can be set in the following ways (order of priority):

• as a field in the tools.yml definition

  output: ~/.local/bin
  

• as a flag to the tool

  godyl --output ~/.local/bin
  

• as an environment variable

  GODYL_OUTPUT=~/.local/bin godyl
  

• in an .env file

  GODYL_OUTPUT=~/.local/bin
  

• by setting the value in a defaults.yml file (see defaults)

  output: ~/.local/bin
  

If none of the above are fulfilled, the default configuration embedded from defaults.yml will be used.

Tools

A YAML file controls the tools to download and install. Alternatively, if the positional argument to the tool is not a YAML file, it will be treated as a single tool name or URL.

Examples are provided in tools.yml.

Simple form

- idelchi/godyl

Above is the simple form to attempt to download the latest release of godyl from idelchi/godyl.

If it is a URL, it will be considered as a source.url type. Otherwise, it will be assumed to be a source.github type on the form owner/repo.

Full form

name: string
description: string
version:
  version: string
  commands: []
  patterns: []
path: string
output: string
exe:
  name: string
  patterns:
    - regex
platform:
  os: string
  architecture:
    type: string
    version: string
  library: string
  extension: string
  distribution: string
aliases: []
values: {}
fallbacks: []
hints:
  - pattern: regex
    weight: string
    must: boolean
source:
  type: string
  github:
    repo: string
    owner: string
    token: string
  url:
    url: string
    token: string
  go:
    command: string
  commands: []
tags:
  - string
strategy: string
extensions:
  - string
skip:
  - condition: string
    reason: string
post: []
mode: string
env:
  key: string

Any field that accepts a list, can also be provided as a string.

For example:

aliases: gd
fallbacks: go
exe:
  patterns: "{{ .Exe }}{{ .EXTENSION }}$"

is equivalent to:

aliases:
  - gd
fallbacks:
  - go
exe:
  patterns:
    - "{{ .Exe }}{{ .EXTENSION }}$"

and

skip:
  reason: "tool is not available on windows"
  condition: '{{ eq .OS "windows" }}'

is equivalent to:

skip:
  - reason: "tool is not available on windows"
    condition: '{{ eq .OS "windows" }}'

Name

Template	Templated	As Template
{{ .Name }}

name is the name of the tool to download.

Usage

• Used as display name and for inferring other fields (see GitHub and exe)

Description

Template	Templated	As Template

description is an optional description of the tool, for documentation purposes.

Version

Template	Templated	As Template
{{ .Version }}

Usage

• As template for other fields

• To compare tool versions if strategy is set to upgrade

• Will be inferred and populated by the source method if not given

• version.version is the version of the tool to download

• version.commands is a list of commands to run to get the version of the tool (for upgrades)

• version.patterns is a list of patterns to use for finding the version of the tool (for upgrades)

• version.commands & version.patterns set according to defaults if not given

Set version.commands to [] if version parsing is not available. This will result in always downloading the tool when strategy is set to upgrade.

Pinning to a specific version will prevent upgrades.

Alternative form

version: v0.1.0

is equivalent to:

version:
  version: v0.1.0

Path

Template	Templated	As Template

path is the path to the tool to download. Currently only supports URLs.

Usage

• Will be inferred and populated by the source method if not given

Output

Template	Templated	As Template

output is the path to the directory where the tool will be installed.

Usage

• Set according to flags and environment variables or defaults if not given

Exe

Template	Templated	As Template
{{ .Exe}}
{{ .Exe.Patterns }}

exe is a dictionary containing the name of the executable and patterns to use for finding the executable.

Usage

• exe.name is the name of the executable, inferred from name if not given
• exe.patterns is a list of patterns to use for finding the executable, inferred from name if not given
• Set according to defaults if not given

Alternative form

exe: godyl

is equivalent to:

exe:
  name: godyl

Platform

Template	Templated	As Template
{{ .Platform.<> }}		See variables

platform is a dictionary containing the platform and architecture information.

Usage

• Any field not given will be inferred from the system
• Set according to defaults if not given

Aliases

Template	Templated	As Template

aliases is a list of aliases for the tool. Will be used to create symlinks (or copies on Windows) for the tool.

Values

Template	Templated	As Template
{{ .Values.<> }}

values is an arbitrary values map, which can be used for templating in other fields.

Fallbacks

Template	Templated	As Template

fallbacks is a list of fallback strategies to use if the tool cannot be found.

They will be tried in order until the tool is found or all have been tried.

Hints

Template	Templated	As Template
{{ .Hints.Weight }}
{{ .Hints.Pattern }}
{{ .Hints.Regex }}
{{ .Hints.Must }}

hints is a list of hints for matching, which can be used to help godyl find the correct tool.

Usage

• Set according to defaults if not given

Source

Type

Template	Templated	As Template

source.type is the source type. Accepted values are:

• github
• url
• go
• commands

GitHub

Template	Templated	As Template

source.github is a dictionary containing the owner, repository and token of the tool.

Usage

• repo and owner will be inferred from name if not given, or set according to defaults (not recommended)
• token will be set according to flags and environment variables or defaults if not given

URL

Template	Templated	As Template

source.url is a dictionary containing the URL and token of the tool.

Usage

• token will be set according to flags and environment variables if not given

Go

Template	Templated	As Template

source.go can be set to the relative path of the go command to download, if non-standard (i.e not matching <name>, cmd/<name> or cmd).

[!WARNING] Go will be downloaded into a temporary directory /tmp/.godyl-go if not present.

Usage

Commands

Template	Templated	As Template

commands is a list of commands to run to install the tool.

Tags

Template	Templated	As Template

tags is a list of tags to filter tools by.

Usage

• the name of the tool will always be added as a tag

Strategy

Template	Templated	As Template

strategy is the strategy to use for updating the tool.

Accepted values are:

• none
• upgrade
• force

Usage

• Set according to flags and environment variables or defaults if not given
• none will skip the tool if it already exists
• upgrade will attempt to parse the version of an existing tool and upgrade if necessary
• force will always download and install the tool

Extensions

Template	Templated	As Template

extensions is a list of extensions to consider when matching the most suitable tool to download (used only for github source type).

Usage

• Set according to defaults if not given
• Can be used to for example prefer .zip files for Windows

Skip

Template	Templated	As Template

skip is a list of conditions under which the tool should be skipped.

Usage

• reason is a description of why the tool was skipped
• condition is a condition to check, and can use templating

Alternative form

skip: <condition>

is equivalent to:

skip:
  - condition: <condition>

Post

Template	Templated	As Template

post is a list of commands to run after the tool has been installed.

Mode

Template	Templated	As Template

mode is the mode to use for the tool.

Accepted values are:

• find
• extract

Usage

• find will download, extract and find the executable
• extract will download the tool and extract it directly to the output directory
• Set according to flags and environment variables or defaults if not given
• Automatically set to extract if the tool is used without tools.yml (e.g. godyl idelchi/godyl)

Defaults

A default configuration may be used to specify default settings for all tools. These will override (or extend in some case) the settings for each tool.

The following is embedded and used by default if no default configuration is provided:

defaults.yml

The example above defines:

• The default output directory for all tools (~/.local/bin)

• Patterns to use for when searching for the executable

  • ^{{ .Exe }}{{ .EXTENSION }}$
  • .\*/{{ .Exe }}{{ .EXTENSION }}$

• Hints to:

  • use the executable name as a pattern (useful for repositories with multiple binaries, such as ahmetb/kubectx)

    pattern: "{{ .Exe }}"
    weight: 1
    

  • prefer .zip files for Windows

    pattern: zip
    weight: '{{ if eq .OS "windows" }}1{{ else }}0{{ end }}'
    

  • prefer the three last versions of the arm architecture, in descending order

    - pattern: "armv{{.ARCH_VERSION}}"
      weight: 3
    - pattern: "armv{{sub .ARCH_VERSION 1}}"
      weight: 2
    - pattern: "armv{{sub .ARCH_VERSION 2}}"
      weight: 1
    

• Extensions to use when filtering assets

  • .exe for Windows, empty for Linux and MacOS
  • .zip for Windows and Darwin
  • .tar.gz for all platforms

• find mode for downloading, extracting and finding the executable

• The default source type as github

• none strategy to skip tools which already exist

• Settings the environment variable GH_TOKEN to the value of GODYL_GITHUB_TOKEN

The full set of default options are:

exe:
output:
platform:
values:
fallbacks:
hints:
source:
tags:
strategy:
env:
mode:

For reference full reference of what values you can set, see the tools section.

Template overview

All functions available in the slim-sprig library are available for use in templates.

Variables

The following table lists the available template variables, where they may be used, and their descriptions:

Variable	Description
{{ .Name }}	The name of the tool or project
{{ .Output }}	The output path template for built artifacts
{{ .Exe }}	The name of the executable
{{ .Env.<> }}	Any environment variable
{{ .Values.<> }}	Custom values for templating
{{ .Version }}	The version of the tool or project
{{ .OS }}	The operating system (e.g., linux, darwin, windows)
{{ .ARCH }}	The architecture type (e.g., amd64, arm64)
{{ .ARCH_VERSION }}	The version of the architecture, if applicable
{{ .LIBRARY }}	The system library (e.g., gnu, musl)
{{ .EXTENSION }}	The file extension specific to the platform
{{ .DISTRIBUTION }}	The distribution name (e.g., debian, alpine)

Allowed in

Only certain fields are templated. Below is a list of fields where templating is allowed, along with examples of how they might be used:

• output

  output: bin/{{ .OS }}-{{ .ARCH }}
  

• skip

  skip:
    reason: "tool is not available for windows"
    condition: '{{ eq .OS "windows" }}'
  

• version

  version: |-
    {{- if has .OS (list "linux" "darwin") -}}
        v0.1.0
    {{- else -}}
        v0.2.0
    {{- end -}}
  

• source.type

  source:
    type: |-
      {{- if has .OS (list "linux" "darwin") -}}
          github
      {{- else -}}
          go
      {{- end -}}
  

• exe.patterns

  exe:
    patterns:
      - "^{{ .OS }}-{{ .Exe}}"
  

• extensions

  extensions:
    - '{{ if eq .OS "windows" }}.exe{{ else }}{{ end }}'
  

• source.commands

  commands:
    - pip install {{ .Exe}}=={{ .Version }}
  

• hints[].pattern

  hints:
    - pattern: "{{ .OS }}"
      must: true
  

• hints[].weight

  hints:
    - pattern: armhf
      weight: |-
        {{- if eq .ARCH "arm" -}}
        1
        {{- else -}}
        0
        {{- end -}}
  

• path

  path: https://get.helm.sh/helm-v{{ .Version }}-{{ .OS }}-{{ .ARCH }}.tar.gz
  

[!NOTE] Not that if Version is not provided, it will be evaluated to an empty string. If it is being inferred, it will be available in the following fields:

• exe.name
• exe.patterns
• extensions
• hints[].pattern
• hints[].weight
• path
• source.commands
• post

Inference

Operating Systems

OS	Inferred from
Linux	linux
Darwin	darwin, macos, mac, osx
Windows	windows, win
FreeBSD	freebsd
Android	android
NetBSD	netbsd
OpenBSD	openbsd

Architectures

Architecture	Inferred from
AMD64	amd64, x86_64, x64, win64
ARM64	arm64, aarch64
AMD32	amd32, x86, i386, i686, win32, 386, 686
ARM32 (v7)	armv7, armv7l, armhf
ARM32 (v6) *	armv6, armv6l
ARM32 (v5)	armv5, armel
ARM32 (v)	arm

Libraries

Library	Inferred from
GNU	gnu
Musl	musl
MSVC	msvc
LibAndroid	android

Notes

All regex expressions are evaluated using search, meaning that ^ and $ are necessary to match the start and end of the string. When running 32-bit userland on a 64-bit Kernel, there's some attempts to infer the matching 32-bit architecture.

However, to be certain that the right binary is downloaded, it's recommended to pass the --arch flag to the tool.