corral

command module
v0.0.6 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jun 24, 2026 License: GPL-3.0 Imports: 4 Imported by: 0

README

Corral logo

Corral

Automatically clone and organise GitHub repositories by visibility and language.

Build Version License


Install

Homebrew (macOS / Linux):

brew install sebastienrousseau/tap/corralctl

Arch Linux (AUR):

yay -S corralctl-bin    # or: paru -S corralctl-bin

From source (requires Go 1.26+):

git clone https://github.com/sebastienrousseau/corral.git
cd corral
make build              # produces ./corralctl
gh auth login
./corralctl <owner>

Prebuilt binaries and .deb/.rpm packages are also attached to each GitHub release.

Then verify the output:

ls ~/Code/

Requires gh, git, and Go 1.26+. Works on macOS, Ubuntu/Debian, Fedora/RHEL, Arch, and WSL2.

The command is corralctl (the project is Corral); the …ctl binary name avoids a clash with the unrelated corral formula in homebrew-core.

Platform-specific prerequisites

macOS:

brew install go git gh

Ubuntu / Debian / WSL2:

sudo apt install golang git

Install gh separately — see the GitHub CLI install guide for your distribution.

Fedora / RHEL:

sudo dnf install golang git gh

WSL2 users: Run all commands inside your Linux distribution, not from PowerShell or CMD. The script works identically to native Linux.

Automation and cron

The tool is idempotent and non-interactive. Safe to run on a schedule:

# crontab -e
0 2 * * * /path/to/corralctl my-username

Existing repositories are skipped. Only new ones are cloned.


Overview

Most cloning tools dump every repository into a single flat directory. Finding anything means scrolling through hundreds of folders with no structure. Corral creates a clean, navigable local mirror — sorted by visibility and language — in one command. No config files.

~/Code/
├── Public/
│   ├── rust/
│   │   └── my-crate/
│   ├── typescript/
│   │   └── my-app/
│   └── other/
│       └── dotfiles/
└── Private/
    └── python/
        └── internal-tool/
  • One command to clone and organise every repository from a user or organisation
  • Safe to re-run at any time — new repos are cloned, and existing ones are pulled to their latest changes (unless --no-sync is passed)
  • Automatic migration from flat ~/Code/<Language>/ layouts to the new visibility-based structure
  • Tested on macOS and Ubuntu with automated tests, 100% coverage, and signed commits.

Architecture

Run once or a hundred times, the directory tree converges on the same state.

graph TD
    A[User Shell] --> B{Corral}
    B --> C[Pre-flight: Git / GitHub API]
    C -- Missing --> Z1[Exit: dependency error]
    C -- OK --> D[GitHub API → Fetch Repos]
    D -- Fails --> Z2[Exit: API error]
    D -- OK --> E[Loop: each repo]
    E --> F{Already cloned?}
    F -- "Yes (default)" --> G1[git pull --rebase --autostash]
    F -- "Yes + --no-sync" --> G2[Skip]
    F -- No --> H{Legacy directory?}
    H -- Yes --> I[Migrate to new layout]
    H -- No --> J[git clone]
    G1 & G2 & I & J --> E
    E -- Done --> K[Remove empty legacy directories]
    K --> L[Print summary]

Features

Structured The only tool that sorts repositories into Public/ and Private/ trees, grouped by primary language
Idempotent Safe to re-run at any time — already-cloned repositories are skipped, only new ones are fetched
Migratory Flat ~/Code/<Language>/ layouts from earlier runs move into the new structure automatically
Platforms First-class support for macOS, Ubuntu/Debian, Fedora/RHEL, Arch, and Windows via WSL2
Zero-config No YAML, no .env, no config files — pass the owner name and run
Fail-safe Pre-flight checks for git and API connectivity with clear error messages on failure
Production-grade 100% test coverage, CI on Ubuntu and macOS, signed commits
Security All commits cryptographically signed (ED25519), CI actions pinned to immutable SHAs, Gitleaks secret scanning

Usage

Parameter Required Default Description
owner Yes GitHub username or organisation
base_dir No $HOME/Code Root directory for the cloned tree
limit No 1000 Maximum repositories to fetch
Option Short Default Description
--base-dir $HOME/Code Root directory for cloned repos
--limit -l 1000 Maximum repositories to fetch
--concurrency -c 1 Number of concurrent clone/sync operations
--dry-run -n off Preview actions without making changes
--orphans -o off Detect and list local repositories no longer on GitHub
--help -h Show help message
--version -v Print the Corral version and exit
--protocol -p https Clone protocol — ssh or https
--no-sync off Skip pulling latest changes for already-cloned repos
--recurse-submodules off Initialise submodules on clone and sync
--output text Output format: text, json, or ndjson
--auth auto Auth mode: auto, token, or gh
--visibility all Filter repositories by visibility: all, public, private
--include-forks off Include forked repositories
--include-archived off Include archived repositories
--languages Comma-separated language allow-list (for example: go,rust)
--exclude-languages Comma-separated language deny-list
--clone-blobless off Use partial clone (--filter=blob:none)
--clone-single-branch off Clone only the default branch
--clone-depth 0 Shallow clone depth (0 disables shallow clone)
--retry-max 4 Max retries for transient GitHub API failures
--retry-min-backoff 500ms Minimum backoff between retries
--retry-max-backoff 8s Maximum backoff between retries

Clone a personal account:

./corralctl my-username

Clone an organisation into a custom directory:

./corralctl my-org ~/Projects 500

Clone via SSH (key-based auth):

./corralctl --protocol ssh my-username

Skip pulling updates for existing clones:

./corralctl --no-sync my-username

Preview what would happen without making changes:

./corralctl --dry-run my-org

Fetch only private Go and Rust repositories, and emit machine-readable JSON:

./corralctl --visibility private --languages go,rust --output json my-org

Use GitHub CLI auth explicitly and perform shallow single-branch clones:

./corralctl --auth gh --clone-single-branch --clone-depth 1 my-username

Detect local repositories that no longer exist on GitHub (orphans):

./corralctl --orphans my-username

Clone with submodules, raising concurrency for a large account:

./corralctl --recurse-submodules --concurrency 16 my-org

Include forks and archived repositories, excluding generated languages:

./corralctl --include-forks --include-archived --exclude-languages makefile,dockerfile my-username

Stream one NDJSON record per repository (ideal for piping into jq):

./corralctl --output ndjson my-org | jq -r 'select(.action=="CLONE") | .repo'

Use a fixed token and tune retry backoff for flaky networks:

GITHUB_TOKEN=ghp_xxx ./corralctl --auth token --retry-max 6 --retry-min-backoff 1s --retry-max-backoff 30s my-org

Print the version:

./corralctl --version

By default (--auth auto), Corral checks GITHUB_TOKEN/GH_TOKEN first, then falls back to gh auth token.


What's Included

Organisation and Layout
  • Visibility sorting separates repositories into Public/ and Private/ trees based on GitHub metadata. GitHub Enterprise INTERNAL repositories are securely routed to the Private/ tree.
  • Language grouping places each repository under its primary language directory, normalised to lowercase
  • Special characters are handled cleanly — C# becomes csharp, C++ becomes cpp, spaces and slashes become underscores
  • Null languages default to other/ so every repository has a home
Legacy Migration
  • Flat layouts from earlier versions (~/Code/<Language>/<repo>) are detected and moved into the new <Visibility>/<Language>/<repo> structure
  • Empty directories left behind after migration are removed automatically
  • Existing clones are never overwritten or deleted — the script only adds, never subtracts
Troubleshooting
Message Cause Solution
ERROR: Required command 'git' not found Git is not installed See Install above
ERROR: GITHUB_TOKEN (or GH_TOKEN) environment variable not set --auth token was set and no token variable is present Run export GITHUB_TOKEN=$(gh auth token) or switch to --auth auto
FAILED: owner/repo Network issue or private repo without token access Check connectivity and verify gh auth status
Frequently Asked Questions
  • Can it back up private repositories? Yes. Any repository visible to the authenticated GITHUB_TOKEN is cloned. PRIVATE and GitHub Enterprise INTERNAL repositories land in the Private/ tree.
  • Does it work with organisations? Yes. Pass the organisation name as the first argument. Both user accounts and organisations are supported.
  • What happens if a repository is deleted on GitHub? The local clone remains untouched. The script never deletes existing directories.
  • What happens if I have uncommitted changes when it syncs? Your local uncommitted work is kept safe. Git automatically stashes your changes (--autostash), rebases the latest commits from the remote branch, and then reapplies your stash. If there is a direct conflict, the sync is cleanly aborted for that specific repository.
  • Does it work on Windows? Yes, through WSL2. Install a Linux distribution from the Microsoft Store, open its terminal, and run the script there. It behaves identically to native Linux.
  • Is it safe to run on a schedule? Yes. The tool is idempotent — existing repos are skipped, only new ones are cloned. No interactive prompts.
How It Compares
Feature Corral ghorg ghcloneall Gist scripts
Organises by language Yes No No No
Organises by visibility Yes No No No
macOS, Linux, and WSL2 Yes (CI on both) Yes Linux only Varies
Zero configuration Yes No No Yes
Idempotent re-runs Yes Yes Yes No
Concurrent operations Yes Yes Yes No
Legacy layout migration Yes No No No
Test suite 100% coverage, CI on 2 OS Yes Limited None

For security policy and vulnerability reporting, see SECURITY.md.


THE ARCHITECTSebastien Rousseau THE ENGINEEUXIS ᛫ Enterprise Unified Execution Intelligence System


License

Licensed under the GNU General Public License v3.0.

Back to Top

Documentation

Overview

Package main is the entry point for the Corral CLI application.

Directories

Path Synopsis
Package cmd provides the command-line interface for the Corral application.
Package cmd provides the command-line interface for the Corral application.
internal
engine
Package engine provides the core concurrency and execution logic for Corral.
Package engine provides the core concurrency and execution logic for Corral.
git
Package git provides helper functions to execute common Git commands by wrapping the system's git binary using os/exec.
Package git provides helper functions to execute common Git commands by wrapping the system's git binary using os/exec.
github
Package github provides functionality to interact with the GitHub API.
Package github provides functionality to interact with the GitHub API.
tui
Package tui provides a Bubble Tea terminal user interface for Corral.
Package tui provides a Bubble Tea terminal user interface for Corral.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL