Documentation
¶
Overview ¶
Package cmd defines all CLI commands available in the Data Master project.
It includes entrypoints for provisioning infrastructure, catalog synchronization, dataset seeding, benchmarking, processing orchestration, and dashboard generation.
Each command is modular and self-contained, and the CLI is implemented using Cobra, allowing flexible composition, argument parsing, and extensibility.
This package is the user-facing interface for executing and automating operations across all stages of the Medallion Architecture (raw, bronze, silver, gold).
Index ¶
- Constants
- func ArtifactsCmd(artifacts embed.FS) *ishell.Cmd
- func AuthCmd() *ishell.Cmd
- func BenchmarkCmd() *ishell.Cmd
- func CatalogCmd() *ishell.Cmd
- func ClearCmd(shell *ishell.Shell) *ishell.Cmd
- func CustomReadlineConfig() *readline.Config
- func DeployCmd(templates, artifacts, scripts embed.FS) *ishell.Cmd
- func ECRCmd(artifacts embed.FS) *ishell.Cmd
- func ExitCmd(shell *ishell.Shell) *ishell.Cmd
- func GrafanaCmd(dashboards embed.FS) *ishell.Cmd
- func Help(helps embed.FS) *ishell.Cmd
- func LambdaCmd(artifacts embed.FS) *ishell.Cmd
- func MigrationCmd(migrations embed.FS) *ishell.Cmd
- func ParseShellFlags(c *ishell.Context, fs *flag.FlagSet) bool
- func ProcessCmd() *ishell.Cmd
- func SeedCmd() *ishell.Cmd
- func StacksCmd(templates embed.FS) *ishell.Cmd
- func StreamCmd() *ishell.Cmd
- func Welcome(shell *ishell.Shell)
- func WhoAmICmd() *ishell.Cmd
- func WithAuth(f WrapperFunc) func(c *ishell.Context)
- type WrapperFunc
Constants ¶
const ( Option1 = "1" Option2 = "2" )
Variables ¶
This section is empty.
Functions ¶
func ArtifactsCmd ¶
ArtifactsCmd returns an interactive shell command that lists all embedded artifacts (.zip for Lambda, .tar for Docker). It reads the embedded filesystem under the predefined artifact path and prints the artifact names with their type.
func AuthCmd ¶
AuthCmd returns an interactive shell command that handles AWS authentication. The user is prompted to authenticate either using a named profile or direct access/secret keys. After selecting the authentication method and region, the credentials are saved locally and tested using the STS GetCallerIdentity call. On success, the authenticated identity is displayed.
func BenchmarkCmd ¶
BenchmarkCmd returns an interactive CLI command that runs the ingestion benchmark using one of the supported execution environments: ECS (Go) or Glue (PySpark).
Usage:
benchmark --run <ecs|glue>
Parameters:
--run: Required flag specifying the implementation to execute. - ecs: Executes the benchmark using a Go binary running on ECS. - glue: Executes the benchmark using a PySpark script on AWS Glue.
Example:
benchmark --run ecs
The command prompts for confirmation before launching the benchmark. After execution, it prints the result or any error encountered.
func CatalogCmd ¶
CatalogCmd returns an interactive shell command that creates or updates AWS Glue catalog tables. The user can optionally specify a data lake layer (--layer) and a comma-separated list of table names (--tables). If no flags are provided, all tables in all layers (bronze, silver, gold) will be processed.
func ClearCmd ¶
ClearCmd returns an interactive shell command that clears the screen output. It invokes the ClearScreen method from the provided shell instance.
func CustomReadlineConfig ¶
CustomReadlineConfig returns a readline.Config with specific settings for interactive shells using ishell.
It disables Meta key sequences that commonly interfere with normal input, especially on buggy terminals that send ESC-prefixed characters by default.
Blocked runes include:
- MetaBackward (Alt+b): moves backward by word
- MetaForward (Alt+f): moves forward by word
- MetaDelete (Alt+d): deletes forward word
- MetaBackspace (Alt+Backspace): deletes backward word
- MetaTranspose (Alt+t): transposes characters
Returns:
- *readline.Config: customized configuration for readline shell input.
func DeployCmd ¶
DeployCmd returns an interactive shell command that deploys infrastructure stacks using embedded CloudFormation templates. The user can deploy a specific stack via the --stack flag, optionally passing parameters with --params (key=value pairs). If no stack is specified, all stacks will be deployed.
func ECRCmd ¶
ECRCmd returns an interactive shell command that publishes one or all Docker images to ECR from embedded .tar artifacts. The user can specify an image name with --name, or publish all available .tar artifacts if not provided.
func ExitCmd ¶
ExitCmd returns an interactive shell command that terminates the program. When executed, it prints a goodbye message and stops the shell session.
func GrafanaCmd ¶
GrafanaCmd returns an interactive shell command that creates or updates Grafana dashboards. The user can optionally specify a single dashboard name with the --dashboard flag. If no dashboard is specified, all available dashboards embedded in the filesystem will be pushed.
func Help ¶
Help returns an interactive shell command that displays help information for available commands. When run without arguments, it lists all registered commands and their short descriptions. When a specific command is provided (e.g. "help deploy"), it tries to load a detailed help template from the embedded filesystem. If not found, it falls back to the built-in help description.
func LambdaCmd ¶
LambdaCmd returns an interactive shell command that deploys one or all Lambda functions from embedded artifacts. The user can specify a function name with --name, or deploy all available .zip artifacts if not provided. Optional flags include --memory (in MB) and --timeout (in seconds).
func MigrationCmd ¶
MigrationCmd returns an interactive shell command that runs database migration scripts. The user can optionally specify a script name using --script to run a single migration, or omit it to run all available migration scripts in order. The scripts are read from the embedded filesystem.
func ParseShellFlags ¶
ParseShellFlags parses shell flags and returns true if parsing was successful. It also prints errors in a consistent format using the misc.Red wrapper.
func ProcessCmd ¶
ProcessCmd triggers the processing pipeline for a specific layer (e.g. silver, gold) with optional filtering by table names.
func SeedCmd ¶
SeedCmd returns an interactive shell command that seeds datasets into Aurora or S3. The user can optionally specify a dataset file with --file to seed a single dataset, or omit it to seed all datasets from the default source (GitHub).
func StacksCmd ¶
StacksCmd returns an interactive shell command that lists all available infrastructure stacks. It reads the embedded templates directory and extracts stack names based on the configured template extension.
func WhoAmICmd ¶
WhoAmICmd returns an interactive shell command that displays the current AWS identity. It uses the AWS STS GetCallerIdentity API to retrieve and print the UserId, Account, and ARN. AWS authentication is required and handled by the WithAuth middleware.
func WithAuth ¶
func WithAuth(f WrapperFunc) func(c *ishell.Context)
WithAuth wraps a command handler with AWS auth validation logic.
Types ¶
type WrapperFunc ¶
WrapperFunc defines the signature for wrapped command handlers.