kcp

KCP CLI

This repository is part of the Confluent organization on GitHub. It is public and open to contributions from the community.

Please see the LICENSE file for contribution terms. Please see the CHANGELOG.md for details of recent updates.

Table of Contents

• Overview
• Installation
• Authentication
• Development
• Examples

Overview

Mission: Simplify and streamline your Kafka migration journey to Confluent Cloud!

kcp helps you migrate your Kafka setups to Confluent Cloud by providing tools to:

• Scan scan and identify resources in existing Kafka deployments.
• Create reports for migration planning and cost analysis.
• Generate migration assets and infrastructure configurations.

Key Features

Installation

Build/Install from Source

[!TIP] Make sure you have Go 1.24+ installed before building from source

# Clone the repository
git clone https://github.com/confluentinc/kcp.git
cd kcp

# Install to system path (requires sudo)
make install

Installing from GitHub Releases

You can also download kcp from GitHub under the releases tab. We provide support for Linux and Darwin arm64/amd64 systems respectively.

Once downloaded, make sure to set the binary permissions to executable by running chmod +x <binary name>.

If you wish to run the downloaded kcp binary from anywhere on your system, you may run the following (requires sudo permissions):

# Update the binary suffix to your respective architecture.
sudo mv ./kcp_<ARCH> /usr/local/bin/kcp

Authentication

Ensure that your terminal session is authenticated with AWS. The kcp CLI uses the standard AWS credential chain and supports multiple authentication methods:

Authentication options:

• Environment variables: Export AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, and optionally AWS_SESSION_TOKEN
• AWS credentials file: Configure with aws configure (requires AWS CLI)
• AWS SSO/Identity Center: Use aws sso login (requires AWS CLI)
• IAM Roles: Assume roles or use instance profiles
• Other tools: Any tool that sets AWS credentials in the standard locations such as granted.

Verify your authentication: The easiest way to test authentication is to run a kcp command that requires AWS access such as kcp scan region, or if you have AWS CLI installed:

aws sts get-caller-identity

Required Permissions

Required IAM permissions for running kcp commands can be found in the README.md generated by kcp init or viewed in the browser here.

Examples

Basic Migration Workflow

1. Initialize configuration:

   kcp init
   

2. Scan your AWS region:

   kcp scan region --region us-east-1
   

3. Scan specific cluster:

   kcp scan cluster \
     --cluster-arn arn:aws:kafka:us-east-1:123456789012:cluster/my-cluster \
     --use-sasl-iam
   

4. Generate reports:

   kcp report region costs \
     --region us-east-1
     --start 2025-07-10 \
     --end 2025-07-11 \
     --daily \
     --tag Environment=Staging \
     --tag Owner=ajanuzi
   

   kcp report region \
     --region us-east-1 \
     --start 2025-07-10 \
     --end 2025-07-11
   

5. Generate migration assets:

   kcp create-asset bastion-host \
     --bastion-host-cidr 10.0.XXX.0/24 \
     --vpc-id vpc-xxxxxxxxx \
     --region us-east-1
   

   kcp create-asset migration-infra \
     --region us-east-1
     --cluster-file cluster.json \
     --vpc-id vpc-xxxxxxxxx
     --type 1 \
     --cc-api-key EXAMPLECCKEY \
     --cc-api-secret EXAMPLECCSECRET \
     --cc-env-name my-new-environment \
     --cc-cluster-name my-new-cluster \
     --cluster-type enterprise \
     --ansible-control-node-subnet-cidr 10.0.10.0/24 \
     --jump-cluster-broker-subnet-config us-east-1a:10.0.50.0/24,us-east-1b:10.0.60.0/24,us-east-1c:10.0.70.0/24 \
     --use-sasl-iam \
     --jump-cluster-broker-iam-role-name msk-cluster-link-role
   

Development

Prerequisites

• Go 1.24+
• Make

Build Commands

# Build for current platform
make build

# Build for Linux
make build-linux

# Build for all platforms
make build-all

# Clean build artifacts
make clean

Testing & Quality

# Format go code
make fmt

# Run tests
make test

# Run tests with coverage
make test-cov

# Run tests with coverage and open UI coverage browser
make test-cov-ui