cq-source-upwind

CloudQuery Upwind Source Plugin

A CloudQuery source plugin for extracting security and vulnerability data from Upwind.

Features

• Vulnerability Findings: Extract comprehensive vulnerability data including CVE details, CVSS scores, affected packages, container images, and cloud resources
• Advanced Filtering: Filter vulnerabilities by exposure, exploitability, severity, cloud accounts, clusters, namespaces, and more
• Memory Efficient: Streaming architecture handles large datasets without loading everything into memory
• Production Ready: Built-in rate limiting, retry logic, and comprehensive error handling

Quick Start

Prerequisites

• Go 1.22 or later
• CloudQuery CLI
• Upwind account with API credentials

Installation

Option 1: Local Development

Build the plugin locally:

git clone https://github.com/tektite-io/cq-source-upwind.git
cd cq-source-upwind
make build

Option 2: Use with CloudQuery (after publishing)

cloudquery sync config.yml

Configuration

Create a config.yml file:

kind: source
spec:
  name: upwind
  registry: local
  path: ./cq-source-upwind
  version: v1.0.0
  tables: ["*"]
  destinations: ["postgresql"]
  
  spec:
    # Required: Upwind API credentials
    client_id: "${UPWIND_CLIENT_ID}"
    client_secret: "${UPWIND_CLIENT_SECRET}"
    organization_id: "${UPWIND_ORGANIZATION_ID}"
    
    # Optional: API region (US, EU, ME), defaults to US
    region: "US"
    
    # Optional: Performance tuning
    max_retries: 3
    max_concurrency: 10
    page_size: 100
    rate_limit_per_second: 10
    
    # Optional: Filter vulnerabilities
    vulnerability_filters:
      severity: "CRITICAL"              # Filter by severity
      is_exploitable: true              # Only exploitable vulnerabilities
      is_exposed: true                  # Only internet-exposed
      is_in_use: true                   # Only packages in use
      # cloud_account_id: "123456789"   # Filter by cloud account
      # cluster_id: "my-cluster"        # Filter by cluster
      # namespace: "production"         # Filter by namespace
      # framework: "golang"             # Filter by package framework
      # image_name: "nginx"             # Filter by image name

---
kind: destination
spec:
  name: postgresql
  path: cloudquery/postgresql
  registry: cloudquery
  version: "v8.12.2"
  
  spec:
    connection_string: "${DATABASE_URL}"

Set your environment variables:

export UPWIND_CLIENT_ID="your-client-id"
export UPWIND_CLIENT_SECRET="your-client-secret"
export UPWIND_ORGANIZATION_ID="your-org-id"
export DATABASE_URL="postgresql://user:pass@localhost:5432/dbname"

Run the sync:

cloudquery sync config.yml

Configuration Reference

Authentication (Required)

Performance Tuning (Optional)

Vulnerability Filters (Optional)

All filters are optional and can be combined:

Tables

upwind_vulnerability_findings

Vulnerability findings with comprehensive context including CVE details, CVSS scores, affected packages, container images, and cloud resources.

Schema: 44 columns including:

• Core finding metadata (ID, status, source, timestamps)
• Vulnerability details (CVE ID, CVSS v2/v3/v4 scores, exploitability)
• Container image information (name, digest, registry, OS)
• Package details (name, version, framework, usage status)
• Cloud resource context (accounts, regions, clusters, namespaces, exposure)

For detailed schema, see generated documentation.

Example Queries

Critical Exploitable Vulnerabilities

SELECT 
  vulnerability_nvd_cve_id,
  vulnerability_nvd_cvss_v3_score,
  package_name,
  package_version,
  resource_name,
  resource_namespace
FROM upwind_vulnerability_findings
WHERE vulnerability_exploitable = true
  AND vulnerability_nvd_cvss_v3_severity = 'CRITICAL'
ORDER BY vulnerability_nvd_cvss_v3_score DESC;

Internet-Exposed Vulnerabilities

SELECT 
  vulnerability_nvd_cve_id,
  resource_name,
  resource_namespace,
  image_name,
  vulnerability_nvd_cvss_v3_severity
FROM upwind_vulnerability_findings
WHERE resource_internet_exposure_ingress_active_communication = true
ORDER BY 
  CASE vulnerability_nvd_cvss_v3_severity
    WHEN 'CRITICAL' THEN 1
    WHEN 'HIGH' THEN 2
    WHEN 'MEDIUM' THEN 3
    WHEN 'LOW' THEN 4
  END;

Vulnerability Summary by Severity

SELECT 
  vulnerability_nvd_cvss_v3_severity,
  COUNT(*) as count,
  COUNT(DISTINCT vulnerability_nvd_cve_id) as unique_cves,
  COUNT(DISTINCT resource_name) as affected_resources
FROM upwind_vulnerability_findings
GROUP BY vulnerability_nvd_cvss_v3_severity
ORDER BY 
  CASE vulnerability_nvd_cvss_v3_severity
    WHEN 'CRITICAL' THEN 1
    WHEN 'HIGH' THEN 2
    WHEN 'MEDIUM' THEN 3
    WHEN 'LOW' THEN 4
  END;

Vulnerabilities by Cloud Account

SELECT 
  resource_cloud_account_name,
  COUNT(DISTINCT vulnerability_nvd_cve_id) as unique_cves,
  COUNT(*) as total_findings,
  SUM(CASE WHEN vulnerability_nvd_cvss_v3_severity = 'CRITICAL' THEN 1 ELSE 0 END) as critical,
  SUM(CASE WHEN vulnerability_nvd_cvss_v3_severity = 'HIGH' THEN 1 ELSE 0 END) as high
FROM upwind_vulnerability_findings
GROUP BY resource_cloud_account_name
ORDER BY critical DESC, high DESC;

Development

Building

make build

Running as gRPC Server (for debugging)

Terminal 1:

make serve

Terminal 2:

# Update config.yml to use grpc registry
cloudquery sync config.yml

Testing

make test

Code Quality

make fmt      # Format code
make lint     # Run linter
make verify   # Run all checks

Troubleshooting

Authentication Errors

Error: 401 Unauthorized or failed to create Upwind client

Solution: Verify your credentials are correct:

echo $UPWIND_CLIENT_ID
echo $UPWIND_CLIENT_SECRET
echo $UPWIND_ORGANIZATION_ID

Ensure the region setting matches your Upwind account region.

Rate Limiting

Error: 429 Too Many Requests

Solution: Reduce request rate in your configuration:

spec:
  spec:
    rate_limit_per_second: 5
    max_concurrency: 5

Memory Issues

Solution: Reduce page size and apply filters:

spec:
  spec:
    page_size: 50
    vulnerability_filters:
      severity: "CRITICAL"
      is_exploitable: true

Configuration Errors

Error: json: unknown field "client_id"

Solution: Ensure plugin configuration is nested under spec.spec:

kind: source
spec:
  name: upwind
  tables: ["*"]
  destinations: ["postgresql"]
  
  spec:  # Plugin config goes here
    client_id: "..."
    client_secret: "..."

Architecture

• Streaming: Memory-efficient channel-based streaming from Upwind API
• Concurrency: Configurable concurrent requests for optimal performance
• Resilience: Automatic retry with exponential backoff
• Observability: Structured logging with zerolog

Links

• Upwind
• Upwind Go SDK
• CloudQuery Documentation
• Creating CloudQuery Plugins

License

This project is licensed under the Mozilla Public License Version 2.0 - see the LICENSE file for details.

Support

• Issues: GitHub Issues
• CloudQuery: CloudQuery Discord
• Upwind: Upwind Support