sdk-go

OpenCTEM SDK

Go SDK for building integrations with the OpenCTEM security platform.

Overview

OpenCTEM SDK provides Go packages for:

• pkg/sensorkit: the sensor runtime in one call (identity, connection, heartbeat, commands, durable outbox, per-task tool sandbox, local policy, preflight checks and the config report)
• pkg/client: the platform protocol client
• pkg/ctis: the CTIS result types
• pkg/httpsec: SSRF-safe HTTP clients
• pkg/conformance: a fake platform and the sensor conformance suite

The scanner wrappers (Semgrep, CodeQL, Trivy, Betterleaks, Nuclei and the recon tools) live in the sensor (github.com/openctemio/sensor), not in the SDK. What each package promises is in docs/STABILITY.md; where the SDK is going is in docs/rfcs/sensor-sdk-v2.md.

Installation

go get github.com/openctemio/sdk-go

Upgrading from v0.6 (agent → sensor)

v0.7.0 renames the agent vocabulary to sensor (core.BaseAgent → core.BaseSensor, client.WithAgentID → client.WithSensorID, …). Upgrade your module with the codemod, which rewrites only SDK identifiers, type-safely, then moves you to the new SDK:

go run github.com/openctemio/sdk-go/cmd/sensor-migrate@v0.7.0 -dry-run   # review
go run github.com/openctemio/sdk-go/cmd/sensor-migrate@v0.7.0            # apply

The wire to the platform (protocol v1) is unchanged, and the SDK migrates a sensor's saved credentials and old AGENT_* settings by itself. See CHANGELOG.md for the full list.

Quick Start

API Client

package main

import (
    "context"
    "github.com/openctemio/sdk-go/pkg/client"
)

func main() {
    // Create client
    c := client.New(
        client.WithBaseURL("http://localhost:8080"),
        client.WithAPIKey("your-api-key"),
    )

    // List assets
    assets, err := c.Assets().List(context.Background())
    if err != nil {
        panic(err)
    }

    // Create finding
    finding := &client.Finding{
        Title:    "SQL Injection",
        Severity: "HIGH",
        // ...
    }
    err = c.Findings().Create(context.Background(), finding)
}

Build a sensor in 30 lines

pkg/sensorkit is the whole sensor runtime in one call. A sensor implements its tools; the kit reads the settings, connects (protocol v2 negotiated on hello, v1 for what an older platform lacks), reports the tools on every heartbeat, waits while the key is rejected, follows the heartbeat doorbell, claims commands into resource-sized slots, delivers results through the durable outbox, renews the API key and drains on SIGTERM.

package main

import (
	"context"
	"time"

	"github.com/openctemio/sdk-go/pkg/core"
	"github.com/openctemio/sdk-go/pkg/sensorkit"
)

var version = "dev"

func main() {
	tool := core.NewBaseScanner(&core.BaseScannerConfig{ // your tool's binary
		Name: "mytool", Binary: "mytool", Capabilities: []string{"sast"},
		DefaultArgs: []string{"scan", "--sarif", "{target}"}, Timeout: 30 * time.Minute,
	})
	kit, err := sensorkit.New(sensorkit.Options{Name: "my-sensor", Version: version})
	sensorkit.Exit(err)    // a clear message and exit code for a bad setting
	kit.AddScanner(tool)   // reported (installed? version?), dispatched, scheduled
	sensorkit.Exit(kit.Run(context.Background())) // blocks; SIGINT/SIGTERM drain
}

Run it with API_URL and API_KEY (a sensor key from Settings > Sensors). examples/minimal-sensor is a complete one with an in-process tool.

Keep the state directory. A key renewal retires the key the sensor was started with; the renewed key is saved in sensor-credentials.json in the state directory (0600) and used on the next start instead of API_KEY. In a container, mount a volume at /var/lib/openctem/state (-v my-sensor-state:/var/lib/openctem/state): without one the kit does not renew, because a recreated container would come back with the retired key. When an administrator regenerates the key and API_KEY is set to the new one, the new key wins. Every heartbeat also carries the process's instance_id, so the platform flags one key running in two places.

Plug your own parts in: kit.AddCollector, kit.AddParser (your tool's output format), kit.HandleCommand("type", executor), kit.UseCommandMiddleware(mw, "type") (wrap every command), Options.Content (managed scanner content on the heartbeat), Options.ScanTargetPolicy, Options.AssetResolver, kit.Tools() (more tools or capabilities). The resolvers (ResolveMaxJobs, ResolveOutbox, ResolveProtocol, CheckCredentials, MigrateSettings) are exported for a sensor with its own flags or configuration file. pkg/core stays available for anything the kit does not cover; the sections below describe it.

Writing a sensor: you implement executors; the SDK owns queue, slots, leases and reporting

The platform's queue is authoritative (api RFC-030). A sensor only says what it can run; core.CommandPoller does the rest:

poller := core.NewCommandPoller(apiClient, executor, &core.CommandPollerConfig{PollInterval: 30 * time.Second})
poller.SetResourceManager(resource.NewManager(resource.ManagerConfig{
	Cap:       0, // no operator cap: slots follow the resources (up to 64)
	Tools:     []string{"nuclei", "trivy"},
	StateFile: "/var/lib/openctem/tool-costs.json",
	Prober:    &resource.Prober{WorkDir: "/var/lib/openctem"},
}))
poller.SetDoorbell(doorbell)
sensor.SetLoadReporter(poller) // heartbeat: load, resources, capacity, queue, running
go poller.Start(ctx)

An executor that starts processes itself calls core.RecordProcessState(ctx, cmd.ProcessState) after Wait so its CPU time and peak memory teach the cost history (the SDK's exec helpers already do).

Tools: register them, the SDK reports them

The platform does not need to know a sensor's tools: the sensor registers what it runs when it starts, and every heartbeat reports it (api RFC-029 §4.3.1). The platform dispatches by that report, and its administrator can only narrow it.

s := core.NewBaseSensor(cfg, apiClient)
s.AddScanner(myScanner)                 // registered: probed with IsInstalled
executor.SetToolRegistry(s.Tools())     // the executor's scanners and collectors too
s.Tools().Register(core.ToolSpec{       // any other tool, with an optional probe
	Name: "zap", Version: "2.15.0", Capabilities: []string{"dast"},
	Probe: func(ctx context.Context) (bool, string, error) { return zapVersion(ctx) },
	Cost:  &resource.ToolCostHint{Cores: 2, MemBytes: 2 << 30, SecondsPerTarget: 60},
})
s.Tools().AddCapabilities("validate")   // served whatever the tools
s.Tools().Limit(strings.Split(os.Getenv("MY_TOOLS"), ",")...) // optional operator allowlist

• A tool whose probe fails is reported as not installed and gets no jobs. Installed tools report their version.
• Probes are cached for 10 minutes (SetProbeTTL, Refresh).
• Feed the slot sizer the same list: resource.ManagerConfig{Tools: s.Tools().Names(), CostHints: s.Tools().CostHints()}.
• SetCapabilityReporter replaces the registry with your own reporter.

Write a tool

A tool is a manifest plus a run function (pkg/tool); the sensor runs it out of process, admits its targets against the local policy and checks every record it emits. A simple tool is declared with the builder:

var Dotenv = tool.Define("dotenv", "1.0.0").
	Targets("http_service").
	Produces("finding:misconfiguration").
	Params(tool.StringParam("path").Label("Path").Default("/.env").Pattern("^/")).
	Handle(func(ctx tool.Context, job *tool.Job, emit tool.Emit) error {
		for _, t := range job.Targets() {
			// probe t.URL(job.Param("path").String()) with ctx.HTTP()
			_ = emit.Misconfiguration(t, tool.Issue{RuleID: "dotenv-exposed", Title: ".env is readable", Severity: "high"})
			ctx.TargetDone(t)
		}
		return nil
	}).
	MustBuild()

Add it to a sensor with kit.AddTool(Dotenv) and test it with testkit.Run. The design: docs/rfcs/sensor-sdk-v2.md.

Packages

What the SDK promises, what is stable, what is moving to the sensor, and how it grows without breaking sensors: docs/STABILITY.md.

Examples

See examples/ for complete examples:

• A tool in Python with no SDK, and how to check it with the conformance kit (python-adapter, docs/adapter-protocol.md)
• A complete sensor on pkg/sensorkit (minimal-sensor)
• Basic API client usage
• Scanner integration
• CI/CD pipeline integration
• Custom scanner development

Building

# Run tests
go test ./...

# Generate proto files
make proto

# Lint
make lint

Contributing

We welcome contributions! Please see CONTRIBUTING.md.

Related Projects

• openctemio/openctem - the platform: API (api/) and web console (web/), formerly openctemio/api and openctemio/ui
• openctemio/sensor - the OpenCTEM sensor (binary openctemio-sensor, image ghcr.io/openctemio/sensor)

Enterprise Edition

For advanced features and enterprise support, see OpenCTEM Enterprise.

License

Apache License 2.0 - see LICENSE.