scorecard-infra

module
v0.3.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 14, 2026 License: Apache-2.0

README

OpenSSF Scorecard Infrastructure

Presubmits OpenSSF Scorecard Contributor-Covenant Go

The infrastructure that runs OpenSSF Scorecard as a service — and the work to make it run anywhere.

Scorecard results are heuristic signals, not a verdict. Nothing here asserts that a repository "is secure" or "is insecure"; every result declares its source, freshness, and completeness. A score of -1 means inconclusive, not failing.

Contents

About The Project

ossf/scorecard is the engine: checks, probes, scoring, output formats. This repository is what surrounds it — the batch pipeline that scans 1M+ repositories every week, the API that serves what it produces, and the design for moving that stack off any single cloud provider.

Path Role State
Batch scanning pipeline cron/ Producer — scans 1M+ repositories weekly and writes results to object storage Production on AWS (live 2026-09-06). Imported from ossf/scorecard.
Results API api/ Serving tier — the service behind api.scorecard.dev Production on AWS. Imported from ossf/scorecard-webapp. This is what ships.
Provider-agnostic design docs/research/ The plan — a reference design for running Scorecard's hosted data services on any cloud or self-hosted target Reference design; not yet an official artifact.

These are one system's present and future, not three unrelated projects. The pipeline is a producer writing Scorecard results to a bucket; the API is the consumer reading them back out over HTTP. Both arrived here from elsewhere, so that the whole hosted data path lives in one place — which is the precondition for moving it.

flowchart LR
    ACT["Scorecard<br/>GitHub Action"]

    subgraph produce["Producer · cron/ — AWS (live 2026-09-06)"]
        direction TB
        CTL["batch controller"] --> WKR["scan workers"]
        WKR --> COR[("weekly results")]
        COR --> S3[("S3 buckets")]
    end

    subgraph serve["Serving tier · api/ — AWS"]
        direction TB
        BKT[("results bucket")]
        API["results API"]
        BKT --> API
        API -- "publish" --> BKT
    end

    COR -- "copied" --> BKT
    ACT -- "POST · Sigstore-verified" --> API
    API --> CDN{{"CDN<br/>api.scorecard.dev"}}
    CDN --> VW["scorecard.dev<br/>viewer"]
    CDN --> BDG["img.shields.io<br/>badge"]
    CDN --> MCP["scorecard-mcp,<br/>other API clients"]

Arrows follow results, not requests. One thing the picture is worth reading carefully for: the Action's POST path is the only one that refreshes the edge promptly, because the cache purge fires from the publish handler; batch-produced results reach clients as the cache expires.

They share a repository and, so far, no code. Converging them is the point of landing them together; it has not happened yet and should be done as its own change rather than incidentally (designs C11 and W10, in migrate-batch-pipeline and migrate-api).

Where the effort is going. Both trees are actively developed. The migration described in scorecard#5208 is complete, and api/ and cron/ are held to the same rules as the rest of the repository.

Getting Started

Prerequisites

Common to everything here:

  • Go matching go.mod (1.25.x). If builds fail with a Go tool version mismatch across many stdlib packages, a stray GOROOT is the cause — prefix commands with env -u GOROOT.

Each component adds its own — an object store and SCM token, ko and protoc, a cluster and its cloud dependencies. They are listed in that component's README, linked from Usage below.

Installation
git clone https://github.com/ossf/scorecard-infra
cd scorecard-infra
make build     # go build ./...
make help      # every target, grouped

Building an individual component:

Component Where
Results API — the service that ships api/README.md
Batch scanning pipeline cron/README.md

Usage

Batch scanning pipeline

cron/ is the batch scanning pipeline behind the weekly public Scorecard scan of 1M+ repositories. It was imported from ossf/scorecard with its full commit history — 466 commits dating to 2020 — because the operational history is the runbook: quota workarounds, shard sizing, SQS ack deadlines, and result formatting migrations exist only as commit messages. cron/initial-graft.md records the terms of that import, including what could not come across and where to trace it.

Production is AWS (live 2026-09-06). See:

  • cron/README.md for the pipeline components, the scan inventories and how to add a repository to the weekly scan, and the build targets.
  • deploy/cron/README.md for the AWS batch plane deployment: infrastructure, configuration, apply order, and verification.
Results API (api.scorecard.dev)

The service behind api.scorecard.dev and api.securityscorecards.dev, imported from ossf/scorecard-webapp with its full history. This is the API that ships. It is what every live consumer already talks to: the project website's result viewer, img.shields.io, the Scorecard GitHub Action's upload path, and scorecard-mcp.

The website that consumed it stays in ossf/scorecard-webapp, which remains its home; only the Go API moved.

Endpoints, unchanged by the migration:

Endpoint Behavior
GET /projects/{platform}/{org}/{repo} Published result; ?commit= for a pinned one
GET /projects/{platform}/{org}/{repo}/badge 302 to img.shields.io (a redirect — it renders nothing itself)
POST /projects/{platform}/{org}/{repo} Publish a result, after verifying its Sigstore certificate, transparency-log entry, and originating workflow

Found a bug in the API? Open an issue in this repository. Bugs in the viewer at scorecard.dev belong in ossf/scorecard-webapp.

See api/README.md for the tree layout, build targets, image publication, and the two counterintuitive things about this tree — and deploy/api/README.md for the deployment itself.

Results API server

cmd/scorecard-api and internal/ hold a separate, provider-agnostic implementation of the results contract. It is not the deployment path, and nothing in production uses it — what ships is api/, above. It is documented in cmd/scorecard-api/README.md.

Provider-agnostic migration

Scorecard's hosted infrastructure is being migrated to a new home. The purpose of the work in docs/research/ is to make that migration provider-agnostic — portable across any cloud or self-hosted target rather than tied to a single provider — with the smallest reliable footprint.

  • docs/research/data-infra.md — the reconciled reference design: object store, serving tier, interchange format, public dataset, and cost transparency. Written to be proposable to the Scorecard Infrastructure Working Group; it is not an official artifact yet, and nothing in it is committed.
  • docs/research/infra-seed-0.md and infra-seed-1.md — the two research passes it reconciles (component-selection breadth; correctness/protocol critique and data model).

This is the arc that gives the two systems above a shared destination. Getting the pipeline out of ossf/scorecard is the precondition for it, not the work itself.

Roadmap

The infrastructure migration — what is moving, what has already moved, and what it means for consumers — is announced and tracked in ossf/scorecard#5208.

Backlog ownership across the Scorecard repositories, including which issues belong here, is in #77.

Per-change plans live in openspec/changes/; openspec list shows their live status.

Contributing

Contributions are welcome. For detailed contributing guidelines — including how to add a repository to the weekly scan, and the rules that apply to the cron/ tree — please see CONTRIBUTING.md.

make build     # go build ./...
make test      # go test ./... -race
make lint      # golangci-lint run ./...   (config in .golangci.yml, aligned with ossf/scorecard)
make help      # every target, grouped — including the pipeline targets

An S3-compatible integration test runs when SCORECARD_TEST_S3_URL is set (e.g. a local self-hosted S3-compatible store), and is skipped otherwise.

This repository is developed spec-first with OpenSpec; see AGENTS.md for conventions and openspec/ for the specs and change proposals.

Participation is governed by our Code of Conduct.

License

Distributed under the Apache-2.0 License. See LICENSE for more information.

Data served from Scorecard is licensed under CDLA Permissive 2.0.

Contact

Project Link: https://github.com/ossf/scorecard-infra

Acknowledgements

Directories

Path Synopsis
api
app/generated/restapi
Package restapi OpenSSF Scorecard API
Package restapi OpenSSF Scorecard API
app/server/internal/cdn
Package cdn implements clients for CDN operations.
Package cdn implements clients for CDN operations.
cmd
scorecard-api command
Command scorecard-api is a cloud-agnostic, self-hosted OpenSSF Scorecard results API server.
Command scorecard-api is a cloud-agnostic, self-hosted OpenSSF Scorecard results API server.
cron
config
Package config defines the configuration values for the cron job.
Package config defines the configuration values for the cron job.
data
Package data contains util fns for reading and writing data handled by the cron job.
Package data contains util fns for reading and writing data handled by the cron job.
internal/bq command
Package main implements the BQ transfer job.
Package main implements the BQ transfer job.
internal/cdn
Package cdn implements clients for CDN operations.
Package cdn implements clients for CDN operations.
internal/cii command
Package main implements the PubSub controller.
Package main implements the PubSub controller.
internal/controller command
Package main implements the PubSub controller.
Package main implements the PubSub controller.
internal/data/add command
Package main adds new project repositories to the projects file.
Package main adds new project repositories to the projects file.
internal/data/validate command
Package main validates an input file with a list of projects.
Package main validates an input file with a list of projects.
internal/githubserver command
Package main implements the GitHub token server.
Package main implements the GitHub token server.
internal/provenance
Package provenance reports which Scorecard engine produced a result.
Package provenance reports which Scorecard engine produced a result.
internal/pubsub
Package pubsub handles interactions with PubSub framework.
Package pubsub handles interactions with PubSub framework.
internal/shuffle command
Package main implements the PubSub controller.
Package main implements the PubSub controller.
internal/webhook command
Package main implements a Webhook which runs a bash script when called.
Package main implements a Webhook which runs a bash script when called.
internal/worker command
Package main implements cron worker job.
Package main implements cron worker job.
monitoring
Package monitoring defines exporters to be used by opencensus package in the cron job.
Package monitoring defines exporters to be used by opencensus package in the cron job.
worker
Package worker implements the generic cron worker logic.
Package worker implements the generic cron worker logic.
internal
config
Package config loads all server configuration from the environment (design D10, 12-factor).
Package config loads all server configuration from the environment (design D10, 12-factor).
fallback
Package fallback fetches an existing latest Scorecard result from an upstream Scorecard API (e.g.
Package fallback fetches an existing latest Scorecard result from an upstream Scorecard API (e.g.
flags
Package flags is the single seam for runtime feature flags (design FF1/FF5).
Package flags is the single seam for runtime feature flags (design FF1/FF5).
httpapi
Package httpapi implements the Scorecard-webapp GET contract over the orchestrator (design D1/D11): GET /projects/{host}/{org}/{repo} (+ ?commit=), /badge, /capabilities, /health, and /readyz.
Package httpapi implements the Scorecard-webapp GET contract over the orchestrator (design D1/D11): GET /projects/{host}/{org}/{repo} (+ ?commit=), /badge, /capabilities, /health, and /readyz.
model
Package model defines a lean in-repo mirror of the OpenSSF Scorecard JSON2 wire format (Result), the repository reference the server routes on (RepoRef), and the provenance metadata (Provenance) that the server declares for every response.
Package model defines a lean in-repo mirror of the OpenSSF Scorecard JSON2 wire format (Result), the repository reference the server routes on (RepoRef), and the provenance metadata (Provenance) that the server declares for every response.
orchestrator
Package orchestrator is the read-through cache and the central serve-vs-scan seam (design D2).
Package orchestrator is the read-through cache and the central serve-vs-scan seam (design D2).
scan
Package scan generates Scorecard results on demand by wrapping pkg/scorecard.Run (design D8).
Package scan generates Scorecard results on demand by wrapping pkg/scorecard.Run (design D8).
store
Package store persists Scorecard results in a cloud-agnostic object store via gocloud.dev/blob.
Package store persists Scorecard results in a cloud-agnostic object store via gocloud.dev/blob.
tokens
Package tokens manages SCM credentials and rate limiting for live scans (design D8).
Package tokens manages SCM credentials and rate limiting for live scans (design D8).

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL