numaflow

package module

v1.7.0 Latest Latest Go to latest Published: Nov 11, 2025 License: Apache-2.0 Imports: 2 Imported by: 0

Details

Valid go.mod file
Redistributable license
Tagged version
Stable version
Learn more about best practices

Repository

github.com/numaproj/numaflow

Links

README ¶

Numaflow

Welcome to Numaflow! A Kubernetes-native, serverless platform for running scalable and reliable event-driven applications. Numaflow decouples event sources and sinks from the processing logic, allowing each component to independently auto-scale based on demand. With out-of-the-box sources and sinks, and built-in observability, developers can focus on their processing logic without worrying about event consumption, writing boilerplate code, or operational complexities. Each step of the pipeline can be written in any programming language, offering unparalleled flexibility in using the best programming language for each step and ease of using the languages you are most familiar with.

Numaflow, created by the Intuit Argo team to address community needs for continuous event processing, leverages their expertise to deliver a scalable and robust, serverless platform for event-driven applications.

Numaflow Pipeline

Use Cases

Event driven applications: Process events as they happen, e.g., updating inventory and sending customer notifications in e-commerce.
Real time analytics: Analyze data instantly, e.g., social media analytics, observability data processing.
Inference on streaming data: Perform real-time predictions, e.g., anomaly detection.
Workflows running in a streaming manner.

Key Features

Kubernetes-native: If you know Kubernetes, you already know how to use Numaflow.
Serverless: Focus on your code and let the system scale up and down based on demand.
Language agnostic: Use your favorite programming language.
Exactly-Once semantics: No input element is duplicated or lost even as pods are rescheduled or restarted.
Auto-scaling with back-pressure: Each vertex automatically scales from zero to whatever is needed.

Data Integrity Guarantees

Minimally provide at-least-once semantics
Provide exactly-once semantics for unbounded and near real-time data sources
Preserving order is not required

Roadmap

Metadata propagation and Open Telemetry Tracing, Deprecate monitor container (1.7)

Demo

Resources

Documentation ¶

Constants ¶

This section is empty.

Variables ¶

This section is empty.

Functions ¶

This section is empty.

Types ¶

type Version ¶

type Version struct {
	Version      string
	BuildDate    string
	GitCommit    string
	GitTag       string
	GitTreeState string
	GoVersion    string
	Compiler     string
	Platform     string
}

Version contains version information

func GetVersion ¶

func GetVersion() Version

GetVersion returns the version information

func (Version) String ¶

func (v Version) String() string

String outputs the version as a string

Source Files ¶

View all Source files

version.go

Directories ¶

Path	Synopsis
cmd
commands
hack
crdgen command
gen-openapi-spec command
json-schema command
pkg
apis/numaflow/v1alpha1 +kubebuilder:object:generate=true +groupName=numaflow.numaproj.io +k8s:deepcopy-gen=package,register +k8s:openapi-gen=true	+kubebuilder:object:generate=true +groupName=numaflow.numaproj.io +k8s:deepcopy-gen=package,register +k8s:openapi-gen=true
apis/proto
apis/proto/daemon Package daemon is a reverse proxy.	Package daemon is a reverse proxy.
apis/proto/isb
apis/proto/mvtxdaemon Package mvtxdaemon is a reverse proxy.	Package mvtxdaemon is a reverse proxy.
apis/proto/watermark
client/clientset/versioned
client/clientset/versioned/fake This package has the automatically generated fake clientset.	This package has the automatically generated fake clientset.
client/clientset/versioned/scheme This package contains the scheme of the automatically generated clientset.	This package contains the scheme of the automatically generated clientset.
client/clientset/versioned/typed/numaflow/v1alpha1 This package has the automatically generated typed clients.	This package has the automatically generated typed clients.
client/clientset/versioned/typed/numaflow/v1alpha1/fake Package fake has the automatically generated clients.	Package fake has the automatically generated clients.
client/informers/externalversions
client/informers/externalversions/internalinterfaces
client/informers/externalversions/numaflow
client/informers/externalversions/numaflow/v1alpha1
client/listers/numaflow/v1alpha1
daemon/client Package client is used to create the daemon service client.	Package client is used to create the daemon service client.
daemon/server
daemon/server/service Package service is built for querying metadata and to expose it over daemon service.	Package service is built for querying metadata and to expose it over daemon service.
daemon/server/service/rater Package rater provides the functionality to calculate the processing rate of each vertex partition.	Package rater provides the functionality to calculate the processing rate of each vertex partition.
daemon/server/service/runtime
daemon/server/service/watermark
isbsvc
metrics
mvtxdaemon/client Package client is used to create the MonoVertex daemon service client.	Package client is used to create the MonoVertex daemon service client.
mvtxdaemon/server
mvtxdaemon/server/service
mvtxdaemon/server/service/rater
mvtxdaemon/server/service/runtime
reconciler Package reconciler defines implementations of the Reconciler interface defined at sigs.k8s.io/controller-runtime/pkg/reconcile.Reconciler.	Package reconciler defines implementations of the Reconciler interface defined at sigs.k8s.io/controller-runtime/pkg/reconcile.Reconciler.
reconciler/cmd
reconciler/isbsvc
reconciler/isbsvc/installer
reconciler/monovertex
reconciler/monovertex/scaling Package scaling provides the autoscaling capability for MonoVertex objects.	Package scaling provides the autoscaling capability for MonoVertex objects.
reconciler/pipeline
reconciler/servingpipeline
reconciler/validator
reconciler/vertex
reconciler/vertex/scaling Package scaling provides the autoscaling capability for Vertex objects.	Package scaling provides the autoscaling capability for Vertex objects.
shared/clients/nats
shared/clients/nats/test
shared/ewma
shared/expr
shared/health-status-code
shared/logging
shared/queue
shared/tls
shared/util
webhook Package webhook is used for running a validating admission webhook.	Package webhook is used for running a validating admission webhook.
webhook/cmd
webhook/validator
server Package server is used for running UX backend server.	Package server is used for running UX backend server.
apis
apis/v1
authn
authz
cmd/server
cmd/serversecretinit
common
routes
test
api-e2e
e2e-api command
fixtures

?	: This menu
/	: Search site
f or F	: Jump to
y or Y	: Canonical URL