numaflow

package module

v0.5.3 Latest Latest Go to latest Published: Jul 8, 2022 License: Apache-2.0 Imports: 2 Imported by: 0

Details

Valid go.mod file
Redistributable license
Tagged version
Stable version
Learn more about best practices

Repository

github.com/numaproj/numaflow

Links

README ¶

Numaflow

Summary

Numaflow is a Kubernetes-native platform for running massive parallel data processing and streaming jobs.

A Numaflow Pipeline is implemented as a Kubernetes custom resource, and consists of one or more source, data processing, and sink vertices.

Numaflow installs in less than a minute and is easier and cheaper for simple data processing applications than full-featured stream processing platforms.

Key Features

Kubernetes-native: If you know Kubernetes, you already know 90% of what you need to use Numaflow.
Language agnostic: Use your favorite programming language.
Exactly-Once semantics: No input element is duplicated or lost even as pods are rescheduled or restarted.

Roadmap

Auto-scaling with back-pressure: Each vertex automatically scales from zero to whatever is needed.
Data aggregation (e.g. group-by)

Resources

Documentation ¶

Constants ¶

This section is empty.

Variables ¶

This section is empty.

Functions ¶

This section is empty.

Types ¶

type Version ¶

type Version struct {
	Version      string
	BuildDate    string
	GitCommit    string
	GitTag       string
	GitTreeState string
	GoVersion    string
	Compiler     string
	Platform     string
}

Version contains version information

func GetVersion ¶

func GetVersion() Version

GetVersion returns the version information

func (Version) String ¶

func (v Version) String() string

String outputs the version as a string

Source Files ¶

View all Source files

version.go

Directories ¶

Path	Synopsis
cmd
commands
controllers
cmd
isbsvc
isbsvc/installer
pipeline
vertex
watchdog
pkg
apis/numaflow/v1alpha1 +kubebuilder:object:generate=true +groupName=numaflow.numaproj.io +k8s:deepcopy-gen=package,register +k8s:openapi-gen=true	+kubebuilder:object:generate=true +groupName=numaflow.numaproj.io +k8s:deepcopy-gen=package,register +k8s:openapi-gen=true
apis/proto/daemon Package daemon is a reverse proxy.	Package daemon is a reverse proxy.
client/clientset/versioned This package has the automatically generated clientset.	This package has the automatically generated clientset.
client/clientset/versioned/fake This package has the automatically generated fake clientset.	This package has the automatically generated fake clientset.
client/clientset/versioned/scheme This package contains the scheme of the automatically generated clientset.	This package contains the scheme of the automatically generated clientset.
client/clientset/versioned/typed/numaflow/v1alpha1 This package has the automatically generated typed clients.	This package has the automatically generated typed clients.
client/clientset/versioned/typed/numaflow/v1alpha1/fake Package fake has the automatically generated clients.	Package fake has the automatically generated clients.
client/informers/externalversions
client/informers/externalversions/internalinterfaces
client/informers/externalversions/numaflow
client/informers/externalversions/numaflow/v1alpha1
client/listers/numaflow/v1alpha1
daemon/client
daemon/server
daemon/server/service
isb
isb/forward Package forward does the Read (fromBuffer) -> Process (UDF) -> Forward (toBuffers) -> Ack (fromBuffer) loop.	Package forward does the Read (fromBuffer) -> Process (UDF) -> Forward (toBuffers) -> Ack (fromBuffer) loop.
isb/jetstream
isb/redis
isb/simplebuffer
isb/testutils
isbsvc
isbsvc/clients
metrics
shared/expr
shared/logging
shared/queue
shared/tls
shared/util
sinks
sinks/kafka
sinks/logger
sinks/udsink
sources
sources/generator Package generator contains an implementation of a in memory generator that generates payloads in json format.	Package generator contains an implementation of a in memory generator that generates payloads in json format.
sources/http
sources/kafka
udf
udf/applier
udf/builtin
udf/builtin/cat
udf/builtin/filter
watermark/fetch
watermark/generic
watermark/processor Package processor is the smallest processor entity for which the watermark will strictly monotonically increase.	Package processor is the smallest processor entity for which the watermark will strictly monotonically increase.
watermark/publish
watermark/store
watermark/store/jetstream Package jetstream package implements the watermark progression using Jetstream as the KV store.	Package jetstream package implements the watermark progression using Jetstream as the KV store.
watermark/store/noop
server
apis
apis/v1
cmd
routes
test
e2e-api command
fixtures

?	: This menu
/	: Search site
f or F	: Jump to
y or Y	: Canonical URL