README
¶
Gitaly
Quick Links: Roadmap | Want to Contribute? |
Gitaly is a Git RPC service for handling all the Git calls made by GitLab.
For documentation generated from the protobuf definitions (in proto/ directory),
see Gitaly RPC documentation.
To see where it fits in please look at GitLab architecture.
Project Goals
Fault-tolerant horizontal scaling of Git storage in GitLab, and particularly, on GitLab.com.
Current Status
All application code accesses Git repositories via Gitaly.
Besides "Git over RPC" functionality, Gitaly also offers an optional high-availability solution.
We are building features according to our roadmap.
Installation
Most users won't install Gitaly on its own. It is already included in your GitLab installation.
Gitaly requires Go 1.24. Run make to compile the executables
required by Gitaly.
Gitaly uses git. Versions 2.47.0 and newer are supported.
Configuration
The administration and reference guide is documented in the GitLab project.
Contributing
See CONTRIBUTING.md and a list of quick win issues.
Name
Gitaly is a tribute to Git and the town of Aly. Where the town of Aly has zero inhabitants most of the year we would like to reduce the number of disk operations to zero for most actions. It doesn't hurt that it sounds like Italy, the capital of which is the destination of all roads. All Git actions in GitLab end up in Gitaly.
Design
The Gitaly and Gitaly cluster documentation details the architecture and design of Gitaly, including a list of known Gitaly consumers.
High Availability
Gitaly offers a High Availability solution known as Gitaly Cluster (product documentation).
- In its current iteration, client traffic goes through Praefect, which then replicates data to multiple Gitaly servers, and stores state in a PostgreSQL database (see Design above).
- We are working on a new distributed replication solution referred to as Raft, notably removing the need for Praefect and its database, and offering stricter consistency guarantees. See this epic for details on the new design and its progress.
Further reading
More about the project and its processes is detailed in the docs.
Distributed Tracing
Gitaly supports distributed tracing through LabKit using OpenTracing APIs.
By default, no tracing implementation is linked into the binary, but different OpenTracing providers can be linked in using build tags/build constraints. This can be done by setting the BUILD_TAGS make variable.
For more details of the supported providers, see LabKit, but as an example, for Jaeger tracing support, include the tags: BUILD_TAGS="tracer_static tracer_static_jaeger".
make BUILD_TAGS="tracer_static tracer_static_jaeger"
Once Gitaly is compiled with an opentracing provider, the tracing configuration is configured via the GITLAB_TRACING environment variable.
For example, to configure Jaeger, you could use the following command:
GITLAB_TRACING=opentracing://jaeger ./gitaly config.toml
Continuous Profiling
Gitaly supports Continuous Profiling through LabKit using Stackdriver Profiler.
For more information on how to set it up, see the LabKit monitoring docs.
Training Videos
The Gitaly Training and Resources Playlist contains videos that give an overview into how Gitaly works, as well as dive deep into different parts of Gitaly and even Git.
Presentations
- TheConf talk on Scaling GitLab Git storage with Gitaly, 2019-08-16
- Infrastructure Team Update 2017-05-11
- Gitaly Basics, 2017-05-01
- Git Paris meetup, 2017-02-22 a high-level overview of what our plans are and where we are.
Documentation
¶
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func UnpackAuxiliaryBinaries ¶
func UnpackAuxiliaryBinaries(destinationDir string, shouldInclude func(binaryName string) bool) error
UnpackAuxiliaryBinaries unpacks the packed auxiliary binaries of Gitaly into destination directory.
Gitaly invoking auxiliary binaries across different releases is a source of backwards compatibility issues. The calling protocol may change and cause issues if we don't carefully maintain the compatibility. Major version changing the module path also causes problems for gob encoding as it effectively changes the name of every type. To avoid having to maintain backwards compatibility between the different Gitaly binaries, we want to pin a given gitaly binary to only ever call the auxiliary binaries of the same build. We achieve this by packing the auxiliary binaries in the main gitaly binary and unpacking them on start to a temporary directory we can call them from. This way updating the gitaly binaries on the disk is atomic and a running gitaly can't call auxiliary binaries from a different version.
Types ¶
This section is empty.
Source Files
Directories
¶
| Path | Synopsis |
|---|---|
|
gitaly
command
|
|
|
gitaly-backup
command
|
|
|
gitaly-blackbox
command
|
|
|
gitaly-debug
command
|
|
|
gitaly-gpg
command
|
|
|
gitaly-hooks
command
|
|
|
gitaly-lfs-smudge
command
|
|
|
gitaly-ssh
command
|
|
|
gitaly-wrapper
command
|
|
|
praefect
command
|
|
|
internal
|
|
|
backoff
Package backoff implements exponential backoff mechanism based on gRPC's backoff algorithm https://github.com/grpc/grpc/blob/master/doc/connection-backoff.md
|
Package backoff implements exponential backoff mechanism based on gRPC's backoff algorithm https://github.com/grpc/grpc/blob/master/doc/connection-backoff.md |
|
bundleuri
Package bundleuri is used to enable the use [Bundle-URI] when the client clones/fetches from the repository.
|
Package bundleuri is used to enable the use [Bundle-URI] when the client clones/fetches from the repository. |
|
cache
Package cache supplies background workers for periodically cleaning the cache folder on all storages listed in the config file.
|
Package cache supplies background workers for periodically cleaning the cache folder on all storages listed in the config file. |
|
dontpanic
Package dontpanic provides function wrappers and supervisors to ensure that wrapped code does not panic and cause program crashes.
|
Package dontpanic provides function wrappers and supervisors to ensure that wrapped code does not panic and cause program crashes. |
|
git/pktline
Package pktline implements utility functions for working with the Git pkt-line format.
|
Package pktline implements utility functions for working with the Git pkt-line format. |
|
gitaly/storage
Package storage contains the storage layer of Gitaly.
|
Package storage contains the storage layer of Gitaly. |
|
gitaly/storage/mode
Package mode contains the file modes that are supported by the storage.
|
Package mode contains the file modes that are supported by the storage. |
|
gitaly/storage/mode/permission
Package permission defines constants for Posix permission bits.
|
Package permission defines constants for Posix permission bits. |
|
gitaly/storage/storagemgr/partition/log
positions.go
|
positions.go |
|
grpc/backchannel
Package backchannel implements connection multiplexing that allows for invoking gRPC methods from the server to the client.
|
Package backchannel implements connection multiplexing that allows for invoking gRPC methods from the server to the client. |
|
grpc/proxy
Package proxy provides a reverse proxy handler for gRPC.
|
Package proxy provides a reverse proxy handler for gRPC. |
|
praefect
Package praefect is a Gitaly reverse proxy for transparently routing gRPC calls to a set of Gitaly services.
|
Package praefect is a Gitaly reverse proxy for transparently routing gRPC calls to a set of Gitaly services. |
|
praefect/datastore
Package datastore provides data models and datastore persistence abstractions for tracking the state of repository replicas.
|
Package datastore provides data models and datastore persistence abstractions for tracking the state of repository replicas. |
|
praefect/datastore/advisorylock
Package advisorylock contains the lock IDs of all advisory locks used in Praefect.
|
Package advisorylock contains the lock IDs of all advisory locks used in Praefect. |
|
praefect/datastore/glsql
Package glsql (Gitaly SQL) is a helper package to work with plain SQL queries.
|
Package glsql (Gitaly SQL) is a helper package to work with plain SQL queries. |
|
streamcache
Package streamcache provides a cache for large blobs (in the order of gigabytes).
|
Package streamcache provides a cache for large blobs (in the order of gigabytes). |
|
unarycache
Package unarycache allows you to cache responses for unary gRPC messages.
|
Package unarycache allows you to cache responses for unary gRPC messages. |
|
proto
|
|
|
Package streamio contains wrappers intended for turning gRPC streams that send/receive messages with a []byte field into io.Writers and io.Readers.
|
Package streamio contains wrappers intended for turning gRPC streams that send/receive messages with a []byte field into io.Writers and io.Readers. |
|
tools
|
|
|
binary-size-checker
command
|
|
|
govulncheck-filter
command
|
|
|
module-updater
command
|
|
|
noticegen
command
|
|
|
panic-parser
command
|
|
|
protoc-gen-gitaly-lint
command
Command protoc-gen-gitaly-lint is designed to be used as a protobuf compiler plugin to verify Gitaly processes are being followed when writing RPC's.
|
Command protoc-gen-gitaly-lint is designed to be used as a protobuf compiler plugin to verify Gitaly processes are being followed when writing RPC's. |
|
protoc-gen-gitaly-protolist
command
Command protoc-gen-gitaly-protolist is designed to be used as a protobuf compiler to generate a list of protobuf files via a publicly accessible variable.
|
Command protoc-gen-gitaly-protolist is designed to be used as a protobuf compiler to generate a list of protobuf files via a publicly accessible variable. |
|
test-boot
command
|