caddystoragevalkey

Caddy x Valkey

This project does not affiliate with Caddy nor Valkey. It only extends Caddy with custom code to integrate the valkey storage into Caddy. Logo Copyright belongs to the corresponding project.

Caddy Storage Valkey

Caddy storage backend module with the native valkey golang client and caddy, nothing else.

This module is still in development. Breaking changes will likely come. No stability checks yet.

Example

{
    storage valkey {
        address 127.0.0.1:6379
    }
}

mydomain.localhost {
    respond "Hello World" 200
}

Why?

Mainly for just building it and integrating the native valkey golang client as a caddy storage module.

Thanks to @gamalan (gamalan/caddy-tlsredis) for showing what a possible implementation for this kind of database could look like.

Installation

Download a caddy binary from caddyserver.com with this package included here.

CLI Download (experimental)

This is equal to the version above but replaces your existing binary with the new one including the package.

Caddy has a feature to add packages to your current installation by running the following command:

caddy add-package github.com/oltdaniel/caddy-storage-valkey

DIY Route

Build a custom binary of the latest caddy release with this module enabled.

CADDY_VERSION=latest xcaddy build --with github.com/oltdaniel/caddy-storage-valkey
./caddy run

Configuration Interface

storage module valkey

[!IMPORTANT]
All important client options should be exposed to the config. If there are any missing that could have any purposes in this specific use-case, please open an Issue.

Examples

# Connecting to single valkey node
storage valkey {
    # Server specific connection information can be passed in url format or as seperate config options
    # See https://github.com/redis/redis-specifications/blob/1252427cdbc497f66a7f8550c6b5f2f35367dc92/uri/redis.txt
    url valkey://localhost:6379/0

    lock_majority 1
    disable_client_cache true
}

storage valkey {
    # Server address can be passed as single entry or list instead of url
    address 127.0.0.1:6379

    db 0

    lock_majority 1
    disable_client_cache true
}

# Connecting with specific user to single node
storage valkey {
    url valkey://caddy:pleasechangeme@localhost:6382
}

storage valkey {
    address localhost:6382

    username caddy
    password pleasechangeme
}

# Connecting to TLS single node
storage valkey {
    url valkeys://localhost:6380

    tls_insecure false
    tls_min_version tlsv1.2

    # Any certificate or key can be passed as a PEM string or filepath as described in the table
    tls_ca_cert <<CACERT
    -----BEGIN CERTIFICATE-----
    ...
    -----END CERTIFICATE-----
    CACERT
}

storage valkey {
    url valkeys://localhost:6380

    tls_insecure false
    tls_min_version tlsv1.2

    # Any certificate or key can be passed as a PEM string or filepath as described in the table
    tls_ca_cert tests/ca.crt
}


# Connecting to TLS client auth single node
storage valkey {
    url valkeys://localhost:6381

    tls_insecure false
    tls_min_version tlsv1.2

    tls_ca_cert <<CACERT
    -----BEGIN CERTIFICATE-----
    ...
    -----END CERTIFICATE-----
    CACERT
    tls_client_cert <<CLICERT
    -----BEGIN CERTIFICATE-----
    ...
    -----END CERTIFICATE-----
    CLICERT
    tls_client_key <<CLIKEY
    -----BEGIN PRIVATE KEY-----
    ...
    -----END PRIVATE KEY-----
    CLIKEY
}

# Connecting to standalone valkey with replicas
storage valkey {
    url valkey://localhost:6379/0

    replica {
        localhost:6376
    }

    lock_majority 1
    disable_client_cache true
    send_to_replicas readonly
}

# Connecting to valkey cluster
storage valkey {
    # See https://github.com/redis/redis-specifications/blob/1252427cdbc497f66a7f8550c6b5f2f35367dc92/uri/redis.txt
    url valkey://localhost:7001?addr=localhost:7002&addr=localhost:7003

    shuffle_init true

    lock_majority 2
    disable_client_cache true
}

# Connecting to valkey sentinels
storage valkey {
    # See https://github.com/redis/redis-specifications/blob/1252427cdbc497f66a7f8550c6b5f2f35367dc92/uri/redis.txt
    url valkey://localhost:7001?addr=localhost:7002&addr=localhost:7003

    sentinel_master_set my_master

    lock_majority 2
    disable_client_cache true
}

Values

More?

Do you have a neat way of using this library in your Caddyfile? Feel free to submit it.

Internals

We use the most simple commands in order to make this work and avoid managing any extra structures. Each file is stored as a Hash, with value, last_modified and size in order to store the content of the file and its metadata without any additional serialization. Walking through directories is simply done by doing an Scan and processing of the records in order to return valid results. This means, there is no additional command to repair any internal structures, as there are only Valkey native data structures and mostly single commands for a single action.

The Lock structure is handled by the sub-package valkeylock of the Valkey Go Client Library and some essential aspects are exposed via the configuration.

Resources

• Valkey Go Client Library
• Old Caddy storage redis module

Development

Clone, create example config and run with xcaddy.

git clone https://github.com/oltdaniel/caddy-storage-valkey.git
cd caddy-storage-valkey

CADDY_VERSION=master xcaddy run
# or
CADDY_VERSION=master xcaddy build --with github.com/oltdaniel/caddy-storage-valkey=.
./caddy run

Additionally, there is a docker-compose.yml which contains a demo setup for many different valkey server setups that can be used for testing.

Testing Performance

In order to test locking and the load the setup can handle, there is a testing script in ./scripts/generate-benchmark.sh which will generate an Caddyfile with a huger number of domains for which internally signed certificates are generated with a lifetime of 1 hour and a storage cleanup intervall of 60 seconds, to stress this storage module.

A test with 10.000 domains, showed the cleanup to take about 90 seconds to finish. There haven't been any long running tests yet or other more extreme tests.

Testing TLS

In order to test the TLS feature locally, there is a small script ./scripts/generate-test-certs.sh which generates all the necessary certificates for testing. The ports in the examples above match the correct Container which is already configured to be used for tetsing each scenario.

License