netx

package
v0.22.0-alpha.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Dec 15, 2020 License: BSD-3-Clause Imports: 16 Imported by: 0

README

Package github.com/ooni/probe-engine/netx

OONI extensions to the net and net/http packages. This code is used by ooni/probe-engine as a low level library to collect network, DNS, and HTTP events occurring during OONI measurements.

This library contains replacements for commonly used standard library interfaces that facilitate seamless network measurements. By using such replacements, as opposed to standard library interfaces, we can:

  • save the timing of HTTP events (e.g. received response headers)
  • save the timing and result of every Connect, Read, Write, Close operation
  • save the timing and result of the TLS handshake (including certificates)

By default, this library uses the system resolver. In addition, it is possible to configure alternative DNS transports and remote servers. We support DNS over UDP, DNS over TCP, DNS over TLS (DoT), and DNS over HTTPS (DoH). When using an alternative transport, we are also able to intercept and save DNS messages, as well as any other interaction with the remote server (e.g., the result of the TLS handshake for DoT and DoH).

This package is a fork of github.com/ooni/netx.

Documentation

Overview

Package netx contains code to perform network measurements.

This library contains replacements for commonly used standard library interfaces that facilitate seamless network measurements. By using such replacements, as opposed to standard library interfaces, we can:

* save the timing of HTTP events (e.g. received response headers) * save the timing and result of every Connect, Read, Write, Close operation * save the timing and result of the TLS handshake (including certificates)

By default, this library uses the system resolver. In addition, it is possible to configure alternative DNS transports and remote servers. We support DNS over UDP, DNS over TCP, DNS over TLS (DoT), and DNS over HTTPS (DoH). When using an alternative transport, we are also able to intercept and save DNS messages, as well as any other interaction with the remote server (e.g., the result of the TLS handshake for DoT and DoH).

We described the design and implementation of the most recent version of this package at <https://github.com/ooni/probe-engine/issues/359>. Such issue also links to a previous design document.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func NewDefaultCertPool added in v0.20.0

func NewDefaultCertPool() *x509.CertPool

NewDefaultCertPool returns a copy of the default x509 certificate pool. This function panics on failure.

Types

type Config added in v0.17.0

type Config struct {
	BaseResolver        Resolver             // default: system resolver
	BogonIsError        bool                 // default: bogon is not error
	ByteCounter         *bytecounter.Counter // default: no explicit byte counting
	CacheResolutions    bool                 // default: no caching
	CertPool            *x509.CertPool       // default: use vendored gocertifi
	ContextByteCounting bool                 // default: no implicit byte counting
	DNSCache            map[string][]string  // default: cache is empty
	DialSaver           *trace.Saver         // default: not saving dials
	Dialer              Dialer               // default: dialer.DNSDialer
	FullResolver        Resolver             // default: base resolver + goodies
	HTTP3Dialer         HTTP3Dialer          // default: dialer.HTTP3DNSDialer
	HTTP3Enabled        bool                 // default: disabled
	HTTPSaver           *trace.Saver         // default: not saving HTTP
	Logger              Logger               // default: no logging
	NoTLSVerify         bool                 // default: perform TLS verify
	ProxyURL            *url.URL             // default: no proxy
	ReadWriteSaver      *trace.Saver         // default: not saving read/write
	ResolveSaver        *trace.Saver         // default: not saving resolves
	TLSConfig           *tls.Config          // default: attempt using h2
	TLSDialer           TLSDialer            // default: dialer.TLSDialer
	TLSSaver            *trace.Saver         // default: not saving TLS
}

Config contains configuration for creating a new transport. When any field of Config is nil/empty, we will use a suitable default.

We use different savers for different kind of events such that the user of this library can choose what to save.

type DNSClient added in v0.17.0

type DNSClient struct {
	Resolver
	// contains filtered or unexported fields
}

DNSClient is a DNS client. It wraps a Resolver and it possibly also wraps an HTTP client, but only when we're using DoH.

func NewDNSClient added in v0.17.0

func NewDNSClient(config Config, URL string) (DNSClient, error)

NewDNSClient creates a new DNS client. The config argument is used to create the underlying Dialer and/or HTTP transport, if needed. The URL argument describes the kind of client that we want to make:

- if the URL is `doh://powerdns`, `doh://google` or `doh://cloudflare` or the URL starts with `https://`, then we create a DoH client.

- if the URL is “ or `system:///`, then we create a system client, i.e. a client using the system resolver.

- if the URL starts with `udp://`, then we create a client using a resolver that uses the specified UDP endpoint.

We return error if the URL does not parse or the URL scheme does not fall into one of the cases described above.

If config.ResolveSaver is not nil and we're creating an underlying resolver where this is possible, we will also save events.

func NewDNSClientWithOverrides added in v0.18.0

func NewDNSClientWithOverrides(config Config, URL, hostOverride, SNIOverride string) (DNSClient, error)

NewDNSClientWithOverrides creates a new DNS client, similar to NewDNSClient, with the option to override the default Hostname and SNI.

func (DNSClient) CloseIdleConnections added in v0.17.0

func (c DNSClient) CloseIdleConnections()

CloseIdleConnections closes idle connections, if any.

type Dialer

type Dialer interface {
	DialContext(ctx context.Context, network, address string) (net.Conn, error)
}

Dialer is the definition of dialer assumed by this package.

func NewDialer

func NewDialer(config Config) Dialer

NewDialer creates a new Dialer from the specified config

type HTTP3Dialer added in v0.20.0

type HTTP3Dialer interface {
	Dial(network, addr string, tlsCfg *tls.Config, cfg *quic.Config) (quic.EarlySession, error)
}

HTTP3Dialer is the definition of a dialer for HTTP3 transport assumed by this package.

func NewHTTP3Dialer added in v0.20.0

func NewHTTP3Dialer(config Config) HTTP3Dialer

NewHTTP3Dialer creates a new DNS Dialer for HTTP3 transport, with the resolver from the specified config

type HTTPRoundTripper added in v0.17.0

type HTTPRoundTripper interface {
	RoundTrip(req *http.Request) (*http.Response, error)
	CloseIdleConnections()
}

HTTPRoundTripper is the definition of http.HTTPRoundTripper used by this package.

func NewHTTPTransport

func NewHTTPTransport(config Config) HTTPRoundTripper

NewHTTPTransport creates a new HTTPRoundTripper. You can further extend the returned HTTPRoundTripper before wrapping it into an http.Client.

type Logger added in v0.17.0

type Logger interface {
	Debugf(format string, v ...interface{})
	Debug(message string)
}

Logger is the logger assumed by this package

type Resolver added in v0.17.0

type Resolver interface {
	LookupHost(ctx context.Context, hostname string) (addrs []string, err error)
	Network() string
	Address() string
}

Resolver is the interface we expect from a resolver

func NewResolver

func NewResolver(config Config) Resolver

NewResolver creates a new resolver from the specified config

type TLSDialer added in v0.17.0

type TLSDialer interface {
	DialTLSContext(ctx context.Context, network, address string) (net.Conn, error)
}

TLSDialer is the definition of a TLS dialer assumed by this package.

func NewTLSDialer added in v0.17.0

func NewTLSDialer(config Config) TLSDialer

NewTLSDialer creates a new TLSDialer from the specified config

Directories

Path Synopsis
Package archival contains data formats used for archival.
Package archival contains data formats used for archival.
Package errorx contains error extensions
Package errorx contains error extensions
Package httptransport contains HTTP transport extensions.
Package httptransport contains HTTP transport extensions.
Package selfcensor contains code that triggers censorship.
Package selfcensor contains code that triggers censorship.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL