Documentation
¶
Overview ¶
Package model contains the shared interfaces and data structures.
Criteria for adding a type to this package ¶
This package should contain two types:
1. important interfaces that are shared by several packages within the codebase, with the objective of separating unrelated pieces of code and making unit testing easier;
2. important pieces of data that are shared across different packages (e.g., the representation of a Measurement).
In general, this package should not contain logic, unless this logic is strictly related to data structures and we cannot implement this logic elsewhere.
Content of this package ¶
The following list (which may not always be up-to-date) summarizes the categories of types that currently belong here and names the files in which they are implemented:
- experiment.go: generic definition of a network experiment and all the required support types;
- keyvaluestore.go: generic definition of a key-value store, used in several places across the codebase;
- logger.go: generic definition of an apex/log compatible logger, used in several places across the codebase;
- measurement.go: data type representing the result of a network measurement, used in many many places;
- netx.go: network extension interfaces and data used everywhere we need to perform network operations;
- ooapi.go: types to communicate with the OONI API.
Index ¶
- Constants
- Variables
- func ArchivalNewHTTPHeadersMap(header http.Header) (out map[string]ArchivalScrubbedMaybeBinaryString)
- func ErrorToStringOrOK(err error) string
- func MeasurementFormatTimeNowUTC() string
- func ScrubMeasurement(m *Measurement, currentIP string) error
- type ArchivalBinaryData
- type ArchivalDNSAnswer
- type ArchivalDNSLookupResult
- type ArchivalExtSpec
- type ArchivalHTTPHeader
- type ArchivalHTTPRequest
- type ArchivalHTTPRequestResult
- type ArchivalHTTPResponse
- type ArchivalHTTPTor
- type ArchivalNetworkEvent
- type ArchivalOpenVPNHandshakeResult
- type ArchivalOpenVPNOptions
- type ArchivalScrubbedMaybeBinaryString
- type ArchivalTCPConnectResult
- type ArchivalTCPConnectStatus
- type ArchivalTLSOrQUICHandshakeResult
- type DNSDecoder
- type DNSEncoder
- type DNSQuery
- type DNSResponse
- type DNSTransport
- type DatabaseMeasurement
- type DatabaseMeasurementURLNetwork
- type DatabaseNetwork
- type DatabaseResult
- type DatabaseResultNetwork
- type DatabaseURL
- type DebugLogger
- type Dialer
- type DialerWrapper
- type Experiment
- type ExperimentArgs
- type ExperimentBuilder
- type ExperimentCallbacks
- type ExperimentInputProcessor
- type ExperimentMeasurer
- type ExperimentOptionInfo
- type ExperimentSession
- type ExperimentTarget
- type ExperimentTargetLoader
- type ExperimentTargetLoaderConfig
- type ExperimentTargetLoaderSession
- type GeoIPASNLookupper
- type GeoIPASNLookupperFunc
- type HTTPClient
- type HTTPSSvc
- type HTTPTransport
- type InfoLogger
- type InputPolicy
- type KeyValueStore
- type LocationASN
- type LocationProvider
- type Logger
- type Measurement
- type MeasurementInput
- type MeasurementSummaryKeys
- type MeasurementSummaryKeysProvider
- type MeasuringNetwork
- type OOAPICheckInConfig
- type OOAPICheckInConfigWebConnectivity
- type OOAPICheckInInfoWebConnectivity
- type OOAPICheckInResult
- type OOAPICheckInResultConfig
- type OOAPICheckInResultNettests
- type OOAPICheckReportIDResponse
- type OOAPICollectorOpenResponse
- type OOAPICollectorUpdateRequest
- type OOAPICollectorUpdateResponse
- type OOAPILoginAuth
- type OOAPILoginCredentials
- type OOAPIMeasurementMeta
- type OOAPIMeasurementMetaConfig
- type OOAPIProbeMetadata
- type OOAPIRegisterRequest
- type OOAPIRegisterResponse
- type OOAPIReportTemplate
- type OOAPIService
- type OOAPITorTarget
- type OOAPIURLInfo
- type OOAPIVPNConfig
- type OOAPIVPNProviderConfig
- type PerformanceTestKeys
- type PrinterCallbacks
- type QUICDialer
- type QUICDialerWrapper
- type ReadableDatabase
- type Resolver
- type RunType
- type Saver
- type SimpleDialer
- type Submitter
- type THDNSResult
- type THHTTPRequestResult
- type THIPInfo
- type THRequest
- type THResponse
- type THTCPConnectResult
- type THTLSHandshakeResult
- type TLSConn
- type TLSDialer
- type TLSHandshaker
- type Trace
- type UDPLikeConn
- type UDPListener
- type UnderlyingNetwork
- type UploadedTotalCount
- type WritableDatabase
Constants ¶
const ( // InputOrQueryBackend indicates that the experiment requires // external input to run and that this kind of input is URLs // from the citizenlab/test-lists repository. If this input // not provided to the experiment, then the code that runs the // experiment is supposed to fetch from URLs from OONI's backends. InputOrQueryBackend = InputPolicy("or_query_backend") // InputStrictlyRequired indicates that the experiment // requires input and we currently don't have an API for // fetching such input. Therefore, either the user specifies // input or the experiment will fail for the lack of input. InputStrictlyRequired = InputPolicy("strictly_required") // InputOptional indicates that the experiment handles input, // if any; otherwise it fetchs input/uses a default. InputOptional = InputPolicy("optional") // InputNone indicates that the experiment does not want any // input and ignores the input if provided with it. InputNone = InputPolicy("none") // We gather input from StaticInput and SourceFiles. If there is // input, we return it. Otherwise, we return an internal static // list of inputs to be used with this experiment. InputOrStaticDefault = InputPolicy("or_static_default") )
const ( // HTTPHeaderAccept is the Accept header used for measuring. HTTPHeaderAccept = "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8" // HTTPHeaderAcceptLanguage is the Accept-Language header used for measuring. HTTPHeaderAcceptLanguage = "en-US,en;q=0.9" // HTTPHeaderUserAgent is the User-Agent header used for measuring. The current header // is 43.03% of the browser population as of 2025-07-05 according to the // https://www.useragents.me/ webpage. HTTPHeaderUserAgent = "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.10 Safari/605.1.1" )
Headers we use for measuring.
const ( // DefaultProbeASN is the default probe ASN as a number. DefaultProbeASN uint = 0 // DefaultProbeCC is the default probe CC. DefaultProbeCC = "ZZ" // DefaultProbeIP is the default probe IP. DefaultProbeIP = "127.0.0.1" // DefaultProbeNetworkName is the default probe network name. DefaultProbeNetworkName = "" // DefaultResolverASN is the default resolver ASN. DefaultResolverASN uint = 0 // DefaultResolverIP is the default resolver IP. DefaultResolverIP = "127.0.0.2" // DefaultResolverNetworkName is the default resolver network name. DefaultResolverNetworkName = "" // DefaultGeoipDB is the default geoip database. DefaultGeoipDB = "engine-internal" )
const ( // DefaultCategoryCode is the default category code to use // when a URL's category code is unknown. DefaultCategoryCode = "MISC" // DefaultCountryCode is the default country code to use // when a URL's country code is unknown. // // We use XX because it is the same string that the URL // prioritization code would return. // // See https://github.com/ooni/backend/blob/f7a93f477111c7278424996815b91e6300d66b83/api/ooniapi/prio.py#L182 DefaultCountryCode = "XX" )
const ( // OOAPIReportDefaultDataFormatVersion is the default data format version. // // See https://github.com/ooni/spec/tree/master/data-formats#history. OOAPIReportDefaultDataFormatVersion = "0.2.0" // DefaultFormat is the default format OOAPIReportDefaultFormat = "json" )
const ( // RunTypeManual indicates that the user manually run `ooniprobe run`. Command // line tools such as miniooni should always use this run type. RunTypeManual = RunType("manual") // RunTypeTimed indicates that the user run `ooniprobe run unattended`, which // is the correct way to run ooniprobe from scripts and cronjobs. RunTypeTimed = RunType("timed") )
const ( // THIPInfoFlagResolvedByProbe indicates that the probe has // resolved this IP address. THIPInfoFlagResolvedByProbe = 1 << iota // THIPInfoFlagResolvedByTH indicates that the test helper // has resolved this IP address. THIPInfoFlagResolvedByTH // THIPInfoFlagIsBogon indicates that the address is a bogon THIPInfoFlagIsBogon // THIPInfoFlagValidForDomain indicates that an IP address // is valid for the domain because it works with TLS THIPInfoFlagValidForDomain )
const MeasurementDateFormat = "2006-01-02 15:04:05"
MeasurementDateFormat is the data format used by a measurement.
const Scrubbed = `[scrubbed]`
Scrubbed is the string that replaces IP addresses.
const THDNSNameError = "dns_name_error"
THDNSNameError is the error returned by the control on NXDOMAIN
Variables ¶
var ( // ArchivalExtDNS is the version of df-002-dnst.md ArchivalExtDNS = ArchivalExtSpec{Name: "dnst", V: 0} // ArchivalExtNetevents is the version of df-008-netevents.md ArchivalExtNetevents = ArchivalExtSpec{Name: "netevents", V: 0} // ArchivalExtHTTP is the version of df-001-httpt.md ArchivalExtHTTP = ArchivalExtSpec{Name: "httpt", V: 0} // ArchivalExtTCPConnect is the version of df-005-tcpconnect.md ArchivalExtTCPConnect = ArchivalExtSpec{Name: "tcpconnect", V: 0} // ArchivalExtTLSHandshake is the version of df-006-tlshandshake.md ArchivalExtTLSHandshake = ArchivalExtSpec{Name: "tlshandshake", V: 0} // ArchivalExtTunnel is the version of df-009-tunnel.md ArchivalExtTunnel = ArchivalExtSpec{Name: "tunnel", V: 0} )
var ( // HTTPUnexpectedStatusCode indicates that we re not getting // the expected (range of) HTTP status code(s). HTTPUnexpectedStatusCode = "http_unexpected_status_code" // HTTPUnexpectedRedirectURL indicates that the redirect URL // returned by the server is not the expected one. HTTPUnexpectedRedirectURL = "http_unexpected_redirect_url" )
Additional strings used to report HTTP errors. They're currently only used by experiment/whatsapp but may be used by more experiments in the future. They must be addressable (i.e., var and not const) because experiments typically want to take their addresses to fill fields with `string|null` type.
var ( // DefaultProbeASNString is the default probe ASN as a string. DefaultProbeASNString = fmt.Sprintf("AS%d", DefaultProbeASN) // DefaultResolverASNString is the default resolver ASN as a string. DefaultResolverASNString = fmt.Sprintf("AS%d", DefaultResolverASN) )
var ErrInvalidBinaryDataFormat = errors.New("model: invalid binary data format")
ErrInvalidBinaryDataFormat is the format returned when marshaling and unmarshaling binary data and the value of "format" is unknown.
var ErrInvalidProbeIP = errors.New("model: invalid probe IP")
ErrInvalidProbeIP indicates that we're dealing with a string that is not the valid serialization of an IP address.
var ErrNoAvailableTestHelpers = errors.New("no available helpers")
ErrNoAvailableTestHelpers is emitted when there are no available test helpers.
Functions ¶
func ArchivalNewHTTPHeadersMap ¶ added in v3.19.0
func ArchivalNewHTTPHeadersMap(header http.Header) (out map[string]ArchivalScrubbedMaybeBinaryString)
ArchivalNewHTTPHeadersMap creates a map representation of HTTP headers
func ErrorToStringOrOK ¶ added in v3.15.0
ErrorToStringOrOK emits "ok" on "<nil>"" values for success.
func MeasurementFormatTimeNowUTC ¶ added in v3.19.0
func MeasurementFormatTimeNowUTC() string
MeasurementFormatTimeNowUTC formats the current time as UTC using the MeasurementDateFormat.
func ScrubMeasurement ¶ added in v3.16.2
func ScrubMeasurement(m *Measurement, currentIP string) error
ScrubMeasurement removes [currentIP] from [m] by rewriting it in place while preserving the underlying types
Types ¶
type ArchivalBinaryData ¶ added in v3.19.0
type ArchivalBinaryData []byte
ArchivalBinaryData is a wrapper for bytes that serializes the enclosed data using the specific ooni/spec data format for binary data.
See https://github.com/ooni/spec/blob/master/data-formats/df-001-httpt.md#maybebinarydata.
func (ArchivalBinaryData) MarshalJSON ¶ added in v3.19.0
func (value ArchivalBinaryData) MarshalJSON() ([]byte, error)
MarshalJSON implements json.Marshaler.
func (*ArchivalBinaryData) UnmarshalJSON ¶ added in v3.19.0
func (value *ArchivalBinaryData) UnmarshalJSON(raw []byte) error
UnmarshalJSON implements json.Unmarshaler.
type ArchivalDNSAnswer ¶
type ArchivalDNSAnswer struct {
ASN int64 `json:"asn,omitempty"`
ASOrgName string `json:"as_org_name,omitempty"`
AnswerType string `json:"answer_type"`
Hostname string `json:"hostname,omitempty"`
IPv4 string `json:"ipv4,omitempty"`
IPv6 string `json:"ipv6,omitempty"`
TTL *uint32 `json:"ttl"`
}
ArchivalDNSAnswer is a DNS answer.
type ArchivalDNSLookupResult ¶
type ArchivalDNSLookupResult struct {
Answers []ArchivalDNSAnswer `json:"answers"`
Engine string `json:"engine"`
Failure *string `json:"failure"`
GetaddrinfoError int64 `json:"getaddrinfo_error,omitempty"`
Hostname string `json:"hostname"`
QueryType string `json:"query_type"`
RawResponse []byte `json:"raw_response,omitempty"`
Rcode int64 `json:"rcode,omitempty"`
ResolverHostname *string `json:"resolver_hostname"`
ResolverPort *string `json:"resolver_port"`
ResolverAddress string `json:"resolver_address"`
T0 float64 `json:"t0,omitempty"`
T float64 `json:"t"`
Tags []string `json:"tags"`
TransactionID int64 `json:"transaction_id,omitempty"`
}
ArchivalDNSLookupResult is the result of a DNS lookup.
See https://github.com/ooni/spec/blob/master/data-formats/df-002-dnst.md.
type ArchivalExtSpec ¶
ArchivalExtSpec describes a data format extension
func (ArchivalExtSpec) AddTo ¶
func (spec ArchivalExtSpec) AddTo(m *Measurement)
AddTo adds the current ExtSpec to the specified measurement
type ArchivalHTTPHeader ¶
type ArchivalHTTPHeader [2]ArchivalScrubbedMaybeBinaryString
ArchivalHTTPHeader is a single HTTP header.
func ArchivalNewHTTPHeadersList ¶ added in v3.19.0
func ArchivalNewHTTPHeadersList(source http.Header) (out []ArchivalHTTPHeader)
ArchivalNewHTTPHeadersList constructs a new ArchivalHTTPHeader list given HTTP headers.
func (*ArchivalHTTPHeader) UnmarshalJSON ¶
func (ahh *ArchivalHTTPHeader) UnmarshalJSON(data []byte) error
UnmarshalJSON implements json.Unmarshaler.
type ArchivalHTTPRequest ¶
type ArchivalHTTPRequest struct {
Body ArchivalScrubbedMaybeBinaryString `json:"body"`
BodyIsTruncated bool `json:"body_is_truncated"`
HeadersList []ArchivalHTTPHeader `json:"headers_list"`
Headers map[string]ArchivalScrubbedMaybeBinaryString `json:"headers"`
Method string `json:"method"`
Tor ArchivalHTTPTor `json:"tor"`
Transport string `json:"x_transport"`
URL string `json:"url"`
}
ArchivalHTTPRequest contains an HTTP request.
Headers are a map in Web Connectivity data format but we have added support for a list since January 2020.
type ArchivalHTTPRequestResult ¶
type ArchivalHTTPRequestResult struct {
Network string `json:"network,omitempty"`
Address string `json:"address,omitempty"`
ALPN string `json:"alpn,omitempty"`
Failure *string `json:"failure"`
Request ArchivalHTTPRequest `json:"request"`
Response ArchivalHTTPResponse `json:"response"`
T0 float64 `json:"t0,omitempty"`
T float64 `json:"t"`
Tags []string `json:"tags"`
TransactionID int64 `json:"transaction_id,omitempty"`
}
ArchivalHTTPRequestResult is the result of sending an HTTP request.
See https://github.com/ooni/spec/blob/master/data-formats/df-001-httpt.md.
type ArchivalHTTPResponse ¶
type ArchivalHTTPResponse struct {
Body ArchivalScrubbedMaybeBinaryString `json:"body"`
BodyIsTruncated bool `json:"body_is_truncated"`
Code int64 `json:"code"`
HeadersList []ArchivalHTTPHeader `json:"headers_list"`
Headers map[string]ArchivalScrubbedMaybeBinaryString `json:"headers"`
// The following fields are not serialised but are useful to simplify
// analysing the measurements in telegram, whatsapp, etc.
Locations []string `json:"-"`
}
ArchivalHTTPResponse contains an HTTP response.
Headers are a map in Web Connectivity data format but we have added support for a list since January 2020.
type ArchivalHTTPTor ¶
type ArchivalHTTPTor struct {
ExitIP *string `json:"exit_ip"`
ExitName *string `json:"exit_name"`
IsTor bool `json:"is_tor"`
}
ArchivalHTTPTor contains Tor information.
type ArchivalNetworkEvent ¶
type ArchivalNetworkEvent struct {
Address string `json:"address,omitempty"`
Failure *string `json:"failure"`
NumBytes int64 `json:"num_bytes,omitempty"`
Operation string `json:"operation"`
Proto string `json:"proto,omitempty"`
T0 float64 `json:"t0,omitempty"`
T float64 `json:"t"`
TransactionID int64 `json:"transaction_id,omitempty"`
Tags []string `json:"tags,omitempty"`
}
ArchivalNetworkEvent is a network event. It contains all the possible fields and most fields are optional. They are only added when it makes sense for them to be there _and_ we have data to show.
See https://github.com/ooni/spec/blob/master/data-formats/df-008-netevents.md.
type ArchivalOpenVPNHandshakeResult ¶ added in v3.23.0
type ArchivalOpenVPNHandshakeResult struct {
Endpoint string `json:"endpoint"`
Failure *string `json:"failure"`
HandshakeTime float64 `json:"handshake_time,omitempty"`
IP string `json:"ip"`
Port int `json:"port"`
Transport string `json:"transport"`
Provider string `json:"provider"`
OpenVPNOptions ArchivalOpenVPNOptions `json:"openvpn_options"`
T0 float64 `json:"t0,omitempty"`
T float64 `json:"t"`
Tags []string `json:"tags"`
TransactionID int64 `json:"transaction_id,omitempty"`
}
ArchivalOpenVPNHandshakeResult contains the result of a OpenVPN handshake.
type ArchivalOpenVPNOptions ¶ added in v3.23.0
type ArchivalOpenVPNOptions struct {
Auth string `json:"auth,omitempty"`
Cipher string `json:"cipher,omitempty"`
Compression string `json:"compression,omitempty"`
}
ArchivalOpenVPNOptions is a subset of [vpnconfig.OpenVPNOptions] that we want to include in the archived result.
type ArchivalScrubbedMaybeBinaryString ¶ added in v3.19.0
type ArchivalScrubbedMaybeBinaryString string
ArchivalScrubbedMaybeBinaryString is a possibly-binary string. When the string is valid UTF-8 we serialize it as itself. Otherwise, we use the binary data format defined by https://github.com/ooni/spec/blob/master/data-formats/df-001-httpt.md#maybebinarydata
As the name implies, the data contained by this type is scrubbed to remove IPv4 and IPv6 addresses and endpoints during JSON serialization, to make it less likely that OONI leaks IP addresses in textual or binary fields such as HTTP headers and bodies.
func (ArchivalScrubbedMaybeBinaryString) MarshalJSON ¶ added in v3.19.0
func (value ArchivalScrubbedMaybeBinaryString) MarshalJSON() ([]byte, error)
MarshalJSON implements json.Marshaler.
func (*ArchivalScrubbedMaybeBinaryString) UnmarshalJSON ¶ added in v3.19.0
func (value *ArchivalScrubbedMaybeBinaryString) UnmarshalJSON(rawData []byte) error
UnmarshalJSON implements json.Unmarshaler.
type ArchivalTCPConnectResult ¶
type ArchivalTCPConnectResult struct {
IP string `json:"ip"`
Port int `json:"port"`
Status ArchivalTCPConnectStatus `json:"status"`
T0 float64 `json:"t0,omitempty"`
T float64 `json:"t"`
Tags []string `json:"tags"`
TransactionID int64 `json:"transaction_id,omitempty"`
}
ArchivalTCPConnectResult contains the result of a TCP connect.
See https://github.com/ooni/spec/blob/master/data-formats/df-005-tcpconnect.md.
type ArchivalTCPConnectStatus ¶
type ArchivalTCPConnectStatus struct {
Blocked *bool `json:"blocked,omitempty"`
Failure *string `json:"failure"`
Success bool `json:"success"`
}
ArchivalTCPConnectStatus is the status of ArchivalTCPConnectResult.
type ArchivalTLSOrQUICHandshakeResult ¶
type ArchivalTLSOrQUICHandshakeResult struct {
Network string `json:"network"`
Address string `json:"address"`
CipherSuite string `json:"cipher_suite"`
Failure *string `json:"failure"`
SoError *string `json:"so_error,omitempty"`
NegotiatedProtocol string `json:"negotiated_protocol"`
NoTLSVerify bool `json:"no_tls_verify"`
PeerCertificates []ArchivalBinaryData `json:"peer_certificates"`
ServerName string `json:"server_name"`
OuterServerName string `json:"outer_server_name,omitempty"`
ECHConfig string `json:"echconfig,omitempty"`
T0 float64 `json:"t0,omitempty"`
T float64 `json:"t"`
Tags []string `json:"tags"`
TLSVersion string `json:"tls_version"`
TransactionID int64 `json:"transaction_id,omitempty"`
}
ArchivalTLSOrQUICHandshakeResult is the result of a TLS or QUIC handshake.
See https://github.com/ooni/spec/blob/master/data-formats/df-006-tlshandshake.md
type DNSDecoder ¶
type DNSDecoder interface {
// DecodeResponse decodes a DNS response message.
//
// Arguments:
//
// - data is the raw reply
//
// This function fails if we cannot parse data as a DNS
// message or the message is not a response.
//
// Regarding the returned response, remember that the Rcode
// MAY still be nonzero (this method does not treat a nonzero
// Rcode as an error when parsing the response).
DecodeResponse(data []byte, query DNSQuery) (DNSResponse, error)
}
The DNSDecoder decodes DNS responses.
type DNSEncoder ¶
type DNSEncoder interface {
// Encode transforms its arguments into a serialized DNS query.
//
// Every time you call Encode, you get a new DNSQuery value
// using a query ID selected at random.
//
// Serialization to bytes is lazy to acommodate DNS transports that
// do not need to serialize and send bytes, e.g., getaddrinfo.
//
// You serialize to bytes using DNSQuery.Bytes. This operation MAY fail
// if the domain name cannot be packed into a DNS message (e.g., it is
// too long to fit into the message).
//
// Arguments:
//
// - domain is the domain for the query (e.g., x.org);
//
// - qtype is the query type (e.g., dns.TypeA);
//
// - padding is whether to add padding to the query.
//
// This function will transform the domain into an FQDN is it's not
// already expressed in the FQDN format.
Encode(domain string, qtype uint16, padding bool) DNSQuery
}
The DNSEncoder encodes DNS queries to bytes
type DNSQuery ¶ added in v3.16.0
type DNSQuery interface {
// Domain is the domain we're querying for.
Domain() string
// Type is the query type.
Type() uint16
// Bytes serializes the query to bytes. This function may fail if we're not
// able to correctly encode the domain into a query message.
//
// The value returned by this function WILL be memoized after the first call,
// so you SHOULD create a new DNSQuery if you need to retry a query.
Bytes() ([]byte, error)
// ID returns the query ID.
ID() uint16
}
DNSQuery is an encoded DNS query ready to be sent using a DNSTransport.
type DNSResponse ¶ added in v3.16.0
type DNSResponse interface {
// Query is the query associated with this response.
Query() DNSQuery
// Bytes returns the bytes from which we parsed the query.
Bytes() []byte
// Rcode returns the response's Rcode.
Rcode() int
// DecodeHTTPS returns information gathered from all the HTTPS
// records found inside of this response.
DecodeHTTPS() (*HTTPSSvc, error)
// DecodeLookupHost returns the addresses in the response matching
// the original query type (one of A and AAAA).
DecodeLookupHost() ([]string, error)
// DecodeNS returns all the NS entries in this response.
DecodeNS() ([]*net.NS, error)
// DecodeCNAME returns the first CNAME entry in this response.
DecodeCNAME() (string, error)
}
DNSResponse is a parsed DNS response ready for further processing.
type DNSTransport ¶
type DNSTransport interface {
// RoundTrip sends a DNS query and receives the reply.
RoundTrip(ctx context.Context, query DNSQuery) (DNSResponse, error)
// RequiresPadding returns whether this transport needs padding.
RequiresPadding() bool
// Network is the network of the round tripper (e.g. "dot").
Network() string
// Address is the address of the round tripper (e.g. "1.1.1.1:853").
Address() string
// CloseIdleConnections closes idle connections, if any.
CloseIdleConnections()
}
DNSTransport represents an abstract DNS transport.
type DatabaseMeasurement ¶ added in v3.17.0
type DatabaseMeasurement struct {
ID int64 `db:"measurement_id,omitempty"`
TestName string `db:"test_name"`
StartTime time.Time `db:"measurement_start_time"`
Runtime float64 `db:"measurement_runtime"` // Fractional number of seconds
IsDone bool `db:"measurement_is_done"`
IsUploaded bool `db:"measurement_is_uploaded"`
IsFailed bool `db:"measurement_is_failed"`
FailureMsg sql.NullString `db:"measurement_failure_msg,omitempty"`
IsUploadFailed bool `db:"measurement_is_upload_failed"`
UploadFailureMsg sql.NullString `db:"measurement_upload_failure_msg,omitempty"`
IsRerun bool `db:"measurement_is_rerun"`
ReportID sql.NullString `db:"report_id,omitempty"`
URLID sql.NullInt64 `db:"url_id,omitempty"` // Used to reference URL
MeasurementID sql.NullInt64 `db:"collector_measurement_id,omitempty"`
IsAnomaly sql.NullBool `db:"is_anomaly,omitempty"`
// FIXME we likely want to support JSON. See: https://github.com/upper/db/issues/462
TestKeys string `db:"test_keys"`
ResultID int64 `db:"result_id"`
ReportFilePath sql.NullString `db:"report_file_path,omitempty"`
MeasurementFilePath sql.NullString `db:"measurement_file_path,omitempty"`
}
Database Measurement model
type DatabaseMeasurementURLNetwork ¶ added in v3.17.0
type DatabaseMeasurementURLNetwork struct {
DatabaseMeasurement `db:",inline"`
DatabaseNetwork `db:",inline"`
DatabaseResult `db:",inline"`
DatabaseURL `db:",inline"`
}
MeasurementURLNetwork is used for the JOIN between Measurement and URL
type DatabaseNetwork ¶ added in v3.17.0
type DatabaseNetwork struct {
ID int64 `db:"network_id,omitempty"`
NetworkName string `db:"network_name"`
NetworkType string `db:"network_type"`
IP string `db:"ip"`
ASN uint `db:"asn"`
CountryCode string `db:"network_country_code"`
}
DatabaseNetwork represents a network tested by the user
type DatabaseResult ¶ added in v3.17.0
type DatabaseResult struct {
ID int64 `db:"result_id,omitempty"`
TestGroupName string `db:"test_group_name"`
StartTime time.Time `db:"result_start_time"`
NetworkID int64 `db:"network_id"` // Used to include a Network
Runtime float64 `db:"result_runtime"` // Runtime is expressed in fractional seconds
IsViewed bool `db:"result_is_viewed"`
IsDone bool `db:"result_is_done"`
IsUploaded bool `db:"result_is_uploaded"`
DataUsageUp float64 `db:"result_data_usage_up"`
DataUsageDown float64 `db:"result_data_usage_down"`
MeasurementDir string `db:"measurement_dir"`
}
Database Result model
type DatabaseResultNetwork ¶ added in v3.17.0
type DatabaseResultNetwork struct {
DatabaseResult `db:",inline"`
DatabaseNetwork `db:",inline"`
AnomalyCount uint64 `db:"anomaly_count"`
TotalCount uint64 `db:"total_count"`
TestKeys string `db:"test_keys"`
}
ResultNetwork is used to represent the structure made from the JOIN between the results and networks tables.
type DatabaseURL ¶ added in v3.17.0
type DatabaseURL struct {
ID sql.NullInt64 `db:"url_id,omitempty"`
URL sql.NullString `db:"url"`
CategoryCode sql.NullString `db:"category_code"`
CountryCode sql.NullString `db:"url_country_code"`
}
DatabaseURL represents URLs from the testing lists
type DebugLogger ¶
type DebugLogger = logmodel.DebugLogger
DebugLogger is a logger emitting only debug messages.
type Dialer ¶
type Dialer interface {
// A Dialer is also a SimpleDialer.
SimpleDialer
// CloseIdleConnections closes idle connections, if any.
CloseIdleConnections()
}
Dialer is a SimpleDialer with the possibility of closing open connections.
type DialerWrapper ¶ added in v3.16.0
DialerWrapper is a type that takes in input a Dialer and returns in output a wrapped Dialer.
type Experiment ¶ added in v3.16.0
type Experiment interface {
// KibiBytesReceived accounts for the KibiBytes received by the experiment.
KibiBytesReceived() float64
// KibiBytesSent is like KibiBytesReceived but for the bytes sent.
KibiBytesSent() float64
// Name returns the experiment name.
Name() string
// ReportID returns the open report's ID, if we have opened a report
// successfully before, or an empty string, otherwise.
ReportID() string
// MeasureWithContext measures the given experiment target.
//
// Return value: either a non-nil measurement and a nil error
// or a nil measurement and a non-nil error.
MeasureWithContext(ctx context.Context, target ExperimentTarget) (measurement *Measurement, err error)
// SubmitAndUpdateMeasurementContext submits a measurement and updates the
// fields whose value has changed as part of the submission.
SubmitAndUpdateMeasurementContext(
ctx context.Context, measurement *Measurement) (string, error)
// OpenReportContext will open a report using the given context
// to possibly limit the lifetime of this operation.
OpenReportContext(ctx context.Context) error
}
Experiment is an experiment instance.
type ExperimentArgs ¶ added in v3.17.0
type ExperimentArgs struct {
// Callbacks contains MANDATORY experiment callbacks.
Callbacks ExperimentCallbacks
// Measurement is the MANDATORY measurement in which the experiment
// must write the results of the measurement.
Measurement *Measurement
// Session is the MANDATORY session the experiment can use.
Session ExperimentSession
// Target is the OPTIONAL target we're measuring.
//
// Only richer-input-aware experiments use this field. These experiments
// SHOULD be defensive and handle the case where this field is nil.
Target ExperimentTarget
}
ExperimentArgs contains the arguments passed to an experiment.
type ExperimentBuilder ¶ added in v3.16.0
type ExperimentBuilder interface {
// Interruptible tells you whether this is an interruptible experiment. This kind
// of experiments (e.g. ndt7) may be interrupted mid way.
Interruptible() bool
// InputPolicy returns the experiment input policy.
InputPolicy() InputPolicy
// Options returns information about the experiment's options.
Options() (map[string]ExperimentOptionInfo, error)
// SetOptionAny sets an option whose value is an any value. We will use reasonable
// heuristics to convert the any value to the proper type of the field whose name is
// contained by the key variable. If we cannot convert the provided any value to
// the proper type, then this function returns an error.
SetOptionAny(key string, value any) error
// SetOptionsAny sets options from a map[string]any. See the documentation of
// the SetOptionAny method for more information.
SetOptionsAny(options map[string]any) error
// SetOptionsJSON uses the given [json.RawMessage] to initialize fields
// of the configuration for running the experiment. The [json.RawMessage], if
// not empty, MUST contain a serialization of the experiment config's
// type. An empty [json.RawMessage] will silently be ignored.
SetOptionsJSON(value json.RawMessage) error
// SetCallbacks sets the experiment's interactive callbacks.
SetCallbacks(callbacks ExperimentCallbacks)
// NewExperiment creates the [Experiment] instance.
NewExperiment() Experiment
// NewTargetLoader creates the [ExperimentTargetLoader] instance.
NewTargetLoader(config *ExperimentTargetLoaderConfig) ExperimentTargetLoader
}
ExperimentBuilder builds an experiment.
type ExperimentCallbacks ¶
type ExperimentCallbacks interface {
// OnProgress provides information about the experiment's progress.
//
// The prog field is a number between 0.0 and 1.0 representing progress, where
// 0.0 corresponds to 0% and 1.0 corresponds to 100%.
OnProgress(prog float64, message string)
}
ExperimentCallbacks contains experiment event-handling callbacks.
type ExperimentInputProcessor ¶ added in v3.16.0
ExperimentInputProcessor processes inputs for an experiment.
type ExperimentMeasurer ¶
type ExperimentMeasurer interface {
// ExperimentName returns the experiment name.
ExperimentName() string
// ExperimentVersion returns the experiment version.
ExperimentVersion() string
// Run runs the experiment with the specified context, session,
// measurement, and experiment calbacks. This method should only
// return an error in case the experiment could not run (e.g.,
// a required input is missing). Otherwise, the code should just
// set the relevant OONI error inside of the measurement and
// return nil. This is important because the caller WILL NOT submit
// the measurement if this method returns an error.
Run(ctx context.Context, args *ExperimentArgs) error
}
ExperimentMeasurer is the interface that allows to run a measurement for a specific experiment.
type ExperimentOptionInfo ¶ added in v3.16.0
type ExperimentOptionInfo struct {
// Doc contains the documentation.
Doc string
// Type contains the type.
Type string
// Value contains the current option value.
Value any
}
ExperimentOptionInfo contains info about an experiment option.
type ExperimentSession ¶
type ExperimentSession interface {
// GetTestHelpersByName returns a list of test helpers with the given name.
GetTestHelpersByName(name string) ([]OOAPIService, bool)
// DefaultHTTPClient returns the default HTTPClient used by the session.
DefaultHTTPClient() HTTPClient
// FetchOpenVPNConfig returns vpn config as a serialized JSON or an error.
FetchOpenVPNConfig(ctx context.Context, provider, cc string) (*OOAPIVPNProviderConfig, error)
// FetchPsiphonConfig returns psiphon's config as a serialized JSON or an error.
FetchPsiphonConfig(ctx context.Context) ([]byte, error)
// FetchTorTargets returns the targets for the Tor experiment or an error.
FetchTorTargets(ctx context.Context, cc string) (map[string]OOAPITorTarget, error)
// Logger returns the logger used by the session.
Logger() Logger
// ProbeCC returns the country code.
ProbeCC() string
// ResolverIP returns the resolver's IP.
ResolverIP() string
// TempDir returns the session's temporary directory.
TempDir() string
// TorArgs returns the arguments we should pass to tor when executing it.
TorArgs() []string
// TorBinary returns the path of the tor binary.
TorBinary() string
// TunnelDir is the directory where to store tunnel information.
TunnelDir() string
// UserAgent returns the user agent we should be using when we're fine
// with identifying ourselves as ooniprobe.
UserAgent() string
}
ExperimentSession is the experiment's view of a session.
type ExperimentTarget ¶ added in v3.23.0
type ExperimentTarget interface {
// Category returns the github.com/citizenlab/test-lists category
// code for this piece of richer input.
//
// Return [DefaultCategoryCode] if there's no applicable category code.
Category() string
// Country returns the country code for this
// piece of richer input.
//
// Return [DefaultCountryCode] if there's not applicable country code.
Country() string
// Input returns the experiment input, which is typically a URL.
Input() string
// Options transforms the options contained by this target
// into a []string containing options as they were provided
// using the command line `-O option=value` syntax.
//
// This method MUST NOT serialize all the options whose name
// starts with the "Safe" prefix. This method MUST skip serializing
// sensitive options, non-scalar options, and zero value options.
//
// Consider using the [experimentconfig] package to serialize.
Options() []string
// String MUST return the experiment input.
//
// Implementation note: previously existing code often times treated
// the input as a string and, crucially, printed it using %s. To be
// robust with respect to introducing richer input, we would like the
// code to print in output the same value as before, which possibly
// is processed by the desktop app. This is the reason why we are
// introducing an explicit String() method and why we say that this
// method MUST return the experiment input.
String() string
}
ExperimentTarget contains a target for the experiment to measure.
type ExperimentTargetLoader ¶ added in v3.23.0
type ExperimentTargetLoader interface {
Load(ctx context.Context) ([]ExperimentTarget, error)
}
ExperimentTargetLoader loads targets from local or remote sources.
type ExperimentTargetLoaderConfig ¶ added in v3.23.0
type ExperimentTargetLoaderConfig struct {
// CheckInConfig contains OPTIONAL options for the CheckIn API. If not set, then we'll create a
// default config. If set but there are fields inside it that are not set, then we will set them
// to a default value.
CheckInConfig *OOAPICheckInConfig
// Session is the MANDATORY current measurement session.
Session ExperimentTargetLoaderSession
// StaticInputs contains OPTIONAL input to be added
// to the resulting input list if possible.
StaticInputs []string
// SourceFiles contains OPTIONAL files to read input
// from. Each file should contain a single input string
// per line. We will fail if any file is unreadable
// as well as if any file is empty.
SourceFiles []string
}
ExperimentTargetLoaderConfig is the configuration to create a new ExperimentTargetLoader.
The zero value is not ready to use; please, init the MANDATORY fields.
type ExperimentTargetLoaderSession ¶ added in v3.23.0
type ExperimentTargetLoaderSession interface {
// CheckIn invokes the check-in API.
CheckIn(ctx context.Context, config *OOAPICheckInConfig) (*OOAPICheckInResult, error)
// FetchOpenVPNConfig fetches the OpenVPN experiment configuration.
FetchOpenVPNConfig(ctx context.Context, provider, cc string) (*OOAPIVPNProviderConfig, error)
// Logger returns the logger to use.
Logger() Logger
// ProbeCC returns the probe country code.
ProbeCC() string
}
ExperimentTargetLoaderSession is the session according to ExperimentTargetLoader.
type GeoIPASNLookupper ¶ added in v3.19.0
GeoIPASNlookupper performs ASN lookups.
type GeoIPASNLookupperFunc ¶ added in v3.19.0
GeoIPASNLookupperFunc transforms a func into a GeoIPASNLookupper.
type HTTPClient ¶
HTTPClient is an http.Client-like interface.
type HTTPSSvc ¶
type HTTPSSvc struct {
// ALPN contains the ALPNs inside the HTTPS reply.
ALPN []string
// IPv4 contains the IPv4 hints (which may be empty).
IPv4 []string
// IPv6 contains the IPv6 hints (which may be empty).
IPv6 []string
// Encrypted ClientHello config decoded from base64 to bytes
// (which may be empty).
Ech []byte
}
HTTPSSvc is the reply to an HTTPS DNS query.
type HTTPTransport ¶
type HTTPTransport interface {
// Network returns the network used by the transport, which
// should be one of "tcp" and "udp".
Network() string
// RoundTrip performs the HTTP round trip.
RoundTrip(req *http.Request) (*http.Response, error)
// CloseIdleConnections closes idle connections.
CloseIdleConnections()
}
HTTPTransport is an http.Transport-like structure.
type InfoLogger ¶
type InfoLogger = logmodel.InfoLogger
InfoLogger is a logger emitting debug and infor messages.
type InputPolicy ¶ added in v3.16.0
type InputPolicy string
InputPolicy describes the experiment policy with respect to input. That is whether it requires input, optionally accepts input, does not want input.
type KeyValueStore ¶
type KeyValueStore interface {
// Get gets the value of the given key or returns an
// error if there is no such key or we cannot read
// from the key-value store.
Get(key string) (value []byte, err error)
// Set sets the value of the given key and returns
// whether the operation was successful or not.
Set(key string, value []byte) (err error)
}
KeyValueStore is a generic key-value store.
type LocationASN ¶ added in v3.19.0
LocationASN contains ASN information related to a location.
type LocationProvider ¶ added in v3.17.0
type LocationProvider interface {
ProbeASN() uint
ProbeASNString() string
ProbeCC() string
ProbeIP() string
ProbeNetworkName() string
ResolverIP() string
}
LocationProvider is an interface that returns the current location. The [engine.Session] struct implements this interface.
type Logger ¶
Logger defines the common interface that a logger should have. It is out of the box compatible with `log.Log` in `apex/log`.
var DiscardLogger Logger = logDiscarder{}
DiscardLogger is the default logger that discards its input
func ValidLoggerOrDefault ¶ added in v3.16.0
ValidLoggerOrDefault is a factory that either returns the logger provided as argument, if not nil, or DiscardLogger.
type Measurement ¶
type Measurement struct {
// Annotations contains results annotations
Annotations map[string]string `json:"annotations,omitempty"`
// DataFormatVersion is the version of the data format
DataFormatVersion string `json:"data_format_version"`
// Extensions contains information about the extensions included
// into the test_keys of this measurement.
Extensions map[string]int64 `json:"extensions,omitempty"`
// ID is the locally generated measurement ID
ID string `json:"id,omitempty"`
// Input is the measurement input
Input MeasurementInput `json:"input"`
// InputHashes contains input hashes
InputHashes []string `json:"input_hashes,omitempty"`
// MeasurementStartTime is the time when the measurement started
MeasurementStartTime string `json:"measurement_start_time"`
// MeasurementStartTimeSaved is the moment in time when we
// started the measurement. This is not included into the JSON
// and is only used within the ./internal pkg as a "zero" time.
MeasurementStartTimeSaved time.Time `json:"-"`
// Options contains command line options
Options []string `json:"options,omitempty"`
// ProbeASN contains the probe autonomous system number
ProbeASN string `json:"probe_asn"`
// ProbeCC contains the probe country code
ProbeCC string `json:"probe_cc"`
// ProbeCity contains the probe city
ProbeCity string `json:"probe_city,omitempty"`
// ProbeIP contains the probe IP
ProbeIP string `json:"probe_ip,omitempty"`
// ProbeNetworkName contains the probe network name
ProbeNetworkName string `json:"probe_network_name"`
// ReportID contains the report ID
ReportID string `json:"report_id"`
// ResolverASN is the ASN of the resolver
ResolverASN string `json:"resolver_asn"`
// ResolverIP is the resolver IP
ResolverIP string `json:"resolver_ip"`
// ResolverNetworkName is the network name of the resolver.
ResolverNetworkName string `json:"resolver_network_name"`
// SoftwareName contains the software name
SoftwareName string `json:"software_name"`
// SoftwareVersion contains the software version
SoftwareVersion string `json:"software_version"`
// TestHelpers contains the test helpers. It seems this structure is more
// complex than we would like. In particular, using a map from string to
// string does not fit into the web_connectivity use case. Hence, for now
// we're going to represent this using interface{}. In going forward we
// may probably want to have more uniform test helpers.
TestHelpers map[string]interface{} `json:"test_helpers,omitempty"`
// TestKeys contains the real test result. This field is opaque because
// each experiment will insert here a different structure.
TestKeys interface{} `json:"test_keys"`
// TestName contains the test name
TestName string `json:"test_name"`
// MeasurementRuntime contains the measurement runtime. The JSON name
// is test_runtime because this is the name expected by the OONI backend
// even though that name is clearly a misleading one.
MeasurementRuntime float64 `json:"test_runtime"`
// TestStartTime contains the test start time
TestStartTime string `json:"test_start_time"`
// TestVersion contains the test version
TestVersion string `json:"test_version"`
}
Measurement is a OONI measurement.
This structure is compatible with the definition of the base data format in https://github.com/ooni/spec/blob/master/data-formats/df-000-base.md.
func (*Measurement) AddAnnotation ¶
func (m *Measurement) AddAnnotation(key, value string)
AddAnnotation adds a single annotations to m.Annotations.
func (*Measurement) AddAnnotations ¶
func (m *Measurement) AddAnnotations(input map[string]string)
AddAnnotations adds the annotations from input to m.Annotations.
type MeasurementInput ¶ added in v3.23.0
type MeasurementInput string
MeasurementInput is the input of an OONI measurement.
func (MeasurementInput) MarshalJSON ¶ added in v3.23.0
func (t MeasurementInput) MarshalJSON() ([]byte, error)
MarshalJSON serializes the MeasurementInput.
type MeasurementSummaryKeys ¶ added in v3.21.0
type MeasurementSummaryKeys interface {
// Anomaly returns whether there was a measurement anomaly.
Anomaly() bool
}
MeasurementSummaryKeys is the measurement summary.
type MeasurementSummaryKeysProvider ¶ added in v3.21.0
type MeasurementSummaryKeysProvider interface {
MeasurementSummaryKeys() MeasurementSummaryKeys
}
MeasurementSummaryKeysProvider is the interface that the experiment test keys should implement when they want to have a customized implementation of the MeasurementSummaryKeys. If there is no such implementation, common code for running experiments returns a default implementation whose Anomaly method always returns false when invoked.
type MeasuringNetwork ¶ added in v3.19.0
type MeasuringNetwork interface {
// NewDialerWithoutResolver creates a [Dialer] with error wrapping and without an attached
// resolver, meaning that you MUST pass TCP or UDP endpoint addresses to this dialer.
//
// The [DialerWrapper] arguments wraps the returned dialer in such a way that we can implement
// the legacy [netx] package. New code MUST NOT use this functionality, which we'd like to remove ASAP.
NewDialerWithoutResolver(dl DebugLogger, w ...DialerWrapper) Dialer
// NewParallelDNSOverHTTPSResolver creates a new DNS-over-HTTPS resolver with error wrapping.
NewParallelDNSOverHTTPSResolver(logger DebugLogger, URL string) Resolver
// NewParallelUDPResolver creates a new Resolver using DNS-over-UDP
// that performs parallel A/AAAA lookups during LookupHost.
//
// The address argument is the UDP endpoint address (e.g., 1.1.1.1:53, [::1]:53).
NewParallelUDPResolver(logger DebugLogger, dialer Dialer, address string) Resolver
// NewQUICDialerWithoutResolver creates a [QUICDialer] with error wrapping and without an attached
// resolver, meaning that you MUST pass UDP endpoint addresses to this dialer.
//
// The [QUICDialerWrapper] arguments wraps the returned dialer in such a way
// that we can implement the legacy [netx] package. New code MUST NOT
// use this functionality, which we'd like to remove ASAP.
NewQUICDialerWithoutResolver(
listener UDPListener, logger DebugLogger, w ...QUICDialerWrapper) QUICDialer
// NewStdlibResolver creates a new Resolver with error wrapping using
// getaddrinfo or &net.Resolver{} depending on `-tags netgo`.
NewStdlibResolver(logger DebugLogger) Resolver
// NewTLSHandshakerStdlib creates a new TLSHandshaker with error wrapping
// that is using the go standard library to manage TLS.
NewTLSHandshakerStdlib(logger DebugLogger) TLSHandshaker
// NewTLSHandshakerUTLS creates a new TLS handshaker using
// gitlab.com/yawning/utls for TLS that implements error wrapping.
//
// The id is the address of something like utls.HelloFirefox_55.
//
// Passing a nil `id` will make this function panic.
NewTLSHandshakerUTLS(logger DebugLogger, id *utls.ClientHelloID) TLSHandshaker
// NewUDPListener creates a new UDPListener with error wrapping.
NewUDPListener() UDPListener
}
MeasuringNetwork defines the constructors required for implementing OONI experiments. All these constructors MUST guarantee proper error wrapping to map Go errors to OONI errors as documented by the [netxlite] package. The [*netxlite.Netx] type is currently the default implementation of this interface. This interface SHOULD always be implemented in terms of an UnderlyingNetwork that allows to switch between the host network and [netemx].
type OOAPICheckInConfig ¶
type OOAPICheckInConfig struct {
// Charging indicate whether the phone is charging.
Charging bool `json:"charging"`
// OnWiFi indicate if the phone is connected to a WiFi.
OnWiFi bool `json:"on_wifi"`
// Platform of the probe.
Platform string `json:"platform"`
// ProbeASN is the probe ASN.
ProbeASN string `json:"probe_asn"`
// ProbeCC is the probe country code.
ProbeCC string `json:"probe_cc"`
// RunType indicated whether the run is "timed" or "manual".
RunType RunType `json:"run_type"`
// SoftwareName of the probe.
SoftwareName string `json:"software_name"`
// SoftwareVersion of the probe.
SoftwareVersion string `json:"software_version"`
// WebConnectivity contains WebConnectivity information.
WebConnectivity OOAPICheckInConfigWebConnectivity `json:"web_connectivity"`
}
OOAPICheckInConfig contains config for a checkin API call.
type OOAPICheckInConfigWebConnectivity ¶
type OOAPICheckInConfigWebConnectivity struct {
// CategoryCodes contains an array of category codes
CategoryCodes []string `json:"category_codes"`
}
OOAPICheckInConfigWebConnectivity is the WebConnectivity portion of OOAPICheckInConfig.
type OOAPICheckInInfoWebConnectivity ¶
type OOAPICheckInInfoWebConnectivity struct {
// ReportID is the report ID the probe should use.
ReportID string `json:"report_id"`
// URLs contains the URL to measure.
URLs []OOAPIURLInfo `json:"urls"`
}
OOAPICheckInInfoWebConnectivity contains the WebConnectivity part of OOAPICheckInInfo.
type OOAPICheckInResult ¶ added in v3.17.0
type OOAPICheckInResult struct {
// Conf contains configuration.
Conf OOAPICheckInResultConfig `json:"conf"`
// ProbeASN contains the probe's ASN.
ProbeASN string `json:"probe_asn"`
// ProbeCC contains the probe's CC.
ProbeCC string `json:"probe_cc"`
// Tests contains information about nettests.
Tests OOAPICheckInResultNettests `json:"tests"`
// UTCTime contains the time in UTC.
UTCTime time.Time `json:"utc_time"`
// V is the version.
V int64 `json:"v"`
}
OOAPICheckInResult is the result returned by the checkin API.
type OOAPICheckInResultConfig ¶ added in v3.17.0
type OOAPICheckInResultConfig struct {
// Features contains feature flags.
Features map[string]bool `json:"features"`
// TestHelpers contains test-helpers information.
TestHelpers map[string][]OOAPIService `json:"test_helpers"`
}
OOAPICheckInResultConfig contains configuration.
type OOAPICheckInResultNettests ¶ added in v3.17.0
type OOAPICheckInResultNettests struct {
// WebConnectivity contains WebConnectivity related information.
WebConnectivity *OOAPICheckInInfoWebConnectivity `json:"web_connectivity"`
}
OOAPICheckInResultNettests contains nettests information returned by the checkin API call.
type OOAPICheckReportIDResponse ¶ added in v3.17.0
type OOAPICheckReportIDResponse struct {
Error string `json:"error"`
Found bool `json:"found"`
V int64 `json:"v"`
}
OOAPICheckReportIDResponse is the check-report-id API response.
type OOAPICollectorOpenResponse ¶ added in v3.17.0
type OOAPICollectorOpenResponse struct {
// BackendVersion is the backend version.
BackendVersion string `json:"backend_version"`
// ReportID is the report ID.
ReportID string `json:"report_id"`
// SupportedFormats contains supported formats.
SupportedFormats []string `json:"supported_formats"`
}
OOAPICollectorOpenResponse is the response returned by the open report API.
type OOAPICollectorUpdateRequest ¶ added in v3.17.0
type OOAPICollectorUpdateRequest struct {
// Format is the Content's data format
Format string `json:"format"`
// Content is the actual report
Content any `json:"content"`
}
OOAPICollectorUpdateRequest is a request for the collector update API.
type OOAPICollectorUpdateResponse ¶ added in v3.17.0
type OOAPICollectorUpdateResponse struct {
// MeasurementUID is the measurement UID.
MeasurementUID string `json:"measurement_uid"`
}
OOAPICollectorUpdateResponse is the response from the collector update API.
type OOAPILoginAuth ¶ added in v3.17.0
OOAPILoginAuth contains authentication info
type OOAPILoginCredentials ¶ added in v3.17.0
type OOAPILoginCredentials struct {
Username string `json:"username"`
Password string `json:"password"`
}
OOAPILoginCredentials contains the login credentials
type OOAPIMeasurementMeta ¶ added in v3.17.0
type OOAPIMeasurementMeta struct {
// Fields returned by the API server whenever we are
// calling /api/v1/measurement_meta.
Anomaly bool `json:"anomaly"`
CategoryCode string `json:"category_code"`
Confirmed bool `json:"confirmed"`
Failure bool `json:"failure"`
Input *string `json:"input"`
MeasurementStartTime time.Time `json:"measurement_start_time"`
ProbeASN int64 `json:"probe_asn"`
ProbeCC string `json:"probe_cc"`
ReportID string `json:"report_id"`
Scores string `json:"scores"`
TestName string `json:"test_name"`
TestStartTime time.Time `json:"test_start_time"`
// This field is only included if the user has specified
// the config.Full option, otherwise it's empty.
RawMeasurement string `json:"raw_measurement"`
}
OOAPIMeasurementMeta contains measurement metadata.
type OOAPIMeasurementMetaConfig ¶ added in v3.17.0
type OOAPIMeasurementMetaConfig struct {
// ReportID is the mandatory report ID.
ReportID string
// Full indicates whether we also want the full measurement body.
Full bool
// Input is the optional input.
Input string
}
OOAPIMeasurementMetaConfig contains configuration for GetMeasurementMeta.
type OOAPIProbeMetadata ¶ added in v3.17.0
type OOAPIProbeMetadata struct {
AvailableBandwidth string `json:"available_bandwidth,omitempty"`
DeviceToken string `json:"device_token,omitempty"`
Language string `json:"language,omitempty"`
NetworkType string `json:"network_type,omitempty"`
Platform string `json:"platform"`
ProbeASN string `json:"probe_asn"`
ProbeCC string `json:"probe_cc"`
ProbeFamily string `json:"probe_family,omitempty"`
ProbeTimezone string `json:"probe_timezone,omitempty"`
SoftwareName string `json:"software_name"`
SoftwareVersion string `json:"software_version"`
SupportedTests []string `json:"supported_tests"`
}
OOAPIProbeMetadata contains metadata about a probe. This message is included into a bunch of messages sent to orchestra.
func (OOAPIProbeMetadata) Valid ¶ added in v3.17.0
func (m OOAPIProbeMetadata) Valid() bool
Valid returns true if metadata is valid, false otherwise. Metadata is considered valid if all the mandatory fields are not empty. If a field is marked `json:",omitempty"` in the structure definition, then it's for sure mandatory. The "device_token" field is mandatory only if the platform is "ios" or "android", because there's currently no device token that we know of for desktop devices.
type OOAPIRegisterRequest ¶ added in v3.17.0
type OOAPIRegisterRequest struct {
OOAPIProbeMetadata
Password string `json:"password"`
}
OOAPIRegisterRequest is a request to the register API.
type OOAPIRegisterResponse ¶ added in v3.17.0
type OOAPIRegisterResponse struct {
ClientID string `json:"client_id"`
}
OOAPIRegisterResponse is a reponse from the register API.
type OOAPIReportTemplate ¶ added in v3.17.0
type OOAPIReportTemplate struct {
// DataFormatVersion is unconditionally set to DefaultDataFormatVersion
// and you don't need to be concerned about it.
DataFormatVersion string `json:"data_format_version"`
// Format is unconditionally set to `json` and you don't need
// to be concerned about it.
Format string `json:"format"`
// ProbeASN is the probe's autonomous system number (e.g. `AS1234`)
ProbeASN string `json:"probe_asn"`
// ProbeCC is the probe's country code (e.g. `IT`)
ProbeCC string `json:"probe_cc"`
// SoftwareName is the app name (e.g. `measurement-kit`)
SoftwareName string `json:"software_name"`
// SoftwareVersion is the app version (e.g. `0.9.1`)
SoftwareVersion string `json:"software_version"`
// TestName is the test name (e.g. `ndt`)
TestName string `json:"test_name"`
// TestStartTime contains the test start time
TestStartTime string `json:"test_start_time"`
// TestVersion is the test version (e.g. `1.0.1`)
TestVersion string `json:"test_version"`
}
OOAPIReportTemplate is the template for opening a report
type OOAPIService ¶
type OOAPIService struct {
// Address is the address of the server.
Address string `json:"address"`
// Type is the type of the service.
Type string `json:"type"`
// Front is the front to use with "cloudfront" type entries.
Front string `json:"front,omitempty"`
}
OOAPIService describes a backend service.
The fields of this struct have the meaning described in v2.0.0 of the OONI bouncer specification defined by https://github.com/ooni/spec/blob/master/backends/bk-004-bouncer.md.
type OOAPITorTarget ¶
type OOAPITorTarget struct {
// Address is the address of the target.
Address string `json:"address"`
// Name is the name of the target.
Name string `json:"name"`
// Params contains optional params for, e.g., pluggable transports.
Params map[string][]string `json:"params"`
// Protocol is the protocol to use with the target.
Protocol string `json:"protocol"`
// Source is the source from which we fetched this specific
// target. Whenever the source is non-empty, we will treat
// this specific target as a private target.
Source string `json:"source"`
}
OOAPITorTarget is a target for the tor experiment.
type OOAPIURLInfo ¶
type OOAPIURLInfo struct {
// CategoryCode is the URL's category (e.g., FEXP, POLT, HUMR).
CategoryCode string `json:"category_code"`
// CountryCode is the URL's ISO country code or ZZ for global URLs.
CountryCode string `json:"country_code"`
// URL is the string-serialized URL.
URL string `json:"url"`
}
OOAPIURLInfo contains information on a test lists URL.
func NewOOAPIURLInfoWithDefaultCategoryAndCountry ¶ added in v3.23.0
func NewOOAPIURLInfoWithDefaultCategoryAndCountry(URL string) *OOAPIURLInfo
NewOOAPIURLInfoWithDefaultCategoryAndCountry constructs a new instance of *OOAPIURLInfo with default category and country code.
func (*OOAPIURLInfo) Category ¶ added in v3.23.0
func (o *OOAPIURLInfo) Category() string
Category implements ExperimentTarget.
func (*OOAPIURLInfo) Country ¶ added in v3.23.0
func (o *OOAPIURLInfo) Country() string
Country implements ExperimentTarget.
func (*OOAPIURLInfo) Input ¶ added in v3.23.0
func (o *OOAPIURLInfo) Input() string
Input implements ExperimentTarget.
func (*OOAPIURLInfo) Options ¶ added in v3.23.0
func (o *OOAPIURLInfo) Options() []string
Options implements ExperimentTarget.
func (*OOAPIURLInfo) String ¶ added in v3.23.0
func (o *OOAPIURLInfo) String() string
String implements ExperimentTarget.
type OOAPIVPNConfig ¶ added in v3.23.0
type OOAPIVPNConfig struct {
// CA is the Certificate Authority for the endpoints by this provider.
CA string `json:"ca"`
// Cert is a valid certificate, for providers that use x509 certificate authentication.
Cert string `json:"cert,omitempty"`
// Key is a valid key, for providers that use x509 certificate authentication.
Key string `json:"key,omitempty"`
// Username is a valid username, for providers that use password authentication.
Username string `json:"username,omitempty"`
// Password is a valid password, for providers that use password authentication.
Password string `json:"password,omitempty"`
}
OOAPIVPNConfig contains the configuration needed to start an OpenVPN connection, returned as part of OOAPIVPNProviderConfig.
type OOAPIVPNProviderConfig ¶ added in v3.23.0
type OOAPIVPNProviderConfig struct {
// Provider is the label for this provider.
Provider string `json:"provider,omitempty"`
// Config is the provider-specific VPN Config.
Config *OOAPIVPNConfig `json:"config"`
// Inputs is an array of valid endpoints for this provider.
Inputs []string `json:"endpoints"`
// DateUpdated is when the credential set was last updated in the server database.
DateUpdated time.Time `json:"date_updated"`
}
OOAPIVPNProviderConfig is a minimal valid configuration subset for the openvpn experiment; at the moment it provides credentials valid for endpoints in a provider, and a list of inputs to be tested on this provider.
type PerformanceTestKeys ¶ added in v3.17.0
type PerformanceTestKeys struct {
Upload float64 `json:"upload"`
Download float64 `json:"download"`
Ping float64 `json:"ping"`
Bitrate float64 `json:"median_bitrate"`
}
PerformanceTestKeys is the result summary for a performance test
type PrinterCallbacks ¶
type PrinterCallbacks struct {
Logger
}
PrinterCallbacks is the default event handler
func NewPrinterCallbacks ¶
func NewPrinterCallbacks(logger Logger) PrinterCallbacks
NewPrinterCallbacks returns a new default callback handler
func (PrinterCallbacks) OnProgress ¶
func (d PrinterCallbacks) OnProgress(percentage float64, message string)
OnProgress provides information about an experiment progress.
type QUICDialer ¶
type QUICDialer interface {
// DialContext establishes a new QUIC session using the given
// network and address. The tlsConfig and the quicConfig arguments
// MUST NOT be nil. Returns either the session or an error.
//
// Recommended tlsConfig setup:
//
// - set ServerName to be the SNI;
//
// - set RootCAs to nil (which causes us to use the default cert pool);
//
// - set NextProtos to []string{"h3"}.
//
// Typically, you want to pass `&quic.Config{}` as quicConfig.
DialContext(ctx context.Context, address string,
tlsConfig *tls.Config, quicConfig *quic.Config) (quic.EarlyConnection, error)
// CloseIdleConnections closes idle connections, if any.
CloseIdleConnections()
}
QUICDialer dials QUIC sessions.
type QUICDialerWrapper ¶ added in v3.16.0
type QUICDialerWrapper interface {
WrapQUICDialer(qd QUICDialer) QUICDialer
}
QUICDialerWrapper is a type that takes in input a QUICDialer and returns in output a wrapped QUICDialer.
type ReadableDatabase ¶ added in v3.17.0
type ReadableDatabase interface {
// ListResults return the list of results
//
// Arguments:
//
// Returns either the complete and incomplete database results or an error
ListResults() ([]DatabaseResultNetwork, []DatabaseResultNetwork, error)
// ListMeasurements given a result ID
//
// Arguments:
//
// - resultID is the id of the result to search measurements from
//
// Returns the measurements under the given result or an error
ListMeasurements(resultID int64) ([]DatabaseMeasurementURLNetwork, error)
// GetMeasurementJSON returns a map[string]interface{} given a database and a measurementID
//
// Arguments:
//
// - msmtID is the measurement id to generate JSON
//
// Returns the measurement JSON or an error
GetMeasurementJSON(msmtID int64) (map[string]interface{}, error)
}
ReadableDatabase only supports reading data.
type Resolver ¶
type Resolver interface {
// LookupHost resolves the given hostname to IP addreses. This function SHOULD handle the
// case in which hostname is an IP address by returning a 1-element list containing the hostname,
// for consistency with [net.Resolver] behaviour.
LookupHost(ctx context.Context, hostname string) (addrs []string, err error)
// Network returns the resolver type. It should be one of:
//
// - go: means we're using whatever resolver the Go stdlib uses
// depending on the current build configuration;
//
// - system: means we've been compiled with `CGO_ENABLED=1`
// so we can bypass the go resolver and call getaddrinfo directly;
//
// - udp: is a custom DNS-over-UDP resolver;
//
// - tcp: is a custom DNS-over-TCP resolver;
//
// - dot: is a custom DNS-over-TLS resolver;
//
// - doh: is a custom DNS-over-HTTPS resolver;
//
// - doh3: is a custom DNS-over-HTTP3 resolver.
//
// See https://github.com/ooni/probe/issues/2029#issuecomment-1140805266
// for an explanation of why it would not be proper to call "netgo" the
// resolver we get by default from the standard library.
Network() string
// Address returns the resolver address (e.g., 8.8.8.8:53).
Address() string
// CloseIdleConnections closes idle connections, if any.
CloseIdleConnections()
// LookupHTTPS issues an HTTPS query for a domain.
LookupHTTPS(
ctx context.Context, domain string) (*HTTPSSvc, error)
// LookupNS issues a NS query for a domain.
LookupNS(ctx context.Context, domain string) ([]*net.NS, error)
}
Resolver performs domain name resolutions.
type Saver ¶ added in v3.16.0
type Saver interface {
SaveMeasurement(m *Measurement) error
}
Saver saves a measurement on some persistent storage.
type SimpleDialer ¶
type SimpleDialer interface {
// DialContext creates a new TCP/UDP connection like [net.DialContext] would do.
//
// The endpoint is an endpoint like the ones accepted by [net.DialContext]. For example,
// x.org:443, 130.192.91.211:443 and [::1]:443. Note that IPv6 addrs are quoted.
//
// This function MUST gracefully handle the case where the endpoint contains an IPv4
// or IPv6 address by skipping DNS resolution and directly using the endpoint.
//
// See https://github.com/ooni/probe-cli/pull/1295#issuecomment-1731243994 for more
// details on why DialContext MUST do that.
DialContext(ctx context.Context, network, endpoint string) (net.Conn, error)
}
SimpleDialer establishes network connections.
type Submitter ¶ added in v3.16.0
type Submitter interface {
// Submit submits the measurement and updates its
// report ID field in case of success.
Submit(ctx context.Context, m *Measurement) (string, error)
}
Submitter submits a measurement to the OONI collector.
type THDNSResult ¶ added in v3.16.0
type THDNSResult struct {
Failure *string `json:"failure"`
Addrs []string `json:"addrs"`
ASNs []int64 `json:"-"` // not visible from the JSON
}
THDNSResult is the result of the DNS lookup performed by the control vantage point.
type THHTTPRequestResult ¶ added in v3.16.0
type THHTTPRequestResult struct {
BodyLength int64 `json:"body_length"`
DiscoveredH3Endpoint string `json:"discovered_h3_endpoint"`
Failure *string `json:"failure"`
Title string `json:"title"`
Headers map[string]string `json:"headers"`
StatusCode int64 `json:"status_code"`
}
THHTTPRequestResult is the result of the HTTP request performed by the control vantage point.
type THIPInfo ¶ added in v3.16.0
type THIPInfo struct {
// ASN contains the address' AS number.
ASN int64 `json:"asn"`
// Flags contains flags describing this address.
Flags int64 `json:"flags"`
}
THIPInfo contains information about IP addresses resolved either by the probe or by the TH and processed by the TH.
type THRequest ¶ added in v3.16.0
type THRequest struct {
HTTPRequest string `json:"http_request"`
HTTPRequestHeaders map[string][]string `json:"http_request_headers"`
TCPConnect []string `json:"tcp_connect"`
// XQUICEnabled is a feature flag that tells the oohelperd to
// conditionally enable QUIC measurements, which are otherwise
// disabled by default. We will honour this flag during the
// v3.17.x release cycle and possibly also for v3.18.x but we
// will eventually enable QUIC for all clients.
XQUICEnabled bool `json:"x_quic_enabled"`
}
THRequest is the request that we send to the control.
See https://github.com/ooni/spec/blob/master/nettests/ts-017-web-connectivity.md
type THResponse ¶ added in v3.16.0
type THResponse struct {
TCPConnect map[string]THTCPConnectResult `json:"tcp_connect"`
TLSHandshake map[string]THTLSHandshakeResult `json:"tls_handshake,omitempty"`
QUICHandshake map[string]THTLSHandshakeResult `json:"quic_handshake"`
HTTPRequest THHTTPRequestResult `json:"http_request"`
HTTP3Request *THHTTPRequestResult `json:"http3_request"` // optional!
DNS THDNSResult `json:"dns"`
IPInfo map[string]*THIPInfo `json:"ip_info,omitempty"`
}
THResponse is the response from the control service.
type THTCPConnectResult ¶ added in v3.16.0
THTCPConnectResult is the result of the TCP connect attempt performed by the control vantage point.
type THTLSHandshakeResult ¶ added in v3.16.0
type THTLSHandshakeResult struct {
ServerName string `json:"server_name"`
Status bool `json:"status"`
Failure *string `json:"failure"`
}
THTLSHandshakeResult is the result of the TLS handshake attempt performed by the control vantage point.
type TLSConn ¶ added in v3.19.0
type TLSConn interface {
// net.Conn is the underlying interface
net.Conn
// ConnectionState returns the ConnectionState according
// to the standard library.
ConnectionState() tls.ConnectionState
// HandshakeContext performs an TLS handshake bounded
// in time by the given context.
HandshakeContext(ctx context.Context) error
// NetConn returns the underlying net.Conn
NetConn() net.Conn
}
TLSConn is the interface representing a *tls.Conn compatible connection, which could possibly be different from a *tls.Conn as long as it implements the interface. You can use, for example, refraction-networking/utls instead of the stdlib.
type TLSDialer ¶
type TLSDialer interface {
// CloseIdleConnections closes idle connections, if any.
CloseIdleConnections()
// DialTLSContext dials a TLS connection. This method will always return
// to you a [TLSConn], so you can always safely cast to it.
//
// The endpoint is an endpoint like the ones accepted by [net.DialContext]. For example,
// x.org:443, 130.192.91.211:443 and [::1]:443. Note that IPv6 addrs are quoted.
//
// This function MUST gracefully handle the case where the endpoint contains an IPv4
// or IPv6 address by skipping DNS resolution and directly using the endpoint.
//
// See https://github.com/ooni/probe-cli/pull/1295#issuecomment-1731243994 for more
// details on why DialTLSContext MUST do that.
DialTLSContext(ctx context.Context, network, address string) (net.Conn, error)
}
TLSDialer is a Dialer dialing TLS connections.
type TLSHandshaker ¶
type TLSHandshaker interface {
// Handshake creates a new TLS connection from the given connection and
// the given config. This function DOES NOT take ownership of the connection
// and it's your responsibility to close it on failure.
//
// Recommended tlsConfig setup:
//
// - set ServerName to be the SNI;
//
// - set RootCAs to nil (which causes us to use the default cert pool);
//
// - set NextProtos to []string{"h2", "http/1.1"} for HTTPS
// and []string{"dot"} for DNS-over-TLS.
Handshake(ctx context.Context, conn net.Conn, tlsConfig *tls.Config) (TLSConn, error)
}
TLSHandshaker is the generic TLS handshaker.
type Trace ¶ added in v3.16.0
type Trace interface {
// TimeNow returns the current time. Normally, this should be the same
// value returned by time.Now but you may want to manipulate the time
// returned when testing to have deterministic tests. To this end, you
// can use functionality exported by the ./internal/testingx pkg.
TimeNow() time.Time
// MaybeWrapNetConn possibly wraps a net.Conn with the caller trace. If there's no
// desire to wrap the net.Conn, this function just returns the original net.Conn.
//
// Arguments:
//
// - conn is the non-nil underlying net.Conn to be wrapped
MaybeWrapNetConn(conn net.Conn) net.Conn
// MaybeWrapUDPLikeConn is like MaybeWrapNetConn but for UDPLikeConn.
//
// Arguments:
//
// - conn is the non-nil underlying UDPLikeConn to be wrapped
MaybeWrapUDPLikeConn(conn UDPLikeConn) UDPLikeConn
// OnDNSRoundTripForLookupHost is used with a DNSTransport and called
// when the RoundTrip terminates.
//
// Arguments:
//
// - started is when we called transport.RoundTrip
//
// - reso is the parent resolver for the trace;
//
// - query is the non-nil DNS query we use for the RoundTrip
//
// - response is a valid DNS response, obtained after the RoundTrip;
//
// - addrs is the list of addresses obtained after the RoundTrip, which
// is empty if the RoundTrip failed
//
// - err is the result of DNSLookup; either an error or nil
//
// - finished is the time right after the RoundTrip
OnDNSRoundTripForLookupHost(started time.Time, reso Resolver, query DNSQuery,
response DNSResponse, addrs []string, err error, finished time.Time)
// OnDelayedDNSResponse is used with a DNSOverUDPTransport and called
// when we get delayed, unexpected DNS responses.
//
// Arguments:
//
// - started is when we started reading the delayed response;
//
// - txp is the DNS transport used with the resolver;
//
// - query is the non-nil DNS query we use for the RoundTrip;
//
// - response is the non-nil valid DNS response, obtained after some delay;
//
// - addrs is the list of addresses obtained after decoding the delayed response,
// which is empty if the response did not contain any addresses, which we
// extract by calling the DecodeLookupHost method.
//
// - err is the result of DecodeLookupHost: either an error or nil;
//
// - finished is when we have read the delayed response.
OnDelayedDNSResponse(started time.Time, txp DNSTransport, query DNSQuery,
resp DNSResponse, addrs []string, err error, finsihed time.Time) error
// OnConnectDone is called when connect terminates.
//
// Arguments:
//
// - started is when we called connect;
//
// - network is the network we're using (one of "tcp" and "udp");
//
// - domain is the domain for which we're calling connect. If the user called
// connect for an IP address and a port, then domain will be an IP address;
//
// - remoteAddr is the TCP endpoint with which we are connecting: it will
// consist of an IP address and a port (e.g., 8.8.8.8:443, [::1]:5421);
//
// - err is the result of connect: either an error or nil;
//
// - finished is when connect returned.
//
// The error passed to this function will always be wrapped such that the
// string returned by Error is an OONI error.
OnConnectDone(
started time.Time, network, domain, remoteAddr string, err error, finished time.Time)
// OnTLSHandshakeStart is called when the TLS handshake starts.
//
// Arguments:
//
// - now is the moment before we start the handshake;
//
// - remoteAddr is the TCP endpoint with which we are connecting: it will
// consist of an IP address and a port (e.g., 8.8.8.8:443, [::1]:5421);
//
// - config is the non-nil TLS config we're using.
OnTLSHandshakeStart(now time.Time, remoteAddr string, config *tls.Config)
// OnTLSHandshakeDone is called when the TLS handshake terminates.
//
// Arguments:
//
// - started is when we started the handshake;
//
// - remoteAddr is the TCP endpoint with which we are connecting: it will
// consist of an IP address and a port (e.g., 8.8.8.8:443, [::1]:5421);
//
// - config is the non-nil TLS config we're using;
//
// - state is the state of the TLS connection after the handshake, where all
// fields are zero-initialized if the handshake failed;
//
// - err is the result of the handshake: either an error or nil;
//
// - finished is right after the handshake.
//
// The error passed to this function will always be wrapped such that the
// string returned by Error is an OONI error.
OnTLSHandshakeDone(started time.Time, remoteAddr string, config *tls.Config,
state tls.ConnectionState, err error, finished time.Time)
// OnQUICHandshakeStart is called before the QUIC handshake.
//
// Arguments:
//
// - now is the moment before we start the handshake;
//
// - remoteAddr is the QUIC endpoint with which we are connecting: it will
// consist of an IP address and a port (e.g., 8.8.8.8:443, [::1]:5421);
//
// - config is the possibly-nil QUIC config we're using.
OnQUICHandshakeStart(now time.Time, remoteAddr string, quicConfig *quic.Config)
// OnQUICHandshakeDone is called after the QUIC handshake.
//
// Arguments:
//
// - started is when we started the handshake;
//
// - remoteAddr is the QUIC endpoint with which we are connecting: it will
// consist of an IP address and a port (e.g., 8.8.8.8:443, [::1]:5421);
//
// - qconn is the QUIC connection we receive after the handshake: either
// a valid quic.EarlyConnection or nil;
//
// - config is the non-nil TLS config we are using;
//
// - err is the result of the handshake: either an error or nil;
//
// - finished is right after the handshake.
//
// The error passed to this function will always be wrapped such that the
// string returned by Error is an OONI error.
OnQUICHandshakeDone(started time.Time, remoteAddr string, qconn quic.EarlyConnection,
config *tls.Config, err error, finished time.Time)
}
Trace allows to collect measurement traces. A trace is injected into netx operations using context.WithValue. Netx code retrieves the trace using context.Value. See docs/design/dd-003-step-by-step.md for the design document explaining why we implemented context-based tracing.
type UDPLikeConn ¶
type UDPLikeConn interface {
// An UDPLikeConn is a net.PacketConn conn.
net.PacketConn
// SetReadBuffer allows setting the read buffer.
SetReadBuffer(bytes int) error
// SyscallConn returns a conn suitable for calling syscalls,
// which is also instrumental to setting the read buffer.
SyscallConn() (syscall.RawConn, error)
}
UDPLikeConn is a net.PacketConn with some extra functions required to convince the QUIC library (quic-go/quic-go) to inflate the receive buffer of the connection.
The QUIC library will treat this connection as a "dumb" net.PacketConn, calling its ReadFrom and WriteTo methods as opposed to more efficient methods that are available under Linux and (maybe?) FreeBSD.
It seems fine to avoid performance optimizations, because they would complicate the implementation on our side and our use cases (blocking and heavy throttling) do not seem to require such optimizations.
See https://github.com/ooni/probe/issues/1754 for a more comprehensive discussion of UDPLikeConn.
type UDPListener ¶ added in v3.19.0
type UDPListener interface {
// Listen creates a new listening UDPLikeConn.
Listen(addr *net.UDPAddr) (UDPLikeConn, error)
}
UDPListener listens for connections over UDP, e.g. QUIC.
type UnderlyingNetwork ¶ added in v3.17.0
type UnderlyingNetwork interface {
// DefaultCertPool returns the underlying cert pool used by the
// network extensions library. You MUST NOT use this function to
// modify the default cert pool since this would lead to a data
// race. Use [netxlite.netxlite.NewMozillaCertPool] if you wish to get
// a copy of the default cert pool that you can modify.
DefaultCertPool() *x509.CertPool
// DialTimeout returns the default timeout to use for dialing.
DialTimeout() time.Duration
// DialContext is equivalent to net.Dialer.DialContext.
DialContext(ctx context.Context, network, address string) (net.Conn, error)
// GetaddrinfoLookupANY is like net.Resolver.LookupHost except that it
// also returns to the caller the CNAME when it is available.
GetaddrinfoLookupANY(ctx context.Context, domain string) ([]string, string, error)
// GetaddrinfoResolverNetwork returns the resolver network.
GetaddrinfoResolverNetwork() string
// ListenTCP is equivalent to net.ListenTCP.
ListenTCP(network string, addr *net.TCPAddr) (net.Listener, error)
// ListenUDP is equivalent to net.ListenUDP.
ListenUDP(network string, addr *net.UDPAddr) (UDPLikeConn, error)
}
UnderlyingNetwork implements the underlying network APIs on top of which we implement network extensions such as MeasuringNetwork.
type UploadedTotalCount ¶ added in v3.17.0
type UploadedTotalCount struct {
UploadedCount int64 `db:",inline"`
TotalCount int64 `db:",inline"`
}
UploadedTotalCount is the count of the measurements which have been uploaded vs the total measurements in a given result set
type WritableDatabase ¶ added in v3.17.0
type WritableDatabase interface {
// CreateNetwork will create a new network in the network table
//
// Arguments:
//
// - loc: loc is the location provider used to instantiate the network
//
// Returns either a database network instance or an error
CreateNetwork(loc LocationProvider) (*DatabaseNetwork, error)
// CreateOrUpdateURL will create a new URL entry to the urls table if it doesn't
// exists, otherwise it will update the category code of the one already in
// there.
//
// Arguments:
//
// - urlStr is the URL string to create or update
//
// - categoryCode is the category code to update
//
// - countryCode is the country code to update
//
// Returns either the new URL id or an error
CreateOrUpdateURL(urlStr string, categoryCode string, countryCode string) (int64, error)
// CreateResult writes the Result to the database a returns a pointer
// to the Result
//
// Arguments:
//
// - homePath is the home directory path to make the results directory
//
// - testGroupName is used to annotate the database the result
//
// - networkID is the id of the underlying network
//
// Returns either a database result instance or an error
CreateResult(homePath string, testGroupName string, networkID int64) (*DatabaseResult, error)
// UpdateUploadedStatus will check if all the measurements inside of a given result set have been
// uploaded and if so will set the is_uploaded flag to true
//
// Arguments:
//
// - result is the database result to update
//
// Returns a non-nil error if update failed
UpdateUploadedStatus(result *DatabaseResult) error
// Finished marks the result as done and sets the runtime
//
// Arguments:
//
// - result is the database result to mark as done
//
// Returns a non-nil error if result could not be marked as done
Finished(result *DatabaseResult) error
// DeleteResult will delete a particular result and the relative measurement on
// disk.
//
// Arguments:
//
// - resultID is the id of the database result to be deleted
//
// Returns a non-nil error if result could not be deleted
DeleteResult(resultID int64) error
// CreateMeasurement writes the measurement to the database a returns a pointer
// to the Measurement
//
// Arguments:
//
// - reportID is the report id to annotate
//
// - testName is the experiment name to use
//
// - measurementDir is the measurement directory path
//
// - resultID is the result id to annotate
//
// - urlID is the id of the URL input
//
// Returns either a database measurement or an error
CreateMeasurement(reportID sql.NullString, testName string, measurementDir string, idx int,
resultID int64, urlID sql.NullInt64) (*DatabaseMeasurement, error)
// AddTestKeys writes the summary to the measurement
//
// Arguments:
//
// - msmt is the database measurement to update
//
// - tk is the testkeys
//
// Returns a non-nil error if measurement update failed
AddTestKeys(msmt *DatabaseMeasurement, tk MeasurementSummaryKeys) error
// Done marks the measurement as completed
//
// Arguments:
//
// - msmt is the database measurement to update
//
// Returns a non-nil error if the measurement could not be marked as done
Done(msmt *DatabaseMeasurement) error
// UploadFailed writes the error string for the upload failure to the measurement
//
// Arguments:
//
// - msmt is the database measurement to update
//
// - failure is the error string to use
//
// Returns a non-nil error if the measurement update failed
UploadFailed(msmt *DatabaseMeasurement, failure string) error
// UploadSucceeded writes the error string for the upload failure to the measurement
//
// Arguments:
//
// - msmt is the database measurement to update
//
// Returns a non-nil error is measurement update failed
UploadSucceeded(msmt *DatabaseMeasurement) error
// Failed writes the error string to the measurement
//
// Arguments:
//
// - msmt is the database measurement to update
//
// - failure is the error string to use
//
// Returns a non-nil error if the measurement update failed
Failed(msmt *DatabaseMeasurement, failure string) error
}
WritableDatabase supports writing and updating data.