engine

package module
v0.2.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Nov 12, 2019 License: BSD-3-Clause Imports: 24 Imported by: 0

README

GoDoc Build Status Coverage Status Go Report Card

OONI probe measurement engine

This repository contains OONI probe's measurement engine. That is, the piece of software that implements OONI nettests.

API

You can browse ooni/probe-engine's API online at godoc.org. We currently don't provide any API stability guarantees.

This repository also allows to build miniooni, a small command line client useful to test the functionality in here without integrating with OONI probe. You can browse the manual of this tool online at godoc.org. We currently don't promise that the miniooni CLI will be stable over time.

Integrating ooni/probe-engine

This software uses Go modules and therefore requires Go v1.11+. We also depend on Measurement Kit, a C++14 library implementing many OONI tests.

Note that export CGO_ENABLED=0 will disable C/C++ extensions and therefore will prevent Measurement Kit tests from being linked into the resulting Go binaries. You may want that in some cases, e.g. when you only want to use OONI tests written in Go.

The Psiphon experiment requires some secrets that you may not have. To disable building it, use the nopsiphon build tag. For example, you can use go build -v -tags nopsiphon ./cmd/miniooni to build a version of the miniooni testing command that does not include Psiphon.

We plan on gradually rewriting all OONI tests in Go, therefore the dependency on Measurement Kit will eventually be removed.

A future version of this document will provide platform specific instruction for installing Measurement Kit and building.

Release procedure

  1. make sure that dependencies are up to date

  2. make sure that resources are up to date

  3. commit, tag, and push

  4. create new release on GitHub

Documentation

Overview

Package engine contains the engine API

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func AllExperiments 

func AllExperiments() []string

AllExperiments returns the name of all experiments

Types

type Callbacks 

type Callbacks interface {
	// OnDataUsage provides information about data usage.
	OnDataUsage(dloadKiB, uploadKiB float64)

	// OnProgress provides information about an experiment progress.
	OnProgress(percentage float64, message string)
}

Callbacks contains event handling callbacks

This is a copy of experiment/handler.Callbacks. Go will make sure the interface will match for us. This means we can have this set of callbacks as part of the toplevel engine API.

type Experiment 

type Experiment struct {
	// contains filtered or unexported fields
}

Experiment is an experiment instance.

func (*Experiment) CloseReport 

func (e *Experiment) CloseReport() error

CloseReport is an idempotent method that closes and open report if one has previously been opened, otherwise it does nothing.

func (*Experiment) LoadMeasurement 

func (e *Experiment) LoadMeasurement(data []byte) (*Measurement, error)

LoadMeasurement loads a measurement from a byte stream. The measurement must be a measurement for this experiment.

func (*Experiment) Measure 

func (e *Experiment) Measure(input string) (*Measurement, error)

Measure performs a measurement with input. We assume that you have configured the available test helpers, either manually or by calling the session's MaybeLookupBackends() method.

func (*Experiment) Name 

func (e *Experiment) Name() string

Name returns the experiment name.

func (*Experiment) OpenReport 

func (e *Experiment) OpenReport() error

OpenReport is an idempotent method to open a report. We assume that you have configured the available collectors, either manually or through using the session's MaybeLookupBackends method.

func (*Experiment) ReportID 

func (e *Experiment) ReportID() string

ReportID returns the open reportID, if we have opened a report successfully before, or an empty string, otherwise.

func (*Experiment) SaveMeasurement 

func (e *Experiment) SaveMeasurement(measurement *Measurement, path string) error

SaveMeasurement saves the measurement at the specified path.

func (*Experiment) SubmitAndUpdateMeasurement 

func (e *Experiment) SubmitAndUpdateMeasurement(measurement *Measurement) error

SubmitAndUpdateMeasurement submits a measurement and updates the fields whose value has changed as part of the submission.

type ExperimentBuilder 

type ExperimentBuilder struct {
	// contains filtered or unexported fields
}

ExperimentBuilder is an experiment builder.

func (*ExperimentBuilder) Build 

func (b *ExperimentBuilder) Build() *Experiment

Build builds the experiment

func (*ExperimentBuilder) NeedsInput 

func (b *ExperimentBuilder) NeedsInput() bool

NeedsInput returns whether the experiment needs input

func (*ExperimentBuilder) Options 

func (b *ExperimentBuilder) Options() (map[string]OptionInfo, error)

Options returns info about all options

func (*ExperimentBuilder) SetCallbacks 

func (b *ExperimentBuilder) SetCallbacks(callbacks Callbacks)

SetCallbacks sets the interactive callbacks

func (*ExperimentBuilder) SetOptionBool 

func (b *ExperimentBuilder) SetOptionBool(key string, value bool) error

SetOptionBool sets a bool option

func (*ExperimentBuilder) SetOptionInt 

func (b *ExperimentBuilder) SetOptionInt(key string, value int64) error

SetOptionInt sets an int option

func (*ExperimentBuilder) SetOptionString 

func (b *ExperimentBuilder) SetOptionString(key, value string) error

SetOptionString sets a string option

type Measurement 

type Measurement struct {
	// contains filtered or unexported fields
}

Measurement is a OONI measurement

func (*Measurement) AddAnnotations 

func (m *Measurement) AddAnnotations(annotations map[string]string)

AddAnnotations adds annotation to the measurement

func (*Measurement) MakeGenericTestKeys 

func (m *Measurement) MakeGenericTestKeys() (map[string]interface{}, error)

MakeGenericTestKeys casts the m.TestKeys to a map[string]interface{}.

Ideally, all tests should have a clear Go structure, well defined, that will be stored in m.TestKeys as an interface. This is not already the case and it's just valid for tests written in Go. Until all tests will be written in Go, we'll keep this glue here to make sure we convert from the engine format to the cli format.

This function will first attempt to cast directly to map[string]interface{}, which is possible for MK tests, and then use JSON serialization and de-serialization only if that's required.

type OptionInfo 

type OptionInfo struct {
	Doc  string
	Type string
}

OptionInfo contains info about an option

type Session 

type Session struct {
	// contains filtered or unexported fields
}

Session is a measurement session

func NewSession 

func NewSession(config SessionConfig) (*Session, error)

NewSession creates a new session or returns an error

func (*Session) AddAvailableHTTPSBouncer 

func (sess *Session) AddAvailableHTTPSBouncer(baseURL string)

AddAvailableHTTPSBouncer adds an HTTPS bouncer to the list of bouncers that we'll try to contact.

func (*Session) AddAvailableHTTPSCollector 

func (sess *Session) AddAvailableHTTPSCollector(baseURL string)

AddAvailableHTTPSCollector adds an HTTPS collector to the list of collectors that we'll try to use.

func (*Session) MaybeLookupBackends 

func (sess *Session) MaybeLookupBackends() error

MaybeLookupBackends is a caching OONI backends lookup call.

func (*Session) MaybeLookupLocation 

func (sess *Session) MaybeLookupLocation() error

MaybeLookupLocation is a caching location lookup call.

func (*Session) NewExperimentBuilder 

func (sess *Session) NewExperimentBuilder(name string) (*ExperimentBuilder, error)

NewExperimentBuilder returns a new experiment builder for the experiment with the given name, or an error if there's no such experiment with the given name

func (*Session) NewTestListsClient 

func (sess *Session) NewTestListsClient() *TestListsClient

NewTestListsClient returns a new TestListsClient that is configured to perform requests in the context of this session

func (*Session) NewTestListsConfig 

func (sess *Session) NewTestListsConfig() *TestListsConfig

NewTestListsConfig returns prefilled settings for TestListsClient where in particular we have set the correct country code

func (*Session) ProbeASN 

func (sess *Session) ProbeASN() uint

ProbeASN returns the ASN of the probe's network.

func (*Session) ProbeASNString 

func (sess *Session) ProbeASNString() string

ProbeASNString is like ProbeASN but returns the "AS<%d>" string where the number is the number returned by ProbeASN.

func (*Session) ProbeCC 

func (sess *Session) ProbeCC() string

ProbeCC returns the probe country code.

func (*Session) ProbeIP 

func (sess *Session) ProbeIP() string

ProbeIP returns the probe IP.

func (*Session) ProbeNetworkName 

func (sess *Session) ProbeNetworkName() string

ProbeNetworkName returns the probe network name.

func (*Session) ResolverIP 

func (sess *Session) ResolverIP() string

ResolverIP returns the resolver IP.

func (*Session) SetIncludeProbeASN 

func (sess *Session) SetIncludeProbeASN(value bool)

SetIncludeProbeASN controls whether to include the ASN

func (*Session) SetIncludeProbeCC 

func (sess *Session) SetIncludeProbeCC(value bool)

SetIncludeProbeCC controls whether to include the country code

func (*Session) SetIncludeProbeIP 

func (sess *Session) SetIncludeProbeIP(value bool)

SetIncludeProbeIP controls whether to include the IP

type SessionConfig 

type SessionConfig struct {
	AssetsDir       string
	Logger          log.Logger
	ProxyURL        *url.URL
	SoftwareName    string
	SoftwareVersion string
	TempDir         string
	TLSConfig       *tls.Config
}

SessionConfig contains the Session config

type TestListsClient 

type TestListsClient struct {
	// contains filtered or unexported fields
}

TestListsClient is a test lists client

func (*TestListsClient) Fetch 

func (c *TestListsClient) Fetch(config *TestListsConfig) ([]URLInfo, error)

Fetch fetches the test list

type TestListsConfig 

type TestListsConfig struct {
	BaseURL             string   // or use default
	AvailableCategories []string // or ask for all
	CountryCode         string   // set by session
	Limit               int      // or get all
}

TestListsConfig contains settings for TestListsClient

type URLInfo 

type URLInfo interface {
	URL() string
	CategoryCode() string
	CountryCode() string
}

URLInfo contains info about URLs

Source Files

View all Source files

Directories

Path Synopsis
Package bouncer contains a OONI bouncer client implementation.
 Package bouncer contains a OONI bouncer client implementation.
cmd
miniooni command
Command miniooni is simple binary for testing purposes.
 Command miniooni is simple binary for testing purposes.
Package collector contains a OONI collector client implementation.
 Package collector contains a OONI collector client implementation.
Package experiment contains network experiment.
 Package experiment contains network experiment.
dash
Package dash contains the dash network experiment.
 Package dash contains the dash network experiment.
example
Package example contains a simple example of experiment.
 Package example contains a simple example of experiment.
fbmessenger
Package fbmessenger contains the Facebook Messenger network experiment.
 Package fbmessenger contains the Facebook Messenger network experiment.
handler
Package handler contains experiment events handler
 Package handler contains experiment events handler
hhfm
Package hhfm contains the HTTP Header Field Manipulation network experiment.
 Package hhfm contains the HTTP Header Field Manipulation network experiment.
hirl
Package hirl contains the HTTP Invalid Request Line network experiment.
 Package hirl contains the HTTP Invalid Request Line network experiment.
mkevent
Package mkevent processes MK events
 Package mkevent processes MK events
mkhelper
Package mkhelper contains common code to get the proper helper and configure it into settings.
 Package mkhelper contains common code to get the proper helper and configure it into settings.
mkrunner
Package mkrunner contains code to run an MK based test
 Package mkrunner contains code to run an MK based test
ndt
Package ndt contains the ndt network experiment.
 Package ndt contains the ndt network experiment.
ndt7
Package ndt7 contains the ndt7 network experiment.
 Package ndt7 contains the ndt7 network experiment.
psiphon
Package psiphon implements the psiphon network experiment.
 Package psiphon implements the psiphon network experiment.
telegram
Package telegram contains the Telegram network experiment.
 Package telegram contains the Telegram network experiment.
web_connectivity
Package web_connectivity contains the Web Connectivity network experiment.
 Package web_connectivity contains the Web Connectivity network experiment.
whatsapp
Package whatsapp contains the WhatsApp network experiment.
 Package whatsapp contains the WhatsApp network experiment.
geoiplookup
iplookup
Package iplookup implements probe IP lookup.
 Package iplookup implements probe IP lookup.
iplookup/akamai
Package akamai lookups the IP using akamai.
 Package akamai lookups the IP using akamai.
iplookup/avast
Package avast lookups the IP using avast.
 Package avast lookups the IP using avast.
iplookup/invalid
Package invalid returns an invalid IP.
 Package invalid returns an invalid IP.
iplookup/ubuntu
Package ubuntu lookups the IP using Ubuntu.
 Package ubuntu lookups the IP using Ubuntu.
mmdblookup
Package mmdblookup performs probe ASN, CC, NetworkName lookups.
 Package mmdblookup performs probe ASN, CC, NetworkName lookups.
resolverlookup
Package resolverlookup discovers the resolver's IP
 Package resolverlookup discovers the resolver's IP
Package httpx contains HTTP extensions.
 Package httpx contains HTTP extensions.
fetch
Package fetch is used to fetch resources.
 Package fetch is used to fetch resources.
httplog
Package httplog implements HTTP event logging.
 Package httplog implements HTTP event logging.
httptracex
Package httptracex contains httptrace extensions.
 Package httptracex contains httptrace extensions.
jsonapi
Package jsonapi interacts with HTTP JSON APIs.
 Package jsonapi interacts with HTTP JSON APIs.
netx
Package netx contains network extensions.
 Package netx contains network extensions.
retryx
Package retryx helps to retry operations
 Package retryx helps to retry operations
Package log defines the logging interface used in this library.
 Package log defines the logging interface used in this library.
Package measurementkit allows to use Measurement Kit.
 Package measurementkit allows to use Measurement Kit.
Package model defines shared data structures.
 Package model defines shared data structures.
orchestra
testlists
Package testlists queries orchestra's test lists.
 Package testlists queries orchestra's test lists.
Package resources contains code to download resources.
 Package resources contains code to download resources.
Package session models a measurement session.
 Package session models a measurement session.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.