Documentation
¶
Overview ¶
mtglib defines a package with MTPROTO proxy.
Since mtg itself is build as an example of how to work with mtglib, it worth to telling a couple of words about a project organization.
A core object of the project is mtglib.Proxy. This is a proxy you expect: that one which you configure, set to serve on a listener and/or shutdown on application termination.
But it also has a core logic unrelated to Telegram per se: anti replay cache, network connectivity (who knows, maybe you want to have a native VMESS integration) and so on.
You can supply such parts to a proxy with interfaces. The rest of the packages in mtg define some default implementations of these interfaces. But if you want to integrate it with, let say, influxdb, you can do it easily.
Index ¶
- Constants
- Variables
- func ParseBanAddresses(ips []string, ttl string, force bool) ([]netip.Addr, time.Duration, error)
- func ParseExpiry(s string) (time.Time, error)
- func ParseExternalAddresses(ips []string, ttl string) ([]netip.Addr, time.Duration, error)
- func ParseKickAddresses(ips []string) ([]netip.Addr, error)
- func ParseQuotaBytes(s string) (int64, error)
- func ParseSpeed(s string) (int64, error)
- func SecretsCanFront(secrets map[string]Secret, domainFrontingHost string) bool
- type AddressReport
- type AntiReplayCache
- type BanJSON
- type BanResult
- type BansReport
- type ClientJSON
- type ClientUsageJSON
- type ClientsReport
- type DenyReason
- type Event
- type EventConcurrencyLimited
- type EventConnectedToDC
- type EventDomainFronting
- type EventFinish
- type EventIPBlocklisted
- type EventIPLimited
- type EventIPListSize
- type EventReplayAttack
- type EventStart
- type EventStream
- type EventThrottled
- type EventTraffic
- type FakeTLSJSON
- type IPBlocklist
- type KickResult
- type ListenerJSON
- type ListenerPolicy
- type Logger
- type ModeSet
- type Network
- type ObfuscatedJSON
- type Proxy
- func (p *Proxy) Ban(name string, addrs []netip.Addr, ttl time.Duration) (BanResult, error)
- func (p *Proxy) DeleteSecret(name string) error
- func (p *Proxy) DomainFrontingAddress() string
- func (p *Proxy) GlobalAdTag() *[AdTagLength]byte
- func (p *Proxy) Kick(name string, addrs []netip.Addr) (int, error)
- func (p *Proxy) PutSecret(name string, s Secret, adTag *[AdTagLength]byte, limits *SecretLimits) error
- func (p *Proxy) ReloadSecrets() error
- func (p *Proxy) ReplaceSecrets(cfg SecretConfig) error
- func (p *Proxy) Serve(listener net.Listener) error
- func (p *Proxy) ServeConn(conn essentials.Conn)
- func (p *Proxy) ServeWithPolicy(listener net.Listener, policy ListenerPolicy) error
- func (p *Proxy) SetExternalAddresses(name string, addrs []netip.Addr, ttl time.Duration) error
- func (p *Proxy) SetGlobalAdTag(tag *[AdTagLength]byte) error
- func (p *Proxy) Shutdown()
- func (p *Proxy) SnapshotSecrets() []SecretInfo
- func (p *Proxy) Unban(name string, addr netip.Addr) (bool, error)
- type ProxyOpts
- type ProxyStats
- func (s *ProxyStats) AddBytesIn(name string, n int64)
- func (s *ProxyStats) AddBytesOut(name string, n int64)
- func (s *ProxyStats) CanConnect(name string) bool
- func (s *ProxyStats) FlushUsage(path string) error
- func (s *ProxyStats) LoadUsage(path string) error
- func (s *ProxyStats) OnAdTagFailed(name string)
- func (s *ProxyStats) OnAdTagSkipped(name string)
- func (s *ProxyStats) OnClientConnsRefused(name string)
- func (s *ProxyStats) OnConnect(name string)
- func (s *ProxyStats) OnDisconnect(name string)
- func (s *ProxyStats) OnIPLimitRefused(name string)
- func (s *ProxyStats) PreRegister(name string)
- func (s *ProxyStats) QuotaUsed(name string, reset QuotaReset) int64
- func (s *ProxyStats) ResetQuota(name string)
- func (s *ProxyStats) ServeHTTP(w http.ResponseWriter, _ *http.Request)
- func (s *ProxyStats) SetDirectionalSpeedLimits(name string, upBytesPerSecond, downBytesPerSecond int64)
- func (s *ProxyStats) SetLastIP(name, ip string)
- func (s *ProxyStats) SetQuotaCeiling(name string, bytes int64)
- func (s *ProxyStats) SetSpeedLimit(name string, bytesPerSecond int64)
- func (s *ProxyStats) SetThrottle(limit int64, interval time.Duration)
- func (s *ProxyStats) UpdateLastSeen(name string)
- type QuotaReset
- type RetiringSecret
- type RotationClock
- type Secret
- func (s Secret) Base64() string
- func (s Secret) DDHex() string
- func (s Secret) Hex() string
- func (s Secret) Hosted() bool
- func (s Secret) MarshalText() ([]byte, error)
- func (s Secret) PlainHex() string
- func (s *Secret) Set(text string) error
- func (s Secret) String() string
- func (s *Secret) UnmarshalText(data []byte) error
- func (s Secret) Valid() bool
- type SecretConfig
- type SecretInfo
- type SecretLimits
- type SecretMode
- type StatsResponse
- type ThrottleJSON
- type UnbanResult
- type UnknownSecretAction
- type UserStatsJSON
Constants ¶
const ( // DefaultExternalAddressTTL is how long an externally reported address // counts when the report names no ttl. A panel is expected to refresh // well inside it. DefaultExternalAddressTTL = 2 * time.Minute // MaxExternalAddressTTL bounds a report's ttl, so a mistaken or hostile // panel cannot pin a user's slots for a day. MaxExternalAddressTTL = time.Hour // MaxExternalAddresses bounds one report, which bounds what one user's // ledger can hold. MaxExternalAddresses = 64 )
const ( // DefaultConcurrency is a default max count of simultaneously connected // clients. DefaultConcurrency = 4096 // DefaultBufferSize is a default size of a copy buffer. // // Deprecated: this setting no longer makes any effect. DefaultBufferSize = 16 * 1024 // 16 kib // DefaultDomainFrontingPort is a default port (HTTPS) to connect to in case // of probe-resistance activity. DefaultDomainFrontingPort = 443 // DefaultIdleTimeout is a default timeout for closing a connection in case of // idling. // // Set to 5 minutes to survive typical mobile sleep periods (2-5 min) and // avoid racing with MTProto ping_delay_disconnect (~60s interval). DefaultIdleTimeout = 5 * time.Minute // DefaultHandshakeTimeout defines a time period during which the // all handshake ceremonies must be completed. DefaultHandshakeTimeout = 10 * time.Second // DefaultTolerateTimeSkewness is a default timeout for time skewness on a // faketls timeout verification. DefaultTolerateTimeSkewness = 3 * time.Second // DefaultPreferIP is a default value for Telegram IP connectivity preference. DefaultPreferIP = "prefer-ipv6" // SecretKeyLength defines a length of the secret bytes used by Telegram and a // proxy. SecretKeyLength = 16 // AdTagLength defines the length in bytes of a Telegram advertising tag // (the ad_tag / proxy tag issued by @MTProxybot). It is carried in the // RPC_PROXY_REQ message sent to a Telegram middle proxy. AdTagLength = 16 // ConnectionIDBytesLength defines a count of random bytes used to generate a // stream/connection ids. ConnectionIDBytesLength = 16 // TCPRelayReadTimeout defines a max time period between two consecuitive // reads from Telegram after which connection will be terminated. This is // required to abort stale connections. TCPRelayReadTimeout = 20 * time.Second // DoppelGangerPerRaid defines a number of requests to each URL // per raid. DoppelGangerPerRaid = 10 // DoppelGangerEach defines a time period between each crawl attempt. DoppelGangerEach = 6 * time.Hour )
const ( // DefaultBanTTL is how long a ban lasts when the request names no ttl. DefaultBanTTL = time.Hour // MaxBanTTL bounds a ban's ttl. A ban is a short-term tool; a leaked link // is retired by re-keying the secret. MaxBanTTL = 30 * 24 * time.Hour )
const ( // FeatureClientBans means POST and GET /secrets/{name}/bans and DELETE // /secrets/{name}/bans/{address...} are served: an operator refuses a // secret to an address -- an IPv4 address or an IPv6 /64 -- for a ttl, in // memory only, ending its live streams; the place is turned away at every // door as a wrong secret is; and GET /secrets/{name}/clients rows carry // banned_until. Addresses are listed and echoed only with track-client-ips. // The API is strict, so a client must see this before it calls the routes. FeatureClientBans = "client-bans" // FeatureClientIPs means the binary understands the track-client-ips // option, so a panel may offer the switch and may expect connected_ips and // last_ip on GET /stats when it is on. It says nothing about whether the // running deployment has actually enabled it -- the fields' presence says // that, and their absence is indistinguishable from an idle secret. FeatureClientIPs = "client-ips" // FeatureClientLimits means the per-address controls inside one secret are // honoured: [secret-limits.<name>] and the secrets API accept client-conns // (client_conns on the API), a cap on the live connections one address may // hold to the secret; client-speed (client_speed), a ceiling on how fast // one address may move bytes; and client-fair (client_fair), which divides // the secret's speed, speed-up and speed-down among its addresses and is // refused without one of them. /stats reads them back as client_conns, // client_speed_limit and client_fair and counts the cap's refusals in // client_conns_refused. The API is strict, so a client must see this before // it sends any of these fields. FeatureClientLimits = "client-limits" // FeatureClientStats means GET /secrets/{name}/clients is served: each // source address of a secret, with the bytes it moved, its live // connections and the per-address rates in force, most bytes first -- // listed only with track-client-ips, the counts answered either way -- and // that /stats carries clients and clients_overflow inside the user object. FeatureClientStats = "client-stats" // FeatureFakeTLS means the proxy speaks the FakeTLS (obfuscated) handshake. FeatureFakeTLS = "faketls" // FeatureHandshakeBudget means the FakeTLS door bounds the CPU it spends // scanning hellos no hint matched, [defense.handshake-budget] is honoured, // and /stats reports the door's counters as the top-level faketls_handshake // object once it has counted something. FeatureHandshakeBudget = "handshake-budget" // FeatureIPLimits means [secret-limits.<name>] and the secrets API accept // a per-user ip_limit, the proxy refuses a connection from one address // too many, /stats reports ip_limit, active_ips and ip_limit_refused, and // GET/PUT/DELETE /secrets/{name}/external-ips let a panel report the // addresses a user holds on other systems so the limit spans them. FeatureIPLimits = "ip-limits" // FeatureListeners means the [[listener]] config table is honoured: extra // sockets can be declared, and /stats reports what each one accepts. FeatureListeners = "listeners" // FeatureQuotaReset means POST /secrets/{name}/reset-quota is served. FeatureQuotaReset = "quota-reset" // FeatureSecretModes means the dd and plain secret forms parse and serve, so // a panel may hand out a link in any of the three wire forms. FeatureSecretModes = "secret-modes" // FeatureSecretsAPI means the GET/PUT/POST/DELETE /secrets routes are served. FeatureSecretsAPI = "secrets-api" // FeatureWebRelay means this binary carries the WEB proxy transport: the // `web-relay` and `generate-web-key` subcommands, and the [web] config // table they read. // // It is advertised by the PROXY, which does not serve that transport, and // that is the point: the panel decides whether to render the WEB settings // and start a second process, and the only thing it can ask is the running // binary. A proxy answering /stats is the one component it always has. FeatureWebRelay = "web-relay" // FeatureWebRelayWatch means the relay follows its config file by itself: // a changed [secrets] table is applied within seconds of the write // settling, with no signal sent. Advertised by the proxy for the same // reason as FeatureWebRelay -- a panel that writes the file and then // signals the relay can stop signalling once it sees this, and the proxy // is the one component it can ask. FeatureWebRelayWatch = "web-relay-watch" // FeatureSpeedLimits means [secret-limits.<name>] and the secrets API // accept a per-user speed, enforced by the proxy across every connection // and door of that user, and /stats reports speed_limit and throttled_ms. FeatureSpeedLimits = "speed-limits" // FeatureSpeedDirections means speed-up and speed-down (speed_up and // speed_down on the API) cap each direction on its own bucket, read back // as speed_limit_up and speed_limit_down. FeatureSpeedDirections = "speed-directions" // FeatureKick means POST /secrets/{name}/kick closes a user's live streams // from the named places, matched at the unit the address limit counts. FeatureKick = "kick" // FeatureClientIPActivity means connected_ips_last_seen accompanies // connected_ips wherever it is reported: when each address last moved a // byte, gated on track-client-ips exactly as the addresses are. FeatureClientIPActivity = "client-ip-activity" // FeatureQuotaCeiling means the quota is enforced on the data path -- a // stream ends at the byte that reaches it and a sweep closes the idle // ones -- rather than only refusing the next connection. FeatureQuotaCeiling = "quota-ceiling" // FeatureSecretRead means GET /secrets/{name} is served. FeatureSecretRead = "secret-read" // FeatureSecretRotation means a secret may be given a new key while its old // one is still accepted until an absolute time: the [[retiring-secret]] // config table is honoured, the `rotate-secret` command exists, and /stats // and GET /secrets/{name} may carry retiring_keys, retiring_until, // retiring_connections and retiring_last_seen inside the secret's own // object -- never as a key under users. FeatureSecretRotation = "secret-rotation" )
Capability names reported in the "features" array of the /stats response. They are a PUBLIC CONTRACT: downstream tooling (the management panel) matches on these exact strings to decide whether an endpoint may be called, so renaming one is a breaking API change, not a refactor. Only add a name once the capability actually exists in the binary — advertising an absent one defeats the point of the array.
const ( // MaxRetiringOverlap is the longest overlap a config should grant: an entry // whose Until is further than this from the load is warned about by the // config layer, refused by run --check and never written by rotate-secret. // The engine does not apply it: such an entry is still bounded by its own // Until, and dropping it now would accept it again once now has grown. MaxRetiringOverlap = 720 * time.Hour // MaxRetiringPerSecret is how many retiring keys one name keeps: the ones // with the latest Until. MaxRetiringPerSecret = 4 // MaxRetiringSecrets is how many retiring keys a set keeps in all, after // the per-name cap: the ones with the latest Until, ties broken by name and // then by key bytes. MaxRetiringSecrets = 4096 )
const MinSpeedBytesPerSecond = 64_000 / 8
MinSpeedBytesPerSecond is the lowest speed limit accepted: 64 kbit/s, which is exactly 8,000 bytes a second. Below that Telegram is not usable, and a bucket that small would make every read a stall; a value under it is a typo in the units far more often than an intention, so it is refused rather than honoured. Spelled in bits deliberately, so that "64kbit" -- the number the error message names -- is itself accepted.
Variables ¶
var ( // ErrSecretEmpty is returned if you are trying to create a proxy but do not // provide a secret. ErrSecretEmpty = errors.New("secret is empty") // ErrSecretInvalid is returned if you are trying to create a proxy but secret // value is invalid (no host or payload are zeroes). ErrSecretInvalid = errors.New("secret is invalid") // ErrNetworkIsNotDefined is returned if you are trying to create a proxy but // network value is undefined. ErrNetworkIsNotDefined = errors.New("network is not defined") // ErrAntiReplayCacheIsNotDefined is returned if you are trying to create a // proxy but anti replay cache value is undefined. ErrAntiReplayCacheIsNotDefined = errors.New("anti-replay cache is not defined") // ErrIPBlocklistIsNotDefined is returned if you are trying to create a proxy // but ip blocklist instance is not defined. ErrIPBlocklistIsNotDefined = errors.New("ip blocklist is not defined") // ErrIPAllowlistIsNotDefined is returned if you are trying to create a proxy // but ip allowlist instance is not defined. ErrIPAllowlistIsNotDefined = errors.New("ip allowlist is not defined") // ErrEventStreamIsNotDefined is returned if you are trying to create a proxy // but event stream instance is not defined. ErrEventStreamIsNotDefined = errors.New("event stream is not defined") // ErrLoggerIsNotDefined is returned if you are trying to create a proxy but // logger is not defined. ErrLoggerIsNotDefined = errors.New("logger is not defined") // ErrReloaderNotConfigured is returned by ReloadSecrets when the proxy was // built without a SecretsReloader, so there is no source to re-read the // secret set from (for example, simple-run mode with no config file). ErrReloaderNotConfigured = errors.New("secrets reloader is not configured") // ErrSecretNotFound is returned by DeleteSecret when no secret with the // requested name exists in the active set. ErrSecretNotFound = errors.New("secret not found") // ErrLastSecret is returned by DeleteSecret when removing the named secret // would leave the proxy with no secrets at all. ErrLastSecret = errors.New("cannot delete the last secret") )
var ( // ErrBanAddresses is wrapped by every validation failure of a ban // request, so the API can map all of them to one status. ErrBanAddresses = errors.New("invalid ban addresses") // ErrBanListFull is a ban request that would take the list past maxBans. ErrBanListFull = errors.New("ban list full") )
var DefaultListenerPolicy = ListenerPolicy{Modes: ModeSetOf(SecretModeFakeTLS)}
DefaultListenerPolicy is what an unstamped connection gets: FakeTLS only. Failing closed here is what keeps a wiring mistake from turning a public port into an obfuscated2 oracle.
var ErrExternalAddresses = errors.New("invalid external addresses")
ErrExternalAddresses is wrapped by every validation failure of an external address report, so the API can map all of them to one status.
var ErrKickAddresses = errors.New("invalid kick addresses")
ErrKickAddresses is wrapped by every validation failure of a kick request, so the API can map all of them to one status.
Functions ¶
func ParseBanAddresses ¶ added in v1.10.0
ParseBanAddresses validates a ban request as the API receives it: at least one place and at most maxBans, since more could never fit, each a literal IP address or an IPv6 /64 (parseBanAddress); a ttl within (0, MaxBanTTL], with an empty one meaning DefaultBanTTL; and, unless force is set, no place whose unit takes in an address that does not stand for one place (clientKeyed: loopback, private, link-local, unspecified, multicast), because behind a front or a PROXY-protocol listener one such address is every client. Every failure wraps ErrBanAddresses.
The unit is what the ban is held at (keyOfAddr), so the check is asked of the address as named and of its unit's first address. That is the whole of it: every IPv6 range of such addresses -- fc00::/7, fe80::/10, ff00::/8 -- is aligned far above a /64, so a /64 is all one kind, except ::/64, which holds :: and ::1 beside ::2 and the IPv4-compatible ::a.b.c.d, and whose first address, ::, is unspecified. An IPv4 address is its own unit.
func ParseExpiry ¶
ParseExpiry parses an expiry deadline in RFC3339 ("2026-12-31T00:00:00Z") or plain date ("2026-12-31") form. An empty string yields the zero time (never).
func ParseExternalAddresses ¶ added in v1.7.0
ParseExternalAddresses validates one report as the API receives it: at most MaxExternalAddresses distinct addresses, each a literal IP with no zone, and a ttl within (0, MaxExternalAddressTTL], with an empty ttl meaning DefaultExternalAddressTTL. Every failure wraps ErrExternalAddresses.
func ParseKickAddresses ¶ added in v1.9.0
ParseKickAddresses validates the places a kick names: at least one literal IP address, each without a zone. Every failure wraps ErrKickAddresses.
func ParseQuotaBytes ¶
ParseQuotaBytes parses a byte size as a bare integer count ("1048576") or a human-friendly unit form ("10GB", "500MiB", ...). An empty string yields 0 (unlimited).
func ParseSpeed ¶ added in v1.6.0
ParseSpeed parses a speed limit into bytes per second. An empty string yields 0 (unlimited). Both the config file and the management API go through this one function, so they cannot disagree about a spelling.
A bare number is bytes per second. Bit units are decimal, as every network link is sold -- "5mbit" is 5,000,000 bits, 625,000 bytes; byte units are binary, as ParseQuotaBytes reads them -- "1MB" is 1,048,576 bytes. A trailing "/s" or "ps" is accepted and means nothing: every value is per second. Case does not matter.
5mbit 500kbit 10gbit 2mbps 1MB/s 512KB/s 625000
func SecretsCanFront ¶ added in v1.1.0
SecretsCanFront reports whether a secret set leaves the always-present FakeTLS listener a cover site to fall back to: at least one ee secret, or an explicit domain-fronting host. Without one, every failed ClientHello on the public port would have nowhere to go, and closing instead of fronting is exactly the distinguisher the fronting path exists to avoid.
Types ¶
type AddressReport ¶ added in v1.7.0
type AddressReport struct {
Name string `json:"name"`
// IPLimit is the configured limit, or absent for none.
IPLimit int `json:"ip_limit,omitempty"`
// ActiveIPs is the number of distinct addresses counted right now: those
// holding a live connection here plus those reported from elsewhere. It
// is a count, so it is never gated.
ActiveIPs int `json:"active_ips,omitempty"`
// ConnectedIPs and ExternalIPs are the addresses themselves, present
// only with track-client-ips, for the reason ProxyOpts.TrackClientIPs
// gives. ExternalExpiresAt is when the reported set stops counting
// unless it is refreshed first.
ConnectedIPs []string `json:"connected_ips,omitempty"`
ExternalIPs []string `json:"external_ips,omitempty"`
ExternalExpiresAt *time.Time `json:"external_expires_at,omitempty"`
// ConnectedIPsLastSeen is when each connected address last moved a byte,
// gated as ConnectedIPs is.
ConnectedIPsLastSeen map[string]time.Time `json:"connected_ips_last_seen,omitempty"`
}
AddressReport is the address view of one user: how many places they count as being in right now, and -- only when the proxy is configured to report addresses at all -- which. IPLimit is filled in by the caller that knows the secret set.
type AntiReplayCache ¶
type AntiReplayCache interface {
// Seen before checks if this set of bytes was observed before or not. If it
// is required to store this information somewhere else, then it has to do
// that.
SeenBefore(data []byte) bool
}
AntiReplayCache is an interface that is used to detect replay attacks based on some traffic fingerprints.
Replay attacks are probe attacks whose main goal is to identify if server software can be classified in some way. For example, if you send some HTTP request to a web server, then you can expect that this server will respond with HTTP response back.
There is a problem though. Let's imagine, that connection is encrypted. Let's imagine, that it is encrypted with some static key like ShadowSocks. In that case, in theory, if you repeat the same bytes, you can get the same responses. Let's imagine, that you've cracked the key. then if you send the same bytes, you can decrypt a response and see its structure. Based on its structure you can identify if this server is SOCKS5, MTPROTO proxy etc.
This is just one example, maybe not the best or not the most relevant. In real life, different organizations use such replay attacks to perform some reverse engineering of the proxy, do some statical analysis to identify server software.
There are many ways how to protect your proxy against them. One is domain fronting which is a core part of mtg. Another one is to collect some 'handshake fingerprints' and forbid duplication.
So, it one is sending the same byte flow right after you (or a couple of hours after), mtg should detect that and reject this connection (or redirect to fronting domain).
type BanJSON ¶ added in v1.10.0
BanJSON is one ban on GET /secrets/{name}/bans: the banned place, an IPv4 address or an IPv6 /64 in prefix form, and the moment the ban lapses, UTC.
type BanResult ¶ added in v1.10.0
type BanResult struct {
Banned int `json:"banned"`
Closed int `json:"closed"`
Addresses []string `json:"addresses,omitempty"`
}
BanResult is the POST /secrets/{name}/bans answer: how many places were banned, how many live streams the ban closed, and -- only with track-client-ips, as PUT external-ips echoes what it was sent -- the places, in the unit they are held at, sorted.
type BansReport ¶ added in v1.10.0
type BansReport struct {
Total int `json:"total"`
Tracking bool `json:"tracking"`
Bans []BanJSON `json:"bans,omitzero"`
}
BansReport is the GET /secrets/{name}/bans body, gated as external_ips is: Total, the number of the secret's bans in force, is a count and is reported whatever Tracking says; Bans, by address, is present only when the proxy lists addresses at all (track-client-ips), and is then an array, empty rather than null when there are none.
type ClientJSON ¶ added in v1.10.0
type ClientJSON struct {
Address string `json:"address"`
ClientUsageJSON
}
ClientJSON is one row of the list: an address -- an IPv4 address, or an IPv6 /64 in prefix form, the unit the address limit and kick count -- and its usage.
The /64 form ("2001:db8::/64") is how the list spells the unit, not something kick reads back: kick takes an address, and any address inside the /64 -- its prefix without the "/64", say -- ends every stream of it. The bans routes take the /64 form as it is, as well as any address inside it.
type ClientUsageJSON ¶ added in v1.10.0
type ClientUsageJSON struct {
Connections int `json:"connections"`
BytesIn int64 `json:"bytes_in"`
BytesOut int64 `json:"bytes_out"`
ThrottledMillis int64 `json:"throttled_ms,omitempty"`
FirstSeen time.Time `json:"first_seen"`
LastSeen time.Time `json:"last_seen"`
SpeedLimit int64 `json:"speed_limit,omitempty"`
SpeedLimitUp int64 `json:"speed_limit_up,omitempty"`
SpeedLimitDown int64 `json:"speed_limit_down,omitempty"`
BannedUntil *time.Time `json:"banned_until,omitempty"`
}
ClientUsageJSON is one address's usage on GET /secrets/{name}/clients, and the rest entry's: everything but the address.
Connections is how many live connections the address holds right now, from the address ledger, which counts every stream of the address whatever entry it holds; for the rest entry, which has no address of its own, it is the streams the rest entry governs. BytesIn and BytesOut are from the client's side, as on the user object. ThrottledMillis is the time the address's streams spent held to its own limiters, absent when none. FirstSeen is when the entry was made and LastSeen the second the address was last seen, both UTC. SpeedLimit, SpeedLimitUp and SpeedLimitDown are the per-address rates in force, in bytes per second, absent where there is none: the combined one is the tighter of the address's ceiling (client-speed) and its fair share of the user's speed, the directional ones its fair shares of speed-up and speed-down. BannedUntil is when a ban on the address lapses, absent when it has none.
type ClientsReport ¶ added in v1.10.0
type ClientsReport struct {
Total int `json:"total"`
Overflow int64 `json:"overflow"`
Tracking bool `json:"tracking"`
Rest *ClientUsageJSON `json:"rest,omitempty"`
Clients []ClientJSON `json:"clients,omitzero"`
}
ClientsReport is the GET /secrets/{name}/clients body.
Total is how many addresses the secret holds an entry for, the rest entry not counted, and Overflow how many connections were sent to the rest entry because the table had no room for their address; both are counts, and are reported whatever Tracking says. Tracking says whether the proxy lists addresses at all (track-client-ips). Only when it does are Rest -- the secret's rest entry, which every unkeyed and overflowed stream shares, absent when the secret has none -- and Clients -- the addresses, most bytes first, ties by address, at most the request's limit -- present. Clients is then an array, empty rather than null when there is nobody.
type DenyReason ¶
type DenyReason int
DenyReason explains why a connection was refused by the per-user limits.
const ( // DenyNone means the connection is allowed. DenyNone DenyReason = iota // DenyDisabled means the secret is administratively disabled. DenyDisabled // DenyExpired means the secret's expiry deadline has passed. DenyExpired // DenyQuota means the secret has used up its data quota. DenyQuota // DenyIPLimit means the connection came from an address the user is not // already counted at, while they are at their address limit. DenyIPLimit // DenyClientConns means the connection came from an address that already // holds the per-address connection cap of live connections to this secret. DenyClientConns // DenyBanned means the connection came from an address an operator banned // from this secret. DenyBanned // DenyKeyRevoked means the connection matched a key its secret no longer // has: the secret set was replaced between the match and the registration, // and the set now in force does not accept that key for the name. DenyKeyRevoked )
func (DenyReason) String ¶
func (d DenyReason) String() string
String returns a short machine-friendly reason label for logs.
type Event ¶
type Event interface {
// StreamID returns an identifier of the stream, connection, request, you name
// it. All events within the same stream returns the same stream id.
StreamID() string
// Timestamp returns a timestamp when this event was generated.
Timestamp() time.Time
}
Event is a data structure which is populated during mtg request processing lifecycle. Each request popluates many events:
- Client connected
- Request is finished
- Connection to Telegram server is established
and so on. All these events are data structures but all of them must conform the same interface.
type EventConcurrencyLimited ¶
type EventConcurrencyLimited struct {
// contains filtered or unexported fields
}
EventConcurrencyLimited is emitted when connection was declined because of the concurrency limit of the worker pool.
func NewEventConcurrencyLimited ¶
func NewEventConcurrencyLimited() EventConcurrencyLimited
NewEventConcurrencyLimited creates a new EventConcurrencyLimited event.
type EventConnectedToDC ¶
type EventConnectedToDC struct {
// RemoteIP is an IP address of the Telegram server proxy has been connected
// to.
RemoteIP net.IP
// DC is an index of the datacenter proxy has been connected to.
DC int
// contains filtered or unexported fields
}
EventConnectedToDC is emitted when mtg proxy has connected to a Telegram server.
func NewEventConnectedToDC ¶
func NewEventConnectedToDC(streamID string, remoteIP net.IP, dc int) EventConnectedToDC
NewEventConnectedToDC creates a new EventConnectedToDC event.
type EventDomainFronting ¶
type EventDomainFronting struct {
// contains filtered or unexported fields
}
EventDomainFronting is emitted when we connect to a front domain instead of Telegram server.
func NewEventDomainFronting ¶
func NewEventDomainFronting(streamID string) EventDomainFronting
NewEventDomainFronting creates a new EventDomainFronting event.
type EventFinish ¶
type EventFinish struct {
// contains filtered or unexported fields
}
EventFinish is emitted when we stop to manage a connection.
func NewEventFinish ¶
func NewEventFinish(streamID string) EventFinish
NewEventFinish creates a new EventFinish event.
type EventIPBlocklisted ¶
type EventIPBlocklisted struct {
RemoteIP net.IP
IsBlockList bool
// contains filtered or unexported fields
}
EventIPBlocklisted is emitted when connection was declined because IP address was found in IP blocklist.
func NewEventIPAllowlisted ¶
func NewEventIPAllowlisted(remoteIP net.IP) EventIPBlocklisted
NewEventIPAllowlisted creates a NewEventIPBlocklisted event with a mark that it is supposed to be for allow list.
func NewEventIPBlocklisted ¶
func NewEventIPBlocklisted(remoteIP net.IP) EventIPBlocklisted
NewEventIPBlocklisted creates a new EventIPBlocklisted event.
type EventIPLimited ¶ added in v1.7.0
type EventIPLimited struct {
SecretName string
// contains filtered or unexported fields
}
EventIPLimited is emitted when a connection is refused because the user is at their address limit and it came from an address they are not counted at. It carries the user and not the address. The refusal's log line is not the same: it is written on the stream logger, whose client-ip comes with it.
func NewEventIPLimited ¶ added in v1.7.0
func NewEventIPLimited(streamID, secretName string) EventIPLimited
NewEventIPLimited creates a new EventIPLimited event.
type EventIPListSize ¶
EventIPListSize is emitted when mtg updates a contents of the ip lists: allowlist or blocklist.
func NewEventIPListSize ¶
func NewEventIPListSize(size int, isBlockList bool) EventIPListSize
NewEventIPListSize creates a new EventIPListSize event.
type EventReplayAttack ¶
type EventReplayAttack struct {
// contains filtered or unexported fields
}
EventReplayAttack is emitted when mtg detects a replay attack on a connection.
func NewEventReplayAttack ¶
func NewEventReplayAttack(streamID string) EventReplayAttack
NewEventReplayAttack creates a new EventReplayAttack event.
type EventStart ¶
type EventStart struct {
// RemoteIP is an IP address of the client.
RemoteIP net.IP
// contains filtered or unexported fields
}
EventStart is emitted when mtg proxy starts to process a new connection.
func NewEventStart ¶
func NewEventStart(streamID string, remoteIP net.IP) EventStart
NewEventStart creates a new EventStart event.
type EventStream ¶
type EventStream interface {
// Send delivers an event to observers. Given context has to be respected. If
// the context is closed, all blocking operations should be released ASAP.
//
// It is possible that context is closed but the message is delivered.
// EventStream implementations should solve this issue somehow.
Send(context.Context, Event)
}
EventStream is an abstraction that accepts a set of events produced by mtg. Its main goal is to inject your logging or monitoring system.
The idea is simple. When mtg works, it emits a set of events during a lifecycle of the requestor: EventStart, EventFinish etc. mtg is a producer which puts these events into a stream. Responsibility of the stream is to deliver this event to consumers/observers. There might be many different observers (for example, you want to have both statsd and prometheus), mtg should know nothing about them.
type EventThrottled ¶
type EventThrottled struct {
SecretName string
// contains filtered or unexported fields
}
EventThrottled is emitted when a connection is rejected because the per-user connection cap has been reached.
func NewEventThrottled ¶
func NewEventThrottled(streamID, secretName string) EventThrottled
NewEventThrottled creates a new EventThrottled event.
type EventTraffic ¶
type EventTraffic struct {
// Traffic is a count of bytes which were transmitted.
Traffic uint
// IsRead defines if we _read_ or _write_ to connection. A rule of thumb is
// simple: EventTraffic is bound to a remote connection. Not to a client one,
// but either to Telegram or front domain one.
//
// In the case of Telegram, isRead means that we've fetched some bytes from
// Telegram to send it to a client.
//
// In the case of the front domain, it means that we've fetched some bytes
// from this domain to send it to a client.
IsRead bool
// contains filtered or unexported fields
}
EventTraffic is emitted when we read/write some bytes on a connection.
func NewEventTraffic ¶
func NewEventTraffic(streamID string, traffic uint, isRead bool) EventTraffic
NewEventTraffic creates a new EventTraffic event.
type FakeTLSJSON ¶ added in v1.10.0
type FakeTLSJSON struct {
Hinted int64 `json:"hinted"`
Scanned int64 `json:"scanned"`
Unmatched int64 `json:"unmatched"`
DeniedIP int64 `json:"denied_ip"`
DeniedGlobal int64 `json:"denied_global"`
Oversized int64 `json:"oversized"`
OverBudgetMatched int64 `json:"over_budget_matched"`
Enforced bool `json:"enforced"`
NsPerBlock float64 `json:"ns_per_block"`
}
FakeTLSJSON is the FakeTLS door's counter block on GET /stats: one number per way a hello past the SNI gate can end, the flag saying whether the handshake budget acts on a refusal, and what the budget believes a SHA-256 block costs. Every count is a total since the process started.
type IPBlocklist ¶
type IPBlocklist interface {
// Contains checks if given IP address belongs to this blocklist If. it is, a
// connection is terminated .
Contains(net.IP) bool
// Run starts a background update procedure for a blocklist
Run(time.Duration)
// Shutdown stops a blocklist. It is assumed that none will access it after.
Shutdown()
}
IPBlocklist filters requests based on IP address.
If this filter has an IP address, then mtg closes a request without reading anything from a socket. It also does not give such request to a worker pool, so in worst cases you can expect that you invoke this object more frequent than defined proxy concurrency.
type KickResult ¶ added in v1.9.0
type KickResult struct {
Closed int `json:"closed"`
}
KickResult is its answer: how many streams were closed.
type ListenerJSON ¶ added in v1.1.0
type ListenerJSON struct {
Bind string `json:"bind"`
Accept []string `json:"accept"`
Public bool `json:"public"`
}
ListenerJSON is one entry of the listener capability report on GET /stats.
type ListenerPolicy ¶ added in v1.1.0
type ListenerPolicy struct {
// Modes is the set of secret forms this socket serves.
Modes ModeSet
// Public is the operator's acknowledgement that a non-loopback obfuscated2
// port is fingerprintable as an MTProto proxy. It also arms HandshakesPerIP.
Public bool
// ProxyProtocol says a PROXY header is expected on this listener. It is
// consumed by the CLI when it builds the socket, not by the proxy.
ProxyProtocol bool
// OnUnknownSecret is the disposal of a refused connection.
OnUnknownSecret UnknownSecretAction
// HandshakesPerIP is the per-IP failed-handshake budget per second (burst
// 2x). 0 disables it, and it is armed only when Public is set.
HandshakesPerIP uint
}
ListenerPolicy is what one listening socket HONOURS. The mode a connection actually uses is always derived (entry path + decrypted tag); this only says which derivations are served. The ZERO VALUE is FakeTLS-only, so an existing Serve(listener) call and a bare ServeConn are byte-for-byte unchanged.
func (ListenerPolicy) Obfuscated ¶ added in v1.1.0
func (p ListenerPolicy) Obfuscated() bool
Obfuscated reports whether this socket serves the pre-FakeTLS entry.
func (ListenerPolicy) Tags ¶ added in v1.1.0
func (p ListenerPolicy) Tags() obfuscation.TagSet
Tags is the set of obfuscated2 connection-type tags this socket honours.
type Logger ¶
type Logger interface {
// Named returns a new logger with a bound name. Name chaining is allowed and
// appreciated.
Named(name string) Logger
// BindInt binds new integer parameter to a new logger instance.
BindInt(name string, value int) Logger
// BindStr binds new string parameter to a new logger instance.
BindStr(name, value string) Logger
// BindJSON binds a new JSON-encoded string to a new logger instance.
BindJSON(name, value string) Logger
// Printf is to support log.Logger behavior.
Printf(format string, args ...any)
// Info puts a message about some normal situation.
Info(msg string)
// InfoError puts a message about some normal situation but this situation is
// related to a given error.
InfoError(msg string, err error)
// Warning puts a message about some extraordinary situation worth to look at.
Warning(msg string)
// WarningError puts a message about some extraordinary situation worth to
// look at. This situation is related to a given error.
WarningError(msg string, err error)
// Debug puts a message useful for debugging only.
Debug(msg string)
// Debug puts a message useful for debugging only. This message is related to
// a given error.
DebugError(msg string, err error)
}
Logger defines an interface of the logger used by mtglib.
Each logger has a name. It is possible to stack names to organize poor-man namespaces. Also, each logger must be able to bind parameters to avoid pushing them all the time.
Example
logger := SomeLogger{} logger = logger.BindStr("ip", net.IP{127, 0, 0, 1})
logger.Info("Hello")
In that case, ip is bound as a parameter. It is a great idea to put this parameter somewhere in a log message.
logger1 = logger.BindStr("param1", "11") logger2 = logger.BindInt("param2",
11)
logger1 should see no param2 and vice versa, logger2 should not see param1 If you attach a parameter to a logger, parents should not know about that.
type ModeSet ¶ added in v1.1.0
type ModeSet uint8
ModeSet is a bitset of SecretMode.
func ModeSetOf ¶ added in v1.1.0
func ModeSetOf(modes ...SecretMode) ModeSet
ModeSetOf builds a set out of the given modes.
func ParseModeSet ¶ added in v1.1.0
ParseModeSet resolves a list of mode names into a set.
func (ModeSet) Has ¶ added in v1.1.0
func (m ModeSet) Has(mode SecretMode) bool
Has reports whether the given mode belongs to this set.
func (ModeSet) Tags ¶ added in v1.1.0
func (m ModeSet) Tags() obfuscation.TagSet
Tags is the set of obfuscated2 connection-type tags this mode set honours at frame offset 56.
dd is padded-intermediate and nothing else -- that is what the mode means. plain is ALL THREE, because a bare 16-byte secret places no constraint on the client's transport: tdlib sends 0xeeeeeeee for it, tdesktop and Telegram-iOS send 0xefefefef, and any of them may pick padded-intermediate. Honouring a subset silently breaks part of the client population.
type Network ¶
type Network interface {
// Dial establishes context-free TCP connections.
Dial(network, address string) (essentials.Conn, error)
// DialContext dials using a context. This is a preferable way of
// establishing TCP connections.
DialContext(ctx context.Context, network, address string) (essentials.Conn, error)
// MakeHTTPClient build an HTTP client with given dial function. If nothing is
// provided, then DialContext of this interface is going to be used.
MakeHTTPClient(func(ctx context.Context, network, address string) (essentials.Conn, error)) *http.Client
// NativeDialer returns a configured instance of native dialer that
// skips proxy connections or any other irrelevant settings.
NativeDialer() *net.Dialer
}
Network defines a knowledge how to work with a network. It may sound fun but it encapsulates all the knowledge how to properly establish connections to remote hosts and configure HTTP clients.
For example, if you want to use SOCKS5 proxy, you probably want to have all traffic routed to this proxy: telegram connections, http requests and so on. This knowledge is encapsulated into instances of such interface.
mtglib uses Network for:
- Dialing to Telegram
- Dialing to front domain
- Doing HTTP requests (for example, for FireHOL ipblocklist).
type ObfuscatedJSON ¶ added in v1.1.0
type ObfuscatedJSON struct {
Trials int64 `json:"trials"`
Misses int64 `json:"misses"`
Denied int64 `json:"denied"`
Draining int64 `json:"draining"`
RateLimited int64 `json:"rate_limited"`
}
ObfuscatedJSON is the obfuscated2 entry counter block on GET /stats. Draining is a live gauge, not a total: it is how many refused connections are sitting in a bounded drain right now.
type Proxy ¶
type Proxy struct {
// contains filtered or unexported fields
}
Proxy is an MTPROTO proxy structure.
func (*Proxy) Ban ¶ added in v1.10.0
Ban bans each of addrs from the named secret for ttl, closes every live stream the secret has at those places, and forgets the secret's hints there. A place is matched at the unit kick matches, so an IPv6 address bans its /64. The secret must exist; ttl must be within (0, MaxBanTTL].
The bans are recorded and the streams found in one critical section under liveMu, the lock admission registers a stream under, so no stream registers between the ban and the kick: one that registered first is found and closed, and one that registers after meets the ban. The streams are closed after the lock is let go, as Kick closes them, because each stream's own unregister takes the lock too. Whether the secret exists is decided under the same lock: a reload swaps its set in before it takes liveMu to drop a removed secret's bans, so a ban recorded here for a secret the set still has is dropped by that reload if it removes it, and a secret already gone is not banned.
func (*Proxy) DeleteSecret ¶
DeleteSecret removes a single secret, its advertising override and its limits. It returns ErrSecretNotFound for an unknown name and ErrLastSecret if the removal would empty the set. The removed secret's live connections are closed, and its retiring entries go with it (forget), whatever their until.
func (*Proxy) DomainFrontingAddress ¶
DomainFrontingAddress returns a host:port pair for a fronting domain. If a fronting host (literal IP or hostname) is configured, it is used instead of the secret's hostname. When secrets use different hostnames, pass the matched secret's host to front the correct domain.
func (*Proxy) GlobalAdTag ¶
func (p *Proxy) GlobalAdTag() *[AdTagLength]byte
GlobalAdTag returns the currently active global advertising tag, or nil when none is set.
func (*Proxy) Kick ¶ added in v1.9.0
Kick closes every live stream of the named secret whose client is at one of addrs and reports how many it closed. An address is matched at the unit the address limit counts, so an IPv6 address kicks its whole /64: the device that rotated its low bits is still the device that was kicked. Streams are snapshotted under liveMu and closed outside it, because each stream's own unregister takes the lock too. The secret must exist; a user with no stream at those places is a kick of zero, not an error.
func (*Proxy) PutSecret ¶
func (p *Proxy) PutSecret(name string, s Secret, adTag *[AdTagLength]byte, limits *SecretLimits) error
PutSecret adds or updates a single secret, optionally setting its per-secret advertising tag (nil clears the override) and governance limits (nil or a zero value clears them). Only a key change closes that secret's live connections; an adtag-only change leaves them running because closeStaleConns keys on the secret key, not the tag. A disable/expiry set here takes effect immediately because swapSecretConfigLocked also closes newly-denied streams. The name's retiring entries are kept, and the old primary never becomes one. The primary wins: a key that repeats one of the name's own entries makes that entry go (rule 6), and a key that is another name's retiring key makes that other name's entry go (rule 5), the key being this name's primary.
func (*Proxy) ReloadSecrets ¶
ReloadSecrets re-reads the secret set through the configured reloader and swaps it in atomically, so a client add, removal, disable or re-key takes effect without restarting the process. Connections whose secret disappeared or was re-keyed are closed; every other live stream keeps running, and the /stats counters carry across the swap. It returns ErrReloaderNotConfigured when the proxy was built without a SecretsReloader, and an error (leaving the current set active) when the reloader fails or yields no valid secret.
func (*Proxy) ReplaceSecrets ¶
func (p *Proxy) ReplaceSecrets(cfg SecretConfig) error
ReplaceSecrets swaps the entire secret configuration (secrets + advertising tags) for cfg, going through the same validation, atomic swap and stale connection close as a file reload.
cfg.Retiring is the COMPLETE list of retiring keys the new set holds, and nil is none, as for a reload: a ReplaceSecrets that leaves it nil ends every overlap in force, and closes the streams on the keys that were retiring. A config file that still grants an overlap grants it again at the next reload, until the entry's own until. The management API's PUT /secrets is the one caller that keeps the overlaps of the names it keeps, through its own step (replaceSecretsKeepingRetiring), not through this method.
func (*Proxy) Serve ¶
Serve starts a proxy on a given listener with the default (FakeTLS-only) policy.
func (*Proxy) ServeConn ¶
func (p *Proxy) ServeConn(conn essentials.Conn)
ServeConn serves a connection. We do not check IP blocklist and concurrency limit here.
func (*Proxy) ServeWithPolicy ¶ added in v1.1.0
func (p *Proxy) ServeWithPolicy(listener net.Listener, policy ListenerPolicy) error
ServeWithPolicy is Serve for a listener whose socket honours something other than FakeTLS. It is Serve's body with one line changed: the accepted conn is stamped with policy before it goes into the worker pool.
func (*Proxy) SetExternalAddresses ¶ added in v1.7.0
SetExternalAddresses replaces the addresses the named user is reported to hold elsewhere. They count against the user's ip limit alongside the addresses of their live connections here, until ttl passes or the next report replaces them. An empty list clears the set. The secret must exist.
func (*Proxy) SetGlobalAdTag ¶
func (p *Proxy) SetGlobalAdTag(tag *[AdTagLength]byte) error
SetGlobalAdTag sets (or, with nil, clears) the global advertising tag. It does not close any live connection: the tag only affects how new connections are routed.
func (*Proxy) Shutdown ¶
func (p *Proxy) Shutdown()
Shutdown 'gracefully' shutdowns all connections. Please remember that it does not close an underlying listener.
func (*Proxy) SnapshotSecrets ¶
func (p *Proxy) SnapshotSecrets() []SecretInfo
SnapshotSecrets returns the active secret set as a name-sorted list, exposing the secret material and both per-secret and effective advertising tags. It reads the atomic snapshot lock-free.
func (*Proxy) Unban ¶ added in v1.10.0
Unban lifts the named secret's ban of addr's place, and says whether there was one. The secret must exist; a place with no ban is lifted of nothing, which is not an error. It takes banMu alone: lifting a ban can only let a stream in that a ban kept out, and admission has nothing to agree on with it.
type ProxyOpts ¶
type ProxyOpts struct {
// Secret defines a secret which should be used by a proxy.
//
// Deprecated: Use Secrets instead for multi-secret support.
// Kept for backward compatibility.
Secret Secret
// Secrets defines a map of named secrets which should be used by a proxy.
// If set, Secret is ignored. During FakeTLS handshake, each secret is
// tried until one validates.
Secrets map[string]Secret
// Network defines a network instance which should be used for all network
// communications made by proxies.
//
// This is a mandatory setting.
Network Network
// AntiReplayCache defines an instance of antireplay cache.
//
// This is a mandatory setting.
AntiReplayCache AntiReplayCache
// IPBlocklist defines an instance of IP blocklist.
//
// This is a mandatory setting.
IPBlocklist IPBlocklist
// IPAllowlist defines a whitelist of IPs to allow to use proxy.
//
// This is an optional setting, ignored by default (no restrictions).
IPAllowlist IPBlocklist
// EventStream defines an instance of event stream.
//
// This ia a mandatory setting.
EventStream EventStream
// Logger defines an instance of the logger.
//
// This is a mandatory setting.
Logger Logger
// BufferSize is a size of the copy buffer in bytes.
//
// Please remember that we multiply this number in 2, because when we relay
// between proxies, we have to create 2 intermediate buffers: to and from.
//
// This is an optional setting.
//
// Deprecated: this setting is no longer makes any effect.
BufferSize uint
// Concurrency is a size of the worker pool for connection management.
//
// If we have more connections than this number, they are going to be
// rejected.
//
// This is an optional setting.
Concurrency uint
// IdleTimeout is a timeout for relay when we have to break a stream.
//
// This is a timeout for any activity. So, if we have any message which will
// pass to either direction, a timer is reset. If we have no any reads or
// writes for this timeout, a connection will be aborted.
//
// This is an optional setting.
IdleTimeout time.Duration
// HandshakeTimeout is a timeout during which all handshake ceremonies must
// be completed, otherwise this process will be aborted
//
// This is an optional setting.
HandshakeTimeout time.Duration
// TolerateTimeSkewness is a time boundary that defines a time range where
// faketls timestamp is acceptable.
//
// This means that if if you got a timestamp X, now is Y, then if |X-Y| <
// TolerateTimeSkewness, then you accept a packet.
//
// This is an optional setting.
TolerateTimeSkewness time.Duration
// PreferIP defines an IP connectivity preference. Valid values are:
// 'prefer-ipv4', 'prefer-ipv6', 'only-ipv4', 'only-ipv6'.
//
// This is an optional setting.
PreferIP string
// AutoUpdate defines if it is required to auto update proxy list from
// Telegram instead of relying on a hardcoded list.
//
// This is an optional setting.
AutoUpdate bool
// DomainFrontingPort is a port we use to connect to a fronting domain.
//
// This is required because secret does not specify a port. It specifies a
// hostname only.
//
// This is an optional setting.
DomainFrontingPort uint
// DomainFrontingHost is the address to use when connecting to the
// fronting domain instead of resolving the hostname from the secret via
// DNS. It can be a literal IP or a hostname; hostnames are resolved at
// dial time via the native dialer (which honours dual-stack and Happy
// Eyeballs).
//
// This is useful when DNS resolution of the secret's hostname is blocked
// or loops back to this server. The hostname from the secret is still
// used for SNI in the TLS handshake.
//
// This is an optional setting.
DomainFrontingHost string
// DomainFrontingIP previously held the dial target for the fronting
// domain. The setting is no longer honoured: setting it logs a warning
// at proxy startup and the value is dropped.
//
// Deprecated: use DomainFrontingHost. Setting this field has no effect.
DomainFrontingIP string
// DomainFrontingProxyProtocol is used if communication between upstream
// endpoint and mtg supports proxy protocol. This is useful in case
// if mtg is also placed behind load balancer, and this will make
// fronting webserver to know about real IP addresses
//
// This is an optional setting.
DomainFrontingProxyProtocol bool
// TrackClientIPs makes GET /stats report per-user client addresses:
// connected_ips, the distinct addresses holding a live connection right
// now, and last_ip, the address of the most recent admitted connection.
//
// It defaults to FALSE, and that default is deliberate. This is a
// circumvention proxy: the people using it are frequently the reason it
// exists, and an operator's answer to "who connected from where" is also
// the answer anyone who takes the server gets. connected_ips costs no new
// retention -- those addresses are already held for the duration of the
// connection, and vanish with it -- but last_ip outlives the connection it
// came from, so a deployment that never asks for it never accumulates it.
//
// Nothing here is written to disk. The usage state file holds quota
// counters only, so an address survives a restart in no form at all.
//
// This is an optional setting.
TrackClientIPs bool
// HandshakeHints makes the FakeTLS door remember, per client address (an
// IPv4 address or an IPv6 /64), the names of up to 16 secrets that recently
// completed a handshake from it, and verify a hello from that address
// against those first. A returning client then costs one HMAC instead of a
// scan over every hosted secret. A hint is only an order to try things in:
// every hinted secret passes the same verification, so it never makes a
// wrong secret match, and a hello that matches nothing can neither read nor
// plant one. Loopback, private, link-local, unspecified and multicast
// addresses get none -- behind a carrier that does not forward the client
// address they would be one shared key, and a multicast address is no
// client at all.
//
// nil follows TrackClientIPs, and that default is the point: the cache holds
// address -> secret name in memory for 30 to 60 minutes after the last
// successful handshake, so it is on exactly when the operator has already
// chosen to keep client addresses past a connection. A non-nil value
// overrides it either way. Nothing here is logged, exported or written to
// disk.
//
// This is an optional setting.
HandshakeHints *bool
// HandshakeBudgetMode is what the FakeTLS door does with a hello it would
// have to scan every hosted secret for (one no hint matched) when that scan
// is over its CPU budget, or when the hello is above the scan cap of 128
// SHA-256 blocks (about 8 KiB; a real client sends 0.5 to 2 KiB):
//
// - "enforce" (or empty, the default) lets a scan the budget can pay for
// within 150 ms wait for it and run, and fronts any other hello at once to
// the cover site exactly as a hello that matched nothing is fronted --
// never closes it;
// - "observe" keeps the same accounting but never waits and never refuses:
// it counts what enforce would have refused, and the refused hellos that
// then matched a secret, and scans anyway;
// - "off" has no budget and no scan cap.
//
// The budget is in real CPU: the cost of a SHA-256 block on this machine is
// measured once per process. Under enforce a legitimate client with no hint
// can be fronted while a flood keeps the budget spent; it retries. With
// hints on (HandshakeHints), a client whose address matched a secret
// recently has its hinted candidates tried before, and outside, the budget;
// a hint miss falls through to a scan that is charged like any other.
//
// The zero values of this and the next two settings are the defaults:
// enforce, 50 ms per address, 25% of the cores.
//
// This is an optional setting.
HandshakeBudgetMode string
// HandshakeBudgetPerIPMillis is the scan CPU one client address (an IPv4
// address or an IPv6 /64) may spend per second, in milliseconds, with a
// burst of four seconds' worth. Loopback, private, link-local and other
// addresses that do not stand for one place have no bucket of their own;
// they share the process-wide one.
//
// nil is 50. 0 disables the per-address bucket, leaving only the
// process-wide one -- which is what peers behind a carrier that does not
// forward client addresses get anyway, since they all arrive from a loopback
// or private address. A front proxy on a public address that sends no PROXY
// header is the opposite case: every client arrives from that one address
// and shares one bucket, so 0 is what it wants. Above 1000 is invalid.
//
// This is an optional setting.
HandshakeBudgetPerIPMillis *uint
// HandshakeBudgetCPUPercent is the share of the cores (GOMAXPROCS, read
// once at start) the whole process may spend scanning, with a burst of one
// second's worth. This is the bound that holds against many addresses and
// against forged ones.
//
// 0 is 25. Above 100 is invalid.
//
// This is an optional setting.
HandshakeBudgetCPUPercent uint
// AllowFallbackOnUnknownDC defines how proxy behaves if unknown DC was
// requested. If this setting is set to false, then such connection will be
// rejected. Otherwise, proxy will chose any DC.
//
// Telegram is designed in a way that any DC can serve any request, the
// problem is a latency.
//
// This is an optional setting.
AllowFallbackOnUnknownDC bool
// UseTestDCs defines if we have to connect to production or to staging DCs of
// Telegram.
//
// This is required if you use mtglib as an integration library for your
// Telegram-related projects.
//
// This is an optional setting.
//
// OBSOLETE and DEPRECATED. Ignored.
UseTestDCs bool
// DCOverrides defines a set of IP addresses that should be used
// with a higher priority to those that are calculated somehow by mtg.
//
// OBSOLETE and DEPRECATED. Ignored.
DCOverrides map[int][]string
// DoppelGangerURLs is a list of URLs that should be crawled by
// mtg to calculate parameters for statistical distribution of a
// traffic for fronting domains. If nothing is given, then predefined
// statistics is going to be used.
DoppelGangerURLs []string
// DoppelGangerPerRaid defines how many time each URL from
// DoppelGangerURLs list should be crawled per raid. We recommend to
// have this number ~10.
DoppelGangerPerRaid uint
// DoppelGangerEach defines a time period between each raid. We recommend
// to use hours here.
DoppelGangerEach time.Duration
// DoppelGangerDRS defines if TLS Dynamic Record Sizing is active.
DoppelGangerDRS bool
// DoppelGangerShaping is how the doppelganger spaces its records in time:
// "record" (or empty, the default) waits a drawn delay before every
// record; "burst" waits once when a connection wakes from idle and then
// sends records back to back.
DoppelGangerShaping string
// DoppelGangerScoutCovers measures the cover sites the ee secrets name --
// their certificate flight's size, for the fake handshake's noise record
// -- when DoppelGangerURLs is empty. Their record timing is not taken:
// pacing keeps the built-in model. It makes outbound HTTPS requests to
// those sites, so a library caller has to ask; the mtg-proxy binary does
// unless [defense.doppelganger] scout-covers = false.
DoppelGangerScoutCovers bool
// APIBindTo is the address to bind the stats HTTP API server to.
// If empty, the stats API server is not started.
//
// This is an optional setting.
APIBindTo string
// APIListener is a socket already bound for the management API. When it
// is set, APIBindTo is ignored and the proxy serves the API on this socket
// instead, taking ownership of it: it is closed when the proxy shuts down.
//
// It exists for an embedder or a test that needs the port to be its own
// before the proxy starts, rather than one the proxy picks and reports.
//
// This is an optional setting.
APIListener net.Listener
// ThrottleMaxConnections is the total connection limit. When total
// connections exceed this value, per-user caps are computed using
// a fair-share algorithm and new connections from over-cap users
// are rejected. 0 disables throttling.
//
// This is an optional setting.
ThrottleMaxConnections uint
// ThrottleCheckInterval is how often the throttle recomputes per-user
// caps. Defaults to 5 seconds.
//
// This is an optional setting.
ThrottleCheckInterval time.Duration
// SecretsReloader re-reads the desired secret configuration from its source
// (for the CLI, the config file passed to `mtg run`). When set, a POST to
// /reload on the stats API triggers Proxy.ReloadSecrets, which calls this
// and swaps the secret set (and advertising tags) in without a restart.
// When nil, /reload reports the capability is unavailable and the proxy
// behaves exactly as before.
//
// This is an optional setting.
SecretsReloader func() (SecretConfig, error)
// GlobalAdTag is the advertising tag (ad_tag from @MTProxybot) applied to
// every secret without a per-secret override. When set, matching clients
// are routed through Telegram middle proxies so a sponsored channel is
// shown. nil disables advertising globally.
//
// This is an optional setting.
GlobalAdTag *[AdTagLength]byte
// SecretAdTags carries per-secret advertising tag overrides keyed by secret
// name. A secret listed here uses its own tag instead of GlobalAdTag.
//
// This is an optional setting.
SecretAdTags map[string][AdTagLength]byte
// SecretLimits carries per-secret governance limits (data quota, expiry,
// disabled flag) keyed by secret name. A secret absent here is unlimited,
// never expires and is enabled.
//
// This is an optional setting.
SecretLimits map[string]SecretLimits
// RetiringSecrets is the complete list of retiring keys the initial set
// holds: old keys of a secret name, each accepted until its own Until after
// the name was given a new primary key, with the name's quota, limits,
// bans and stats. nil means none. It has the meaning SecretConfig.Retiring
// has, and an entry the set cannot hold is dropped and logged, never an
// error.
//
// This is an optional setting.
RetiringSecrets []RetiringSecret
// RotationClock is the clock retiring keys expire by. One per process,
// shared by whatever reads the configuration and the proxy, so that both
// judge an overlap by the same instant and neither is ever handed a time
// earlier than one the other has seen. nil makes NewProxy build one of its
// own on the wall clock.
//
// This is an optional setting.
RotationClock *RotationClock
// UsageStateFile is the path of a JSON file where per-secret quota usage is
// persisted so it survives restarts. When empty, quota accounting is
// in-memory only and resets when the process restarts.
//
// This is an optional setting.
UsageStateFile string
// EmitTraffic controls whether per-read/write EventTraffic events are
// produced for relayed connections. These events are consumed only by the
// statsd/prometheus observers, so leave it false when no metrics sink is
// configured: doing so skips a heap allocation (plus time.Now, a streamID
// hash and a channel send) on every relay read and write. The CLI sets it
// automatically when statsd or prometheus is enabled.
//
// This is an optional setting.
EmitTraffic bool
// PublicIPv4 and PublicIPv6 advertise this proxy's externally reachable
// addresses. They are used as the proxy's own address in the middle-proxy
// RPC handshake and RPC_PROXY_REQ; required to be correct behind NAT.
//
// These are optional settings.
PublicIPv4 net.IP
PublicIPv6 net.IP
// AdvertisedPort is the port reported as the proxy's own port in
// RPC_PROXY_REQ. Usually the first bind port.
//
// This is an optional setting.
AdvertisedPort int
// Listeners describes every socket this proxy was configured to serve, for
// the capability report on GET /stats. It is display only: the policy that
// actually governs a connection travels on the connection itself.
//
// This is an optional setting.
Listeners []ListenerJSON
// ListenerModes is the union of every configured listener's accept set. It
// is what tells NewProxy that a dd or plain secret has no door, so the
// operator gets a warning instead of a client that silently cannot connect.
//
// This is an optional setting.
ListenerModes ModeSet
// APIToken, when non-empty, requires every management-API request to carry
// an "Authorization: Bearer <token>" header. Empty keeps the API
// unauthenticated (relying on the localhost-bind convention).
//
// This is an optional setting.
APIToken string
}
ProxyOpts is a structure with settings to mtg proxy.
This is not required per se, but this is to shorten function signature and give an ability to conveniently provide default values.
type ProxyStats ¶
type ProxyStats struct {
// contains filtered or unexported fields
}
ProxyStats tracks per-secret connection stats with atomic counters. Thread-safe for concurrent access from proxy goroutines.
func NewProxyStats ¶
func NewProxyStats() *ProxyStats
NewProxyStats creates a new ProxyStats instance.
func (*ProxyStats) AddBytesIn ¶
func (s *ProxyStats) AddBytesIn(name string, n int64)
AddBytesIn adds to the bytes-in counter for the given secret.
func (*ProxyStats) AddBytesOut ¶
func (s *ProxyStats) AddBytesOut(name string, n int64)
AddBytesOut adds to the bytes-out counter for the given secret.
func (*ProxyStats) CanConnect ¶
func (s *ProxyStats) CanConnect(name string) bool
CanConnect returns true if the user is allowed to open a new connection under the current throttle caps. If throttling is not configured or the user has no cap, it always returns true.
func (*ProxyStats) FlushUsage ¶
func (s *ProxyStats) FlushUsage(path string) error
FlushUsage writes the current quota usage of every known secret to path atomically (write-temp-then-rename), so a crash never leaves a truncated file.
func (*ProxyStats) LoadUsage ¶
func (s *ProxyStats) LoadUsage(path string) error
LoadUsage restores persisted quota usage from path. A missing file is not an error (there is simply nothing to restore yet).
func (*ProxyStats) OnAdTagFailed ¶ added in v1.4.0
func (s *ProxyStats) OnAdTagFailed(name string)
OnAdTagFailed counts one ad-tagged stream whose transport the middle proxy does handle, and which went direct anyway because the middle proxy could not be reached or turned it away. Kept apart from OnAdTagSkipped: a skip is a design outcome, a failure is an operational one, and only the second means something is wrong.
func (*ProxyStats) OnAdTagSkipped ¶ added in v1.1.0
func (s *ProxyStats) OnAdTagSkipped(name string)
OnAdTagSkipped counts one ad-tagged stream that could not be routed through the middle proxy and went direct instead, losing only its sponsored channel.
func (*ProxyStats) OnClientConnsRefused ¶ added in v1.10.0
func (s *ProxyStats) OnClientConnsRefused(name string)
OnClientConnsRefused counts one connection refused by the per-address connection cap.
func (*ProxyStats) OnConnect ¶
func (s *ProxyStats) OnConnect(name string)
OnConnect increments the active connection count for the given secret.
func (*ProxyStats) OnDisconnect ¶
func (s *ProxyStats) OnDisconnect(name string)
OnDisconnect decrements the active connection count for the given secret.
func (*ProxyStats) OnIPLimitRefused ¶ added in v1.7.0
func (s *ProxyStats) OnIPLimitRefused(name string)
OnIPLimitRefused counts one connection refused by the user's address limit.
func (*ProxyStats) PreRegister ¶
func (s *ProxyStats) PreRegister(name string)
PreRegister adds a secret name to the stats map so it appears in output even if no connections have been made yet.
func (*ProxyStats) QuotaUsed ¶
func (s *ProxyStats) QuotaUsed(name string, reset QuotaReset) int64
QuotaUsed returns the bytes counted against the secret's current quota period, rolling the monthly period over first when it is due.
func (*ProxyStats) ResetQuota ¶
func (s *ProxyStats) ResetQuota(name string)
ResetQuota zeroes the used-bytes counter for the secret and restarts its quota period from now.
func (*ProxyStats) ServeHTTP ¶
func (s *ProxyStats) ServeHTTP(w http.ResponseWriter, _ *http.Request)
func (*ProxyStats) SetDirectionalSpeedLimits ¶ added in v1.9.0
func (s *ProxyStats) SetDirectionalSpeedLimits(name string, upBytesPerSecond, downBytesPerSecond int64)
SetDirectionalSpeedLimits installs a user's per-direction caps, or unlimited for zero, re-rating the buckets in place the way SetSpeedLimit does.
func (*ProxyStats) SetLastIP ¶ added in v1.2.0
func (s *ProxyStats) SetLastIP(name, ip string)
SetLastIP records the client address of the most recent admitted connection for the given secret. Callers gate this on the proxy's tracking setting; it is not checked here, so a caller that reaches this has already decided.
func (*ProxyStats) SetQuotaCeiling ¶ added in v1.8.0
func (s *ProxyStats) SetQuotaCeiling(name string, bytes int64)
SetQuotaCeiling installs the quota the data path holds a user to, or unlimited for zero. The live counter is untouched: a ceiling lowered below it ends the user's streams on their next byte and the sweep closes the idle ones.
func (*ProxyStats) SetSpeedLimit ¶ added in v1.6.0
func (s *ProxyStats) SetSpeedLimit(name string, bytesPerSecond int64)
SetSpeedLimit installs a user's speed limit, or unlimited for zero, re-rating the bucket their live connections are drawing from.
func (*ProxyStats) SetThrottle ¶
func (s *ProxyStats) SetThrottle(limit int64, interval time.Duration)
SetThrottle configures connection throttling. Must be called before startThrottleLoop and before any connections arrive.
func (*ProxyStats) UpdateLastSeen ¶
func (s *ProxyStats) UpdateLastSeen(name string)
UpdateLastSeen sets the last-seen timestamp for the given secret to now.
type QuotaReset ¶
type QuotaReset int32
QuotaReset defines how a per-secret data quota is replenished.
const ( // QuotaResetNone means the quota is a single lifetime allowance that is // never automatically replenished (it can still be cleared explicitly via // the management API). QuotaResetNone QuotaReset = iota // QuotaResetMonthly means the used-bytes counter is reset to zero at the // start of every calendar month. QuotaResetMonthly )
func ParseQuotaReset ¶
func ParseQuotaReset(s string) (QuotaReset, error)
ParseQuotaReset parses the reset policy spelling. Empty and "none" both map to QuotaResetNone.
func (QuotaReset) String ¶
func (q QuotaReset) String() string
String returns the config/API spelling of the reset policy.
type RetiringSecret ¶ added in v1.10.0
RetiringSecret is a key a secret name still accepts until Until, after the name was given a new primary key. Its traffic is the name's: same quota, limits, bans, stats.
type RotationClock ¶ added in v1.10.0
type RotationClock struct {
// contains filtered or unexported fields
}
RotationClock is the wall clock retiring keys expire by, never allowed to run backwards within a process. One instance per process, shared by whatever reads a config and the proxy, so that both judge an overlap by the same instant: an NTP step back after an overlap's until must not make a later reload accept the old key again.
It is a high-water mark over a time source. Now returns the larger of what the source says and the latest time it has returned. When the source is more than a second behind that mark, the clock reports the episode once, through the callback given to NewRotationClock, and judges by the mark all the same.
Every time a clock returns is a wall time with no monotonic reading, whichever way it was reached, so that it compares with an Until read from a file, and with one built from time.Now().Add(...), by the wall clock alone. (A nil *RotationClock is not a clock but time.Now, and returns it as it is.)
The mark is kept in unix nanoseconds, so the source must return times within that range, the years 1678 to 2262; time.Now always does. Outside it the reading wraps, and the clock would hold a mark that means nothing.
The zero value is usable: it is a high-water clock on time.Now that reports nothing. A nil *RotationClock reads time.Now with no mark at all. A RotationClock must not be copied after first use.
func NewRotationClock ¶ added in v1.10.0
func NewRotationClock(now func() time.Time, onBehind func(time.Duration)) *RotationClock
NewRotationClock builds a clock on the given time source (nil is time.Now). onBehind, if not nil, is called with how far the source was behind the clock's mark, once for each episode of the source being more than a second behind it.
The source must return times within the unix-nanosecond range, the years 1678 to 2262.
onBehind is called synchronously, from inside Now, by whichever caller meets the episode first. The proxy calls Now while it holds its reload lock, and while a stream registers under its registry lock, so onBehind must be quick: log and return. It must not block and must not call back into the proxy.
func (*RotationClock) Now ¶ added in v1.10.0
func (c *RotationClock) Now() time.Time
Now is the time to judge an expiry by: the larger of the source's time and the latest time this clock has returned, which it then holds. A nil receiver reads time.Now. It is safe for concurrent use.
The result of a non-nil clock is a wall time with no monotonic reading, on every path: the source's reading with it stripped, or the mark.
A source at or past the mark moves the mark and ends a behind episode. A source more than a second behind the mark starts one, and onBehind is called for it once, whichever goroutine gets there first, until a read reaches the mark again. A source behind the mark by a second or less neither starts nor ends an episode.
type Secret ¶
type Secret struct {
// Key is a set of bytes used for traffic authentication.
Key [SecretKeyLength]byte
// Host is a domain fronting hostname. Required by, and meaningful only to,
// SecretModeFakeTLS: dd and plain never send SNI and never front.
Host string
// Mode is the wire form. The ZERO VALUE is SecretModeFakeTLS, so every
// existing Secret{Key: k, Host: h} literal in the tree and its tests keeps
// its exact current meaning with no edit.
Mode SecretMode
}
Secret is a data structure that presents a secret.
Telegram secret is not a simple string like "ee367a189aee18fa31c190054efd4a8e9573746f726167652e676f6f676c65617069732e636f6d". Actually, this is a serialized datastructure of 3 parts: a mode prefix, a key and a host.
ee367a189aee18fa31c190054efd4a8e9573746f726167652e676f6f676c65617069732e636f6d |-|-------------------------------|------------------------------------------- p key hostname
There are 3 wire forms, and the length discriminates between them before the first byte does:
16 bytes - "plain": a bare key, obfuscated2, no TLS.
17 bytes starting 0xdd - "dd": obfuscated2 with padded-intermediate framing.
0xee + 16 bytes + a host - "ee": faketls plus protection from statistical
attacks on a length. This is what mtg generates
and what every modern client should be given.
After the prefix we have 16 bytes of the key. This is a random generated secret data of the proxy and this data is used to derive authentication schemas. These secrets are mixed into hmacs and sha256 checksums which are used to build AEAD ciphers for obfuscated2 protocol and ensure faketls handshake. The key is identical in all 3 forms: the prefix tells the CLIENT how to connect, it does not change the cryptography.
Host is a domain fronting hostname in latin1 (ASCII) encoding. This hostname should be used for SNI in faketls and MTG verifies it. Also, this is when mtg gets about a domain fronting hostname. Only the 'ee' form carries it.
Secrets can be serialized into 2 forms: hex and base64. If you decode both forms into bytes, you'll get the same byte array. Telegram clients nowadays accept all forms.
func GenerateSecret ¶
GenerateSecret makes a new secret with a given hostname.
func GenerateSecretMode ¶ added in v1.1.0
func GenerateSecretMode(mode SecretMode, hostname string) (Secret, error)
GenerateSecretMode makes a new secret of the given mode. hostname is used only by SecretModeFakeTLS and must be empty for the other two.
func ParseSecret ¶
ParseSecret parses a secret (both hex and base64 forms).
func (Secret) DDHex ¶ added in v1.1.0
DDHex renders THIS KEY in the dd link form, regardless of stored Mode. The three forms are presentations of the same 16 bytes.
func (Secret) Hosted ¶ added in v1.1.0
Hosted reports whether this secret can serve the FakeTLS entry: it has an SNI hostname to be matched by and a cover site to front to.
func (Secret) MarshalText ¶
MarshalText is to support text.Marshaller interface.
func (Secret) PlainHex ¶ added in v1.1.0
PlainHex renders this key as a bare 32-hex-character link secret.
func (*Secret) UnmarshalText ¶
UnmarshalText is to support text.Unmarshaller interface.
func (Secret) Valid ¶
Valid checks if this secret is valid and can be used in proxy.
A key is always required; a hostname is required only by the ee form, which is the only one that carries it on the wire. Demanding a Host from a dd/plain secret would mean storing a hostname that is a lie, and that lie would then enter secretSet.hostnames and make SNI matching answer to a domain no real ee client ever presents.
type SecretConfig ¶
type SecretConfig struct {
// Secrets is the name->secret map. It must be non-empty and every secret
// must be valid.
Secrets map[string]Secret
// SecretAdTags carries optional per-secret advertising tag overrides keyed
// by secret name. A name absent here falls back to GlobalAdTag.
SecretAdTags map[string][AdTagLength]byte
// GlobalAdTag is the advertising tag applied to any secret without an
// override. nil means no advertising (the direct-DC path).
GlobalAdTag *[AdTagLength]byte
// Limits carries optional per-secret governance limits (data quota, expiry,
// disabled flag) keyed by secret name. A name absent here has no limits. It
// is swapped in atomically together with the secrets, so a reload or API
// mutation can change limits without a restart.
Limits map[string]SecretLimits
// Retiring is the COMPLETE list of retiring keys the swapped-in set is to
// hold: old keys of a name, each accepted until its own Until after the
// name was given a new primary key. nil means none, wherever a
// SecretConfig is consumed -- ReloadSecrets, ReplaceSecrets and every
// other swap -- never "keep what is there". An entry the set cannot hold
// (a value Secret.Valid() refuses, expired, for a name not in Secrets, on
// another name's key or its own primary's, a duplicate, past a cap) is
// dropped and logged, never an error.
//
// For embedders: a ReplaceSecrets with a nil Retiring ends every overlap
// in force. The next reload of a config file that still grants an
// overlap grants it again, until its Until.
Retiring []RetiringSecret
}
SecretConfig is the full set of runtime-swappable secret state produced by a SecretsReloader: the named secrets plus the optional advertising tags. It is what POST /reload re-reads from the config file and what the management API mutators build before an atomic swap.
type SecretInfo ¶
type SecretInfo struct {
Name string `json:"name"`
Secret string `json:"secret"`
Host string `json:"host"`
AdTag string `json:"ad_tag,omitempty"`
EffectiveAdTag string `json:"effective_ad_tag,omitempty"`
Quota int64 `json:"quota,omitempty"`
QuotaUsed int64 `json:"quota_used,omitempty"`
QuotaRemaining *int64 `json:"quota_remaining,omitempty"`
QuotaReset string `json:"quota_reset,omitempty"`
ExpiresAt *time.Time `json:"expires_at,omitempty"`
Disabled bool `json:"disabled,omitempty"`
// SpeedLimit is the user's cap in bytes per second, upload and download
// combined across every connection, or absent for unlimited. It is set
// with a human spelling ("5mbit", "1MB/s") and read back as a number, the
// same way quota is.
SpeedLimit int64 `json:"speed_limit,omitempty"`
// SpeedLimitUp and SpeedLimitDown are the per-direction caps, from the
// client's side, in the same unit; absent for unlimited.
SpeedLimitUp int64 `json:"speed_limit_up,omitempty"`
SpeedLimitDown int64 `json:"speed_limit_down,omitempty"`
// IPLimit is how many distinct addresses the user may be at once, or
// absent for unlimited; ActiveIPs is how many they count as being at
// right now. Neither is an address, so neither is gated.
IPLimit int `json:"ip_limit,omitempty"`
ActiveIPs int `json:"active_ips,omitempty"`
// ClientConns is how many live connections one address may hold to this
// secret at once, or absent for unlimited. A limit, not an address.
ClientConns int `json:"client_conns,omitempty"`
// ClientSpeedLimit is the per-address ceiling in bytes per second, set as
// client_speed with a human spelling and read back as a number, the way
// speed is read back as speed_limit; absent for unlimited.
ClientSpeedLimit int64 `json:"client_speed_limit,omitempty"`
// ClientFair is whether the secret's speed limits are divided among its
// addresses, client_fair; absent when they are not. A switch, not an
// address.
ClientFair bool `json:"client_fair,omitempty"`
// Mode is the wire form this secret serialises to: "ee", "dd" or "plain".
Mode string `json:"mode"`
// Unserved is true when no configured listener can serve this secret's form:
// a dd or plain secret on a proxy with no [[listener]]. The secret is stored
// and valid, it just has no door. It is NOT an error -- one odd client must
// never take an inbound offline -- but it is the thing an operator needs to
// see.
Unserved bool `json:"unserved,omitempty"`
// Connections and LastSeen are the runtime view of this secret, carried
// here so a caller rendering one user does not have to fetch and filter the
// whole of GET /stats to learn whether anybody is on it. They are the same
// numbers /stats reports, read from the same counters.
Connections int64 `json:"connections"`
LastSeen *time.Time `json:"last_seen"`
// ConnectedIPs and LastIP mirror the same fields on GET /stats, and are
// filled in on exactly the same condition: only when the proxy is
// configured with track-client-ips. See ProxyOpts.TrackClientIPs for why
// that is off by default.
//
// Connections and LastSeen above are NOT gated, because they are not
// addresses -- knowing that a user has two connections says nothing about
// where they are.
ConnectedIPs []string `json:"connected_ips,omitempty"`
LastIP string `json:"last_ip,omitempty"`
// ConnectedIPsLastSeen is when each connected address last moved a byte,
// gated as ConnectedIPs is.
ConnectedIPsLastSeen map[string]time.Time `json:"connected_ips_last_seen,omitempty"`
// ExternalIPs and ExternalExpiresAt are the addresses a panel reported
// this user present at elsewhere and when that report lapses, gated
// exactly as ConnectedIPs is.
ExternalIPs []string `json:"external_ips,omitempty"`
ExternalExpiresAt *time.Time `json:"external_expires_at,omitempty"`
// RetiringKeys, RetiringUntil, RetiringConnections and RetiringLastSeen mirror
// the same four fields on GET /stats, read from the same set and the same
// counters, and absent for a secret with no unexpired previous key. A count,
// two times and a count: none is an address, so none is gated, and none is
// ever a key.
RetiringKeys int `json:"retiring_keys,omitempty"`
RetiringUntil *time.Time `json:"retiring_until,omitempty"`
RetiringConnections int `json:"retiring_connections,omitempty"`
RetiringLastSeen *time.Time `json:"retiring_last_seen,omitempty"`
}
SecretInfo is the JSON representation of a single named secret returned by GET /secrets. AdTag is the per-secret override (empty when none) and EffectiveAdTag is the tag actually applied after global fallback. The quota and expiry fields describe the secret's governance limits (omitted when unset), and QuotaUsed/QuotaRemaining report live usage.
type SecretLimits ¶
type SecretLimits struct {
// QuotaBytes is the maximum number of relayed bytes (client read + write)
// allowed in the current quota period. 0 means unlimited.
QuotaBytes int64
// QuotaReset selects how QuotaBytes is replenished.
QuotaReset QuotaReset
// ExpiresAt is the instant after which the secret stops working. The zero
// time means the secret never expires.
ExpiresAt time.Time
// Disabled, when true, rejects every new connection for the secret without
// removing it from the set.
Disabled bool
// SpeedBytesPerSecond is the sustained throughput this user is allowed,
// in bytes per second, upload and download combined, across every
// connection and every door they use at once. 0 means unlimited.
//
// It is a per-user allowance, not a per-connection one: two devices on
// the same secret share it. It is enforced by holding reads and writes
// to the rate rather than by buffering, so backpressure reaches the
// client's own TCP window and nothing is queued in memory.
SpeedBytesPerSecond int64
// SpeedUpBytesPerSecond and SpeedDownBytesPerSecond cap one direction each,
// from the client's side: up is what the client sends, down is what it
// receives. Each is its own bucket, so a user may be given a slow upload
// and a fast download, and both apply on top of the combined cap when that
// is set too. 0 means unlimited.
SpeedUpBytesPerSecond int64
SpeedDownBytesPerSecond int64
// IPLimit is how many distinct addresses this user may hold connections
// from at once, counting the addresses of their live connections here
// together with any a panel has reported them present at elsewhere. 0
// means unlimited.
//
// A connection from an address already counted is always admitted -- a
// second socket from the same phone is the same device -- and one from a
// new address is refused while the count is at the limit. An IPv6
// address counts as its /64. Refusal happens at admission, after the
// secret matched, so only a holder of the valid secret can observe it.
IPLimit int
// ClientConns is how many live connections one address may hold to this
// secret at once, client-conns: the per-address connection cap. 0 means
// unlimited.
//
// It is counted from the address ledger -- the same unit and the same live
// count IPLimit reads, so a v4-mapped address is its IPv4 address and an
// IPv6 address is its /64 -- and not from the per-address table, so it
// holds when the table is full and counts the connections that were open
// before it was set. A connection from an address already holding that
// many is refused after the handshake, as an IPLimit refusal is. An
// address that does not stand for one place (loopback, private and the
// like: behind a carrier that forwards no client address, every client) is
// never capped. Distinct from IPLimit, which counts places: a Telegram
// client opens several connections, so this is set to a few times that.
ClientConns int
// ClientSpeedBytesPerSecond is how fast one address may move bytes on this
// secret, upload and download combined across all of its connections,
// client-speed: the per-address ceiling. 0 means unlimited.
//
// It is the address's own bucket, beside the user's SpeedBytesPerSecond,
// which still holds the secret as a whole: an address is held to whichever
// is reached first, and two addresses are each held to it, not to it
// between them. An address is the unit IPLimit counts (an IPv4 address, an
// IPv6 /64). Every peer that does not stand for one place, and every
// address the per-address table has no room for, shares the secret's one
// rest entry, which has no ceiling: it stands for many clients at once, and
// one address's ceiling would hold all of them together.
ClientSpeedBytesPerSecond int64
// ClientFair, client-fair, divides the secret's own speed limits among the
// addresses using it: a work-conserving share, worked out again every
// second, in which an address that wants less than an even split keeps
// what it wants and the others share the rest equally, however many
// connections each holds. Each of SpeedBytesPerSecond,
// SpeedUpBytesPerSecond and SpeedDownBytesPerSecond that is set is divided
// on its own, so it needs at least one of them. Without one of speed /
// speed-up / speed-down the engine treats it as off -- nothing is divided,
// though as a per-address control it still keeps the secret's addresses'
// entries; only the config and the HTTP API refuse it. The rest entry --
// every address that does not stand for one place -- holds one share
// between them.
ClientFair bool
}
SecretLimits carries the optional per-user governance limits attached to a named secret: a data quota (with an optional monthly reset), a validity deadline, and an on/off switch. The zero value means "no limits" — unlimited traffic, never expires, enabled — so a secret without an entry is unrestricted.
func (SecretLimits) IsZero ¶
func (l SecretLimits) IsZero() bool
IsZero reports whether the limits impose no restriction at all, i.e. the secret is unlimited, never-expiring and enabled. Callers use it to avoid materialising empty entries in maps and API responses.
type SecretMode ¶ added in v1.1.0
type SecretMode uint8
SecretMode is the wire form a secret is PRESENTED in. It is not an access control: the same Key derives the same ciphers in all three modes, so which mode a connection actually uses is decided by the listener it arrived on (ListenerPolicy) plus the transport tag it decrypts to. Mode never gates admission -- see doFakeTLSHandshake, which selects HMAC candidates by Hosted(), and doObfuscatedEntry, which trial-decrypts against every key.
const ( SecretModeFakeTLS SecretMode = iota // "ee" + key + hostname; the ZERO VALUE SecretModeDD // "dd" + key; obfuscated2, no TLS SecretModePlain // bare 16-byte key; obfuscated2, no TLS )
func ParseSecretMode ¶ added in v1.1.0
func ParseSecretMode(s string) (SecretMode, error)
ParseSecretMode resolves a mode name. "faketls" is accepted as a synonym of "ee" because that is what the mode does; "ee" is what the link says.
func (SecretMode) MarshalText ¶ added in v1.1.0
func (m SecretMode) MarshalText() ([]byte, error)
MarshalText is to support text.Marshaller interface.
func (SecretMode) String ¶ added in v1.1.0
func (m SecretMode) String() string
String is to support fmt.Stringer interface.
func (*SecretMode) UnmarshalText ¶ added in v1.1.0
func (m *SecretMode) UnmarshalText(b []byte) error
UnmarshalText is to support text.Unmarshaller interface.
type StatsResponse ¶
type StatsResponse struct {
StartedAt time.Time `json:"started_at"`
UptimeSeconds int64 `json:"uptime_seconds"`
TotalConnections int64 `json:"total_connections"`
Features []string `json:"features"`
Throttle *ThrottleJSON `json:"throttle,omitempty"`
// Listeners and Obfuscated describe the extra obfuscated2 entry. Both are
// omitted when no [[listener]] is configured, so a deployment that never
// declared one keeps a byte-identical response.
Listeners []ListenerJSON `json:"listeners,omitempty"`
Obfuscated *ObfuscatedJSON `json:"obfuscated,omitempty"`
// FakeTLSHandshake is what the FakeTLS door decided about the hellos that
// reached its secret scan. It is omitted until the door has counted
// something, and it is a top-level object on purpose: nothing about the door
// is a user, and the panel bills every key under Users as one.
FakeTLSHandshake *FakeTLSJSON `json:"faketls_handshake,omitempty"`
Users map[string]UserStatsJSON `json:"users"`
}
StatsResponse is the JSON response for the stats endpoint.
type ThrottleJSON ¶
type ThrottleJSON struct {
Active bool `json:"active"`
Limit int64 `json:"limit"`
Caps map[string]int64 `json:"caps,omitempty"`
}
ThrottleJSON is the throttle portion of the stats JSON response.
type UnbanResult ¶ added in v1.10.0
type UnbanResult struct {
Removed int `json:"removed"`
}
UnbanResult is the DELETE /secrets/{name}/bans/{address...} answer: 1 when a ban was lifted, 0 when the place had none.
type UnknownSecretAction ¶ added in v1.1.0
type UnknownSecretAction uint8
UnknownSecretAction is how an obfuscated2 listener disposes of a connection it refused. The ZERO VALUE is close, which is the right answer on loopback.
const ( // UnknownSecretClose closes a refused connection immediately. UnknownSecretClose UnknownSecretAction = iota // UnknownSecretDrain reads and discards from a refused connection, bounded // in both bytes and time, before closing. On a public listener a socket that // neither answers nor hangs up is less distinctive than a prompt reset. UnknownSecretDrain )
type UserStatsJSON ¶
type UserStatsJSON struct {
Connections int64 `json:"connections"`
BytesIn int64 `json:"bytes_in"`
BytesOut int64 `json:"bytes_out"`
LastSeen *time.Time `json:"last_seen"`
QuotaUsed int64 `json:"quota_used"`
Quota int64 `json:"quota,omitempty"`
QuotaRemaining *int64 `json:"quota_remaining,omitempty"`
QuotaReset string `json:"quota_reset,omitempty"`
ExpiresAt *time.Time `json:"expires_at,omitempty"`
Disabled bool `json:"disabled,omitempty"`
// QuotaCutoffs counts streams ended by reaching the quota, since the
// process started. A quota nobody reaches shows none.
QuotaCutoffs int64 `json:"quota_cutoffs,omitempty"`
// AdTagSkipped counts streams that carried an ad tag but took the direct
// path anyway because their transport has no middle-proxy reframer. An
// operator who monetises through @MTProxybot has to be able to see this: a
// silent skip is a revenue bug nobody can diagnose.
AdTagSkipped int64 `json:"ad_tag_skipped,omitempty"`
// AdTagFailed counts streams that carried an ad tag, on a transport the
// middle proxy handles, and took the direct path because the middle proxy
// could not be reached or refused them. Unlike a skip this is not by
// design: a proxy on a network that cannot reach core.telegram.org, or
// whose ad tag was revoked, shows up here and nowhere else.
AdTagFailed int64 `json:"ad_tag_failed,omitempty"`
// SpeedLimit is the user's configured cap in bytes per second, and
// ThrottledMillis the time their connections have spent held to it. A
// user with a limit and no throttled time is not reaching it; one whose
// throttled time grows by a second a second is living at it.
SpeedLimit int64 `json:"speed_limit,omitempty"`
ThrottledMillis int64 `json:"throttled_ms,omitempty"`
// SpeedLimitUp and SpeedLimitDown are the per-direction caps in bytes per
// second, from the client's side, or absent for unlimited. ThrottledMillis
// above counts the time held to any of the three buckets, and to the
// per-address limiters of the addresses the user connects from.
SpeedLimitUp int64 `json:"speed_limit_up,omitempty"`
SpeedLimitDown int64 `json:"speed_limit_down,omitempty"`
// IPLimit is the user's address limit; ActiveIPs is how many distinct
// addresses they count as being at right now, the addresses of their
// live connections here plus any reported from elsewhere; and
// IPLimitRefused is how many connections were turned away for arriving
// from one address too many. The first two are absent when zero, the
// count is never gated: how many places a user is in says nothing about
// where.
IPLimit int `json:"ip_limit,omitempty"`
ActiveIPs int `json:"active_ips,omitempty"`
IPLimitRefused int64 `json:"ip_limit_refused,omitempty"`
// ClientConns is the per-address connection cap, read back, and
// ClientConnsRefused is how many connections were turned away for arriving
// from an address that already held that many. Scalars inside the user
// object, never a key under users and never an address: both are absent
// when zero, and neither is gated.
ClientConns int `json:"client_conns,omitempty"`
ClientConnsRefused int64 `json:"client_conns_refused,omitempty"`
// ClientSpeedLimit is the per-address ceiling in bytes per second, read
// back as SpeedLimit is, and absent for unlimited. Time an address spent
// held to it is in ThrottledMillis.
ClientSpeedLimit int64 `json:"client_speed_limit,omitempty"`
// ClientFair is whether the user's speed limits are divided among the
// addresses they connect from, read back, and absent when they are not.
// Time an address spent held to its share is in ThrottledMillis.
ClientFair bool `json:"client_fair,omitempty"`
// Clients is how many source addresses the per-address table holds an
// entry for, the shared rest entry not counted, and ClientsOverflow how
// many connections were sent to the rest entry because the table had no
// room for their address, since the secret was added. Counts, never
// addresses: both are absent when zero and neither is gated. The addresses
// themselves are on GET /secrets/{name}/clients, never here.
Clients int `json:"clients,omitempty"`
ClientsOverflow int64 `json:"clients_overflow,omitempty"`
// ConnectedIPs lists the distinct client addresses holding a live
// connection right now, sorted. LastIP is the address of the most recent
// admitted connection, which outlives that connection. ExternalIPs are
// the addresses a panel reported the user present at elsewhere, and
// ExternalExpiresAt is when that report lapses unless refreshed.
//
// All are filled in only when the proxy is configured with
// track-client-ips, and all are omitted when empty, so /stats on a
// deployment that never enabled it is byte-identical to before they
// existed. See ProxyOpts.TrackClientIPs for why that is the default.
ConnectedIPs []string `json:"connected_ips,omitempty"`
LastIP string `json:"last_ip,omitempty"`
ExternalIPs []string `json:"external_ips,omitempty"`
ExternalExpiresAt *time.Time `json:"external_expires_at,omitempty"`
// ConnectedIPsLastSeen is, for each address in ConnectedIPs, the second a
// stream from it last moved a byte: the latest across the streams sharing
// the address. Gated and omitted exactly as ConnectedIPs is, so a panel
// can tell a device that fell silent from one still talking.
ConnectedIPsLastSeen map[string]time.Time `json:"connected_ips_last_seen,omitempty"`
// RetiringKeys, RetiringUntil, RetiringConnections and RetiringLastSeen are
// the read-back of a rotation's overlap: how many previous keys of the secret
// are still accepted, when the last of them stops being (UTC), how many live
// streams hold a key that is not the secret's current one, and when a stream
// last came in on a previous key. Four scalars inside the user object, never
// a key under users and never a key's value: all are absent for a secret
// with no unexpired previous key, so /stats on a deployment that never
// rotates is byte-identical to before they existed, and none is gated.
// handleStats fills them from the set in force, onto users that are already
// there; ProxyStats itself knows nothing of keys.
RetiringKeys int `json:"retiring_keys,omitempty"`
RetiringUntil *time.Time `json:"retiring_until,omitempty"`
RetiringConnections int `json:"retiring_connections,omitempty"`
RetiringLastSeen *time.Time `json:"retiring_last_seen,omitempty"`
}
UserStatsJSON is the per-user portion of the stats JSON response. The quota and expiry fields describe the secret's configured limits and are filled in by the proxy (which holds the secret snapshot); ProxyStats itself only fills the runtime counters, including QuotaUsed.
Source Files
¶
- addr_limiter.go
- address_limit.go
- blocked_tracker.go
- client_stats.go
- client_table.go
- conns.go
- counting_conn.go
- events.go
- fair_share.go
- faketls_counters.go
- handshake_budget.go
- handshake_hints.go
- handshake_limiter.go
- init.go
- mode.go
- proxy.go
- proxy_api.go
- proxy_bans.go
- proxy_entry.go
- proxy_kick.go
- proxy_opts.go
- proxy_retiring_view.go
- proxy_secrets.go
- proxy_serve.go
- proxy_stats.go
- proxy_telegram.go
- rotation_clock.go
- secret.go
- secret_limits.go
- secret_rotation.go
- speed_limit.go
- stream_context.go
- water_level.go
Directories
¶
| Path | Synopsis |
|---|---|
|
Package dcprobe verifies that a TCP endpoint is a real Telegram DC by performing the unauthenticated first step of the MTProto handshake (req_pq_multi -> resPQ) on top of mtg's existing obfuscated2 transport.
|
Package dcprobe verifies that a TCP endpoint is a real Telegram DC by performing the unauthenticated first step of the MTProto handshake (req_pq_multi -> resPQ) on top of mtg's existing obfuscated2 transport. |
|
internal
|
|
|
middleproxy
Package middleproxy implements the client side of Telegram's MTProto middle proxy (RPC) protocol.
|
Package middleproxy implements the client side of Telegram's MTProto middle proxy (RPC) protocol. |