updates

package
v0.165.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 12, 2026 License: MIT Imports: 23 Imported by: 51

Documentation

Overview

Package updates provides a Telegram's internalState synchronization manager.

It guarantees that all internalState-sensitive updates will be performed in correct order.

Limitations:

  1. Manager cannot verify stateless types of updates (tg.UpdatesClass without Seq, or tg.UpdateClass without Pts or Qts).

  2. Due to the fact that updates.getDifference and updates.getChannelDifference do not return event sequences, manager cannot guarantee the correctness of these operations. We rely on the server here.

  3. Manager cannot fully recover the channel gap if there is a ChannelDifferenceTooLong error. The gap range and the latest messages returned by the server are reported via Config.OnChannelGap (not the handler, as they may have been delivered already), so missed messages can be fetched manually. See: https://core.telegram.org/constructor/updates.channelDifferenceTooLong

Failed differences

A failed updates.getDifference or updates.getChannelDifference is retried with backoff (1 s doubling up to 60 s, then every 60–80 s), like TDLib and Telegram Desktop; FLOOD_WAIT is waited out. A lost authorization is not retried. PERSISTENT_TIMESTAMP_INVALID and PERSISTENT_TIMESTAMP_EMPTY on the common sequence reset the state from updates.getState and call Config.OnTooLong, as the updates in between are lost. CHANNEL_INVALID stops tracking the channel, like CHANNEL_PRIVATE. Config.Clock drives the retries.

Observability

Events are logged via Config.Logger: debug logs for every difference request, gap, tracked channel and update received before Run, warnings for abnormal ones (e.g. a channel gap whose pts did not advance).

With Config.MeterProvider, the manager records these OpenTelemetry metrics (meter "github.com/gotd/td/telegram/updates"). Attributes never carry channel or user IDs.

  • gotd.updates.difference.requests (counter): getDifference (kind=common) and getChannelDifference (kind=channel) calls by result: empty, slice, full, too_long, error, inaccessible.
  • gotd.updates.difference.duration (histogram, s): duration of those calls, same attributes.
  • gotd.updates.difference.triggers (counter): why the difference was requested, by kind and reason (e.g. startup, pts-gap-timeout, idle-timeout, channel-too-long, and retry or channel-retry for a retry of a failed difference).
  • gotd.updates.gaps (counter): sequence gaps by kind, seq (pts, qts, seq) and action: detected, resolved_by_waiting, resolved_by_pending. Gaps not resolved in time trigger a difference with a *-gap-timeout reason.
  • gotd.updates.channel.too_long (counter): channelDifferenceTooLong, i.e. Config.OnChannelGap calls, by dialog: ok, stale (pts did not advance), invalid (no dialog pts).
  • gotd.updates.channel.gap.size (histogram, {pts}): RemotePts - LocalPts of those gaps.
  • gotd.updates.channel.gap.messages (histogram, {message}): length of ChannelGap.Messages.
  • gotd.updates.before_run (counter, {batch}): updates received before Run by action: passed, buffered, dropped, replayed, passed_raw (replayed to the handler as-is because the state was fetched from the server).
  • gotd.updates.channel.track (counter): channel tracking by source (api for Manager.TrackChannel, stored, update, too_long) and result (started, already, pending, stopped, canceled, error).
  • gotd.updates.channels.tracked (up-down counter): channels tracked by running managers.
  • gotd.updates.channel.inaccessible (counter): channels removed from tracking after CHANNEL_PRIVATE or CHANNEL_INVALID.
  • gotd.updates.stopped (counter): calls that returned ErrStopped, by op: handle, handle_affected, track_channel.
  • gotd.updates.dispatched (counter, {update}): updates passed to the handler, by source: live or difference.

Index

Constants

This section is empty.

Variables

View Source
var ErrStopped = errors.New("updates manager stopped")

ErrStopped is returned by Handle, HandleAffected and TrackChannel after Run stopped, until Reset.

Functions

This section is empty.

Types

type API added in v0.80.0

type API interface {
	UpdatesGetState(ctx context.Context) (*tg.UpdatesState, error)
	UpdatesGetDifference(ctx context.Context, request *tg.UpdatesGetDifferenceRequest) (tg.UpdatesDifferenceClass, error)
	UpdatesGetChannelDifference(ctx context.Context, request *tg.UpdatesGetChannelDifferenceRequest) (tg.UpdatesChannelDifferenceClass, error)
}

API is the interface which contains Telegram RPC methods used by manager for internalState synchronization.

type AuthOptions added in v0.80.0

type AuthOptions struct {
	IsBot   bool
	Forget  bool
	OnStart func(ctx context.Context)
}

type BeforeRunMode

type BeforeRunMode int

BeforeRunMode is what Manager does with updates received before Run.

const (
	// BeforeRunPass passes updates received before Run to Handler as-is,
	// without order verification, gap recovery or short updates
	// transformation. Default.
	BeforeRunPass BeforeRunMode = iota
	// BeforeRunBuffer keeps updates received before Run, at most
	// Config.BeforeRunBufferSize batches, and pushes them through gap recovery
	// once Run starts. Updates over the limit are dropped. Updates received
	// while they are replayed are buffered too, so they are handled in order.
	//
	// If Run fetches the state from the server (AuthOptions.Forget, or no
	// stored state), the buffered updates are already covered by it, so they
	// are passed to Handler as-is, like with BeforeRunPass.
	//
	// Use it when handlers are ready only after authorization, e.g.
	// UpdateDispatcher handlers registered in Run's OnStart.
	BeforeRunBuffer
	// BeforeRunDrop drops updates received before Run.
	BeforeRunDrop
)

type ChannelAccessHasher added in v0.51.0

type ChannelAccessHasher interface {
	SetChannelAccessHash(ctx context.Context, userID, channelID, accessHash int64) error
	GetChannelAccessHash(ctx context.Context, userID, channelID int64) (accessHash int64, found bool, err error)
}

ChannelAccessHasher stores users channel access hashes.

type ChannelGap

type ChannelGap struct {
	// ChannelID is the channel ID.
	ChannelID int64
	// AccessHash is the channel access hash.
	AccessHash int64
	// LocalPts is the local pts before the gap.
	LocalPts int
	// RemotePts is the new pts after the gap (pts of the channel dialog).
	RemotePts int
	// TopMessage is the ID of the latest channel message (from the
	// channel dialog), zero if unknown.
	TopMessage int
	// Messages are the latest channel messages returned with the gap. They
	// may include messages delivered to Handler before the gap, so they are
	// not passed to Handler.
	Messages []tg.MessageClass
	// Users and Chats are entities of Messages.
	Users []tg.UserClass
	Chats []tg.ChatClass
}

ChannelGap describes a channel gap that the manager cannot recover (updates.channelDifferenceTooLong).

type Config

type Config struct {
	// Handler where updates will be passed.
	Handler telegram.UpdateHandler
	// StopOnApplyError makes Run return on Handler or StateStorage errors
	// while applying updates, instead of logging them and continuing
	// (default).
	//
	// When enabled, a failed Handler call is not followed by the checkpoint
	// write of its apply path, so the pts, qts, seq or channel pts is not
	// advanced in memory or in Storage; a failed StateStorage write does not
	// advance the corresponding in-memory state; a difference slice is not
	// continued. Errors of RPC calls keep their usual handling.
	//
	// This is an error policy, not a transaction: completed steps and
	// concurrent work (e.g. other channels) are not rolled back, a Storage
	// may commit before returning an error, and a handler may have partially
	// applied the failed batch. Handlers must be idempotent, as updates
	// after the last successful checkpoint are delivered again. After
	// resolving the failure, restart with a new Manager and Forget=false to
	// resume from the stored state.
	//
	// Updates outside any sequence that a channel difference forwards to the
	// common state are handed off before the channel pts is stored, so a
	// failure to handle them does not hold that pts back.
	StopOnApplyError bool
	// Callback called if manager cannot
	// recover channel gap (optional).
	//
	// See also OnChannelGap, which provides the gap range.
	OnChannelTooLong func(channelID int64)
	// Callback called if manager cannot recover channel gap, i.e. on
	// updates.channelDifferenceTooLong, with the gap range (optional).
	//
	// The latest channel messages returned by the server with
	// channelDifferenceTooLong are passed in gap.Messages, not to Handler:
	// they may include messages delivered before the gap. Messages between
	// the last known message and gap.Messages should be fetched manually,
	// e.g. via messages.getHistory.
	//
	// Called after OnChannelTooLong.
	OnChannelGap func(gap ChannelGap)
	// Callback called when the manager loses access to a channel, detected via
	// CHANNEL_PRIVATE or CHANNEL_INVALID on updates.getChannelDifference (e.g. the account was
	// kicked/banned or the channel was deleted). The channel is removed from
	// the update manager after this call (optional).
	OnChannelInaccessible func(channelID int64)
	// Callback called if manager cannot recover
	// common state gap, i.e. on updates.differenceTooLong (optional).
	OnTooLong func()
	// Callback called when the manager fails to load the locally stored user
	// state during Run with forget=false (no state found), so the state is
	// fetched from the server and a full resynchronization is performed.
	//
	// Useful to detect that updates missed while offline may need to be fetched
	// manually, e.g. via messages.getHistory (optional).
	OnLoadUserStateFailed func()
	// Callback called when the manager fails to load the locally stored state of
	// a channel during Run with forget=false (its access hash is missing), so
	// the channel is skipped.
	//
	// Useful to detect that the channel may need to be resynchronized manually,
	// e.g. by calling Manager.TrackChannel with its access hash, which resumes
	// from the stored pts (optional).
	OnLoadChannelStateFailed func(channelID int64)
	// State storage.
	// In-mem used if not provided.
	Storage StateStorage
	// Channel access hash storage. In-mem used if not provided.
	//
	// The manager needs the access hash of a channel to track it. Hashes are
	// learned from the entities of updates and differences; for a channel
	// the manager has not seen there yet (e.g. known only from dialogs), its
	// first update costs an extra updates.getDifference to find the hash.
	// Pass a hasher that also knows the peers the rest of the app has seen,
	// such as *peers.Manager, or a persistent one (FileStorage, the contrib
	// storages), to avoid that and to keep channels across restarts.
	AccessHasher ChannelAccessHasher
	// User access hash storage. If not provided, AccessHasher is used when
	// it implements UserAccessHasher (e.g. *peers.Manager, FileStorage),
	// otherwise an in-mem one.
	UserAccessHasher UserAccessHasher
	// Logger (optional).
	Logger log.Logger
	// TracerProvider (optional).
	TracerProvider trace.TracerProvider
	// MeterProvider for metrics of the manager (optional). No metrics by
	// default. See the package documentation for the list.
	MeterProvider metric.MeterProvider
	// MaxChannelDifferenceConcurrency limits how many updates.getChannelDifference
	// requests may be in flight at once across all tracked channels. Non-positive
	// values (<= 0, default 0) mean unlimited: every channel recovers its gap as
	// before. A positive value bounds the burst so an account in many active
	// channels does not exceed Telegram's per-account method rate limit.
	MaxChannelDifferenceConcurrency int
	// BeforeRun is what to do with updates received before Run, e.g. while
	// the client connects and authorizes. Default is BeforeRunPass.
	//
	// Dropped updates (BeforeRunDrop, or over BeforeRunBufferSize) of the
	// common or a tracked channel sequence are recovered via getDifference
	// once the next update reveals the gap.
	BeforeRun BeforeRunMode
	// BeforeRunBufferSize limits how many update batches are kept with
	// BeforeRunBuffer. Non-positive means 100.
	BeforeRunBufferSize int

	// Clock is used for retries of failed differences. Defaults to
	// clock.System.
	Clock clock.Clock
}

Config of the manager.

type FileStorage

type FileStorage struct {
	// contains filtered or unexported fields
}

FileStorage is a StateStorage, ChannelAccessHasher and UserAccessHasher that keeps the update state of every user, the pts of their channels and the channel and user access hashes in one JSON file, so that a restarted client gets the updates it missed while stopped.

Every change rewrites the file atomically (a temporary file in the same directory, then rename), so the file is never left half-written, but a crash of the machine may lose the last changes and lead to updates being delivered again. It suits bots and small accounts; for accounts in many active channels use a database-backed storage, e.g. from gotd/contrib.

Use the same FileStorage as Config.Storage and Config.AccessHasher: channel state is restored on start only together with its access hash.

A FileStorage must not be shared by processes. It is safe for concurrent use.

func NewFileStorage

func NewFileStorage(path string) *FileStorage

NewFileStorage returns a FileStorage that keeps the state in the file at path. The file is created on the first change, and its directory must exist.

func (*FileStorage) ForEachChannels

func (s *FileStorage) ForEachChannels(
	ctx context.Context, userID int64,
	f func(ctx context.Context, channelID int64, pts int) error,
) error

ForEachChannels implements StateStorage.

func (*FileStorage) GetChannelAccessHash

func (s *FileStorage) GetChannelAccessHash(_ context.Context, userID, channelID int64) (int64, bool, error)

GetChannelAccessHash implements ChannelAccessHasher.

func (*FileStorage) GetChannelPts

func (s *FileStorage) GetChannelPts(_ context.Context, userID, channelID int64) (int, bool, error)

GetChannelPts implements StateStorage.

func (*FileStorage) GetState

func (s *FileStorage) GetState(_ context.Context, userID int64) (State, bool, error)

GetState implements StateStorage.

func (*FileStorage) GetUserAccessHash

func (s *FileStorage) GetUserAccessHash(_ context.Context, userID, targetUserID int64) (int64, bool, error)

GetUserAccessHash implements UserAccessHasher.

func (*FileStorage) SetChannelAccessHash

func (s *FileStorage) SetChannelAccessHash(_ context.Context, userID, channelID, accessHash int64) error

SetChannelAccessHash implements ChannelAccessHasher.

func (*FileStorage) SetChannelPts

func (s *FileStorage) SetChannelPts(_ context.Context, userID, channelID int64, pts int) error

SetChannelPts implements StateStorage.

func (*FileStorage) SetDate

func (s *FileStorage) SetDate(_ context.Context, userID int64, date int) error

SetDate implements StateStorage.

func (*FileStorage) SetDateSeq

func (s *FileStorage) SetDateSeq(_ context.Context, userID int64, date, seq int) error

SetDateSeq implements StateStorage.

func (*FileStorage) SetPts

func (s *FileStorage) SetPts(_ context.Context, userID int64, pts int) error

SetPts implements StateStorage.

func (*FileStorage) SetQts

func (s *FileStorage) SetQts(_ context.Context, userID int64, qts int) error

SetQts implements StateStorage.

func (*FileStorage) SetSeq

func (s *FileStorage) SetSeq(_ context.Context, userID int64, seq int) error

SetSeq implements StateStorage.

func (*FileStorage) SetState

func (s *FileStorage) SetState(_ context.Context, userID int64, state State) error

SetState implements StateStorage. It forgets the pts of the user's channels, but keeps their access hashes.

func (*FileStorage) SetUserAccessHash

func (s *FileStorage) SetUserAccessHash(_ context.Context, userID, targetUserID, accessHash int64) error

SetUserAccessHash implements UserAccessHasher.

type Manager added in v0.51.0

type Manager struct {
	// contains filtered or unexported fields
}

Manager deals with gaps.

Important: Updates produced by this manager may contain negative Pts/Qts/Seq values in tg.UpdateClass/tg.UpdatesClass (does not affects to the tg.MessageClass).

This is because telegram server does not return these sequences for getDifference/getChannelDifference results. You SHOULD NOT use them in update handlers at all.

func New

func New(cfg Config) *Manager

New creates new manager.

func (*Manager) Handle added in v0.51.0

func (m *Manager) Handle(ctx context.Context, u tg.UpdatesClass) error

Handle handles updates.

Important: If Run method not called, updates are handled according to Config.BeforeRun: by default, they are passed to the provided handler as-is without any order verification or short updates transformation.

func (*Manager) HandleAffected added in v0.157.0

func (m *Manager) HandleAffected(ctx context.Context, channelID int64, pts, ptsCount int) error

HandleAffected applies the pts increment carried by a messages.affectedMessages or messages.affectedHistory RPC result, keeping the local pts in sync after a self-initiated read or delete (e.g. messages.readHistory, messages.deleteMessages).

Without this, such results silently advance the server pts while the local pts stays behind, so the next genuine update looks like a gap and is postponed until the next pts-changing update or a getDifference (see issue #1382).

channelID must be the channel the result belongs to, or 0 for the common (user/basic-group) sequence. The affected pts for a channel is applied only when that channel is already tracked. It is a no-op before Run is called.

Most callers should not call this directly: use the hook middleware in telegram/updates/hook, which wires it from RPC results automatically.

func (*Manager) Reset added in v0.80.0

func (m *Manager) Reset()

Reset notifies manager about user logout.

func (*Manager) Run added in v0.80.0

func (m *Manager) Run(ctx context.Context, api API, userID int64, opt AuthOptions) error

Run notifies manager about user authentication on the telegram server.

If forget is true, local internalState (if exist) will be overwritten with remote internalState.

func (*Manager) TrackChannel

func (m *Manager) TrackChannel(ctx context.Context, channelID, accessHash int64, pts int) error

TrackChannel starts tracking updates of the channel, if it is not tracked yet.

The manager tracks only channels with a stored pts and access hash, and learns about other channels only from their next update, so messages posted before it are missed. Use TrackChannel to track channels joined while the client was offline, or channels reported via Config.OnLoadChannelStateFailed, e.g. with pts of the channel dialog from messages.getDialogs (Dialog.Pts).

If the channel has a stored pts, tracking starts from it, so missed updates are fetched via updates.getChannelDifference. Otherwise it starts from the given pts, which must be positive.

Can be called before Run or from AuthOptions.OnStart: such channels start being tracked when the manager starts, and errors are only logged.

type State

type State struct {
	Pts, Qts, Date, Seq int
}

State is the user internalState.

type StateStorage added in v0.51.0

type StateStorage interface {
	GetState(ctx context.Context, userID int64) (state State, found bool, err error)
	SetState(ctx context.Context, userID int64, state State) error
	SetPts(ctx context.Context, userID int64, pts int) error
	SetQts(ctx context.Context, userID int64, qts int) error
	SetDate(ctx context.Context, userID int64, date int) error
	SetSeq(ctx context.Context, userID int64, seq int) error
	SetDateSeq(ctx context.Context, userID int64, date, seq int) error
	GetChannelPts(ctx context.Context, userID, channelID int64) (pts int, found bool, err error)
	SetChannelPts(ctx context.Context, userID, channelID int64, pts int) error
	ForEachChannels(ctx context.Context, userID int64, f func(ctx context.Context, channelID int64, pts int) error) error
}

StateStorage is the users internalState storage.

Note: SetPts, SetQts, SetDate, SetSeq, SetDateSeq should return error if user internalState does not exist.

type UserAccessHasher added in v0.150.0

type UserAccessHasher interface {
	SetUserAccessHash(ctx context.Context, userID, targetUserID, accessHash int64) error
	GetUserAccessHash(ctx context.Context, userID, targetUserID int64) (accessHash int64, found bool, err error)
}

UserAccessHasher stores user access hashes observed by the updates manager so it can decide whether an incoming update references a user it already knows. Mirrors ChannelAccessHasher. In-mem used if not provided to Config.

Directories

Path Synopsis
Package hook contains telegram update hook middleware.
Package hook contains telegram update hook middleware.
internal
e2e
Package e2e contains end-to-end updates processing test.
Package e2e contains end-to-end updates processing test.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL