rdpgfx

package
v0.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Sep 19, 2026 License: GPL-3.0 Imports: 12 Imported by: 0

Documentation

Overview

clear.go 实现 MS-RDPEGFX 2.2.4 ClearCodec（TS_CLEARCODEC_BITMAP_STREAM） 的完整解码，算法对齐 FreeRDP libfreerdp/codec/clear.c。

流结构：glyphFlags(1) + seqNumber(1) + [glyph 段] + residualByteCount(4) + bandsByteCount(4) + subcodecByteCount(4) + 三段载荷。

Index

Constants

View Source 
const (
	ReplayKindWireToSurface1 byte = 1
	ReplayKindWireToSurface2 byte = 2
	ReplayKindCacheToSurface byte = 3
	ReplayKindSurfaceToCache byte = 4
	ReplayKindSolidFill      byte = 5
	ReplayKindSurfaceToSurf  byte = 6
	ReplayKindEvictCache     byte = 7
	ReplayKindResetGraphics  byte = 8
)

Replay harness record kinds (see SetPduRecorder).

View Source 
const (
	ChannelName = plugin.RDPGFX_DVC_CHANNEL_NAME
)

Variables

This section is empty.

Functions

func DecodeRLGR3ForDebug 

func DecodeRLGR3ForDebug(data []byte, outputSize int) []int16

DecodeRLGR3ForDebug exposes the RLGR3 decoder for offline verification.

func DecodeSurfaceRFX 

func DecodeSurfaceRFX(data []byte, width, height int) []byte

DecodeSurfaceRFX decodes non-progressive RemoteFX (MS-RDPRFX) encoded data into a top-down BGRA pixel buffer suitable for surface bitmap commands.

func SetH264Backend 

func SetH264Backend(b *H264DecoderBackend)

SetH264Backend registers the H.264 decoder backend. Must be called before any RDP session is started. When the h264 build tag is set, example/h264_ffmpeg.go calls this in its init().

Types

type BitmapUpdate 

type BitmapUpdate struct {
	DestLeft, DestTop, DestRight, DestBottom int
	Width, Height                            int
	Bpp                                      int    // bytes per pixel (always 4)
	Data                                     []byte // BGRA pixel data — see lifecycle note above
}

BitmapUpdate represents a rendered bitmap region.

Lifecycle: Data is borrowed from an internal buffer pool and is only valid for the duration of the synchronous onBitmap callback. After the callback returns, the slice may be returned to the pool and overwritten by subsequent updates. Callers that need to retain the pixels (e.g. to hand them to an asynchronous paint goroutine) MUST copy the bytes before the callback returns.

type GfxCacheEntry 

type GfxCacheEntry struct {
	Key           uint64
	Width, Height int
	Bpp           uint16
	Data          []byte
}

GfxCacheEntry 是一条跨连接保留的位图缓存条目（MS-RDPEGFX 持久位图缓存）。 Key 为服务器在 SurfaceToCache 中给出的 cacheKey（持久身份），Data 为当时 的 BGRA 像素副本。重连后经 CacheImportOffer 上报，服务器按前缀导入并 重新分配槽位，后续 CacheToSurface 即可直接回贴、无需重传像素。

type GfxCacheStore 

type GfxCacheStore interface {
	Persist(key uint64, w, h int, bpp uint16, data []byte)
	Export() []GfxCacheEntry
	Get(key uint64) (GfxCacheEntry, bool)
	Keys() []uint64
}

GfxCacheStore 在浏览器侧持久化缓存条目（v1：页面内存，跨手动重连保留）。 Persist 随每条 SurfaceToCache 调用；Export 在每次连接 caps 确认后调用一次， 返回待上报的条目（顺序即 CacheImportReply 的前缀导入顺序）。 Get/Keys 供 bitmap 管线持久缓存（6.4b M2）按需查取与枚举键。

type GfxHandler 

type GfxHandler struct {
	// contains filtered or unexported fields
}

GfxHandler implements the RDPGFX (MS-RDPEGFX) protocol.

func NewGfxHandler 

func NewGfxHandler(onBitmap func([]BitmapUpdate)) *GfxHandler

NewGfxHandler creates a new RDPGFX handler.

func (*GfxHandler) Close 

func (g *GfxHandler) Close()

Close shuts down the GfxHandler's background goroutines. Safe to call multiple times; subsequent calls are no-ops.

h264dec is intentionally NOT freed here: decodeLoop (goroutine 21) may be in the middle of avcodec_send_packet when Close is called from the transport goroutine, which would cause a use-after-free SIGSEGV. Instead, decodeLoop defers cleanup of h264dec so it always runs after the last Decode call.

func (*GfxHandler) CodecStats 

func (g *GfxHandler) CodecStats() map[uint16]int64

CodecStats returns cumulative surface-bitmap bytes per codec id (WTS1 + WTS2 combined). Useful for bandwidth diagnostics: e.g. a session dominated by codec 3 (RemoteFX) vs 0x0B/0x0E (AVC420/444) vs 8 (ClearCodec). Keys 100+cmdId carry the per-command PDU counters (100+0..100+63).

func (*GfxHandler) DebugSurfacePixel 

func (g *GfxHandler) DebugSurfacePixel(x, y int) (uint8, uint8, uint8, uint8, bool)

DebugSurfacePixel 诊断：返回第一个 mapped surface 上 (x,y) 的 BGRA 原始值

func (*GfxHandler) DiagStats 

func (g *GfxHandler) DiagStats() (frames, qdepth, fintvMs, decUs int64)

DiagStats 返回实时诊断指标：累计解码帧数、当前解码队列深度、 帧间隔（毫秒）与单消息解码耗时 EMA（微秒）。解码耗时必须保留 微秒精度——WebCodecs 路径单消息只有几百微秒，取整到毫秒恒为 0。 统计循环定期输出，用于判断解码吞吐是否跟不上到达速率、服务器帧率是否异常。

func (*GfxHandler) NotifyLocalInput 

func (g *GfxHandler) NotifyLocalInput()

NotifyLocalInput tells the graphics pipeline that a real local input event was just sent to the server. If the decoder has already been silent longer than the HW stall threshold, arm a short watchdog so recovery no longer depends on the next H.264 packet arriving.

func (*GfxHandler) OnChannelCreated 

func (g *GfxHandler) OnChannelCreated()

OnChannelCreated is called after the DVC CREATE_RSP has been sent. It sends RESET_GRAPHICS (MS-RDPEGFX 3.2.1.3：通道初始化后、caps 协商前， 客户端先发 RESET_GRAPHICS 声明会话尺寸) 再发 CAPS_ADVERTISE 启动管线。 服务器收到后重建表面并重发完整桌面帧——首次连接无害，断线重附会话时 则强制全量重绘，消除重连后的画面残留。 实测 Win10 19041 依赖该 reset 启动图形管线（省略后服务器不发 caps confirm 直接断连）；Server 2025 对 reset 后的管线重建会崩（0x112F）， 由上层对该错误做一次性 bitmap 回退兜底。

func (*GfxHandler) Process 

func (g *GfxHandler) Process(data []byte)

Process handles a complete RDPGFX payload (may contain multiple PDUs). Data arrives wrapped in ZGFX (RDP8 Bulk Compression) segments (MS-RDPEGFX 2.2.4).

Called on the network read goroutine. Decompression happens here; the decompressed payload is then queued for asynchronous processing (including frame ACKs and decode) on the decode goroutine. This keeps the read goroutine free from any socket.Write calls that could cause TCP deadlock when both sides try to write simultaneously.

func (*GfxHandler) ReplayPDU 

func (g *GfxHandler) ReplayPDU(kind byte, codecId uint16, surfW, surfH uint16, x, y, w, h uint32, payload []byte)

ReplayPDU feeds one recorded payload through the normal decode path (offline replay harness for garbled-screen debugging).

func (*GfxHandler) ReplaySurface 

func (g *GfxHandler) ReplaySurface(id uint16, w, h uint16)

ReplaySurface ensures a replay surface with the given id/dimensions exists.

func (*GfxHandler) ReplaySurfacePixels 

func (g *GfxHandler) ReplaySurfacePixels(id uint16) []byte

ReplaySurfacePixels returns the live pixel buffer of a replay surface.

func (*GfxHandler) SetAVC444Disabled 

func (g *GfxHandler) SetAVC444Disabled(v bool)

SetAVC444Disabled controls whether AVC444/AVC444v2 is advertised to the server. When disabled, CAPS_ADVERTISE only includes v8.0 and v8.1, so the server will encode frames using AVC420 (4:2:0) only and never send LC=2 chroma-upgrade data. This avoids the colour degradation caused by servers (e.g. VirtualBox VRDE) that send LC=2 frames without including stream2 in LC=0 IDR packets. Must be called before the channel is opened.

func (*GfxHandler) SetAVCDisabled 

func (g *GfxHandler) SetAVCDisabled(v bool)

SetAVCDisabled advertises v10.x caps with RDPGFX_CAPS_FLAG_AVC_DISABLED while keeping the RDPGFX channel alive: the server then encodes with ClearCodec and RFX Progressive only ("RemoteFX mode"), never H.264. Must be called before the channel is opened.

func (*GfxHandler) SetDecoderBrokenCallback 

func (g *GfxHandler) SetDecoderBrokenCallback(fn func())

SetDecoderBrokenCallback registers a function that is called once when the H.264 decoder becomes permanently unrecoverable (all soft resets exhausted). The callback should reconnect the RDP session so a fresh decoder can be created from scratch.

func (*GfxHandler) SetH264RawCallback 

func (g *GfxHandler) SetH264RawCallback(fn func(destX, destY, w, h int, isKey bool, data []byte, regions []int32))

SetH264RawCallback registers a function that receives raw H.264 NAL unit data when the built-in decoder is unavailable (h264dec == nil). This allows the caller to hand off decoding to an external engine such as the browser WebCodecs VideoDecoder API.

destX and destY are the top-left canvas coordinates of the decoded frame. isKey is true when the NAL data starts a new GOP (IDR frame).

func (*GfxHandler) SetI420Callback 

func (g *GfxHandler) SetI420Callback(fn func(destX, destY, w, h int, y []byte, yStride int, u []byte, uStride int, v []byte, vStride int))

SetI420Callback registers a callback that receives I420 planar data when an H.264 frame is decoded and the underlying decoder supports I420 extraction. When set, H264 frames are NOT emitted via the normal OnBitmap path; the caller is responsible for rendering the I420 data directly (e.g. via an SDL2 IYUV texture). When the I420 fast path is used, the BGRA surface backing store is not updated for that frame. Set fn to nil to disable and revert to normal OnBitmap delivery.

func (*GfxHandler) SetKeyframeRequestFunc 

func (g *GfxHandler) SetKeyframeRequestFunc(fn func())

SetKeyframeRequestFunc registers a function that is called after each soft decoder reset to ask the server for a fresh IDR keyframe. This speeds up recovery: without it the decoder waits for the server to spontaneously send a keyframe. A typical implementation calls pdu.SendRefreshRect with the current screen dimensions.

func (*GfxHandler) SetNV12Callback 

func (g *GfxHandler) SetNV12Callback(fn func(destX, destY, w, h int, y []byte, yStride int, uv []byte, uvStride int))

SetNV12Callback registers a callback that receives native NV12 planar data when H.264 decoding produces NV12. When set for AVC420 frames, the normal OnBitmap path is bypassed for frames that can be delivered as NV12; callers should upload the Y and UV planes directly (for example with SDL2 SDL_UpdateNVTexture). Set fn to nil to disable.

func (*GfxHandler) SetPduRecorder 

func (g *GfxHandler) SetPduRecorder(fn func(kind byte, codecId uint16, surfW, surfH, x, y, w, h uint32, payload []byte))

SetPduRecorder installs a callback that receives every wire-to-surface bitmap payload for offline replay analysis. Pass nil to disable.

func (*GfxHandler) SetPersistentCacheStore 

func (g *GfxHandler) SetPersistentCacheStore(s GfxCacheStore)

SetPersistentCacheStore 安装持久缓存桥；必须在连接建立前调用。

func (*GfxHandler) SetQueueDepthHint 

func (g *GfxHandler) SetQueueDepthHint(depth uint32)

func (*GfxHandler) SetSendFunc 

func (g *GfxHandler) SetSendFunc(fn func([]byte))

SetSendFunc sets the function used to send RDPGFX responses via DVC.

func (*GfxHandler) SetSessionSize 

func (g *GfxHandler) SetSessionSize(width, height uint16)

SetSessionSize 注入会话尺寸，供通道初始化的 RESET_GRAPHICS 使用。

type H264BrokenReason 

type H264BrokenReason int

H264BrokenReason describes why a decoder became unrecoverable. 

const (
	H264BrokenReasonNone H264BrokenReason = iota
	H264BrokenReasonInitFailure
	H264BrokenReasonHWStall
	H264BrokenReasonNoIDR
)

func (H264BrokenReason) String 

func (r H264BrokenReason) String() string

type H264Decoder 

type H264Decoder interface {
	// Decode decodes H.264 NAL units and returns a decoded frame.
	// Returns nil frame (no error) when the decoder needs more input data.
	Decode(h264Data []byte) (*H264Frame, error)
	// NeedsKeyframe reports whether the decoder is waiting for a keyframe.
	NeedsKeyframe() bool
	// NeedsIDR reports whether the decoder is explicitly waiting for an IDR frame.
	NeedsIDR() bool
	// IsBroken reports whether the decoder is permanently unrecoverable.
	IsBroken() bool
	// BrokenReason reports why the decoder became unrecoverable.
	BrokenReason() H264BrokenReason
	// ForceBroken marks the decoder unrecoverable for the given reason.
	ForceBroken(reason H264BrokenReason)
	// HardResetCount returns the number of hard resets performed so far.
	HardResetCount() int
	// LastReceiveTime returns the wall-clock time of the most recent Decode() call.
	LastReceiveTime() time.Time
	// Close releases all resources held by the decoder.
	Close()
}

H264Decoder decodes H.264 Annex B bitstream data into BGRA frames.

type H264DecoderBackend 

type H264DecoderBackend struct {
	// NewHW creates a hardware-preferred decoder with an optional watchdog channel.
	// watchdogCh may be nil for the initial decoder (no watchdog).
	NewHW func(watchdogCh chan<- struct{}) H264Decoder
	// NewSW creates a software-only decoder without a watchdog (for aux decoders).
	NewSW func() H264Decoder
	// NewSWFallback creates a software-only decoder with a watchdog
	// (used as the post-VideoToolbox-stall fallback decoder).
	NewSWFallback func(watchdogCh chan<- struct{}) H264Decoder
}

H264DecoderBackend holds factory functions for creating H264Decoder instances. Register a backend via SetH264Backend before starting any RDP session. Typically called from an init() function in the application binary.

type H264Frame 

type H264Frame struct {
	Data          []byte // BGRA pixel data, 4 bytes per pixel (nil when Dropped)
	Width, Height int
	// Dropped is true when the decoder intentionally discarded this frame
	// (e.g. zero-filled VideoToolbox IOSurface) rather than experiencing a
	// genuine codec stall.  Callers must skip bitmap updates but must NOT
	// request a keyframe or flag the decoder as broken.
	Dropped bool
}

H264Frame holds a decoded H.264 frame in BGRA pixel format.

type H264FrameI420 

type H264FrameI420 struct {
	Y, U, V                   []byte
	YStride, UStride, VStride int
	Width, Height             int
	FullRange                 bool // true when the source used full-range (JPEG/PC) YUV
}

H264FrameI420 holds a decoded H.264 frame in planar I420 (YUV420P) format. SDL2 can render I420 natively via hardware-accelerated YUV→RGB shaders using a PIXELFORMAT_IYUV texture, eliminating CPU-side colour conversion. Plane slices borrow ring-buffer memory; the caller must copy all slices before the next Decode call.

type H264FrameNV12 

type H264FrameNV12 struct {
	Y, UV     []byte
	YStride   int
	UVStride  int
	Width     int
	Height    int
	FullRange bool
}

H264FrameNV12 holds a decoded H.264 frame in NV12 format (Y plane plus interleaved UV plane). VideoToolbox commonly transfers hardware-decoded H.264 frames as NV12; SDL2 can upload NV12 directly, avoiding the CPU-side NV12->I420 deinterleave needed by the I420 path.

type I420Decoder 

type I420Decoder interface {
	// DecodeWithI420 decodes H.264 NAL data and returns both a BGRA frame
	// and an optional I420 frame for GPU-accelerated rendering.  The I420
	// frame is nil when the pixel format is not directly convertible;
	// callers must fall back to the BGRA frame in that case.
	DecodeWithI420(h264Data []byte) (*H264Frame, *H264FrameI420, error)
}

I420Decoder is an optional interface that an H264Decoder may implement to produce I420 output alongside the normal BGRA frame. Callers detect support via a type assertion.

type NV12Decoder 

type NV12Decoder interface {
	DecodeWithNV12(h264Data []byte) (*H264Frame, *H264FrameNV12, error)
}

NV12Decoder is an optional interface for decoders that can expose native NV12 output. Callers should fall back to I420 or BGRA when the returned NV12 frame is nil.

type RegionHinter 

type RegionHinter interface {
	SetRegionHint(rects [][4]uint16)
}

RegionHinter is an optional interface implemented by decoders that support region-aware YUV→BGRA conversion. When SetRegionHint is called immediately before Decode, the decoder only converts pixels within the specified dirty rectangles, skipping unchanged areas of the frame. Each element of rects is [left, top, right, bottom].

type ScanResult 

type ScanResult struct {
	HasKeyFrame      bool
	SPSStart, SPSEnd int
	PPSStart, PPSEnd int
}

ScanResult holds the IDR-presence flag and SPS/PPS NAL boundaries (offsets into the original packet, including Annex B start code) discovered during a single linear walk of an Annex B H.264 packet.

func ScanH264Packet 

func ScanH264Packet(data []byte) ScanResult

ScanH264Packet walks an Annex B H.264 packet exactly once, returning whether it contains any IDR slice (NAL type 5) or SPS (NAL type 7) NAL unit and recording the byte ranges for the most recent SPS/PPS NALs found.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.