netpoll

package
v2.5.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 9, 2026 License: Apache-2.0 Imports: 13 Imported by: 0

Documentation

Overview

Package netpoll provides a portable event-driven interface for network I/O.

The underlying facility of event notification is OS-specific:

With the help of the netpoll package, you can easily build your own high-performance event-driven network applications based on epoll/kqueue.

The Poller represents the event notification facility whose backend is epoll or kqueue. The OpenPoller function creates a new Poller instance:

poller, err := netpoll.OpenPoller()
if err != nil {
	// handle error
}

defer poller.Close()

addr, err := net.ResolveTCPAddr("tcp", "127.0.0.1:9090")
if err != nil {
	// handle error
}
c, err := net.DialTCP("tcp", nil, addr)
if err != nil {
	// handle error
}

f, err := c.File()
if err != nil {
	// handle error
}

closeClient := func() {
	c.Close()
	f.Close()
}
defer closeClient()

The PollAttachment consists of a file descriptor and its callback function. PollAttachment is used to register a file descriptor to Poller. The callback function is called when an event occurs on the file descriptor:

pa := netpoll.PollAttachment{
	FD: int(f.Fd()),
	Callback: func(fd int, event netpoll.IOEvent, flags netpoll.IOFlags) error {
		if netpoll.IsErrorEvent(event, flags) {
			closeClient()
			return errors.ErrEngineShutdown
		}

		if netpoll.IsReadEvent(event) {
			buf := make([]byte, 64)
			// Read data from the connection.
			_, err := c.Read(buf)
			if err != nil {
				closeClient()
				return errors.ErrEngineShutdown
			}
			// Process the data...
		}

		if netpoll.IsWriteEvent(event) {
			// Write data to the connection.
			_, err := c.Write([]byte("hello"))
			if err != nil {
				closeClient()
				return errors.ErrEngineShutdown
			}
		}

		return nil
	}}

if err := poller.AddReadWrite(&pa, false); err != nil {
	// handle error
}

The Poller.Polling function starts the event loop monitoring file descriptors and waiting for I/O events to occur:

poller.Polling(func(fd int, event netpoll.IOEvent, flags netpoll.IOFlags) error {
	return pa.Callback(fd, event, flags)
})

Or

poller.Polling()

if you've enabled the build tag `poll_opt`.

Example
ln, err := net.Listen("tcp", "127.0.0.1:9090")
if err != nil {
	panic(fmt.Sprintf("Error listening: %v", err))
}

defer ln.Close() //nolint:errcheck

ctx, cancel := signal.NotifyContext(context.Background(), os.Interrupt, os.Kill)
defer cancel()

go func() {
	c, err := ln.Accept()
	if err != nil {
		panic(fmt.Sprintf("Error accepting connection: %v", err))
	}

	defer c.Close() //nolint:errcheck

	buf := make([]byte, 64)

	for {
		select {
		case <-ctx.Done():
			cancel()
			fmt.Printf("Signal received: %v\n", ctx.Err())
			return
		default:
		}

		_, err := c.Read(buf)
		if err != nil {
			panic(fmt.Sprintf("Error reading data from client: %v", err))
		}
		fmt.Printf("Received data from client: %s\n", buf)

		_, err = c.Write([]byte("Hello, client!"))
		if err != nil {
			panic(fmt.Sprintf("Error writing data to client: %v", err))
		}
		fmt.Println("Sent data to client")

		time.Sleep(200 * time.Millisecond)
	}
}()

// Wait for the server to start running.
time.Sleep(500 * time.Millisecond)

poller, err := netpoll.OpenPoller()
if err != nil {
	panic(fmt.Sprintf("Error opening poller: %v", err))
}

defer poller.Close() //nolint:errcheck

addr, err := net.ResolveTCPAddr("tcp", ln.Addr().String())
if err != nil {
	panic(fmt.Sprintf("Error resolving TCP address: %v", err))
}
c, err := net.DialTCP("tcp", nil, addr)
if err != nil {
	panic(fmt.Sprintf("Error dialing TCP address: %v", err))
}

f, err := c.File()
if err != nil {
	panic(fmt.Sprintf("Error getting file from connection: %v", err))
}

closeClient := func() {
	c.Close() //nolint:errcheck
	f.Close() //nolint:errcheck
}
defer closeClient()

sendData := true

pa := netpoll.PollAttachment{
	FD: int(f.Fd()),
	Callback: func(fd int, event netpoll.IOEvent, flags netpoll.IOFlags) error { //nolint:revive
		if netpoll.IsErrorEvent(event, flags) {
			closeClient()
			return errors.ErrEngineShutdown
		}

		if netpoll.IsReadEvent(event) {
			sendData = true
			buf := make([]byte, 64)
			_, err := c.Read(buf)
			if err != nil {
				closeClient()
				fmt.Println("Error reading data from server:", err)
				return errors.ErrEngineShutdown
			}
			fmt.Printf("Received data from server: %s\n", buf)
			// Process the data...
		}

		if netpoll.IsWriteEvent(event) && sendData {
			sendData = false
			// Write data to the connection...
			_, err := c.Write([]byte("Hello, server!"))
			if err != nil {
				closeClient()
				fmt.Println("Error writing data to server:", err)
				return errors.ErrEngineShutdown
			}
			fmt.Println("Sent data to server")
		}

		return nil
	},
}

if err := poller.AddReadWrite(&pa, false); err != nil {
	panic(fmt.Sprintf("Error adding file descriptor to poller: %v", err))
}

err = examplePoll(poller, &pa)

fmt.Printf("Poller exited with error: %v", err)

Index

Examples

Constants

View Source
const (
	// InitPollEventsCap represents the initial capacity of poller event-list.
	InitPollEventsCap = 128
	// MaxPollEventsCap is the maximum limitation of events that the poller can process.
	MaxPollEventsCap = 1024
	// MinPollEventsCap is the minimum limitation of events that the poller can process.
	MinPollEventsCap = 32
	// MaxAsyncTasksAtOneTime is the maximum amount of asynchronous tasks that the event-loop will process at one time.
	MaxAsyncTasksAtOneTime = 256
	// ReadEvents represents readable events that are polled by epoll.
	//
	// FIX L-12: EPOLLPRI is a priority (out-of-band) event. Nothing here enables
	// SO_OOBINLINE or SO_SELECT_ERR_QUEUE, so the kernel does not put TCP urgent
	// data on the error queue these sockets read from and no EPOLLPRI is ever
	// generated for them; the bit is inert today. It is kept because it is what
	// upstream requests, but the coupling is implicit: enabling either socket
	// option later would start surfacing EPOLLPRI, and because IsReadEvent tests
	// ReadEvents, a level-triggered EPOLLPRI that nothing consumes would re-arm
	// every iteration — a read event that read(2) cannot clear. Remove the bit at
	// the same time as adding such an option, not after.
	ReadEvents = unix.EPOLLIN | unix.EPOLLPRI
	// WriteEvents represents writeable events that are polled by epoll.
	WriteEvents = unix.EPOLLOUT
	// ReadWriteEvents represents both readable and writeable events.
	ReadWriteEvents = ReadEvents | WriteEvents
	// ErrEvents represents exceptional events that occurred.
	//
	// FIX L-12: EPOLLRDHUP is deliberately absent. It is requested in edge-triggered
	// mode (see AddRead/ModReadWrite), and on a peer half-close the kernel reports it
	// together with EPOLLIN — because tcp_poll sets both when the receive side is
	// shut down. Adding it here would therefore classify an ordinary orderly
	// shutdown as an error event and tear the connection down before the data the
	// peer sent before closing was read. The read path already turns that same
	// condition into io.EOF when read(2) returns 0, which is where the close belongs.
	ErrEvents = unix.EPOLLERR | unix.EPOLLHUP
)

Variables

This section is empty.

Functions

func IsErrorEvent

func IsErrorEvent(event IOEvent, _ IOFlags) bool

IsErrorEvent checks if the event is an error event.

func IsReadEvent

func IsReadEvent(event IOEvent) bool

IsReadEvent checks if the event is a read event.

func IsWriteEvent

func IsWriteEvent(event IOEvent) bool

IsWriteEvent checks if the event is a write event.

Types

type IOEvent

type IOEvent = uint32

IOEvent is the integer type of I/O events on Linux.

type IOFlags

type IOFlags = uint16

IOFlags represents the flags of IO events.

type PollAttachment

type PollAttachment struct {
	FD       int
	Callback PollEventHandler
}

PollAttachment is the user data which is about to be stored in "void *ptr" of epoll_data or "void *udata" of kevent.

Lifetime contract, which matters under the poll_opt build tags: there the kernel is handed a raw pointer to the PollAttachment, and the Go garbage collector cannot see it — it scans Go memory, not the event registration held inside the kernel. The pointer is taken with unsafe.Pointer, which does not keep the object alive on its own. The caller must therefore

  • keep a reachable reference to the PollAttachment for as long as its file descriptor is registered, and
  • not copy or replace it in the meantime, since the poller will restore whatever address it was given when the event arrives.

Dropping the last reference while the descriptor is still registered leaves the kernel pointing at freed memory, and the next event for that descriptor makes the poller call a callback out of a dangling pointer. Delete the descriptor from the poller before releasing the attachment. The default (non-poll_opt) build carries the file descriptor number instead of a pointer and has no such requirement.

type PollEventHandler

type PollEventHandler func(int, IOEvent, IOFlags) error

PollEventHandler is the callback for I/O events notified by the poller.

type Poller

type Poller struct {
	// contains filtered or unexported fields
}

Poller represents a poller which is in charge of monitoring file-descriptors.

func OpenPoller

func OpenPoller() (poller *Poller, err error)

OpenPoller instantiates a poller.

func (*Poller) AddRead

func (p *Poller) AddRead(pa *PollAttachment, edgeTriggered bool) error

AddRead registers the given file descriptor with readable event to the poller.

func (*Poller) AddReadWrite

func (p *Poller) AddReadWrite(pa *PollAttachment, edgeTriggered bool) error

AddReadWrite registers the given file descriptor with readable and writable events to the poller.

func (*Poller) AddWrite

func (p *Poller) AddWrite(pa *PollAttachment, edgeTriggered bool) error

AddWrite registers the given file descriptor with writable event to the poller.

func (*Poller) Close

func (p *Poller) Close() error

Close closes the poller.

Close is idempotent: it marks the poller as closed by setting both descriptors to -1, so closing twice is a no-op instead of a second close(2) on a descriptor number the runtime may have handed out again in the meantime.

func (*Poller) Delete

func (p *Poller) Delete(fd int) error

Delete removes the given file descriptor from the poller.

func (*Poller) ModRead

func (p *Poller) ModRead(pa *PollAttachment, edgeTriggered bool) error

ModRead modifies the given file descriptor with readable event in the poller.

func (*Poller) ModReadWrite

func (p *Poller) ModReadWrite(pa *PollAttachment, edgeTriggered bool) error

ModReadWrite modifies the given file descriptor with readable and writable events in the poller.

func (*Poller) Polling

func (p *Poller) Polling(callback PollEventHandler) error

Polling blocks the current goroutine, monitoring the registered file descriptors.

func (*Poller) SetAdaptiveYield

func (y *Poller) SetAdaptiveYield(adaptive bool)

SetAdaptiveYield makes an enabled yield adaptive: the poller yields only for yieldWindow batches after each wake-up through its task queues and goes straight to the blocking wait otherwise. It must be called before Polling starts, and has no effect while the yield is disabled by SetYield. Pollers other than the epoll ones ignore it and keep yielding unconditionally.

Skipping the yield does not affect correctness, only when goroutines that the callbacks made runnable start: they stay in the run queue of this P and are picked up by an idle P (which the runtime wakes for them) or, when every P is busy, once the loop blocks in the kernel and the runtime hands its P over, which can take up to a scheduler monitor tick. That is why a process with a single P never skips the yield: there is no other P that could take the goroutines, and the yield itself is cheap there because no thread has to be woken.

func (*Poller) SetYield

func (y *Poller) SetYield(enabled bool)

SetYield switches the yield-before-blocking behaviour described on pollYield on or off. It must be called before Polling starts; it is on by default.

func (*Poller) Trigger

func (p *Poller) Trigger(priority queue.EventPriority, fn queue.Func, param any) (err error)

Trigger enqueues task and wakes up the poller to process pending tasks.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL