Documentation
¶
Overview ¶
Package grpc is the gRPC wire transport for the runtime/transport.Service contract. It lives in the runtime (which owns the contract) and is imported by modeld to serve it, so the dependency arrow stays one-way: modeld -> runtime. Both the server (serves any transport.Service) and the client (dials the lease leader and implements transport.Service/Session) live here.
The bindings are hand-written against a registered JSON codec rather than generated from a .proto, so the transport is live without a protoc toolchain. Payloads are plain Go structs (the transport.* types are JSON-friendly).
Index ¶
- func Serve(ctx context.Context, lis net.Listener, svc transport.Service, ...) error
- type Client
- func (c *Client) Close() error
- func (c *Client) Describe(ctx context.Context, req transport.OpenSessionRequest) (transport.ModelInfo, error)
- func (c *Client) Health(ctx context.Context) (HealthStatus, error)
- func (c *Client) OpenSession(ctx context.Context, req transport.OpenSessionRequest) (transport.Session, error)
- type HealthStatus
- type Server
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
Types ¶
type Client ¶
type Client struct {
// contains filtered or unexported fields
}
Client dials a modeld owner endpoint and implements transport.Service over the wire. owner is the lease instance id it expects to be serving; it is fenced on every call so a client never acts on a stale owner after a takeover.
func DialLeader ¶
func DialLeader(endpoint, expectedOwner string, opts ...grpclib.DialOption) (*Client, error)
DialLeader connects to the owner's advertised endpoint and fences every call with the expected owner instance id resolved from the lease record.
func (*Client) Describe ¶
func (c *Client) Describe(ctx context.Context, req transport.OpenSessionRequest) (transport.ModelInfo, error)
Describe reports a model's capabilities as read by the daemon from the model metadata. The model is identified by req's typed handle (Type + Path); Config is ignored.
func (*Client) Health ¶
func (c *Client) Health(ctx context.Context) (HealthStatus, error)
Health pings the owner for liveness. It is not fenced — the caller uses the returned InstanceID to confirm the lease holder is the process answering.
func (*Client) OpenSession ¶
type HealthStatus ¶
type HealthStatus struct {
InstanceID string // the owner instance actually serving this endpoint
Ready bool
Backend string // inference backend the owner serves ("llama"/"openvino"/"none")
}
HealthStatus is the result of the unfenced liveness probe.
type Server ¶
type Server struct {
// contains filtered or unexported fields
}
Server adapts a transport.Service to the gRPC wire. It is generic: modeld supplies the concrete (CGO-backed) Service and the owner instance id from the lease. Sessions live here keyed by an opaque handle that encodes the owner instance, so a handle minted by a previous owner is rejected after takeover.
func NewServer ¶
NewServer wraps a transport.Service. instanceID is the owner's lease instance id used for fencing; pass "" to disable fencing (the unwired/local path). backend is the served inference backend ("llama"/"openvino"/"none"/"") echoed on the health probe so the runtime learns the daemon's mode over the wire.
func (*Server) Register ¶
func (s *Server) Register(reg grpclib.ServiceRegistrar)
Register mounts the service on a gRPC server.
Source Files