Documentation
¶
Overview ¶
Package devicelease guards physical accelerator residency across modeld data roots. The normal modeld lease stays per-data-root for service discovery; this wrapper adds a second, fixed-location lease keyed by the backend's resolved accelerator device so two daemons cannot make models resident on the same GPU.
Index ¶
- Constants
- type Option
- type Service
- func (s *Service) Describe(ctx context.Context, req transport.OpenSessionRequest) (transport.ModelInfo, error)
- func (s *Service) Embed(ctx context.Context, req transport.EmbedRequest) (transport.EmbedResult, error)
- func (s *Service) OpenSession(ctx context.Context, req transport.OpenSessionRequest) (transport.Session, error)
Constants ¶
View Source
const ( // EnvLeaseDir overrides the directory used for fixed-location device leases. // It exists for tests and for operators with non-standard cache locations. EnvLeaseDir = "CONTENOX_MODELD_DEVICE_LEASE_DIR" )
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Option ¶
type Option func(*Service)
func WithLeaseDir ¶
WithLeaseDir sets the directory where device lease files are stored.
type Service ¶
type Service struct {
// contains filtered or unexported fields
}
Service wraps a transport.Service with accelerator-device ownership.
func (*Service) Embed ¶
func (s *Service) Embed(ctx context.Context, req transport.EmbedRequest) (transport.EmbedResult, error)
func (*Service) OpenSession ¶
Source Files
Click to show internal directories.
Click to hide internal directories.