Documentation
¶
Index ¶
- Variables
- func HighestFlaggedCategory(result *Result) moderationpb.FlaggedCategory
- func NewIntegration(attestor *commonpb.PublicKey) ocp_integration.Moderation
- func ValidateAttestation(attestation *moderationpb.ModerationAttestation, attestor *commonpb.PublicKey, ...) error
- type Client
- type Integration
- type Result
- type Server
Constants ¶
This section is empty.
Variables ¶
View Source
var ( ErrInvalidAttestation = errors.New("invalid attestation") ErrInvalidSignature = errors.New("invalid attestation signature") ErrInvalidAttestor = errors.New("attestor public key mismatch") ErrInvalidContentHash = errors.New("content hash mismatch") ErrExpiredAttestation = errors.New("attestation is expired") )
View Source
var ErrUnsupportedLanguage = errors.New("unsupported language")
ErrUnsupportedLanguage is returned when the moderation service does not support the language of the provided text.
Functions ¶
func HighestFlaggedCategory ¶ added in v1.19.0
func HighestFlaggedCategory(result *Result) moderationpb.FlaggedCategory
HighestFlaggedCategory maps a moderation Result's flagged categories onto the proto FlaggedCategory vocabulary, returning the highest-scoring one that maps to a well-defined category (falling back to OTHER). It is the shared mapping used both by this service's responses and by callers that surface a moderation verdict elsewhere (e.g. a blob's rejection metadata).
func NewIntegration ¶
func NewIntegration(attestor *commonpb.PublicKey) ocp_integration.Moderation
func ValidateAttestation ¶
func ValidateAttestation(attestation *moderationpb.ModerationAttestation, attestor *commonpb.PublicKey, content any) error
ValidateAttestation validates a moderation attestation against the expected attestor public key and content. Content must be a string (for text) or []byte (for images).
Types ¶
type Client ¶
type Client interface {
// ClassifyText classifies the provided text for moderation. The result
// indicates whether the text was flagged and includes per-category scores.
// This is the generic classification that should be applied to all text.
// For usecase-specific moderation, an additional call must be made.
ClassifyText(ctx context.Context, text string) (*Result, error)
// ClassifyImage classifies the provided image data for moderation. The
// result indicates whether the image was flagged and includes per-category
// scores.
ClassifyImage(ctx context.Context, data []byte) (*Result, error)
// ClassifyCurrencyName checks whether a currency name infringes on
// existing trademarks, impersonates known brands, or is otherwise
// misleading. The result includes per-category scores for areas like
// cryptocurrency, financial_service, impersonation, etc.
ClassifyCurrencyName(ctx context.Context, name string) (*Result, error)
// ClassifyUsername checks a user-chosen username on two independent axes:
// who the handle claims to be, and what it says.
//
// A handle is unique and globally addressable, so holding one is a claim to
// be its subject. It is scored for squatting a brand, token, exchange,
// financial service, government body, public figure, or the Flipcash
// platform itself, and for reading as an official support or staff role
// rather than as a person. This is the reverse of ClassifyDisplayName,
// which deliberately allows any name: a display name is not unique, so it
// claims nothing.
//
// A handle is also text other users see when they pay, message, or search
// for its holder, so it is scored for harmful content under the same
// categories ClassifyDisplayName uses. Overlapping ClassifyText there is
// deliberate: a handle is at most 15 characters with no whitespace, rarely
// enough for the general classifier to identify a language, so it often
// declines with ErrUnsupportedLanguage and leaves this the only classifier
// that saw the handle.
//
// The result includes per-category scores for areas like impersonation,
// official_role, hate, sexual, etc.
ClassifyUsername(ctx context.Context, username string) (*Result, error)
// ClassifyDisplayName checks whether a user-chosen display name abuses the
// name field to advertise, solicit, or expose others to harmful content.
// The result includes per-category scores for areas like solicitation,
// contact_info, hate, etc.
//
// It deliberately does not score impersonation: users are free to call
// themselves whatever they like, including the name of a real person or
// brand. It is also distinct from ClassifyText, which is tuned for prose
// and has little to work with in a one- or two-word name.
ClassifyDisplayName(ctx context.Context, name string) (*Result, error)
}
type Integration ¶
type Integration struct {
// contains filtered or unexported fields
}
func (*Integration) ValidateAttestation ¶
func (i *Integration) ValidateAttestation(_ context.Context, _ *ocp_common.Account, rawAttestation []byte, content any) (bool, error)
type Server ¶
type Server struct {
moderationpb.UnimplementedModerationServer
// contains filtered or unexported fields
}
func (*Server) ModerateImage ¶
func (s *Server) ModerateImage(ctx context.Context, req *moderationpb.ModerateImageRequest) (*moderationpb.ModerateImageResponse, error)
func (*Server) ModerateText ¶
func (s *Server) ModerateText(ctx context.Context, req *moderationpb.ModerateTextRequest) (*moderationpb.ModerateTextResponse, error)
Click to show internal directories.
Click to hide internal directories.