multimodal

package
v0.12.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jun 23, 2026 License: MIT Imports: 19 Imported by: 0

Documentation

Overview

SIVA — Split-Image Visual Adversarial attack module (arXiv 2602.08136).

SIVA splits a source image into N fragments and submits them in a single multi-image request alongside an operator-supplied harmful instruction. The attack exploits the model's tendency to reason across fragmented visual context as a single coherent scene, defeating safety filters that only inspect images in isolation.

Scope (v0.9.0):

  • The module's contribution is the *protocol* — image fragmentation via image.SubImage + multi-image submission via ImageProvider.QueryWithImages
  • an image-blind refusal post-check.
  • Text-overlay rendering (writing adversarial text onto each fragment) is out of scope because it requires pulling in a font library that the rest of the codebase avoids. Operators wanting pre-overlaid fragments should render externally and pass a directory of pre-built fragments via Metadata["overlay_dir"]; v0.10.0 may revisit inline rendering.

Per the v0.9.0 plan SIVA is NOT safety-gated — the module never sends an image without an operator-supplied prompt, and refusal is the expected majority outcome on production-tier vision models. Operators are still expected to confine testing to authorized targets.

VSH — Virtual Scenario Hypnosis attack module.

VSH wraps a harmful objective in a narrative-fiction scenario and pairs it with a single contextually-plausible scene image, exploiting the model's tendency to relax refusal logic inside a clearly-marked fictional frame. Source: ScienceDirect S0031320325010520 ("Virtual Scenario Hypnosis: Multimodal Jailbreak via Narrative Engineering").

VSH and SIVA share infrastructure:

  • common.ImageProvider capability gate.
  • imageBlindIndicators / matchesAny post-check helper (defined in siva.go).
  • loadImage (defined in siva.go).

VSH differs from SIVA in payload shape: VSH submits ONE image (a benign scene) plus a narratively-framed prompt; SIVA submits N image fragments of a single source plus a direct instruction.

Per the v0.9.0 plan VSH is NOT safety-gated for the same reasons as SIVA: the attack only runs when an operator supplies the objective.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type AdversarialGenerator

type AdversarialGenerator struct {
	// contains filtered or unexported fields
}

AdversarialGenerator creates adversarial examples

func NewAdversarialGenerator

func NewAdversarialGenerator() *AdversarialGenerator

NewAdversarialGenerator creates adversarial generator

func (*AdversarialGenerator) GenerateModelSpecific

func (ag *AdversarialGenerator) GenerateModelSpecific(img image.Image, request AttackRequest) []byte

GenerateModelSpecific creates model-specific adversarial examples

func (*AdversarialGenerator) GeneratePerturbation

func (ag *AdversarialGenerator) GeneratePerturbation(img image.Image) []byte

GeneratePerturbation creates general adversarial perturbation

type AttackRequest

type AttackRequest struct {
	Type       AttackType
	Modalities []Modality
	Target     interface{}
	Objective  string
	Parameters map[string]interface{}
}

AttackRequest defines attack parameters

type AttackResponse

type AttackResponse struct {
	AttackID string
	Success  bool
	Results  []AttackResult
	Payload  interface{}
}

AttackResponse contains attack results

type AttackResult

type AttackResult struct {
	Modality  Modality
	Success   bool
	Response  string
	Extracted interface{}
	Timestamp time.Time
}

AttackResult contains attack outcome

type AttackStatus

type AttackStatus string

AttackStatus tracks attack state

const (
	StatusPreparing AttackStatus = "preparing"
	StatusExecuting AttackStatus = "executing"
	StatusSuccess   AttackStatus = "success"
	StatusFailed    AttackStatus = "failed"
)

type AttackType

type AttackType string

AttackType categorizes multi-modal attacks

const (
	AttackImageInjection    AttackType = "image_injection"
	AttackAudioManipulation AttackType = "audio_manipulation"
	AttackDocumentExploit   AttackType = "document_exploit"
	AttackVideoPayload      AttackType = "video_payload"
	AttackHybridAttack      AttackType = "hybrid_attack"
	AttackSteganographic    AttackType = "steganographic"
)

type AudioAttacker

type AudioAttacker struct {
	// contains filtered or unexported fields
}

AudioAttacker performs audio-based attacks

func NewAudioAttacker

func NewAudioAttacker(config MultiModalConfig) *AudioAttacker

NewAudioAttacker creates audio attacker

func (*AudioAttacker) GeneratePayload

func (aa *AudioAttacker) GeneratePayload(request AttackRequest) (*AudioPayload, error)

GeneratePayload creates audio attack payload

type AudioEncoder

type AudioEncoder struct{}

func NewAudioEncoder

func NewAudioEncoder() *AudioEncoder

func (*AudioEncoder) GenerateUltrasonic

func (a *AudioEncoder) GenerateUltrasonic(payload string) []byte

type AudioInjector

type AudioInjector struct{}

func NewAudioInjector

func NewAudioInjector() *AudioInjector

type AudioManipulator

type AudioManipulator struct{}

func NewAudioManipulator

func NewAudioManipulator() *AudioManipulator

type AudioPayload

type AudioPayload struct {
	Audio          []byte
	SampleRate     int
	Channels       int
	Duration       time.Duration
	EmbeddedPrompt string
	Subliminal     []byte
}

AudioPayload represents audio attack payload

type ComplexityLevel

type ComplexityLevel int

ComplexityLevel defines attack complexity

const (
	ComplexityLow ComplexityLevel = iota
	ComplexityMedium
	ComplexityHigh
	ComplexityExtreme
)

type CrossModalTrigger

type CrossModalTrigger struct {
	Source    Modality
	Target    Modality
	Condition string
	Action    string
}

CrossModalTrigger triggers across modalities

type DocumentAttacker

type DocumentAttacker struct {
	// contains filtered or unexported fields
}

DocumentAttacker performs document-based attacks

func NewDocumentAttacker

func NewDocumentAttacker(config MultiModalConfig) *DocumentAttacker

NewDocumentAttacker creates document attacker

func (*DocumentAttacker) GeneratePayload

func (da *DocumentAttacker) GeneratePayload(request AttackRequest) (*DocumentPayload, error)

GeneratePayload creates document attack payload

type DocumentFormat

type DocumentFormat string

DocumentFormat represents document type

const (
	FormatPDF  DocumentFormat = "pdf"
	FormatDOCX DocumentFormat = "docx"
	FormatXLSX DocumentFormat = "xlsx"
	FormatHTML DocumentFormat = "html"
	FormatXML  DocumentFormat = "xml"
	FormatJSON DocumentFormat = "json"
)

type DocumentPayload

type DocumentPayload struct {
	Format         DocumentFormat
	Content        []byte
	EmbeddedFiles  []EmbeddedFile
	Macros         []Macro
	ExploitVectors []ExploitVector
}

DocumentPayload represents document attack payload

type EmbeddedFile

type EmbeddedFile struct{}

type ExploitVector

type ExploitVector struct{}

type Font

type Font struct {
	Name       string
	Size       int
	Color      color.Color
	Background color.Color
}

Font represents a font for text rendering

type FormatExploiter

type FormatExploiter struct{}

func NewFormatExploiter

func NewFormatExploiter() *FormatExploiter

type FrameManipulator

type FrameManipulator struct{}

func NewFrameManipulator

func NewFrameManipulator() *FrameManipulator

type HiddenChannel

type HiddenChannel struct{}

type HybridPayload

type HybridPayload struct {
	ID         string
	Components map[Modality]interface{}
	Sequence   []ModalitySequence
	Triggers   []CrossModalTrigger
}

HybridPayload represents combined multi-modal payload

type HybridPayloadGenerator

type HybridPayloadGenerator struct {
	// contains filtered or unexported fields
}

HybridPayloadGenerator creates multi-modal hybrid attacks

func NewHybridPayloadGenerator

func NewHybridPayloadGenerator(config MultiModalConfig) *HybridPayloadGenerator

NewHybridPayloadGenerator creates hybrid generator

func (*HybridPayloadGenerator) GeneratePayload

func (hpg *HybridPayloadGenerator) GeneratePayload(request AttackRequest) (*HybridPayload, error)

GeneratePayload creates hybrid attack payload

type ImageAttacker

type ImageAttacker struct {
	// contains filtered or unexported fields
}

ImageAttacker performs image-based attacks

func NewImageAttacker

func NewImageAttacker(config MultiModalConfig) *ImageAttacker

NewImageAttacker creates an image attacker

func (*ImageAttacker) GeneratePayload

func (ia *ImageAttacker) GeneratePayload(request AttackRequest) (*ImagePayload, error)

GeneratePayload creates image-based attack payload

type ImagePayload

type ImagePayload struct {
	Image           image.Image
	EncodedPayload  string
	Metadata        map[string]string
	HiddenText      string
	AdversarialData []byte
}

ImagePayload represents an image attack payload

type ImagePayloadEncoder

type ImagePayloadEncoder struct{}

Placeholder types

func NewImagePayloadEncoder

func NewImagePayloadEncoder() *ImagePayloadEncoder

Placeholder constructors

func (*ImagePayloadEncoder) EncodeInImage

func (i *ImagePayloadEncoder) EncodeInImage(img image.Image, payload string) string

Method implementations for placeholder types

type Macro

type Macro struct{}

type MacroGenerator

type MacroGenerator struct{}

func NewMacroGenerator

func NewMacroGenerator() *MacroGenerator

type MetadataEncoder

type MetadataEncoder struct{}

func NewMetadataEncoder

func NewMetadataEncoder() *MetadataEncoder

type Modality

type Modality string

Modality represents input type

const (
	ModalityText     Modality = "text"
	ModalityImage    Modality = "image"
	ModalityAudio    Modality = "audio"
	ModalityVideo    Modality = "video"
	ModalityDocument Modality = "document"
)

type ModalitySequence

type ModalitySequence struct {
	Modality Modality
	Delay    time.Duration
	Payload  interface{}
}

ModalitySequence defines execution order

type ModelProfile

type ModelProfile struct {
	Name            string
	Architecture    string
	InputSize       image.Point
	Vulnerabilities []string
}

ModelProfile contains model-specific information

type MultiModalAttack

type MultiModalAttack struct {
	ID         string
	Type       AttackType
	Modalities []Modality
	Payload    interface{}
	Status     AttackStatus
	StartTime  time.Time
	Results    []AttackResult
	// contains filtered or unexported fields
}

MultiModalAttack represents an active multi-modal attack

type MultiModalAttacker

type MultiModalAttacker struct {
	// contains filtered or unexported fields
}

MultiModalAttacker performs attacks using multiple input modalities

func NewMultiModalAttacker

func NewMultiModalAttacker(config MultiModalConfig) *MultiModalAttacker

NewMultiModalAttacker creates a multi-modal attacker

func (*MultiModalAttacker) ExecuteAttack

func (mma *MultiModalAttacker) ExecuteAttack(ctx context.Context, request AttackRequest) (*AttackResponse, error)

ExecuteAttack performs a multi-modal attack

type MultiModalConfig

type MultiModalConfig struct {
	MaxImageSize        int64
	MaxAudioDuration    time.Duration
	MaxDocumentSize     int64
	EnableSteganography bool
	EnableAdvancedOCR   bool
	PayloadComplexity   ComplexityLevel
}

MultiModalConfig configures multi-modal attacks

type OCRManipulator

type OCRManipulator struct {
	// contains filtered or unexported fields
}

OCRManipulator manipulates OCR text in images

func NewOCRManipulator

func NewOCRManipulator() *OCRManipulator

NewOCRManipulator creates OCR manipulator

func (*OCRManipulator) EmbedInImage

func (om *OCRManipulator) EmbedInImage(img image.Image, text string)

EmbedInImage embeds hidden text in image

func (*OCRManipulator) GenerateHiddenText

func (om *OCRManipulator) GenerateHiddenText(objective string) string

GenerateHiddenText creates OCR-exploitable text

type PayloadCombinator

type PayloadCombinator interface {
	Combine(payloads map[Modality]interface{}) (interface{}, error)
}

PayloadCombinator combines multiple modalities

type PayloadEmbedder

type PayloadEmbedder struct{}

func NewPayloadEmbedder

func NewPayloadEmbedder() *PayloadEmbedder

type PerturbationMethod

type PerturbationMethod interface {
	Generate(img image.Image, epsilon float64) []byte
}

PerturbationMethod generates adversarial perturbations

type SIVAModule added in v0.9.0

type SIVAModule struct{}

SIVAModule implements attacks.AttackModule for SIVA.

func (*SIVAModule) Category added in v0.9.0

func (m *SIVAModule) Category() common.AttackCategory

Category returns CategoryMultimodal.

func (*SIVAModule) Description added in v0.9.0

func (m *SIVAModule) Description() string

Description summarizes the technique.

func (*SIVAModule) Execute added in v0.9.0

func (m *SIVAModule) Execute(
	ctx context.Context,
	provider common.Provider,
	config common.AttackConfig,
) (*common.AttackResult, error)

Execute runs the SIVA attack.

Phases:

  1. Capability gate — provider must implement common.ImageProvider.
  2. Resolve source image (Metadata["source_path"], default "src/attacks/multimodal/testdata/siva_source.png" for tests).
  3. Split into N fragments via image.SubImage; encode each as PNG.
  4. Submit multi-image request with operator-supplied instruction.
  5. Post-check for image-blind refusal indicators → SkipModelRefusedImage.
  6. Score response against config.SuccessIndicators.

func (*SIVAModule) Name added in v0.9.0

func (m *SIVAModule) Name() string

Name returns the registered technique name.

func (*SIVAModule) Techniques added in v0.9.0

func (m *SIVAModule) Techniques() []common.TechniqueInfo

Techniques returns the OWASP and metadata bundle.

type TextObfuscator

type TextObfuscator interface {
	Obfuscate(text string) string
}

TextObfuscator obfuscates text

type VSHModule added in v0.9.0

type VSHModule struct{}

VSHModule implements attacks.AttackModule for VSH.

func (*VSHModule) Category added in v0.9.0

func (m *VSHModule) Category() common.AttackCategory

Category returns CategoryMultimodal (shared with SIVA).

func (*VSHModule) Description added in v0.9.0

func (m *VSHModule) Description() string

Description summarizes the technique.

func (*VSHModule) Execute added in v0.9.0

func (m *VSHModule) Execute(
	ctx context.Context,
	provider common.Provider,
	config common.AttackConfig,
) (*common.AttackResult, error)

Execute runs the VSH attack.

Phases:

  1. Capability gate — provider must implement common.ImageProvider.
  2. Resolve scenario template + objective.
  3. Load scene image.
  4. Submit single-image request.
  5. Image-blind post-check → SkipModelRefusedImage.
  6. Score response against config.SuccessIndicators.

func (*VSHModule) Name added in v0.9.0

func (m *VSHModule) Name() string

Name returns the registered technique name.

func (*VSHModule) Techniques added in v0.9.0

func (m *VSHModule) Techniques() []common.TechniqueInfo

Techniques returns the OWASP and metadata bundle.

type VideoAttacker

type VideoAttacker struct {
	// contains filtered or unexported fields
}

VideoAttacker performs video-based attacks

func NewVideoAttacker

func NewVideoAttacker(config MultiModalConfig) *VideoAttacker

NewVideoAttacker creates video attacker

func (*VideoAttacker) GeneratePayload

func (va *VideoAttacker) GeneratePayload(request AttackRequest) (*VideoPayload, error)

GeneratePayload creates video attack payload

type VideoFrame

type VideoFrame struct {
	Index     int
	Image     image.Image
	Timestamp time.Duration
	Payload   []byte
}

VideoFrame represents a video frame

type VideoPayload

type VideoPayload struct {
	Frames         []VideoFrame
	Audio          []byte
	Duration       time.Duration
	FrameRate      int
	Resolution     image.Point
	HiddenChannels []HiddenChannel
}

VideoPayload represents video attack payload

type VoiceSynthesizer

type VoiceSynthesizer struct{}

func NewVoiceSynthesizer

func NewVoiceSynthesizer() *VoiceSynthesizer

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL