rimportv2

package
v0.0.0-...-36d6306 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jul 5, 2026 License: Apache-2.0 Imports: 50 Imported by: 0

Documentation

Overview

Package rimportv2 provides a modern unified import service with TypeSpec compliance. It implements automatic format detection and supports multiple import formats.

Package rimportv2 provides a modern HAR import service with TypeSpec compliance. It implements a simple, maintainable architecture with dependency injection for core services, functional options pattern for configuration, and comprehensive error handling for local development tool workflows.

Package rimportv2 provides a modern unified import service with TypeSpec compliance. It implements automatic format detection and supports multiple import formats.

Index

Constants

View Source
const ReasonValidJSON = "Valid JSON; "

Variables

View Source
var (
	ErrInvalidHARFormat  = errors.New("invalid HAR format")
	ErrPermissionDenied  = errors.New("permission denied")
	ErrStorageFailed     = errors.New("storage operation failed")
	ErrWorkspaceNotFound = errors.New("workspace not found")
	ErrFormatDetection   = errors.New("format detection failed")
	ErrUnsupportedFormat = errors.New("unsupported format")
	ErrInvalidData       = errors.New("invalid data provided")
	ErrTranslationFailed = errors.New("translation failed")
	ErrValidationFailed  = errors.New("validation failed")
	ErrEmptyData         = errors.New("empty data provided")
	ErrDataTooLarge      = errors.New("data exceeds size limit")
	ErrTimeout           = errors.New("operation timed out")
)

Common errors for the rimportv2 service

View Source
var ErrCycleDetected = fmt.Errorf("cycle detected in dependency graph")

ErrCycleDetected is returned when a cycle is detected in the dependency graph.

View Source
var ErrPrivateIP = errors.New("URL resolves to a private or internal IP address")

ErrPrivateIP is returned when a URL resolves to a private or internal IP address.

Functions

func CreateImportV2Service

func CreateImportV2Service(srv *ImportV2RPC, options []connect.HandlerOption) (*api.Service, error)

CreateImportV2Service creates the service registration for rimportv2 This follows the exact same pattern as rimport.CreateService function

func IsURL

func IsURL(s string) bool

IsURL checks if a string looks like a fetchable HTTP(S) URL. Returns false for curl commands and other non-URL text.

func IsUTF8

func IsUTF8(data []byte) bool

IsUTF8 checks if data is valid UTF-8

func IsValidationError

func IsValidationError(err error) bool

IsValidationError checks if the error is a validation error

func NewHARTranslatorForTesting

func NewHARTranslatorForTesting() *defaultHARTranslator

NewHARTranslatorForTesting creates a new HAR translator for testing purposes This provides access to the HAR translator for test files while keeping the main implementation private

func NewValidationError

func NewValidationError(field, message string) error

NewValidationError creates a new validation error

func NewValidationErrorWithCause

func NewValidationErrorWithCause(field string, cause error) error

NewValidationErrorWithCause creates a new validation error with an underlying cause

func TopologicalSort

func TopologicalSort[T any](
	entities []T,
	getID func(T) idwrap.IDWrap,
	getParentID func(T) *idwrap.IDWrap,
) ([]T, error)

TopologicalSort sorts entities so parents come before children using Kahn's algorithm.

This enables handling arbitrary depth parent-child relationships (delta chains of any length). Entities with external parents (parent ID not in the current batch) are treated as roots, since their parents are assumed to already exist in the database.

Parameters:

  • entities: slice of entities to sort
  • getID: function to extract the ID from an entity
  • getParentID: function to extract the parent ID from an entity (nil if no parent)

Returns:

  • sorted slice of entities (parents before children)
  • error if a cycle is detected

func TopologicalSortWithFallback

func TopologicalSortWithFallback[T any](
	entities []T,
	getID func(T) idwrap.IDWrap,
	getParentID func(T) *idwrap.IDWrap,
	onCycle func(entities []T),
) []T

TopologicalSortWithFallback sorts entities topologically, falling back to original order on cycle.

This is useful when cycles shouldn't occur in valid data but we want to be defensive. If a cycle is detected:

  1. A warning is logged (via the provided logger callback)
  2. The original order is returned (best effort)

Parameters:

  • entities: slice of entities to sort
  • getID: function to extract the ID from an entity
  • getParentID: function to extract the parent ID from an entity (nil if no parent)
  • onCycle: optional callback invoked when a cycle is detected (can be nil)

Returns:

  • sorted slice of entities (parents before children), or original order if cycle detected

Types

type CURLTranslator

type CURLTranslator struct {
	// contains filtered or unexported fields
}

CURLTranslator implements Translator for CURL command format

func NewCURLTranslator

func NewCURLTranslator() *CURLTranslator

NewCURLTranslator creates a new CURL translator

func (*CURLTranslator) GetFormat

func (t *CURLTranslator) GetFormat() Format

func (*CURLTranslator) Translate

func (t *CURLTranslator) Translate(ctx context.Context, data []byte, workspaceID idwrap.IDWrap) (*TranslationResult, error)

func (*CURLTranslator) Validate

func (t *CURLTranslator) Validate(data []byte) error

type Deduplicator

type Deduplicator struct {
	// contains filtered or unexported fields
}

func NewDeduplicator

func NewDeduplicator(httpService shttp.HTTPService, fileService sfile.FileService, globalMu *sync.Mutex) *Deduplicator

func (*Deduplicator) FindFile

func (d *Deduplicator) FindFile(ctx context.Context, file *mfile.File, logicalPath string) (idwrap.IDWrap, string, error)

FindFile checks if a file with the same path and type already exists.

func (*Deduplicator) FindHTTP

func (d *Deduplicator) FindHTTP(
	ctx context.Context,
	req *mhttp.HTTP,
	headers []mhttp.HTTPHeader,
	params []mhttp.HTTPSearchParam,
	bodyRaw *mhttp.HTTPBodyRaw,
	bodyForms []mhttp.HTTPBodyForm,
	bodyUrlEncoded []mhttp.HTTPBodyUrlencoded,
	parentContentHash string,
) (idwrap.IDWrap, string, error)

FindHTTP checks if an identical HTTP request already exists. Returns the existing ID if found, its content hash, and an error if hashing fails.

func (*Deduplicator) ResolveFile

func (d *Deduplicator) ResolveFile(ctx context.Context, file *mfile.File, logicalPath string) (idwrap.IDWrap, bool, error)

ResolveFile ensures the file/folder structure exists. logicalPath example: "/com/example/api/users/GET_Users.request" Returns the ID and a boolean indicating if it was newly created.

func (*Deduplicator) ResolveHTTP

func (d *Deduplicator) ResolveHTTP(
	ctx context.Context,
	req *mhttp.HTTP,
	headers []mhttp.HTTPHeader,
	params []mhttp.HTTPSearchParam,
	bodyRaw *mhttp.HTTPBodyRaw,
	bodyForms []mhttp.HTTPBodyForm,
	bodyUrlEncoded []mhttp.HTTPBodyUrlencoded,
	parentContentHash string,
) (idwrap.IDWrap, bool, string, error)

ResolveHTTP ensures the HTTP request exists. Returns the ID of the existing (or newly created) request and a boolean indicating if it was newly created.

func (*Deduplicator) UpdatePathCache

func (d *Deduplicator) UpdatePathCache(pathHash string, id idwrap.IDWrap)

UpdatePathCache manually adds an entry to the path cache.

type DefaultImporter

type DefaultImporter struct {
	// contains filtered or unexported fields
}

DefaultImporter implements the Importer interface using existing modern services It coordinates HAR processing and storage operations

func NewImporter

func NewImporter(
	db *sql.DB,
	workspaceService sworkspace.WorkspaceService,
	httpService *shttp.HTTPService,
	flowService *sflow.FlowService,
	fileService *sfile.FileService,
	httpHeaderService shttp.HttpHeaderService,
	httpSearchParamService *shttp.HttpSearchParamService,
	httpBodyFormService *shttp.HttpBodyFormService,
	httpBodyUrlEncodedService *shttp.HttpBodyUrlEncodedService,
	bodyService *shttp.HttpBodyRawService,
	httpAssertService *shttp.HttpAssertService,
	nodeService *sflow.NodeService,
	nodeRequestService *sflow.NodeRequestService,
	edgeService *sflow.EdgeService,
	envService senv.EnvironmentService,
	varService senv.VariableService,
) *DefaultImporter

NewImporter creates a new DefaultImporter with service dependencies

func (*DefaultImporter) ImportAndStore

func (imp *DefaultImporter) ImportAndStore(ctx context.Context, data []byte, workspaceID idwrap.IDWrap) (*harv2.HarResolved, error)

ImportAndStore processes HAR data and returns resolved models

func (*DefaultImporter) ImportAndStoreUnified

func (imp *DefaultImporter) ImportAndStoreUnified(ctx context.Context, data []byte, workspaceID idwrap.IDWrap) (*TranslationResult, error)

ImportAndStoreUnified processes any supported format and returns unified translation results

func (*DefaultImporter) SetMutationPublisher

func (imp *DefaultImporter) SetMutationPublisher(p mutation.Publisher)

SetMutationPublisher registers the publisher used to dispatch sync events after a successful import. Wired in serverrun once both the flow and HTTP services have been constructed (their MutationPublisher() methods route to the right streamers).

func (*DefaultImporter) StoreDomainVariables

func (imp *DefaultImporter) StoreDomainVariables(ctx context.Context, workspaceID idwrap.IDWrap, domainData []ImportDomainData) ([]menv.Env, []menv.Variable, []menv.Variable, error)

StoreDomainVariables adds domain-to-variable mappings to all existing environments in the workspace. The domain URL is stored as the variable value so users can easily change the base URL by modifying the environment variable.

func (*DefaultImporter) StoreFiles

func (imp *DefaultImporter) StoreFiles(ctx context.Context, files []*mfile.File) error

StoreFiles stores file entities using the modern file service

func (*DefaultImporter) StoreFlow

func (imp *DefaultImporter) StoreFlow(ctx context.Context, flow *mflow.Flow) error

StoreFlow stores the flow entity using the modern flow service

func (*DefaultImporter) StoreFlows

func (imp *DefaultImporter) StoreFlows(ctx context.Context, flows []*mflow.Flow) error

StoreFlows stores multiple flow entities using the modern flow service

func (*DefaultImporter) StoreHTTPEntities

func (imp *DefaultImporter) StoreHTTPEntities(ctx context.Context, httpReqs []*mhttp.HTTP) error

StoreHTTPEntities stores HTTP request entities using the modern HTTP service

func (*DefaultImporter) StoreImportResults

func (imp *DefaultImporter) StoreImportResults(ctx context.Context, results *ImportResults) error

StoreImportResults performs a coordinated storage of all import results

func (*DefaultImporter) StoreUnifiedResults

func (imp *DefaultImporter) StoreUnifiedResults(ctx context.Context, results *TranslationResult) (map[idwrap.IDWrap]bool, map[idwrap.IDWrap]bool, []menv.Variable, []menv.Variable, error)

StoreUnifiedResults performs a coordinated storage of all unified translation results

type DefaultURLFetcher

type DefaultURLFetcher struct {
	// contains filtered or unexported fields
}

DefaultURLFetcher implements URLFetcher using net/http.

func NewURLFetcher

func NewURLFetcher() *DefaultURLFetcher

NewURLFetcher creates a new DefaultURLFetcher with SSRF protection. The underlying transport resolves DNS and rejects private/internal IPs before establishing a TCP connection, protecting against both direct and redirect-based SSRF attacks.

func (*DefaultURLFetcher) Fetch

func (f *DefaultURLFetcher) Fetch(ctx context.Context, rawURL string) ([]byte, error)

Fetch downloads content from the given URL.

type DefaultValidator

type DefaultValidator struct {
	// contains filtered or unexported fields
}

DefaultValidator implements the Validator interface

func NewValidator

func NewValidator(us *suser.UserService, userReader *sworkspace.UserReader) *DefaultValidator

NewValidator creates a new DefaultValidator with user service dependency

func (*DefaultValidator) ValidateDataSize

func (v *DefaultValidator) ValidateDataSize(ctx context.Context, data []byte) error

ValidateDataSize validates the size of import data

func (*DefaultValidator) ValidateFormatSupport

func (v *DefaultValidator) ValidateFormatSupport(ctx context.Context, format Format) error

ValidateFormatSupport validates that a format is supported

func (*DefaultValidator) ValidateImportRequest

func (v *DefaultValidator) ValidateImportRequest(ctx context.Context, req *ImportRequest) error

ValidateImportRequest validates the incoming import request

func (*DefaultValidator) ValidateWorkspaceAccess

func (v *DefaultValidator) ValidateWorkspaceAccess(ctx context.Context, workspaceID idwrap.IDWrap) error

ValidateWorkspaceAccess validates that the user has access to the workspace

type DetectionResult

type DetectionResult struct {
	Format     Format
	Confidence float64 // 0.0 to 1.0
	Reason     string  // Human-readable explanation
}

DetectionResult represents the result of format detection with confidence

type Format

type Format int

Format represents the supported import formats

const (
	FormatUnknown Format = iota
	FormatHAR
	FormatYAML
	FormatJSON
	FormatCURL
	FormatPostman
	FormatOpenAPI
)

func (Format) String

func (f Format) String() string

String returns the string representation of the format

type FormatDetector

type FormatDetector struct {
	// contains filtered or unexported fields
}

FormatDetector implements automatic format detection with confidence scoring

func NewFormatDetector

func NewFormatDetector() *FormatDetector

NewFormatDetector creates a new format detector with compiled patterns

func (*FormatDetector) DetectAndValidate

func (fd *FormatDetector) DetectAndValidate(data []byte) (*DetectionResult, error)

DetectAndValidate performs format detection and validation in one step

func (*FormatDetector) DetectFormat

func (fd *FormatDetector) DetectFormat(data []byte) *DetectionResult

DetectFormat automatically detects the format of input data with confidence scoring

func (*FormatDetector) ValidateFormat

func (fd *FormatDetector) ValidateFormat(data []byte, format Format) error

ValidateFormat performs additional validation on detected format

type HARTranslator

type HARTranslator struct {
	// contains filtered or unexported fields
}

HARTranslator implements Translator for HAR format

func NewHARTranslator

func NewHARTranslator(httpService *shttp.HTTPService) *HARTranslator

NewHARTranslator creates a new HAR translator

func (*HARTranslator) GetFormat

func (t *HARTranslator) GetFormat() Format

func (*HARTranslator) Translate

func (t *HARTranslator) Translate(ctx context.Context, data []byte, workspaceID idwrap.IDWrap) (*TranslationResult, error)

func (*HARTranslator) Validate

func (t *HARTranslator) Validate(data []byte) error

type HashableBodyRaw

type HashableBodyRaw struct {
	Data            string `json:"data"`
	CompressionType string `json:"compression_type"`
}

type HashableHTTP

type HashableHTTP struct {
	Method       string           `json:"method"`
	URL          string           `json:"url"`
	Headers      []HashableHeader `json:"headers"`
	Params       []HashableParam  `json:"params"`
	BodyKind     int8             `json:"body_kind"`
	BodyRaw      *HashableBodyRaw `json:"body_raw,omitempty"`
	Description  string           `json:"description,omitempty"`
	ParentHttpID string           `json:"parent_http_id,omitempty"`
	IsDelta      bool             `json:"is_delta"`
}

HashableHTTP represents the significant content of an HTTP request for deduplication. Excludes: ID, WorkspaceID, ParentID (handled by identity hash logic), CreatedAt, UpdatedAt.

type HashableHeader

type HashableHeader struct {
	Key   string `json:"key"`
	Value string `json:"value"`
}

type HashableParam

type HashableParam struct {
	Key   string `json:"key"`
	Value string `json:"value"`
}

type ImportConstraints

type ImportConstraints struct {
	MaxDataSizeBytes int64         // Maximum size of import data
	SupportedFormats []Format      // List of supported formats
	AllowedMimeTypes []string      // Allowed MIME types for file uploads
	Timeout          time.Duration // Operation timeout
}

ImportConstraints defines validation constraints for import operations

func DefaultConstraints

func DefaultConstraints() *ImportConstraints

DefaultConstraints returns sensible default constraints

type ImportDomainData

type ImportDomainData struct {
	Enabled  bool
	Domain   string
	Variable string
}

ImportDomainData represents domain variable configuration

type ImportMissingDataKind

type ImportMissingDataKind int32

ImportMissingDataKind represents the type of missing data

const (
	ImportMissingDataKind_UNSPECIFIED ImportMissingDataKind = 0
	ImportMissingDataKind_DOMAIN      ImportMissingDataKind = 1
)

type ImportRequest

type ImportRequest struct {
	WorkspaceID           idwrap.IDWrap
	Name                  string
	Data                  []byte
	TextData              string
	DomainData            []ImportDomainData
	DomainDataWasProvided bool // True if domainData was explicitly provided (even if empty array)
}

ImportRequest represents the incoming import request with domain data

type ImportResponse

type ImportResponse struct {
	MissingData ImportMissingDataKind
	Domains     []string
}

ImportResponse represents the response to an import request

type ImportResults

type ImportResults struct {
	Flow     *mflow.Flow
	HTTPReqs []*mhttp.HTTP
	Files    []*mfile.File // ALL files: HTTP, folders, AND flow files (ContentType=Flow)

	HTTPHeaders        []*mhttp.HTTPHeader
	HTTPSearchParams   []*mhttp.HTTPSearchParam
	HTTPBodyForms      []*mhttp.HTTPBodyForm
	HTTPBodyUrlEncoded []*mhttp.HTTPBodyUrlencoded
	HTTPBodyRaws       []*mhttp.HTTPBodyRaw
	HTTPAsserts        []*mhttp.HTTPAssert

	// Flow-specific entities
	Nodes        []mflow.Node
	RequestNodes []mflow.NodeRequest
	Edges        []mflow.Edge

	// Environment variables created during import (for domain-to-variable mappings)
	CreatedEnvs []menv.Env
	CreatedVars []menv.Variable
	UpdatedVars []menv.Variable

	Domains     []string
	WorkspaceID idwrap.IDWrap
	MissingData ImportMissingDataKind

	// Tracking for deduplication to prevent redundant sync events
	DeduplicatedFiles    map[idwrap.IDWrap]bool
	DeduplicatedHTTPReqs map[idwrap.IDWrap]bool
}

ImportResults represents the complete results of an import operation

type ImportServices

ImportServices groups all service dependencies

func (*ImportServices) Validate

func (s *ImportServices) Validate() error

type ImportV2Deps

type ImportV2Deps struct {
	DB        *sql.DB
	Logger    *slog.Logger
	Services  ImportServices
	Readers   ImportV2Readers
	Streamers ImportStreamers
}

func (*ImportV2Deps) Validate

func (d *ImportV2Deps) Validate() error

type ImportV2RPC

type ImportV2RPC struct {
	Logger *slog.Logger

	// Streamers for real-time updates
	FlowStream               eventstream.SyncStreamer[rflowv2.FlowTopic, rflowv2.FlowEvent]
	NodeStream               eventstream.SyncStreamer[rflowv2.NodeTopic, rflowv2.NodeEvent]
	EdgeStream               eventstream.SyncStreamer[rflowv2.EdgeTopic, rflowv2.EdgeEvent]
	HttpStream               eventstream.SyncStreamer[rhttp.HttpTopic, rhttp.HttpEvent]
	HttpHeaderStream         eventstream.SyncStreamer[rhttp.HttpHeaderTopic, rhttp.HttpHeaderEvent]
	HttpSearchParamStream    eventstream.SyncStreamer[rhttp.HttpSearchParamTopic, rhttp.HttpSearchParamEvent]
	HttpBodyFormStream       eventstream.SyncStreamer[rhttp.HttpBodyFormTopic, rhttp.HttpBodyFormEvent]
	HttpBodyUrlEncodedStream eventstream.SyncStreamer[rhttp.HttpBodyUrlEncodedTopic, rhttp.HttpBodyUrlEncodedEvent]
	HttpBodyRawStream        eventstream.SyncStreamer[rhttp.HttpBodyRawTopic, rhttp.HttpBodyRawEvent]
	HttpAssertStream         eventstream.SyncStreamer[rhttp.HttpAssertTopic, rhttp.HttpAssertEvent]
	FileStream               eventstream.SyncStreamer[rfile.FileTopic, rfile.FileEvent]
	EnvStream                eventstream.SyncStreamer[renv.EnvironmentTopic, renv.EnvironmentEvent]
	EnvVarStream             eventstream.SyncStreamer[renv.EnvironmentVariableTopic, renv.EnvironmentVariableEvent]

	// Services exposed for testing
	HttpService               *shttp.HTTPService
	FlowService               *sflow.FlowService
	FileService               *sfile.FileService
	HttpHeaderService         shttp.HttpHeaderService
	HttpSearchParamService    *shttp.HttpSearchParamService
	HttpBodyFormService       *shttp.HttpBodyFormService
	HttpBodyUrlEncodedService *shttp.HttpBodyUrlEncodedService
	HttpBodyRawService        *shttp.HttpBodyRawService
	HttpAssertService         *shttp.HttpAssertService
	NodeService               *sflow.NodeService
	NodeRequestService        *sflow.NodeRequestService
	EdgeService               *sflow.EdgeService
	EnvService                senv.EnvironmentService
	VarService                senv.VariableService
	// contains filtered or unexported fields
}

ImportV2RPC implements the Connect RPC interface for HAR import v2

func NewImportV2RPC

func NewImportV2RPC(deps ImportV2Deps) *ImportV2RPC

NewImportV2RPC creates a new ImportV2RPC handler with all required dependencies

func (*ImportV2RPC) Import

Import implements the Import RPC method from the TypeSpec interface This method delegates to the internal service after proper validation and setup

func (*ImportV2RPC) ImportUnifiedInternal

func (h *ImportV2RPC) ImportUnifiedInternal(ctx context.Context, req *ImportRequest) (*ImportResults, error)

ImportUnifiedInternal exposes the internal unified import logic for other server components

func (*ImportV2RPC) SetMutationPublisher

func (h *ImportV2RPC) SetMutationPublisher(p mutation.Publisher)

SetMutationPublisher registers the publisher used to dispatch sync events for entities created during an import. Without it, the desktop UI's TanStack DB collections only see the new rows after a manual refresh. Wired in serverrun once both the flow + http services have been built.

type ImportV2Readers

type ImportV2Readers struct {
	Workspace *sworkspace.WorkspaceReader
	User      *sworkspace.UserReader
}

func (*ImportV2Readers) Validate

func (r *ImportV2Readers) Validate() error

type Importer

type Importer interface {
	// Process and store HAR data with modern models (legacy compatibility)
	ImportAndStore(ctx context.Context, data []byte, workspaceID idwrap.IDWrap) (*harv2.HarResolved, error)
	// Process and store any supported format with automatic detection
	ImportAndStoreUnified(ctx context.Context, data []byte, workspaceID idwrap.IDWrap) (*TranslationResult, error)
	// Store individual entity types
	StoreHTTPEntities(ctx context.Context, httpReqs []*mhttp.HTTP) error
	StoreFiles(ctx context.Context, files []*mfile.File) error
	StoreFlow(ctx context.Context, flow *mflow.Flow) error
	// Store multiple flow entities using the modern flow service
	StoreFlows(ctx context.Context, flows []*mflow.Flow) error

	// StoreImportResults performs a coordinated storage of all import results (legacy)
	StoreImportResults(ctx context.Context, results *ImportResults) error

	// StoreUnifiedResults performs a coordinated storage of all unified translation results
	StoreUnifiedResults(ctx context.Context, results *TranslationResult) (map[idwrap.IDWrap]bool, map[idwrap.IDWrap]bool, []menv.Variable, []menv.Variable, error)

	// Store domain-to-variable mappings to all existing environments
	StoreDomainVariables(ctx context.Context, workspaceID idwrap.IDWrap, domainData []ImportDomainData) (createdEnvs []menv.Env, createdVars []menv.Variable, updatedVars []menv.Variable, err error)
}

Importer handles the complete import pipeline: format detection, processing and storage

type IntegrityError

type IntegrityError struct {
	EntityType string        // "file", "request_node", "http"
	EntityID   idwrap.IDWrap // ID of the problematic entity
	Field      string        // Which field has the issue
	RefID      idwrap.IDWrap // The referenced ID that is missing/invalid
	Message    string
}

IntegrityError represents a specific integrity violation

func (IntegrityError) Error

func (e IntegrityError) Error() string

type IntegrityReport

type IntegrityReport struct {
	Errors []IntegrityError
}

IntegrityReport contains all integrity violations found

func ValidateImportIntegrity

func ValidateImportIntegrity(
	ctx context.Context,
	workspaceID idwrap.IDWrap,
	fileService *sfile.FileService,
	httpService *shttp.HTTPService,
	nodeService *sflow.NodeService,
	nodeRequestService *sflow.NodeRequestService,
) (*IntegrityReport, error)

ValidateImportIntegrity checks that all imported entities are correctly linked. This should be called after an import to verify data consistency.

func ValidateTranslationResult

func ValidateTranslationResult(result *TranslationResult) *IntegrityReport

ValidateTranslationResult validates the in-memory translation result before storage. This catches issues at the translation layer before they hit the database.

func (*IntegrityReport) AddError

func (r *IntegrityReport) AddError(entityType string, entityID idwrap.IDWrap, field string, refID idwrap.IDWrap, message string)

func (*IntegrityReport) Error

func (r *IntegrityReport) Error() string

func (*IntegrityReport) HasErrors

func (r *IntegrityReport) HasErrors() bool

type JSONTranslator

type JSONTranslator struct {
	// contains filtered or unexported fields
}

JSONTranslator implements Translator for generic JSON format

func NewJSONTranslator

func NewJSONTranslator() *JSONTranslator

NewJSONTranslator creates a new JSON translator

func (*JSONTranslator) GetFormat

func (t *JSONTranslator) GetFormat() Format

func (*JSONTranslator) Translate

func (t *JSONTranslator) Translate(ctx context.Context, data []byte, workspaceID idwrap.IDWrap) (*TranslationResult, error)

func (*JSONTranslator) Validate

func (t *JSONTranslator) Validate(data []byte) error

type OpenAPITranslator

type OpenAPITranslator struct {
	// contains filtered or unexported fields
}

OpenAPITranslator implements Translator for OpenAPI/Swagger spec format

func NewOpenAPITranslator

func NewOpenAPITranslator() *OpenAPITranslator

NewOpenAPITranslator creates a new OpenAPI translator

func (*OpenAPITranslator) GetFormat

func (t *OpenAPITranslator) GetFormat() Format

func (*OpenAPITranslator) Translate

func (t *OpenAPITranslator) Translate(ctx context.Context, data []byte, workspaceID idwrap.IDWrap) (*TranslationResult, error)

func (*OpenAPITranslator) Validate

func (t *OpenAPITranslator) Validate(data []byte) error

type PostmanTranslator

type PostmanTranslator struct {
	// contains filtered or unexported fields
}

PostmanTranslator implements Translator for Postman collection format

func NewPostmanTranslator

func NewPostmanTranslator() *PostmanTranslator

NewPostmanTranslator creates a new Postman translator

func (*PostmanTranslator) GetFormat

func (t *PostmanTranslator) GetFormat() Format

func (*PostmanTranslator) Translate

func (t *PostmanTranslator) Translate(ctx context.Context, data []byte, workspaceID idwrap.IDWrap) (*TranslationResult, error)

func (*PostmanTranslator) Validate

func (t *PostmanTranslator) Validate(data []byte) error

type Service

type Service struct {
	// contains filtered or unexported fields
}

Service implements the main business logic for unified import

func NewService

func NewService(importer Importer, validator Validator, opts ...ServiceOption) *Service

NewService creates a new Service with dependency injection and optional configuration Required dependencies: importer and validator Optional dependencies can be configured using ServiceOption functions

func (*Service) DetectFormat

func (s *Service) DetectFormat(ctx context.Context, data []byte) (*DetectionResult, error)

DetectFormat performs format detection on the provided data

func (*Service) GetSupportedFormats

func (s *Service) GetSupportedFormats() []Format

GetSupportedFormats returns all supported import formats

func (*Service) Import

func (s *Service) Import(ctx context.Context, req *ImportRequest) (*ImportResults, error)

Import processes a HAR file and stores the results using modern models

func (*Service) ImportUnified

func (s *Service) ImportUnified(ctx context.Context, req *ImportRequest) (*ImportResults, error)

ImportUnified processes any supported format with automatic detection. It handles three input modes:

  1. Binary data (req.Data) - file uploads (Postman JSON, HAR, etc.)
  2. Text data (req.TextData) - pasted curl commands, raw JSON/YAML
  3. URL text (req.TextData is a URL) - fetches content from URL (e.g., swagger.json URL)

func (*Service) ImportUnifiedWithTextData

func (s *Service) ImportUnifiedWithTextData(ctx context.Context, req *ImportRequest) (*ImportResults, error)

ImportUnifiedWithTextData processes any supported format from text with automatic detection

func (*Service) ImportWithTextData

func (s *Service) ImportWithTextData(ctx context.Context, req *ImportRequest) (*ImportResults, error)

ImportWithTextData processes HAR data from text format

func (*Service) ValidateAndSanitizeRequest

func (s *Service) ValidateAndSanitizeRequest(ctx context.Context, req *ImportRequest) (*ImportRequest, error)

ValidateAndSanitizeRequest validates and sanitizes import request data

func (*Service) ValidateFormat

func (s *Service) ValidateFormat(ctx context.Context, data []byte, format Format) error

ValidateFormat validates data for a specific format

func (*Service) ValidateImportRequestExtended

func (s *Service) ValidateImportRequestExtended(ctx context.Context, req *ImportRequest, constraints *ImportConstraints) error

ValidateImportRequestExtended performs comprehensive validation of import requests

type ServiceOption

type ServiceOption func(*Service)

ServiceOption configures the Service during construction

func WithHTTPService

func WithHTTPService(httpService *shttp.HTTPService) ServiceOption

WithHTTPService sets the HTTP service for the service (required for HAR import overwrite detection)

func WithLogger

func WithLogger(logger *slog.Logger) ServiceOption

WithLogger sets a custom logger

func WithTimeout

func WithTimeout(timeout time.Duration) ServiceOption

WithTimeout sets the processing timeout for HAR operations

func WithURLFetcher

func WithURLFetcher(fetcher URLFetcher) ServiceOption

WithURLFetcher sets a custom URL fetcher (useful for testing)

type TranslationOptions

type TranslationOptions struct {
	WorkspaceID       idwrap.IDWrap
	GenerateFiles     bool
	FileOrder         int
	EnableCompression bool
	CompressionType   int8
}

TranslationOptions provides options for the translation process

func DefaultTranslationOptions

func DefaultTranslationOptions(workspaceID idwrap.IDWrap) *TranslationOptions

DefaultTranslationOptions returns sensible default options

func (*TranslationOptions) MergeWithDefaults

func (opts *TranslationOptions) MergeWithDefaults(workspaceID idwrap.IDWrap) *TranslationOptions

MergeWithDefaults creates a complete TranslationOptions by merging with defaults

func (*TranslationOptions) WithCompression

func (opts *TranslationOptions) WithCompression(enable bool, compressionType int8) *TranslationOptions

WithCompression configures compression options

func (*TranslationOptions) WithFiles

func (opts *TranslationOptions) WithFiles(generate bool) *TranslationOptions

WithFiles configures the translation to generate files

type TranslationResult

type TranslationResult struct {
	// Core entities
	HTTPRequests []mhttp.HTTP
	Files        []mfile.File // ALL files: HTTP, folders, AND flow files (ContentType=Flow)
	Flows        []mflow.Flow

	// Associated HTTP data (headers, params, bodies)
	Headers        []mhttp.HTTPHeader
	SearchParams   []mhttp.HTTPSearchParam
	BodyForms      []mhttp.HTTPBodyForm
	BodyUrlencoded []mhttp.HTTPBodyUrlencoded
	BodyRaw        []mhttp.HTTPBodyRaw
	Asserts        []mhttp.HTTPAssert

	// Flow-specific entities
	Nodes        []mflow.Node
	RequestNodes []mflow.NodeRequest
	Edges        []mflow.Edge

	// Flow node implementations (type-specific data)
	JSNodes        []mflow.NodeJS
	ConditionNodes []mflow.NodeIf
	ForNodes       []mflow.NodeFor
	ForEachNodes   []mflow.NodeForEach
	AINodes        []mflow.NodeAI
	FlowVariables  []mflow.FlowVariable

	// Variables (collection or environment level)
	Variables []menv.Variable

	// Metadata
	DetectedFormat Format
	Domains        []string
	ProcessedAt    int64
	WorkspaceID    idwrap.IDWrap
}

TranslationResult represents the unified result from any translator

type Translator

type Translator interface {
	// Translate converts input data to the unified TranslationResult format
	Translate(ctx context.Context, data []byte, workspaceID idwrap.IDWrap) (*TranslationResult, error)

	// GetFormat returns the format this translator handles
	GetFormat() Format

	// Validate checks if the data is valid for this format
	Validate(data []byte) error
}

Translator defines the unified interface for all format translators

type TranslatorRegistry

type TranslatorRegistry struct {
	// contains filtered or unexported fields
}

TranslatorRegistry manages all available translators

func NewTranslatorRegistry

func NewTranslatorRegistry(httpService *shttp.HTTPService) *TranslatorRegistry

NewTranslatorRegistry creates a new registry with all available translators

func (*TranslatorRegistry) DetectAndTranslate

func (r *TranslatorRegistry) DetectAndTranslate(ctx context.Context, data []byte, workspaceID idwrap.IDWrap) (*TranslationResult, error)

DetectAndTranslate detects format and translates data in one step

func (*TranslatorRegistry) GetSupportedFormats

func (r *TranslatorRegistry) GetSupportedFormats() []Format

GetSupportedFormats returns all supported formats

func (*TranslatorRegistry) GetTranslator

func (r *TranslatorRegistry) GetTranslator(format Format) (Translator, error)

GetTranslator returns the translator for the given format

func (*TranslatorRegistry) RegisterTranslator

func (r *TranslatorRegistry) RegisterTranslator(translator Translator)

RegisterTranslator adds a translator to the registry

func (*TranslatorRegistry) ValidateFormat

func (r *TranslatorRegistry) ValidateFormat(data []byte, format Format) error

ValidateFormat validates data for a specific format

type URLFetcher

type URLFetcher interface {
	Fetch(ctx context.Context, rawURL string) ([]byte, error)
}

URLFetcher fetches content from URLs. It's an interface to allow testing.

type ValidationError

type ValidationError struct {
	Field   string
	Message string
	Err     error
}

ValidationError represents an input validation error

func (*ValidationError) Error

func (e *ValidationError) Error() string

func (*ValidationError) Unwrap

func (e *ValidationError) Unwrap() error

type Validator

type Validator interface {
	ValidateImportRequest(ctx context.Context, req *ImportRequest) error
	ValidateWorkspaceAccess(ctx context.Context, workspaceID idwrap.IDWrap) error
	ValidateDataSize(ctx context.Context, data []byte) error
	ValidateFormatSupport(ctx context.Context, format Format) error
}

Validator handles input validation for import requests

type YAMLTranslator

type YAMLTranslator struct {
	// contains filtered or unexported fields
}

YAMLTranslator implements Translator for YAML flow format

func NewYAMLTranslator

func NewYAMLTranslator() *YAMLTranslator

NewYAMLTranslator creates a new YAML translator

func (*YAMLTranslator) GetFormat

func (t *YAMLTranslator) GetFormat() Format

func (*YAMLTranslator) Translate

func (t *YAMLTranslator) Translate(ctx context.Context, data []byte, workspaceID idwrap.IDWrap) (*TranslationResult, error)

func (*YAMLTranslator) Validate

func (t *YAMLTranslator) Validate(data []byte) error

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL