Documentation
¶
Overview ¶
Package copier provides tools and utilities for copying and modifying GTFS feeds.
Index ¶
- type AfterValidator
- type AfterWrite
- type Copier
- type EntityMarker
- type ErrorHandler
- type ErrorThresholdFileResult
- type ErrorThresholdResult
- type ExpandFilter
- type Extension
- type Filter
- type Marker
- type Options
- type Prepare
- type RequiredMinEntitiesFileResult
- type RequiredMinEntitiesResult
- type Result
- func Copy(ctx context.Context, reader adapters.Reader, writer adapters.Writer, ...) (*Result, error)
- func CopyWithOptions(ctx context.Context, reader adapters.Reader, writer adapters.Writer, ...) (*Result, error)
- func NewResult(errorLimit int) *Result
- func QuietCopy(ctx context.Context, reader adapters.Reader, writer adapters.Writer, ...) (*Result, error)
- func (cr *Result) CheckErrorThreshold(thresholds map[string]float64) ErrorThresholdResult
- func (cr *Result) CheckRequiredMinEntities(requirements map[string]int) RequiredMinEntitiesResult
- func (cr *Result) DisplayErrors()
- func (cr *Result) DisplaySummary()
- func (cr *Result) DisplayWarnings()
- func (cr *Result) HandleEntityErrors(ent tt.Entity, errs []error, warns []error)
- func (cr *Result) HandleError(fn string, errs []error)
- func (cr *Result) HandleSourceErrors(fn string, errs []error, warns []error)
- type ValidationError
- type ValidationErrorGroup
- type Validator
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type AfterValidator ¶ added in v0.8.1
AfterValidator is called for each fully validated entity before writing.
type AfterWrite ¶ added in v0.8.8
AfterWrite is called for after writing each entity.
type Copier ¶
type Copier struct {
// book keeping
EntityMap *tt.EntityMap
// contains filtered or unexported fields
}
Copier copies from Reader to Writer
func NewCopier ¶
func NewCopier(ctx context.Context, reader adapters.Reader, writer adapters.Writer, opts Options) (*Copier, error)
NewCopier creates and initializes a new Copier.
func (*Copier) Copy ¶
Copy copies Base GTFS entities from the Reader to the Writer, returning the summary as a Result.
func (*Copier) CopyEntities ¶ added in v0.10.0
func (*Copier) CopyEntity ¶
CopyEntity performs validation and saves errors and warnings.
type EntityMarker ¶ added in v1.2.4
EntityMarker is a marker interface that checks if an entity is marked.
type ErrorHandler ¶
type ErrorHandler interface {
HandleEntityErrors(tt.Entity, []error, []error)
HandleSourceErrors(string, []error, []error)
}
ErrorHandler is called on each source file and entity; errors can be nil
type ErrorThresholdFileResult ¶ added in v1.3.0
type ErrorThresholdFileResult struct {
TotalCount int
ErrorCount int
ErrorPercent float64
Threshold float64
OK bool
}
ErrorThresholdFileResult contains threshold check results for a single file.
type ErrorThresholdResult ¶ added in v1.3.0
type ErrorThresholdResult struct {
OK bool
Details map[string]ErrorThresholdFileResult
}
ErrorThresholdResult contains the result of checking error thresholds per file.
type ExpandFilter ¶ added in v0.11.0
type Extension ¶ added in v0.8.1
type Extension interface {
Copy(adapters.EntityCopier) error
}
Extension is run after normal copying has completed.
type Options ¶ added in v0.8.0
type Options struct {
// Batch size
BatchSize int
// Skip most validation filters
NoValidators bool
// Skip shape cache
NoShapeCache bool
// Attempt to save an entity that returns validation errors
AllowEntityErrors bool
AllowReferenceErrors bool
// Interpolate any missing StopTime values: ArrivalTime/DepartureTime/ShapeDistTraveled
InterpolateStopTimes bool
// Create a stop-to-stop Shape for Trips without a ShapeID.
CreateMissingShapes bool
// Create missing Calendar entries
NormalizeServiceIDs bool
// Normalize timezones, e.g. US/Pacific -> America/Los_Angeles
NormalizeTimezones bool
// Simplify Calendars that use mostly CalendarDates
SimplifyCalendars bool
// Convert extended route types to primitives
UseBasicRouteTypes bool
// Copy extra files (requires CSV input)
CopyExtraFiles bool
// Simplify shapes
SimplifyShapes float64
// Convert route network_id to networks.txt/route_networks.txt
NormalizeNetworks bool
// DeduplicateStopTimes
DeduplicateJourneyPatterns bool
// Error limit
ErrorLimit int
// Logging level
Quiet bool
// Default error handler
ErrorHandler ErrorHandler
// Entity selection strategy
Marker Marker
// Journey Pattern Key Function
JourneyPatternKey func(*gtfs.Trip) string
// Named extensions
ExtensionDefs []string
// contains filtered or unexported fields
}
Options defines the settable options for a Copier.
func (*Options) AddExtension ¶ added in v0.18.0
func (*Options) AddExtensionWithLevel ¶ added in v1.0.0
type RequiredMinEntitiesFileResult ¶ added in v1.3.0
RequiredMinEntitiesFileResult contains the check result for a single file.
type RequiredMinEntitiesResult ¶ added in v1.3.0
type RequiredMinEntitiesResult struct {
OK bool
Details map[string]RequiredMinEntitiesFileResult
}
RequiredMinEntitiesResult contains the result of checking required minimum entity counts per file.
type Result ¶ added in v0.8.0
type Result struct {
InterpolatedStopTimeCount int
EntityCount map[string]int
GeneratedCount map[string]int
SkipEntityErrorCount map[string]int
SkipEntityReferenceCount map[string]int
SkipEntityFilterCount map[string]int
SkipEntityMarkedCount map[string]int
Errors map[string]*ValidationErrorGroup
Warnings map[string]*ValidationErrorGroup
ErrorLimit int
}
Result stores Copier results and statistics.
func Copy ¶ added in v0.18.0
func Copy(ctx context.Context, reader adapters.Reader, writer adapters.Writer, optfns ...func(*Options)) (*Result, error)
Copy with options builder
func CopyWithOptions ¶ added in v1.0.0
func QuietCopy ¶ added in v0.18.0
func QuietCopy(ctx context.Context, reader adapters.Reader, writer adapters.Writer, optfns ...func(*Options)) (*Result, error)
Quiet copy
func (*Result) CheckErrorThreshold ¶ added in v1.3.0
func (cr *Result) CheckErrorThreshold(thresholds map[string]float64) ErrorThresholdResult
CheckErrorThreshold checks if any file exceeds its error threshold percentage. Thresholds are specified as 0-100 (e.g., 5 means 5%, 10 means 10%). A threshold of 0 means any error is a failure. Files are only checked if they have an explicit threshold or a default "*" threshold is set. The thresholds map uses filename as key (e.g., "stops.txt") with "*" as the default. The error rate is calculated as (entity errors + reference errors) / total entities * 100. Entities skipped by filters or markers do not count toward the error rate.
func (*Result) CheckRequiredMinEntities ¶ added in v1.3.0
func (cr *Result) CheckRequiredMinEntities(requirements map[string]int) RequiredMinEntitiesResult
CheckRequiredMinEntities checks if files meet their required minimum entity counts. The requirements map uses filename as key (e.g., "agency.txt") and minimum count as value. Returns a result indicating which files failed and why.
func (*Result) DisplayErrors ¶ added in v0.8.0
func (cr *Result) DisplayErrors()
DisplayErrors shows individual errors in log.Info
func (*Result) DisplaySummary ¶ added in v0.8.0
func (cr *Result) DisplaySummary()
DisplaySummary shows entity and error counts in log.Info
func (*Result) DisplayWarnings ¶ added in v0.8.0
func (cr *Result) DisplayWarnings()
DisplayWarnings shows individual warnings in log.Info
func (*Result) HandleEntityErrors ¶ added in v0.8.0
HandleEntityErrors .
func (*Result) HandleError ¶ added in v0.8.0
HandleError .
type ValidationError ¶ added in v0.16.0
type ValidationError struct {
Filename string `db:"-"`
Field string `db:"-"`
ErrorCode string `db:"-"`
Line int
GroupKey string
Message string
EntityID string
Value string
Geometry tt.Geometry
EntityJson tt.Map
}
func (ValidationError) Error ¶ added in v0.16.0
func (e ValidationError) Error() string
type ValidationErrorGroup ¶ added in v0.16.0
type ValidationErrorGroup struct {
Filename string
Field string
ErrorType string
ErrorCode string
GroupKey string
Level int
Count int
Limit int `db:"-"`
Errors []ValidationError `db:"-"`
}
ValidationErrorGroup helps group errors together with a maximum limit on the number stored.
func NewValidationErrorGroup ¶ added in v0.16.0
func NewValidationErrorGroup(err error, limit int) *ValidationErrorGroup
func (*ValidationErrorGroup) Add ¶ added in v0.16.0
func (e *ValidationErrorGroup) Add(err error)
Add an error to the error group.
func (*ValidationErrorGroup) Key ¶ added in v0.16.0
func (eg *ValidationErrorGroup) Key() string