Documentation
¶
Overview ¶
Package histimport parses the history exports of other apps (TezLab, ABRP) into drive and charge candidates. It is pure: no database, no network, no logging of row values. The caller (the import API) does the dedupe against stored rows and the writes.
The export formats are not documented anywhere public, so each source is a Format: a table of header aliases per field, built from the columns we assume and meant to be edited once real exports are in hand (slice 2b). The parser fails closed: a required field with no matching header refuses the file (MissingColumnsError, header names only), a unit or zone the file does not state is asked of the user (NeedChoiceError), a row that fails a check is skipped and counted by reason, and a file with more than 20% failing rows is refused (TooManyBadRowsError).
Index ¶
- Constants
- Variables
- func ExternalID(source, rivianVehicleID, kind string, start time.Time) string
- type Charge
- type ColumnSpec
- type Dim
- type Drive
- type ExpectedLayout
- type Field
- type Format
- type Kind
- type MissingColumnsError
- type NeedChoiceError
- type Options
- type Report
- type Result
- type Table
- type TooManyBadRowsError
Constants ¶
const (
FStart, FEnd = "start", "end"
FTime = "time"
FSessionID = "session"
FDistance = "distance"
FStartSoC, FEndSoC = "start_soc", "end_soc"
FSoC = "soc"
FStartOdo, FEndOdo = "start_odometer", "end_odometer"
FOdometer = "odometer"
FStartLat, FStartLon = "start_lat", "start_lon"
FEndLat, FEndLon = "end_lat", "end_lon"
FLat, FLon = "lat", "lon"
FMaxSpeed, FAvgSpeed = "max_speed", "avg_speed"
FSpeed = "speed"
FEnergyUsed = "energy_used"
FEnergyAdded = "energy_added"
FMaxPower, FAvgPower = "max_power", "avg_power"
FPower = "power"
FCost, FCurrency = "cost", "currency"
FVehicle = "vehicle"
)
Canonical field names.
const ( ReasonBadNumber = "bad_number" ReasonBadTime = "bad_time" ReasonMissing = "missing_value" ReasonEndBeforeBeg = "ended_before_started" ReasonFuture = "ended_in_future" ReasonTooOld = "before_2021" ReasonTooLong = "too_long" ReasonSoCRange = "soc_out_of_range" ReasonDistance = "bad_distance" ReasonSpeed = "speed_too_high" ReasonEnergy = "energy_out_of_range" ReasonPower = "power_out_of_range" ReasonRunTooLong = "run_too_long" ReasonDuplicate = "duplicate_in_file" ReasonOtherVeh = "other_vehicle" ReasonParked = "parked" ReasonEmpty = "empty_session" )
Skip reasons. The first group counts toward MaxBadRowFraction (the row or session is wrong); the second is expected noise.
const ( MaxRows = 500_000 MaxCols = 200 MaxSessions = 100_000 // drives or charges one file may produce; a decade of ten drives a day is under 40k MaxCSVBytes = 200 << 20 MaxXLSXZipBytes = 50 << 20 MaxXLSXUnzipped = 200 << 20 )
Caps on what a file may hold. The upload is capped at 50 MB by the caller; these bound what it can expand to.
const AliasProvenance = "assumed: no published column list for TezLab or ABRP exports (checked Nov 13)"
Provenance of the header aliases, checked Nov 13: neither TezLab (support article 86, "Exporting Data"), Tessie (article 68, "Import from TezLab") nor ABRP (help article 7182124 "My Drives", and the feedback threads on export separators, temperatures and odometer) publishes a column list, a unit or a time format. So every alias below is an assumption drawn from the vocabulary exports of this kind use, and none is pinned to a real file. The parser is built for that: a file whose required columns do not match is refused with the header names it has and the names we expected (RequiredColumns), never imported in part.
const MaxBadRowFraction = 0.20
MaxBadRowFraction: a file with more failing rows than this is refused as a wrong mapping, not noise (design §2).
Variables ¶
var ( TezLabDrives = Format{ Source: "tezlab_import", Kind: KindDrives, Fields: []Field{ {Name: FStart, Dim: DimTime, Required: true, Aliases: []string{"start time", "started at", "start", "start date", "departure time", "begin time", "date"}}, {Name: FEnd, Dim: DimTime, Required: true, Aliases: []string{"end time", "ended at", "end", "end date", "arrival time", "finish time"}}, {Name: FDistance, Dim: DimDistance, Required: true, Aliases: []string{"distance", "drive distance", "trip distance", "distance driven"}}, {Name: FStartSoC, Dim: DimSoC, Aliases: []string{"start battery", "start soc", "start battery level", "starting battery", "starting soc", "battery start"}}, {Name: FEndSoC, Dim: DimSoC, Aliases: []string{"end battery", "end soc", "end battery level", "ending battery", "ending soc", "battery end"}}, {Name: FStartOdo, Dim: DimDistance, Aliases: []string{"start odometer", "odometer start", "starting odometer"}}, {Name: FEndOdo, Dim: DimDistance, Aliases: []string{"end odometer", "odometer end", "ending odometer"}}, {Name: FStartLat, Dim: DimAngle, Aliases: []string{"start latitude", "start lat", "origin latitude"}}, {Name: FStartLon, Dim: DimAngle, Aliases: []string{"start longitude", "start lon", "start lng", "origin longitude"}}, {Name: FEndLat, Dim: DimAngle, Aliases: []string{"end latitude", "end lat", "destination latitude"}}, {Name: FEndLon, Dim: DimAngle, Aliases: []string{"end longitude", "end lon", "end lng", "destination longitude"}}, {Name: FMaxSpeed, Dim: DimSpeed, Aliases: []string{"max speed", "maximum speed", "top speed"}}, {Name: FAvgSpeed, Dim: DimSpeed, Aliases: []string{"avg speed", "average speed"}}, {Name: FEnergyUsed, Dim: DimEnergy, Aliases: []string{"energy used", "energy consumed", "consumed energy"}}, {Name: FVehicle, Dim: DimText, Aliases: []string{"vehicle", "vehicle name", "car", "vin"}}, }, } TezLabCharges = Format{ Source: "tezlab_import", Kind: KindCharges, EnergyBasis: "unknown", Fields: []Field{ {Name: FStart, Dim: DimTime, Required: true, Aliases: []string{"start time", "started at", "start", "start date", "begin time", "date"}}, {Name: FEnd, Dim: DimTime, Required: true, Aliases: []string{"end time", "ended at", "end", "end date", "finish time"}}, {Name: FStartSoC, Dim: DimSoC, Aliases: []string{"start battery", "start soc", "start battery level", "starting battery", "starting soc", "battery start"}}, {Name: FEndSoC, Dim: DimSoC, Aliases: []string{"end battery", "end soc", "end battery level", "ending battery", "ending soc", "battery end"}}, {Name: FEnergyAdded, Dim: DimEnergy, Aliases: []string{"energy added", "added energy", "charge energy added", "energy", "charged"}}, {Name: FMaxPower, Dim: DimPower, Aliases: []string{"max power", "peak power", "maximum power", "max charge rate"}}, {Name: FAvgPower, Dim: DimPower, Aliases: []string{"avg power", "average power", "avg charge rate"}}, {Name: FStartLat, Dim: DimAngle, Aliases: []string{"latitude", "lat", "location latitude"}}, {Name: FStartLon, Dim: DimAngle, Aliases: []string{"longitude", "lon", "lng", "location longitude"}}, {Name: FCost, Dim: DimMoney, Aliases: []string{"cost", "total cost", "charge cost", "price"}}, {Name: FCurrency, Dim: DimText, Aliases: []string{"currency", "cost currency"}}, {Name: FVehicle, Dim: DimText, Aliases: []string{"vehicle", "vehicle name", "car", "vin"}}, }, } )
TezLab exports one CSV per kind (Activity tab: Drives, Charges). Its columns are not published, so these aliases are assumptions (see AliasProvenance); a spelling that turns up in a real file is added here, never guessed in the parser. The energy basis of its charges is unknown, so it is recorded as "unknown" and kept out of efficiency ratios.
var ABRPPoints = Format{ Source: "abrp_import", Kind: KindPoints, Fields: []Field{ {Name: FTime, Dim: DimTime, Required: true, Aliases: []string{"time", "timestamp", "date", "datetime", "date time", "utc"}}, {Name: FLat, Dim: DimAngle, Required: true, Aliases: []string{"latitude", "lat"}}, {Name: FLon, Dim: DimAngle, Required: true, Aliases: []string{"longitude", "lon", "lng", "long"}}, {Name: FSoC, Dim: DimSoC, Required: true, Aliases: []string{"soc", "state of charge", "battery level", "battery", "battery soc"}}, {Name: FSpeed, Dim: DimSpeed, Aliases: []string{"speed", "vehicle speed", "gps speed"}}, {Name: FOdometer, Dim: DimDistance, Aliases: []string{"odometer", "odo"}}, {Name: FSessionID, Dim: DimText, Aliases: []string{"drive id", "session", "session id", "trip id", "trip", "drive"}}, {Name: FVehicle, Dim: DimText, Aliases: []string{"vehicle", "vehicle name", "car", "vin"}}, }, }
ABRP's Drive & charge history export is an .xlsx of recorded points (design §1; the format is reported by users, not documented). The aliases are assumptions (see AliasProvenance). Drives and charges are derived from the points (points.go); points do not go into vehicle_state. SoC on a Rivian is whole percent only, so energy from SoC steps is coarse by about 1.3 kWh.
var ErrTooBig = errors.New("histimport: file is too big")
ErrTooBig: the file exceeds a cap (rows, columns, bytes).
Functions ¶
Types ¶
type Charge ¶
type Charge struct {
StartedAt, EndedAt time.Time
StartSoCPct, EndSoCPct float64
EnergyAddedKWh float64
// EnergyBasis is what EnergyAddedKWh measures, from the format: "battery", "wall" or "unknown". EnergyEstimated: it came from SoC × pack.
EnergyBasis string
EnergyEstimated bool
MaxPowerKW float64
AvgPowerKW float64
Lat, Lon float64
SessionType string // "dc" when the peak power is at least 25 kW, else "ac"
Cost float64
Currency string
HasCost bool
Source string
VehicleLabel string
}
Charge is a parsed charge.
type ColumnSpec ¶ added in v0.41.24
ColumnSpec is one required column of a format: the canonical field and the header spellings the parser accepts for it (first few).
type Dim ¶
type Dim int
Dim is the physical dimension of a field, which says what its header unit can be.
type Drive ¶
type Drive struct {
StartedAt, EndedAt time.Time
StartSoCPct, EndSoCPct float64
StartOdometerMi, EndOdometerMi float64
DistanceMi float64
StartLat, StartLon float64
EndLat, EndLon float64
MaxSpeedMph, AvgSpeedMph float64
// EnergyUsedKWh is the pack-side energy: the file's own consumed energy when its header says so, else (start SoC - end SoC) × the pack
// given in Options. EnergyEstimated marks the second case; zero with no pack.
EnergyUsedKWh float64
EnergyEstimated bool
RoutePolyline string // precision 5; only for sources that carry points
Source string
VehicleLabel string // the file's own label for the vehicle, "" when none
}
Drive is a parsed drive, in the units and zone the rest of Rivolt uses (miles, kWh, percent, UTC). It maps onto drives.Drive; the import API sets the id, vehicle and import batch.
type ExpectedLayout ¶ added in v0.41.24
type ExpectedLayout struct {
Kind string `json:"kind"` // "drives", "charges" or "points"
Columns []ColumnSpec `json:"columns"`
}
ExpectedLayout is one file layout of a source and the columns it requires.
func RequiredColumns ¶ added in v0.41.24
func RequiredColumns(formats []Format) []ExpectedLayout
RequiredColumns lists, per format, the required fields and up to three accepted spellings of each, in the format's own order. It carries no file content, so it can go to the user.
type Field ¶
Field is one canonical field and the header spellings that mean it. Aliases are compared after normalising (see baseName): case, spaces, punctuation and any unit word are ignored, so "Distance (km)", "distance_km" and "DISTANCE" are all "distance".
type Format ¶
type Format struct {
Source string // "tezlab_import", "abrp_import": the Source of the rows it produces
Kind Kind
Fields []Field
EnergyBasis string // what the file's charge energy measures: "battery", "wall" or "unknown" (recorded on the row, never guessed)
}
Format describes one export file: which source it comes from, what its rows are, and its columns. Formats are data, so a corrected or additional spelling is a one-line change (slice 2b pins them to real exports).
type MissingColumnsError ¶
type MissingColumnsError struct {
Missing []string // canonical names of the required fields not found
Header []string
}
MissingColumnsError refuses a file with no header for a required field. It carries header names only, never row values, so it can be shown to the user and sent to support.
func (*MissingColumnsError) Error ¶
func (e *MissingColumnsError) Error() string
type NeedChoiceError ¶
type NeedChoiceError struct {
Choices []string // "distance_unit" (km|mi), "energy_unit" (kwh|wh), "timezone", "vehicle"
Vehicles []string // for "vehicle": the labels the file holds
}
NeedChoiceError: the file does not say something the numbers depend on, so the user must (the preview asks). Never guessed.
func (*NeedChoiceError) Error ¶
func (e *NeedChoiceError) Error() string
type Options ¶
type Options struct {
Location *time.Location // zone of times that carry none
DistanceUnit string // "km" or "mi", for headers that state no unit
EnergyUnit string // "kwh" or "wh", for energy and power headers that state none
PackKWh float64 // usable pack, for energy derived from SoC; 0 leaves such energy out
Vehicle string // the file's own label of the vehicle to keep, when the file holds several
Now time.Time // for the no-future check; zero means time.Now()
}
Options are what the user confirmed in the preview. A zero value is "not stated": the parser asks (NeedChoiceError) when the file does not state it either.
type Report ¶
type Report struct {
Rows int // data rows read
Accepted int // sessions produced
Skipped map[string]int // sessions or rows left out, by reason (a points file counts points for parse failures and sessions for the rest)
BadRows int // rows behind failing sessions: what MaxBadRowFraction is measured on
Ambiguous int // local times inside a DST fall-back hour, kept as the first instant
// DroppedCoords counts session positions that were off the globe (or 0,0) and were left out; the session itself is kept.
DroppedCoords int
// Assumptions the numbers rest on, shown in the preview.
SoCScale string // "percent" or "fraction" (file said, or every value was ≤ 1)
DistanceUnit string
EnergyUnit string
Vehicles []string // distinct vehicle labels in the file (at most 20)
First, Last time.Time
}
Report says what happened to every row, so the preview can show that new + skipped = rows.
type Table ¶
type Table interface {
Header() []string
// Next returns the next non-empty row, or io.EOF.
Next() ([]string, error)
}
Table is a header plus a stream of rows. The row slice returned by Next is only valid until the next call.
type TooManyBadRowsError ¶
TooManyBadRowsError refuses a file whose rows mostly fail: its columns do not mean what the format says.
func (*TooManyBadRowsError) Error ¶
func (e *TooManyBadRowsError) Error() string