Documentation
¶
Overview ¶
Package histimport parses the history exports of other apps (TezLab, ABRP) into drive and charge candidates. It is pure: no database, no network, no logging of row values. The caller (the import API) does the dedupe against stored rows and the writes.
The export formats are not documented anywhere public, so each source is a Format: a table of header aliases per field, built from the columns we assume and meant to be edited once real exports are in hand (slice 2b). The parser fails closed: a required field with no matching header refuses the file (MissingColumnsError, header names only), a unit or zone the file does not state is asked of the user (NeedChoiceError), a row that fails a check is skipped and counted by reason, and a file with more than 20% failing rows is refused (TooManyBadRowsError).
Index ¶
Constants ¶
const (
FStart, FEnd = "start", "end"
FTime = "time"
FSessionID = "session"
FDistance = "distance"
FStartSoC, FEndSoC = "start_soc", "end_soc"
FSoC = "soc"
FStartOdo, FEndOdo = "start_odometer", "end_odometer"
FOdometer = "odometer"
FStartLat, FStartLon = "start_lat", "start_lon"
FEndLat, FEndLon = "end_lat", "end_lon"
FLat, FLon = "lat", "lon"
FMaxSpeed, FAvgSpeed = "max_speed", "avg_speed"
FSpeed = "speed"
FEnergyUsed = "energy_used"
FEnergyAdded = "energy_added"
FMaxPower, FAvgPower = "max_power", "avg_power"
FPower = "power"
FCost, FCurrency = "cost", "currency"
FVehicle = "vehicle"
)
Canonical field names.
const ( ReasonBadNumber = "bad_number" ReasonBadTime = "bad_time" ReasonMissing = "missing_value" ReasonEndBeforeBeg = "ended_before_started" ReasonFuture = "ended_in_future" ReasonTooOld = "before_2021" ReasonTooLong = "too_long" ReasonSoCRange = "soc_out_of_range" ReasonDistance = "bad_distance" ReasonSpeed = "speed_too_high" ReasonEnergy = "energy_out_of_range" ReasonPower = "power_out_of_range" ReasonRunTooLong = "run_too_long" ReasonDuplicate = "duplicate_in_file" ReasonOtherVeh = "other_vehicle" ReasonParked = "parked" ReasonEmpty = "empty_session" )
Skip reasons. The first group counts toward MaxBadRowFraction (the row or session is wrong); the second is expected noise.
const ( MaxRows = 500_000 MaxCols = 200 MaxSessions = 100_000 // drives or charges one file may produce; a decade of ten drives a day is under 40k MaxCSVBytes = 200 << 20 MaxXLSXZipBytes = 50 << 20 MaxXLSXUnzipped = 200 << 20 )
Caps on what a file may hold. The upload is capped at 50 MB by the caller; these bound what it can expand to.
const MaxBadRowFraction = 0.20
MaxBadRowFraction: a file with more failing rows than this is refused as a wrong mapping, not noise (design §2).
Variables ¶
var ( TezLabDrives = Format{ Source: "tezlab_import", Kind: KindDrives, Fields: []Field{ {Name: FStart, Dim: DimTime, Required: true, Aliases: []string{"start time", "started at", "start", "start date", "departure time", "begin time", "date"}}, {Name: FEnd, Dim: DimTime, Required: true, Aliases: []string{"end time", "ended at", "end", "end date", "arrival time", "finish time"}}, {Name: FDistance, Dim: DimDistance, Required: true, Aliases: []string{"distance", "drive distance", "trip distance", "distance driven"}}, {Name: FStartSoC, Dim: DimSoC, Aliases: []string{"start battery", "start soc", "start battery level", "starting battery", "starting soc", "battery start"}}, {Name: FEndSoC, Dim: DimSoC, Aliases: []string{"end battery", "end soc", "end battery level", "ending battery", "ending soc", "battery end"}}, {Name: FStartOdo, Dim: DimDistance, Aliases: []string{"start odometer", "odometer start", "starting odometer"}}, {Name: FEndOdo, Dim: DimDistance, Aliases: []string{"end odometer", "odometer end", "ending odometer"}}, {Name: FStartLat, Dim: DimAngle, Aliases: []string{"start latitude", "start lat", "origin latitude"}}, {Name: FStartLon, Dim: DimAngle, Aliases: []string{"start longitude", "start lon", "start lng", "origin longitude"}}, {Name: FEndLat, Dim: DimAngle, Aliases: []string{"end latitude", "end lat", "destination latitude"}}, {Name: FEndLon, Dim: DimAngle, Aliases: []string{"end longitude", "end lon", "end lng", "destination longitude"}}, {Name: FMaxSpeed, Dim: DimSpeed, Aliases: []string{"max speed", "maximum speed", "top speed"}}, {Name: FAvgSpeed, Dim: DimSpeed, Aliases: []string{"avg speed", "average speed"}}, {Name: FEnergyUsed, Dim: DimEnergy, Aliases: []string{"energy used", "energy consumed", "consumed energy"}}, {Name: FVehicle, Dim: DimText, Aliases: []string{"vehicle", "vehicle name", "car", "vin"}}, }, } TezLabCharges = Format{ Source: "tezlab_import", Kind: KindCharges, EnergyBasis: "unknown", Fields: []Field{ {Name: FStart, Dim: DimTime, Required: true, Aliases: []string{"start time", "started at", "start", "start date", "begin time", "date"}}, {Name: FEnd, Dim: DimTime, Required: true, Aliases: []string{"end time", "ended at", "end", "end date", "finish time"}}, {Name: FStartSoC, Dim: DimSoC, Aliases: []string{"start battery", "start soc", "start battery level", "starting battery", "starting soc", "battery start"}}, {Name: FEndSoC, Dim: DimSoC, Aliases: []string{"end battery", "end soc", "end battery level", "ending battery", "ending soc", "battery end"}}, {Name: FEnergyAdded, Dim: DimEnergy, Aliases: []string{"energy added", "added energy", "charge energy added", "energy", "charged"}}, {Name: FMaxPower, Dim: DimPower, Aliases: []string{"max power", "peak power", "maximum power", "max charge rate"}}, {Name: FAvgPower, Dim: DimPower, Aliases: []string{"avg power", "average power", "avg charge rate"}}, {Name: FStartLat, Dim: DimAngle, Aliases: []string{"latitude", "lat", "location latitude"}}, {Name: FStartLon, Dim: DimAngle, Aliases: []string{"longitude", "lon", "lng", "location longitude"}}, {Name: FCost, Dim: DimMoney, Aliases: []string{"cost", "total cost", "charge cost", "price"}}, {Name: FCurrency, Dim: DimText, Aliases: []string{"currency", "cost currency"}}, {Name: FVehicle, Dim: DimText, Aliases: []string{"vehicle", "vehicle name", "car", "vin"}}, }, } )
TezLab exports one CSV per kind (Activity tab: Drives, Charges). Nothing public documents its columns, so these aliases are assumptions (design §1) to be pinned to a real export in slice 2b. Add a spelling here, never a guess in the parser. The energy basis of its charges is unknown until then, so it is recorded as "unknown" and kept out of efficiency ratios.
var ABRPPoints = Format{ Source: "abrp_import", Kind: KindPoints, Fields: []Field{ {Name: FTime, Dim: DimTime, Required: true, Aliases: []string{"time", "timestamp", "date", "datetime", "date time", "utc"}}, {Name: FLat, Dim: DimAngle, Required: true, Aliases: []string{"latitude", "lat"}}, {Name: FLon, Dim: DimAngle, Required: true, Aliases: []string{"longitude", "lon", "lng", "long"}}, {Name: FSoC, Dim: DimSoC, Required: true, Aliases: []string{"soc", "state of charge", "battery level", "battery", "battery soc"}}, {Name: FSpeed, Dim: DimSpeed, Aliases: []string{"speed", "vehicle speed", "gps speed"}}, {Name: FOdometer, Dim: DimDistance, Aliases: []string{"odometer", "odo"}}, {Name: FSessionID, Dim: DimText, Aliases: []string{"drive id", "session", "session id", "trip id", "trip", "drive"}}, {Name: FVehicle, Dim: DimText, Aliases: []string{"vehicle", "vehicle name", "car", "vin"}}, }, }
ABRP's Drive & charge history export is an .xlsx of recorded points (design §1; the format is reported by users, not documented). The aliases are assumptions to be pinned in slice 2b. Drives and charges are derived from the points (points.go); points do not go into vehicle_state. SoC on a Rivian is whole percent only, so energy from SoC steps is coarse by about 1.3 kWh.
var ErrTooBig = errors.New("histimport: file is too big")
ErrTooBig: the file exceeds a cap (rows, columns, bytes).
Functions ¶
Types ¶
type Charge ¶
type Charge struct {
StartedAt, EndedAt time.Time
StartSoCPct, EndSoCPct float64
EnergyAddedKWh float64
// EnergyBasis is what EnergyAddedKWh measures, from the format: "battery", "wall" or "unknown". EnergyEstimated: it came from SoC × pack.
EnergyBasis string
EnergyEstimated bool
MaxPowerKW float64
AvgPowerKW float64
Lat, Lon float64
SessionType string // "dc" when the peak power is at least 25 kW, else "ac"
Cost float64
Currency string
HasCost bool
Source string
VehicleLabel string
}
Charge is a parsed charge.
type Dim ¶
type Dim int
Dim is the physical dimension of a field, which says what its header unit can be.
type Drive ¶
type Drive struct {
StartedAt, EndedAt time.Time
StartSoCPct, EndSoCPct float64
StartOdometerMi, EndOdometerMi float64
DistanceMi float64
StartLat, StartLon float64
EndLat, EndLon float64
MaxSpeedMph, AvgSpeedMph float64
// EnergyUsedKWh is the pack-side energy: the file's own consumed energy when its header says so, else (start SoC - end SoC) × the pack
// given in Options. EnergyEstimated marks the second case; zero with no pack.
EnergyUsedKWh float64
EnergyEstimated bool
RoutePolyline string // precision 5; only for sources that carry points
Source string
VehicleLabel string // the file's own label for the vehicle, "" when none
}
Drive is a parsed drive, in the units and zone the rest of Rivolt uses (miles, kWh, percent, UTC). It maps onto drives.Drive; the import API sets the id, vehicle and import batch.
type Field ¶
Field is one canonical field and the header spellings that mean it. Aliases are compared after normalising (see baseName): case, spaces, punctuation and any unit word are ignored, so "Distance (km)", "distance_km" and "DISTANCE" are all "distance".
type Format ¶
type Format struct {
Source string // "tezlab_import", "abrp_import": the Source of the rows it produces
Kind Kind
Fields []Field
EnergyBasis string // what the file's charge energy measures: "battery", "wall" or "unknown" (recorded on the row, never guessed)
}
Format describes one export file: which source it comes from, what its rows are, and its columns. Formats are data, so a corrected or additional spelling is a one-line change (slice 2b pins them to real exports).
type MissingColumnsError ¶
type MissingColumnsError struct {
Missing []string // canonical names of the required fields not found
Header []string
}
MissingColumnsError refuses a file with no header for a required field. It carries header names only, never row values, so it can be shown to the user and sent to support.
func (*MissingColumnsError) Error ¶
func (e *MissingColumnsError) Error() string
type NeedChoiceError ¶
type NeedChoiceError struct {
Choices []string // "distance_unit" (km|mi), "energy_unit" (kwh|wh), "timezone", "vehicle"
Vehicles []string // for "vehicle": the labels the file holds
}
NeedChoiceError: the file does not say something the numbers depend on, so the user must (the preview asks). Never guessed.
func (*NeedChoiceError) Error ¶
func (e *NeedChoiceError) Error() string
type Options ¶
type Options struct {
Location *time.Location // zone of times that carry none
DistanceUnit string // "km" or "mi", for headers that state no unit
EnergyUnit string // "kwh" or "wh", for energy and power headers that state none
PackKWh float64 // usable pack, for energy derived from SoC; 0 leaves such energy out
Vehicle string // the file's own label of the vehicle to keep, when the file holds several
Now time.Time // for the no-future check; zero means time.Now()
}
Options are what the user confirmed in the preview. A zero value is "not stated": the parser asks (NeedChoiceError) when the file does not state it either.
type Report ¶
type Report struct {
Rows int // data rows read
Accepted int // sessions produced
Skipped map[string]int // sessions or rows left out, by reason (a points file counts points for parse failures and sessions for the rest)
BadRows int // rows behind failing sessions: what MaxBadRowFraction is measured on
Ambiguous int // local times inside a DST fall-back hour, kept as the first instant
// DroppedCoords counts session positions that were off the globe (or 0,0) and were left out; the session itself is kept.
DroppedCoords int
// Assumptions the numbers rest on, shown in the preview.
SoCScale string // "percent" or "fraction" (file said, or every value was ≤ 1)
DistanceUnit string
EnergyUnit string
Vehicles []string // distinct vehicle labels in the file (at most 20)
First, Last time.Time
}
Report says what happened to every row, so the preview can show that new + skipped = rows.
type Table ¶
type Table interface {
Header() []string
// Next returns the next non-empty row, or io.EOF.
Next() ([]string, error)
}
Table is a header plus a stream of rows. The row slice returned by Next is only valid until the next call.
type TooManyBadRowsError ¶
TooManyBadRowsError refuses a file whose rows mostly fail: its columns do not mean what the format says.
func (*TooManyBadRowsError) Error ¶
func (e *TooManyBadRowsError) Error() string