hupd

func EnsureMeta ¶

func EnsureMeta(
	ctx context.Context,
	metaPath, metaURL string,
	bar ProgressReporter,
	mu *sync.Mutex,
	log *slog.Logger,
) error

EnsureMeta downloads the HUPD metadata Feather file from metaURL to metaPath unless the file already exists on disk. The download is written to a sibling temp file and atomically renamed on success, so a partial download never leaves a corrupt file at metaPath.

bar and mu are used for progress reporting: mu must be held while calling bar methods so the caller's progress bar is updated from a single goroutine. log may be nil (no progress lines are emitted in that case).

func FmtCount ¶

func FmtCount(n int64) string

FmtCount renders n with thousands separators, e.g. 1234567 -> "1,234,567".

func FmtRate ¶

func FmtRate(perSec float64) string

FmtRate renders a per-second rate compactly, e.g. 12345 -> "12.3K/s".

func NormalizeFamily ¶

func NormalizeFamily(fps []string) []string

NormalizeFamily maps fps through NormalizeUSID, discarding empty results.

package main

import (
	"fmt"

	"github.com/Qubut/epo-processor/internal/hupd"
)

func main() {
	// Filters a family-member list to US-normalised numeric IDs only.
	// Non-US identifiers and unparseable entries are silently dropped.
	ids := hupd.NormalizeFamily([]string{
		"US9114971B2",
		"EP1234567A1", // non-US, dropped
		"US20120043352A1",
		"", // empty, dropped
	})
	for _, id := range ids {
		fmt.Println(id)
	}
}

Output:
9114971
20120043352

func NormalizeHUPDPatentNumber ¶

func NormalizeHUPDPatentNumber(s string) string

NormalizeHUPDPatentNumber returns the granted-patent-number stem (bare numeric string in HUPD).

package main

import (
	"fmt"

	"github.com/Qubut/epo-processor/internal/hupd"
)

func main() {
	// HUPD stores granted patent numbers as plain numeric strings.
	// Normalisation trims whitespace; the string is returned unchanged
	// if it is already a bare number.
	fmt.Println(hupd.NormalizeHUPDPatentNumber("  9114971  "))
	fmt.Println(hupd.NormalizeHUPDPatentNumber("9114971"))
}

Output:
9114971
9114971

func NormalizeHUPDPublication ¶

func NormalizeHUPDPublication(raw string) string

NormalizeHUPDPublication converts a raw HUPD publication_number into the same canonical stem produced by NormalizeUSID for an EPO patent identifier.

HUPD stores publication numbers in USPTO format with a trailing application date, e.g. "US20100138160A1-20100603". USPTO uses an 11-digit body (4-digit year + 7-digit serial) whereas EPO/DOCDB uses 10 (year + 6-digit serial), dropping the serial's leading zero. Without this reconciliation a HUPD publication ("US20100138160A1") and the matching EPO patent_id ("US2010138160A1") normalise to different stems ("20100138160" vs "2010138160") and never match — which previously left the EPO↔HUPD self-link (DatasetRecord.EPOHUPDPaths) empty for every application-style US patent.

Mirrors the Python reference: publication_number.split('-')[0] then s[:6] + s[7:] (drop the extra serial digit at index 6).

package main

import (
	"fmt"

	"github.com/Qubut/epo-processor/internal/hupd"
)

func main() {
	// HUPD stores publication numbers in USPTO format (11-digit body) with a
	// trailing application date. EPO/DOCDB uses a 10-digit body, dropping the
	// serial's leading zero. NormalizeHUPDPublication reconciles them so a HUPD
	// publication and the matching EPO patent_id collapse to the same stem.
	fmt.Println(hupd.NormalizeHUPDPublication("US20100138160A1-20100603"))
	fmt.Println(hupd.NormalizeUSID("US2010138160A1")) // EPO patent_id, same stem
}

Output:
2010138160
2010138160

func NormalizeUSID ¶

func NormalizeUSID(id string) string

NormalizeUSID returns the canonical numeric stem of a US-prefixed EPO identifier. "US9114971B2" → "9114971", "US20120043352A1" → "20120043352". Returns "" for non-US identifiers and unparseable inputs.

package main

import (
	"fmt"

	"github.com/Qubut/epo-processor/internal/hupd"
)

func main() {
	// Strips the "US" prefix and kind code to yield the bare numeric string
	// that matches the EPO patent_id after the same normalisation.
	fmt.Println(hupd.NormalizeUSID("US9114971B2"))
	fmt.Println(hupd.NormalizeUSID("US20120043352A1"))
	fmt.Println(hupd.NormalizeUSID("EP1234567A1")) // non-US → empty
}

Output:
9114971
20120043352

func ReadIDs ¶

func ReadIDs(path string) ([2]string, error)

ReadIDs reads one HUPD JSON and returns its two normalized US identifiers (patent_number, publication_number). Either may be empty.

func ScanIDs ¶

func ScanIDs(
	ctx context.Context,
	dir string,
	workers int,
	bar ProgressReporter,
	mu *sync.Mutex,
	log *slog.Logger,
) (map[string][]string, error)

ScanIDs walks dir recursively, reads the patent_number and publication_number header fields from each .json file in parallel (up to workers goroutines), and returns a normalised-ID → []path index identical in shape to the one produced by ScanMeta.

Use ScanIDs when the Feather metadata file is unavailable or when only a subset of the full HUPD dataset is present on disk.

bar and mu are used for progress reporting (see EnsureMeta). log may be nil.

func ScanMeta ¶

func ScanMeta(
	ctx context.Context,
	metaPath, metaURL, hupdDir string,
	bar ProgressReporter,
	mu *sync.Mutex,
	log *slog.Logger,
) (map[string][]string, error)

ScanMeta builds the normalised-ID → []path index from the HUPD Feather metadata file. If metaPath does not exist it is downloaded first via EnsureMeta.

Columns consumed: application_number, patent_number, publication_number, filing_date. Per-row paths are inferred as hupdDir/<year>/<year>/<app>.json where year is derived from the filing_date column.

Each unique normalised ID (NormalizeUSID) maps to one or more paths; multiple paths arise when the same patent appears under more than one application number in the index.

bar and mu are used for progress reporting (see EnsureMeta).