Documentation
¶
Index ¶
- Variables
- func DefaultExcludePatterns() []string
- func IsDefaultExclude(pattern string) bool
- type DocSummary
- type ExcludeEntry
- type PageEntry
- type Store
- func (s *Store) AddExclude(dir, pattern string) (*ExcludeEntry, error)
- func (s *Store) AutoIndexAllowed(dir string) (bool, error)
- func (s *Store) DeleteAllByPrefix(dirPrefix string) error
- func (s *Store) DeleteByDoc(docPath string) error
- func (s *Store) DeleteExclude(excludeID string) error
- func (s *Store) GetByDoc(docPath string) ([]*PageEntry, error)
- func (s *Store) IsDocIndexed(docPath string) (bool, error)
- func (s *Store) ListDocModTimes(dirPrefix string) (map[string]int64, error)
- func (s *Store) ListDocPaths(dirPrefix string) ([]string, error)
- func (s *Store) ListDocsSummary(dirPrefix string) ([]*DocSummary, error)
- func (s *Store) ListDocxFiles(dirPrefix string) ([]*PageEntry, error)
- func (s *Store) ListExcludes(dir string) ([]*ExcludeEntry, error)
- func (s *Store) ListPDFFiles(dirPrefix string) ([]*PageEntry, error)
- func (s *Store) ListSheetFiles(dirPrefix string) ([]*PageEntry, error)
- func (s *Store) ListTextFiles(dirPrefix string) ([]*PageEntry, error)
- func (s *Store) SeedDefaultExcludes(dir string) error
- func (s *Store) SetAutoIndexAllowed(dir string, allowed bool) error
- func (s *Store) UpdateLabels(docPath string, pageNum int, labels []string) error
- func (s *Store) Upsert(entry *PageEntry) error
Constants ¶
This section is empty.
Variables ¶
var SheetExtensions = []string{".xlsx", ".xlsm", ".csv", ".tsv"}
SheetExtensions are the file extensions treated as spreadsheets everywhere the document index understands them. Every stored page of such a document belongs to one sheet (a delimited file is the single sheet "Sheet1"), so callers group the pages by DocPath instead of counting rows as files.
This slice is the single owner of the list: the store's query clauses (see sheetPathMatchConditions / sheetPathExclusionConditions) and the indexer's extension dispatch read it. internal/sheet (the leaf profiling package) and internal/server/query_sheet_loader keep their own copies, since importing docindex from a leaf is not allowed and those files sit outside this package's seam — parity is pinned by tests in internal/docindex and internal/indexer.
.xls and .ods are deliberately in NO list (excelize cannot read BIFF/ODF), so they belong to neither the sheet pages nor the text listing.
Functions ¶
func DefaultExcludePatterns ¶ added in v0.39.0
func DefaultExcludePatterns() []string
DefaultExcludePatterns returns the shipped list, for callers that need to describe or apply it without going through the store.
func IsDefaultExclude ¶ added in v0.39.0
IsDefaultExclude reports whether pattern is one ogcode ships. Keeping the answer in Go rather than a column means the shipped list stays the single source of truth, and a release that changes it is reflected the next time the panel reads — with no migration and no rows carrying a stale origin.
Types ¶
type DocSummary ¶
type DocSummary struct {
DocPath string `json:"docPath"`
PageCount int `json:"pageCount"`
Pages []*PageEntry `json:"pages,omitempty"`
IndexedAt int64 `json:"indexedAt"`
}
DocSummary holds a document's metadata for the UI listing. Pages is only populated by callers that explicitly attach full page data; the docs listing leaves it empty to keep the payload small.
type ExcludeEntry ¶ added in v0.7.0
type ExcludeEntry struct {
ID string `json:"id"`
Directory string `json:"directory"`
Pattern string `json:"pattern"`
CreatedAt int64 `json:"createdAt"`
// Default marks a pattern ogcode ships rather than one the user wrote. It is
// derived from the list at read time, never stored, so the panel's label
// cannot drift from the list it describes.
Default bool `json:"default"`
}
ExcludeEntry is a pattern that the indexer skips during a walk. Patterns are matched against directory names and file basenames. Glob wildcards (e.g. *.min.js) are supported via filepath.Match.
type PageEntry ¶
type PageEntry struct {
ID string `json:"id"`
DocPath string `json:"docPath"`
PageNum int `json:"pageNum"`
Keywords []string `json:"keywords"`
Labels []string `json:"labels"`
IndexedAt int64 `json:"indexedAt"`
// ModTime is the file's modification time in Unix milliseconds as it stood
// when the page was indexed. It is what lets a later run recognise a file
// that has been rewritten and re-index it, rather than skipping it because
// its path is already known. 0 means no time was recorded (a row written
// before the column existed, or a stat that failed), which reads as stale.
ModTime int64 `json:"modTime"`
}
PageEntry represents a single indexed page in a PDF document.
type Store ¶
type Store struct {
// contains filtered or unexported fields
}
Store provides persistence for doc_page_index entries.
func (*Store) AddExclude ¶ added in v0.7.0
func (s *Store) AddExclude(dir, pattern string) (*ExcludeEntry, error)
AddExclude inserts a new exclude pattern for a directory. Duplicate patterns are ignored.
func (*Store) AutoIndexAllowed ¶ added in v0.47.0
AutoIndexAllowed reports whether the project at dir has consented to being indexed automatically once it passes the file limit the caller enforces.
The answer is per directory and defaults to false: a project nobody has answered for is asked, not assumed, so the missing-row case and a stored 0 both read the same way. A nil store or an empty dir is likewise false rather than an error — the callers treat "no answer" and "no project" the same way.
func (*Store) DeleteAllByPrefix ¶
DeleteAllByPrefix deletes all entries for docs under dirPrefix (directory boundary semantics, see dirPrefixFilter).
func (*Store) DeleteByDoc ¶
DeleteByDoc removes all PageEntry rows for a given document path.
func (*Store) DeleteExclude ¶ added in v0.7.0
DeleteExclude removes an exclude entry by ID.
func (*Store) IsDocIndexed ¶ added in v0.7.0
IsDocIndexed reports whether any pages for the given document path exist in the index.
func (*Store) ListDocModTimes ¶ added in v0.39.0
ListDocModTimes returns the newest recorded mod_time (Unix milliseconds) for every indexed doc_path under dirPrefix (directory boundary semantics, see dirPrefixFilter). A run compares these against the files' times on disk to tell a rewritten file from an untouched one; a path absent from the map is simply not indexed yet. A value of 0 means no time was recorded, which reads as stale.
func (*Store) ListDocPaths ¶ added in v0.19.3
ListDocPaths returns the distinct doc_path values under dirPrefix (directory boundary semantics, see dirPrefixFilter). It is used by the indexer to detect stale entries for files that no longer exist on disk.
func (*Store) ListDocsSummary ¶
func (s *Store) ListDocsSummary(dirPrefix string) ([]*DocSummary, error)
ListDocsSummary returns one DocSummary per unique doc_path whose path is under dirPrefix. It aggregates in a single query and does not load per-page label data — callers that need full pages should use GetByDoc.
func (*Store) ListDocxFiles ¶ added in v0.13.6
ListDocxFiles returns all indexed DOCX entries (page-level) whose doc_path is under dirPrefix (directory-boundary semantics, see dirPrefixFilter), ordered by path then page number. Each DOCX document will have one entry per pseudo-page; callers group them by DocPath.
func (*Store) ListExcludes ¶ added in v0.7.0
func (s *Store) ListExcludes(dir string) ([]*ExcludeEntry, error)
ListExcludes returns all exclude patterns stored for the given directory.
func (*Store) ListPDFFiles ¶ added in v0.13.6
ListPDFFiles returns all indexed PDF entries (page-level) whose doc_path is under dirPrefix (directory-boundary semantics, see dirPrefixFilter), ordered by path then page number. Each PDF document will have one entry per page; callers group them by DocPath.
func (*Store) ListSheetFiles ¶ added in v0.55.0
ListSheetFiles returns all indexed spreadsheet entries (page-level) whose doc_path is under dirPrefix (directory-boundary semantics, see dirPrefixFilter), ordered by path then page number. Each spreadsheet gets one entry per sheet (a delimited file is one sheet named "Sheet1"), so callers group them by DocPath. The extensions come from SheetExtensions; .xls/.ods are in no list and are indexed by neither.
func (*Store) ListTextFiles ¶ added in v0.7.0
ListTextFiles returns all indexed entries that belong to a single-entry text/code file whose doc_path is under dirPrefix (the boundary is the directory separator, so a sibling folder sharing the prefix never matches), ordered by path. Each text/code file has exactly one entry (page 1). Multi-page documents — PDF, DOCX and spreadsheets (one page per sheet, see internal/sheet) — are excluded here by extension, and the page_num = 1 filter keeps a future multi-page kind that is not yet excluded from leaking its later pages in as files: only a first page can be a single-entry file. Callers of the multi-page queries must group pages by DocPath rather than count rows as files.
func (*Store) SeedDefaultExcludes ¶ added in v0.7.0
SeedDefaultExcludes inserts the shipped patterns for a directory, once.
The unit of "once" is a row in index_exclude_seed, not the number of exclude rows the directory happens to hold. Seeding is therefore idempotent, and — the point of the marker — a default the user deletes stays deleted: seeding keys on whether this directory was ever seeded, so a later run does not read a shorter list and helpfully put back what was removed.
INSERT OR IGNORE keeps a pattern the user already wrote exactly as they wrote it, so seeding can only add.
func (*Store) SetAutoIndexAllowed ¶ added in v0.47.0
SetAutoIndexAllowed records the project's answer. It replaces any prior answer for the directory, so a toggle is last-write-wins, and it is safe on a nil store or an empty dir (a no-op).
func (*Store) UpdateLabels ¶
UpdateLabels updates only the labels field for a specific page.
Source Files