store

package
v0.7.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Feb 9, 2026 License: MIT Imports: 14 Imported by: 0

Documentation

Overview

Package store provides database access for msgvault.

Index

Constants

View Source 
const (
	SyncStatusRunning   = "running"
	SyncStatusCompleted = "completed"
	SyncStatusFailed    = "failed"
)

Variables

This section is empty.

Functions

func IsSystemLabel added in v0.4.0 

func IsSystemLabel(sourceLabelID string) bool

IsSystemLabel returns true if the given Gmail label ID represents a system label.

Types

type APIAttachment added in v0.7.0 

type APIAttachment struct {
	Filename string
	MimeType string
	Size     int64
}

APIAttachment represents attachment metadata for API responses.

type APIMessage added in v0.7.0 

type APIMessage struct {
	ID             int64
	Subject        string
	From           string
	To             []string
	SentAt         time.Time
	Snippet        string
	Labels         []string
	HasAttachments bool
	SizeEstimate   int64
	Body           string
	Headers        map[string]string
	Attachments    []APIAttachment
}

APIMessage represents a message for API responses.

type Checkpoint 

type Checkpoint struct {
	PageToken         string
	MessagesProcessed int64
	MessagesAdded     int64
	MessagesUpdated   int64
	ErrorsCount       int64
}

Checkpoint represents sync progress for resumption.

type Label 

type Label struct {
	ID            int64
	SourceID      sql.NullInt64
	SourceLabelID sql.NullString
	Name          string
	LabelType     sql.NullString
}

Label represents a Gmail label.

type LabelInfo added in v0.4.0 

type LabelInfo struct {
	Name string
	Type string // "system" or "user"
}

LabelInfo holds the name and type for a label to be ensured.

type Message 

type Message struct {
	ID              int64
	ConversationID  int64
	SourceID        int64
	SourceMessageID string
	MessageType     string // "email"
	SentAt          sql.NullTime
	ReceivedAt      sql.NullTime
	InternalDate    sql.NullTime
	SenderID        sql.NullInt64
	IsFromMe        bool
	Subject         sql.NullString
	Snippet         sql.NullString
	SizeEstimate    int64
	HasAttachments  bool
	AttachmentCount int
	DeletedAt       sql.NullTime
	ArchivedAt      time.Time
}

Message represents a message in the database.

type MessageInspection added in v0.4.0 

type MessageInspection struct {
	SentAt               string
	InternalDate         string
	DeletedFromSourceAt  sql.NullTime
	ThreadSourceID       string
	BodyText             string
	RawDataExists        bool
	RecipientCounts      map[string]int    // recipient_type -> count
	RecipientDisplayName map[string]string // "type:email" -> display_name
}

MessageInspection contains detailed message data for test assertions.

type Participant 

type Participant struct {
	ID           int64
	EmailAddress sql.NullString
	DisplayName  sql.NullString
	Domain       sql.NullString
}

Participant represents a person in the participants table.

type Source 

type Source struct {
	ID           int64
	SourceType   string // "gmail"
	Identifier   string // email address
	DisplayName  sql.NullString
	GoogleUserID sql.NullString
	LastSyncAt   sql.NullTime
	SyncCursor   sql.NullString // historyId for Gmail
	CreatedAt    time.Time
	UpdatedAt    time.Time
}

Source represents a Gmail account or other message source.

type Stats 

type Stats struct {
	MessageCount    int64
	ThreadCount     int64
	AttachmentCount int64
	LabelCount      int64
	SourceCount     int64
	DatabaseSize    int64
}

Stats holds database statistics.

type Store 

type Store struct {
	// contains filtered or unexported fields
}

Store provides database operations for msgvault.

func Open 

func Open(dbPath string) (*Store, error)

Open opens or creates the database at the given path. Currently only SQLite is supported. PostgreSQL URLs will return an error.

func (*Store) AddMessageLabels added in v0.7.0 

func (s *Store) AddMessageLabels(messageID int64, labelIDs []int64) error

AddMessageLabels adds labels to a message without removing existing ones. Uses INSERT OR IGNORE to skip labels that already exist.

func (*Store) BackfillFTS added in v0.6.4 

func (s *Store) BackfillFTS(progress func(done, total int64)) (int64, error)

BackfillFTS populates the FTS table from existing message data. Processes in batches to avoid blocking for minutes on large archives. The progress callback (if non-nil) is called after each batch with (position in ID range, total ID range). Each batch is committed independently so partial progress is preserved if interrupted. Returns the number of rows inserted. No-op if FTS5 is not available.

func (*Store) Close 

func (s *Store) Close() error

Close closes the database connection.

func (*Store) CompleteSync 

func (s *Store) CompleteSync(syncID int64, finalHistoryID string) error

CompleteSync marks a sync as successfully completed.

func (*Store) CountMessagesForSource 

func (s *Store) CountMessagesForSource(sourceID int64) (int64, error)

CountMessagesForSource returns the count of messages for a specific source (account).

func (*Store) CountMessagesWithRaw 

func (s *Store) CountMessagesWithRaw(sourceID int64) (int64, error)

CountMessagesWithRaw returns the count of messages that have raw MIME stored.

func (*Store) DB 

func (s *Store) DB() *sql.DB

DB returns the underlying database connection for advanced queries.

func (*Store) EnsureConversation 

func (s *Store) EnsureConversation(sourceID int64, sourceConversationID, title string) (int64, error)

EnsureConversation gets or creates a conversation (thread) for a message.

func (*Store) EnsureLabel 

func (s *Store) EnsureLabel(sourceID int64, sourceLabelID, name, labelType string) (int64, error)

EnsureLabel gets or creates a label.

func (*Store) EnsureLabelsBatch 

func (s *Store) EnsureLabelsBatch(sourceID int64, labels map[string]LabelInfo) (map[string]int64, error)

EnsureLabelsBatch ensures all labels exist and returns a map of source_label_id -> internal ID.

func (*Store) EnsureParticipant 

func (s *Store) EnsureParticipant(email, displayName, domain string) (int64, error)

EnsureParticipant gets or creates a participant by email.

func (*Store) EnsureParticipantsBatch 

func (s *Store) EnsureParticipantsBatch(addresses []mime.Address) (map[string]int64, error)

EnsureParticipantsBatch gets or creates participants in batch. Returns a map of email -> participant ID.

func (*Store) FTS5Available added in v0.6.4 

func (s *Store) FTS5Available() bool

FTS5Available returns whether FTS5 full-text search is available.

func (*Store) FailSync 

func (s *Store) FailSync(syncID int64, errMsg string) error

FailSync marks a sync as failed with an error message.

func (*Store) GetActiveSync 

func (s *Store) GetActiveSync(sourceID int64) (*SyncRun, error)

GetActiveSync returns the most recent running sync for a source, if any.

func (*Store) GetLastSuccessfulSync 

func (s *Store) GetLastSuccessfulSync(sourceID int64) (*SyncRun, error)

GetLastSuccessfulSync returns the most recent successful sync for a source.

func (*Store) GetMessage added in v0.7.0 

func (s *Store) GetMessage(id int64) (*APIMessage, error)

GetMessage returns a single message with full details. Only this method accesses message_bodies (single PK lookup).

func (*Store) GetMessageRaw 

func (s *Store) GetMessageRaw(messageID int64) ([]byte, error)

GetMessageRaw retrieves and decompresses the raw MIME data for a message.

func (*Store) GetOrCreateSource 

func (s *Store) GetOrCreateSource(sourceType, identifier string) (*Source, error)

GetOrCreateSource gets or creates a source by type and identifier.

func (*Store) GetRandomMessageIDs 

func (s *Store) GetRandomMessageIDs(sourceID int64, limit int) ([]int64, error)

GetRandomMessageIDs returns a random sample of message IDs for a source. Uses reservoir sampling with random offsets for O(limit) performance on large tables, falling back to ORDER BY RANDOM() for small tables where the overhead isn't significant.

func (*Store) GetSourceByIdentifier 

func (s *Store) GetSourceByIdentifier(identifier string) (*Source, error)

GetSourceByIdentifier returns a source by its identifier (email address).

func (*Store) GetStats 

func (s *Store) GetStats() (*Stats, error)

GetStats returns statistics about the database.

func (*Store) InitSchema 

func (s *Store) InitSchema() error

InitSchema initializes the database schema. This creates all tables if they don't exist.

func (*Store) InspectAttachment added in v0.6.5 

func (s *Store) InspectAttachment(sourceMessageID string) (filename, mimeType string, err error)

InspectAttachment returns the filename and mime_type for the first attachment (by ID) of a message.

func (*Store) InspectBodyText added in v0.4.0 

func (s *Store) InspectBodyText(sourceMessageID string) (string, error)

InspectBodyText returns the body_text for a message.

func (*Store) InspectDeletedFromSource added in v0.4.0 

func (s *Store) InspectDeletedFromSource(sourceMessageID string) (bool, error)

InspectDeletedFromSource checks whether a message has deleted_from_source_at set.

func (*Store) InspectDisplayName added in v0.4.0 

func (s *Store) InspectDisplayName(sourceMessageID, recipientType, email string) (string, error)

InspectDisplayName returns the display name for a recipient of a message.

func (*Store) InspectMessage added in v0.4.0 

func (s *Store) InspectMessage(sourceMessageID string) (*MessageInspection, error)

InspectMessage retrieves detailed message information for test assertions. This consolidates multiple schema-aware queries into a single method, keeping schema knowledge in the store package.

func (*Store) InspectMessageDates added in v0.4.0 

func (s *Store) InspectMessageDates(sourceMessageID string) (sentAt, internalDate string, err error)

InspectMessageDates returns sent_at and internal_date for a message.

func (*Store) InspectParticipantDisplayName added in v0.6.5 

func (s *Store) InspectParticipantDisplayName(email string) (string, error)

InspectParticipantDisplayName returns the display_name from the participants table for an email.

func (*Store) InspectRawDataExists added in v0.4.0 

func (s *Store) InspectRawDataExists(sourceMessageID string) (bool, error)

InspectRawDataExists checks that raw MIME data exists for a message.

func (*Store) InspectRecipientCount added in v0.4.0 

func (s *Store) InspectRecipientCount(sourceMessageID, recipientType string) (int, error)

InspectRecipientCount returns the count of recipients of a given type for a message.

func (*Store) InspectThreadSourceID added in v0.4.0 

func (s *Store) InspectThreadSourceID(sourceMessageID string) (string, error)

InspectThreadSourceID returns the source_conversation_id for a message's thread.

func (*Store) ListMessages added in v0.7.0 

func (s *Store) ListMessages(offset, limit int) ([]APIMessage, int64, error)

ListMessages returns a paginated list of messages with batch-loaded recipients and labels.

func (*Store) ListSources added in v0.3.0 

func (s *Store) ListSources(sourceType string) ([]*Source, error)

ListSources returns all sources, optionally filtered by source type. Pass an empty string to return all sources.

func (*Store) MarkMessageDeleted 

func (s *Store) MarkMessageDeleted(sourceID int64, sourceMessageID string) error

MarkMessageDeleted marks a message as deleted from the source.

func (*Store) MarkMessageDeletedByGmailID 

func (s *Store) MarkMessageDeletedByGmailID(permanent bool, gmailID string) error

MarkMessageDeletedByGmailID marks a message as deleted by its Gmail ID. This is used by the deletion executor which only has the Gmail message ID. The permanent flag indicates whether the message was permanently deleted or just trashed.

func (*Store) MarkMessagesDeletedBatch added in v0.7.0 

func (s *Store) MarkMessagesDeletedBatch(sourceID int64, sourceMessageIDs []string) error

MarkMessagesDeletedBatch marks multiple messages as deleted from the source in a single transaction.

func (*Store) MarkMessagesDeletedByGmailIDBatch added in v0.7.0 

func (s *Store) MarkMessagesDeletedByGmailIDBatch(gmailIDs []string) error

MarkMessagesDeletedByGmailIDBatch marks multiple messages as deleted by their Gmail IDs in batched UPDATE statements. Much faster than individual MarkMessageDeletedByGmailID calls because it issues one UPDATE per chunk instead of one per message.

Uses best-effort semantics: if a chunk fails, it falls back to individual updates for that chunk and continues with remaining chunks. Returns the first error encountered (if any) after processing all IDs.

func (*Store) MessageExistsBatch 

func (s *Store) MessageExistsBatch(sourceID int64, sourceMessageIDs []string) (map[string]int64, error)

MessageExistsBatch checks which message IDs already exist in the database. Returns a map of source_message_id -> internal message_id for existing messages.

func (*Store) NeedsFTSBackfill added in v0.6.4 

func (s *Store) NeedsFTSBackfill() bool

NeedsFTSBackfill reports whether the FTS table needs to be populated. Uses MAX(id) comparisons (instant B-tree lookups) instead of COUNT(*) to avoid full table scans on large databases.

func (*Store) Rebind 

func (s *Store) Rebind(query string) string

Rebind converts a query with ? placeholders to the appropriate format for the current database driver. Currently SQLite-only (no conversion needed). When PostgreSQL support is added, this will convert ? to $1, $2, etc.

func (*Store) RemoveMessageLabels added in v0.7.0 

func (s *Store) RemoveMessageLabels(messageID int64, labelIDs []int64) error

RemoveMessageLabels removes specific labels from a message.

func (*Store) ReplaceMessageLabels 

func (s *Store) ReplaceMessageLabels(messageID int64, labelIDs []int64) error

ReplaceMessageLabels replaces all labels for a message atomically.

func (*Store) ReplaceMessageRecipients 

func (s *Store) ReplaceMessageRecipients(messageID int64, recipientType string, participantIDs []int64, displayNames []string) error

ReplaceMessageRecipients replaces all recipients for a message atomically.

func (*Store) SearchMessages added in v0.7.0 

func (s *Store) SearchMessages(query string, offset, limit int) ([]APIMessage, int64, error)

SearchMessages searches messages using FTS5, with batch-loaded recipients and labels.

func (*Store) StartSync 

func (s *Store) StartSync(sourceID int64, syncType string) (int64, error)

StartSync creates a new sync run record and returns its ID.

func (*Store) UpdateSourceDisplayName added in v0.5.0 

func (s *Store) UpdateSourceDisplayName(sourceID int64, displayName string) error

UpdateSourceDisplayName updates the display name for a source.

func (*Store) UpdateSourceSyncCursor 

func (s *Store) UpdateSourceSyncCursor(sourceID int64, cursor string) error

UpdateSourceSyncCursor updates the sync cursor (historyId) for a source.

func (*Store) UpdateSyncCheckpoint 

func (s *Store) UpdateSyncCheckpoint(syncID int64, cp *Checkpoint) error

UpdateSyncCheckpoint saves progress for resumption.

func (*Store) UpsertAttachment 

func (s *Store) UpsertAttachment(messageID int64, filename, mimeType, storagePath, contentHash string, size int) error

UpsertAttachment stores an attachment record.

func (*Store) UpsertFTS added in v0.6.4 

func (s *Store) UpsertFTS(messageID int64, subject, bodyText, fromAddr, toAddrs, ccAddrs string) error

UpsertFTS inserts or replaces an FTS row for a message. No-op if FTS5 is not available.

func (*Store) UpsertMessage 

func (s *Store) UpsertMessage(msg *Message) (int64, error)

UpsertMessage inserts or updates a message.

func (*Store) UpsertMessageBody 

func (s *Store) UpsertMessageBody(messageID int64, bodyText, bodyHTML sql.NullString) error

UpsertMessageBody stores the body text and HTML for a message in the separate message_bodies table.

func (*Store) UpsertMessageRaw 

func (s *Store) UpsertMessageRaw(messageID int64, rawData []byte) error

UpsertMessageRaw stores the compressed raw MIME data for a message.

type SyncRun 

type SyncRun struct {
	ID                int64
	SourceID          int64
	StartedAt         time.Time
	CompletedAt       sql.NullTime
	Status            string // SyncStatusRunning, SyncStatusCompleted, SyncStatusFailed
	MessagesProcessed int64
	MessagesAdded     int64
	MessagesUpdated   int64
	ErrorsCount       int64
	ErrorMessage      sql.NullString
	CursorBefore      sql.NullString // Page token for resumption
	CursorAfter       sql.NullString // Final history ID
}

SyncRun represents a sync operation in progress or completed.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.