recally

package
v0.49.3 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 17, 2026 License: AGPL-3.0 Imports: 15 Imported by: 0

Documentation

Overview

Package recally provides file operations for article storage.

Package recally provides database operations for articles and feeds.

Package recally provides the internal reading assistant library for Stella.

Package recally provides deterministic URL normalization helpers.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func ExtractSlug 

func ExtractSlug(value string) string

ExtractSlug generates a filesystem-safe slug from a title.

func NormalizeURL 

func NormalizeURL(rawURL string) string

NormalizeURL performs deterministic normalization with no network access.

func SplitFrontmatter 

func SplitFrontmatter(content string) (string, string)

func ValidateFeedSubscription added in v0.43.0 

func ValidateFeedSubscription(rawURL string, kind FeedKind) error

ValidateFeedSubscription reports whether rawURL can be subscribed as kind. X/Twitter hosts are only subscribable as profile timelines (/<handle>) — they never serve RSS, so lists, search, individual posts, and bookmarks are rejected regardless of kind. RSS feeds are validated when fetched, so any non-X URL is accepted here.

Types

type Article 

type Article struct {
	ID           string            `json:"id"`
	UserID       string            `json:"user_id"`
	AgentID      *string           `json:"agent_id,omitempty"`
	URL          string            `json:"url"`
	CanonicalURL string            `json:"canonical_url"`
	SourceType   SourceType        `json:"source_type"`
	Title        string            `json:"title"`
	Author       string            `json:"author"`
	Summary      string            `json:"summary"`
	Tags         []string          `json:"tags"`
	Status       ArticleStatus     `json:"status"`
	Starred      bool              `json:"starred"`
	FilePath     string            `json:"file_path"`
	Metadata     map[string]string `json:"metadata"`
	PublishedAt  *time.Time        `json:"published_at,omitempty"`
	SavedAt      time.Time         `json:"saved_at"`
	ReadAt       *time.Time        `json:"read_at,omitempty"`
	CreatedAt    time.Time         `json:"created_at"`
	UpdatedAt    time.Time         `json:"updated_at"`
	IsNew        bool              `json:"is_new"`
}

Article represents a saved article with its metadata.

func (*Article) FromSQLCArticle 

func (a *Article) FromSQLCArticle(sa sqlc.RecallyArticle)

FromSQLCArticle populates an Article from a sqlc Article.

type ArticleFilter 

type ArticleFilter struct {
	Status     ArticleStatus
	SourceType SourceType
	Starred    *bool
	Limit      int
	Offset     int
}

ArticleFilter provides filtering options for listing articles.

type ArticleStatus 

type ArticleStatus string

ArticleStatus represents the reading status of an article. 

const (
	StatusUnread   ArticleStatus = "unread"
	StatusRead     ArticleStatus = "read"
	StatusArchived ArticleStatus = "archived"
)

func (ArticleStatus) Valid added in v0.38.0 

func (s ArticleStatus) Valid() bool

Valid reports whether s is a known article status.

type Digest 

type Digest struct {
	UserID               string     `json:"user_id"`
	Date                 time.Time  `json:"date"`
	SavedYesterday       []Article  `json:"saved_yesterday"`
	SavedYesterdayCount  int        `json:"saved_yesterday_count"`
	UnreadCount          int64      `json:"unread_count"`
	ReadCount            int64      `json:"read_count"`
	ArchivedCount        int64      `json:"archived_count"`
	StarredCount         int64      `json:"starred_count"`
	WorthRevisiting      []Article  `json:"worth_revisiting"`
	WorthRevisitingCount int        `json:"worth_revisiting_count"`
	TopTags              []TagCount `json:"top_tags"`
	TotalArticles        int64      `json:"total_articles"`
}

Digest contains daily reading statistics and summaries.

type DigestSection added in v0.30.0 

type DigestSection = string

DigestSection identifies which section of a digest an article belongs to. 

const (
	DigestSectionSavedYesterday  DigestSection = "saved_yesterday"
	DigestSectionWorthRevisiting DigestSection = "worth_revisiting"
)

type Feed 

type Feed struct {
	ID            string            `json:"id"`
	UserID        string            `json:"user_id"`
	AgentID       *string           `json:"agent_id,omitempty"`
	URL           string            `json:"url"`
	Kind          FeedKind          `json:"kind"`
	Metadata      map[string]string `json:"metadata"`
	Title         string            `json:"title"`
	Description   string            `json:"description"`
	CheckInterval string            `json:"check_interval"`
	LastCheckedAt *time.Time        `json:"last_checked_at,omitempty"`
	LastETag      string            `json:"last_etag"`
	LastModified  string            `json:"last_modified"`
	Enabled       bool              `json:"enabled"`
	CreatedAt     time.Time         `json:"created_at"`
	UpdatedAt     time.Time         `json:"updated_at"`
}

Feed represents an RSS feed subscription.

func (*Feed) FromSQLCFeed 

func (f *Feed) FromSQLCFeed(sf sqlc.RecallyFeed)

FromSQLCFeed populates a Feed from a sqlc RecallyFeed.

type FeedEntry 

type FeedEntry struct {
	ID           string         `json:"id"`
	FeedID       string         `json:"feed_id"`
	GUID         string         `json:"guid"`
	URL          string         `json:"url"`
	Title        string         `json:"title"`
	Status       RSSEntryStatus `json:"status"`
	ArticleID    *string        `json:"article_id,omitempty"`
	Attempts     int            `json:"attempts"`
	ErrorMsg     string         `json:"error_msg"`
	DiscoveredAt time.Time      `json:"discovered_at"`
	ProcessedAt  *time.Time     `json:"processed_at,omitempty"`
}

FeedEntry represents an entry/item from an RSS feed.

func (*FeedEntry) FromSQLCFeedEntry 

func (e *FeedEntry) FromSQLCFeedEntry(se sqlc.RecallyFeedEntry)

FromSQLCFeedEntry populates a FeedEntry from a sqlc RecallyFeedEntry.

type FeedEntryFilter 

type FeedEntryFilter struct {
	Status RSSEntryStatus
	Limit  int
	Offset int
}

FeedEntryFilter provides filtering options for listing feed entries.

type FeedKind added in v0.43.0 

type FeedKind string

FeedKind is the extraction-dispatch hint for a feed subscription. Go stores it and routes scheduling on it, but does not branch business logic on it — per-source item discovery lives in the recally skill. 

const (
	FeedKindRSS     FeedKind = "rss"
	FeedKindTwitter FeedKind = "twitter"
	FeedKindWebsite FeedKind = "website"
)

func SniffFeedKind added in v0.43.0 

func SniffFeedKind(rawURL string) FeedKind

SniffFeedKind infers a feed kind from a subscription URL. x.com / twitter.com hosts map to twitter; everything else defaults to rss. The caller may override the result with an explicit kind. URL shape is enforced separately by ValidateFeedSubscription.

func (FeedKind) Valid added in v0.43.0 

func (k FeedKind) Valid() bool

Valid reports whether k is a known feed kind.

type FeedPollResult 

type FeedPollResult struct {
	Feed       Feed        `json:"feed"`
	NewEntries []FeedEntry `json:"new_entries"`
	Errors     []string    `json:"errors,omitempty"`
}

FeedPollResult contains the result of polling a feed.

type FileManager 

type FileManager struct {
	// contains filtered or unexported fields
}

FileManager handles reading and writing article files to disk.

func NewFileManager 

func NewFileManager(stellaHome string) *FileManager

NewFileManager creates a new FileManager instance.

func (*FileManager) ArticleExists 

func (fm *FileManager) ArticleExists(path string) bool

ArticleExists reports whether an article file exists.

func (*FileManager) ArticlePath 

func (fm *FileManager) ArticlePath(userID string, articleID, title string, savedAt time.Time) string

ArticlePath generates the storage path for an article file.

func (*FileManager) DeleteArticle 

func (fm *FileManager) DeleteArticle(path string) error

DeleteArticle removes an article file from disk.

func (*FileManager) EnsureLibrary 

func (fm *FileManager) EnsureLibrary(userID string) error

EnsureLibrary creates the library directory structure if it doesn't exist.

func (*FileManager) ListArticles 

func (fm *FileManager) ListArticles(userID string) ([]string, error)

ListArticles returns all article markdown files in a user's library.

func (*FileManager) ReadArticle 

func (fm *FileManager) ReadArticle(path string) (string, error)

ReadArticle reads an article file and returns the body content without frontmatter.

func (*FileManager) ReadArticleFull 

func (fm *FileManager) ReadArticleFull(path string) (string, error)

ReadArticleFull reads the entire article file including frontmatter.

func (*FileManager) RebuildIndex 

func (fm *FileManager) RebuildIndex(userID string) ([]string, error)

RebuildIndex lists article files for future DB reindexing workflows.

func (*FileManager) RelativePath 

func (fm *FileManager) RelativePath(absolutePath string) string

RelativePath returns a path relative to STELLA_HOME.

func (*FileManager) WriteArticle 

func (fm *FileManager) WriteArticle(path string, article *Article, content string) error

WriteArticle writes an article to disk with YAML frontmatter.

type ListResult 

type ListResult struct {
	Items   []Article `json:"items"`
	Total   int       `json:"total"`
	HasMore bool      `json:"has_more"`
}

ListResult contains pagination info for list operations.

type RSSEntryStatus 

type RSSEntryStatus string

RSSEntryStatus represents the processing status of an RSS feed entry. 

const (
	EntryStatusPending RSSEntryStatus = "pending"
	EntryStatusSaved   RSSEntryStatus = "saved"
	EntryStatusSkipped RSSEntryStatus = "skipped"
	EntryStatusError   RSSEntryStatus = "error"
)

type SaveRequest 

type SaveRequest struct {
	URL          string            `json:"url"`
	CanonicalURL string            `json:"canonical_url,omitempty"`
	SourceType   SourceType        `json:"source_type"`
	Title        string            `json:"title"`
	Author       string            `json:"author"`
	Summary      string            `json:"summary"`
	Tags         []string          `json:"tags"`
	Content      string            `json:"content"`
	Metadata     map[string]string `json:"metadata"`
	PublishedAt  *time.Time        `json:"published_at,omitempty"`
	AgentID      *string           `json:"agent_id,omitempty"`
}

SaveRequest contains all data needed to save an article.

type SearchResult 

type SearchResult struct {
	Article
	Rank float64 `json:"rank,omitempty"`
}

SearchResult contains an article with its search relevance.

type SourceType 

type SourceType string

SourceType represents the type of content source. 

const (
	SourceTypeWeb     SourceType = "web"
	SourceTypeTwitter SourceType = "twitter"
	SourceTypeYouTube SourceType = "youtube"
	SourceTypeGitHub  SourceType = "github"
	SourceTypeRSS     SourceType = "rss"
	SourceTypePDF     SourceType = "pdf"
)

func (SourceType) Valid added in v0.38.0 

func (s SourceType) Valid() bool

Valid reports whether s is a known source type.

type Store 

type Store struct {
	// contains filtered or unexported fields
}

Store provides database operations for the recally package.

func NewStore 

func NewStore(db *sql.DB) *Store

NewStore creates a new Store instance.

func (*Store) CreateFeed 

func (s *Store) CreateFeed(ctx context.Context, userID string, feedURL string, kind FeedKind, metadata map[string]string, title, description string, agentID *string) (*Feed, error)

CreateFeed creates a new feed subscription. kind is the extraction-dispatch hint ("rss", "twitter"); metadata carries source-specific bookkeeping such as a stable numeric user id.

func (*Store) CreateFeedEntry 

func (s *Store) CreateFeedEntry(ctx context.Context, feedID, guid, entryURL, title string) (*FeedEntry, error)

CreateFeedEntry creates a new feed entry. Returns nil, nil when the entry already exists (ON CONFLICT DO NOTHING).

func (*Store) DeleteArticle 

func (s *Store) DeleteArticle(ctx context.Context, userID string, articleID string) error

DeleteArticle removes an article from the database.

func (*Store) DeleteFeed 

func (s *Store) DeleteFeed(ctx context.Context, userID string, feedID string) error

DeleteFeed removes a feed and all its entries.

func (*Store) GetArticle 

func (s *Store) GetArticle(ctx context.Context, userID string, articleID string) (*Article, error)

GetArticle retrieves an article by ID.

func (*Store) GetArticleByCanonicalURL 

func (s *Store) GetArticleByCanonicalURL(ctx context.Context, userID string, canonicalURL string) (*Article, error)

GetArticleByCanonicalURL retrieves an article by canonical URL for a user.

func (*Store) GetDigest 

func (s *Store) GetDigest(ctx context.Context, userID string) (*Digest, error)

GetDigest generates a daily reading digest for a user.

func (*Store) GetFeed 

func (s *Store) GetFeed(ctx context.Context, userID string, feedID string) (*Feed, error)

GetFeed retrieves a feed by ID.

func (*Store) GetFeedByURL 

func (s *Store) GetFeedByURL(ctx context.Context, userID string, feedURL string) (*Feed, error)

GetFeedByURL retrieves a feed by URL for a user.

func (*Store) GetFeedEntry 

func (s *Store) GetFeedEntry(ctx context.Context, feedID string, entryID string) (*FeedEntry, error)

GetFeedEntry retrieves a feed entry by ID.

func (*Store) GetStoredDigestByDate added in v0.30.0 

func (s *Store) GetStoredDigestByDate(ctx context.Context, userID string, date string) (*StoredDigest, error)

GetStoredDigestByDate returns a persisted digest with full article objects.

func (*Store) ListArticles 

func (s *Store) ListArticles(ctx context.Context, userID string, filter ArticleFilter) ([]Article, error)

ListArticles lists articles for a user with optional filtering.

func (*Store) ListFeeds 

func (s *Store) ListFeeds(ctx context.Context, userID string, limit, offset int) ([]Feed, error)

ListFeeds lists all feeds for a user.

func (*Store) ListPendingFeedEntries 

func (s *Store) ListPendingFeedEntries(ctx context.Context, feedID string, limit, offset int) ([]FeedEntry, error)

ListPendingFeedEntries lists pending entries for processing.

func (*Store) ListStoredDigests added in v0.30.0 

func (s *Store) ListStoredDigests(ctx context.Context, userID string, limit, offset int64) ([]StoredDigestSummary, int64, error)

ListStoredDigests returns a paginated list of lightweight digest summaries.

func (*Store) MarkFeedEntry 

func (s *Store) MarkFeedEntry(ctx context.Context, feedID string, entryID string, status RSSEntryStatus, articleID *string, errorMsg string) (*FeedEntry, error)

MarkFeedEntry updates the status of a feed entry after processing.

func (*Store) SaveArticle 

func (s *Store) SaveArticle(ctx context.Context, userID string, req SaveRequest) (*Article, bool, error)

SaveArticle saves or updates an article by canonical URL.

func (*Store) SaveDigest added in v0.30.0 

func (s *Store) SaveDigest(ctx context.Context, userID string, narrative, date string) (*StoredDigest, error)

SaveDigest persists a daily digest snapshot. If one already exists for that date it is replaced. Counts are snapshotted from the live digest.

func (*Store) SearchArticles 

func (s *Store) SearchArticles(ctx context.Context, userID string, query string, limit int) ([]Article, error)

SearchArticles searches articles by title, summary, tags, or author using the FTS5 index, ordered by BM25 relevance (title hits rank highest). Queries with no token of 3+ runes use an unranked LIKE fallback instead.

func (*Store) UpdateArticle 

func (s *Store) UpdateArticle(ctx context.Context, userID string, articleID string, updates map[string]any) (*Article, error)

UpdateArticle updates article metadata.

func (*Store) UpdateArticleFilePath 

func (s *Store) UpdateArticleFilePath(ctx context.Context, userID, articleID, filePath string) error

UpdateArticleFilePath updates only the file path of an article.

func (*Store) UpdateFeed 

func (s *Store) UpdateFeed(ctx context.Context, userID string, feedID string, updates map[string]any) (*Feed, error)

UpdateFeed updates feed metadata.

type StoredDigest added in v0.30.0 

type StoredDigest struct {
	ID                   string     `json:"id"`
	UserID               string     `json:"user_id"`
	Date                 string     `json:"date"`
	Narrative            string     `json:"narrative"`
	SavedYesterday       []Article  `json:"saved_yesterday"`
	SavedYesterdayCount  int64      `json:"saved_yesterday_count"`
	UnreadCount          int64      `json:"unread_count"`
	ReadCount            int64      `json:"read_count"`
	ArchivedCount        int64      `json:"archived_count"`
	StarredCount         int64      `json:"starred_count"`
	WorthRevisiting      []Article  `json:"worth_revisiting"`
	WorthRevisitingCount int64      `json:"worth_revisiting_count"`
	TopTags              []TagCount `json:"top_tags"`
	TotalArticles        int64      `json:"total_articles"`
	CreatedAt            time.Time  `json:"created_at"`
	UpdatedAt            time.Time  `json:"updated_at"`
}

StoredDigest is a persisted daily digest snapshot with full article objects.

type StoredDigestSummary added in v0.30.0 

type StoredDigestSummary struct {
	ID                   string    `json:"id"`
	Date                 string    `json:"date"`
	Narrative            string    `json:"narrative"`
	SavedYesterdayCount  int64     `json:"saved_yesterday_count"`
	WorthRevisitingCount int64     `json:"worth_revisiting_count"`
	TotalArticles        int64     `json:"total_articles"`
	CreatedAt            time.Time `json:"created_at"`
}

StoredDigestSummary is a lightweight digest record for list views.

type TagCount 

type TagCount struct {
	Tag   string `json:"tag"`
	Count int64  `json:"count"`
}

TagCount represents a tag with its frequency.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.