database

package
v1.0.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 12, 2025 License: AGPL-3.0 Imports: 28 Imported by: 0

Documentation

Overview

TODO: handle `modified` time sqlite database management

Copyright (c) 2025 Chakib Ben Ziane <contact@blob42.xyz> and [`gosuki` contributors](https://github.com/blob42/gosuki/graphs/contributors). All rights reserved.

SPDX-License-Identifier: AGPL-3.0-or-later

This file is part of GoSuki.

GoSuki is free software: you can redistribute it and/or modify it under the terms of the GNU Affero General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.

GoSuki is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more details.

You should have received a copy of the GNU Affero General Public License along with gosuki. If not, see <http://www.gnu.org/licenses/>.

Package database provides functionality for managing and synchronizing SQLite databases, specifically for bookmark data. It includes methods for syncing data between databases, caching, and disk persistence.

The package supports:

  • Syncing data from one database to another (upsert or update)
  • Synchronizing in-memory cache databases to disk
  • Copying entire databases
  • Scheduling periodic sync operations
  • Handling SQLite-specific constraints and errors

The `SyncTo` method implements a manual UPSERT operation, which attempts to insert records from a source database to a destination database. If an insertion fails due to a constraint (e.g., duplicate URL), it will attempt to update the existing record.

The `SyncToDisk` method provides a way to sync a database to a specified disk path, using SQLite's backup API for efficient copying.

The `SyncFromDisk` method allows for restoring data from a disk file into a database.

The `CopyTo` method is used to copy an entire database from one location to another.

The `SyncToCache` method is used to sync a database to an in-memory cache, either by copying the entire database or by performing a sync operation if the cache is not empty.

This package also includes a scheduler for debounced sync operations to disk, which prevents excessive disk writes and ensures that syncs happen at regular intervals.

The package uses the `sqlx` package for database operations and `log` for logging.

See the individual function documentation for more details about their usage and behavior.

Index

Constants

View Source 
const (
	CacheName = "memcache"
	//MemcacheFmt = "file:%s?mode=memory&cache=shared"
	//BufferFmt   = "file:%s?mode=memory&cache=shared"
	DBTypeInMemoryDSN = "file:%s?mode=memory&cache=shared"
	DBTypeCacheDSN    = DBTypeInMemoryDSN
)
View Source 
const (
	DBFileName = "gosuki.db"

	DBTypeFileDSN = "file:%s"

	// Opening DBs with this driver allows to track connections
	// This is used to perform sqlite backup
	DriverBackupMode = "sqlite_hook_backup"

	GosukiMainTable = "bookmarks"
)
View Source 
const (
	WhereQueryBookmarks = `
	URL like '%%%s%%' OR metadata like '%%%s%%' OR tags like '%%%s%%'
	`

	WhereQueryBookmarksFuzzy = `
	fuzzy('%s', URL) OR fuzzy('%s', metadata) OR fuzzy('%s', tags)
	`

	WhereQueryBookmarksByTag = `
		(URL LIKE '%%%s%%' OR metadata LIKE '%%%s%%') AND tags LIKE '%%%s%%'
	`
	WhereQueryBookmarksByTagFuzzy = `
		(fuzzy('%s', URL) OR fuzzy('%s', metadata)) AND tags LIKE '%%%s%%'
	`

	QQueryPaginate = ` LIMIT %d OFFSET %d`
)
View Source 
const (
	// metadata: name or title of resource
	// modified: time.Now().Unix()
	//
	// flags: designed to be extended in future using bitwise masks
	// Masks:
	//     0b00000001: set title immutable ((do not change title when updating the bookmarks from the web ))
	QCreateBookmarksTable = `` /* 279-byte string literal not displayed */

)

Database schemas used for the creation of new databases

View Source 
const TagSep = ","

Default separator used to join tags in the DB

Variables

View Source 
var (
	// Global in memory cache of gosuki database
	// Main in memory db, is synced with disc
	// `CacheDB` is a memory replica of disk db
	Cache = &CacheDB{}
)
View Source 
var (

	// Default sqlite3 driver
	DriverDefault = "sqlite3_gosuki"
)
View Source 
var (
	ErrVfsLocked = errors.New("vfs locked")
)

Functions

func CountTotalBookmarks 

func CountTotalBookmarks(ctx context.Context) (uint, error)

func DebugPrintRow 

func DebugPrintRow(rows *sql.Rows)

Print debug a single row (does not run rows.next())

func DebugPrintRows 

func DebugPrintRows(rows *sql.Rows)

Print debug Rows results

func DotxQuery 

func DotxQuery(file string) (*dotsqlx.DotSqlx, error)

Loads a dotsql <file> and, wraps it with dotsqlx

func DotxQueryEmbedFS 

func DotxQueryEmbedFS(fs embed.FS, filename string) (*dotsqlx.DotSqlx, error)

Loads a dotsql from an embedded FS

func GetDBDir 

func GetDBDir() string

Get database directory path

func GetDBFullPath 

func GetDBFullPath() string

func GetDBPath 

func GetDBPath() string

func Init 

func Init()

func InitDiskConn 

func InitDiskConn(dbPath string) error

Initialize the connection to ondisk gosuki db

func LoadBookmarks 

func LoadBookmarks(load loadFunc, modName string) error

internal loading function called by modules

func RegisterSqliteHooks 

func RegisterSqliteHooks()

RegisterSqliteHooks registers a SQLite backup hook with additional connection tracking.

func SQLFuncFoo 

func SQLFuncFoo(in string) string

Testing custom func

func SQLFuzzy 

func SQLFuzzy(test, in string) bool

func ScheduleSyncToDisk 

func ScheduleSyncToDisk()

TODO: add `force` param to force sync

func StartSyncScheduler 

func StartSyncScheduler()

func SyncTreeToBuffer 

func SyncTreeToBuffer(node *Node, buffer *DB)

func SyncURLIndexToBuffer 

func SyncURLIndexToBuffer(urls []string, index Index, buffer *DB)

Types

type Bookmark 

type Bookmark = gosuki.Bookmark

type CacheDB 

type CacheDB struct {
	*DB
}

func GetCacheDB 

func GetCacheDB() *CacheDB

func (*CacheDB) IsInitialized 

func (c *CacheDB) IsInitialized() bool

type DB 

type DB struct {
	Name       string
	Path       string
	Handle     *sqlx.DB
	EngineMode string
	AttachedTo []string
	Type       DBType

	SQLXOpener
	LockChecker
	// contains filtered or unexported fields
}

DB encapsulates an sql.DB struct. All interactions with memory/buffer and disk databases are done through the DB instance. 

var (

	//FIXME: hard coded path
	DefaultDBPath = "~/.local/share/gosuki/"

	// Handle to on-disk gosuki database
	DiskDB *DB
)

func NewBuffer 

func NewBuffer(name string) (*DB, error)

func NewDB 

func NewDB(name string, dbPath string, dbFormat string, opts ...DsnOptions) *DB

dbPath is empty string ("") when using in memory sqlite db Call to Init() required before using

func (*DB) AttachTo 

func (db *DB) AttachTo(attached *DB)

func (*DB) Close 

func (db *DB) Close() error

func (*DB) CopyTo 

func (src *DB) CopyTo(dst *DB)

Copy from src DB to dst DB Source DB os overwritten

func (*DB) CountRows 

func (db *DB) CountRows(table string) int

func (*DB) Init 

func (db *DB) Init() (*DB, error)

We should export Open() in its own method and wrap with interface so we can mock it and test the lock status in Init() Initialize a sqlite database with Gosuki Schema if not already done

func (*DB) InitSchema 

func (db *DB) InitSchema() error

func (*DB) InsertBookmark 

func (db *DB) InsertBookmark(bk *Bookmark)

Inserts a bookmarks to the passed DB In case of conflict follow the default rules which for sqlite is a fail with the error `sqlite3.ErrConstraint`

func (*DB) IsEmpty 

func (db *DB) IsEmpty() (bool, error)

func (*DB) Locked 

func (db *DB) Locked() (bool, error)

func (*DB) PrintBookmarks 

func (db *DB) PrintBookmarks() error

func (*DB) SyncFromDisk 

func (dst *DB) SyncFromDisk(dbpath string) error

func (*DB) SyncTo 

func (src *DB) SyncTo(dst *DB)

Manual UPSERT: For every row in `src` try to insert it into `dst`. if if fails then try to update it. It means `src` is synced to `dst`

func (*DB) SyncToCache 

func (src *DB) SyncToCache() error

func (*DB) SyncToDisk 

func (src *DB) SyncToDisk(dbpath string) error

func (*DB) UpsertBookmark 

func (db *DB) UpsertBookmark(bk *Bookmark) error

Inserts or updates a bookmarks to the passed DB In case of a conflict for a UNIQUE URL constraint, update the existing bookmark TODO: use context

type DBConfig 

type DBConfig struct {
	SyncInterval time.Duration `toml:"sync-interval" mapstructure:"sync-interval"`
	DBPath       string        `toml:"db-path" mapstructure:"db-path"`
}

type DBError 

type DBError struct {
	// Database object where error occured
	DBName string

	// Error that occured
	Err error
}

func DBErr 

func DBErr(dbName string, err error) DBError

func (DBError) Error 

func (e DBError) Error() string

type DBType 

type DBType int
const (
	DBTypeInMemory DBType = iota
	DBTypeRegularFile
)
const (
	DBGosuki DBType = iota
	DBForeign
)

Differentiate between gosukidb.sqlite and other sqlite DBs

type DsnOptions 

type DsnOptions map[string]string

type Index 

type Index = *hashmap.RBTree

This is a RedBlack Tree Hashmap that holds in memory the last state of the bookmark tree.It is used as fast db queries. Each URL holds a pointer to a node in [nodeTree]

type LockChecker 

type LockChecker interface {
	Locked() (bool, error)
}

type Node 

type Node = tree.Node

URLTree Node

type Opener 

type Opener interface {
	Open(driver string, dsn string) error
}

type PaginationParams 

type PaginationParams struct {
	Page int
	Size int
}

func DefaultPagination 

func DefaultPagination() *PaginationParams

type QueryResult 

type QueryResult struct {
	Bookmarks []*gosuki.Bookmark
	Total     uint
}

func BookmarksByTag 

func BookmarksByTag(
	ctx context.Context,
	tag string,
	pagination *PaginationParams,
) (*QueryResult, error)

func ListBookmarks 

func ListBookmarks(
	ctx context.Context,
	pagination *PaginationParams,
) (*QueryResult, error)

func QueryBookmarks 

func QueryBookmarks(
	ctx context.Context,
	query string,
	fuzzy bool,
	pagination *PaginationParams,
) (*QueryResult, error)

func QueryBookmarksByTag 

func QueryBookmarksByTag(
	ctx context.Context,
	query,
	tag string,
	fuzzy bool,
	pagination *PaginationParams,
) (*QueryResult, error)

type RawBookmark 

type RawBookmark struct {
	ID  uint64
	URL string `db:"URL"`

	// Usually used for the bookmark title
	Metadata string

	Tags string
	Desc string

	// Last modified
	Modified uint64

	// kept for buku compat, not used for now
	Flags int

	Module string
}

type RawBookmarks 

type RawBookmarks []*RawBookmark

func (RawBookmarks) AsBookmarks 

func (raws RawBookmarks) AsBookmarks() []*gosuki.Bookmark

type SQLXDBOpener 

type SQLXDBOpener struct {
	// contains filtered or unexported fields
}

func (*SQLXDBOpener) Get 

func (o *SQLXDBOpener) Get() *sqlx.DB

func (*SQLXDBOpener) Open 

func (o *SQLXDBOpener) Open(driver string, dataSourceName string) error

type SQLXOpener 

type SQLXOpener interface {
	Opener
	Get() *sqlx.DB
}

type Tags 

type Tags struct {
	// contains filtered or unexported fields
}

func NewTags 

func NewTags(tags []string, delim string) *Tags

Reads tags from a slice of strings

func TagsFromString 

func TagsFromString(s, delim string) *Tags

Builds a list of tags from a string as a Tags struct. It also removes empty tags

func (*Tags) Add 

func (t *Tags) Add(tag string)

func (*Tags) Extend 

func (t *Tags) Extend(tags []string) *Tags

func (*Tags) Get 

func (t *Tags) Get() []string

Returns the list of tags as slice

func (*Tags) PreSanitize 

func (t *Tags) PreSanitize() *Tags

Sanitize the list of tags before saving them to the DB

func (Tags) String 

func (t Tags) String(wrap bool) string

String representation of the tags. It can wrap the tags with the delim if wrap is true. This is done for compatibility with Buku DB format.

func (Tags) StringWrap 

func (t Tags) StringWrap() string

String representation of the tags. It wraps the tags with the delim.

type VFSLockChecker 

type VFSLockChecker struct {
	// contains filtered or unexported fields
}

func (*VFSLockChecker) Locked 

func (checker *VFSLockChecker) Locked() (bool, error)

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.