db

package
v0.0.3 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Nov 29, 2025 License: MIT Imports: 12 Imported by: 0

Documentation

Overview

Package db works with the rest of the orm to interface between a database and the ORM abstraction of reading and querying a database. It is architected so that both SQL and NoSQL databases can be used with the abstraction, though currently only SQL databases are supported. This allows you to potentially write code that is completely agnostic to what kind of database you are using. Even if 100% portability is not achievable in your use case, the ORM and database abstraction layers should be able to handle most of your needs, such that if you ever did have to port to a different database, it would minimize the amount of custom code you would need to write.

Generally, SQL and NoSQL databases work very differently. However, many SQL databases have recently added NoSQL capabilities, like storing and searching JSON text. Similarly, NoSQL databases have added features to enable searching a database through relationships, similar to SQL capabilities.

The general approach Goradd takes is to describe data with key/value pairs. This fits in well with SQL, as key/value pairs are just table-column/field pairs. NoSQL works with key-value pairs as well.

Relationships between structures can be, either one-to-one, one-to-many, or many-to-many. By keeping the description at a higher level, we allow databases to implement those relationships in the way that works best.

SQL implements one-to-many relationships using foreign keys. In the data description, you will see a Reference type of Relationship, which points from the many to the one, and a ReverseRelationship, which is a kind of virtual representation of pointing from the one side to the many. ReverseRelationship lists are populated at the time of a query. Many-to-many relationships use an intermediate table, called an Association Table, that has foreign keys pointing in both directions.

The other major difference between SQL and NoSQL databases is the built-in capabilities to do aggregate calculations. In SQL, you generally can create a filtered list of records and ask SQL to sum all the values from a particular field. Some NoSQL databases can do this, and some cannot. The ones that cannot expect the programmer to do their own filtering and summing. GoRADD handles this difference by allowing individual GORADD database drivers to be written that add some aggregate capabilities to a database, and also providing ways for individual developers to simply create their own custom queries that will be non-portable between databases. In any case, there is always a way to do what you want to do, just some databases are easier to work with.

Index

Constants

View Source 
const (
	DriverTypeMysql    = "mysql"
	DriverTypePostgres = "postgres"
	DriverTypeSQLite   = "sqlite"
)

List of supported database drivers

View Source 
const (
	LogDatabase  = "database" // the database key
	LogSql       = "query"
	LogArgs      = "args"
	LogError     = "error"
	LogComponent = "component"
	LogTable     = "table"
	LogColumn    = "column"
	LogFilename  = "filename"
	LogStartTime = "start"
	LogEndTime   = "end"
	LogDuration  = "duration"
)

Logging keys for slog contextual fields

Variables

View Source 
var InstanceId = time.Now().UnixMicro() & int64(^(uint64(recordVersionMask) << 38))

InstanceId is an id used to identify the current instance of the application. It is important when multiple instances of the same application that uses the ORM are accessing the same database.

The value is assigned automatically as the microsecond that the application starts at, in an 8-year interval. If your application is running behind a load balancer that can assign instance ids that are unique and sequential, you might consider setting InstanceId to that value.

View Source 
var IntPrimaryKeyFunc func() int64
View Source 
var RecordVersionFunc func(prev int64) int64

RecordVersionFunc is a function that will return a new record version value given a previous value. Only set this one if the default behavior does not work for you. See RecordVersion.

Functions

func AddDatabase 

func AddDatabase(d DatabaseI, key string)

AddDatabase adds a database to the global database store. Only call this during app startup.

func Associate 

func Associate[J, K any](ctx context.Context,
	d DatabaseI,
	assnTable string,
	srcColumnName string,
	pk J,
	relatedColumnName string,
	relatedPk K) error

Associate adds a record to the assnTable table.

func AssociateOnly 

func AssociateOnly[J, K any](ctx context.Context,
	d DatabaseI,
	assnTable string,
	srcColumnName string,
	pk J,
	relatedColumnName string,
	relatedPks []K) error

AssociateOnly resets a many-many relationship in the database. The assnTable is the name of the association table that contains the many-many relationships. The srcColumnName is the name of the column that points to the primary key in the source table. The value of that column is pk. The relatedColumnName is the name of the column in the association table that points to the destination table's primary key. with relatedPks having all the primary keys of objects that should be associated with the object with primary key pk. All previous associations with the source object are deleted.

func DatabaseIter 

func DatabaseIter() iter.Seq2[string, DatabaseI]

DatabaseIter returns an iterator over the databases in key order.

func NewIntPrimaryKey 

func NewIntPrimaryKey() int64

NewIntPrimaryKey returns a 64-bit generated primary key, similar to a Snowflake key. Such keys are quick to generate, are quick to sort in the database, and provide reasonable assurances of uniqueness even when multiple instances are generatiung keys. However, they are not suitable for external use, since the keys may be easily guessed based on previous keys. The default is based on the current time stamp plus 8 bits of entropy, giving a means to ensure the multiple instances of the app are unlikely to generate the same key. Keys will not repeat in this scenario for 2 years, and even then will be onlikely to collide. If the default doesn't work for you, set the InPrimaryKeyFunc variable to a function that generates keys. There are many Snowflake and Snowflake like libraries that you can use for this purpose.

func NewOptimisticLockError 

func NewOptimisticLockError(table string, pkValue any, err error) error

NewOptimisticLockError returns a new error related to optimistic locking.

Test and get the values using: 

 if myerr, ok := anyutil.As[*OptimisticLockError](err); ok {
	// process error
 }

func NewQueryError 

func NewQueryError(operation, query string, args []any, err error) error

func NewRecordNotFoundError 

func NewRecordNotFoundError(table string, pkValue any) error

NewRecordNotFoundError returns a new error stating that a record was not found. The message should describe the search used that failed.

func NewUniqueValueError 

func NewUniqueValueError(table string, valuesByColumn map[string]any, err error) error

NewUniqueValueError returns a new error stating that a record could not be saved because a unique value or values in the new record conflicted with values in a different record.

func RecordVersion 

func RecordVersion(prev int64) (v int64)

RecordVersion produces an atomically unique record version value that is different from prev. The value returned can be used to determine when a record has changed for optimistic locking. Many database implementations provide a mechanism to do row-level locking, and in those cases a basic incrementer would work. This only needs to be unique within a record and previous versions of that single record.

However, some NoSQL databases (DynamoDB for one) do not have a mechanism to lock rows ahead of changes, but rather expect a condition to be given to the database to check whether a value (like a version number) remains constant throughout the transaction, and will report an error after the transaction attempt has been made. In these situations, it is important that all instances of the application produce unique numbers for version changes.

A suitable default method is used that will guarantee uniqueness provided that:

  1. Instances are restarted at least every 8 years,
  2. Multiple instances are not cold started at the same microsecond, and
  3. Individual instances do not create more than 67 million records before being restarted.

Even if these parameters cannot be guaranteed, it is still extremely unlikely that a collision will occur. You can replace the default method by setting the RecordVersionFunction value.

func WalkCursor 

func WalkCursor[T any](cursor Cursor[T], fn func(index int, item *T) error) (rerr error)

WalkCursor iterates through a cursor, calls the handler for each item, and ensures the cursor is closed. The handler receives both the item and its 0-based index.

func WithConstraintsOff 

func WithConstraintsOff(ctx context.Context, d DatabaseI, f func(ctx context.Context) error) error

WithConstraintsOff turns off constraints for databases that support foreign key constraints. Otherwise, will just call f with ctx.

func WithTransaction 

func WithTransaction(ctx context.Context, d DatabaseI, f func(ctx context.Context) error) error

WithTransaction wraps the function f in a database transaction if the driver supports transactions. Otherwise, just executes f with ctx.

While the ORM by default will wrap individual database calls with a timeout, it will not apply this timeout to a transaction. It is up to you to pass a context that has a timeout to prevent the overall transaction from hanging.

Types

type AutoPrimaryKeyJsonUnmarshaller 

type AutoPrimaryKeyJsonUnmarshaller interface {
	AutoPrimaryKeyJsonUnmarshal(any) AutoPrimaryKey
}

AutoPrimaryKeyJsonUnmarshaller is the interface for database implementations that need to specially handle the process of unmarshalling a json value for an AutoPrimaryKey. For example, MongoDB exports this as a hex string, but cannot import that without a helper.

type Copier 

type Copier interface {
	Copy() interface{}
}

Copier implements the copy interface, that returns a deep copy of an object.

type Cursor 

type Cursor[T any] interface {
	Next() (*T, error)
	Close() error
}

type DatabaseI 

type DatabaseI interface {
	// Update sets specific fields of a single record that exists in the database.
	// optLockFieldName is the name of a version field that will implement an optimistic locking check while doing the update.
	// If optLockFieldName is provided:
	//   - That field will be used to limit the update,
	//   - That field will be updated with a new version and returned in changes.
	//   - If the record was previously deleted or updated, an OptimisticLockError will be returned.
	//     You will need to query further to determine if the record still exists.
	//
	// Otherwise, if optLockFieldName is blank, and the record we are attempting to change does not exist, the database
	// will not be altered, and no error will be returned.
	Update(ctx context.Context, table string, primaryKey map[string]any, changes map[string]any, optLockFieldName string, optLockFieldValue int64) error
	// Insert will insert a new record into the database with the given values.
	// If autoPkKey is specified and it is not present in fields, it will be generated by the database or
	// the driver, and returned in fields.
	// All references to auto primary keys should be marked as references.
	// Make sure fields has all the required values for the record.
	Insert(ctx context.Context, table string, fields map[string]any, autoPkKey string) error
	// Delete will delete a single record from the database.
	// If optLockFieldName is provided, the optLockFieldValue will also constrain Delete, and if no
	// records are found, it will return an OptimisticLockError. If optLockFieldName is empty, and
	// no record is found, a NoRecordFound error will be returned.
	// Care should be exercised when calling this directly, since linked records are not modified in any way.
	// If this record has linked records, the database structure may be corrupted.
	Delete(ctx context.Context, table string, primaryKey map[string]any, optLockFieldName string, optLockFieldValue int64) error
	// DeleteWhere will delete records from table with the criteria where.
	// If where is empty, all records in the table will be deleted.
	// The values in where are initially AND'd. Maps in where will be OR'd, and maps inside OR'd values will be
	// AND'd etc.
	DeleteWhere(ctx context.Context, table string, where map[string]any) error
	// Query executes a simple query on a single table using fields, where the keys of fields are the names of database fields to select,
	// and the values are the types of data to return for each field.
	// If orderBy is not nil, it specifies field names to sort the data on, in ascending order.
	// If the database supports transactions and row locking, and a transaction is active, it will lock the rows read, and
	// depending on the setting in the transaction, it will be either a read or a write lock.
	Query(ctx context.Context, table string, fields map[string]ReceiverType, where map[string]any, orderBy []string) (CursorI, error)
	// BuilderQuery performs a complex query using a query builder.
	// The data returned will depend on the command inside the builder.
	BuilderQuery(ctx context.Context, builder *Builder) (any, error)
}

DatabaseI is the interface that describes the behaviors required for a database implementation.

Time values are converted to whatever time format the database prefers.

JSON values must already be encoded as strings or []byte values.

If where is not nil, it specifies fields and values that will limit the search. Multiple field-value combinations will be Or'd together. If a value is a map[string]any type, its key is ignored, and the keys and values of the enclosed type will be And'd together. This Or-And pattern is recursive. If a value is a slice of int or strings, those values will be put in an "IN" test. For example, {"vals":[]int{1,2,3}} will result in SQL of "vals IN (1,2,3)".

func GetDatabase 

func GetDatabase(key string) DatabaseI

GetDatabase returns the database given the database's key.

type Decoder 

type Decoder interface {
	Decode(v interface{}) error
}

Decoder provides support to the codegenerated structures, allowing the decoder to be mocked.

type Encoder 

type Encoder interface {
	Encode(v interface{}) error
}

Encoder provides support to the codegenerated structures, allowing the encoder to be mocked.

type OptimisticLockError 

type OptimisticLockError struct {
	Table   string
	PkValue any
	Err     error // wrapped error coming from database driver if there is one
}

OptimisticLockError reports errors related to optimistic locking. i.e. when the same record was changed by a different user prior to a save completing.

func (*OptimisticLockError) Error 

func (e *OptimisticLockError) Error() string

func (*OptimisticLockError) Unwrap 

func (e *OptimisticLockError) Unwrap() error

type QueryError 

type QueryError struct {
	// Operation is the call into the database, or database function that returned the error
	Operation string
	// Query is the query that was attempted
	Query string
	// Args are the arguments sent with the query
	Args []any
	// Error is the underlying error returned
	Err error
}

QueryError indicates an error occurred while querying a database. This could mean a syntax error with the query, a problem with the database, a problem with the connection to the database, etc. Unique value collisions will be returned as a UniqueValueError.

func (*QueryError) Error 

func (e *QueryError) Error() string

func (*QueryError) Unwrap 

func (e *QueryError) Unwrap() error

type RecordNotFoundError 

type RecordNotFoundError struct {
	Table   string
	PkValue any
}

RecordNotFoundError indicates a record was expected in the database, but was not found. The record may have been deleted simultaneously by another process.

func (*RecordNotFoundError) Error 

func (e *RecordNotFoundError) Error() string

type SchemaExtractor 

type SchemaExtractor interface {
	ExtractSchema(options map[string]any) schema.Database
}

type SchemaRebuilder 

type SchemaRebuilder interface {
	DestroySchema(ctx context.Context, s schema.Database) error
	CreateSchema(ctx context.Context, s schema.Database) error
}

type UniqueValueError 

type UniqueValueError struct {
	Table          string
	ValuesByColumn map[string]any
	Err            error
}

UniqueValueError indicates a record failed to save because a value in that record has a unique index and the value was found in another record. If the error is generated by the database driver, it will just have a value for Err. If it is detected by the ORM, Table, Columns and Values will be set.

func (*UniqueValueError) Error 

func (e *UniqueValueError) Error() string

func (*UniqueValueError) Unwrap 

func (e *UniqueValueError) Unwrap() error

type ValueMap 

type ValueMap map[string]any

func NewValueMap 

func NewValueMap() ValueMap

func (ValueMap) Copy 

func (m ValueMap) Copy() interface{}

Copy does a deep copy and supports the deep copy interface

Source Files

View all Source files

Directories

Path Synopsis
Package jointree supports the query buildNodeTree process.
 Package jointree supports the query buildNodeTree process.
sql
Package sql contains helper functions that connect a standard Go database/sql object to the GoRADD system.
 Package sql contains helper functions that connect a standard Go database/sql object to the GoRADD system.
mysql
pgsql
sqlite

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.