search

gFly Search

Search module supports full-text search with relevance ranking, semantic search using embeddings, and optional Database and ElasticSearch integration.

Copyright © 2023, gFly
https://www.gFly.dev
All rights reserved.

Usage

Install

go get -u github.com/gflydev/search@v1.0.0

---

Table of Contents

1. Overview
2. Package Structure
3. Core Concepts
   • Driver
   • Searchable Interface
   • Engine
   • Builder
   • Result & Hit
4. Drivers
   • DatabaseDriver
   • ElasticsearchDriver
5. Making a Model Searchable
6. Searching
   • Basic Search
   • Filtering
   • Sorting
   • Pagination
   • Count Only
   • Ad-hoc Search Without Searchable
7. Indexing (Elasticsearch)
   • Index a Single Model
   • Remove a Model
   • Bulk Index
8. Hydrating Results
9. Complete Example
10. Best Practices

---

Overview

Search Query
    ↓
Engine.For(model) / Engine.Index(name, fields...)
    ↓
Builder  →  .Query()  .Where()  .OrderBy()  .Page()  .PerPage()
    ↓
Driver.Search(Request)
    ↓
Result { Total, Hits[]{ID, Score, Data} }
    ↓
Hydrate IDs → []Model  (fetch from DB via mb.GetModelByID)

Two drivers are available out of the box:

Driver	Backend	Indexing needed?	Fuzzy search
DatabaseDriver	PostgreSQL ILIKE	No	No
ElasticsearchDriver	Elasticsearch 7.x/8.x	Yes	Yes

---

Package Structure

├── driver.go               → Driver interface, Filter, Order, Document types
├── request.go              → Request struct
├── result.go               → Result, Hit types
├── searchable.go           → Searchable interface
├── engine.go               → Engine (entry point)
├── builder.go              → Builder (fluent query API)
├── database_driver.go      → DatabaseDriver implementation
└── elasticsearch_driver.go → ElasticsearchDriver implementation

---

Core Concepts

Driver

Driver is the interface every search backend must implement:

type Driver interface {
    Search(req Request) (*Result, error)
    Index(indexName string, id any, data core.Data) error
    Remove(indexName string, id any) error
    BulkIndex(indexName string, docs []Document) error
}

Index, Remove, and BulkIndex are no-ops for the DatabaseDriver — the database is already the source of truth.

---

Searchable Interface

Any model that should be searchable must implement the four-method Searchable interface:

type Searchable interface {
    SearchIndex() string            // table name (DB) or index name (ES)
    SearchKey() any                 // primary key
    SearchableFields() []string     // columns for ILIKE (DB driver)
    ToSearchDocument() core.Data   // document written to ES
}

---

Engine

Engine is the main entry point. Create one per driver and reuse it:

engine := search.New(search.NewDatabaseDriver())

Method	Description
engine.For(model Searchable)	Returns a Builder pre-configured from the model
engine.Index(name, fields...)	Returns a Builder with explicit index and field list
engine.IndexModel(model)	Indexes a single model (ES)
engine.RemoveModel(model)	Removes a model from the index (ES)
engine.BulkIndex(models)	Bulk-indexes a slice of models (ES)

---

Builder

Builder provides the fluent query API:

engine.For(models.User{}).
    Query("alice").
    Where("users.status", "active").
    OrderBy("users.fullname", search.Asc).
    Page(1).PerPage(20).
    Search()

Method	Description
.Query(q string)	Full-text search string
.Where(field, value)	Equality filter (AND-ed, multiple allowed)
.OrderBy(field, direction)	Sort by field (search.Asc / search.Desc)
.Page(n int)	1-based page number (default: 1)
.PerPage(n int)	Results per page (default: 15)
.Search()	Execute and return *Result
.Paginate()	Alias for .Search()
.Count()	Return total matching count only

---

Result & Hit

type Result struct {
    Total   int64   // total matching documents (before pagination)
    Page    int
    PerPage int
    Hits    []Hit
}

type Hit struct {
    ID    any        // primary key
    Score float64    // 1.0 for DB driver, ES _score for ES driver
    Data  core.Data // ES _source document (nil for DB driver)
}

Convenience helpers on *Result:

result.IDs()     // []any
result.IntIDs()  // []int  (skips non-int IDs)

---

Drivers

DatabaseDriver

Uses PostgreSQL ILIKE (case-insensitive) queries. No additional infrastructure is required.

driver := search.NewDatabaseDriver()
engine := search.New(driver)

Options:

driver := &search.DatabaseDriver{
    CaseSensitive: true,  // use LIKE instead of ILIKE (default: false)
    IDField:       "id",  // primary key column name (default: "id")
}

How it works:

For a Query("alice") across ["users.fullname", "users.email"] the driver executes:

-- Count
SELECT COUNT(*) FROM users
WHERE (users.fullname ILIKE '%alice%' OR users.email ILIKE '%alice%')

-- Paginated IDs
SELECT users.id FROM users
WHERE (users.fullname ILIKE '%alice%' OR users.email ILIKE '%alice%')
ORDER BY users.fullname ASC
LIMIT 20 OFFSET 0

---

ElasticsearchDriver

Uses the Elasticsearch REST API via Go's standard net/http — no extra dependency required.

driver := search.NewElasticsearchDriver(search.ElasticsearchConfig{
    Host:     "http://localhost:9200",  // required
    Username: "",                       // optional (HTTP Basic Auth)
    Password: "",
    Timeout:  10 * time.Second,        // optional (default: 10 s)
})
engine := search.New(driver)

Docker Compose (already configured):

es:
  image: elasticsearch:7.17.28
  ports:
    - "${CONTAINER_ES_PORT:-9200}:9200"

Start with: make es.run

How it works:

A Query("alice") across ["fullname", "email"] with Where("status", "active") generates:

{
  "query": {
    "bool": {
      "must": [{
        "multi_match": {
          "query": "alice",
          "fields": ["fullname", "email"],
          "type": "best_fields",
          "fuzziness": "AUTO"
        }
      }],
      "filter": [{ "term": { "status": "active" } }]
    }
  },
  "from": 0,
  "size": 20
}

The driver returns ES _id and _score in each Hit. The _source document is available via hit.Data.

---

Making a Model Searchable

Add a dedicated file inside the model's package (e.g. user_searchable.go):

package models

import "github.com/gflydev/core"

func (u User) SearchIndex() string { return TableUser }
func (u User) SearchKey() any      { return u.ID }

func (u User) SearchableFields() []string {
    return []string{
        "fullname",
        "email",
        "phone",
    }
}

func (u User) ToSearchDocument() core.Data {
    return core.Data{
        "id":       u.ID,
        "fullname": u.Fullname,
        "email":    u.Email,
        "phone":    u.Phone,
        "status":   string(u.Status),
    }
}

Note: SearchableFields should use qualified column names (table.column) when the DatabaseDriver query involves JOINs to avoid ambiguity. For the ElasticsearchDriver the field names are the plain document field names stored in the index.

---

Searching

Basic Search

engine := search.New(search.NewDatabaseDriver())

result, err := engine.For(models.User{}).
    Query("alice").
    Search()

// result.Total → total matching records
// result.Hits  → []Hit with ID and Score

Filtering

Additional Where calls are AND-ed together:

result, err := engine.For(models.User{}).
    Query("alice").
    Where("users.status", "active").
    Where("users.deleted_at", nil).
    Search()

For the DatabaseDriver, filter field names must be qualified (table.column). For the ElasticsearchDriver, filter field names match the document field names in the index.

Sorting

result, err := engine.For(models.User{}).
    Query("alice").
    OrderBy("users.created_at", search.Desc).
    Search()

Constants: search.Asc, search.Desc

Pagination

result, err := engine.For(models.User{}).
    Query("alice").
    Page(2).PerPage(15).
    Paginate()

// result.Total   → total count across all pages
// result.Page    → 2
// result.PerPage → 15
// result.Hits    → up to 15 hits for page 2

Count Only

total, err := engine.For(models.User{}).
    Query("alice").
    Where("users.status", "active").
    Count()

Ad-hoc Search Without Searchable

When you do not want to implement the full Searchable interface, use engine.Index() directly:

result, err := engine.
    Index("users", "users.fullname", "users.email").
    Query("alice").
    Where("users.status", "active").
    PerPage(10).
    Search()

---

Indexing (Elasticsearch)

The DatabaseDriver does not require indexing — skip this section if you only use PostgreSQL search.

Index a Single Model

Call after creating or updating a model:

esEngine := search.New(search.NewElasticsearchDriver(cfg))

// After create
user, err := services.CreateUser(dto)
esEngine.IndexModel(user)

// After update
user, err := services.UpdateUser(dto)
esEngine.IndexModel(*user)

Remove a Model

Call before or after deleting a model:

esEngine.RemoveModel(user)
services.DeleteUserByID(user.ID)

Bulk Index

Use for initial import or scheduled full re-sync:

var users []models.User
// ... fetch all users from DB ...

searchables := make([]search.Searchable, len(users))
for i, u := range users {
    searchables[i] = u
}

err := esEngine.BulkIndex(searchables)

---

Hydrating Results

Both drivers return Hit.ID values (primary keys). Fetch full model structs from the database:

result, err := engine.For(models.User{}).Query("alice").Search()
if err != nil {
    return err
}

users := make([]models.User, 0, len(result.Hits))
for _, id := range result.IntIDs() {
    user, err := mb.GetModelByID[models.User](id)
    if err != nil {
        continue // skip missing records
    }
    users = append(users, *user)
}

Elasticsearch only: hit.Data contains the raw _source document already available in the Result. If the indexed document contains all required fields you can use hit.Data directly and skip the database round-trip.

---

Complete Example

1. Implement Searchable (internal/domain/models/user_searchable.go)

package models

import "github.com/gflydev/core"

func (u User) SearchIndex() string { return TableUser }
func (u User) SearchKey() any      { return u.ID }

func (u User) SearchableFields() []string {
    return []string{
        "fullname",
        "email",
        "phone",
    }
}

func (u User) ToSearchDocument() core.Data {
    return core.Data{
        "id":       u.ID,
        "fullname": u.Fullname,
        "email":    u.Email,
        "phone":    u.Phone,
        "status":   string(u.Status),
    }
}

2. Create engines (internal/services/search_engines.go)

package services

import "github.com/gflydev/search"

var DBSearchEngine = search.New(search.NewDatabaseDriver())

var ESSearchEngine = search.New(search.NewElasticsearchDriver(
    search.ElasticsearchConfig{Host: "http://localhost:9200"},
))

3. Use in a service (internal/services/user_services.go)

func SearchUsers(keyword, status string, page, perPage int) ([]models.User, int64, error) {
    builder := DBSearchEngine.For(models.User{}).
        Query(keyword).
        Page(page).PerPage(perPage)

    if status != "" {
        builder = builder.Where("status", status)
    }

    result, err := builder.Search()
    if err != nil {
        return nil, 0, err
    }

    users := make([]models.User, 0, len(result.Hits))
    for _, id := range result.IntIDs() {
        user, err := mb.GetModelByID[models.User](id)
        if err != nil {
            continue
        }
        users = append(users, *user)
    }

    return users, result.Total, nil
}

4. Use in a controller (internal/http/controllers/api/user/)

func (h *SearchUsersApi) Handle(c *core.Ctx) error {
    keyword := c.QueryParam("q")
    status  := c.QueryParam("status")
    page, _ := strconv.Atoi(c.QueryParam("page"))
    if page < 1 {
        page = 1
    }

    users, total, err := services.SearchUsers(keyword, status, page, 20)
    if err != nil {
        return c.Error(http.Error{Code: "SEARCH_ERROR", Message: err.Error()})
    }

    data := http.ToListResponse(users, transformers.ToUserResponse)
    return c.Success(http.List[response.User]{
        Meta: http.Meta{Page: page, PerPage: 20, Total: int(total)},
        Data: data,
    })
}

---

Best Practices

Topic	Recommendation
Driver selection	Use DatabaseDriver for simple keyword search and small datasets. Switch to ElasticsearchDriver when you need fuzzy matching, relevance scoring, or high query throughput.
Field qualification	Always prefix DatabaseDriver field names with the table name (users.email) when the query involves JOINs to prevent ambiguous column errors.
ES indexing	Call engine.IndexModel inside the service layer, immediately after a successful create or update, to keep the index in sync.
Bulk re-index	Run engine.BulkIndex as a console command or scheduled job (internal/console/) rather than inline in a request handler.
Hydration	Prefer fetching full models from PostgreSQL using mb.GetModelByID rather than relying on the ES _source document, unless the source contains all required fields and a second DB round-trip is unacceptable.
Engine reuse	Create engine instances once (e.g. package-level vars in a service file) and reuse them — each search.New() call is lightweight but the underlying HTTP client should be shared.
Empty query	An empty Query("") with the DatabaseDriver skips the ILIKE conditions and returns all rows matching only the Where filters, behaving like a normal filtered list.
Sorting with ES	Only sort on keyword or numeric field types in ES. Sorting on text fields requires a .keyword sub-field mapping (field.keyword).