modware-stock

modware-stock

gRPC microservice for managing biological stocks (strains and plasmids) in dictyBase. Backed by ArangoDB.

On every create, update, or delete mutation the service publishes a protobuf-serialized event to NATS so that downstream services can react without polling.

• Running the Server
  • Core Flags
  • ArangoDB Collection Flags
  • Ontology Flags
• Stock IDs
• NATS Events
• gRPC API
  • Service Methods
  • Connect with Go
  • Create a Strain
  • Get a Strain
  • Stock Autocomplete
  • Stock Full Search
    • How the Full Search AQL Statement Is Built
  • Count Stocks
  • Update a Strain
  • Update a Plasmid
  • List Strains with Pagination and Filters
  • Filtering
  • Batch Lookup by IDs
  • Create a Plasmid
  • Delete a Stock
  • Load with Existing ID
• ArangoDB Schema

Running the Server

modware-stock start-server \
  --port 9560 \
  --arangodb-database dictybase \
  --arangodb-user root \
  --arangodb-pass secret \
  --arangodb-host arangodb \
  --arangodb-port 8529 \
  --nats-host nats \
  --nats-port 4222

Core Flags

Global flags: --log-format (json/text, default json), --log-level (debug/warn/error/fatal/panic, default error).

ArangoDB Collection Flags

These flags control the names of collections and graphs created in ArangoDB. Defaults match the dictyBase production configuration.

Ontology Flags

These flags configure which ontologies are used to classify and tag stocks. The referenced ontologies must be loaded into ArangoDB before the service starts (see OboJSONFileUpload).

Stock IDs

Auto-generated IDs follow the pattern DB(S|P)[0-9]{5,}:

• Strains: DBS0370001, DBS0370002, … (prefix DBS)
• Plasmids: DBP0370001, DBP0370002, … (prefix DBP)

The numeric suffix is generated by ArangoDB's autoincrement key generator, seeded by --keyoffset (default 370000). IDs assigned by LoadStrain/LoadPlasmid must match the same regex; the service validates them before persisting.

NATS Events

The service publishes events after every mutating operation. Payloads are protobuf-serialized Strain or Plasmid messages.

Deserialize using proto.Unmarshal with the generated types from github.com/dictyBase/go-genproto/dictybaseapis/stock.

gRPC API

Full protobuf definitions: dictybaseapis/stock.proto.

Service Methods

Connect with Go

import (
    "context"
    "log"

    "github.com/dictyBase/go-genproto/dictybaseapis/stock"
    "google.golang.org/grpc"
    "google.golang.org/grpc/credentials/insecure"
)

func main() {
    conn, err := grpc.NewClient(
        "localhost:9560",
        grpc.WithTransportCredentials(insecure.NewCredentials()),
    )
    if err != nil {
        log.Fatal(err)
    }
    defer conn.Close()

    client := stock.NewStockServiceClient(conn)
    ctx := context.Background()
    _ = client // use client methods below
}

Create a Strain

created_by, updated_by, depositor, label, and species are required. All other fields are optional.

resp, err := client.CreateStrain(ctx, &stock.NewStrain{
    Data: &stock.NewStrain_Data{
        Type: "strain",
        Attributes: &stock.NewStrainAttributes{
            CreatedBy: "user@dictybase.org",
            UpdatedBy: "user@dictybase.org",
            Depositor: "Jane Doe",
            Label:     "DBS0123456",
            Species:   "Dictyostelium discoideum",
            Genes:     []string{"gene1"},
            Names:     []string{"myStrain"},
        },
    },
})
// resp.Data.Id — auto-generated ID (e.g. "DBS0370001")
// resp.Data.Attributes.DictyStrainProperty — defaults to the server's --strain-term value if omitted

Get a Strain

The GetStrain method returns the strain with the given stock ID.

resp, err := client.GetStrain(ctx, &stock.StockId{Id: "DBS0350123"})

Stock Autocomplete

The AutocompleteStock method gives type-ahead suggestions for a query of 3 characters or more, with stock ID, kind, matched field, display text and score (default 5, maximum 50).

resp, err := client.AutocompleteStock(ctx, &stock.StockAutocompleteParameters{
    Data: &stock.StockAutocompleteParameters_Data{
        Type: "stock",
        Attributes: &stock.StockAutocompleteAttributes{
            Query: "DBS03",
        },
    },
})
// resp.Data — slice of suggestions (id, entity, field, display_text, score)
// resp.Meta.Total — number of suggestions in this response

// Restrict the suggestions to strains and request more results
resp, err := client.AutocompleteStock(ctx, &stock.StockAutocompleteParameters{
    Data: &stock.StockAutocompleteParameters_Data{
        Type: "stock",
        Attributes: &stock.StockAutocompleteAttributes{
            Query:  "sadA",
            Limit:  20,
            Entity: stock.StockEntity_STOCK_ENTITY_STRAIN,
        },
    },
})

Stock Full Search

The SearchStock method gives a ranked list of stocks for a query of 2 characters or more, with stock ID, kind, matched field, display text, score, summary, stock_label, and stock_type (maximum 50, no pagination).

resp, err := client.SearchStock(ctx, &stock.StockSearchParameters{
    Data: &stock.StockSearchParameters_Data{
        Type: "stock",
        Attributes: &stock.StockSearchAttributes{
            Query: "actin binding",
        },
    },
})
// resp.Data — ranked slice of results
// Each result has: id, entity, field, display_text, score, summary, stock_label, stock_type
// resp.Meta.Total — number of results in this response

// Restrict the results to plasmids and request fewer results
resp, err := client.SearchStock(ctx, &stock.StockSearchParameters{
    Data: &stock.StockSearchParameters_Data{
        Type: "stock",
        Attributes: &stock.StockSearchAttributes{
            Query:  "pDM",
            Limit:  10,
            Entity: stock.StockEntity_STOCK_ENTITY_PLASMID,
        },
    },
})

The server runs one generated AQL statement per search: 19 LET search branches over the stock_full_search view, plus a merge block that keeps the best-scoring hit per stock. How the Full Search AQL Statement Is Built walks through that statement step by step, from the field definitions to the rendered AQL.

Count Stocks

The CountStocks method returns the total number of strains and plasmids, with the count of each subtype group. Strain groups follow the searchStocks mapping: gwdi (REMI-seq), bacterial (bacterial strain), and regular (every other strain, including strains without an ontology term). Plasmid groups are goldenbraid (GB vector) and regular. Both entity records always list all of their subtype groups, with 0 when a group has no stocks.

resp, err := client.CountStocks(ctx, &emptypb.Empty{})
// resp.Strains.Total — total number of strains
// resp.Strains.Subtypes — [{regular, n}, {gwdi, n}, {bacterial, n}]
// resp.Plasmids.Total — total number of plasmids
// resp.Plasmids.Subtypes — [{regular, n}, {goldenbraid, n}]

The same RPC is also available as an HTTP endpoint, served by grpc-gateway on the same --port as the gRPC server:

curl http://localhost:9560/v1/stocks/counts
# {"strains":{"total":"12","subtypes":[{"name":"regular","count":"5"},{"name":"gwdi","count":"7"},{"name":"bacterial","count":"0"}]},"plasmids":{"total":"4","subtypes":[{"name":"regular","count":"1"},{"name":"goldenbraid","count":"3"}]}}

Update a Strain

Only updated_by is required. All other attributes are optional; unset fields are left unchanged.

resp, err := client.UpdateStrain(ctx, &stock.StrainUpdate{
    Data: &stock.StrainUpdate_Data{
        Type: "strain",
        Id:   "DBS0350123",
        Attributes: &stock.StrainUpdateAttributes{
            UpdatedBy:           "user@dictybase.org",
            Summary:             "Updated description",
            DictyStrainProperty: "REMI-seq",
        },
    },
})

Update a Plasmid

Only updated_by is required. All other attributes are optional.

resp, err := client.UpdatePlasmid(ctx, &stock.PlasmidUpdate{
    Data: &stock.PlasmidUpdate_Data{
        Type: "plasmid",
        Id:   "DBP0350456",
        Attributes: &stock.PlasmidUpdateAttributes{
            UpdatedBy: "user@dictybase.org",
            Summary:   "Updated plasmid description",
            Sequence:  "ATCG...",
        },
    },
})

List Strains with Pagination and Filters

ListStrains and ListPlasmids use cursor-based pagination. The default page size is 10.

resp, err := client.ListStrains(ctx, &stock.StockParameters{
    Limit:  10,
    Cursor: 0,
    Filter: "depositor===Jane Doe",
})
// resp.Data           — slice of strains for this page
// resp.Meta.NextCursor — pass as Cursor to retrieve the next page (0 means last page)
// resp.Meta.Total      — total number of records matching the filter
// resp.Meta.Limit      — the effective page size

// Fetch the next page
if resp.Meta.NextCursor != 0 {
    next, err := client.ListStrains(ctx, &stock.StockParameters{
        Limit:  10,
        Cursor: resp.Meta.NextCursor,
        Filter: "depositor===Jane Doe",
    })
}

Filtering

ListStrains and ListPlasmids accept a filter string in StockParameters. Filters follow the syntax field operator value and can be combined with boolean connectors.

Boolean Connectors

AND takes precedence over OR. Use ; and , to build compound expressions without parentheses.

String Operators

Date Operators

Date values accept YYYY-MM-DD, YYYY-MM, or YYYY formats.

Array Operators

Used for fields that hold repeated values (e.g. gene, name).

Filterable Fields

Examples

// Strains deposited by a specific user, of a particular species
resp, err := client.ListStrains(ctx, &stock.StockParameters{
    Limit:  10,
    Filter: "depositor===Jane Doe;species===Dictyostelium discoideum",
})

// Strains created after a date
resp, err := client.ListStrains(ctx, &stock.StockParameters{
    Limit:  10,
    Filter: "created_at$>=2022-01-01",
})

// Strains associated with a gene (array field)
resp, err := client.ListStrains(ctx, &stock.StockParameters{
    Limit:  10,
    Filter: "gene@==act15",
})

// Plasmids whose name contains "pDM"
resp, err := client.ListPlasmids(ctx, &stock.StockParameters{
    Limit:  10,
    Filter: "plasmid_name=~pDM",
})

// Strains tagged with a specific ontology term
resp, err := client.ListStrains(ctx, &stock.StockParameters{
    Limit:  10,
    Filter: "tag===REMI-seq",
})

Batch Lookup by IDs

ListStrainsByIds returns strains for a list of IDs in a single call, without pagination metadata. IDs must match DB(S|P)[0-9]{5,}.

resp, err := client.ListStrainsByIds(ctx, &stock.StockIdList{
    Id: []string{"DBS0350123", "DBS0350456"},
})

Create a Plasmid

created_by, updated_by, depositor, and name are required.

resp, err := client.CreatePlasmid(ctx, &stock.NewPlasmid{
    Data: &stock.NewPlasmid_Data{
        Type: "plasmid",
        Attributes: &stock.NewPlasmidAttributes{
            CreatedBy: "user@dictybase.org",
            UpdatedBy: "user@dictybase.org",
            Depositor: "Jane Doe",
            Name:      "pDM304",
            Genes:     []string{"act15"},
            Sequence:  "ATCG...",
        },
    },
})

Delete a Stock

RemoveStock accepts both strain and plasmid IDs. The operation is idempotent.

_, err := client.RemoveStock(ctx, &stock.StockId{Id: "DBS0350123"})

Load with Existing ID

LoadStrain and LoadPlasmid are intended for data migration. Unlike Create*, they accept a pre-assigned ID and explicit created_at/updated_at timestamps, preserving historical records without re-sequencing IDs.

resp, err := client.LoadStrain(ctx, &stock.ExistingStrain{
    Data: &stock.ExistingStrain_Data{
        Type: "strain",
        Id:   "DBS0350000",
        Attributes: &stock.ExistingStrainAttributes{
            CreatedBy: "import@dictybase.org",
            UpdatedBy: "import@dictybase.org",
            CreatedAt: timestamppb.New(time.Date(2020, 1, 1, 0, 0, 0, 0, time.UTC)),
            UpdatedAt: timestamppb.New(time.Date(2020, 1, 1, 0, 0, 0, 0, time.UTC)),
            Label:     "DBS0350000",
            Species:   "Dictyostelium discoideum",
            Depositor: "Legacy Import",
        },
    },
})

ArangoDB Schema

erDiagram
    stock_key_generator ||--o| stock : "auto-generates ID"

    stock ||--o| stockprop : "stock_type edge"
    stock ||--o| stock : "parent_strain edge"
    stock ||--o{ cvterm : "stock_term edge"

    cv ||--o{ cvterm : "contains"

    stock {
        string _key PK
        string stock_id UK
        string created_by
        string updated_by
        string depositor
        string summary
        string editable_summary
        list genes
        list dbxrefs
        list publications
        timestamp created_at
        timestamp updated_at
    }

    stockprop {
        string _key PK
        string label
        string species
        string plasmid
        list names
        string dicty_strain_property
        string name
        string image_map
        string sequence
    }

    stock_key_generator {
        string _key PK
        int offset
    }

    cvterm {
        string _key PK
        string label
        string name
    }

    cv {
        string _key PK
        string namespace
    }

The stock collection holds fields shared by both strains and plasmids. Type-specific fields (label, species, names for strains; name, sequence, image_map for plasmids) live in a linked stockprop document. Ontology classification is stored as graph edges rather than embedded fields, allowing stocks to be queried and grouped by ontology term without denormalizing the data.

Edge collections and their graphs: