ixapi

package
v0.1.27 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Dec 15, 2025 License: BSD-2-Clause Imports: 20 Imported by: 0

README

Intelligence X API Go client

The Go package ixapi uses the Intelligence X API to perform searches and return results.

There is a full working command line program ix that uses this package.

Using the package

To download the package:

go get -u github.com/IntelligenceX/SDK/Go/ixapi

Then import it in your code:

import "github.com/IntelligenceX/SDK/Go/ixapi"

The code has a default public API key and URL embedded. If you received your own API key, make sure to specify it in the Init function.

Following code performs a search and queries the results. Selector is the search term.

search := ixapi.IntelligenceXAPI{}
search.Init("", "")
results, selectorInvalid, err := search.Search(ctx, Selector, ixapi.SortXScoreDesc, 100, ixapi.DefaultWaitSortTime, ixapi.DefaultTimeoutGetResults)

These are all functions available of the IntelligenceXAPI struct:

Init                    Initializes the API with optional API URL and Key
SearchStart             Starts a search and returns the search ID
SearchStartAdvanced     Starts a search with optional parameters and returns the search ID
SearchGetResults        Returns available results
SearchTerminate         Terminates a search
FilePreview             Returns the preview (max first 1000 characters) of an item
FileRead                Returns the full item data
SearchGetResultsAll     Returns all results within a timeout
SetAPIKey               Sets API URL and Key to use

These are high-level functions that search and return the results immediately:

Search                  Starts a search and queries all results
SearchWithDates         Starts a search with dates and queries all results

Source

IX API Downloaded from https://github.com/IntelligenceX/SDK/blob/master/Go/ixapi at 2025-03-14

Documentation

Index

Constants

View Source
const (
	SortNone       = 0 // No sorting
	SortXScoreAsc  = 1 // X-Score ascending = Least relevant first
	SortXScoreDesc = 2 // X-Score descending = Most relevant first
	SortDateAsc    = 3 // Date ascending = Oldest first
	SortDateDesc   = 4 // Date descending = Newest first
)

Sort orders

View Source
const DefaultTimeoutGetResults = 20 * time.Second

DefaultTimeoutGetResults is the suggested timeout after which the search will be terminated.

View Source
const DefaultWaitSortTime = 400 * time.Millisecond

DefaultWaitSortTime is the suggested time to give the API to process and sort all the results, before the client queries them.

View Source
const TagLanguage = 0

TagLanguage is ISO 639-1 defined

Variables

This section is empty.

Functions

func SimhashCompareItems

func SimhashCompareItems(Item1, Item2 *Item) uint8

SimhashCompareItems compares 2 items for data equalness and returns the hamming distance. The closer to 0 the more equal they are. Never compare Simhashes directly because with different content types and even on the same type with different encoding they use different algorithms.

Types

type CsvItem

type CsvItem struct {
	Name     string `json:"name"`
	Date     string `json:"date"`
	Bucket   string `json:"bucket"`
	Media    string `json:"media"`
	Content  string `json:"content"`
	Type     string `json:"type"`
	Size     int64  `json:"size"`
	SystemID string `json:"system id"`
}

type IntelligenceXAPI

type IntelligenceXAPI struct {
	URL string    // The API URL. Always ending with slash.
	Key uuid.UUID // The API key assigned by Intelligence X. Contact the company to receive one.

	// additional input. Set before calling Init
	ProxyURL string // Proxy to use
	BindToIP string // Bind to a specific IPv4 or IPv6

	// one client for the session
	Client              http.Client
	RetryAttempts       int // in case of underlying transport failure
	UserAgent           string
	HTTPMaxResponseSize int64

	HTTPTimeout time.Duration

	WriteCounter *WriteCounter
}

IntelligenceXAPI holds all information for communicating with the Intelligence X API. Call Init() first.

func (*IntelligenceXAPI) DownloadZip added in v0.1.8

func (api *IntelligenceXAPI) DownloadZip(ctx context.Context, searchID uuid.UUID, Limit int, OutputFile string) (err error)

DownloadZip reads the data of an item.

func (*IntelligenceXAPI) FilePreview

func (api *IntelligenceXAPI) FilePreview(ctx context.Context, item *Item) (text string, err error)

FilePreview loads the preview of an item. Previews are always capped at 1000 characters.

func (*IntelligenceXAPI) FileRead

func (api *IntelligenceXAPI) FileRead(ctx context.Context, item *Item, Limit int64) (data []byte, err error)

FileRead reads the data of an item.

func (*IntelligenceXAPI) Init

func (api *IntelligenceXAPI) Init(URL string, Key string)

Init initializes the IX API. URL and Key may be empty to use defaults.

func (*IntelligenceXAPI) Search

func (api *IntelligenceXAPI) Search(ctx context.Context, Selector string, Sort, Limit int, WaitSort, TimeoutGetResults time.Duration) (records []SearchResult, selectorInvalid bool, err error)

Search starts a search and queries all results. It takes a selector as input. WaitSort should be a few hundred ms, giving the API time to sort the results before querying them. Limit is the max count of results to query per bucket. The total number of results returned might be higher. TimeoutGetResults is the max amount of time for querying all results. This should be at least a few seconds but a timeout of 10-30 seconds makes sense. If the input selector is invalid (not a strong selector), the function returns selectorInvalid set to true with no error reported.

func (*IntelligenceXAPI) SearchGetResults

func (api *IntelligenceXAPI) SearchGetResults(ctx context.Context, searchID uuid.UUID, Limit int) (records []SearchResult, status int, err error)

SearchGetResults returns results Status: 0 = Success with results (continue), 1 = No more results available (this response might still have results), 2 = Search ID not found, 3 = No results yet available keep trying, 4 = Error

func (*IntelligenceXAPI) SearchGetResultsAll

func (api *IntelligenceXAPI) SearchGetResultsAll(ctx context.Context, searchID uuid.UUID, Limit int, Timeout time.Duration) (records []SearchResult, err error)

SearchGetResultsAll returns all results up to Limit and up to the given Timeout. It will automatically terminate the search before returning. Unless the underlying API requests report and error, no error will be returned. Deadline exceeded is treated as no error.

func (*IntelligenceXAPI) SearchStart

func (api *IntelligenceXAPI) SearchStart(ctx context.Context, Term string) (searchID uuid.UUID, selectorInvalid bool, err error)

SearchStart starts a search

func (*IntelligenceXAPI) SearchStartAdvanced

func (api *IntelligenceXAPI) SearchStartAdvanced(ctx context.Context, Input IntelligentSearchRequest) (searchID uuid.UUID, selectorInvalid bool, err error)

SearchStartAdvanced starts a search and allows the caller to set any advanced filter

func (*IntelligenceXAPI) SearchTerminate

func (api *IntelligenceXAPI) SearchTerminate(ctx context.Context, searchID uuid.UUID) (err error)

SearchTerminate terminates a search

func (*IntelligenceXAPI) SearchWithDates

func (api *IntelligenceXAPI) SearchWithDates(ctx context.Context, Selector string, DateFrom, DateTo time.Time, Sort, Limit int, WaitSort, TimeoutGetResults time.Duration) (searchID *uuid.UUID, records []SearchResult, selectorInvalid bool, err error)

SearchWithDates starts a search and queries all results. It takes a selector and dates as input. Sorting is newest first. WaitSort should be a few hundred ms, giving the API time to sort the results before querying them. Limit is the max count of results to query per bucket. The total number of results returned might be higher. TimeoutGetResults is the max amount of time for querying all results. This should be at least a few seconds but a timeout of 10-30 seconds makes sense.

func (*IntelligenceXAPI) SetAPIKey

func (api *IntelligenceXAPI) SetAPIKey(URL string, Key string)

SetAPIKey sets the API URL and Key. URL and Key may be empty to use defaults.

type IntelligentSearchRequest

type IntelligentSearchRequest struct {
	Term        string        `json:"term"`       // Search term submitted by the user, e.g. "Document 1.docx" or "email@example.com"
	Buckets     []string      `json:"buckets"`    // Bucket identifiers
	Timeout     time.Duration `json:"timeout"`    // Timeout in seconds. May be limited by API config. 0 means default.
	MaxResults  int           `json:"maxresults"` // Total number of max results per bucket. May be limited by API config. 0 means default.
	DateFrom    string        `json:"datefrom"`   // Date from, both from/to are required if set, format "2006-01-02 15:04"
	DateTo      string        `json:"dateto"`     // Date to, both from/to are required if set, format "2006-01-02 15:04"
	Sort        int           `json:"sort"`       // Sort order: 0 = no sorting, 1 = X-Score ASC, 2 = X-Score DESC, 3 = Date ASC, 4 = Date DESC
	Media       int           `json:"media"`      // Media: 0 = not defined, otherwise MediaX as defined in ixservice
	TerminateID []uuid.UUID   `json:"terminate"`  // Optional: Previous search IDs to terminate (normal search or Phonebook). This is if the user makes a new search from the same tab. Same as first calling /intelligent/search/terminate.
}

IntelligentSearchRequest is the information from the human for the search.

type IntelligentSearchResponse

type IntelligentSearchResponse struct {
	ID                  uuid.UUID `json:"id"`                  // id of the search job. This is used to get the results.
	SoftSelectorWarning bool      `json:"softselectorwarning"` // Warning of soft selectors, typically garbage in which results into garbage out
	Status              int       `json:"status"`              // Status of the search: 0 = Success (ID valid), 1 = Invalid Term, 2 = Error Max Concurrent Searches
}

IntelligentSearchResponse is the result to the initial search request

type IntelligentSearchResult

type IntelligentSearchResult struct {
	Records []SearchResult `json:"records"` // The result records
	Status  int            `json:"status"`  // Status: 0 = Success with results, 1 = No more results available, 2 = Search ID not found, 3 = No results yet available keep trying
}

IntelligentSearchResult contains the result items

type Item

type Item struct {
	ID          uint      `json:"id" gorm:"primarykey"`
	SystemID    string    `gorm:"column:system_id;index:,unique;" json:"systemid"` // System identifier uniquely identifying the item
	StorageID   string    `json:"storageid"`                                       // Storage identifier, empty if not stored/available, otherwise a 64-byte blake2b hash hex-encoded
	InStore     bool      `json:"instore"`                                         // Whether the data of the item is in store and the storage id is valid. Also used to indicate update when false but storage id is set.
	Size        int64     `json:"size"`                                            // Size in bytes of the item data
	AccessLevel int       `json:"accesslevel"`                                     // Native access level of the item (0 = Public..)
	Type        int       `json:"type"`                                            // Low-level content type (0 = Binary..)
	Media       int       `json:"media"`                                           // High-level media type (User, Paste, Tweet, Forum Post..)
	Added       time.Time `json:"added"`                                           // When the item was added to the system
	Date        time.Time `json:"date"`                                            // Full time stamp item when it was discovered or created
	Name        string    `json:"name"`                                            // Name or title
	Description string    `json:"description"`                                     // Full description, text only
	XScore      int       `json:"xscore"`                                          // X-Score, ranking its relevancy. 0-100, default 50
	Simhash     uint64    `json:"simhash"`                                         // Simhash, depending on content type. Use hamming distance to compare equality of items data.
	Bucket      string    `json:"bucket"`                                          // Bucket
	Filename    string    `json:"filename"`                                        // FileName
	Downloaded  bool      `json:"downloaded"`                                      // If file already downloaded

	// Tags are meta-data tags helping in classification of the items data. They reveal for example the language or a topic. Different to key-values they have hard-coded classes that
	// allow anyone to take action on them.
	Tags []Tag `json:"tags"`

	// Relations lists all related items.
	Relations []Relationship `json:"relations"`
}

Item represents any items meta-data. It origins from Indexed and is sent as search results. All fields except the identifier are optional and may be zero. It is perfectly valid that a service only knows partial information (like a name or storage id) of a given item.

func (*Item) BeforeCreate added in v0.1.23

func (item *Item) BeforeCreate(tx *gorm.DB) (err error)

func (*Item) GetExtension

func (item *Item) GetExtension() string

func (*Item) GetTag

func (item *Item) GetTag(Class int16) (Value string)

GetTag gets a tags value for the first occurrence. Empty if not found.

func (Item) TableName

func (Item) TableName() string

type PanelSearchResultTag

type PanelSearchResultTag struct {
	ID             uint   `json:"id" gorm:"primarykey"`
	SearchResultID uint   `json:"search_result_id"`
	Class          int16  `json:"class"`  // Class of tag
	ClassH         string `json:"classh"` // Class of tag, human friendly
	Value          string `json:"value"`  // The value
	ValueH         string `json:"valueh"` // Value, human friendly
}

PanelSearchResultTag represents a tag in human form.

func (PanelSearchResultTag) TableName

func (PanelSearchResultTag) TableName() string

type Relationship

type Relationship struct {
	ID       uint   `json:"id" gorm:"primarykey"`
	ItemID   uint   `json:"item_id";gorm:"uniqueIndex:idx_relation_target_r"`
	Target   string `json:"target";gorm:"uniqueIndex:idx_relation_target_r"`   // Target item systemid
	Relation int    `json:"relation";gorm:"uniqueIndex:idx_relation_target_r"` // The relationship, see RelationX
}

Relationship defines a relation between 2 items.

func (Relationship) TableName

func (Relationship) TableName() string

type SearchResult

type SearchResult struct {
	Item
	AccessLevelH string                 `json:"accesslevelh"` // Human friendly access level info
	MediaH       string                 `json:"mediah"`       // Human friendly media type info
	SimhashH     string                 `json:"simhashh"`     // Human friendly simhash
	TypeH        string                 `json:"typeh"`        // Human friendly content type info
	TagsH        []PanelSearchResultTag `json:"tagsh"`        // Human friendly tags
	RandomID     string                 `json:"randomid"`     // Random ID
	BucketH      string                 `json:"bucketh"`      // Human friendly bucket name
	Group        string                 `json:"group"`        // File Group
	IndexFile    string                 `json:"indexfile"`    // Index file ID
}

SearchResult represents a single result record. The entire record IS the de-facto result. Every field is optional and may be empty.

func (*SearchResult) GetCsv

func (sr *SearchResult) GetCsv() *CsvItem

func (SearchResult) TableName

func (SearchResult) TableName() string

type Tag

type Tag struct {
	ID     uint   `json:"id" gorm:"primarykey"`
	ItemID uint   `json:"item_id";gorm:"uniqueIndex:idx_tag_class_v"`
	Class  int16  `json:"class";gorm:"uniqueIndex:idx_tag_class_v"` // Class of tag
	Value  string `json:"value";gorm:"uniqueIndex:idx_tag_class_v"` // The value
}

Tag classifies the items data

func (Tag) TableName

func (Tag) TableName() string

type WriteCounter added in v0.1.8

type WriteCounter struct {
	Total uint64
}

WriteCounter counts the number of bytes written to it. It implements to the io.Writer interface and we can pass this into io.TeeReader() which will report progress on each write cycle.

func (*WriteCounter) Write added in v0.1.8

func (wc *WriteCounter) Write(p []byte) (int, error)

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL