README
¶
BricksLLM: AI Gateway For Putting LLM In Production
BricksLLM is a cloud native AI gateway written in Go. Currently, it serves as a proxy to OpenAI. We let you create API keys that have rate limits, cost limits and TTLs. The API keys can be used in both development and production to achieve fine-grained access control that is not provided by OpenAI at the moment. The proxy is compatible with OpenAI API and its SDKs.
The vision of BricksLLM is to support many more large language models such as LLama2, Claude, PaLM2 etc, and streamline LLM operations.
Roadmap
- Access control via API key with rate limit, cost limit and ttl
- Logging integration
- Statsd integration
- Custom Provider Integration
- PII detection and masking 🚧
Getting Started
The easiest way to get started with BricksLLM is through BricksLLM-Docker.
Step 1 - Clone BricksLLM-Docker repository
git clone https://github.com/bricks-cloud/BricksLLM-Docker
Step 2 - Change to BricksLLM-Docker directory
cd BricksLLM-Docker
Step 3 - Deploy BricksLLM locally with Postgresql and Redis
docker-compose up
You can run this in detach mode use the -d flag: docker-compose up -d
Step 4 - Create a provider setting
curl -X PUT http://localhost:8001/api/provider-settings \
-H "Content-Type: application/json" \
-d '{
"provider":"openai",
"setting": {
"apikey": "YOUR_OPENAI_KEY"
}
}'
Copy the id from the response.
Step 5 - Create a Bricks API key
Use id from the previous step as settingId to create a key with a rate limit of 2 req/min and a spend limit of 25 cents.
curl -X PUT http://localhost:8001/api/key-management/keys \
-H "Content-Type: application/json" \
-d '{
"name": "My Secret Key",
"key": "my-secret-key",
"tags": ["mykey"],
"settingId": "ID_FROM_STEP_FOUR",
"rateLimitOverTime": 2,
"rateLimitUnit": "m",
"costLimitInUsd": 0.25
}'
Congratulations you are done!!!
Then, just redirect your requests to us and use OpenAI as you would normally. For example:
curl -X POST http://localhost:8002/api/providers/openai/v1/chat/completions \
-H "Authorization: Bearer my-secret-key" \
-H "Content-Type: application/json" \
-d '{
"model": "gpt-3.5-turbo",
"messages": [
{
"role": "system",
"content": "hi"
}
]
}'
Or if you're using an SDK, you could change its baseURL to point to us. For example:
// OpenAI Node SDK v4
import OpenAI from 'openai';
const openai = new OpenAI({
apiKey: "some-secret-key", // key created earlier
baseURL: "http://localhost:8002/api/providers/openai/v1", // redirect to us
});
How to Update?
For updating to the latest version
docker pull luyuanxin1995/bricksllm:latest
For updating to a particular version
docker pull luyuanxin1995/bricksllm:1.4.0
Documentation
Environment variables
Name type description default POSTGRESQL_HOSTSrequired Hosts for Postgresql DB. Separated by , localhostPOSTGRESQL_DB_NAMEoptional Name for Postgresql DB. POSTGRESQL_USERNAMErequired Postgresql DB username POSTGRESQL_PASSWORDrequired Postgresql DB password POSTGRESQL_SSL_MODEoptional Postgresql SSL mode disablePOSTGRESQL_PORToptional The port that Postgresql DB runs on 5432POSTGRESQL_READ_TIME_OUToptional Timeout for Postgresql read operations 2sPOSTGRESQL_WRITE_TIME_OUToptional Timeout for Postgresql write operations 1sREDIS_HOSTSrequired Host for Redis. Separated by , localhostREDIS_PASSWORDoptional Redis Password REDIS_PORToptional The port that Redis DB runs on 6379REDIS_READ_TIME_OUToptional Timeout for Redis read operations 1sREDIS_WRITE_TIME_OUToptional Timeout for Redis write operations 500msIN_MEMORY_DB_UPDATE_INTERVALoptional The interval BricksLLM API gateway polls Postgresql DB for latest key configurations 1sSTATS_PROVIDERoptional This value can only be datadog. Required for integration with Datadog. PROXY_TIMEOUToptional This value can only be datadog. Required for integration with Datadog.
Configuration Endpoints
The configuration server runs on Port 8001.
Get keys: GET /api/key-management/keys
This endpoint is set up for retrieving key configurations using a query param called tag.
name type data type description tagoptional string Identifier attached to a key configuration tagsoptional array of string Identifiers attached to a key configuration provideroptional string Provider attached to a key provider configuration. Its value can only be openai.
http code content-type 400,500application/json
Field type example status int400 title stringrequest body reader error type string/errors/request-body-read detail stringsomething is wrong instance string/api/key-management/keys
Response Body []KeyConfiguration
Fields of KeyConfiguration
Field type example description name stringspike's developer key Name of the API key. createdAt int641257894000 Key configuration creation time in unix. updatedAt int641257894000 Key configuration update time in unix. revoked booleantrue Indicator for whether the key is revoked. revokedReason stringThe key has expired Reason for why the key is revoked. tags []string["org-tag-12345"] Identifiers associated with the key. keyId string550e8400-e29b-41d4-a716-446655440000 Unique identifier for the key. costLimitInUsd float645.5Total spend limit of the API key. costLimitInUsdOverTime float642Total spend within period of time. This field is required if costLimitInUsdUnit is specified. costLimitInUsdUnit enumd Time unit for costLimitInUsdOverTime. Possible values are [ m,h,d,mo].rateLimitOverTime int2rate limit over period of time. This field is required if rateLimitUnit is specified. rateLimitUnit stringm Time unit for rateLimitOverTime. Possible values are [ h,m,s,d]ttl string2d time to live. Available units are [ s,m,h]
Create key: PUT /api/key-management/keys
This endpoint is set up for retrieving key configurations using a query param called tag.
PathConfig
Field required type example description path required string/api/providers/openai/v1/chat/completion Allowed path method required stringPOST HTTP Method
Field required type example description name required stringspike's developer key Name of the API key. tags optional []string["org-tag-12345"]Identifiers associated with the key. key required stringabcdef12345 API key settingId required string98daa3ae-961d-4253-bf6a-322a32fdca3d API key costLimitInUsd optional float645.5Total spend limit of the API key. costLimitInUsdOverTime optional float642Total spend within period of time. This field is required if costLimitInUsdUnit is specified. costLimitInUsdUnit optional enumd Time unit for costLimitInUsdOverTime. Possible values are [ m,h,d,mo].rateLimitOverTime optional int2 rate limit over period of time. This field is required if rateLimitUnit is specified. rateLimitUnit optional enumm Time unit for rateLimitOverTime. Possible values are [ h,m,s,d]ttl optional string2d time to live. Available units are [ s,m,h]allowedPaths optional []PathConfig2d Pathes allowed for access
http code content-type 400,500application/json
Field type example status int400title stringrequest body reader error type string/errors/request-body-read detail stringsomething is wrong instance string/api/key-management/keys
Field type example description name stringspike's developer key Name of the API key. createdAt int641257894000Key configuration creation time in unix. updatedAt int641257894000Key configuration update time in unix. revoked booleantrueIndicator for whether the key is revoked. revokedReason stringThe key has expired Reason for why the key is revoked. tags []string["org-tag-12345"] Identifiers associated with the key. keyId string550e8400-e29b-41d4-a716-446655440000 Unique identifier for the key. costLimitInUsd float645.5Total spend limit of the API key. costLimitInUsdOverTime float642 Total spend within period of time. This field is required if costLimitInUsdUnit is specified. costLimitInUsdUnit enumd Time unit for costLimitInUsdOverTime. Possible values are [ m,h,d,mo].rateLimitOverTime int2rate limit over period of time. This field is required if rateLimitUnit is specified. rateLimitOverTime int2rate limit over period of time. This field is required if rateLimitUnit is specified. rateLimitUnit stringm Time unit for rateLimitOverTime. Possible values are [ h,m,s,d].ttl string2d time to live. Available units are [ s,m,h]allowedPaths []PathConfig[{ "path": "/api/providers/openai/v1/chat/completion", method: "POST"}]Allowed paths that can be accessed using the key.
Update key: PATCH /api/key-management/keys/{keyId}
This endpoint is set up for updating key configurations using key id.
name type data type description keyIdrequired string Unique key configuration identifier.
PathConfig
Field required type example description path required string/api/providers/openai/v1/chat/completion Allowed path method required stringPOST HTTP Method
Field required type example description name optional stringspike's developer key Name of the API key. tags optional []string["org-tag-12345"]Identifiers associated with the key. revoked optional booleantrueIndicator for whether the key is revoked. revokedReason optional stringThe key has expired Reason for why the key is revoked. allowedPaths optional []PathConfig2d Pathes allowed for access.
http code content-type 400,500application/json
Field type example status int400 title stringrequest body reader error type string/errors/request-body-read detail stringsomething is wrong instance string/api/key-management/keys
Field type example description name stringspike's developer key Name of the API key. createdAt int641257894000Key configuration creation time in unix. updatedAt int641257894000Key configuration update time in unix. revoked booleantrueIndicator for whether the key is revoked. revokedReason stringThe key has expired Reason for why the key is revoked. tags []string["org-tag-12345"]Identifiers associated with the key. keyId string550e8400-e29b-41d4-a716-446655440000 Unique identifier for the key. costLimitInUsd float645.5Total spend limit of the API key. costLimitInUsdOverTime float642Total spend within period of time. This field is required if costLimitInUsdUnit is specified. costLimitInUsdUnit enumd Time unit for costLimitInUsdOverTime. Possible values are [ m,h,d,mo].rateLimitOverTime int2rate limit over period of time. This field is required if rateLimitUnit is specified. rateLimitUnit stringm Time unit for rateLimitOverTime. Possible values are [ h,m,s,d]ttl string2d time to live. Available units are [ s,m,h]allowedPaths []PathConfig[{ "path": "/api/providers/openai/v1/chat/completion", method: "POST"}]Allowed paths that can be accessed using the key.
Create a provider setting: POST /api/provider-settings
This endpoint is creating a provider setting.
Field required type example description provider required enumopenai This value can only be openai,anthropicandazureas for now.setting required Setting{ "apikey": "YOUR_OPENAI_KEY" }A map of values used for authenticating with the selected provider. name optional stringYOUR_PROVIDER_SETTING_NAME This field is used for giving a name to provider setting allowedModels []string["text-embedding-ada-002"]Allowed models for this provider setting.
Setting
Field required type example description apiKey required stringxx-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxThis value is required. resourceName required stringYOUR_AZURE_RESOURCE_NAMEThis value is required when the provider is azure.
http code content-type 400,500application/json
Field type example status int400title stringrequest body reader error type string/errors/request-body-read detail stringsomething is wrong instance string/api/provider-settings
Field type example description createdAt int641699933571Unix timestamp for creation time. updatedAt int641699933571Unix timestamp for update time. provider enumopenaiThis value can only be openaias for now.id string98daa3ae-961d-4253-bf6a-322a32fdca3d This value is a unique identifier. name stringYOUR_PROVIDER_SETTING_NAME Provider setting name. allowedModels []string["text-embedding-ada-002"]Allowed models for this provider setting.
Get all provider settings: GET /api/provider-settings
This endpoint is getting all provider settings.
Field type type example description
http code content-type 500application/json
Field type example status int400title stringrequest body reader error type string/errors/request-body-read detail stringsomething is wrong instance string/api/provider-settings
[]ProviderSetting
ProviderSetting
Field type example description createdAt int641699933571Unix timestamp for creation time. updatedAt int641699933571Unix timestamp for update time. provider enumopenaiThis value can only be openaias for now.id string98daa3ae-961d-4253-bf6a-322a32fdca3d This value is a unique identifier. name stringYOUR_PROVIDER_SETTING_NAME Provider setting name. allowedModels []string["text-embedding-ada-002"]Allowed models for this provider setting.
Update a provider setting: PATCH /api/provider-settings/:id
This endpoint is updating a provider setting .
name type data type description idrequired stringUnique identifier for the provider setting that you want to update.
Field required type example description setting required Setting{ "apikey": "YOUR_OPENAI_KEY" }A map of values used for authenticating with the selected provider. name optional stringYOUR_PROVIDER_SETTING_NAME This field is used for giving a name to provider setting allowedModels []string["text-embedding-ada-002"]Allowed models for this provider setting.
Setting
Field required type example description apiKey required stringxx-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxThis value is required. resourceName required stringYOUR_AZURE_RESOURCE_NAMEThis value is required when the provider is azure.deploymentId required stringYOUR_DEPLOYMENT_IDThis value is required when the provider is azure.
http code content-type 400,500application/json
Field type example status int400title stringrequest body reader error type string/errors/request-body-read detail stringsomething is wrong instance string/api/provider-settings
Field type example description createdAt int641699933571Unix timestamp for creation time. updatedAt int641699933571Unix timestamp for update time. provider enumopenaiThis value can only be openaias for now.id string98daa3ae-961d-4253-bf6a-322a32fdca3dThis value is a unique identifier name stringYOUR_PROVIDER_SETTING_NAME Provider setting name. allowedModels []string["text-embedding-ada-002"]Allowed models for this provider setting.
Retrieve Metrics: POST /api/reporting/events
This endpoint is retrieving aggregated metrics given an array of key ids and tags.
Field required type example description keyIds required []string["key-1", "key-2", "key-3" ]Array of ids that specicify the keys that you want to aggregate stats from. tags required []string["tag-1", "tag-2"]Array of tags that specicify the keys that you want to aggregate stats from. start required int641699933571Start timestamp for the requested timeseries data. end required int641699933571End timestamp for the requested timeseries data. increment required int60This field is the increment in seconds for the requested timeseries data.
http code content-type 500application/json
Field type example status int400title stringrequest body reader error type string/errors/request-body-read detail stringsomething is wrong instance string/api/provider-settings
Field type example description dataPoints []dataPoint[{ "timeStamp": 1699933571, "numberOfRequests": 1, "costInUsd": 0.8, "latencyInMs": 600, "promptTokenCount": 0, "completionTokenCount": 0, "successCount": 1 }]Unix timestamp for creation time. latencyInMsMedian float64656.7Median latency for the given time period. latencyInMs99th float64555.799th percentile latency for the given time period.
Datapoint
Field type example description timeStamp int641702504746Unix timestamp for the data point numberOfRequests int64100Aggregated number of http requests over the given time increment. costInUsd float641.7Aggregated cost of proxied requests in USD over the given time increment. latencyInMs int555Aggregated latency in milliseconds of http requests over the given time increment. promptTokenCount int25Aggregated prompt token counts over the given time increment. completionTokenCount int4000Aggregated completion token counts over the given time increment. successCount int555Aggregated number of successful http requests over the given time increment. keyId int555.7key Id associated with the event. model stringgpt-3.5-turbomodel associated with the event.
Get events: GET /api/events
This endpoint is for getting events.
name type data type description customIdoptional string Custom identifier attached to an event
http code content-type 500application/json
Field type example status int400title stringrequest body reader error type string/errors/request-body-read detail stringsomething is wrong instance string/api/provider-settings
[]Event
Event
Field type example description id int641699933571Unique identifier associated with the event. created_at int641699933571Unix timestamp for creation time. tags int64["YOUR_TAG"]Tags of the key. key_id stringYOUR_KEY_ID Key Id associated with the proxy request. cost_in_usd float640.0004 Cost incured by the proxy request. model stringgpt-4-1105-preview Model used in the proxy request. provider stringopenaiProvider for the proxy request. status int200Http status. prompt_token_count int8Prompt token count of the proxy request. completion_token_count int16Completion token counts of the proxy request. latency_in_ms int160Provider setting name. path string/api/v1/chat/completion Provider setting name. method stringPOST Http method for the assoicated proxu request. custom_id stringYOUR_CUSTOM_ID Custom Id passed by the user in the headers of proxy requests.
Create custom provider: POST /api/custom/providers
This endpoint is creating custom providers.
Field required type example description path required string/chat/completionPath associated with the custom provider route. It must be unique within the custom provider. target_url required stringhttps://api.openai.com/v1/chat/completionsProxy destination URL for the custom provider route. model_location required stringmodelJSON field for the model in the HTTP request. request_prompt_location required stringmessages.#.contentJSON field for the prompt request in the HTTP request. response_completion_location required stringchoices.#.message.contentJSON field for the completion content in the HTTP response. stream_location required stringstreamJSON field for the stream boolean in the HTTP request. stream_end_word required string[DONE]End word for the stream. stream_response_completion_location required stringchoices.#.delta.contentJSON field for the completion content in the streaming response. stream_max_empty_messages required int10Number of max empty messages in stream.
Field required type example description provider required stringbricksUnique identifier associated with the route config. route_configs required []RouteConfig{{ "path": "/chat/completions", "target_url": "https://api.openai.com/v1/chat/completions" }}Route configurations for the custom provider. authentication_param optional stringapikeyThe authentication parameter required for.
http code content-type 500,400application/json
Field type example status int400title stringrequest body reader error type string/errors/request-body-read detail stringsomething is wrong instance string/api/custom/providers
Field type example description id int641699933571Unique identifier associated with the event. created_at int641699933571Unix timestamp for creation time. updated_at int641699933571Unix timestamp for update time. provider stringbricksUnique identifier associated with the route config. route_configs []RouteConfig{{ "path": "/chat/completions", "target_url": "https://api.openai.com/v1/chat/completions" }}Start timestamp for the requested timeseries data. authentication_param stringapikeyThe authentication parameter required for.
Update custom provider: PATCH /api/custom/providers/:id
This endpoint is updating a custom provider.
Field required type example description path required string/chat/completionPath associated with the custom provider route. It must be unique within the custom provider. target_url required stringhttps://api.openai.com/v1/chat/completionsProxy destination URL for the custom provider route. model_location required stringmodelJSON field for the model in the HTTP request. request_prompt_location required stringmessages.#.contentJSON field for the prompt request in the HTTP request. response_completion_location required stringchoices.#.message.contentJSON field for the completion content in the HTTP response. stream_location required stringstreamJSON field for the stream boolean in the HTTP request. stream_end_word required string[DONE]End word for the stream. stream_response_completion_location required stringchoices.#.delta.contentJSON field for the completion content in the streaming response. stream_max_empty_messages required int10Number of max empty messages in stream.
Field required type example description route_configs optional []RouteConfig{{ "path": "/chat/completions", "target_url": "https://api.openai.com/v1/chat/completions" }}Route configurations for the custom provider. authentication_param optional stringapikeyThe authentication parameter required for.
http code content-type 500,404,400application/json
Field type example status int400title stringrequest body reader error type string/errors/request-body-read detail stringsomething is wrong instance string/api/custom/providers
Field type example description id int641699933571Unique identifier associated with the event. created_at int641699933571Unix timestamp for creation time. updated_at int641699933571Unix timestamp for update time. provider stringbricksUnique identifier associated with the route config. route_configs []RouteConfig{{ "path": "/chat/completions", "target_url": "https://api.openai.com/v1/chat/completions" }}Start timestamp for the requested timeseries data. authentication_param stringapikeyThe authentication parameter required for.
Get custom providers: GET /api/custom/providers
This endpoint is for getting custom providers.
Field required type example description path required string/chat/completionPath associated with the custom provider route. It must be unique within the custom provider. target_url required stringhttps://api.openai.com/v1/chat/completionsProxy destination URL for the custom provider route. model_location required stringmodelJSON field for the model in the HTTP request. request_prompt_location required stringmessages.#.contentJSON field for the prompt request in the HTTP request. response_completion_location required stringchoices.#.message.contentJSON field for the completion content in the HTTP response. stream_location required stringstreamJSON field for the stream boolean in the HTTP request. stream_end_word required string[DONE]End word for the stream. stream_response_completion_location required stringchoices.#.delta.contentJSON field for the completion content in the streaming response. stream_max_empty_messages required int10Number of max empty messages in stream.
Field required type example description route_configs optional []RouteConfig{{ "path": "/chat/completions", "target_url": "https://api.openai.com/v1/chat/completions" }}Route configurations for the custom provider. authentication_param optional stringapikeyThe authentication parameter required for.
http code content-type 500application/json
Field type example status int400title stringrequest body reader error type string/errors/request-body-read detail stringsomething is wrong instance string/api/custom/providers
[]Provider
Provider
Field type example description id int641699933571Unique identifier associated with the event. created_at int641699933571Unix timestamp for creation time. updated_at int641699933571Unix timestamp for update time. provider stringbricksUnique identifier associated with the route config. route_configs []RouteConfig{{ "path": "/chat/completions", "target_url": "https://api.openai.com/v1/chat/completions" }}Start timestamp for the requested timeseries data. authentication_param stringapikeyThe authentication parameter required for.
OpenAI Proxy
The OpenAI proxy runs on Port 8002.
name type data type description x-custom-event-idoptional stringCustom Id that can be used to retrieve an event associated with each proxy request.
Chat Completion
Call OpenAI chat completions: POST /api/providers/openai/v1/chat/completions
This endpoint is set up for proxying OpenAI chat completion requests. Documentation for this endpoint can be found here.
Embeddings
Call OpenAI embeddings: POST /api/providers/openai/v1/embeddings
This endpoint is set up for proxying OpenAI embedding requests. Documentation for this endpoint can be found here.
Moderations
Call OpenAI moderations: POST /api/providers/openai/v1/moderations
This endpoint is set up for proxying OpenAI moderation requests. Documentation for this endpoint can be found here.
Models
Get OpenAI models: GET /api/providers/openai/v1/models
This endpoint is set up for retrieving OpenAI models. Documentation for this endpoint can be found here.
Retrieve an OpenAI model: GET /api/providers/openai/v1/models/:model
This endpoint is set up for retrieving an OpenAI model. Documentation for this endpoint can be found here.
Files
List files: GET /api/providers/openai/v1/files
This endpoint is set up for list OpenAI files. Documentation for this endpoint can be found here.
Upload a file: POST /api/providers/openai/v1/files
This endpoint is set up for creating an OpenAI file. Documentation for this endpoint can be found here.
Delete a file: POST /api/providers/openai/v1/files/:file_id
This endpoint is set up for creating an OpenAI file. Documentation for this endpoint can be found here.
Retrieve a file: GET /api/providers/openai/v1/files/:file_id
This endpoint is set up for retrieving an OpenAI file. Documentation for this endpoint can be found here.
Retrieve file content: GET /api/providers/openai/v1/files/:file_id/content
This endpoint is set up for retrieving an OpenAI file content. Documentation for this endpoint can be found here.
Images
Generate images: POST /api/providers/openai/v1/images/generations
This endpoint is set up for generating OpenAI images. Documentation for this endpoint can be found here.
Edit images: POST /api/providers/openai/v1/images/edits
This endpoint is set up for editting OpenAI generated images. Documentation for this endpoint can be found here.
Create image variations: POST /api/providers/openai/v1/images/variations
This endpoint is set up for creating OpenAI image variations. Documentation for this endpoint can be found here.
Voices
Create speech: POST /api/providers/openai/v1/audio/speech
This endpoint is set up for creating speeches. Documentation for this endpoint can be found here.
Create transcriptions: POST /api/providers/openai/v1/audio/transcriptions
This endpoint is set up for editting generated images. Documentation for this endpoint can be found here.
Create translations: POST /api/providers/openai/v1/audios/translations
This endpoint is set up for creating translations. Documentation for this endpoint can be found here.
Assistants
Create assistant: POST /api/providers/openai/v1/assistants
This endpoint is set up for creating an OpenAI assistant. Documentation for this endpoint can be found here.
Retrieve assistant: GET /api/providers/openai/v1/assistants/:assistant_id
This endpoint is set up for retrieving an OpenAI assistant. Documentation for this endpoint can be found here.
Modify assistant: POST /api/providers/openai/v1/assistants/:assistant_id
This endpoint is set up for modifying an OpenAI assistant. Documentation for this endpoint can be found here.
Delete assistant: DELETE /api/providers/openai/v1/assistants/:assistant_id
This endpoint is set up for deleting an OpenAI assistant. Documentation for this endpoint can be found here.
List assistants: GET /api/providers/openai/v1/assistants
This endpoint is set up for listing OpenAI assistants. Documentation for this endpoint can be found here.
Create assistant file: POST /api/providers/openai/v1/assistants/:assistant_id/files
This endpoint is set up for creating an OpenAI assistant file. Documentation for this endpoint can be found here.
Retrieve assistant file: GET /api/providers/openai/v1/assistants/:assistant_id/files/:file_id
This endpoint is set up for retrieving an OpenAI assistant file. Documentation for this endpoint can be found here.
Delete assistant file: DELETE /api/providers/openai/v1/assistants/:assistant_id/files/:file_id
This endpoint is set up for deleting an OpenAI assistant file. Documentation for this endpoint can be found here.
List assistant files: GET /api/providers/openai/v1/assistants/:assistant_id/files
This endpoint is set up for retrieving OpenAI assistant files. Documentation for this endpoint can be found here.
Create thread: POST /api/providers/openai/v1/threads
This endpoint is set up for creating an OpenAI thread. Documentation for this endpoint can be found here.
Retrieve thread: GET /api/providers/openai/v1/threads/:thread_id
This endpoint is set up for retrieving an OpenAI thread. Documentation for this endpoint can be found here.
Modify thread: POST /api/providers/openai/v1/threads/:thread_id
This endpoint is set up for modifying an OpenAI thread. Documentation for this endpoint can be found here.
Delete thread: DELETE /api/providers/openai/v1/threads/:thread_id
This endpoint is set up for deleting an OpenAI thread. Documentation for this endpoint can be found here.
Create message: POST /api/providers/openai/v1/threads/:thread_id/messages
This endpoint is set up for creating an OpenAI message. Documentation for this endpoint can be found here.
Retrieve message: GET /api/providers/openai/v1/threads/:thread_id/messages/:message_id
This endpoint is set up for retrieving an OpenAI message. Documentation for this endpoint can be found here.
Modify message: POST /api/providers/openai/v1/files/:file_id/content
This endpoint is set up for modifying an OpenAI message. Documentation for this endpoint can be found here.
List messages: GET /api/providers/openai/v1/threads/:thread_id/messages
This endpoint is set up for listing OpenAI messages. Documentation for this endpoint can be found here.
Retrieve message file: GET /api/providers/openai/v1/threads/:thread_id/messages/:message_id/files/:file_id
This endpoint is set up for retrieving an OpenAI message file. Documentation for this endpoint can be found here.
List message files: GET /api/providers/openai/v1/threads/:thread_id/messages/:message_id/files
This endpoint is set up for retrieving OpenAI message files. Documentation for this endpoint can be found here.
Create run: POST /api/providers/openai/v1/threads/:thread_id/runs
This endpoint is set up for creating an OpenAI run. Documentation for this endpoint can be found here.
Retrieve run: GET /api/providers/openai/v1/threads/:thread_id/runs/:run_id
This endpoint is set up for retrieving an OpenAI run. Documentation for this endpoint can be found here.
Modify run: POST /api/providers/openai/v1/threads/:thread_id/runs/:run_id
This endpoint is set up for modifying an OpenAI run. Documentation for this endpoint can be found here.
List runs: GET /api/providers/openai/v1/threads/runs
This endpoint is set up for retrieving OpenAI runs. Documentation for this endpoint can be found here.
Submit tool outputs to run: POST /api/providers/openai/v1/threads/runs
This endpoint is set up for submitting tool outputs to an OpenAI run. Documentation for this endpoint can be found here.
Cancel a run: POST /api/providers/openai/v1/threads/:thread_id/runs/:run_id/cancel
This endpoint is set up for cancellling an OpenAI run. Documentation for this endpoint can be found here.
Create thread and run: POST /api/providers/openai/v1/threads/runs
This endpoint is set up for creating an OpenAI thread and run. Documentation for this endpoint can be found here.
Retrieve run step: GET /api/providers/openai/v1/threads/:thread_id/runs/:run_id/steps/:step_id
This endpoint is set up for retrieving an OpenAI run step. Documentation for this endpoint can be found here.
List run steps: GET /api/providers/openai/v1/threads/:thread_id/runs/:run_id/steps
This endpoint is set up for listing OpenAI run steps. Documentation for this endpoint can be found here.
Azure OpenAI Proxy
The custom provider proxy runs on Port 8002.
Create Azure OpenAI chat completion: POST /api/providers/azure/openai/deployments/:deploymentId/chat/completions?api-version={API_VERSION}
This endpoint is set up for proxying Anthropic completion requests. Documentation for this endpoint can be found here.
Create Azure OpenAI embeddings: POST /api/providers/azure/openai/deployments/:deploymentId/embeddings?api-version={API_VERSION}
This endpoint is set up for proxying Azure OpenAI completion requests. Documentation for this endpoint can be found here.
Anthropic Proxy
The custom provider proxy runs on Port 8002.
Create Anthropic completion: POST /api/providers/anthropic/v1/complete
This endpoint is set up for proxying Anthropic completion requests. Documentation for this endpoint can be found here.
Custom Provider Proxy
The custom provider proxy runs on Port 8002.
Call custom providers: POST /api/custom/providers/:provider/*
First you need to use create custom providers endpoint to create custom providers. Then create corresponding provider setting for the newly created custom provider. Afterward, you can start creating keys associated with the custom provider, and use the keys to access this endpoint by placing the created key in Authorization: Bearer YOUR_BRICKSLLM_KEY as part of your HTTP request headers.
Directories