morrigan

Morrigan

OpenAI/ChatGPT Backend with conversation and API

Features

• Import documents of knowledge base from a table (CSV), save them into PostgreSQL
• Based on the title and content of the document, generate vector of documents with Embedding API
• Summarize the questions and generate corresponding vectors
• Implement high-quality Q&A using vector search
• Welcome message and preset messages
• Chat History for Conversation (based on redis)
• RESTful API
• Support text/event-stream
• Login with OAuth2 client for general Security Provider
• Inner MCP support

Supported Frontend

APIs

Get welcome message and new conversation ID

Get user information of that has been verified or signed in

Post chat prompt and return Streaming messages

Getting started

test -e .env || cp .env.example .env
# Edit .env and change api key of OpenAI
# Embedding and replacing frontend resources

make deps

forego start

# or

make dist

Prepare preset data file in YAML

welcome:
  content: Hello, I am your virtual assistant. How can I help you?
  role: assistant

messages:
  - content: You are a helpful assistant.
    role: system
  - content: When is my birthday?
    role: user
  - content: How would I know?
    role: assistant

 # more messages

Prepare database

CREATE USER morrigan WITH LOGIN PASSWORD 'mydbusersecret';
CREATE DATABASE morrigan WITH OWNER = morrigan ENCODING = 'UTF8';
GRANT ALL PRIVILEGES ON DATABASE morrigan to morrigan;


\c morrigan

-- optional: install extension from https://github.com/pgvector/pgvector
CREATE EXTENSION vector;

Command line usage

USAGE:
   morrigan [global options] command [command options] [arguments...]

COMMANDS:
   usage, env                   show usage
   initdb                       init database schema
   import                       import documents from a csv
   embedding, embedding-pormpt  read prompt documents and embedding
   web, run                     run a web server
   help, h                      Shows a list of commands or help for one command

GLOBAL OPTIONS:
   --help, -h  show help

Change settings with environment

Show all local settings

go run . usage

Example:

MORRIGAN_OPENAI_API_KEY=oo-xx
MORRIGAN_HTTP_LISTEN=:3002

# optional preset data
MORRIGAN_PRESET_FILE=./data/preset.yaml

# optional OAuth2 login
MORRIGAN_AUTH_REQUIRED=true
OAUTH_PREFIX=https://portal.my-company.xyz

# optional proxy
HTTPS_PROXY=socks5://proxy.my-company.xyz:1081

The operation steps for generating data.

1. Prepare a CSV file for the corpus document.
2. Import documents.
3. Generate Questions and Anwsers from documents with Completion.
4. Generate Prompts and vector from QAs with Embedding
5. Done and go to chat

CSV template of documents

go run . initdb
go run . import mycompany.csv
go run . embedding

Attach frontend resources

1. Go to frontend project directory
2. Build frontend pages and accompanying static resources.
3. Copy them into ./htdocs

Example:

cd ../chatgpt-svelte
npm run build
rsync -a --delete dist/* ../morrigan/htdocs/
cd -

During the development and debugging phase, you can still use with proxy to collaborate with the front-end project.