README
¶
System Test Plan
General Guidelines
- Create a single testing package to hold all system tests.
- Establish a Test User so that generated files are in one directory.
- Give each test a project name that is unique and descriptive.
- Add info on memory usage and duration to log.
- Put request.yaml files into a directory, name each file using RequestName + StepName.
- Make bibleId a variable in each test.
- Give each request.yaml a func Test{RequestName}{StepName}.
- Each func can process the request with a variable list of BibleIds.
- There is a system wide list of BibleId's to test as well as an optional list in each test.
- There is a test harness that submits request to the server or cli.
- The test harness will always move output to a specific test output directory.
- Each test func will have some kind of verification, at least a record or line count.
- Each test function will have a comment that describes what manual verification is recommended.
Terminology
- Plain Text refers to the DBP plain text dataset.
- Plain Text Edit refers to DBP plain text with headings extracted from USX.
- USX Text Edit refers to USX text edited to contain only content in Script.
- Script refers to Excel spreadsheets used in process of recording Audio Bible.
- Whisper is a Speech to Text program produced by OpenAI.
- FastText is a Word Encoding program produced by Meta.
- Aeneas is a Timestamp generating program.
- Librosa is an MFCC generating program.
- Gonum/float is a go library for floating point
- Pandas is a python dataset manipulation tool frequently used by AI model researchers.
Script level text tests
-
01 Load Plain Text Script from BB-API, CSV output
-
02 Load Plain Text Edit from BB-API, JSON output
-
03 Load USX Text Edit from BB-API, CSV output
-
04 Load Script Using File, Sqlite output
-
05 Compare Plain Text to Plain Text Edit, HTML output
-
06 Compare Plain Text Edit to USX Edit, HTML output
-
07 Compare USX Edit to Script, HTML output (Mark Scott use case)
-
08 Compare Plain Text Edit to Script, HTML output
-
Note: Part of the purpose of these comparison tests is to verify that each kind of data parsing is correct.
Script level audio tests
-
09 Load Audio from BB-API do Whisper speech to text, output the text as JSON
-
10 Compare text to USXEdit, output HTML (Mark Scott use case)
-
11 Load Audio from POST, do speech to text, output JSON (OBT use case)
-
12 Reuse request with second and third passage, output JSON (OBT use case)
-
13 Load Project Text with Plain Text Edit, because it is verse aligned. Load time stamps from BB-API, Output CSV
-
14 Load same Text, and timestamp with aeneas, compare the timestamps using gonum/float to find in difference they are equal (float.SubTo, float.Abs, float.Max. And, I need to look at Sandeep’s method
-
15 Load Script and script timestamps, compare with above case where the script is verse aligned.
-
16 Output sql script to insert a fileset's timestamp data into DBP (Jon's OBT use case)
-
Note: The comparison code to do this has not yet been written (5/1/24)
-
17 Load audios and compute timestamps and mfcc. Verification is limited to record counts. output CSV
-
18 Repeat and output JSON
-
19 Load CSV file into Pandas
-
20 Load Json file into Pandas
Word level text tests
- Using existing Plain Text Edit, generate words, validate word split, fast text words, generate CSV
- Using existing USX Text Edit, generate words, validate word split, fast text words, generate JSON
- Load CSV file into Pandas
- Load Json file into Pandas
Word level audio tests
- Load audio into Word Plain Text Test, timestamp words using script level timestamps, generate MFCC, output CSV
- Load audio into Word USX Text Edit, timestamp words using script level timestamps, generate MFCC, output JSON
- Load CSV output into Pandas
- Load JSON output into Pandas
Full Test
- Load USX Text Edit, Audio, timestamp, compute MFCC, word Split, timestamp words, compute MFCC of words, fast text encode words, output JSON.
- Output CSV on same database.
Error Tests
- List here a number of error conditions, output error
Documentation
¶
Index ¶
- Constants
- func APITestUtility(request string, tests []APITest, t *testing.T)
- func CLIExec(requestYaml string, t *testing.T) (string, string)
- func DirectSqlTest(request string, tests []SqliteTest, t *testing.T) string
- func DirectTestUtility(requestYaml string, tests []CtlTest, t *testing.T)
- func ExtractFilename(stdout string) string
- func FCBHDatasetExec(requestYaml string, t *testing.T) (string, string)
- func NumCVSFileLines(filename string, t *testing.T) int
- func NumCVSLines(content []byte, t *testing.T) int
- func NumFileLines(filename string, t *testing.T) int
- func NumHTMLFileLines(filename string, t *testing.T) int
- func NumJSONFileLines(filename string, t *testing.T) int
- func NumJSONLines(content []byte, t *testing.T) int
- func SelectScalarInt(conn *sql.DB, sql string, t *testing.T) int
- type APITest
- type CtlTest
- type SqliteTest
Constants ¶
const ( HOST = `http://localhost:8080/` UPLOADHOST = `http://localhost:7777/upload` OUTPUT = `/Users/gary/FCBH2024/systemtest/` )
Variables ¶
This section is empty.
Functions ¶
func DirectSqlTest ¶
func DirectSqlTest(request string, tests []SqliteTest, t *testing.T) string
DirectSqlTest requires a sqlite database as output to perform tests on the result
Source Files