Documentation
¶
Overview ¶
Package lines provides a virtual table to read data line-by-line.
It is particularly useful for line-oriented datasets, like ndjson or JSON Lines, when paired with SQLite's JSON support.
https://github.com/asg017/sqlite-lines
Example ¶
package main
import (
"database/sql"
"fmt"
"log"
"net/http"
"github.com/ncruces/go-sqlite3"
"github.com/ncruces/go-sqlite3/driver"
"github.com/ncruces/go-sqlite3/ext/lines"
_ "github.com/ncruces/go-sqlite3/embed"
)
func main() {
db, err := driver.Open(":memory:", func(c *sqlite3.Conn) error {
lines.Register(c)
return nil
})
if err != nil {
log.Fatal(err)
}
defer db.Close()
res, err := http.Get("https://storage.googleapis.com/quickdraw_dataset/full/simplified/calendar.ndjson")
if err != nil {
log.Fatal(err)
}
defer res.Body.Close()
rows, err := db.Query(`
SELECT
line ->> '$.countrycode' as countrycode,
COUNT(*)
FROM lines_read(?)
GROUP BY 1
ORDER BY 2 DESC
LIMIT 5`,
sqlite3.Pointer(res.Body))
if err != nil {
log.Fatal(err)
}
defer rows.Close()
var countrycode sql.RawBytes
var count int
for rows.Next() {
err := rows.Scan(&countrycode, &count)
if err != nil {
log.Fatal(err)
}
fmt.Printf("%s: %d\n", countrycode, count)
}
if err := rows.Err(); err != nil {
log.Fatal(err)
}
}
Output: US: 141001 GB: 22560 CA: 11759 RU: 9250 DE: 8748
Index ¶
Examples ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func Register ¶
Register registers the lines and lines_read table-valued functions. The lines function reads from a database blob or text. The lines_read function reads from a file or an io.Reader. If a filename is specified, os.Open is used to open the file.
func RegisterFS ¶ added in v0.12.0
RegisterFS registers the lines and lines_read table-valued functions. The lines function reads from a database blob or text. The lines_read function reads from a file or an io.Reader. If a filename is specified, fsys is used to open the file.
Types ¶
This section is empty.
Source Files