fireboltgosdk

Firebolt Go SDK

Connect to Firebolt using Go. The Firebolt Go driver is an implementation of database/sql/driver.

Installation

go get github.com/firebolt-db/firebolt-go-sdk

DSN (Data source name)

Cloud instance

All information for the connection should be specified using the DSN string. The firebolt dsn string has the following format:

firebolt://[/database]?account_name=account_name&client_id=client_id&client_secret=client_secret[&engine=engine]

• client_id - client id of the service account.
• client_secret - client secret of the service account.
• account_name - the name of Firebolt account to log in to.
• database - (optional) the name of the database to connect to.
• engine - (optional) the name of the engine to run SQL on.

Core instance

For the core instance, the DSN string has the following format:

firebolt://[/database]?url=core_instance_url[&client_side_lb=false]

• url - the URL of the core instance to connect to. It should contain the full URL, including schema. E.g. http://localhost:3473.
• database - (optional) the name of the database to connect to.
• client_side_lb - (optional, default true) enables client-side round-robin load balancing. The SDK resolves the hostname in url to its underlying IP addresses and distributes requests across them. This prevents Go's default connection pooling from pinning all requests to a single pod when url points to a Kubernetes service. Set to false to disable.
• client_side_lb_dns_ttl - (optional, default 30s) how often the round-robin resolver re-resolves the hostname to discover new or removed nodes. Accepts any Go duration string (e.g. 5s, 500ms, 2m). Lower values give faster failover; higher values reduce DNS traffic. Only takes effect when client_side_lb is enabled.

Custom HTTP transport

The SDK ships with sensible HTTP transport defaults (30s dial timeout, 10s TLS handshake timeout, 30s keep-alive, 90s idle connection timeout). If you need to tune these -- for example, to increase the dial timeout for high-latency networks or the idle connection timeout for long-lived batch pipelines -- use OpenConnectorWithDSN together with WithTransport:

package main

import (
	"database/sql"
	"log"
	"net"
	"time"

	firebolt "github.com/firebolt-db/firebolt-go-sdk"
	"github.com/firebolt-db/firebolt-go-sdk/client"
)

func main() {
	// Start from the SDK defaults and override what you need.
	transport := client.DefaultTransport()
	transport.DialContext = (&net.Dialer{
		Timeout:   60 * time.Second,
		KeepAlive: 60 * time.Second,
	}).DialContext
	transport.TLSHandshakeTimeout = 20 * time.Second
	transport.IdleConnTimeout = 2 * time.Minute

	dsn := "firebolt:///mydb?url=http://firebolt:3473&client_side_lb_dns_ttl=5s"

	// OpenConnectorWithDSN parses the DSN and applies driver options.
	connector, err := firebolt.OpenConnectorWithDSN(dsn, firebolt.WithTransport(transport))
	if err != nil {
		log.Fatal(err)
	}

	// sql.OpenDB returns the same *sql.DB you get from sql.Open,
	// but with your custom transport in effect.
	db := sql.OpenDB(connector)
	defer db.Close()

	// Use db as usual ...
}

client.DefaultTransport() returns a new *http.Transport each time, so you can safely create different transports for different use cases. WithTransport accepts any http.RoundTripper, so you can also wrap the transport with middleware (e.g. otelhttp.NewTransport for OpenTelemetry tracing). When no custom transport is provided (i.e. when using sql.Open), the SDK uses its built-in defaults.

Querying example

Here is an example of establishing a connection and executing a simple select query. For it to run successfully, you have to specify your credentials, and have a default engine up and running.

package main

import (
	"database/sql"
	"fmt"
	"log"

	// we need to import firebolt-go-sdk in order to register the driver
	_ "github.com/firebolt-db/firebolt-go-sdk"
)

func main() {

	// set your Firebolt credentials to construct a dsn string
	clientId := ""
	clientSecret := ""
	accountName := ""
	databaseName := ""
	engineName := ""
	dsn := fmt.Sprintf("firebolt:///%s?account_name=%s&client_id=%s&client_secret=%s&engine=%s", databaseName, accountName, clientId, clientSecret, engineName)

	// open a Firebolt connection
	db, err := sql.Open("firebolt", dsn)
	if err != nil {
		log.Fatalf("error during opening a driver: %v", err)
	}

	// create a table
	_, err = db.Query("CREATE TABLE test_table(id INT, value TEXT)")
	if err != nil {
		log.Fatalf("error during select query: %v", err)
	}

	// execute a parametrized insert (only ? placeholders are supported)
	_, err = db.Query("INSERT INTO test_table VALUES (?, ?)", 1, "my value")
	if err != nil {
		log.Fatalf("error during select query: %v", err)
	}

	// execute a simple select query
	rows, err := db.Query("SELECT id FROM test_table")
	if err != nil {
		log.Fatalf("error during select query: %v", err)
	}

	// iterate over the result
	defer func() {
		if err := rows.Close(); err != nil {
			log.Printf("error during rows.Close(): %v\n", err)
		}
	}()

	for rows.Next() {
		var id int
		if err := rows.Scan(&id); err != nil {
			log.Fatalf("error during scan: %v", err)
		}
		log.Print(id)
	}

	if err := rows.Err(); err != nil {
		log.Fatalf("error during rows iteration: %v\n", err)
	}
}

Streaming example

In order to stream the query result (and not store it in memory fully), you need to pass a special context with streaming enabled.

Warning: If you enable streaming the result, the query execution might finish successfully, but the actual error might be returned during the iteration over the rows.

Here is an example of how to do it:

package main

import (
	"context"
	"database/sql"
	"fmt"
	"log"

	// we need to import firebolt-go-sdk in order to register the driver
	_ "github.com/firebolt-db/firebolt-go-sdk"
	fireboltContext "github.com/firebolt-db/firebolt-go-sdk/context"
)

func main() {
	// set your Firebolt credentials to construct a dsn string
	clientId := ""
	clientSecret := ""
	accountName := ""
	databaseName := ""
	engineName := ""
	dsn := fmt.Sprintf("firebolt:///%s?account_name=%s&client_id=%s&client_secret=%s&engine=%s", databaseName, accountName, clientId, clientSecret, engineName)

	// open a Firebolt connection
	db, err := sql.Open("firebolt", dsn)
	if err != nil {
		log.Fatalf("error during opening a driver: %v", err)
	}

	// create a streaming context
	streamingCtx := fireboltContext.WithStreaming(context.Background())

	// execute a large select query
	rows, err := db.QueryContext(streamingCtx, "SELECT 'abc' FROM generate_series(1, 100000000)")
	if err != nil {
		log.Fatalf("error during select query: %v", err)
	}

	// iterating over the result is exactly the same as in the previous example
	defer func() {
		if err := rows.Close(); err != nil {
			log.Printf("error during rows.Close(): %v\n", err)
		}
	}()

	for rows.Next() {
		var data string
		if err := rows.Scan(&data); err != nil {
			log.Fatalf("error during scan: %v", err)
		}
		log.Print(data)
	}

	if err := rows.Err(); err != nil {
		log.Fatalf("error during rows iteration: %v\n", err)
	}
}

Errors in streaming

If you enable streaming the result, the query execution might finish successfully, but the actual error might be returned during the iteration over the rows.

Prepared statements

The SDK supports two types of prepared statements:

1. Native - client-side prepared statements. Uses ? as a placeholder for parameters.
2. FBNumeric - server-side prepared statements. Uses $i as a placeholder for parameters.

You can manually create a prepared statement using the Prepare method of the sql.DB object. You can also use the Exec method of the sql.DB object to execute a prepared statement directly without creating it first.

Example of client-side prepared statements

package main
import (
    "database/sql"
    "fmt"
    "log"

    // we need to import firebolt-go-sdk in order to register the driver
    _ "github.com/firebolt-db/firebolt-go-sdk"
)
func main() {
    // set your Firebolt credentials to construct a dsn string
    clientId := ""
    clientSecret := ""
    accountName := ""
    databaseName := ""
    engineName := ""
    dsn := fmt.Sprintf("firebolt:///%s?account_name=%s&client_id=%s&client_secret=%s&engine=%s", databaseName, accountName, clientId, clientSecret, engineName)

	// open a Firebolt connection
	db, err := sql.Open("firebolt", dsn)
	if err != nil {
		log.Fatalf("error during opening a driver: %v", err)
	}
	defer db.Close()
	log.Printf("successfully opened a driver with dsn: %s", dsn)

	// Initially created client-side prepared statement
	nativeStmt, err := db.Prepare("INSERT INTO test_table VALUES (?, ?)")
	if err != nil {
		log.Fatalf("error preparing native statement: %v", err)
	}
	defer nativeStmt.Close()
	log.Printf("successfully prepared a native statement")

	_, err = nativeStmt.Exec(1, "value")
	if err != nil {
		log.Fatalf("error executing native prepared statement: %v", err)
	}
	log.Printf("successfully executed native prepared statement with args: 1, \"value\"")

	// Executing the same statement directly using Exec
	_, err = db.Exec("INSERT INTO test_table VALUES (?, ?)", 2, "another value")
	if err != nil {
		log.Fatalf("error executing native prepared statement directly: %v", err)
	}
	log.Printf("successfully executed native prepared statement directly with args: 2, \"another_value\"")
}

Example of server-side prepared statements

package main

import (
	"context"
	"database/sql"
	"fmt"
	"log"

	// we need to import firebolt-go-sdk in order to register the driver
	_ "github.com/firebolt-db/firebolt-go-sdk"
	fireboltContext "github.com/firebolt-db/firebolt-go-sdk/context"
)

func main() {
	// set your Firebolt credentials to construct a dsn string
	clientId := ""
	clientSecret := ""
	accountName := ""
	databaseName := ""
	engineName := ""
	dsn := fmt.Sprintf("firebolt:///%s?account_name=%s&client_id=%s&client_secret=%s&engine=%s", databaseName, accountName, clientId, clientSecret, engineName)

	// open a Firebolt connection
	db, err := sql.Open("firebolt", dsn)
	if err != nil {
		log.Fatalf("error during opening a driver: %v", err)
	}
	defer db.Close()
	log.Printf("successfully opened a driver with dsn: %s", dsn)

	// We need to specify the prepared statement style in the context. Native is used by default.
	serverSideCtx := fireboltContext.WithPreparedStatementsStyle(context.Background(), fireboltContext.PreparedStatementsStyleFbNumeric)

	// Initially created server-side prepared statement
	fbnumericStmt, err := db.PrepareContext(serverSideCtx, "INSERT INTO test_table VALUES ($1, $2)")
	if err != nil {
		log.Fatalf("error preparing FBNumeric statement: %v", err)
	}
	defer fbnumericStmt.Close()
	log.Printf("successfully prepared a native statement")

	_, err = fbnumericStmt.Exec(1, "value")
	if err != nil {
		log.Fatalf("error executing FBNumeric prepared statement: %v", err)
	}
	log.Printf("successfully executed native prepared statement with args: 1, \"value\"")

	// Executing the same statement directly using Exec
	_, err = db.ExecContext(serverSideCtx, "INSERT INTO test_table VALUES ($1, $2)", 2, "another value")
	if err != nil {
		log.Fatalf("error executing FBNumeric prepared statement directly: %v", err)
	}
	log.Printf("successfully executed native prepared statement directly with args: 2, \"another_value\"")
}

Server-side asynchronous execution

The SDK supports server-side asynchronous execution of queries. This allows you to execute long-running queries on the background and retrieve the results later.

Note: the asynchronous execution does not support returning query results. It is useful for long-running queries that do not require immediate results, such as data loading or transformation tasks.

Here is an example of how to use server-side asynchronous execution:

package main

import (
    "context"
    "database/sql"
    "fmt"
    "log"

    firebolt "github.com/firebolt-db/firebolt-go-sdk"
)

func main() {
    // set your Firebolt credentials to construct a dsn string
    clientId := ""
    clientSecret := ""
    accountName := ""
    databaseName := ""
    engineName := ""
    dsn := fmt.Sprintf("firebolt:///%s?account_name=%s&client_id=%s&client_secret=%s&engine=%s", databaseName, accountName, clientId, clientSecret, engineName)

    // open a Firebolt connection
    db, err := sql.Open("firebolt", dsn)
    if err != nil {
        log.Fatalf("error during opening a driver: %v", err)
    }
    defer db.Close()
    log.Printf("successfully opened a driver with dsn: %s", dsn)


    token, err := firebolt.ExecAsync(db, "INSERT INTO test_table VALUES (?, ?)", 1, "async value")
    if err != nil {
        log.Fatalf("error executing asynchronous query: %v", err)
    }

	log.Print("started asynchronous query execution")

	for {
		running, err := firebolt.IsAsyncQueryRunning(db, token)
		if err != nil {
			log.Fatalf("Failed to check async query status: %v", err)
		}
		if !running {
			break
		}
    }

	success, err := firebolt.IsAsyncQuerySuccessful(db, token)
	if err != nil {
		log.Fatalf("Failed to check async query success: %v", err)
	}
	if success {
		log.Println("asynchronous query executed successfully")
	} else {
        log.Println("asynchronous query failed")
    }
}

Transaction support

The SDK supports transactions using the sql.Tx interface. Firebolt uses snapshot isolation for DML (INSERT, UPDATE, DELETE) statements and strict serializability for DDL (CREATE, ALTER, DROP) statements. Here is an example of how to use transactions:

package main

import (
    "context"
    "database/sql"
    "fmt"
    "log"

    firebolt "github.com/firebolt-db/firebolt-go-sdk"
)

func main() {
	// set your Firebolt credentials to construct a dsn string
	clientId := ""
	clientSecret := ""
	accountName := ""
	databaseName := ""
	engineName := ""
	dsn := fmt.Sprintf("firebolt:///%s?account_name=%s&client_id=%s&client_secret=%s&engine=%s", databaseName, accountName, clientId, clientSecret, engineName)

	// open a Firebolt connection
	db, err := sql.Open("firebolt", dsn)
	if err != nil {
		log.Fatalf("error during opening a driver: %v", err)
	}
	defer db.Close()
	log.Printf("successfully opened a driver with dsn: %s", dsn)

	// Start a transaction
	tx, err := db.Begin()
	if err != nil {
        log.Fatalf("error starting transaction: %v", err)
    }

	// Execute a sql query within the transaction
	_, err = tx.ExecContext(context.Background(), "INSERT INTO test_table VALUES (?, ?)", 1, "value")

	// Rollback the transaction if there was an error
	if err != nil {
        log.Printf("error executing query within transaction: %v", err)
        if rollbackErr := tx.Rollback(); rollbackErr != nil {
            log.Fatalf("error rolling back transaction: %v", rollbackErr)
        }
        return
    }

	// Commit the transaction if everything was successful
	if err = tx.Commit(); err != nil {
        log.Fatalf("error committing transaction: %v", err)
    } else {
        log.Println("transaction committed successfully")
    }
}

Batch insert

The SDK supports high-performance batch insertion. Data is buffered client-side, serialised to Parquet, and uploaded via multipart form POST when Send() is called. Two modes are available and can be mixed freely.

Access the batch API by unwrapping the raw driver connection via (*sql.Conn).Raw:

Row-wise insertion

Append one row at a time — convenient when iterating over records:

package main

import (
	"context"
	"database/sql"
	"fmt"
	"log"

	firebolt "github.com/firebolt-db/firebolt-go-sdk"
)

func main() {
	dsn := "firebolt:///mydb?account_name=acct&client_id=id&client_secret=secret&engine=eng"
	db, err := sql.Open("firebolt", dsn)
	if err != nil {
		log.Fatal(err)
	}
	defer db.Close()

	ctx := context.Background()
	conn, err := db.Conn(ctx)
	if err != nil {
		log.Fatal(err)
	}
	defer conn.Close()

	err = conn.Raw(func(driverConn interface{}) error {
		batch, err := driverConn.(firebolt.BatchConnection).PrepareBatch(
			ctx, "INSERT INTO events (id, name, active)")
		if err != nil {
			return err
		}

		// Append one row at a time
		for i := int32(0); i < 1000; i++ {
			if err := batch.Append(i, fmt.Sprintf("event_%d", i), true); err != nil {
				return err
			}
		}

		return batch.Send(ctx)
	})
	if err != nil {
		log.Fatalf("batch insert failed: %v", err)
	}
}

Columnar insertion

Append entire typed slices per column — ideal when data is already in columnar layout:

err = conn.Raw(func(driverConn interface{}) error {
    batch, err := driverConn.(firebolt.BatchConnection).PrepareBatch(
        ctx, "INSERT INTO events (id, name, active)")
    if err != nil {
        return err
    }

    // Append whole columns at once
    if err := batch.Column(0).Append([]int32{1, 2, 3}); err != nil {
        return err
    }
    if err := batch.Column(1).Append([]string{"a", "b", "c"}); err != nil {
        return err
    }
    if err := batch.Column(2).Append([]bool{true, false, true}); err != nil {
        return err
    }

    return batch.Send(ctx)
})

Notes

• PrepareBatch requires an explicit column list in the INSERT statement (e.g. INSERT INTO t (col1, col2)). Column types are discovered automatically.
• Both modes can be mixed in the same batch; all columns must have the same number of rows when Send() is called.
• After a successful Send() the batch is reset and can be reused for another round of appends.
• Call Abort() to discard buffered data without sending.
• Supported column types: int/integer, long/bigint, float/real, double, text, boolean, date, timestamp, timestampntz, timestamptz, bytea, array(T), and nullable variants.

Error handling

The SDK provides specific error types that can be checked using Go's errors.Is() function. Here's how to handle different types of errors:

package main

import (
	"database/sql"
	"errors"
	"fmt"
	"log"

	_ "github.com/firebolt-db/firebolt-go-sdk"
	fireboltErrors "github.com/firebolt-db/firebolt-go-sdk/errors"
)

func main() {
	// set your Firebolt credentials to construct a dsn string
	clientId := ""
	clientSecret := ""
	accountName := ""
	databaseName := ""
	engineName := ""

	// Example 1: Invalid DSN format (using account-name instead of account_name)
	invalidDSN := fmt.Sprintf("firebolt:///%s?account-name=%s&client_id=%s&client_secret=%s&engine=%s",
		databaseName, accountName, clientId, clientSecret, engineName)
	db, err := sql.Open("firebolt", invalidDSN)
	if err != nil {
		if errors.Is(err, fireboltErrors.DSNParseError) {
			log.Println("Invalid DSN format, please update your DSN and try again")
		} else {
			log.Fatalf("Unexpected error type: %v", err)
		}
	}

	// Example 2: Invalid credentials
	invalidCredentialsDSN := fmt.Sprintf("firebolt:///%s?account_name=%s&client_id=%s&client_secret=%s&engine=%s",
		databaseName, accountName, "invalid", "invalid", engineName)
	db, err = sql.Open("firebolt", invalidCredentialsDSN)
	if err != nil {
		if errors.Is(err, fireboltErrors.AuthenticationError) {
			log.Println("Authentication error. Please check your credentials and try again")
		} else {
			log.Fatalf("Unexpected error type: %v", err)
		}
	}

	// Example 3: Invalid account name
    invalidAccountDSN := fmt.Sprintf("firebolt:///%s?account_name=%s&client_id=%s&client_secret=%s&engine=%s",
        databaseName, "invalid", clientId, clientSecret, engineName)
    db, err = sql.Open("firebolt", invalidAccountDSN)
	if err != nil {
        if errors.Is(err, fireboltErrors.InvalidAccountError) {
            log.Println("Invalid account name. Please check your account name and try again")
        } else {
            log.Fatalf("Unexpected error type: %v", err)
        }
    }

	// Example 4: Invalid SQL query
	dsn := fmt.Sprintf("firebolt:///%s?account_name=%s&client_id=%s&client_secret=%s&engine=%s",
		databaseName, accountName, clientId, clientSecret, engineName)
	db, err = sql.Open("firebolt", dsn)
	if err != nil {
		log.Fatalf("Failed to open connection: %v", err)
	}
	defer db.Close()

	// Try to execute an invalid SQL query
	_, err = db.Query("SELECT * FROM non_existent_table")
	if err != nil {
		if errors.Is(err, fireboltErrors.QueryExecutionError) {
			log.Printf("Error during query execution. Please fix your SQL query and try again")
		} else {
			log.Fatalf("Unexpected error type: %v", err)
		}
	}
}

The SDK provides the following error types:

• DSNParseError: Provided DSN string format is invalid
• AuthenticationError: Authentication failure
• QueryExecutionError: SQL query execution error
• AuthorizationError:A user doesn't have permission to perform an action
• InvalidAccountError: Provided account name is invalid or no permissions to access the account

Each error type can be checked using errors.Is(err, errorType). This allows for specific error handling based on the type of error encountered.

Limitations

Although, all interfaces are available, not all of them are implemented or could be implemented:

• driver.Result is a dummy implementation and doesn't return the real result values.
• Named query parameters are not supported.
• Batch insert requires an explicit column list; omitting it (as INSERT INTO t) is not supported.
• AppendStruct (struct-based batch insertion) is not supported.
• Decimal and nested struct column types are not supported in batch inserts.