metricsview

package
v0.78.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Dec 22, 2025 License: Apache-2.0 Imports: 16 Imported by: 0

README

runtime/metricsview/

This package contains the core logic for Rill's metrics layer (semantic layer). Namely, that includes validating metrics views and generating SQL queries for querying dimensions and measures defined in a metrics view.

Some other code files relevant to metrics views that are not found in this package:

  • proto/rill/runtime/v1/resources.proto: defines the spec for a metrics view
  • runtime/parser/parse_metrics_view.go: contains the logic for parsing a metrics view spec from a YAML file

Documentation

Index

Constants

View Source
const ChartsJSONSchema = `` /* 11543-byte string literal not displayed */

ChartsJSONSchema defines the JSON schema for chart specifications, Used by the MCP create_chart tool to validate chart configurations.

View Source
const QueryJSONSchema = `` /* 11145-byte string literal not displayed */

Variables

View Source
var ErrForbidden = errors.New("action not allowed")

ErrForbidden is returned when a query violates the constraints of MetricsViewSecurity.

View Source
var ExpressionJSONSchema = jsonschemautil.MustExtractDefAsSchema(QueryJSONSchema, "Expression")

Functions

func AnalyzeExpressionFields added in v0.78.0

func AnalyzeExpressionFields(e *Expression) []string

AnalyzeExpressionFields analyzes a metrics expression and returns the field names referenced in it.

func AnalyzeQueryFields added in v0.78.0

func AnalyzeQueryFields(q *Query) []string

AnalyzeQueryFields returns a list of all fields (dimensions and measures) that are part of the query.

func ExpressionToExport added in v0.64.1

func ExpressionToExport(e *Expression) (string, error)

func ExpressionToProto added in v0.78.0

func ExpressionToProto(expr *Expression) *runtimev1.Expression

func ExpressionToSQL added in v0.64.1

func ExpressionToSQL(e *Expression) (string, error)

func TypeSchemas added in v0.78.0

func TypeSchemas() map[reflect.Type]*jsonschema.Schema

TypeSchemas returns a map of JSON schemas for this package's query types (currently Query and Expression). This is designed to integrate with jsonschema.ForOptions to enable JSON schema inference on types that have sub-fields that use this package's types.

Types

type AST

type AST struct {
	// Root of the AST
	Root *SelectNode
	// List of CTEs to add to the query
	CTEs []*SelectNode

	// Contextual info for building the AST
	MetricsView *runtimev1.MetricsViewSpec
	Security    MetricsViewSecurity
	Query       *Query
	Dialect     drivers.Dialect
	// contains filtered or unexported fields
}

AST is the abstract syntax tree for a metrics SQL query.

func NewAST

func NewAST(mv *runtimev1.MetricsViewSpec, sec MetricsViewSecurity, qry *Query, dialect drivers.Dialect) (*AST, error)

NewAST builds a new SQL AST based on a metrics query.

Dynamic time ranges in the qry must be resolved to static start/end timestamps before calling this function. This is due to NewAST not being able (or intended) to resolve external time anchors such as watermarks.

The qry's PivotOn must be empty. Pivot queries must be rewritten/handled upstream of NewAST.

func (*AST) AddMeasureField added in v0.78.0

func (a *AST) AddMeasureField(n *SelectNode, m *runtimev1.MetricsViewSpec_Measure) error

AddMeasureField adds a measure field to the given SelectNode. Depending on the measure type, it may rewrite the SelectNode to accommodate the measure.

func (*AST) AddTimeRange added in v0.78.0

func (a *AST) AddTimeRange(n *SelectNode, tr *TimeRange) error

AddTimeRange adds a time range to the given SelectNode's WHERE clause.

func (*AST) ConvertToCTE added in v0.78.0

func (a *AST) ConvertToCTE(n *SelectNode)

ConvertToCTE util func that sets IsCTE and only adds to a.CTEs if IsCTE was false

func (*AST) GenerateIdentifier added in v0.78.0

func (a *AST) GenerateIdentifier() string

GenerateIdentifier generates a unique table identifier for use in the AST.

func (*AST) LookupDimension added in v0.78.0

func (a *AST) LookupDimension(name string, visible bool) (*runtimev1.MetricsViewSpec_Dimension, error)

lookupDimension finds a dimension spec in the metrics view. If visible is true, it returns an error if the security policy does not grant access to the dimension.

func (*AST) LookupMeasure added in v0.78.0

func (a *AST) LookupMeasure(name string, visible bool) (*runtimev1.MetricsViewSpec_Measure, error)

lookupMeasure finds a measure spec in the metrics view. If visible is true, it returns an error if the security policy does not grant access to the measure.

func (*AST) ResolveDimension added in v0.78.0

func (a *AST) ResolveDimension(qd Dimension, visible bool) (*runtimev1.MetricsViewSpec_Dimension, error)

ResolveDimension returns a dimension spec for the given dimension query. If the dimension query specifies a computed dimension, it constructs a dimension spec to match it.

func (*AST) ResolveMeasure added in v0.78.0

func (a *AST) ResolveMeasure(qm Measure, visible bool) (*runtimev1.MetricsViewSpec_Measure, error)

ResolveMeasure returns a measure spec for the given measure query. If the measure query specifies a computed measure, it constructs a measure spec to match it.

func (*AST) SQL

func (a *AST) SQL() (string, []any, error)

SQL builds a SQL query from the AST. It returns the query and query arguments to be passed to the database driver.

func (*AST) SQLForExpression added in v0.78.0

func (ast *AST) SQLForExpression(e *Expression, n *SelectNode, pseudoHaving, visible bool) (string, []any, error)

SQLForExpression generates a SQL expression for a query expression. pseudoHaving is true if the expression is allowed to reference measure expressions. visible is true if the expression is only allowed to reference dimensions and measures that are exposed by the security policy.

func (*AST) WrapSelect added in v0.78.0

func (a *AST) WrapSelect(s *SelectNode, innerAlias string)

WrapSelect rewrites the given node with a wrapping SELECT that includes the same dimensions and measures as the original node. The innerAlias is used as the alias of the inner SELECT in the new outer SELECT. Example: wrapSelect("SELECT a, count(*) as b FROM c", "t") -> "SELECT t.a, t.b FROM (SELECT a, count(*) as b FROM c) t".

type AnnotationsQuery added in v0.78.0

type AnnotationsQuery struct {
	MetricsView string     `json:"metrics_view" mapstructure:"metrics_view"`
	Measures    []string   `json:"measures" mapstructure:"measures"`
	TimeRange   *TimeRange `json:"time_range" mapstructure:"time_range"`
	Limit       *int64     `json:"limit" mapstructure:"limit"`
	Offset      *int64     `json:"offset" mapstructure:"offset"`
	TimeZone    string     `json:"time_zone" mapstructure:"time_zone"`
	TimeGrain   TimeGrain  `json:"time_grain" mapstructure:"time_grain"`
	Priority    int        `json:"priority" mapstructure:"priority"`
}

func (*AnnotationsQuery) AsMap added in v0.78.0

func (q *AnnotationsQuery) AsMap() (map[string]any, error)

type Condition

type Condition struct {
	Operator    Operator      `json:"op" mapstructure:"op"`
	Expressions []*Expression `json:"exprs" mapstructure:"exprs"`
}

type Dimension

type Dimension struct {
	Name    string            `json:"name" mapstructure:"name"`
	Compute *DimensionCompute `json:"compute" mapstructure:"compute"`
}

type DimensionCompute

type DimensionCompute struct {
	TimeFloor *DimensionComputeTimeFloor `json:"time_floor" mapstructure:"time_floor"`
}

type DimensionComputeTimeFloor

type DimensionComputeTimeFloor struct {
	Dimension string    `json:"dimension" mapstructure:"dimension"`
	Grain     TimeGrain `json:"grain" mapstructure:"grain"`
}

type ExprNode

type ExprNode struct {
	Expr string
	Args []any
}

ExprNode represents an expression for a WHERE clause.

func (*ExprNode) And added in v0.78.0

func (n *ExprNode) And(expr string, args []any) *ExprNode

And returns a new node that is the AND of the current node and the given expression.

type Expression

type Expression struct {
	Name      string     `json:"name" mapstructure:"name"`
	Value     any        `json:"val" mapstructure:"val"`
	Condition *Condition `json:"cond" mapstructure:"cond"`
	Subquery  *Subquery  `json:"subquery" mapstructure:"subquery"`
}

func NewExpressionFromProto

func NewExpressionFromProto(expr *runtimev1.Expression) *Expression

func (*Expression) AsMap added in v0.78.0

func (e *Expression) AsMap() (map[string]any, error)

type FieldNode

type FieldNode struct {
	Name        string
	DisplayName string
	Expr        string
	Unnest      bool
	TreatNullAs string // only used for measures
}

FieldNode represents a column in a SELECT clause. It also carries metadata related to the dimension/measure it was derived from. The Name must always match a the name of a dimension/measure in the metrics view or a computed field specified in the request. This means that if two columns in different places in the AST have the same Name, they're guaranteed to resolve to the same value.

type JoinType added in v0.47.1

type JoinType string

JoinType represents types of SQL joins.

const (
	JoinTypeUnspecified JoinType = ""
	JoinTypeFull        JoinType = "FULL OUTER"
	JoinTypeLeft        JoinType = "LEFT OUTER"
	JoinTypeRight       JoinType = "RIGHT OUTER"
	JoinTypeCross       JoinType = "CROSS"
)

type Measure

type Measure struct {
	Name    string          `json:"name" mapstructure:"name"`
	Compute *MeasureCompute `json:"compute" mapstructure:"compute"`
}

type MeasureCompute

type MeasureCompute struct {
	Count           bool                           `json:"count" mapstructure:"count"`
	CountDistinct   *MeasureComputeCountDistinct   `json:"count_distinct" mapstructure:"count_distinct"`
	ComparisonValue *MeasureComputeComparisonValue `json:"comparison_value" mapstructure:"comparison_value"`
	ComparisonDelta *MeasureComputeComparisonDelta `json:"comparison_delta" mapstructure:"comparison_delta"`
	ComparisonRatio *MeasureComputeComparisonRatio `json:"comparison_ratio" mapstructure:"comparison_ratio"`
	PercentOfTotal  *MeasureComputePercentOfTotal  `json:"percent_of_total" mapstructure:"percent_of_total"`
	URI             *MeasureComputeURI             `json:"uri" mapstructure:"uri"`
	ComparisonTime  *MeasureComputeComparisonTime  `json:"comparison_time" mapstructure:"comparison_time"`
}

func (*MeasureCompute) Validate

func (m *MeasureCompute) Validate() error

type MeasureComputeComparisonDelta

type MeasureComputeComparisonDelta struct {
	Measure string `json:"measure" mapstructure:"measure"`
}

type MeasureComputeComparisonRatio

type MeasureComputeComparisonRatio struct {
	Measure string `json:"measure" mapstructure:"measure"`
}

type MeasureComputeComparisonTime added in v0.78.0

type MeasureComputeComparisonTime struct {
	Dimension string `json:"dimension" mapstructure:"dimension"`
}

type MeasureComputeComparisonValue

type MeasureComputeComparisonValue struct {
	Measure string `json:"measure" mapstructure:"measure"`
}

type MeasureComputeCountDistinct

type MeasureComputeCountDistinct struct {
	Dimension string `json:"dimension" mapstructure:"dimension"`
}

type MeasureComputePercentOfTotal added in v0.49.0

type MeasureComputePercentOfTotal struct {
	Measure string   `json:"measure" mapstructure:"measure"`
	Total   *float64 `json:"total" mapstructure:"total"`
}

type MeasureComputeURI added in v0.49.0

type MeasureComputeURI struct {
	Dimension string `json:"dimension" mapstructure:"dimension"`
}

type MetricsViewSecurity added in v0.78.0

type MetricsViewSecurity interface {
	CanAccessField(field string) bool
	RowFilter() string
	QueryFilter() *runtimev1.Expression
}

MetricsViewSecurity defines access restrictions to a metrics view. The interface is currently a subset of the *runtime.ResolvedSecurity concrete type.

type Operator

type Operator string
const (
	OperatorUnspecified Operator = ""
	OperatorEq          Operator = "eq"
	OperatorNeq         Operator = "neq"
	OperatorLt          Operator = "lt"
	OperatorLte         Operator = "lte"
	OperatorGt          Operator = "gt"
	OperatorGte         Operator = "gte"
	OperatorIn          Operator = "in"
	OperatorNin         Operator = "nin"
	OperatorIlike       Operator = "ilike"
	OperatorNilike      Operator = "nilike"
	OperatorOr          Operator = "or"
	OperatorAnd         Operator = "and"
)

func (Operator) Valid

func (o Operator) Valid() bool

type OrderFieldNode

type OrderFieldNode struct {
	Name string
	Desc bool
}

OrderFieldNode represents a field in an ORDER BY clause.

type Query

type Query struct {
	MetricsView         string      `json:"metrics_view" mapstructure:"metrics_view"`
	Dimensions          []Dimension `json:"dimensions" mapstructure:"dimensions"`
	Measures            []Measure   `json:"measures" mapstructure:"measures"`
	PivotOn             []string    `json:"pivot_on" mapstructure:"pivot_on"`
	Spine               *Spine      `json:"spine" mapstructure:"spine"`
	Sort                []Sort      `json:"sort" mapstructure:"sort"`
	TimeRange           *TimeRange  `json:"time_range" mapstructure:"time_range"`
	ComparisonTimeRange *TimeRange  `json:"comparison_time_range" mapstructure:"comparison_time_range"`
	Where               *Expression `json:"where" mapstructure:"where"`
	Having              *Expression `json:"having" mapstructure:"having"`
	Limit               *int64      `json:"limit" mapstructure:"limit"`
	Offset              *int64      `json:"offset" mapstructure:"offset"`
	TimeZone            string      `json:"time_zone" mapstructure:"time_zone"`
	UseDisplayNames     bool        `json:"use_display_names" mapstructure:"use_display_names"`
	Rows                bool        `json:"rows" mapstructure:"rows"`
}

func (*Query) AsMap added in v0.57.1

func (q *Query) AsMap() (map[string]any, error)

func (*Query) Validate added in v0.60.7

func (q *Query) Validate() error

type SearchQuery added in v0.49.0

type SearchQuery struct {
	MetricsView string      `mapstructure:"metrics_view"`
	Dimensions  []string    `mapstructure:"dimensions"`
	Search      string      `mapstructure:"search"`
	Where       *Expression `mapstructure:"where"`
	Having      *Expression `mapstructure:"having"`
	TimeRange   *TimeRange  `mapstructure:"time_range"`
	Limit       *int64      `mapstructure:"limit"`
}

type SearchResult added in v0.49.0

type SearchResult struct {
	Dimension string
	Value     any
}

type SelectNode

type SelectNode struct {
	RawSelect            *ExprNode        // Raw SQL SELECT statement to use
	Alias                string           // Alias for the node used by outer SELECTs to reference it.
	IsCTE                bool             // Whether this node is a Common Table Expression
	DimFields            []FieldNode      // Dimensions fields to select
	MeasureFields        []FieldNode      // Measure fields to select
	FromTable            *string          // Underlying table expression to select from (if set, FromSelect must not be set)
	FromSelect           *SelectNode      // Sub-select to select from (if set, FromTable must not be set)
	SpineSelect          *SelectNode      // Sub-select that returns a spine of dimensions. Currently it will be right-joined onto FromSelect.
	LeftJoinSelects      []*SelectNode    // Sub-selects to left join onto FromSelect, to enable "per-dimension" measures
	CrossJoinSelects     []*SelectNode    // sub-selects to cross join onto FromSelect
	JoinComparisonSelect *SelectNode      // Sub-select to join onto FromSelect for comparison measures
	JoinComparisonType   JoinType         // Type of join to use for JoinComparisonSelect
	Unnests              []string         // Unnest expressions to add in the FROM clause
	Group                bool             // Whether the SELECT is grouped. If yes, it will group by all DimFields.
	Where                *ExprNode        // Expression for the WHERE clause
	TimeWhere            *ExprNode        // Expression for the time range to add to the WHERE clause
	Having               *ExprNode        // Expression for the HAVING clause. If HAVING is not allowed in the current context, it will added as a WHERE in a wrapping SELECT.
	OrderBy              []OrderFieldNode // Fields to order by
	Limit                *int64           // Limit for the query
	Offset               *int64           // Offset for the query
}

SelectNode represents a query that computes measures by dimensions. The from/join clauses are not all compatible. The allowed combinations are:

  • FromTable
  • FromSelect and optionally SpineSelect and/or LeftJoinSelects
  • FromSelect and optionally JoinComparisonSelect (for comparison CTE based optimization, this combination is used, both should be set and one of them will be used as CTE)

func (*SelectNode) HasMeasure added in v0.78.0

func (n *SelectNode) HasMeasure(name string) bool

HasMeasure checks if the given measure name is already present in the given SelectNode. See hasName for details about name checks.

func (*SelectNode) HasName added in v0.78.0

func (n *SelectNode) HasName(name string) bool

HasName checks if the given name is present as either a dimension or measure field in the node. It relies on field names always resolving to the same value regardless of where in the AST they're referenced. I.e. a name always corresponds to a dimension/measure name in the metrics view or as a computed field in the query. NOTE: Even if it returns false, the measure may still be present in a sub-select (which can happen if it was added as a referenced measure of a derived measure).

type Sort

type Sort struct {
	Name string `json:"name" mapstructure:"name"`
	Desc bool   `json:"desc" mapstructure:"desc"`
}

type Spine

type Spine struct {
	Where     *WhereSpine `json:"where" mapstructure:"where"`
	TimeRange *TimeSpine  `json:"time" mapstructure:"time"`
}

type Subquery

type Subquery struct {
	Dimension Dimension   `json:"dimension" mapstructure:"dimension"`
	Measures  []Measure   `json:"measures" mapstructure:"measures"`
	Where     *Expression `json:"where" mapstructure:"where"`
	Having    *Expression `json:"having" mapstructure:"having"`
}

type TimeGrain

type TimeGrain string
const (
	TimeGrainUnspecified TimeGrain = ""
	TimeGrainMillisecond TimeGrain = "millisecond"
	TimeGrainSecond      TimeGrain = "second"
	TimeGrainMinute      TimeGrain = "minute"
	TimeGrainHour        TimeGrain = "hour"
	TimeGrainDay         TimeGrain = "day"
	TimeGrainWeek        TimeGrain = "week"
	TimeGrainMonth       TimeGrain = "month"
	TimeGrainQuarter     TimeGrain = "quarter"
	TimeGrainYear        TimeGrain = "year"
)

func TimeGrainFromProto

func TimeGrainFromProto(t runtimev1.TimeGrain) TimeGrain

func (TimeGrain) ToProto

func (t TimeGrain) ToProto() runtimev1.TimeGrain

func (TimeGrain) ToTimeutil

func (t TimeGrain) ToTimeutil() timeutil.TimeGrain

func (TimeGrain) Valid

func (t TimeGrain) Valid() bool

type TimeRange

type TimeRange struct {
	Start         time.Time `json:"start" mapstructure:"start"`
	End           time.Time `json:"end" mapstructure:"end"`
	Expression    string    `json:"expression" mapstructure:"expression"`
	IsoDuration   string    `json:"iso_duration" mapstructure:"iso_duration"`
	IsoOffset     string    `json:"iso_offset" mapstructure:"iso_offset"`
	RoundToGrain  TimeGrain `json:"round_to_grain" mapstructure:"round_to_grain"`
	TimeDimension string    `json:"time_dimension" mapstructure:"time_dimension"` // optional time dimension to use for time-based operations, if not specified, the default time dimension in the metrics view is used
}

func NewTimeRangeFromProto added in v0.78.0

func NewTimeRangeFromProto(tr *runtimev1.TimeRange) *TimeRange

func (*TimeRange) AsMap added in v0.78.0

func (tr *TimeRange) AsMap() (map[string]any, error)

func (*TimeRange) IsZero

func (tr *TimeRange) IsZero() bool

type TimeSpine

type TimeSpine struct {
	Start         time.Time `json:"start" mapstructure:"start"`
	End           time.Time `json:"end" mapstructure:"end"`
	Grain         TimeGrain `json:"grain" mapstructure:"grain"`
	TimeDimension string    `json:"time_dimension" mapstructure:"time_dimension"` // optional time dimension to use for time-based operations, if not specified, the default time dimension in the metrics view is used
}

type TimestampsResult added in v0.54.0

type TimestampsResult struct {
	Min       time.Time
	Max       time.Time
	Watermark time.Time
	Now       time.Time
}

type WhereSpine

type WhereSpine struct {
	Expression *Expression `json:"expr" mapstructure:"expr"`
}

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL