Documentation
¶
Index ¶
- type CachedStructuredLogEstimator
- func (e *CachedStructuredLogEstimator) Close()
- func (e *CachedStructuredLogEstimator) EstimateWithTaskSlot(ctx context.Context, taskSlotKey string, ...) (*EstimateResult, error)
- func (e *CachedStructuredLogEstimator) EstimateWithTaskSlotNonBlocking(ctx context.Context, taskSlotKey string, ...) (*EstimateResult, bool, error)
- type EstimateResult
- type EstimatedCountPreset
- type EstimatorProvider
- type LogProbeClient
- type LogSamplingProbeFetcher
- type LoggingClientProbeFetcher
- type LoggingMonitoringMatcher
- func Comment(comment string) LoggingMonitoringMatcher
- func CustomFilter(rawQuery string) LoggingMonitoringMatcher
- func LogID(vm ValueMatcher) LoggingMonitoringMatcher
- func ResourceLabel(labelKey string, vm ValueMatcher) LoggingMonitoringMatcher
- func WithComment(matcher LoggingMonitoringMatcher, comment string) LoggingMonitoringMatcher
- type MetricLogCountFetcher
- type MonitoringClientFetcher
- type StructuredLogEstimator
- type StructuredLogQuery
- type ValueMatcher
- func ContainsAny(subs ...string) ValueMatcher
- func Exact(val string) ValueMatcher
- func FromSetFilter(filter *gcpqueryutil.SetFilterParseResult, isSubstring bool) ValueMatcher
- func NoneOf(vals ...string) ValueMatcher
- func NotContainsAny(subs ...string) ValueMatcher
- func OneOf(vals ...string) ValueMatcher
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type CachedStructuredLogEstimator ¶
type CachedStructuredLogEstimator struct {
// contains filtered or unexported fields
}
CachedStructuredLogEstimator manages cached log volume estimates, singleflight in-flight deduplication, and cancellation of outdated in-flight estimation queries per task slot.
func NewCachedStructuredLogEstimator ¶
func NewCachedStructuredLogEstimator() *CachedStructuredLogEstimator
NewCachedStructuredLogEstimator creates a new CachedStructuredLogEstimator instance.
func (*CachedStructuredLogEstimator) Close ¶
func (e *CachedStructuredLogEstimator) Close()
Close cancels all active in-flight requests and clears the estimation cache.
func (*CachedStructuredLogEstimator) EstimateWithTaskSlot ¶
func (e *CachedStructuredLogEstimator) EstimateWithTaskSlot( ctx context.Context, taskSlotKey string, container googlecloud.ResourceContainer, query *StructuredLogQuery, startTime, endTime time.Time, provider EstimatorProvider, ) (*EstimateResult, error)
EstimateWithTaskSlot estimates the log volume for the given query, returning cached results when available. If a query for the same input is already in flight, it deduplicates the request and waits for the shared result. If a previous in-flight estimation for taskSlotKey has a different query or time range, it cancels the previous request.
func (*CachedStructuredLogEstimator) EstimateWithTaskSlotNonBlocking ¶
func (e *CachedStructuredLogEstimator) EstimateWithTaskSlotNonBlocking( ctx context.Context, taskSlotKey string, container googlecloud.ResourceContainer, query *StructuredLogQuery, startTime, endTime time.Time, provider EstimatorProvider, ) (*EstimateResult, bool, error)
EstimateWithTaskSlotNonBlocking estimates the log volume for the given query asynchronously without blocking the caller. It returns (res, false, nil) if the result is already cached. It returns (nil, true, nil) if the estimation is currently in-flight (pending). It returns (nil, false, err) if an error occurred during estimation.
type EstimateResult ¶
type EstimateResult struct {
// MetricCount is the base count obtained from Cloud Monitoring metric.
MetricCount int64 `json:"metricCount"`
// EstimatedCount is the estimated total count after applying custom filters.
EstimatedCount int64 `json:"estimatedCount"`
// CustomFilterRatio is the ratio r (0.0 to 1.0) applied to MetricCount.
CustomFilterRatio float64 `json:"customFilterRatio"`
// IsExact is true if no custom filter ratio estimation was required.
IsExact bool `json:"isExact"`
// Preset indicates a rough estimation tier if defined on the query.
Preset EstimatedCountPreset `json:"preset,omitempty"`
}
EstimateResult represents the volume estimate.
type EstimatedCountPreset ¶
type EstimatedCountPreset int
EstimatedCountPreset represents a predefined rough estimation tier.
const ( // EstimatedCountPresetNone indicates that standard numeric estimation should be performed. EstimatedCountPresetNone EstimatedCountPreset = iota // EstimatedCountPresetFew indicates that the query matches only a small number of logs. EstimatedCountPresetFew )
func (EstimatedCountPreset) String ¶
func (p EstimatedCountPreset) String() string
String returns the string representation of the preset.
type EstimatorProvider ¶
type EstimatorProvider func(ctx context.Context, container googlecloud.ResourceContainer) (*StructuredLogEstimator, error)
EstimatorProvider is a factory function to construct a StructuredLogEstimator for a container.
type LogProbeClient ¶
type LogProbeClient interface {
FetchProbe(ctx context.Context, filter string, pageSize int32) ([]*loggingpb.LogEntry, error)
}
LogProbeClient is an interface for probing Cloud Logging.
type LogSamplingProbeFetcher ¶
type LogSamplingProbeFetcher interface {
// ProbeLogTimestamps returns timestamps of matching log entries (up to maxEntries, ordered by timestamp desc).
ProbeLogTimestamps(ctx context.Context, container googlecloud.ResourceContainer, filter string, maxEntries int32) ([]time.Time, error)
}
LogSamplingProbeFetcher is an interface for probing Cloud Logging sample logs.
type LoggingClientProbeFetcher ¶
LoggingClientProbeFetcher implements LogSamplingProbeFetcher using *logging.Client.
func (*LoggingClientProbeFetcher) ProbeLogTimestamps ¶
func (f *LoggingClientProbeFetcher) ProbeLogTimestamps(ctx context.Context, container googlecloud.ResourceContainer, filter string, maxEntries int32) ([]time.Time, error)
ProbeLogTimestamps queries recent matching log entries from Cloud Logging to extract timestamps.
type LoggingMonitoringMatcher ¶
type LoggingMonitoringMatcher interface {
// SupportMetrics returns true if this filter condition can be natively represented in Cloud Monitoring.
SupportMetrics() bool
// ToLoggingQuery returns the filter fragment for Cloud Logging queries.
ToLoggingQuery() string
// ToMonitoringFilter returns the filter fragment for Cloud Monitoring metric filters.
ToMonitoringFilter() string
}
LoggingMonitoringMatcher defines a self-contained filter clause for both Cloud Logging query strings and Cloud Monitoring metric filters.
func Comment ¶
func Comment(comment string) LoggingMonitoringMatcher
Comment creates a LoggingMonitoringMatcher for a standalone comment-only line. The comment text is formatted as "-- <comment>". Since comments do not filter log entries, SupportMetrics returns true and ToMonitoringFilter returns an empty string.
func CustomFilter ¶
func CustomFilter(rawQuery string) LoggingMonitoringMatcher
CustomFilter creates a LoggingMonitoringMatcher for payload/custom filters. Cloud Monitoring does not support payload filters natively, so SupportMetrics returns false.
func LogID ¶
func LogID(vm ValueMatcher) LoggingMonitoringMatcher
LogID creates a LoggingMonitoringMatcher for log IDs (LOG_ID macro in Logging, metric.labels.log in Monitoring).
func ResourceLabel ¶
func ResourceLabel(labelKey string, vm ValueMatcher) LoggingMonitoringMatcher
ResourceLabel creates a LoggingMonitoringMatcher for a resource.labels.<key> field.
func WithComment ¶
func WithComment(matcher LoggingMonitoringMatcher, comment string) LoggingMonitoringMatcher
WithComment wraps an existing LoggingMonitoringMatcher to append a trailing comment (" -- <comment>") to its Cloud Logging query representation. Cloud Monitoring metric evaluation is delegated to the wrapped matcher without appending the comment.
type MetricLogCountFetcher ¶
type MetricLogCountFetcher interface {
QueryMetricCount(ctx context.Context, container googlecloud.ResourceContainer, metricFilter string, startTime, endTime time.Time) (int64, error)
}
MetricLogCountFetcher is a function or client capable of querying Cloud Monitoring for log entry counts.
type MonitoringClientFetcher ¶
type MonitoringClientFetcher struct {
Client *monitoring.MetricClient
}
MonitoringClientFetcher implements MetricLogCountFetcher using *monitoring.MetricClient.
func (*MonitoringClientFetcher) QueryMetricCount ¶
func (f *MonitoringClientFetcher) QueryMetricCount(ctx context.Context, container googlecloud.ResourceContainer, metricFilter string, startTime, endTime time.Time) (int64, error)
QueryMetricCount queries Cloud Monitoring for logging.googleapis.com/log_entry_count.
type StructuredLogEstimator ¶
type StructuredLogEstimator struct {
MetricFetcher MetricLogCountFetcher
ProbeFetcher LogSamplingProbeFetcher
CallOptionInjector *googlecloud.CallOptionInjector
TargetSamples int32
}
StructuredLogEstimator estimates log volume for StructuredLogQuery.
func NewStructuredLogEstimator ¶
func NewStructuredLogEstimator(metricFetcher MetricLogCountFetcher, probeFetcher LogSamplingProbeFetcher, callOptionInjector *googlecloud.CallOptionInjector) *StructuredLogEstimator
NewStructuredLogEstimator creates a StructuredLogEstimator with defaults.
func NewStructuredLogEstimatorFromClients ¶
func NewStructuredLogEstimatorFromClients(loggingClient *logging.Client, metricClient *monitoring.MetricClient, callOptionInjector *googlecloud.CallOptionInjector) *StructuredLogEstimator
NewStructuredLogEstimatorFromClients constructs a StructuredLogEstimator directly from Google Cloud SDK clients.
func (*StructuredLogEstimator) Estimate ¶
func (e *StructuredLogEstimator) Estimate(ctx context.Context, container googlecloud.ResourceContainer, query *StructuredLogQuery, startTime, endTime time.Time) (*EstimateResult, error)
Estimate estimates the total log volume for the given StructuredLogQuery over the time interval. Cloud Monitoring queries for all resource types are executed concurrently. If custom filters require sampling, a time-window bounded sampling probe is executed.
type StructuredLogQuery ¶
type StructuredLogQuery struct {
// ResourceTypes specifies the resource.type strings (e.g. ["k8s_container"] or ["gke_cluster", "gke_nodepool"]).
ResourceTypes []string
// IgnoreMetricsResourceType specifies the resource.type strings that should NOT be queried via Cloud Monitoring metrics
// (e.g. resource types like "gke_nodepool" that are valid in Cloud Logging but do not exist as MonitoredResourceDescriptors in Cloud Monitoring).
IgnoreMetricsResourceType []string
// Filters are the individual filter matchers for this query.
Filters []LoggingMonitoringMatcher
// Incomplete indicates that the query is missing required parameters (e.g. ProjectID or ClusterName)
// and should not trigger log volume estimation.
Incomplete bool
// Preset specifies a predefined rough estimation tier (e.g. EstimatedCountPresetFew)
// that bypasses Cloud Monitoring and probe queries.
Preset EstimatedCountPreset
}
StructuredLogQuery represents a decomposed Cloud Logging and Cloud Monitoring query.
func (*StructuredLogQuery) AllFiltersSupportMetrics ¶
func (q *StructuredLogQuery) AllFiltersSupportMetrics() bool
AllFiltersSupportMetrics returns true if all matchers in Filters can be natively evaluated in Cloud Monitoring and no resource types are excluded from metrics querying.
func (*StructuredLogQuery) GenerateBaseLoggingQuery ¶
func (q *StructuredLogQuery) GenerateBaseLoggingQuery() string
GenerateBaseLoggingQuery generates the Cloud Logging filter string containing only matchers supported by Cloud Monitoring.
func (*StructuredLogQuery) GenerateCloudLoggingQuery ¶
func (q *StructuredLogQuery) GenerateCloudLoggingQuery() string
GenerateCloudLoggingQuery generates the Cloud Logging filter string.
func (*StructuredLogQuery) GenerateMonitoringMetricFilters ¶
func (q *StructuredLogQuery) GenerateMonitoringMetricFilters() []string
GenerateMonitoringMetricFilters generates the filter strings for logging.googleapis.com/log_entry_count in Cloud Monitoring (one per resource type).
type ValueMatcher ¶
type ValueMatcher interface {
// ToLoggingFieldQuery returns the Cloud Logging query fragment for a named field.
ToLoggingFieldQuery(fieldName string) string
// ToMonitoringFieldFilter returns the Cloud Monitoring metric filter fragment for a named field.
ToMonitoringFieldFilter(fieldName string) string
// ToLoggingLogIDQuery returns the Cloud Logging query fragment for LOG_ID(...).
ToLoggingLogIDQuery() string
// ToMonitoringLogIDFilter returns the Cloud Monitoring metric filter fragment for metric.labels.log.
ToMonitoringLogIDFilter() string
}
ValueMatcher defines how a set of values or patterns expands for both general field names (e.g. resource.labels.key) and LogID macros.
func ContainsAny ¶
func ContainsAny(subs ...string) ValueMatcher
ContainsAny creates a ValueMatcher matching any of the substrings.
func Exact ¶
func Exact(val string) ValueMatcher
Exact creates a ValueMatcher for an exact single value.
func FromSetFilter ¶
func FromSetFilter(filter *gcpqueryutil.SetFilterParseResult, isSubstring bool) ValueMatcher
FromSetFilter converts a KHI SetFilterParseResult into a ValueMatcher.
func NoneOf ¶
func NoneOf(vals ...string) ValueMatcher
NoneOf creates a ValueMatcher that excludes all provided values (NOT).
func NotContainsAny ¶
func NotContainsAny(subs ...string) ValueMatcher
NotContainsAny creates a ValueMatcher excluding all of the substrings.
func OneOf ¶
func OneOf(vals ...string) ValueMatcher
OneOf creates a ValueMatcher that matches any of the provided values (OR).