stool

stool

stool is a CLI tool to analyze access pattern from Nginx access log.

Installation

You can install stool using the following command:

go install github.com/haijima/stool@latest

or you can download binaries from Releases.

Commands and Options

Commands

• stool param: Show the parameter statistics for each endpoint
• stool scenario: Show the access patterns of users
• stool transition: Show the transition between endpoints
• stool trend: Show the count of accesses for each endpoint over time
• stool genconf: Generate configuration file

Options

Global Options

• --config string : Config file (default is $XDG_CONFIG_HOME/.stool.yaml)
• -f, --file string : Access log file to profile.
• --filter string : Filter log lines by regular expression
• --log_labels stringToString : Comma-separated list of key=value pairs to override log labels (default [])
• -m, --matching_groups strings : Comma-separated list of regular expression patterns to group matched URIs. For example: --matching_groups "/users/.*,/items/.*".
• --no_color: Disable colorized output
• --time_format string : The format to parse time field on log file (default "02/Jan/2006:15:04:05 -0700").

Options for stool param

• --format string: The stat output format {table|md|csv|tsv} (default "table")
• -n, --num int: The number of parameters to show (default 5)
• --stat: Show statistics of the parameters
• -t, --type string: The type of the parameter {path|query|all} (default "all")

Options for stool scenario

• --format string : The output format {dot|mermaid|csv} (default "dot").

Options for stool transition

• --format string : The output format {dot|mermaid|csv} (default "dot").

Options for stool trend

• --format string : The output format {table|md|csv} (default "table")
• -i, --interval int : The time (in seconds) of the interval. Access counts are cumulated at each interval. ( default 5).
• --sort string : Comma-separated list of "<sort keys>:<order>" Sort keys are {method|uri|sum|count0|count1|countN}. Orders are [asc|desc]. e.g. "sum:desc,count0:asc" ( default "sum:desc")

Options for stool genconf

• --format string : The output format {toml|yaml|json} (default "toml")

Config file

You can specify the default options in a config file.

The config file is searched in the following order:

1. --config option
2. $CURRENT_DIR/.stool.yaml
3. $XDG_CONFIG_HOME/stool/.stool.yaml
4. $HOME/.config/stool/.stool.yaml when $XDG_CONFIG_HOME is not set
5. $HOME/.stool.yaml

YAML, JSON or TOML format is supported.

Examples

stool param --file path/to/access.log --num 10 --stat "/users/(?P<userId>[^/]+)$"

stool scenario --file path/to/access.log --matching_groups "/users/.*,/items/.*" --format dot | dot -T svg -o scenario.svg && open scenario.svg

stool transition --file path/to/access.log --matching_groups "/users/.*,/items/.*" --format dot | dot -T svg -o transition.svg && open transition.svg

stool trend --file path/to/access.log --matching_groups "/users/.*,/items/.*" --interval 10

stool genconf path/to/main.go --format yaml >> .stool.yaml

Prerequisites

Graphviz

Graphviz is required to visualize .dot files that are generated by stool scenario or stool transition command.

Nginx Log format

1. ngx_http_userid_module is required to get the user ID from the cookie. See Module ngx_http_userid_module for details.
2. stool can handle LTSV formatted log file.

The example of Nginx log setting is shown below:

userid on; # Enable $uid_got and $uid_set

log_format ltsv "time:$time_local"
                "\thost:$remote_addr"
                "\tforwardedfor:$http_x_forwarded_for"
                "\treq:$request"
                "\tstatus:$status"
                "\tmethod:$request_method"
                "\turi:$request_uri"
                "\tsize:$body_bytes_sent"
                "\treferer:$http_referer"
                "\tua:$http_user_agent"
                "\treqtime:$request_time"
                "\tcache:$upstream_http_x_cache"
                "\truntime:$upstream_http_x_runtime"
                "\tapptime:$upstream_response_time"
                "\tuidgot:$uid_got"
                "\tuidset:$uid_set"
                "\tvhost:$host";

License

This tool is licensed under the MIT License. See the LICENSE file for details.