jog

jog

Command line tool to view structured (JSON) log, as regular flat line format

Background

Structured log, AKA. JSON line log, is great for log collectors but hard to read by developers themselves, usually during local development. Jog helps to on-the-fly convert those structured JSON log to traditional space-separated flat line log, friendly for developers. It then removes the need to maintenain different output format for different environments - for ex. we don't need any more to configure JSON log for test / production but flat line log for local development.

Features

Feature request is welcomed, for ex. new JSON log format. Submit issue for that please.

• Support various of formats without customization. For ex.:

  • Logstash
  • GOLANG Uber zap
  • Node.js Bunyan (https://github.com/trentm/node-bunyan)

• Follow mode like tail -f, with optional beginning from latest specified lines like tail -n.

• Read from stdin (stream) or local file

• Straightforward filtering:

  • by logger level
  • by time range, relative or absolute

• Support JSON log mixed with non-JSON text, includes:

  • Mixed with regular flat log lines, for ex., springboot banner, and PM2 banner
  • Extra non-JSON prefix, followed by JSON log, for ex., docker-compose multi-services log

• Supports nested escaped JSON value (escaped by \"...\")

• Compressed logger name - only first letters of package names are outputed

• Print line number as line prefix

• Customizable although usually you no need it. Run jog -t to export default configuration, or see ./static_files/DefaultConfiguration.yml

  • Output pattern
  • Hightlight startup line
  • Colorization
  • Print unknown fields as 'others'
  • For fields that not explictly in output pattern, print as 'others'
  • Show/hide fields

• A GOLANG application, so single across-platform executable binary, support Mac OSX, Windows, Linux.

Usage:

Download the executable binary (https://github.com/qiangyt/jog/releases/) to $PATH. For ex., for Mac OSX and Linux,

   sudo curl -L https://github.com/qiangyt/jog/releases/download/v0.9.18/jog.$(echo `uname -s` | tr A-Z a-z) -o /usr/local/bin/jog
   sudo chmod +x /usr/local/bin/jog

• View a local JSON log file: jog sample.log

  Or follows begining from latest 20 lines: jog -n 20 -f sample.log

• Follow stdin stream, for ex. docker: docker logs -f my.container | ./jog -n 20

• Check full usage: jog -h

  Usage:
    jog  [option...]  <your JSON log file path>
       or
    <stdin stream>  |  jog  [option...]
  
  Examples:
     1) follow with last 10 lines:         jog -f app-20200701-1.log
      2) follow with specified lines:       jog -n 100 -f app-20200701-1.log
      3) with specified config file:        jog -c another.jog.yml app-20200701-1.log
      4) view docker-compose log:           docker-compose logs | jog
      5) print the default template:        jog -t
      6) only shows WARN & ERROR level:     jog -l warn -l error app-20200701-1.log
      7) shows with timestamp range:        jog --after 2020-7-1 --before 2020-7-3 app-20200701-1.log
      8) with WARN level foreground color set to RED: jog -cs fields.level.enums.WARN.color=FgRed app-20200701-1.log
      9) view the WARN level config item:   jog -cg fields.level.enums.WARN
     10) disable colorization:              jog -cs colorization=false app-20200701-1.log
  
  Options:
    -a,  --after <timestamp>                                    'after' time filter. Auto-detect the timestamp format
     -b,  --before <timestamp>                                   'before' time filter. Auto-detect the timestamp format
    -c,  --config <config file path>                            Specify config YAML file path. The default is .jog.yaml or $HOME/.jog.yaml
    -cs, --config-set <config item path>=<config item value>    Set value to specified config item
    -cg, --config-get <config item path>                        Get value to specified config item
    -d,  --debug                                                Print more error detail
    -f,  --follow                                               Follow mode - follow log output
    -h,  --help                                                 Display this information
    -l,  --level <level value>                                  Filter by log level. For ex. --level warn
    -n,  --lines <number of tail lines>                         Number of tail lines. 10 by default, for follow mode
    -t,  --template                                             Print a configuration YAML file template
    -V,  --version                                              Display app version information
  

Build

• Install GOLANG version >= 1.13

• In current directory, run ./build.sh

Status

Not yet ready for first release, still keep refactoring and fixing and adding new features. I create pre-release even for single bug fix or small feature. I won't test much before version 1.0 is ready. Just feel free to use it since it wouldn't affect something.

License

MIT