httplog

go-httplog

A Go implementation of format-driven logging a là Apache mod_log_config.

Creating a logger.

   import "github.com/graygnuorg/go-httplog"
   
   logger := httplog.New(`%a %l %u %t "%r" %>s %b "%{Referer}i" "%{User-Agent}i"`, nil)

This creates a logger that will format its entries according to the supplied format string. Second argument to New is a list of IP addresses or CIDRs used to identify trusted IPs for the purpose of extracting originator IP address from the X-Forwarded-For or similar header (see below). Nil means use originator address from the request.

Once created, the logger can be associated with the output file:

   if err := logger.OpenFile("/var/log/access.log"); err != nil {
          panic(err)
   }

Logging

To log a request, call

   hl.Log(request, metrics)

Its arguments are:

• request *http.Request A request being logged.

• metrics httplog.Metrics Additional data about request processing.

Metrics supply information about request processing, such as response status code, size, etc. This argument can be nil if these data are not available or are of no interest. If supplied, it must be a pointer to an object conforming to the following interface:

• Listen() string Returns the IP address and port of the server handling the requests, in format IP:PORT. This information is used by conversion specifiers %A and %p.

• Handler() string Handler name, for the %R specifier. Its exact meaning is up to the calling implementation.

• Ident() string Remote user logname, if determined. Used by %l.

• Status() int HTTP response status code for %s or %>s.

• Duration() time.Duration Time taken to serve the request. For %D, %T.

• Size() int64 Size of the response in bytes, not counting headers. For %B and %b specifiers.

Format

The format string consists of literal characters, C-style control characters, and conversion specifiers. Literal characters are any characters, not preceded by % or \. They are copied to the output verbatim. Control characters are \n, \t, and \\ which represent newlines, tabs, and backslashes on the output. Conversion specifiers start with % and expand to certain characteristics of the request or metrics.

Supported conversion specifiers are:

• %% The percent sign.

• %a Client IP address of the request. Determined using X-Forwarded-From header, if available.

• %{c}a Underlying peer IP address of the connection.

• %A Local IP-address. Obtained from Metrics.Listen().

• %B Size of response in bytes, excluding HTTP headers. Uses Metrics.Size().

• %b Same as above, but in CLF format, i.e. a '-' rather than a 0 when no bytes are sent.

• %D The time taken to serve the request, in microseconds.

• %{VAR}e The contents of the environment variable VAR.

• %h Remote hostname.

• %H The request protocol.

• %{FIELD}i The contents of FIELD: header line in the request (only the first value in case of multiple headers).

• %l Remote logname, as returned by Metrics.Ident().

• %m The request method.

• %p The canonical port of the server serving the request, as reported by Metrics.Listen().

• %q The query string (prepended with a ? if a query string exists, otherwise an empty string).

• %r First line of request.

• %R Name of the handler generating the response, as returned by Metrics.Handler().

• %s, %>s Response status, as returned by Metrics.Status().

• %t Time the request was received, in the format [02/Jan/2016:15:04:05 -0700].

• %{FORMAT}t Time the request was received, in the form given by strftime(3) FORMAT. The format can optionally be prefixed with begin:, if the time was taken at the beginning of the request processing or end, if it was taken at the end, Default is begin:. Additionally, the following format tokens are supported:

  • sec Number of seconds since the Epoch.
  • msec Number of milliseconds since the Epoch.
  • usec Number of microseconds since the Epoch.
  • msec_frac Millisecond fraction.
  • usec_frac Microsecond fraction

• %T The time taken to serve the request, in seconds. Reported by Metrics.Duration().

• %{UNIT}T The time taken to serve the request, in UNITs (ms, us, s for milliseconds, microseconds, or seconds, correspondingly).

• %u Remote user if the request was authenticated. Supplied by r.URL.User.

• %U The URL path requested, not including any query string.

• %v, %V The canonical name of the server serving the request (given by r.Host).

Example

Here's an example of using httplog together with the httpsnoop package:

   import (
        "net/http"
        "github.com/felixge/httpsnoop" 
        "github.com/graygnuorg/go-httplog"
   )

   // Listener address.
   var ListenAddress string = ":8080"
   // Default log format.
   var LogFormat string = `%a %u "%r" %s %D`
   
   // Implement the Metrics structure.
   type Metrics struct {
        httpsnoop.Metrics    // Most values are taken from there.
   }

   func (m *Metrics) Listen() string {
        return ListenAddress
   }

   func (m *Metrics) Ident() string {
        return ""
   }

   func (m *Metrics) Handler() string {
        return ""
   }

   func (m *Metrics) Status() int {
        return m.Metrics.Code
   }

   func (m *Metrics) Duration() time.Duration {
        return m.Metrics.Duration
   }

   func (m *Metrics) Size() int64 {
        return m.Metrics.Written
   }

   func (m *Metrics) Size() int64 {
        return m.Metrics.Written
   }

   // Create new logger.
   hl := httplog.New(LogFormat, nil)

// Create multiplexer and HTTP server
   mux := &http.ServeMux{}
   s := &http.Server{
        Addr: ListenAddress,
        Handler: http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
	    defer r.Body.Close()
	    // Call ServerHTTP and obtain resulting metrics.
            m := httpsnoop.CaptureMetricsFn(w, func (w http.ResponseWriter) {
	        mux.ServeHTTP(w, r)
            })
	    // Log it.
            hl.Log(r, &Metrics{m})
        }),
   }
   s.ListenAndServe()