Ben Ye
5 years ago
committed by
Ben Kochie
158 changed files with 3434 additions and 4636 deletions
@ -1,2 +1,4 @@
|
||||
// Used in HTTP handlers, any error is handled by the server itself. |
||||
(net/http.ResponseWriter).Write |
||||
// Never check for logger errors. |
||||
(github.com/go-kit/kit/log.Logger).Log |
||||
@ -0,0 +1,151 @@
|
||||
# package log |
||||
|
||||
`package log` provides a minimal interface for structured logging in services. |
||||
It may be wrapped to encode conventions, enforce type-safety, provide leveled |
||||
logging, and so on. It can be used for both typical application log events, |
||||
and log-structured data streams. |
||||
|
||||
## Structured logging |
||||
|
||||
Structured logging is, basically, conceding to the reality that logs are |
||||
_data_, and warrant some level of schematic rigor. Using a stricter, |
||||
key/value-oriented message format for our logs, containing contextual and |
||||
semantic information, makes it much easier to get insight into the |
||||
operational activity of the systems we build. Consequently, `package log` is |
||||
of the strong belief that "[the benefits of structured logging outweigh the |
||||
minimal effort involved](https://www.thoughtworks.com/radar/techniques/structured-logging)". |
||||
|
||||
Migrating from unstructured to structured logging is probably a lot easier |
||||
than you'd expect. |
||||
|
||||
```go |
||||
// Unstructured |
||||
log.Printf("HTTP server listening on %s", addr) |
||||
|
||||
// Structured |
||||
logger.Log("transport", "HTTP", "addr", addr, "msg", "listening") |
||||
``` |
||||
|
||||
## Usage |
||||
|
||||
### Typical application logging |
||||
|
||||
```go |
||||
w := log.NewSyncWriter(os.Stderr) |
||||
logger := log.NewLogfmtLogger(w) |
||||
logger.Log("question", "what is the meaning of life?", "answer", 42) |
||||
|
||||
// Output: |
||||
// question="what is the meaning of life?" answer=42 |
||||
``` |
||||
|
||||
### Contextual Loggers |
||||
|
||||
```go |
||||
func main() { |
||||
var logger log.Logger |
||||
logger = log.NewLogfmtLogger(log.NewSyncWriter(os.Stderr)) |
||||
logger = log.With(logger, "instance_id", 123) |
||||
|
||||
logger.Log("msg", "starting") |
||||
NewWorker(log.With(logger, "component", "worker")).Run() |
||||
NewSlacker(log.With(logger, "component", "slacker")).Run() |
||||
} |
||||
|
||||
// Output: |
||||
// instance_id=123 msg=starting |
||||
// instance_id=123 component=worker msg=running |
||||
// instance_id=123 component=slacker msg=running |
||||
``` |
||||
|
||||
### Interact with stdlib logger |
||||
|
||||
Redirect stdlib logger to Go kit logger. |
||||
|
||||
```go |
||||
import ( |
||||
"os" |
||||
stdlog "log" |
||||
kitlog "github.com/go-kit/kit/log" |
||||
) |
||||
|
||||
func main() { |
||||
logger := kitlog.NewJSONLogger(kitlog.NewSyncWriter(os.Stdout)) |
||||
stdlog.SetOutput(kitlog.NewStdlibAdapter(logger)) |
||||
stdlog.Print("I sure like pie") |
||||
} |
||||
|
||||
// Output: |
||||
// {"msg":"I sure like pie","ts":"2016/01/01 12:34:56"} |
||||
``` |
||||
|
||||
Or, if, for legacy reasons, you need to pipe all of your logging through the |
||||
stdlib log package, you can redirect Go kit logger to the stdlib logger. |
||||
|
||||
```go |
||||
logger := kitlog.NewLogfmtLogger(kitlog.StdlibWriter{}) |
||||
logger.Log("legacy", true, "msg", "at least it's something") |
||||
|
||||
// Output: |
||||
// 2016/01/01 12:34:56 legacy=true msg="at least it's something" |
||||
``` |
||||
|
||||
### Timestamps and callers |
||||
|
||||
```go |
||||
var logger log.Logger |
||||
logger = log.NewLogfmtLogger(log.NewSyncWriter(os.Stderr)) |
||||
logger = log.With(logger, "ts", log.DefaultTimestampUTC, "caller", log.DefaultCaller) |
||||
|
||||
logger.Log("msg", "hello") |
||||
|
||||
// Output: |
||||
// ts=2016-01-01T12:34:56Z caller=main.go:15 msg=hello |
||||
``` |
||||
|
||||
## Levels |
||||
|
||||
Log levels are supported via the [level package](https://godoc.org/github.com/go-kit/kit/log/level). |
||||
|
||||
## Supported output formats |
||||
|
||||
- [Logfmt](https://brandur.org/logfmt) ([see also](https://blog.codeship.com/logfmt-a-log-format-thats-easy-to-read-and-write)) |
||||
- JSON |
||||
|
||||
## Enhancements |
||||
|
||||
`package log` is centered on the one-method Logger interface. |
||||
|
||||
```go |
||||
type Logger interface { |
||||
Log(keyvals ...interface{}) error |
||||
} |
||||
``` |
||||
|
||||
This interface, and its supporting code like is the product of much iteration |
||||
and evaluation. For more details on the evolution of the Logger interface, |
||||
see [The Hunt for a Logger Interface](http://go-talks.appspot.com/github.com/ChrisHines/talks/structured-logging/structured-logging.slide#1), |
||||
a talk by [Chris Hines](https://github.com/ChrisHines). |
||||
Also, please see |
||||
[#63](https://github.com/go-kit/kit/issues/63), |
||||
[#76](https://github.com/go-kit/kit/pull/76), |
||||
[#131](https://github.com/go-kit/kit/issues/131), |
||||
[#157](https://github.com/go-kit/kit/pull/157), |
||||
[#164](https://github.com/go-kit/kit/issues/164), and |
||||
[#252](https://github.com/go-kit/kit/pull/252) |
||||
to review historical conversations about package log and the Logger interface. |
||||
|
||||
Value-add packages and suggestions, |
||||
like improvements to [the leveled logger](https://godoc.org/github.com/go-kit/kit/log/level), |
||||
are of course welcome. Good proposals should |
||||
|
||||
- Be composable with [contextual loggers](https://godoc.org/github.com/go-kit/kit/log#With), |
||||
- Not break the behavior of [log.Caller](https://godoc.org/github.com/go-kit/kit/log#Caller) in any wrapped contextual loggers, and |
||||
- Be friendly to packages that accept only an unadorned log.Logger. |
||||
|
||||
## Benchmarks & comparisons |
||||
|
||||
There are a few Go logging benchmarks and comparisons that include Go kit's package log. |
||||
|
||||
- [imkira/go-loggers-bench](https://github.com/imkira/go-loggers-bench) includes kit/log |
||||
- [uber-common/zap](https://github.com/uber-common/zap), a zero-alloc logging library, includes a comparison with kit/log |
||||
@ -0,0 +1,116 @@
|
||||
// Package log provides a structured logger.
|
||||
//
|
||||
// Structured logging produces logs easily consumed later by humans or
|
||||
// machines. Humans might be interested in debugging errors, or tracing
|
||||
// specific requests. Machines might be interested in counting interesting
|
||||
// events, or aggregating information for off-line processing. In both cases,
|
||||
// it is important that the log messages are structured and actionable.
|
||||
// Package log is designed to encourage both of these best practices.
|
||||
//
|
||||
// Basic Usage
|
||||
//
|
||||
// The fundamental interface is Logger. Loggers create log events from
|
||||
// key/value data. The Logger interface has a single method, Log, which
|
||||
// accepts a sequence of alternating key/value pairs, which this package names
|
||||
// keyvals.
|
||||
//
|
||||
// type Logger interface {
|
||||
// Log(keyvals ...interface{}) error
|
||||
// }
|
||||
//
|
||||
// Here is an example of a function using a Logger to create log events.
|
||||
//
|
||||
// func RunTask(task Task, logger log.Logger) string {
|
||||
// logger.Log("taskID", task.ID, "event", "starting task")
|
||||
// ...
|
||||
// logger.Log("taskID", task.ID, "event", "task complete")
|
||||
// }
|
||||
//
|
||||
// The keys in the above example are "taskID" and "event". The values are
|
||||
// task.ID, "starting task", and "task complete". Every key is followed
|
||||
// immediately by its value.
|
||||
//
|
||||
// Keys are usually plain strings. Values may be any type that has a sensible
|
||||
// encoding in the chosen log format. With structured logging it is a good
|
||||
// idea to log simple values without formatting them. This practice allows
|
||||
// the chosen logger to encode values in the most appropriate way.
|
||||
//
|
||||
// Contextual Loggers
|
||||
//
|
||||
// A contextual logger stores keyvals that it includes in all log events.
|
||||
// Building appropriate contextual loggers reduces repetition and aids
|
||||
// consistency in the resulting log output. With and WithPrefix add context to
|
||||
// a logger. We can use With to improve the RunTask example.
|
||||
//
|
||||
// func RunTask(task Task, logger log.Logger) string {
|
||||
// logger = log.With(logger, "taskID", task.ID)
|
||||
// logger.Log("event", "starting task")
|
||||
// ...
|
||||
// taskHelper(task.Cmd, logger)
|
||||
// ...
|
||||
// logger.Log("event", "task complete")
|
||||
// }
|
||||
//
|
||||
// The improved version emits the same log events as the original for the
|
||||
// first and last calls to Log. Passing the contextual logger to taskHelper
|
||||
// enables each log event created by taskHelper to include the task.ID even
|
||||
// though taskHelper does not have access to that value. Using contextual
|
||||
// loggers this way simplifies producing log output that enables tracing the
|
||||
// life cycle of individual tasks. (See the Contextual example for the full
|
||||
// code of the above snippet.)
|
||||
//
|
||||
// Dynamic Contextual Values
|
||||
//
|
||||
// A Valuer function stored in a contextual logger generates a new value each
|
||||
// time an event is logged. The Valuer example demonstrates how this feature
|
||||
// works.
|
||||
//
|
||||
// Valuers provide the basis for consistently logging timestamps and source
|
||||
// code location. The log package defines several valuers for that purpose.
|
||||
// See Timestamp, DefaultTimestamp, DefaultTimestampUTC, Caller, and
|
||||
// DefaultCaller. A common logger initialization sequence that ensures all log
|
||||
// entries contain a timestamp and source location looks like this:
|
||||
//
|
||||
// logger := log.NewLogfmtLogger(log.NewSyncWriter(os.Stdout))
|
||||
// logger = log.With(logger, "ts", log.DefaultTimestampUTC, "caller", log.DefaultCaller)
|
||||
//
|
||||
// Concurrent Safety
|
||||
//
|
||||
// Applications with multiple goroutines want each log event written to the
|
||||
// same logger to remain separate from other log events. Package log provides
|
||||
// two simple solutions for concurrent safe logging.
|
||||
//
|
||||
// NewSyncWriter wraps an io.Writer and serializes each call to its Write
|
||||
// method. Using a SyncWriter has the benefit that the smallest practical
|
||||
// portion of the logging logic is performed within a mutex, but it requires
|
||||
// the formatting Logger to make only one call to Write per log event.
|
||||
//
|
||||
// NewSyncLogger wraps any Logger and serializes each call to its Log method.
|
||||
// Using a SyncLogger has the benefit that it guarantees each log event is
|
||||
// handled atomically within the wrapped logger, but it typically serializes
|
||||
// both the formatting and output logic. Use a SyncLogger if the formatting
|
||||
// logger may perform multiple writes per log event.
|
||||
//
|
||||
// Error Handling
|
||||
//
|
||||
// This package relies on the practice of wrapping or decorating loggers with
|
||||
// other loggers to provide composable pieces of functionality. It also means
|
||||
// that Logger.Log must return an error because some
|
||||
// implementations—especially those that output log data to an io.Writer—may
|
||||
// encounter errors that cannot be handled locally. This in turn means that
|
||||
// Loggers that wrap other loggers should return errors from the wrapped
|
||||
// logger up the stack.
|
||||
//
|
||||
// Fortunately, the decorator pattern also provides a way to avoid the
|
||||
// necessity to check for errors every time an application calls Logger.Log.
|
||||
// An application required to panic whenever its Logger encounters
|
||||
// an error could initialize its logger as follows.
|
||||
//
|
||||
// fmtlogger := log.NewLogfmtLogger(log.NewSyncWriter(os.Stdout))
|
||||
// logger := log.LoggerFunc(func(keyvals ...interface{}) error {
|
||||
// if err := fmtlogger.Log(keyvals...); err != nil {
|
||||
// panic(err)
|
||||
// }
|
||||
// return nil
|
||||
// })
|
||||
package log |
||||
@ -0,0 +1,89 @@
|
||||
package log |
||||
|
||||
import ( |
||||
"encoding" |
||||
"encoding/json" |
||||
"fmt" |
||||
"io" |
||||
"reflect" |
||||
) |
||||
|
||||
type jsonLogger struct { |
||||
io.Writer |
||||
} |
||||
|
||||
// NewJSONLogger returns a Logger that encodes keyvals to the Writer as a
|
||||
// single JSON object. Each log event produces no more than one call to
|
||||
// w.Write. The passed Writer must be safe for concurrent use by multiple
|
||||
// goroutines if the returned Logger will be used concurrently.
|
||||
func NewJSONLogger(w io.Writer) Logger { |
||||
return &jsonLogger{w} |
||||
} |
||||
|
||||
func (l *jsonLogger) Log(keyvals ...interface{}) error { |
||||
n := (len(keyvals) + 1) / 2 // +1 to handle case when len is odd
|
||||
m := make(map[string]interface{}, n) |
||||
for i := 0; i < len(keyvals); i += 2 { |
||||
k := keyvals[i] |
||||
var v interface{} = ErrMissingValue |
||||
if i+1 < len(keyvals) { |
||||
v = keyvals[i+1] |
||||
} |
||||
merge(m, k, v) |
||||
} |
||||
return json.NewEncoder(l.Writer).Encode(m) |
||||
} |
||||
|
||||
func merge(dst map[string]interface{}, k, v interface{}) { |
||||
var key string |
||||
switch x := k.(type) { |
||||
case string: |
||||
key = x |
||||
case fmt.Stringer: |
||||
key = safeString(x) |
||||
default: |
||||
key = fmt.Sprint(x) |
||||
} |
||||
|
||||
// We want json.Marshaler and encoding.TextMarshaller to take priority over
|
||||
// err.Error() and v.String(). But json.Marshall (called later) does that by
|
||||
// default so we force a no-op if it's one of those 2 case.
|
||||
switch x := v.(type) { |
||||
case json.Marshaler: |
||||
case encoding.TextMarshaler: |
||||
case error: |
||||
v = safeError(x) |
||||
case fmt.Stringer: |
||||
v = safeString(x) |
||||
} |
||||
|
||||
dst[key] = v |
||||
} |
||||
|
||||
func safeString(str fmt.Stringer) (s string) { |
||||
defer func() { |
||||
if panicVal := recover(); panicVal != nil { |
||||
if v := reflect.ValueOf(str); v.Kind() == reflect.Ptr && v.IsNil() { |
||||
s = "NULL" |
||||
} else { |
||||
panic(panicVal) |
||||
} |
||||
} |
||||
}() |
||||
s = str.String() |
||||
return |
||||
} |
||||
|
||||
func safeError(err error) (s interface{}) { |
||||
defer func() { |
||||
if panicVal := recover(); panicVal != nil { |
||||
if v := reflect.ValueOf(err); v.Kind() == reflect.Ptr && v.IsNil() { |
||||
s = nil |
||||
} else { |
||||
panic(panicVal) |
||||
} |
||||
} |
||||
}() |
||||
s = err.Error() |
||||
return |
||||
} |
||||
@ -0,0 +1,22 @@
|
||||
// Package level implements leveled logging on top of Go kit's log package. To
|
||||
// use the level package, create a logger as per normal in your func main, and
|
||||
// wrap it with level.NewFilter.
|
||||
//
|
||||
// var logger log.Logger
|
||||
// logger = log.NewLogfmtLogger(os.Stderr)
|
||||
// logger = level.NewFilter(logger, level.AllowInfo()) // <--
|
||||
// logger = log.With(logger, "ts", log.DefaultTimestampUTC)
|
||||
//
|
||||
// Then, at the callsites, use one of the level.Debug, Info, Warn, or Error
|
||||
// helper methods to emit leveled log events.
|
||||
//
|
||||
// logger.Log("foo", "bar") // as normal, no level
|
||||
// level.Debug(logger).Log("request_id", reqID, "trace_data", trace.Get())
|
||||
// if value > 100 {
|
||||
// level.Error(logger).Log("value", value)
|
||||
// }
|
||||
//
|
||||
// NewFilter allows precise control over what happens when a log event is
|
||||
// emitted without a level key, or if a squelched level is used. Check the
|
||||
// Option functions for details.
|
||||
package level |
||||
@ -0,0 +1,205 @@
|
||||
package level |
||||
|
||||
import "github.com/go-kit/kit/log" |
||||
|
||||
// Error returns a logger that includes a Key/ErrorValue pair.
|
||||
func Error(logger log.Logger) log.Logger { |
||||
return log.WithPrefix(logger, Key(), ErrorValue()) |
||||
} |
||||
|
||||
// Warn returns a logger that includes a Key/WarnValue pair.
|
||||
func Warn(logger log.Logger) log.Logger { |
||||
return log.WithPrefix(logger, Key(), WarnValue()) |
||||
} |
||||
|
||||
// Info returns a logger that includes a Key/InfoValue pair.
|
||||
func Info(logger log.Logger) log.Logger { |
||||
return log.WithPrefix(logger, Key(), InfoValue()) |
||||
} |
||||
|
||||
// Debug returns a logger that includes a Key/DebugValue pair.
|
||||
func Debug(logger log.Logger) log.Logger { |
||||
return log.WithPrefix(logger, Key(), DebugValue()) |
||||
} |
||||
|
||||
// NewFilter wraps next and implements level filtering. See the commentary on
|
||||
// the Option functions for a detailed description of how to configure levels.
|
||||
// If no options are provided, all leveled log events created with Debug,
|
||||
// Info, Warn or Error helper methods are squelched and non-leveled log
|
||||
// events are passed to next unmodified.
|
||||
func NewFilter(next log.Logger, options ...Option) log.Logger { |
||||
l := &logger{ |
||||
next: next, |
||||
} |
||||
for _, option := range options { |
||||
option(l) |
||||
} |
||||
return l |
||||
} |
||||
|
||||
type logger struct { |
||||
next log.Logger |
||||
allowed level |
||||
squelchNoLevel bool |
||||
errNotAllowed error |
||||
errNoLevel error |
||||
} |
||||
|
||||
func (l *logger) Log(keyvals ...interface{}) error { |
||||
var hasLevel, levelAllowed bool |
||||
for i := 1; i < len(keyvals); i += 2 { |
||||
if v, ok := keyvals[i].(*levelValue); ok { |
||||
hasLevel = true |
||||
levelAllowed = l.allowed&v.level != 0 |
||||
break |
||||
} |
||||
} |
||||
if !hasLevel && l.squelchNoLevel { |
||||
return l.errNoLevel |
||||
} |
||||
if hasLevel && !levelAllowed { |
||||
return l.errNotAllowed |
||||
} |
||||
return l.next.Log(keyvals...) |
||||
} |
||||
|
||||
// Option sets a parameter for the leveled logger.
|
||||
type Option func(*logger) |
||||
|
||||
// AllowAll is an alias for AllowDebug.
|
||||
func AllowAll() Option { |
||||
return AllowDebug() |
||||
} |
||||
|
||||
// AllowDebug allows error, warn, info and debug level log events to pass.
|
||||
func AllowDebug() Option { |
||||
return allowed(levelError | levelWarn | levelInfo | levelDebug) |
||||
} |
||||
|
||||
// AllowInfo allows error, warn and info level log events to pass.
|
||||
func AllowInfo() Option { |
||||
return allowed(levelError | levelWarn | levelInfo) |
||||
} |
||||
|
||||
// AllowWarn allows error and warn level log events to pass.
|
||||
func AllowWarn() Option { |
||||
return allowed(levelError | levelWarn) |
||||
} |
||||
|
||||
// AllowError allows only error level log events to pass.
|
||||
func AllowError() Option { |
||||
return allowed(levelError) |
||||
} |
||||
|
||||
// AllowNone allows no leveled log events to pass.
|
||||
func AllowNone() Option { |
||||
return allowed(0) |
||||
} |
||||
|
||||
func allowed(allowed level) Option { |
||||
return func(l *logger) { l.allowed = allowed } |
||||
} |
||||
|
||||
// ErrNotAllowed sets the error to return from Log when it squelches a log
|
||||
// event disallowed by the configured Allow[Level] option. By default,
|
||||
// ErrNotAllowed is nil; in this case the log event is squelched with no
|
||||
// error.
|
||||
func ErrNotAllowed(err error) Option { |
||||
return func(l *logger) { l.errNotAllowed = err } |
||||
} |
||||
|
||||
// SquelchNoLevel instructs Log to squelch log events with no level, so that
|
||||
// they don't proceed through to the wrapped logger. If SquelchNoLevel is set
|
||||
// to true and a log event is squelched in this way, the error value
|
||||
// configured with ErrNoLevel is returned to the caller.
|
||||
func SquelchNoLevel(squelch bool) Option { |
||||
return func(l *logger) { l.squelchNoLevel = squelch } |
||||
} |
||||
|
||||
// ErrNoLevel sets the error to return from Log when it squelches a log event
|
||||
// with no level. By default, ErrNoLevel is nil; in this case the log event is
|
||||
// squelched with no error.
|
||||
func ErrNoLevel(err error) Option { |
||||
return func(l *logger) { l.errNoLevel = err } |
||||
} |
||||
|
||||
// NewInjector wraps next and returns a logger that adds a Key/level pair to
|
||||
// the beginning of log events that don't already contain a level. In effect,
|
||||
// this gives a default level to logs without a level.
|
||||
func NewInjector(next log.Logger, level Value) log.Logger { |
||||
return &injector{ |
||||
next: next, |
||||
level: level, |
||||
} |
||||
} |
||||
|
||||
type injector struct { |
||||
next log.Logger |
||||
level interface{} |
||||
} |
||||
|
||||
func (l *injector) Log(keyvals ...interface{}) error { |
||||
for i := 1; i < len(keyvals); i += 2 { |
||||
if _, ok := keyvals[i].(*levelValue); ok { |
||||
return l.next.Log(keyvals...) |
||||
} |
||||
} |
||||
kvs := make([]interface{}, len(keyvals)+2) |
||||
kvs[0], kvs[1] = key, l.level |
||||
copy(kvs[2:], keyvals) |
||||
return l.next.Log(kvs...) |
||||
} |
||||
|
||||
// Value is the interface that each of the canonical level values implement.
|
||||
// It contains unexported methods that prevent types from other packages from
|
||||
// implementing it and guaranteeing that NewFilter can distinguish the levels
|
||||
// defined in this package from all other values.
|
||||
type Value interface { |
||||
String() string |
||||
levelVal() |
||||
} |
||||
|
||||
// Key returns the unique key added to log events by the loggers in this
|
||||
// package.
|
||||
func Key() interface{} { return key } |
||||
|
||||
// ErrorValue returns the unique value added to log events by Error.
|
||||
func ErrorValue() Value { return errorValue } |
||||
|
||||
// WarnValue returns the unique value added to log events by Warn.
|
||||
func WarnValue() Value { return warnValue } |
||||
|
||||
// InfoValue returns the unique value added to log events by Info.
|
||||
func InfoValue() Value { return infoValue } |
||||
|
||||
// DebugValue returns the unique value added to log events by Warn.
|
||||
func DebugValue() Value { return debugValue } |
||||
|
||||
var ( |
||||
// key is of type interface{} so that it allocates once during package
|
||||
// initialization and avoids allocating every time the value is added to a
|
||||
// []interface{} later.
|
||||
key interface{} = "level" |
||||
|
||||
errorValue = &levelValue{level: levelError, name: "error"} |
||||
warnValue = &levelValue{level: levelWarn, name: "warn"} |
||||
infoValue = &levelValue{level: levelInfo, name: "info"} |
||||
debugValue = &levelValue{level: levelDebug, name: "debug"} |
||||
) |
||||
|
||||
type level byte |
||||
|
||||
const ( |
||||
levelDebug level = 1 << iota |
||||
levelInfo |
||||
levelWarn |
||||
levelError |
||||
) |
||||
|
||||
type levelValue struct { |
||||
name string |
||||
level |
||||
} |
||||
|
||||
func (v *levelValue) String() string { return v.name } |
||||
func (v *levelValue) levelVal() {} |
||||
@ -0,0 +1,135 @@
|
||||
package log |
||||
|
||||
import "errors" |
||||
|
||||
// Logger is the fundamental interface for all log operations. Log creates a
|
||||
// log event from keyvals, a variadic sequence of alternating keys and values.
|
||||
// Implementations must be safe for concurrent use by multiple goroutines. In
|
||||
// particular, any implementation of Logger that appends to keyvals or
|
||||
// modifies or retains any of its elements must make a copy first.
|
||||
type Logger interface { |
||||
Log(keyvals ...interface{}) error |
||||
} |
||||
|
||||
// ErrMissingValue is appended to keyvals slices with odd length to substitute
|
||||
// the missing value.
|
||||
var ErrMissingValue = errors.New("(MISSING)") |
||||
|
||||
// With returns a new contextual logger with keyvals prepended to those passed
|
||||
// to calls to Log. If logger is also a contextual logger created by With or
|
||||
// WithPrefix, keyvals is appended to the existing context.
|
||||
//
|
||||
// The returned Logger replaces all value elements (odd indexes) containing a
|
||||
// Valuer with their generated value for each call to its Log method.
|
||||
func With(logger Logger, keyvals ...interface{}) Logger { |
||||
if len(keyvals) == 0 { |
||||
return logger |
||||
} |
||||
l := newContext(logger) |
||||
kvs := append(l.keyvals, keyvals...) |
||||
if len(kvs)%2 != 0 { |
||||
kvs = append(kvs, ErrMissingValue) |
||||
} |
||||
return &context{ |
||||
logger: l.logger, |
||||
// Limiting the capacity of the stored keyvals ensures that a new
|
||||
// backing array is created if the slice must grow in Log or With.
|
||||
// Using the extra capacity without copying risks a data race that
|
||||
// would violate the Logger interface contract.
|
||||
keyvals: kvs[:len(kvs):len(kvs)], |
||||
hasValuer: l.hasValuer || containsValuer(keyvals), |
||||
} |
||||
} |
||||
|
||||
// WithPrefix returns a new contextual logger with keyvals prepended to those
|
||||
// passed to calls to Log. If logger is also a contextual logger created by
|
||||
// With or WithPrefix, keyvals is prepended to the existing context.
|
||||
//
|
||||
// The returned Logger replaces all value elements (odd indexes) containing a
|
||||
// Valuer with their generated value for each call to its Log method.
|
||||
func WithPrefix(logger Logger, keyvals ...interface{}) Logger { |
||||
if len(keyvals) == 0 { |
||||
return logger |
||||
} |
||||
l := newContext(logger) |
||||
// Limiting the capacity of the stored keyvals ensures that a new
|
||||
// backing array is created if the slice must grow in Log or With.
|
||||
// Using the extra capacity without copying risks a data race that
|
||||
// would violate the Logger interface contract.
|
||||
n := len(l.keyvals) + len(keyvals) |
||||
if len(keyvals)%2 != 0 { |
||||
n++ |
||||
} |
||||
kvs := make([]interface{}, 0, n) |
||||
kvs = append(kvs, keyvals...) |
||||
if len(kvs)%2 != 0 { |
||||
kvs = append(kvs, ErrMissingValue) |
||||
} |
||||
kvs = append(kvs, l.keyvals...) |
||||
return &context{ |
||||
logger: l.logger, |
||||
keyvals: kvs, |
||||
hasValuer: l.hasValuer || containsValuer(keyvals), |
||||
} |
||||
} |
||||
|
||||
// context is the Logger implementation returned by With and WithPrefix. It
|
||||
// wraps a Logger and holds keyvals that it includes in all log events. Its
|
||||
// Log method calls bindValues to generate values for each Valuer in the
|
||||
// context keyvals.
|
||||
//
|
||||
// A context must always have the same number of stack frames between calls to
|
||||
// its Log method and the eventual binding of Valuers to their value. This
|
||||
// requirement comes from the functional requirement to allow a context to
|
||||
// resolve application call site information for a Caller stored in the
|
||||
// context. To do this we must be able to predict the number of logging
|
||||
// functions on the stack when bindValues is called.
|
||||
//
|
||||
// Two implementation details provide the needed stack depth consistency.
|
||||
//
|
||||
// 1. newContext avoids introducing an additional layer when asked to
|
||||
// wrap another context.
|
||||
// 2. With and WithPrefix avoid introducing an additional layer by
|
||||
// returning a newly constructed context with a merged keyvals rather
|
||||
// than simply wrapping the existing context.
|
||||
type context struct { |
||||
logger Logger |
||||
keyvals []interface{} |
||||
hasValuer bool |
||||
} |
||||
|
||||
func newContext(logger Logger) *context { |
||||
if c, ok := logger.(*context); ok { |
||||
return c |
||||
} |
||||
return &context{logger: logger} |
||||
} |
||||
|
||||
// Log replaces all value elements (odd indexes) containing a Valuer in the
|
||||
// stored context with their generated value, appends keyvals, and passes the
|
||||
// result to the wrapped Logger.
|
||||
func (l *context) Log(keyvals ...interface{}) error { |
||||
kvs := append(l.keyvals, keyvals...) |
||||
if len(kvs)%2 != 0 { |
||||
kvs = append(kvs, ErrMissingValue) |
||||
} |
||||
if l.hasValuer { |
||||
// If no keyvals were appended above then we must copy l.keyvals so
|
||||
// that future log events will reevaluate the stored Valuers.
|
||||
if len(keyvals) == 0 { |
||||
kvs = append([]interface{}{}, l.keyvals...) |
||||
} |
||||
bindValues(kvs[:len(l.keyvals)]) |
||||
} |
||||
return l.logger.Log(kvs...) |
||||
} |
||||
|
||||
// LoggerFunc is an adapter to allow use of ordinary functions as Loggers. If
|
||||
// f is a function with the appropriate signature, LoggerFunc(f) is a Logger
|
||||
// object that calls f.
|
||||
type LoggerFunc func(...interface{}) error |
||||
|
||||
// Log implements Logger by calling f(keyvals...).
|
||||
func (f LoggerFunc) Log(keyvals ...interface{}) error { |
||||
return f(keyvals...) |
||||
} |
||||
@ -0,0 +1,62 @@
|
||||
package log |
||||
|
||||
import ( |
||||
"bytes" |
||||
"io" |
||||
"sync" |
||||
|
||||
"github.com/go-logfmt/logfmt" |
||||
) |
||||
|
||||
type logfmtEncoder struct { |
||||
*logfmt.Encoder |
||||
buf bytes.Buffer |
||||
} |
||||
|
||||
func (l *logfmtEncoder) Reset() { |
||||
l.Encoder.Reset() |
||||
l.buf.Reset() |
||||
} |
||||
|
||||
var logfmtEncoderPool = sync.Pool{ |
||||
New: func() interface{} { |
||||
var enc logfmtEncoder |
||||
enc.Encoder = logfmt.NewEncoder(&enc.buf) |
||||
return &enc |
||||
}, |
||||
} |
||||
|
||||
type logfmtLogger struct { |
||||
w io.Writer |
||||
} |
||||
|
||||
// NewLogfmtLogger returns a logger that encodes keyvals to the Writer in
|
||||
// logfmt format. Each log event produces no more than one call to w.Write.
|
||||
// The passed Writer must be safe for concurrent use by multiple goroutines if
|
||||
// the returned Logger will be used concurrently.
|
||||
func NewLogfmtLogger(w io.Writer) Logger { |
||||
return &logfmtLogger{w} |
||||
} |
||||
|
||||
func (l logfmtLogger) Log(keyvals ...interface{}) error { |
||||
enc := logfmtEncoderPool.Get().(*logfmtEncoder) |
||||
enc.Reset() |
||||
defer logfmtEncoderPool.Put(enc) |
||||
|
||||
if err := enc.EncodeKeyvals(keyvals...); err != nil { |
||||
return err |
||||
} |
||||
|
||||
// Add newline to the end of the buffer
|
||||
if err := enc.EndRecord(); err != nil { |
||||
return err |
||||
} |
||||
|
||||
// The Logger interface requires implementations to be safe for concurrent
|
||||
// use by multiple goroutines. For this implementation that means making
|
||||
// only one call to l.w.Write() for each call to Log.
|
||||
if _, err := l.w.Write(enc.buf.Bytes()); err != nil { |
||||
return err |
||||
} |
||||
return nil |
||||
} |
||||
@ -0,0 +1,8 @@
|
||||
package log |
||||
|
||||
type nopLogger struct{} |
||||
|
||||
// NewNopLogger returns a logger that doesn't do anything.
|
||||
func NewNopLogger() Logger { return nopLogger{} } |
||||
|
||||
func (nopLogger) Log(...interface{}) error { return nil } |
||||
@ -0,0 +1,116 @@
|
||||
package log |
||||
|
||||
import ( |
||||
"io" |
||||
"log" |
||||
"regexp" |
||||
"strings" |
||||
) |
||||
|
||||
// StdlibWriter implements io.Writer by invoking the stdlib log.Print. It's
|
||||
// designed to be passed to a Go kit logger as the writer, for cases where
|
||||
// it's necessary to redirect all Go kit log output to the stdlib logger.
|
||||
//
|
||||
// If you have any choice in the matter, you shouldn't use this. Prefer to
|
||||
// redirect the stdlib log to the Go kit logger via NewStdlibAdapter.
|
||||
type StdlibWriter struct{} |
||||
|
||||
// Write implements io.Writer.
|
||||
func (w StdlibWriter) Write(p []byte) (int, error) { |
||||
log.Print(strings.TrimSpace(string(p))) |
||||
return len(p), nil |
||||
} |
||||
|
||||
// StdlibAdapter wraps a Logger and allows it to be passed to the stdlib
|
||||
// logger's SetOutput. It will extract date/timestamps, filenames, and
|
||||
// messages, and place them under relevant keys.
|
||||
type StdlibAdapter struct { |
||||
Logger |
||||
timestampKey string |
||||
fileKey string |
||||
messageKey string |
||||
} |
||||
|
||||
// StdlibAdapterOption sets a parameter for the StdlibAdapter.
|
||||
type StdlibAdapterOption func(*StdlibAdapter) |
||||
|
||||
// TimestampKey sets the key for the timestamp field. By default, it's "ts".
|
||||
func TimestampKey(key string) StdlibAdapterOption { |
||||
return func(a *StdlibAdapter) { a.timestampKey = key } |
||||
} |
||||
|
||||
// FileKey sets the key for the file and line field. By default, it's "caller".
|
||||
func FileKey(key string) StdlibAdapterOption { |
||||
return func(a *StdlibAdapter) { a.fileKey = key } |
||||
} |
||||
|
||||
// MessageKey sets the key for the actual log message. By default, it's "msg".
|
||||
func MessageKey(key string) StdlibAdapterOption { |
||||
return func(a *StdlibAdapter) { a.messageKey = key } |
||||
} |
||||
|
||||
// NewStdlibAdapter returns a new StdlibAdapter wrapper around the passed
|
||||
// logger. It's designed to be passed to log.SetOutput.
|
||||
func NewStdlibAdapter(logger Logger, options ...StdlibAdapterOption) io.Writer { |
||||
a := StdlibAdapter{ |
||||
Logger: logger, |
||||
timestampKey: "ts", |
||||
fileKey: "caller", |
||||
messageKey: "msg", |
||||
} |
||||
for _, option := range options { |
||||
option(&a) |
||||
} |
||||
return a |
||||
} |
||||
|
||||
func (a StdlibAdapter) Write(p []byte) (int, error) { |
||||
result := subexps(p) |
||||
keyvals := []interface{}{} |
||||
var timestamp string |
||||
if date, ok := result["date"]; ok && date != "" { |
||||
timestamp = date |
||||
} |
||||
if time, ok := result["time"]; ok && time != "" { |
||||
if timestamp != "" { |
||||
timestamp += " " |
||||
} |
||||
timestamp += time |
||||
} |
||||
if timestamp != "" { |
||||
keyvals = append(keyvals, a.timestampKey, timestamp) |
||||
} |
||||
if file, ok := result["file"]; ok && file != "" { |
||||
keyvals = append(keyvals, a.fileKey, file) |
||||
} |
||||
if msg, ok := result["msg"]; ok { |
||||
keyvals = append(keyvals, a.messageKey, msg) |
||||
} |
||||
if err := a.Logger.Log(keyvals...); err != nil { |
||||
return 0, err |
||||
} |
||||
return len(p), nil |
||||
} |
||||
|
||||
const ( |
||||
logRegexpDate = `(?P<date>[0-9]{4}/[0-9]{2}/[0-9]{2})?[ ]?` |
||||
logRegexpTime = `(?P<time>[0-9]{2}:[0-9]{2}:[0-9]{2}(\.[0-9]+)?)?[ ]?` |
||||
logRegexpFile = `(?P<file>.+?:[0-9]+)?` |
||||
logRegexpMsg = `(: )?(?P<msg>.*)` |
||||
) |
||||
|
||||
var ( |
||||
logRegexp = regexp.MustCompile(logRegexpDate + logRegexpTime + logRegexpFile + logRegexpMsg) |
||||
) |
||||
|
||||
func subexps(line []byte) map[string]string { |
||||
m := logRegexp.FindSubmatch(line) |
||||
if len(m) < len(logRegexp.SubexpNames()) { |
||||
return map[string]string{} |
||||
} |
||||
result := map[string]string{} |
||||
for i, name := range logRegexp.SubexpNames() { |
||||
result[name] = string(m[i]) |
||||
} |
||||
return result |
||||
} |
||||
@ -0,0 +1,116 @@
|
||||
package log |
||||
|
||||
import ( |
||||
"io" |
||||
"sync" |
||||
"sync/atomic" |
||||
) |
||||
|
||||
// SwapLogger wraps another logger that may be safely replaced while other
|
||||
// goroutines use the SwapLogger concurrently. The zero value for a SwapLogger
|
||||
// will discard all log events without error.
|
||||
//
|
||||
// SwapLogger serves well as a package global logger that can be changed by
|
||||
// importers.
|
||||
type SwapLogger struct { |
||||
logger atomic.Value |
||||
} |
||||
|
||||
type loggerStruct struct { |
||||
Logger |
||||
} |
||||
|
||||
// Log implements the Logger interface by forwarding keyvals to the currently
|
||||
// wrapped logger. It does not log anything if the wrapped logger is nil.
|
||||
func (l *SwapLogger) Log(keyvals ...interface{}) error { |
||||
s, ok := l.logger.Load().(loggerStruct) |
||||
if !ok || s.Logger == nil { |
||||
return nil |
||||
} |
||||
return s.Log(keyvals...) |
||||
} |
||||
|
||||
// Swap replaces the currently wrapped logger with logger. Swap may be called
|
||||
// concurrently with calls to Log from other goroutines.
|
||||
func (l *SwapLogger) Swap(logger Logger) { |
||||
l.logger.Store(loggerStruct{logger}) |
||||
} |
||||
|
||||
// NewSyncWriter returns a new writer that is safe for concurrent use by
|
||||
// multiple goroutines. Writes to the returned writer are passed on to w. If
|
||||
// another write is already in progress, the calling goroutine blocks until
|
||||
// the writer is available.
|
||||
//
|
||||
// If w implements the following interface, so does the returned writer.
|
||||
//
|
||||
// interface {
|
||||
// Fd() uintptr
|
||||
// }
|
||||
func NewSyncWriter(w io.Writer) io.Writer { |
||||
switch w := w.(type) { |
||||
case fdWriter: |
||||
return &fdSyncWriter{fdWriter: w} |
||||
default: |
||||
return &syncWriter{Writer: w} |
||||
} |
||||
} |
||||
|
||||
// syncWriter synchronizes concurrent writes to an io.Writer.
|
||||
type syncWriter struct { |
||||
sync.Mutex |
||||
io.Writer |
||||
} |
||||
|
||||
// Write writes p to the underlying io.Writer. If another write is already in
|
||||
// progress, the calling goroutine blocks until the syncWriter is available.
|
||||
func (w *syncWriter) Write(p []byte) (n int, err error) { |
||||
w.Lock() |
||||
n, err = w.Writer.Write(p) |
||||
w.Unlock() |
||||
return n, err |
||||
} |
||||
|
||||
// fdWriter is an io.Writer that also has an Fd method. The most common
|
||||
// example of an fdWriter is an *os.File.
|
||||
type fdWriter interface { |
||||
io.Writer |
||||
Fd() uintptr |
||||
} |
||||
|
||||
// fdSyncWriter synchronizes concurrent writes to an fdWriter.
|
||||
type fdSyncWriter struct { |
||||
sync.Mutex |
||||
fdWriter |
||||
} |
||||
|
||||
// Write writes p to the underlying io.Writer. If another write is already in
|
||||
// progress, the calling goroutine blocks until the fdSyncWriter is available.
|
||||
func (w *fdSyncWriter) Write(p []byte) (n int, err error) { |
||||
w.Lock() |
||||
n, err = w.fdWriter.Write(p) |
||||
w.Unlock() |
||||
return n, err |
||||
} |
||||
|
||||
// syncLogger provides concurrent safe logging for another Logger.
|
||||
type syncLogger struct { |
||||
mu sync.Mutex |
||||
logger Logger |
||||
} |
||||
|
||||
// NewSyncLogger returns a logger that synchronizes concurrent use of the
|
||||
// wrapped logger. When multiple goroutines use the SyncLogger concurrently
|
||||
// only one goroutine will be allowed to log to the wrapped logger at a time.
|
||||
// The other goroutines will block until the logger is available.
|
||||
func NewSyncLogger(logger Logger) Logger { |
||||
return &syncLogger{logger: logger} |
||||
} |
||||
|
||||
// Log logs keyvals to the underlying Logger. If another log is already in
|
||||
// progress, the calling goroutine blocks until the syncLogger is available.
|
||||
func (l *syncLogger) Log(keyvals ...interface{}) error { |
||||
l.mu.Lock() |
||||
err := l.logger.Log(keyvals...) |
||||
l.mu.Unlock() |
||||
return err |
||||
} |
||||
@ -0,0 +1,110 @@
|
||||
package log |
||||
|
||||
import ( |
||||
"runtime" |
||||
"strconv" |
||||
"strings" |
||||
"time" |
||||
) |
||||
|
||||
// A Valuer generates a log value. When passed to With or WithPrefix in a
|
||||
// value element (odd indexes), it represents a dynamic value which is re-
|
||||
// evaluated with each log event.
|
||||
type Valuer func() interface{} |
||||
|
||||
// bindValues replaces all value elements (odd indexes) containing a Valuer
|
||||
// with their generated value.
|
||||
func bindValues(keyvals []interface{}) { |
||||
for i := 1; i < len(keyvals); i += 2 { |
||||
if v, ok := keyvals[i].(Valuer); ok { |
||||
keyvals[i] = v() |
||||
} |
||||
} |
||||
} |
||||
|
||||
// containsValuer returns true if any of the value elements (odd indexes)
|
||||
// contain a Valuer.
|
||||
func containsValuer(keyvals []interface{}) bool { |
||||
for i := 1; i < len(keyvals); i += 2 { |
||||
if _, ok := keyvals[i].(Valuer); ok { |
||||
return true |
||||
} |
||||
} |
||||
return false |
||||
} |
||||
|
||||
// Timestamp returns a timestamp Valuer. It invokes the t function to get the
|
||||
// time; unless you are doing something tricky, pass time.Now.
|
||||
//
|
||||
// Most users will want to use DefaultTimestamp or DefaultTimestampUTC, which
|
||||
// are TimestampFormats that use the RFC3339Nano format.
|
||||
func Timestamp(t func() time.Time) Valuer { |
||||
return func() interface{} { return t() } |
||||
} |
||||
|
||||
// TimestampFormat returns a timestamp Valuer with a custom time format. It
|
||||
// invokes the t function to get the time to format; unless you are doing
|
||||
// something tricky, pass time.Now. The layout string is passed to
|
||||
// Time.Format.
|
||||
//
|
||||
// Most users will want to use DefaultTimestamp or DefaultTimestampUTC, which
|
||||
// are TimestampFormats that use the RFC3339Nano format.
|
||||
func TimestampFormat(t func() time.Time, layout string) Valuer { |
||||
return func() interface{} { |
||||
return timeFormat{ |
||||
time: t(), |
||||
layout: layout, |
||||
} |
||||
} |
||||
} |
||||
|
||||
// A timeFormat represents an instant in time and a layout used when
|
||||
// marshaling to a text format.
|
||||
type timeFormat struct { |
||||
time time.Time |
||||
layout string |
||||
} |
||||
|
||||
func (tf timeFormat) String() string { |
||||
return tf.time.Format(tf.layout) |
||||
} |
||||
|
||||
// MarshalText implements encoding.TextMarshaller.
|
||||
func (tf timeFormat) MarshalText() (text []byte, err error) { |
||||
// The following code adapted from the standard library time.Time.Format
|
||||
// method. Using the same undocumented magic constant to extend the size
|
||||
// of the buffer as seen there.
|
||||
b := make([]byte, 0, len(tf.layout)+10) |
||||
b = tf.time.AppendFormat(b, tf.layout) |
||||
return b, nil |
||||
} |
||||
|
||||
// Caller returns a Valuer that returns a file and line from a specified depth
|
||||
// in the callstack. Users will probably want to use DefaultCaller.
|
||||
func Caller(depth int) Valuer { |
||||
return func() interface{} { |
||||
_, file, line, _ := runtime.Caller(depth) |
||||
idx := strings.LastIndexByte(file, '/') |
||||
// using idx+1 below handles both of following cases:
|
||||
// idx == -1 because no "/" was found, or
|
||||
// idx >= 0 and we want to start at the character after the found "/".
|
||||
return file[idx+1:] + ":" + strconv.Itoa(line) |
||||
} |
||||
} |
||||
|
||||
var ( |
||||
// DefaultTimestamp is a Valuer that returns the current wallclock time,
|
||||
// respecting time zones, when bound.
|
||||
DefaultTimestamp = TimestampFormat(time.Now, time.RFC3339Nano) |
||||
|
||||
// DefaultTimestampUTC is a Valuer that returns the current time in UTC
|
||||
// when bound.
|
||||
DefaultTimestampUTC = TimestampFormat( |
||||
func() time.Time { return time.Now().UTC() }, |
||||
time.RFC3339Nano, |
||||
) |
||||
|
||||
// DefaultCaller is a Valuer that returns the file and line where the Log
|
||||
// method was invoked. It can only be used with log.With.
|
||||
DefaultCaller = Caller(3) |
||||
) |
||||
@ -0,0 +1,4 @@
|
||||
_testdata/ |
||||
_testdata2/ |
||||
logfmt-fuzz.zip |
||||
logfmt.test.exe |
||||
Some files were not shown because too many files have changed in this diff Show More
Loading…
Reference in new issue