Quick Start Guide

Observability

GoFr by default manages observability in different ways once the server starts:

Logs

Logs offer real-time information, providing valuable insights and immediate visibility into the ongoing state and activities of the system. It helps in identifying errors, debugging and troubleshooting, monitor performance, analyzing application usage, communications etc.

GoFr logger allows to customize log level which provides flexibility to adjust logs based on specific needs.

Logs are generated only for events equal to or above the specified log level, by default GoFr logs at INFO level. Log Level can be changed by setting the environment variable LOG_LEVEL value to WARN,DEBUG,ERROR,NOTICE or FATAL.

When GoFr server runs, it prints log for reading configs, database connection, requests, database queries, missing configs etc. They contain information such as request's correlation ID, status codes, request time etc.

Pretty Printed Logs

Logs are well-structured, they are of type JSON when exported to a file, such that they can be pushed to logging systems such as Loki, elastic search etc.

Metrics

Metrics enable performance monitoring by providing insights into response times, latency, throughput, resource utilization, tracking CPU, memory, and disk I/O consumption across services, facilitating capacity planning and scalability efforts.

Metrics play a pivotal role in fault detection and troubleshooting, offering visibility into system behavior.

They are instrumental in measuring and meeting service-level agreements (SLAs) to ensure expected performance and reliability.

GoFr publishes metrics to port: 2121 on /metrics endpoint in Prometheus format.

NameTypeDescription
app_go_numGCgaugeNumber of completed Garbage Collector cycles
app_go_routinesgaugeNumber of Go routines running
app_go_sysgaugeNumber of total bytes of memory
app_sys_memory_allocgaugeNumber of bytes allocated for heap objects
app_sys_total_allocgaugeNumber of cumulative bytes allocated for heap objects
app_infogaugeNumber of instances running with info of app and framework
app_http_responsehistogramResponse time of HTTP requests in seconds
app_http_service_responsehistogramResponse time of HTTP service requests in seconds
app_sql_open_connectionsgaugeNumber of open SQL connections
app_sql_inUse_connectionsgaugeNumber of inUse SQL connections
app_sql_statshistogramResponse time of SQL queries in milliseconds
app_redis_statshistogramResponse time of Redis commands in milliseconds
app_pubsub_publish_total_countcounterNumber of total publish operations
app_pubsub_publish_success_countcounterNumber of successful publish operations
app_pubsub_subscribe_total_countcounterNumber of total subscribe operations
app_pubsub_subscribe_success_countcounterNumber of successful subscribe operations

For example: When running application locally, you can access /metrics endpoint on port 2121 from: http://localhost:2121/metrics

GoFr also supports creating custom metrics.

Tracing

Tracing is a powerful tool for gaining insights into your application's behavior, identifying bottlenecks, and improving system performance. A trace is a tree of spans. It is a collective of observable signals showing the path of work through a system. A trace on its own is distinguishable by a TraceID.

In complex distributed systems, understanding how requests flow through the system is crucial for troubleshooting performance issues and identifying bottlenecks. Traditional logging approaches often fall short, providing limited visibility into the intricate interactions between components.

Automated Tracing in GoFr

GoFr automatically exports traces for all requests and responses. GoFr uses OpenTelemetry , a popular tracing framework, to automatically add traces to all requests and responses.

Automatic Correlation ID Propagation:

When a request enters your GoFr application, GoFr automatically generates a correlation-ID X-Correlation-ID and adds it to the response headers. This correlation ID is then propagated to all downstream requests. This means that you can track a request as it travels through your distributed system by simply looking at the correlation ID in the request headers.

Configuration & Usage:

GoFr has support for following trace-exporters:

1. Zipkin:

To see the traces install zipkin image using the following Docker command:

  docker run --name gofr-zipkin -p 2005:9411 -d openzipkin/zipkin:latest

Add Tracer configs in .env file, your .env will be updated to

APP_NAME=test-service
HTTP_PORT=9000

REDIS_HOST=localhost
REDIS_PORT=6379

DB_HOST=localhost
DB_USER=root
DB_PASSWORD=root123
DB_NAME=test_db
DB_PORT=3306

# tracing configs
TRACE_EXPORTER=zipkin
TRACER_HOST=localhost
TRACER_PORT=2005

LOG_LEVEL=DEBUG

NOTE: If the value of TRACER_PORT is not provided, GoFr uses port 9411 by default.

Open zipkin and search by TraceID (correlationID) to see the trace.

Zipkin traces

2. Jaeger:

To see the traces install jaeger image using the following Docker command:

docker run -d --name jaeger \
  -e COLLECTOR_OTLP_ENABLED=true \
  -p 16686:16686 \
  -p 14317:4317 \
  -p 14318:4318 \
  jaegertracing/all-in-one:1.41

Add Jaeger Tracer configs in .env file, your .env will be updated to

# ... no change in other env variables

# tracing configs
TRACE_EXPORTER=jaeger
TRACER_HOST=localhost
TRACER_PORT=14317

Open zipkin and search by TraceID (correlationID) to see the trace.

Jaeger traces

3. GoFr Tracer:

GoFr tracer is GoFr's own custom trace exporter as well as collector. You can search a trace by its TraceID (correlationID) in GoFr's own tracer service available anywhere, anytime.

Add GoFr Tracer configs in .env file, your .env will be updated to

# ... no change in other env variables

# tracing configs
TRACE_EXPORTER=gofr

Open gofr-tracer and search by TraceID (correlationID) to see the trace.