Last Updated on June 23, 2024

In distributed systems like Spring Boot applications, observability refers to the practice of understanding a system’s internal health and behavior by analyzing its external outputs.

In this blog post series we will explore components of observability, see how observability is implemented in spring framework and go through examples of logging, metrics and tracing implementation.

Logging vs. Metrics vs. Tracing

Logging can be considered the first line of defense for finding bugs and other blockages. It is also the most simplest way of observing your system programs.

Logs provide valuable details about user actions, system interactions, errors, and warnings like, Number of login attempts, API request latency and Database connection failures

Metrics provide quantitative data points that reflect various aspects of your application’s performance. 

This includes measurements like HTTP request latency, database connection pool size, memory usage and thread pool activity.

Tracing sheds light on the intricate flow of a request as it traverses various components within your application.

It allows you to visualize the entire journey of a request, pinpointing potential bottlenecks and inefficiencies.

Examples of tracing would be E-commerce checkout or social media post creation process.

Observability stands on three pillars – Logs, Traces, and Metrics.

You can correlate log messages with specific metrics or trace events to get a richer context. 

For instance, a high error rate in logs might be linked to a spike in CPU usage metric, and tracing that error can pinpoint the exact service causing the overload. 

Logs provide the “why” behind what metrics show, while metrics offer quantifiable data to understand the severity of issues revealed in logs. 

Traces visualize the flow, helping you see how different parts of the system are interconnected and contributing to the overall behavior.

Spring Boot Micrometer

Spring Boot Actuator integrates seamlessly with Spring Boot Micrometer, a tool for collecting and exporting dimensional metrics about your application.

This combined approach provides a robust and flexible monitoring solution.

It is a sub-project of Spring Boot that provides a set of features for monitoring and managing your Spring Boot applications in production environments.

It essentially equips your application with built-in tools to track its health, performance, and configuration.

To start with you need to include actuator dependency in pom.xml

<dependency>
  <groupId>org.springframework.boot</groupId>
  <artifactId>spring-boot-starter-actuator</artifactId>
</dependency>

Micrometer Metrics

Actuator provides a built-in endpoint (/actuator/metrics) that exposes these metrics collected by Micrometer.

The specific metrics you’ll see depend on the dependencies and configurations in your Spring Boot application.

JVM Metrics:

  • Memory usage (heap, non-heap)
  • Garbage collection activity (frequency, duration)
  • Thread pool usage statistics (active threads, queue size)
  • Class loading information (number of loaded classes)

HTTP Metrics:

  • Request counts, broken down by HTTP method (GET, POST, etc.) and endpoint
  • Response times (distribution of request processing times)
  • HTTP status codes (number of successful requests, errors)

Cache Metrics:

  • Cache hits and misses (for various caches used in your application)
  • Cache size and eviction statistics

Database Metrics:

  • Database connection pool size and usage
  • SQL statement execution times (if using Micrometer integrations for specific databases)

We can say that this is a fairly comprehensive list already for out-of-the-box behavior (just dependant on which beans to try

To try these HTTP requests/responses out we’ll create a quick demo app using jsonplaceholder API for testing and prototyping to test some of the integration areas.

We can define an interface for accessing posts endpoint:

import org.springframework.web.service.annotation.GetExchange;
import java.util.List;

public interface MyInterface {
   @GetExchange("/posts")
   List<Post> getAll();
}

then client proxy in a config file

@Configuration
public class AppConfig {
    @Bean
    MyInterface restClient() {
        RestClient restClient = RestClient.create("https://jsonplaceholder.typicode.com");
        HttpServiceProxyFactory factory = HttpServiceProxyFactory.builderFor(RestClientAdapter.create(restClient)).build();
        return factory.createClient(MyInterface.class);
    }
}

Controller accessing the root endpoint

import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;

import java.util.List;

@RestController
@RequestMapping("/")
public class MyController {

    private final MyInterface myInterface;

    public MyController(MyInterface myInterface) {
        this.myInterface = myInterface;
    }

    @GetMapping("")
    List<Post> getAll() {
        return myInterface.getAll();
    }
}

We see there is a new name called “http.server.requests” and by accessing:

GET /actuator/metrics/http.server.reguests

{
  "name": "http.server.requests",
  "baseUnit": "seconds",
  "measurements": [
    {
      "statistic": "COUNT",
      "value": 13
    },
    {
      "statistic": "TOTAL_TIME",
      "value": 9.603813307
    },
    {
      "statistic": "MAX",
      "value": 0.016824811
    }
  ],
.......

Custom Metrics

To create and publish your own custom metrics, spring creates a micrometer MeterRegistry instance by default, that you can then inject and use to create custom counters, gauges and timers.

We can define a counter simply counting how many times an endpoint was called

import io.micrometer.core.instrument.Counter;
import io.micrometer.core.instrument.MeterRegistry;

@RestController
public class MyController {

    private final Counter myCounter;

    public MyController(MeterRegistry registry) {
        myCounter = Counter.builder("hello.counter")
                .description("Counts hello endpoint access")
                .tags("region", "us-east")
                .register(registry);
    }

    @GetMapping("/hello")
    public String sayHello() {
        myCounter.increment();
        return "Hello World!";
    }
}

Gives us

GET /actuator/metrics/hello.counter

{
  "name": "hello.counter",
  "description": "Counts hello access",
  "measurements": [
    {
      "statistic": "COUNT",
      "value": 3.0
    }
  ],
  ..........

Micrometer Observation API

What if we want to observe our custom method (e.g., processing an order, rendering a web page), Micrometer handles capturing relevant metrics, tracing information, and even logs (depending on your configuration) associated with that observation.

Spring Micrometer Observation API integrates seamlessly with existing Micrometer features and libraries for various monitoring systems.

It declares an @Observed annotation with an aspect implementation based on AspectJ.

To get this to work, we need to add the AOP dependency to our project:

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-aop</artifactId>
</dependency>

Then we write service:

import io.micrometer.observation.annotation.Observed;
import org.springframework.stereotype.Service;

@Observed(name = "myService")
@Service
public class MyService {
    public String sayHello() {
        return "Hello World!";
    }
}

In our controller:

 @GetMapping("/hello")
    String sayHello() {
        return myService.sayHello();
    }

and if we execute GET /hello endpoint several times

using http://localhost:8080/actuator/metrics/myService and we’ll get a result like this

{
  "name": "myService",
  "baseUnit": "seconds",
  "measurements": [
    {
      "statistic": "COUNT",
      "value": 2.0
    },
    {
      "statistic": "TOTAL_TIME",
      "value": 0.036353101
    },
    {
      "statistic": "MAX",
      "value": 0.035743267
    }
  ],
...............

The Micrometer Observation API is a relatively new approach introduced in Spring Boot 3 that simplifies how you instrument your application for observability. It acts as a higher-level abstraction on top of existing Micrometer capabilities.

Aggregation and Visualization

Actuator primarily focuses on exposing metrics for a single application instance. It doesn’t offer built-in mechanisms for aggregating and visualizing metrics across multiple instances.

In the following blog post we will explore how various monitoring systems like Prometheus, Datadog, and Grafana allow you to collect metrics data from multiple application instances and aggregate and visualize them into a central location.

Related Posts

Scroll to Top