To support Prometheus as the metrics reporter, an HTTP endpoint starts with the context of /metrics in default port 9797 when starting the Ballerina service.

Configure advanced metrics

You can set up prometheus for your Ballerina project using configuration similar to the following in your Config.toml file.

[ballerina.observe]
metricsEnabled=true
metricsReporter="prometheus"

[ballerinax.prometheus]
port=9797
host="0.0.0.0"

Set up the external systems for metrics

There are mainly two systems involved in collecting and visualizing the metrics. Prometheus is used to collect the metrics from the Ballerina service while Grafana can be used to connect to Prometheus and visualize the metrics on the dashboard.

Set up Prometheus

Prometheus is used as the monitoring system, which pulls out the metrics collected from the Ballerina /metrics service. This section focuses on the quick installation of Prometheus with Docker and the configuration required to collect metrics from the Ballerina service with the default configurations. Follow the steps below to configure Prometheus.

Tip: There are many other ways to install Prometheus and you can find possible options from the installation guide. The easiest option is to use precompiled binaries listed in Downloads.

1. Create a prometheus.yml file in a directory.

2. Add the following content to the prometheus.yml file.

   global:
     scrape_interval:     15s
     evaluation_interval: 15s
   
   scrape_configs:
     - job_name: 'prometheus'
       static_configs:
         - targets: ['a.b.c.d:9797']

   Here, the 'a.b.c.d:9797' targets should contain the host and port of the /metrics service that is exposed from Ballerina for metrics collection. Add the IP of the host in which the Ballerina service is running as a.b.c.d and its port (default 9797). If you need more information, go to the Prometheus documentation.

   If your Ballerina service is running on localhost and Prometheus in a Docker container, add the target as host.docker.internal:9797 to access the localhost from Docker.

3. Start the Prometheus server in a Docker container with the command below.

   $ docker run -p 19090:9090 -v <path_to_prometheus.yml>:/etc/prometheus/ prom/prometheus
   

4. Go to http://localhost:19090/ and check whether you can see the Prometheus graph. Ballerina metrics should appear in Prometheus graph's metrics list when the Ballerina service is started. You can also use the following command to get the metrics.

   $ curl http://localhost:9797/metrics
   

Set up Grafana

Let’s use Grafana to visualize metrics in a dashboard. For this, we need to install Grafana and configure Prometheus as a data source. Follow the steps below to configure Grafana.

1. Start Grafana as a Docker container with the command below.

   $ docker run -d --name=grafana -p 3000:3000 grafana/grafana
   

   For more information, go to Grafana in Docker Hub.

2. Go to http://localhost:3000/ to access the Grafana dashboard running on Docker.

3. Login to the dashboard with the default user, username: admin and password: admin

4. Add Prometheus as a data source with Browser access configuration as provided below.

   

5. Import the Grafana dashboard designed to visualize Ballerina metrics from https://grafana.com/dashboards/5841 as shown below.

This dashboard consists of service and client invocation level metrics in near real-time view.

The Ballerina HTTP Service Metrics Dashboard Panel will be as shown below.