import ballerina/http;
import ballerina/log;
import ballerina/runtime;
http:Client backendClientEP = new("http://localhost:8080", {
        circuitBreaker: {
            rollingWindow: {
                timeWindowInMillis: 10000,
                bucketSizeInMillis: 2000,
                requestVolumeThreshold: 0            },
            failureThreshold: 0.2,
            resetTimeInMillis: 10000,
            statusCodes: [400, 404, 500]        },
        timeoutInMillis: 2000
    });
@http:ServiceConfig {
    basePath: "/cb"
}
service circuitbreaker on new http:Listener(9090) {
    @http:ResourceConfig {
        methods: ["GET"],
        path: "/"
    }
    resource function invokeEndpoint(http:Caller caller, http:Request request) {
        var backendResponse = backendClientEP->forward("/hello", request);
        if (backendResponse is http:Response) {
            var responseToCaller = caller->respond(backendResponse);
            if (responseToCaller is http:ListenerError) {
                log:printError("Error sending response", responseToCaller);
            }
        } else {
            http:Response response = new;
            response.statusCode = http:STATUS_INTERNAL_SERVER_ERROR;
            response.setPayload(<string> backendResponse.detail()?.message);
            var responseToCaller = caller->respond(response);
            if (responseToCaller is http:ListenerError) {
                log:printError("Error sending response", responseToCaller);
            }
        }
    }
}int counter = 1;@http:ServiceConfig {
    basePath: "/hello"
}
service helloWorld on new http:Listener(8080) {
    @http:ResourceConfig {
        methods: ["GET"],
        path: "/"
    }
    resource function sayHello(http:Caller caller, http:Request req) {
        if (counter % 5 == 0) {
            runtime:sleep(5000);            var result = caller->respond("Hello World!!!");
            handleRespondResult(result);
        } else if (counter % 5 == 3) {
            http:Response res = new;
            res.statusCode = 500;
            res.setPayload(
                    "Internal error occurred while processing the request.");
            var result = caller->respond(res);
            handleRespondResult(result);
        } else {
            var result = caller->respond("Hello World!!!");
            handleRespondResult(result);
        }
        counter = counter + 1;
    }
}function handleRespondResult(error? result) {
    if (result is http:ListenerError) {
        log:printError("Error sending response from mock service", result);
    }
}

Circuit Breaker

The Circuit Breaker is used to gracefully handle network related errors, which occur when using the HTTP Client.

import ballerina/http;
import ballerina/log;
import ballerina/runtime;
http:Client backendClientEP = new("http://localhost:8080", {

The circuit breaker looks for errors across a rolling time window. After the circuit is broken, it does not send requests to the backend until the resetTime.

        circuitBreaker: {

Configuration options that control the behavior of the circuit breaker.

            rollingWindow: {

Failure calculation window. This is how long the circuit breaker keeps the statistics for the operations.

                timeWindowInMillis: 10000,

Time period in milliseconds for which the failure threshold is calculated.

                bucketSizeInMillis: 2000,

The granularity (in milliseconds) at which the time window slides. The RollingWindow is divided into buckets and slides by these increments. For an example, if this timeWindowInMillis is set to 10000 milliseconds and bucketSizeInMillis 2000, then RollingWindow breaks into sub windows with 2-second buckets. Stats are collected with respect to the buckets. As time rolls, a new bucket will be appended to the end of the window and the old bucket will be removed.

                requestVolumeThreshold: 0

Minimum number of requests in a RollingWindow that will trip the circuit.

            },
            failureThreshold: 0.2,

The threshold for request failures. When this threshold exceeds, the circuit trips. This is the ratio between failures and total requests and the ratio is considered only within the configured RollingWindow.

            resetTimeInMillis: 10000,

The time period (in milliseconds) to wait before attempting to make another request to the upstream service. When the failure threshold exceeds, the circuit trips to OPEN state. Once the circuit is in OPEN state circuit breaker waits for the time configured in resetTimeInMillis and switch the circuit to the HALF_OPEN state.

            statusCodes: [400, 404, 500]

HTTP response status codes that are considered as failures

        },
        timeoutInMillis: 2000
    });
@http:ServiceConfig {
    basePath: "/cb"
}
service circuitbreaker on new http:Listener(9090) {

Create an HTTP service bound to the endpoint (circuitBreakerEP).

    @http:ResourceConfig {
        methods: ["GET"],
        path: "/"
    }

Create a REST resource within the API.

    resource function invokeEndpoint(http:Caller caller, http:Request request) {
        var backendResponse = backendClientEP->forward("/hello", request);

The parameters include a reference to the caller and an object of the request data.

        if (backendResponse is http:Response) {
            var responseToCaller = caller->respond(backendResponse);
            if (responseToCaller is http:ListenerError) {
                log:printError("Error sending response", responseToCaller);
            }
        } else {
            http:Response response = new;
            response.statusCode = http:STATUS_INTERNAL_SERVER_ERROR;
            response.setPayload(<string> backendResponse.detail()?.message);
            var responseToCaller = caller->respond(response);
            if (responseToCaller is http:ListenerError) {
                log:printError("Error sending response", responseToCaller);
            }
        }
    }
}

If the backendResponse is an http:Response, it is sent back to the client. If backendResponse is an http:ClientError, an internal server error is returned to the client.

int counter = 1;

This sample service is used to mock connection timeouts and service outages. Mock a service outage by stopping/starting this service. This should run separately from the circuitBreakerDemo service.

@http:ServiceConfig {
    basePath: "/hello"
}
service helloWorld on new http:Listener(8080) {
    @http:ResourceConfig {
        methods: ["GET"],
        path: "/"
    }
    resource function sayHello(http:Caller caller, http:Request req) {
        if (counter % 5 == 0) {
            runtime:sleep(5000);

Delay the response by 5000 milliseconds to mimic the network level delays.

            var result = caller->respond("Hello World!!!");
            handleRespondResult(result);
        } else if (counter % 5 == 3) {
            http:Response res = new;
            res.statusCode = 500;
            res.setPayload(
                    "Internal error occurred while processing the request.");
            var result = caller->respond(res);
            handleRespondResult(result);
        } else {
            var result = caller->respond("Hello World!!!");
            handleRespondResult(result);
        }
        counter = counter + 1;
    }
}
function handleRespondResult(error? result) {
    if (result is http:ListenerError) {
        log:printError("Error sending response from mock service", result);
    }
}
# To start the services, navigate to the directory that contains the
# `.bal` file and use the `ballerina run` command.
$ ballerina run http_circuit_breaker.bal
[ballerina/http] started HTTP/WS listener 0.0.0.0:9090
[ballerina/http] started HTTP/WS listener 0.0.0.0:8080
2019-09-04 17:08:02,595 INFO  [ballerina/http/resiliency] - CircuitBreaker failure threshold exceeded. Circuit tripped from CLOSE to OPEN state.
2019-09-04 17:08:12,514 INFO  [ballerina/http/resiliency] - CircuitBreaker reset timeout reached. Circuit switched from OPEN to HALF_OPEN state.
2019-09-04 17:08:14,993 INFO  [ballerina/http/resiliency] - CircuitBreaker trial run  was successful. Circuit switched from HALF_OPEN to CLOSE state.
2019-09-04 17:08:20,003 ERROR [] - Error sending response from mock service : error {ballerina/http}GenericListenerError message=Connection between remote client and host is closed
2019-09-04 17:08:22,082 INFO  [ballerina/http/resiliency] - CircuitBreaker failure threshold exceeded. Circuit tripped from CLOSE to OPEN state.
2019-09-04 17:08:32,755 INFO  [ballerina/http/resiliency] - CircuitBreaker reset timeout reached. Circuit switched from OPEN to HALF_OPEN state.
2019-09-04 17:08:35,268 INFO  [ballerina/http/resiliency] - CircuitBreaker trial run  was successful. Circuit switched from HALF_OPEN to CLOSE state.
# The first two requests complete without any errors.
$ curl -v http://localhost:9090/cb
*   Trying ::1...
* TCP_NODELAY set
* Connected to localhost (::1) port 9090 (#0)
> GET /cb HTTP/1.1
> Host: localhost:9090
> User-Agent: curl/7.54.0
> Accept: */*
>
< HTTP/1.1 200 OK
< content-type: text/plain
< date: Wed, 4 Sep 2019 17:07:58 +0530
< server: ballerina/1.0.0-beta
< content-length: 14
<
* Connection #0 to host localhost left intact
Hello World!!!
$ curl -v http://localhost:9090/cb
*   Trying ::1...
* TCP_NODELAY set
* Connected to localhost (::1) port 9090 (#0)
> GET /cb HTTP/1.1
> Host: localhost:9090
> User-Agent: curl/7.54.0
> Accept: */*
>
< HTTP/1.1 200 OK
< content-type: text/plain
< date: Wed, 4 Sep 2019 17:08:00 +0530
< server: ballerina/1.0.0-beta
< content-length: 14
<
* Connection #0 to host localhost left intact
Hello World!!!
# The third request responds with a `500 Internal Server Error` because the
# mock service sends a `500` http status code when responding to every third
# request.
$ curl -v http://localhost:9090/cb
*   Trying ::1...
* TCP_NODELAY set
* Connected to localhost (::1) port 9090 (#0)
> GET /cb HTTP/1.1
> Host: localhost:9090
> User-Agent: curl/7.54.0
> Accept: */*
>
< HTTP/1.1 500 Internal Server Error
< content-type: text/plain
< date: Wed, 4 Sep 2019 17:08:00 +0530
< server: ballerina/1.0.0-beta
< content-length: 53
<
* Connection #0 to host localhost left intact
Internal error occurred while processing the request.
# Subsequent requests fail immediately since the reset timeout period has not
# elapsed.
$ curl -v http://localhost:9090/cb
*   Trying ::1...
* TCP_NODELAY set
* Connected to localhost (::1) port 9090 (#0)
> GET /cb HTTP/1.1
> Host: localhost:9090
> User-Agent: curl/7.54.0
> Accept: */*
>
< HTTP/1.1 500 Internal Server Error
< content-type: text/plain
< content-length: 99
< server: ballerina/1.0.0-beta
< date: Wed, 4 Sep 2019 17:08:02 +0530
<
* Connection #0 to host localhost left intact
Upstream service unavailable. Requests to upstream service will be suspended for 8340 milliseconds.
$ curl -v http://localhost:9090/cb
*   Trying ::1...
* TCP_NODELAY set
* Connected to localhost (::1) port 9090 (#0)
> GET /cb HTTP/1.1
> Host: localhost:9090
> User-Agent: curl/7.54.0
> Accept: */*
>
< HTTP/1.1 500 Internal Server Error
< content-type: text/plain
< content-length: 99
< server: ballerina/1.0.0-beta
< date: Wed, 4 Sep 2019 17:08:04 +0530
<
* Connection #0 to host localhost left intact
Upstream service unavailable. Requests to upstream service will be suspended for 6079 milliseconds.
$ curl -v http://localhost:9090/cb
*   Trying ::1...
* TCP_NODELAY set
* Connected to localhost (::1) port 9090 (#0)
> GET /cb HTTP/1.1
> Host: localhost:9090
> User-Agent: curl/7.54.0
> Accept: */*
>
< HTTP/1.1 500 Internal Server Error
< content-type: text/plain
< content-length: 99
< server: ballerina/1.0.0-beta
< date: Wed, 4 Sep 2019 17:08:07 +0530
<
* Connection #0 to host localhost left intact
Upstream service unavailable. Requests to upstream service will be suspended for 3475 milliseconds.
# The request sent immediately after the timeout period expires, is the trial
# request. It is sent to see if the backend service is back to normal.
# If this request is successful, the circuit is set to `close` and normal
# functionality resumes.
$ curl -v http://localhost:9090/cb
*   Trying ::1...
* TCP_NODELAY set
* Connected to localhost (::1) port 9090 (#0)
> GET /cb HTTP/1.1
> Host: localhost:9090
> User-Agent: curl/7.54.0
> Accept: */*
>
< HTTP/1.1 200 OK
< content-type: text/plain
< date: Wed, 4 Sep 2019 17:08:12 +0530
< server: ballerina/1.0.0-beta
< content-length: 14
<
* Connection #0 to host localhost left intact
Hello World!!!
# The fifth request times out because the mock service times out when
# responding to every fifth request.
$ curl -v http://localhost:9090/cb
*   Trying ::1...
* TCP_NODELAY set
* Connected to localhost (::1) port 9090 (#0)
> GET /cb HTTP/1.1
> Host: localhost:9090
> User-Agent: curl/7.54.0
> Accept: */*
>
< HTTP/1.1 500 Internal Server Error
< content-type: text/plain
< content-length: 57
< server: ballerina/1.0.0-beta
< date: Wed, 4 Sep 2019 17:08:17 +0530
<
* Connection #0 to host localhost left intact
Idle timeout triggered before initiating inbound response
# Subsequent requests fail immediately since the reset timeout period has not
# elapsed.
$ curl -v http://localhost:9090/cb
*   Trying ::1...
* TCP_NODELAY set
* Connected to localhost (::1) port 9090 (#0)
> GET /cb HTTP/1.1
> Host: localhost:9090
> User-Agent: curl/7.54.0
> Accept: */*
>
< HTTP/1.1 500 Internal Server Error
< content-type: text/plain
< content-length: 99
< server: ballerina/1.0.0-beta
< date: Wed, 4 Sep 2019 17:08:22 +0530
<
* Connection #0 to host localhost left intact
Upstream service unavailable. Requests to upstream service will be suspended for 4917 milliseconds.
# The request sent immediately after the timeout period expires, is the trial
# request. It is sent to see if the backend service is back to normal.
# If this request is successful, the circuit is set to `close` and normal
# functionality resumes.
$ curl -v http://localhost:9090/cb
*   Trying ::1...
* TCP_NODELAY set
* Connected to localhost (::1) port 9090 (#0)
> GET /cb HTTP/1.1
> Host: localhost:9090
> User-Agent: curl/7.54.0
> Accept: */*
>
< HTTP/1.1 200 OK
< content-type: text/plain
< date: Wed, 4 Sep 2019 17:08:32 +0530
< server: ballerina/1.0.0-beta
< content-length: 14
<
* Connection #0 to host localhost left intact
Hello World!!!
# Since the immediate request after the timeout expired was successful, the
# requests sent after that complete normally.
$ curl -v http://localhost:9090/cb
*   Trying ::1...
* TCP_NODELAY set
* Connected to localhost (::1) port 9090 (#0)
> GET /cb HTTP/1.1
> Host: localhost:9090
> User-Agent: curl/7.54.0
> Accept: */*
>
< HTTP/1.1 200 OK
< content-type: text/plain
< date: Wed, 4 Sep 2019 17:08:35 +0530
< server: ballerina/1.0.0-beta
< content-length: 14
<
* Connection #0 to host localhost left intact
Hello World!!!