Manage data persistence with bal persist

This guide helps you understand the basics of the `bal persist` feature, which allows you to manage data persistence easily. This same data service can also be written using the Ballerina SQL connectors, which require you to write SQL queries to perform CRUD operations against the DB servers. With the `bal persist` feature, you only need to write the `data model`. Based on the model, the client object and record types are generated to interact with the data store.

Set up the prerequisites

To complete this tutorial, you need:

Ballerina 2201.9.0 (Swan Lake) or greater
A text editor

Tip: Preferably, Visual Studio Code with the Ballerina extension installed.

A command terminal

Understand the implementation

This guide describes how to interact with the data store and perform operations against it using bal persist. This will cover the basic use case of creating, reading, updating, and deleting records on a data store in an organization. It also elaborates on how you can create an HTTP RESTful API using Ballerina that can be used to perform basic CRUD operations on the data store.

Data Service Architecture

Info: This guide uses an in-memory data store for simplicity. However, the described methodology can also be applied to work with all other supported data stores. For more information about other supported data stores, see Supported Data Stores.

Create a Ballerina package

Ballerina uses packages to group code. You need to create a Ballerina package and write the business logic in it. In the terminal, execute the command below to create the Ballerina package for the implementation.

Info: For more information on Ballerina packages, see Organize Ballerina code.


$ bal new rainier

This creates a directory named rainier with the files below.


.
├── rainier
│   ├── Ballerina.toml
│   └── main.bal

Initialize `bal persist` in the project

The bal persist initialization takes care of the basics needed to start the development. In the terminal, execute the command below to initialize bal persist in the Ballerina package.


$ bal persist init

With the bal persist init command, the persist directory is created in the Ballerina package. Inside the persist directory, an empty schema file named model.bal is also created. This file is used to define the application's data model.

This changes the Ballerina package as follows.


├── rainier
│   ├── persist
│   │		└── model.bal
│   ├── Ballerina.toml
│   └── main.bal

The next step is to define your data model in the schema file in the persist/model.bal file.

Model your data

The data model can be defined in the schema file inside the persist directory. You can use the empty file added by the persist init command for this.

For more information on defining a data model, see the Data Model Definition documentation. In order to simplify this demonstration, only the Employee entity is added to the schema file. You can add as many as you want for your application and add relationships between entities. Once defined, the schema file will be as follows.


import ballerina/persist as _;
import ballerina/time;

public type Employee record {|
   readonly string id;
   string firstName;
   string lastName;
   string email;
   string phone;
   time:Date hireDate;
   string? managerId;
   string jobTitle;
|};

This model is used to set up the underlying datastore (e.g., set up the tables in the underlying database) and also as the base for the generated client APIs.

Generate the client object, types, and scripts

Now, you can generate the client objects, types, and setup scripts(if needed) for your model by running the command in your terminal.

Note: The client APIs can be generated using the bal persist generate command which is one time generation and the generated client code is a part of the project. We can also integrate the client code generation with the build process of the project by executing the bal persist add command. This will add the client code generation as a build task in the Ballerina.toml file. For more information, see Persist CLI Commands.


$ bal persist generate --datastore inmemory --module store

This changes the Ballerina package as follows.


├── rainier
│   ├── modules
│   │		└── store
│   │		│	├── persist_client.bal
│   │		│	└── persist_types.bal
│   ├── persist
│   │		└── model.bal
│   ├── Ballerina.toml
│   └── main.bal

The persist generate will parse the persist/model.bal definition file and generate the following.

File	Description
`persist_client.bal`	This is the client that is used to persist and retrieve data.
`persist_types.bal`	This contains the record types that will be used to persist and retrieve data.

Note: All of the above generated BAL files should not be modified.

Note: If you use other data stores, additional files will get created to handle runtime configurations and scripts to set up the database/worksheet. Since in-memory is used, you don’t need to do any configuration or setting-up before running the application.

Query your database with client API

First, you need to instantiate the generated client object inside the rainier.store module like below.


import rainier.store;

store:Client sClient = check new();

You can use this sClient as the client to manage data persistence. It provides five resource methods to manage data persistence. Let’s explore each of the methods of the client separately. You can use the main.bal file created inside your project to explore the functionalities.

Note: If you are using SQL database, additional two remote functions will be generated to perform native SQL queries. You can use these functions to perform complex queries that are not supported by the client API.

Create a new `Employee` record

Let’s start with a query to create a new Employee record in the database and log the results to the console. Update the following in your main.bal file.


import ballerina/io;
import rainier.store;


final store:Client sClient = check new();


public function main() returns error? {
   store:EmployeeInsert employee1 = {
       id: "emp_01",
       firstName: "John",
       lastName: "Doe",
       email: "johnd@xyz.com",
       phone: "1234567890",
       hireDate: {
           year: 2020,
           month: 10,
           day: 10
       },
       managerId: "mng_01",
       jobTitle: "Software Engineer"
   };


   string[] employeeIds = check sClient->/employees.post([employee1]);
   io:println("Inserted employee id: " + employeeIds[0]);
}

You can run your Ballerina project with the following command.


$ bal run
Compiling source
        foo/rainier:0.1.0


Running executable

Inserted employee id: emp_01

This creates the first database record with the client API. The next sections describe how to read data from the database.

Retrieve all `Employee` records

Let’s try to fetch all the records inserted into the database. The client API offers the get resource method, which returns a stream of the return type. The return type can be either a complete Employee record or a custom record with a subset of fields.

Replace the previous code and add the new get call instead.


import ballerina/io;
import rainier.store;
import ballerina/persist;


final store:Client sClient = check new ();


public function main() returns error? {
   // Get the complete `Employee` record.
   stream<store:Employee, persist:Error?> employees = sClient->/employees;
   check from var employee in employees
       do {
           io:println(employee);
       };


   // Get only the `id`, `firstName`, and `lastName` fields.
   stream<EmployeeName, persist:Error?> empNames = sClient->/employees;
   check from var name in empNames
       do {
           io:println(name);
       };
}


type EmployeeName record {|
   string id;
   string firstName;
   string lastName;
|};

Run the Ballerina project again.


$ bal run

Compiling source
        foo/rainier:0.1.0


Running executable

{"id":"emp_01","firstName":"John","lastName":"Doe","email":"johnd@xyz.com","phone":"1234567890","hireDate":{"year":2020,"month":10,"day":10},"manager_id":"mng_01","job_title":"Software Engineer"}

{"employee_id":"emp_01","first_name":"John","last_name":"Doe"}

Note: Even if there is a single Employee record, the resource method returns a Ballerina stream object and you need to iterate through the stream to access the records. The client also provides the get by key method, which returns only one record. Let’s explore that method.

Retrieve the `Employee` record by key

If you know the key of the record you need to fetch from the database, you can pass the key as a path param to the get method.

Replace the previous code and add the new get by key call instead,


import ballerina/io;
import rainier.store;


final store:Client sClient = check new ();


public function main() returns error? {
   string empId = "emp_01";
   // Get the complete `Employee` record.
   store:Employee employee = check sClient->/employees/[empId];
   io:println(employee);


   // Get only the `employee_id`, `first_name`, and `last_name` fields.
   EmployeeName employeeName = check sClient->/employees/[empId];
   io:println(employeeName);
}


type EmployeeName record {|
   string id;
   string firstName;
   string lastName;
|};

Run the Ballerina project again.


$ bal run

Compiling source
        foo/rainier:0.1.0


Running executable

{"id":"emp_01","firstName":"John","lastName":"Doe","email":"johnd@xyz.com","phone":"1234567890","hireDate":{"year":2020,"month":10,"day":10},"managerId":"mng_01","jobTitle":"Software Engineer"}

{"id":"emp_01","firstName":"John","lastName":"Doe"}

You will get the same results as you have only one record. Let’s explore the update function.

Update the `Employee` record

With the client API, you can easily update the record with the given ID. You only need to pass the fields that need to be updated with the new values. In this case, The job title of the given employee is updated from Software Engineer to Senior Software Engineer.

Replace the previous code and add the new update call instead.


import ballerina/io;
import rainier.store;


final store:Client sClient = check new ();


public function main() returns error? {
   string empId = "emp_01";


   // Update the job title of the employee with the given ID.
   store:Employee employee = check sClient->/employees/[empId].put({
       jobTitle: "Senior Software Engineer"
   });
   io:println(employee);
}

Run the Ballerina project again.


$ bal run

Compiling source
        foo/rainier:0.1.0


Running executable

{"id":"emp_01","firstName":"John","lastName":"Doe","email":"johnd@xyz.com","phone":"1234567890","hireDate":{"year":2020,"month":10,"day":10},"managerId":"mng_01","jobTitle":"Senior Software Engineer"}

Now, you have successfully executed the CREATE, READ, and UPDATE queries against your database. Let’s explore the final DELETE query in the next section.

Delete the `Employee` record

Similar to the client update call, you can use the delete action to delete the record with the given ID.

Replace the previous code and add the new delete call instead.


import ballerina/io;
import ballerina/persist;
import rainier.store;


final store:Client sClient = check new ();


public function main() returns error? {
   string empId = "emp_01";


   // Delete the employee with the given ID.
   _ = check sClient->/employees/[empId].delete();


   // Check if the employee is deleted.
   store:Employee|persist:Error result = sClient->/employees/[empId].get();
   if result is persist:NotFoundError {
       io:println("Employee not found");
   } else {
       io:println("Employee found");
   }
}

Run the Ballerina project again.


$ bal run

Compiling source
        foo/rainier:0.1.0


Running executable

Employee not found

You just run all the queries against your database successfully. Your next task is to expose the database queries via an HTTP RESTful API to build a data service.

Expose the database via an HTTP RESTful API

After you have defined the methods necessary to manipulate the database, expose these selectively via an HTTP RESTful API.

For this, you need to create an HTTP service, and within the service, you can define resource methods to provide access to the database. The complete code of the service will be as follows.

Replace the previous code and add the service code instead.


import ballerina/http;
import ballerina/persist;
import rainier.store;


final store:Client sClient = check new();


service /employees on new http:Listener(8080) {


   isolated resource function post .(store:EmployeeInsert emp) returns string|error? {
       string[] employeeIds = check sClient->/employees.post([emp]);
       return employeeIds[0];
   }
  
   isolated resource function get [string id]() returns store:Employee|error {
       return check sClient->/employees/[id];
   }
  
   resource function get .() returns store:Employee[]|error? {
       stream<store:Employee, persist:Error?> resultStream = sClient->/employees;
       return check from store:Employee employee in resultStream
           select employee;
   }


   isolated resource function put [string id](store:EmployeeUpdate emp) returns store:Employee|error? {
       return check sClient->/employees/[id].put(emp);
   }
  
   isolated resource function delete [string id]() returns store:Employee|error? {
       return check sClient->/employees/[id].delete();      
   }
}

Run the Ballerina project again,


$ bal run

Compiling source
        foo/rainier:0.1.0


Running executable

Info: This creates an /employees endpoint on port 8080, which can be accessed via a browser at http://locahost:8080/employees.

Try the service

Invoke the defined resource method by sending the POST request below to http://localhost:8080/employees with the required data as a JSON payload.


$ curl -X POST http://localhost:8080/employees/
    -H 'Content-Type: application/json'
    -d '{
        "id": "6",
        "firstName": "test",
        "lastName": "test",
        "email": "test@test.com",
        "phone": "882 771 110",
        "hireDate": {
            "year": 2021,
            "month": 12,
            "day": 16
        },
        "managerId": "mng_01",
        "jobTitle": "Sales Manager"
    }'

The entered employee ID 6 will be returned as the response.

Why Ballerina

Get started with integration

Integration tools

References

Development tutorials

Build and run

Source code & dependencies

Style guide

Test, debug & document the code

Test Ballerina code

Bal persist

Run in the cloud

Configurability

Observability

Java interoperability

Build a GraalVM executable

Reuse code with Ballerina Central

Troubleshoot the runtime

Resources

Featured scenarios

Learn the language

Manage data persistence with bal persist

Set up the prerequisites

Understand the implementation

Create a Ballerina package

Initialize bal persist in the project

Model your data

Generate the client object, types, and scripts

Query your database with client API

Create a new Employee record

Retrieve all Employee records

Retrieve the Employee record by key

Update the Employee record

Delete the Employee record

Expose the database via an HTTP RESTful API

Try the service

Initialize `bal persist` in the project

Create a new `Employee` record

Retrieve all `Employee` records

Retrieve the `Employee` record by key

Update the `Employee` record

Delete the `Employee` record