Marshalling Arrow Types in Ktor

Support for kotlinx.serialization in Arrow 1.2

6 min readJul 16, 2023

A Scottish Archer — An Archer from the Scottish Royal Company

Introduction

Last week saw the release of Arrow version 1.2. Included in this release is support for the kotlinx.serialization multiplatform library. This gives us an alternative to Jackson for marshalling Arrow types as JSON.

The most obvious application is to microservices. So to explore this new functionality let’s:

Write a demo service in Ktor.
Add endpoints that return Arrow types.
See what JSON content is sent to the client.

NB All the code shown below can be found in this GitHub repository. Feel free to clone it, if you just want to run the service and view the outputs.

Creating the Project

We can build a new project via the Ktor Project Generator. I named mine ktor-arrow-marshalling.

In the plugins section we need to add the Routing, Content Negotiation and kotlinx.serialization plugins — as shown below:

The plugin screen in the Ktor Project Generator

We can then generate and download the project, unpack the zip file and open the resulting folder in IntelliJ IDEA.

Adding Arrow to the Project

Next we need to add support for Arrow 1.2 into our project. We configure which version of Arrow we are using in gradle.properties:

arrow_version=1.2.0

At the top of build.gradle.kts we declare this version:

val arrow_version: String by project

Then within the dependencies section we declare both arrow-core and the brand new arrow-core-serialization library:

implementation("io.arrow-kt:arrow-core:$arrow_version")
implementation("io.arrow-kt:arrow-core-serialization:$arrow_version")

Creating Our Model

I’ve stuck with the default package of com.example. Let’s create a com.example.model subpackage, and add some types for books:

@file:UseSerializers(
    NonEmptySetSerializer::class
)

package com.example.model

import arrow.core.*
import arrow.core.serialization.NonEmptySetSerializer
import kotlinx.serialization.Serializable
import kotlinx.serialization.UseSerializers

enum class Genre {
    Scifi,
    Philosophy,
    History,
    Poetry
}

@Serializable
data class Book(
    val title: String, 
    val authors: NonEmptySet<String>, 
    val genre: Genre
)

Normally we would only need to add the Serializable annotation to the Book type, in order to enable instances to be marshalled as JSON. But Book uses the NonEmptySet collection from Arrow. So we need to manually include the serializer for that type.

We do this via the UseSerializers annotation, which you can see at the top of the file. Points to note are:

The serializer type is distinct from, but named after, the type it marshalls. So for NonEmptySet we have NonEmptySetSerializer.
We specify the type of the serializer rather than create an instance. So the declaration used is NonEmptySetSerializer::class.
Because the annotation has a target of file it needs to go at the very top, even above the package declaration. This is logical but feels strange at first.
You only need to include serializers for the types you are using. If you omit one then the kotlinx-serialization compiler plugin will helpfully generate a reminder to assist you 😏.

Creating Our Repository

Now that we have our model types, let’s create a data repository for books. For the purposes of the demo we want this repository to make use of multiple Arrow Core types, ideally in combination:

object BookStore {
    private val stock = nonEmptySetOf(
        Book("Permutation City", nonEmptySetOf("Greg Egan"), Scifi) to 1,
        Book("Project Hail Mary", nonEmptySetOf("Andy Weir"), Scifi) to 2,
        Book("The Three-Body Problem", nonEmptySetOf("Cixin Liu"), Scifi) to 15,
        Book("This is How You Lose the Time War", nonEmptySetOf("Amal El-Mohtar", "Max Gladstone"), Scifi) to 0,
        Book("A Problem from Hell", nonEmptySetOf("Samantha Power"), History) to 8,
        Book("When Genius Failed", nonEmptySetOf("Roger Lowenstein"), History) to 6,
        Book("Brexit Unfolded", nonEmptySetOf("Chris Grey"), History) to 0,
        Book("Lila", nonEmptySetOf("Robert Pirsig"), Philosophy) to 0,
        Book("The Sleepwalkers", nonEmptySetOf("Arthur Koestler"), Philosophy) to 7
    )

    fun findByGenre(genre: Genre): Either<String, List<Book>> = either {
        val results = stock.filter { it.first.genre == genre }
        ensure(results.isNotEmpty()) {
            "No books in genre $genre"
        }
        results.map { it.first }
    }

    fun findByTitle(title: String): Option<Either<String, Book>> = option {
        val result = stock.find {
            it.first.title == title
        }

        ensureNotNull(result)
        either {
            val(book, numInStock) = result
            ensure(numInStock > 0) {
                "${book.title} is out of stock"
            }
            book
        }
    }
}

As you can see I have created a singleton called BookStore, containing a private property called stock.

Stock is a non-empty set of Pair<Book, Int>. The integer holds the number of copies of the book currently in stock. We also have two query methods, findByGenre and findByTitle.

The findByGenre method works as follows:

We filter to find all the books in the selected genre.
If there are no books in that genre we return a Left containing an explanation.
If there are books in the genre we use map to select their names and place the resulting list of strings into a Right.

The findByTitle method works as follows:

We find the first book with a matching title
If no such book exists we return a None
If the book exists we return a Some containing an Either
If the book is out of stock then the Either will be a Left<String>
If the book is in stock then the Either will be a Right<Book>

Creating Routes

Our final task is to expose the BookStore functionality over HTTP. We can take the Serialization.kt file created by the Ktor Project Generator and rewrite it as follows:

@file:UseSerializers(
    EitherSerializer::class,
    OptionSerializer::class
)

package com.example.plugins

import arrow.core.Either
import arrow.core.Option
import arrow.core.serialization.EitherSerializer
import arrow.core.serialization.NonEmptySetSerializer
import arrow.core.serialization.OptionSerializer
import com.example.model.*
import io.ktor.http.*
import io.ktor.serialization.kotlinx.json.*
import io.ktor.server.plugins.contentnegotiation.*
import io.ktor.server.response.*
import io.ktor.server.application.*
import io.ktor.server.routing.*
import kotlinx.serialization.Serializable
import kotlinx.serialization.UseSerializers

@Serializable
data class TitleSearchResult(val result: Option<Either<String, Book>>)

@Serializable
data class GenreSearchResult(val result: Either<String, List<Book>>)

fun Application.configureSerialization() {
    install(ContentNegotiation) {
        json()
    }
    routing {
        route("/books") {
            get("/byGenre/{genre}") {
                val genreParam = call.parameters["genre"]
                if (genreParam != null) {
                    val genre = Genre.valueOf(genreParam)
                    call.respond(GenreSearchResult(BookStore.findByGenre(genre)))
                    return@get
                }
                call.respond(HttpStatusCode.BadRequest)
            }
            get("/byTitle/{title}") {
                val title = call.parameters["title"]
                if (title != null) {
                    call.respond(TitleSearchResult(BookStore.findByTitle(title)))
                    return@get
                }
                call.respond(HttpStatusCode.BadRequest)
            }
        }
    }
}

We have created two endpoints for clients:

/books/byGenre/{genre} exposes our BookStore.findByGenre method.
/books/byTitle/{title} exposes our BookStore.findByTitle method.

Other interesting points to note are:

We need two ‘wrapper’ types to contain our Arrow results. I’ve called these TitleSearchResult and GenreSearchResult.
As before we place the UseSerializers annotation at the very top of the file. Within it we declare the serializers for the Arrow types we will be using in the current code.

Viewing the Marshalled Data

All that remains is to fire up the project and send some sample requests. You can launch the project by running the main method in Application.kt. Alternatively you could use the application → run task in Gradle.

These are the sample requests I ran:

Before reading further you might like to predict what these will return. You can run them in the browser or via a tool like Postman. If you’re using IntelliJ IDEA Ultimate you can create an HTTP Request Scratch File.

When I send a request to /books/byGenre/Poetry this is the response:

{
  "result": {
    "left": "No books in genre Poetry"
  }
}

As you can see the fact that this is the Left case is preserved in the JSON.

If I send a request to /books/byGenre/Philosophy then I get back the Right case — containing a list of books:

{
  "result": {
    "right": [
      {
        "title": "Lila",
        "authors": [
          "Robert Pirsig"
        ],
        "genre": "Philosophy"
      },
      {
        "title": "The Sleepwalkers",
        "authors": [
          "Arthur Koestler"
        ],
        "genre": "Philosophy"
      }
    ]
  }
}

If I search for a book, with a made up title, then a None will be returned from our BookStore. In the JSON this just appears as a null. So a request to /books/byTitle/The Four-Body Problem returns:

{
  "result": null
}

If I request a valid title, and the book is in stock, then I get its details in a Right within a Some. The usage of Some is not encoded in the JSON. So a request to /books/byTitle/The Three-Body Problem returns:

{
  "result": {
    "right": {
      "title": "The Three-Body Problem",
      "authors": [
        "Cixin Liu"
      ],
      "genre": "Scifi"
    }
  }
}

Finally if the title is valid but out of stock we get an explanation in a Left within a Some. Once again the use of Some is not encoded in the JSON. So a request to /books/byTitle/Lila returns:

{
  "result": {
    "left": "Lila is out of stock"
  }
}

Summary

We have seen that you can now use Arrow Core datatypes within Ktor services that make use of kotlinx-serialization. This is great news for those server-side developers who lean towards the functional programming style, but also for Kotlin Multiplatform in general.

In a future blog we will see how you could use this functionality on the client side — specifically in a User Interface written using the Ktor Client and Compose Multiplatform.