Show Navigation

Message Queues with Grails and Micronaut Kafka

Learn how to use message queues with Grails and Micronaut Kafka

Authors: Sergio del Amo

Grails Version: 4

1 Training

Apache Grails Training

Apache Grails is now part of the Apache Software Foundation. The community-maintained training catalog is being migrated; in the meantime see the Learning page for current resources, recorded talks, and links to other community-supplied training material.

2 Getting Started

In this guide we will show you how to setup and use Micronaut Kafka with a Grails application.

2.1 What you will need

To complete this guide, you will need the following:

  • Some time on your hands

  • A decent text editor or IDE

  • JDK 11 or greater installed with JAVA_HOME configured appropriately

2.2 How to complete the guide

To get started do the following:

or

The Grails guides repositories contain three folders:

  • docker

  • complete

  • complete-analytics

In this guide you are going to create two Grails Applications. Both complete and complete-analytics apps are completed examples. It is the result of working through the steps presented by the guide and applying those changes.

To run Kafka in Docker the docker folder contains a docker compose file. You need Docker and Docker Compose installed.

To complete the guide, follow the instructions in the next sections.

You can go right to the completed example if you cd into grails-guides/grails-micronaut-kafka/complete and grails-guides/grails-micronaut-kafka/complete-analytics.

3 Application Overview

In this guide, we setup a message queue to work across two different applications. In this guide, we have an app which lists books and details of books. We want to keep track of the number of times each book is viewed. We add a separate analytics app that keeps track of the number of times each one is viewed.

4 Running Kafka

A fast way to start using Kafka is via Docker. Create this docker-compose.yml file:

docker/docker-compose.yml
link:docker/docker-compose.yml[role=include]
1 Zookeeper uses port 2181 by default, but you can change the value if necessary.
2 Kafka uses port 9092 by default, but you can change the value if necessary.

Start Zookeeper and Kafka (use CTRL-C to stop both)

$ docker-compose up

Alternatively you can install and run a local Kafka instance.

5 Books Application

Create a Grails application with the rest-api profile.

grails create-app example.grails.complete --profile=rest-api

First, add a Book domain:

grails-app/domain/example/grails/Book.groovy
package example.grails

class Book {
    String isbn
    String name

    static constraints = {
        isbn unique: true, blank: false, nullable: false
        name blank: false, nullable: false
    }
}

Create default CRUD actions for Book leveraging GORM data services:

grails-app/services/example/grails/BookGormService.groovy
package example.grails

import grails.gorm.services.Service

@Service(Book)
interface BookGormService {
    Book saveBook(Book book)

    List<Book> findAll()

    Book findByIsbn(String isbn)
}

Then we need to actually create the book data with our Bootstrap.groovy:

grails-app/init/example/grails/Bootstrap.groovy
package example.grails

import groovy.transform.CompileStatic

@CompileStatic
class BootStrap {

    BookGormService bookGormService

    def init = { servletContext ->
        [
                new Book(isbn: '1491950358', name: 'Building Microservices'),
                new Book(isbn: '1680502395', name: 'Release It!'),
                new Book(isbn: '0321601912', name: 'Continuous Delivery')
        ].each {book ->
            bookGormService.saveBook(book)
        }
    }
    def destroy = {
    }
}

Add the Micronaut Kafka dependency:

build.gradle
implementation "io.micronaut:micronaut-inject-groovy"
implementation("io.micronaut.kafka:micronaut-kafka:3.1.0")

The app connects to a Kafka broker running on localhost:9092. Add the following configuration:

grails-app/conf/application.yml
kafka:
    bootstrap:
        servers: localhost:9092

Create an interface to send messages to Kafka. The Micronaut framework will implement the interface at compilation time:

src/main/groovy/example/grails/AnalyticsClient.groovy
package example.grails

import io.micronaut.configuration.kafka.annotation.KafkaClient
import io.micronaut.configuration.kafka.annotation.Topic

@KafkaClient
interface AnalyticsClient {

    @Topic('analytics') (1)
    Map updateAnalytics(Map book) (2)
}
1 Set the topic name
2 Send the book information. The Micronaut Framework will automatically convert it to JSON before sending it.

Create a controller which fetches books and notifies Kafka with the AnalyticsClient:

grails-app/controllers/example/grails/BooksController.groovy
package example.grails

import groovy.transform.CompileStatic
import org.springframework.beans.factory.annotation.Autowired

@CompileStatic
class BooksController {

    BookGormService bookGormService

    @Autowired
    AnalyticsClient analyticsClient

    static allowedMethods = [
            index: 'GET',
            show: 'GET'
    ]

    def index() {
        [books: bookGormService.findAll()]
    }

    def show(String isbn) {
        Book book = bookGormService.findByIsbn(isbn)
        if (!book) {
            response.status = 404
            return
        }
        analyticsClient.updateAnalytics([isbn: book.isbn])
        render(template: 'book', model: [book: book])
    }
}

Add the following mapping to UrlMappings:

grails-app/controllers/example/grails/UrlMappings.groovy
        "/books/$isbn" {
            controller = 'books'
            action = 'show'
        }

Create two JSON Views for the controller’s actions:

grails-app/views/books/_book.gson
import example.grails.Book

model {
    Book book
}
json {
    isbn book.isbn
    name book.name
}
grails-app/views/books/index.gson
import example.grails.Book
model {
    List<Book> books = []
}
json tmpl.book(books)

6 Building Analytics app

Create a new Grails application for this additional app. For example by using Grails Application Forge or the command line:

$ grails create-app example.grails.complete-analytics --profile=rest-api

For the multi app part of this guide we will need to be able to run both apps simultaneously. To avoid a running port conflict update your app’s application.yml to include the following:

grails-app/conf/application.yml
link:complete-analytics/grails-app/conf/application.yml[role=include]

Create a Domain class BookAnalytics which will keep track of how many times a book has been viewed:

grails-app/domain/example/grails/BookAnalytics.groovy
link:complete-analytics/grails-app/domain/example/grails/BookAnalytics.groovy[role=include]

Create a GORM Data service for this domain class:

grails-app/services/example/grails/BookAnalyticsGormService.groovy
link:complete-analytics/grails-app/services/example/grails/BookAnalyticsGormService.groovy[role=include]
1 Implement update operations using JPA-QL

Create a controller which uses the previous service:

grails-app/controllers/example/grails/AnalyticsController.groovy
link:complete-analytics/grails-app/controllers/example/grails/AnalyticsController.groovy[role=include]

Create two JSON Views:

grails-app/views/analytics/_bookAnalytics.gson
link:complete-analytics/grails-app/views/analytics/_bookAnalytics.gson[role=include]
grails-app/views/analytics/index.gson
link:complete-analytics/grails-app/views/analytics/index.gson[role=include]

Create a new class to act as a consumer of the messages sent to Kafka by the books microservice. The Micronaut framework will implement logic to invoke the consumer at compile time. Create the AnalyticsListener class:

src/main/groovy/example/grails/AnalyticsListener.groovy
link:complete-analytics/src/main/groovy/example/grails/AnalyticsListener.groovy[role=include]
1 Do not load this bean for the test environment - this lets us run the tests without having Kafka running
2 Annotate the class with @KafkaListener to indicate that this bean will consume messages from Kafka
3 Constructor injection for BookAnalyticsGormService
4 Annotate the method with @Topic and specify the topic name to use

7 Running the apps

Start Kafka:

$ cd docker
docker$ docker-compose up

Start the books microservice:

$ cd complete
complete$ ./gradlew bootRun

Start the analytics microservice:

$ cd complete-analytics
complete-analytics$ ./gradlew bootRun

Execute a curl request to get one book:

$ curl http://localhost:8080/books/1491950358
{"isbn":"1491950358","name":"Building Microservices"}

Now, use curl to see the analytics:

$ curl http://localhost:8081/analytics
[{"bookIsbn":"1491950358","count":1}]

Update the curl command to the books microservice to retrieve other books and repeat the invocations, then re-run the curl command to the analytics microservice to see that the counts increase.

8 Next Steps

To further your understanding read through the Micronaut Kafka plugin documentation.

9 Do you need help with Grails?

Help with Apache Grails

Apache Grails is supported by an active community of contributors and the Apache Software Foundation. If you need help working through a guide, want to discuss the framework, or have run into something that looks like a bug, the channels below are the right place to start.

For Grails plugins, see the matching project on the apache org or the plugin’s own GitHub repository.