dependencies {
implementation 'org.apache.grails:grails-spring-ai'
implementation 'org.springframework.ai:spring-ai-starter-model-openai'
testImplementation 'org.apache.grails:grails-testing-support-ai'
}22 Using AI in Your Application
Version: 8.1.0-SNAPSHOT
Table of Contents
22 Using AI in Your Application
This chapter is about using AI models from an application: answering questions, summarising, classifying, searching by meaning and letting a model call the application’s own services. It is distinct from the AI tooling that helps develop a Grails application, which the Agent Skills section covers.
Grails integrates Spring AI through the grails-spring-ai plugin. Spring AI gives an application one API, the ChatClient, over the chat models of OpenAI, Anthropic, Ollama, Google, Amazon Bedrock, Mistral and others, along with embeddings, vector stores, tool calling and the Model Context Protocol.
The plugin adds what a Grails application needs on top of that:
-
The Spring AI version is managed by the Grails BOM, so an application declares Spring AI starters without versions and gets the line that matches its Grails release.
-
A
ChatClient.Builderis auto-configured whichever provider is on the classpath. -
Defaults that suit a web application: a failed model call is retried a few times rather than for minutes, and prompts and completions are logged in the development environment only.
-
A chat response an action streams is written to the client as server-sent events, through the asynchronous request support of Grails.
-
Testing support with a scripted chat model and a deterministic embedding model, so unit and integration tests need no provider and no credentials.
The following sections cover installing the plugin, the model providers to choose from, its configuration, using the chat client, streaming responses and testing. The Spring AI reference documents the Spring AI API itself.
22.1 Installation
Add the plugin and one provider starter. Spring AI’s version comes from the Grails BOM, so neither needs a version:
build.gradle
The plugin provides the chat client and the auto-configuration every provider shares; the starter provides the model. Spring AI implements each provider as a separate module, so the application names the one it talks to, and grails-spring-ai does not pick one for it. Any provider works in place of OpenAI here; Model Providers lists them with what each needs. The same goes for the other Spring AI starters, such as a vector store or the MCP client: declare them without a version and the BOM supplies it.
The provider’s credentials go in its own properties, such as spring.ai.openai.api-key. Read them from the environment rather than committing them:
Credentials come from the environment: Spring AI reads the key from the provider’s environment variable, OPENAI_API_KEY here, ANTHROPIC_API_KEY for Anthropic, when the configuration sets none. An application that gets the key from elsewhere sets spring.ai.openai.api-key itself; do not set it to a placeholder such as ${OPENAI_API_KEY}, which is sent as the key, and fails as one, when the variable is unset.
A new application gets all of this from the grails-spring-ai feature, or from one of the spring-ai-openai, spring-ai-anthropic and spring-ai-ollama features, which add the provider’s starter and its configuration as well. The provider features exclude one another, since a second starter would auto-configure a second chat model with nothing to choose between them:
$ grails -t forge create-app my-app --features spring-ai-openai22.2 Model Providers
The grails-spring-ai plugin is provider-neutral on purpose. It brings the ChatClient API and the auto-configuration every provider shares, but no model: Spring AI implements each provider as a module of its own, with that provider’s SDK or HTTP client, its auto-configuration and its spring.ai.<provider>.* properties. A provider’s starter bundles those, and the ChatClient.Builder is only auto-configured once a ChatModel bean exists, which is what the starter contributes. The plugin does not choose one for the application because each starter carries its own dependencies and credentials, and because many applications use one provider in development and another in production.
So an application declares the plugin plus one starter per kind of model it uses, chat and embedding being the common pair. Every starter below is managed by the Grails BOM, so none needs a version. The property shown is the one a provider needs at a minimum; each provider’s page in the Spring AI reference lists the rest, including how to pick a model with spring.ai.<provider>.chat.model and the generation options.
Chat Models
| Provider | Starter | Needs | Spring AI reference |
|---|---|---|---|
OpenAI |
|
|
|
Anthropic |
|
|
|
Ollama, local models |
|
|
|
Google GenAI, Gemini on AI Studio or Vertex AI |
|
|
|
Amazon Bedrock Converse |
|
AWS credentials and |
|
Mistral AI |
|
|
|
DeepSeek |
|
|
The chat model comparison in the Spring AI reference shows which providers support images and other media, tool calling, structured JSON output and local deployment.
OpenAI-Compatible Services
Many services speak the OpenAI API. They need no starter of their own: the OpenAI starter is pointed at them with a base URL and the service’s key, and the model is named with spring.ai.openai.chat.model. Groq, for example:
grails-app/conf/application.yml
spring:
ai:
openai:
base-url: https://api.groq.com/openai/v1
api-key: ${GROQ_API_KEY}
chat:
model: llama-3.3-70b-versatileThe Spring AI reference has a page per service with its base URL and model names: Azure OpenAI, Groq, NVIDIA, Perplexity and Docker Model Runner. MiniMax works the same way through the Anthropic starter, with spring.ai.anthropic.base-url. OCI Generative AI is maintained outside Spring AI since 2.0 and is not managed by the Grails BOM.
A server on the developer’s own machine, such as vLLM, LM Studio, Docker Model Runner or Ollama’s OpenAI endpoint, usually wants no key at all. An empty key puts the client in no-auth mode, and it sends no Authorization header:
spring:
ai:
openai:
base-url: http://localhost:12434/engines/v1
api-key: ''
chat:
model: ai/llama3.2Each kind of model can go to a different service, since spring.ai.openai.chat.base-url, spring.ai.openai.embedding.base-url and the matching api-key properties override the shared ones per kind.
Compatibility is a floor, not the provider. What a provider adds beyond the OpenAI API, such as Anthropic’s prompt caching and PDF input, Bedrock’s IAM credentials, Gemini’s audio and video input or Ollama’s model pulling, is only reachable through that provider’s own starter in the tables above.
Embedding Models
An embedding model turns text into the vectors a vector store searches by meaning. Most chat providers offer one, and the chat starter brings it along; the dedicated starters are for embedding-only services.
| Provider | Starter | Needs | Spring AI reference |
|---|---|---|---|
OpenAI |
|
|
|
Ollama |
|
|
|
Google GenAI |
|
|
|
Vertex AI, text and multimodal |
|
A Google Cloud project and location |
|
Amazon Bedrock, Cohere and Titan |
|
AWS credentials and |
|
Mistral AI |
|
|
|
Transformers, ONNX models in the JVM |
|
Nothing, the model runs locally |
|
PostgresML |
|
A PostgreSQL database with the PostgresML extension |
Image, Audio and Moderation Models
| Kind | Provider | Starter | Spring AI reference |
|---|---|---|---|
Image generation |
OpenAI |
|
|
Image generation |
Google GenAI |
|
|
Image generation |
Stability AI |
|
|
Transcription |
OpenAI |
|
|
Text to speech |
OpenAI |
|
|
Text to speech |
ElevenLabs |
|
|
Moderation |
OpenAI |
|
|
Moderation |
Mistral AI |
|
Several Providers at Once
Two starters that both provide a chat model would both auto-configure one, and the ChatClient.Builder needs a single ChatModel. The selection properties say which provider configures which kind of model, and none turns a kind off:
grails-app/conf/application.yml
spring:
ai:
model:
chat: anthropic
embedding: openaiEach kind of model has a selection property of its own; Configuration lists them. To use two chat providers side by side, build a second ChatClient from the other provider’s model bean, as the ChatClient reference shows. The test environment uses the same mechanism: the testing support sets spring.ai.model.chat and spring.ai.model.embedding to none so the provider backs off and its stubs take over, see Testing.
22.3 Configuration
Spring AI is configured through its own spring.ai properties, which the Spring AI reference documents provider by provider. The plugin ships defaults below the application’s configuration, so anything set in application.yml wins:
spring:
ai:
retry:
max-attempts: 3
environments:
development:
spring:
ai:
chat:
client:
observations:
log-prompt: true
log-completion: true
observations:
include-error-logging: true-
Spring AI retries a failed model call ten times, backing off for up to three minutes. A web request cannot wait that long, so the plugin retries three times. Set
spring.ai.retry.max-attemptsto change it. -
Spring AI logs neither prompts nor completions. In the development environment the plugin turns both on, along with the logging of errors from the model, so a developer sees what was sent and what came back; in test and production nothing is logged. Spring AI logs through Micrometer observations, which need an
ObservationRegistrybean; the plugin brings Spring Boot’s auto-configuration of one, so the logging works without the actuator.
Choosing the Provider and the Model
The plugin selects no provider and no credential; the starter the application adds does, see Model Providers. Every provider auto-configures each kind of model it offers, so with one starter nothing more is needed. With more than one starter on the classpath, or to turn a kind of model off, the selection properties name the provider for each kind. Their value is the provider’s name, such as openai, anthropic, ollama, google-genai, bedrock-converse, mistral or deepseek, or none:
| Property | Selects the provider of |
|---|---|
|
the chat model, which the |
|
the embedding model, which vector stores use |
|
the image generation model |
|
the transcription model |
|
the text to speech model |
|
the moderation model |
Which model a provider uses, and with what options, is set under the provider’s own prefix, for example spring.ai.openai.chat.model and spring.ai.openai.chat.temperature, or spring.ai.anthropic.chat.model; each provider’s page in the Spring AI reference lists them with their defaults.
Streaming is configured under grails.async.streaming, see Streaming Model Responses, and the test stubs under grails.ai.stub, see Testing.
22.4 Using the Chat Client
A ChatClient.Builder is auto-configured over the provider’s chat model. Inject it into a service and build a client; the builder is a prototype, so each service can give its client its own defaults:
grails-app/services/my/app/AnswerService.groovy
package my.app
import org.springframework.ai.chat.client.ChatClient
class AnswerService {
ChatClient chatClient
AnswerService(ChatClient.Builder chatClientBuilder) {
chatClient = chatClientBuilder
.defaultSystem('You answer questions about the Grails framework, briefly.')
.build()
}
String answer(String question) {
chatClient.prompt()
.user(question)
.call()
.content()
}
}The fluent API maps a response onto a class as well. Spring AI derives a JSON schema from the class, asks the model to follow it and converts the answer:
class Summary {
String title
List<String> keyPoints
}
Summary summarise(String text) {
chatClient.prompt()
.user("Summarise the following text.\n\n$text")
.call()
.entity(Summary)
}Methods of a service become tools the model may call when they carry Spring AI’s @Tool annotation and the service is passed to the client:
import org.springframework.ai.tool.annotation.Tool
import org.springframework.ai.tool.annotation.ToolParam
class BookService {
@Tool(description = 'Finds the books of an author')
List<String> findBooks(@ToolParam(description = 'The name of the author') String author) {
Book.findAllByAuthor(author)*.title
}
}
chatClient.prompt()
.user('Which books did Graeme Rocher write?')
.tools(bookService)
.call()
.content()Chat memory, retrieval augmented generation, the MCP client and the other Spring AI features work as the Spring AI reference describes them. A memory advisor needs a conversation id on every call; derive it from the current user and session rather than a constant, so conversations stay apart.
22.5 Streaming Model Responses
A model answers token by token, and a client that shows the tokens as they arrive feels far faster than one that waits for the whole answer. The ChatClient streams through stream(), which returns a Reactor Flux, and a Grails controller action can return that Flux directly:
grails-app/controllers/my/app/ChatController.groovy
package my.app
import org.springframework.ai.chat.client.ChatClient
class ChatController {
ChatClient.Builder chatClientBuilder
def ask(String question) {
chatClientBuilder.build()
.prompt()
.user(question)
.stream()
.content()
}
}Grails handles the request asynchronously, commits the response as text/event-stream and writes every item the Flux emits to the client as a server-sent event, so the text of the answer arrives as the model produces it, and ends the response with an event named complete that the client closes on; should the model fail part way, the stream still ends with that event, its data then being error. Returning stream().chatResponse() works as well: the plugin renders each chat response as the text it carries. A browser reads the events with EventSource:
const answer = document.getElementById('answer');
const source = new EventSource('/chat/ask?question=' + encodeURIComponent(question));
source.onmessage = event => { answer.textContent += event.data; };
source.addEventListener('complete', () => source.close());This is the general streaming support of Grails, which any Reactive Streams Publisher returned from an action gets; the Streaming Responses section of the asynchronous programming chapter describes the event format, how other kinds of item are rendered and the grails.async.streaming settings. A stream has no timeout unless grails.async.streaming.timeout sets one, so a slow or reasoning model that takes minutes to answer is not cut off; set it to bound how long an answer may take:
grails-app/conf/application.yml
grails:
async:
streaming:
timeout: 5mA Mono, such as stream().content().collect(…) or the result of a single call, is not streamed: it is resolved, and its value is the action’s result as a returned `Promise’s value is.
22.6 Testing
Tests should not call a model: answers vary, calls cost money and need credentials, and a test that fails because a provider was slow helps no one. The grails-testing-support-ai module gives tests a scripted chat model and a deterministic embedding model instead.
Unit Tests
A specification that implements grails.testing.ai.AiUnitTest gets a chatModel, an embeddingModel, a chatClientBuilder over the scripted model and, when Spring AI’s vector store module is on the classpath, an in-memory vectorStore in its application context. A service under test gets them by injection, as it would get the real ones. Script the answers and assert on the prompts:
import grails.testing.ai.AiUnitTest
import grails.testing.services.ServiceUnitTest
import spock.lang.Specification
class AnswerServiceSpec extends Specification implements ServiceUnitTest<AnswerService>, AiUnitTest {
void 'answers with the model'() {
given:
respondWith('42')
expect:
service.answer('What is six times seven?') == '42'
prompts.last().userMessage.text == 'What is six times seven?'
}
}Scripted answers are consumed in order; an answer may also be a closure that receives the Prompt and returns the text. When no answer is left, the model echoes the user’s message, so a test that does not care about the answer needs no script; set chatModel.strict = true, in the feature method or in setup(), for a test that must fail instead when the script runs out; it is turned off again with the script before every feature method. A streamed call answers one word at a time, so code that assembles a stream is exercised as it would be against a real model. The script, the recorded prompts and the contents of the vector store are cleared before every feature method.
The stubs are registered before the specification’s setupSpec, once the beans of its doWithSpring() and beanRegistrar() exist, and each only when the context holds no bean of its type, so a chat model, embedding model, chat client builder or vector store the specification defines is used instead. The chat model then has to be a ScriptedChatModel for chatModel and respondWith to find it. The stub embedding model honours grails.ai.stub.embeddingDimensions set through doWithConfig.
Integration and Functional Tests
While grails-testing-support-ai is on the test classpath and the application runs in the test environment, every spring.ai.model.* selection property and spring.ai.vectorstore.type default to none, so no provider auto-configures a chat, embedding, image, audio or moderation model and no vector store starter needs a database, and the scripted chat model, the stub embedding model and an in-memory vector store stand in for the chat model, the embedding model and the store. The test environment then needs neither an API key nor a database. A provider or store the application names in its configuration is kept, and the stub for that kind of model backs off. A chat model, embedding model or vector store the application defines itself, in resources.groovy, doWithSpring or its beanRegistrar, is the one injected: the stubs are fallback beans, which Spring passes over whenever another candidate exists. The org.apache.grails.testing.ai.ScriptedChatModel bean can be injected into a test and scripted the same way; the in-memory store is a ResettableSimpleVectorStore, whose clear() empties it between tests that share a context.
Outside the test environment the stubs stay off, and a warning says the module was found, so a production application never has its model replaced; grails.ai.stub.enabled: true turns them on there deliberately. To run tests against the real provider instead, disable the stubs for the test environment and configure the provider as usual:
grails-app/conf/application.yml
environments:
test:
grails:
ai:
stub:
enabled: false| Property | Default | Description |
|---|---|---|
|
|
Whether the stubs stand in for the configured provider’s models while the module is on the classpath. |
|
|
The number of dimensions of the vectors the stub embedding model produces. |