AI Chart Generation

[rolfsmeds]

Description

A collection of Flow APIs that make it easy to build application features that allow end users to generate their own charts based on application data using AI.

Tier

Pro

License

Proprietary

Motivation

Background

Charts is one of the common ways to visualize application data. Many applications need a lot of different charts.

Problem

It is often difficult for designers and developers to predict all the charts that the application's end users will need. Implementing many different charts can be a lot of effort, and implementing highly customizable or configurable charts is difficult.

Solution

Provide end-users with tools for creating their own charts from scratch based on the data available to the application by describing the desired chart to an LLM.

The API builds on top of the AI Chat feature: using the same AiOrchestrator and LLMProvider interface, with chart-specific behavior provided through ChartAiController. Additionally, a DatabaseProvider interface is used to acquire the database schema and execute the generated query, and a DataConverter interface for converting the return data from the executed query to the format needed for the chart.

The LLM will generate an SQL query and a HighCharts configuration. The query will be executed through the DatabaseProvider and its result converted into a DataSeries format appropriate for the type of chart. The controller applies the chart configuration and sets the data on the Chart instance.

Usage could look something like this:

Chart chart = new Chart();
MessageInput messageInput = new MessageInput();

// Back-end services
LLMProvider llmProvider = new SpringAiLLMProvider(chatClient);
DatabaseProvider databaseProvider = new MyDatabaseProvider();

// Chart controller (provides tools and system prompt)
ChartAiController chartController = new ChartAiController(chart, databaseProvider);
String systemPrompt = ChartAiController.getSystemPrompt();

// Generic orchestrator configured for chart generation
AiOrchestrator orchestrator = AiOrchestrator.builder(llmProvider, systemPrompt)
    .withController(chartController)
    .withInput(messageInput)
    .build();

layout.add(messageInput, chart);

public interface DatabaseProvider {
    /**
     * @return Text description of the database schema
     */
    String getSchema();

    /**
     * @return List of rows, where each row is a map of column name to value
     */
    List<Map<String, Object>> executeQuery(String query);
}

Notes

• The same AiOrchestrator is used for both chat and chart features, with chart-specific behavior added through ChartAiController.
• The same DatabaseProvider could be reused for generating Grids instead of Charts (outside the scope of this PRD).
• For security, we strongly recommend that the database connection used for this uses a DB account with read-only access limited to the relevant tables or views.

Requirements

• ChartAiController implementing AiController that provides chart-specific tools to AiOrchestrator.
• ChartAiController has necessary tools (for example, getSchema, getCurrentState, updateData, updateConfig)
• ChartAiController.getSystemPrompt() provides a default system prompt with information about HighCharts data series format and Vaadin Charts API.
• DataConverter interface for converting DB query results into DataSeries format. Default implementation provided, custom implementations supported via setDataConverter().
• State management: getState(), restoreState(), and state change listeners for persistence.
• DatabaseProvider interface with methods for getting the DB schema and executing a query.
• Documentation
  • Emphasize the necessity of using a DB account with read-only access to only the relevant tables or views.
• License check (remove if not needed)
• Feature flag (remove if not needed)

Nice-to-haves

• Default DatabaseProvider implementations for popular databases.

Risks, limitations and breaking changes

Risks

• End users may not trust the charts produced by the LLM through this feature, as there is no user-friendly way to validate that the query produce what they expect. Possible mitigation options include somehow visualizing the generated query.

Limitations

• We may not be able to generalize the return type from the executed query to support all chart types out of the box.
• We may not be able to support all possible database providers.

Breaking changes

None identified.

Out of scope

Although in many cases the generated charts will be added to a Dashboard, that functionality is outside the scope of this PRD. We will later consider including automatic Dashboard configuration as part of the AI-powered functionality (e.g. through a separate DashboardAiController).

Materials

No response

Metrics

No response

Pre-implementation checklist

• Estimated (estimate entered into Estimate custom field)
• Product Manager sign-off
• Engineering Manager sign-off

Pre-release checklist

• Documented (link to documentation provided in sub-issue or comment)
• UX/DX tests conducted and blockers addressed
• Approved for release by Product Manager

Security review

Peer reviewed