From a42b85fb4c4913ccf347673299448e0a72f7d1a0 Mon Sep 17 00:00:00 2001 From: Liam Sommer <161324156+liamsommer-mx@users.noreply.github.com> Date: Tue, 16 Jun 2026 15:06:28 +0200 Subject: [PATCH 1/5] Revise AWS Authentication documentation for Credentials usage Updated documentation to reflect the use of a Credentials object for various operations instead of static credentials. --- .../modules/aws/amazon-bedrock.md | 13 ++++++++++--- 1 file changed, 10 insertions(+), 3 deletions(-) diff --git a/content/en/docs/marketplace/platform-supported-content/modules/aws/amazon-bedrock.md b/content/en/docs/marketplace/platform-supported-content/modules/aws/amazon-bedrock.md index 43541d8058a..d52235597b4 100644 --- a/content/en/docs/marketplace/platform-supported-content/modules/aws/amazon-bedrock.md +++ b/content/en/docs/marketplace/platform-supported-content/modules/aws/amazon-bedrock.md @@ -61,6 +61,13 @@ As of version 3.0.0 of the [AWS Authentication Connector](https://marketplace.me The AWS Authentication Connector supports both **static credentials** and **temporary credentials**. For more information and detailed instructions please refer to the [AWS Authentication Connector documentation page](/appstore/modules/aws/aws-authentication/). +To use other authentication ways, you can create an `Credentials` object in your custom way and pass it to the operations. +- Chat Completions: associate the `Credentials` object to the `ChatCompletionsRequest_Extension` +- Embeddings: associate the `Credentials` object to the `CohereEmbedOptions_Extension` +- Image Generation: associate the `Credentials` object to the `TitanImageOptions_Extension` +- Retrieve: associate the `Credentials` object to the `RetrieveRequest_Extension` +- Retrieve and Generate: associate the `Credentials` object to the `RetrieveAndGenerateRequest_Extension` + ### Syncing Available Models, Knowledge Bases, and Agents You can use the `SNIP_Settings_Admin_ConfigOverview` snippet under **_USE_ME > SyncBedrockMetadata > ReusableUI** on an administrator page to sync models, knowledge bases and agents for the selected region at runtime. Admins on the page first need to configure the settings of the [AWS Authentication Connector](#configure-authentication) module, by selecting the AWS region and how to authenticate. When saving the settings or when you sync the models for the current region, AWS metadata services are called to create persistent objects in the database which you can view in the tables at the bottom of the snippet. @@ -724,7 +731,7 @@ The input and output for this service are shown in the table below: | Input | Output | | --- | --- | -| `ENUM_Region (enumeration)`, `UseStaticCredentials (boolean)` | `Count (integer)` | +| `ENUM_Region (enumeration)`, `Credentials (object)` | `Count (integer)` | The operation returns an integer that indicates how many objects were created or changed during the syncing process. @@ -737,7 +744,7 @@ The input and output for this service are shown in the table below: | Input | Output | | --- | --- | -| `ENUM_Region (enumeration)`, `UseStaticCredentials (boolean)` | `Count (integer)` | +| `ENUM_Region (enumeration)`, `Credentials (object)` | `Count (integer)` | The operation returns an integer that indicates how many objects were created or changed during the syncing process. @@ -750,7 +757,7 @@ The input and output for this service are shown in the table below: | Input | Output | | --- | --- | -| `ENUM_Region (enumeration)`, `UseStaticCredentials (boolean)` | `Count (integer)` | +| `ENUM_Region (enumeration)`, `Credentials (object)` | `Count (integer)` | The operation returns an integer that indicates how many objects were created or changed during the syncing process. From aba40431acb971cf6b3df9eef5d0654d713048ad Mon Sep 17 00:00:00 2001 From: MarkvanMents Date: Wed, 17 Jun 2026 16:59:58 +0200 Subject: [PATCH 2/5] Improve description of creating Credentials object --- .../modules/aws/amazon-bedrock.md | 15 ++++++++------- 1 file changed, 8 insertions(+), 7 deletions(-) diff --git a/content/en/docs/marketplace/platform-supported-content/modules/aws/amazon-bedrock.md b/content/en/docs/marketplace/platform-supported-content/modules/aws/amazon-bedrock.md index d52235597b4..04335ae0b3b 100644 --- a/content/en/docs/marketplace/platform-supported-content/modules/aws/amazon-bedrock.md +++ b/content/en/docs/marketplace/platform-supported-content/modules/aws/amazon-bedrock.md @@ -61,19 +61,20 @@ As of version 3.0.0 of the [AWS Authentication Connector](https://marketplace.me The AWS Authentication Connector supports both **static credentials** and **temporary credentials**. For more information and detailed instructions please refer to the [AWS Authentication Connector documentation page](/appstore/modules/aws/aws-authentication/). -To use other authentication ways, you can create an `Credentials` object in your custom way and pass it to the operations. -- Chat Completions: associate the `Credentials` object to the `ChatCompletionsRequest_Extension` -- Embeddings: associate the `Credentials` object to the `CohereEmbedOptions_Extension` -- Image Generation: associate the `Credentials` object to the `TitanImageOptions_Extension` -- Retrieve: associate the `Credentials` object to the `RetrieveRequest_Extension` -- Retrieve and Generate: associate the `Credentials` object to the `RetrieveAndGenerateRequest_Extension` +To use other authentication methods, you can create a `Credentials` object programmatically using custom logic and pass it to the operations using the following associations: + +* Chat Completions: associate the `Credentials` object with the `ChatCompletionsRequest_Extension` +* Embeddings: associate the `Credentials` object with the `CohereEmbedOptions_Extension` +* Image Generation: associate the `Credentials` object with the `TitanImageOptions_Extension` +* Retrieve: associate the `Credentials` object with the `RetrieveRequest_Extension` +* Retrieve and Generate: associate the `Credentials` object with the `RetrieveAndGenerateRequest_Extension` ### Syncing Available Models, Knowledge Bases, and Agents You can use the `SNIP_Settings_Admin_ConfigOverview` snippet under **_USE_ME > SyncBedrockMetadata > ReusableUI** on an administrator page to sync models, knowledge bases and agents for the selected region at runtime. Admins on the page first need to configure the settings of the [AWS Authentication Connector](#configure-authentication) module, by selecting the AWS region and how to authenticate. When saving the settings or when you sync the models for the current region, AWS metadata services are called to create persistent objects in the database which you can view in the tables at the bottom of the snippet. {{% alert color="info" %}} -All models are shown, even those which you cannot accesss. Access to models must be configured in the AWS console. +All models are shown, even those which you cannot access. Access to models must be configured in the AWS console. {{% /alert %}} #### Syncing Resources Using Microflows From d7c45fed9a04dbda1354b72ad3eb30e4a113d88a Mon Sep 17 00:00:00 2001 From: katarzyna-koltun-mx <108737161+katarzyna-koltun-mx@users.noreply.github.com> Date: Mon, 22 Jun 2026 09:36:51 +0200 Subject: [PATCH 3/5] Update amazon-bedrock.md --- .../modules/aws/amazon-bedrock.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/content/en/docs/marketplace/platform-supported-content/modules/aws/amazon-bedrock.md b/content/en/docs/marketplace/platform-supported-content/modules/aws/amazon-bedrock.md index 04335ae0b3b..74f1682eb9d 100644 --- a/content/en/docs/marketplace/platform-supported-content/modules/aws/amazon-bedrock.md +++ b/content/en/docs/marketplace/platform-supported-content/modules/aws/amazon-bedrock.md @@ -63,11 +63,11 @@ The AWS Authentication Connector supports both **static credentials** and **temp To use other authentication methods, you can create a `Credentials` object programmatically using custom logic and pass it to the operations using the following associations: -* Chat Completions: associate the `Credentials` object with the `ChatCompletionsRequest_Extension` -* Embeddings: associate the `Credentials` object with the `CohereEmbedOptions_Extension` -* Image Generation: associate the `Credentials` object with the `TitanImageOptions_Extension` -* Retrieve: associate the `Credentials` object with the `RetrieveRequest_Extension` -* Retrieve and Generate: associate the `Credentials` object with the `RetrieveAndGenerateRequest_Extension` +* Chat Completions - Associate the `Credentials` object with the `ChatCompletionsRequest_Extension`. +* Embeddings - Associate the `Credentials` object with the `CohereEmbedOptions_Extension`. +* Image Generation - Associate the `Credentials` object with the `TitanImageOptions_Extension`. +* Retrieve - Associate the `Credentials` object with the `RetrieveRequest_Extension`. +* Retrieve and Generate - Associate the `Credentials` object with the `RetrieveAndGenerateRequest_Extension`. ### Syncing Available Models, Knowledge Bases, and Agents From 2bc71044f0791b479f3429ab5deee5279f4cc420 Mon Sep 17 00:00:00 2001 From: Dana Breseman Date: Mon, 22 Jun 2026 13:04:09 +0200 Subject: [PATCH 4/5] Move Bedrock doc into Agents Kit versioning --- .../external-platforms/bedrock.md | 873 ++++++++++++++++- .../external-platforms/bedrock.md | 881 +++++++++++++++++- .../modules/aws/amazon-bedrock.md | 871 +---------------- 3 files changed, 1744 insertions(+), 881 deletions(-) diff --git a/content/en/docs/genai/v1/reference-guide/external-platforms/bedrock.md b/content/en/docs/genai/v1/reference-guide/external-platforms/bedrock.md index 0df65dd53e1..c6dc7097dad 100644 --- a/content/en/docs/genai/v1/reference-guide/external-platforms/bedrock.md +++ b/content/en/docs/genai/v1/reference-guide/external-platforms/bedrock.md @@ -1,8 +1,8 @@ --- title: "Amazon Bedrock" url: /agents/agents-kit-1/reference-guide/external-connectors/bedrock/ -weight: 10 -description: "Agents Kit 1: Describes the Amazon Bedrock GenAI service." +description: "Agents Kit 1: Describes the Amazon Bedrock connector." +weight: 20 aliases: - /appstore/modules/genai/bedrock/ - /appstore/modules/genai/reference-guide/external-connectors/bedrock/ @@ -10,12 +10,871 @@ aliases: ## Introduction -[Amazon Bedrock](https://aws.amazon.com/bedrock/) is a fully managed service that makes foundation models (FMs) from Amazon and leading AI startups available through an API. You can choose from various foundation models to find the model that is best suited for your use case. With the Amazon Bedrock serverless experience, you can quickly get started and easily experiment with all kinds of generative AI functionality such as leading large language models, knowledge bases or agents. +The [Amazon Bedrock connector](https://marketplace.mendix.com/link/component/215042) enables you to enrich your Mendix app with generative AI capabilities by connecting it to [Amazon Bedrock](https://aws.amazon.com/bedrock/). -## Available Model Families +Amazon Bedrock is a fully managed service that makes foundation models (FMs) from Amazon and leading AI startups available through an API, so you can choose from various FMs to find the model that is best suited for your use case. With the Amazon Bedrock serverless experience, you can quickly get started, easily experiment with FMs, and seamlessly integrate and deploy them into your applications using AWS tools and capabilities. -For more information about the model families that Amazon Bedrock supports, refer to the list of Model Providers on the [Amazon Bedrock](https://aws.amazon.com/bedrock/) webpage. +### Available Model Families -## Integrating Your Mendix App with Amazon Bedrock +For details about the model families that Amazon Bedrock supports, refer to the list of Model Providers on the [Amazon Bedrock](https://aws.amazon.com/bedrock/) webpage. -To allow your Mendix app to use Amazon Bedrock functionalities, install and configure the [Amazon Bedrock connector](/appstore/modules/aws/amazon-bedrock/). +### Typical Use Cases + +Typical use cases include the following: + +* Chatting with an AI assistant, including a chat history. +* Building an AI agent to answer questions about proprietary data. +* Generating images based on text prompts and displaying them in the Mendix app. +* Generating embedding vectors for text inputs. + +Typical use cases for generative AI are described in the [Typical LLM Use Cases](/agents/get-started/#llm-use-cases). + +### Prerequisites {#prerequisites} + +The Amazon Bedrock connector requires Mendix Studio Pro version 10.24.0 or above. + +To authenticate with Amazon Web Services (AWS), you must install and configure the [AWS Authentication connector version 4.1.1 or above](https://marketplace.mendix.com/link/component/120333). It is crucial for the Amazon Bedrock connector to function correctly. For more information about installing and configuring the AWS Authentication connector, see [AWS Authentication](/appstore/modules/aws/aws-authentication/). + +You must have the latest [GenAI Commons](https://marketplace.mendix.com/link/component/239448) version. To make integration of generative AI capabilities as easy as possible, the Amazon Bedrock connector depends on the generic domain model and operations provided by the GenAI Commons module. + +To ensure that your app can connect to Bedrock, you must also install and configure the [Encryption module](/appstore/modules/encryption/#configuration). + +### Licensing and Cost + +This connector is available as a free download from the Mendix Marketplace, but the AWS service to which it connects may incur a usage cost. For more information, refer to AWS documentation. + +The pricing of the Amazon Bedrock Connectors is dependent on the Foundational Model that you use. You can find information about the pricing in the Foundational Model overview in the AWS console. + +{{% alert color="info" %}} +Most AWS services provide a free tier that allows easy access to most services. To find out if this service is included in the free tier, see [AWS Free Tier](https://aws.amazon.com/free/). To calculate the potential cost of using an AWS service outside of the free tier, use the [AWS Cost calculator](https://calculator.aws/). +{{% /alert %}} + +Depending on your use case, your deployment environment, and the type of app that you want to build, you may also need a license for your Mendix app. For more information, refer to [Licensing Apps](/developerportal/deploy/licensing-apps-outside-mxcloud/). + +## Installation + +Follow the instructions in [How to Use Marketplace Content](/appstore/use-content/) to import the Amazon Bedrock connector into your app. + +## Configuration + +After you install the connector, you can find it in the **App Explorer**, in the **AmazonBedrockConnector** section. The connector provides a [domain model and several activities](#technical-reference) that you can use to connect your app to Amazon Bedrock. Each activity can be implemented by using it in a microflow. To ensure that your app can connect to the AWS service, you must also configure AWS authentication for the connector. + +### Configuring AWS Authentication {#configure-authentication} + +To use the Amazon Bedrock service, you must authenticate with AWS. To do so, you must set up a configuration profile in your Mendix app. After you set up the configuration profile, the connector module handles the authentication internally. + +As of version 3.0.0 of the [AWS Authentication Connector](https://marketplace.mendix.com/link/component/120333), all the resources and logic required to set up authentication are centralized inside the AWS Authentication Connector module. + +The AWS Authentication Connector supports both **static credentials** and **temporary credentials**. For more information and detailed instructions, refer to the [AWS Authentication Connector documentation page](/appstore/modules/aws/aws-authentication/). + +### Syncing Available Models, Knowledge Bases, and Agents + +You can use the `SNIP_Settings_Admin_ConfigOverview` snippet under **_USE_ME > SyncBedrockMetadata > ReusableUI** on an administrator page to sync models, knowledge bases, and agents for the selected region at runtime. Admins on the page first need to configure the settings of the [AWS Authentication Connector](#configure-authentication) module by selecting the AWS region and how to authenticate. When saving the settings or when you sync the models for the current region, AWS metadata services are called to create persistent objects in the database that you can view in the tables at the bottom of the snippet. + +{{% alert color="info" %}} +All models are shown, even those which you cannot access. Access to models must be configured in the AWS console. +{{% /alert %}} + +#### Syncing Resources Using Microflows + +If you do not want to configure the resources in an admin user interface, you can use the following actions in your custom microflows: + +* Sync Models & Knowledge Bases & Agents +* [Sync Models](#sync-models) +* [Sync Knowledge Bases](#sync-knowledge-bases) +* [Sync Agents](#sync-agents) + +The following actions only list the available resources as non-persistent objects: + +* List Foundation Models +* List Agents +* List Data Sources + +#### Adding Custom Models to your Bedrock Configuration {#adding-custom-model} + +When syncing your Bedrock configuration, only publicly available foundation models can be retrieved using the `List Foundation Models` action from the Bedrock Connector. To use cross-region inference (CRI) or provisioned-throughput models that you have setup in your AWS console, you must manually add these models to your database. To add a custom model, follow the steps below: + +1. Click **Add model** above the model's table. +2. Fill out the form with the relevant information of your custom model. + + The display name is shown in the model dropdowns while choosing a model to use. The model ID is required to use the model with Amazon Bedrock. The other fields must be filled out according to the functionalities the model can use. + +3. Save the model. + +You can now use the custom model with the actions of your choice. + +### Using Amazon Bedrock Models + +To use Amazon Bedrock models, keep in mind some specific requirements, as listed below. + +#### Model Lifecycle + +Amazon Bedrock models have a lifecycle that consists of the Active, Legacy, and EOL stages. For more information, see [Model lifecycle](https://docs.aws.amazon.com/bedrock/latest/userguide/model-lifecycle.html). Models are no longer available for use after they reach the EOL state. To ensure that your application functions as intended, make sure that you regularly monitor the state of the model that you are using. For example, you may want to use an API call to retrieve the status of the model and alert you once it reaches the Legacy state. To programmatically get information about available models and their lifecycle status, you can use the **ListFoundationModels** operation. + +### Configuring a Microflow for an AWS Service + +After you configure the authentication profile for Amazon Bedrock, you can implement the functions of the connector by using the provided activities in microflows. The most important actions are available in the toolbox or in the [GenAI Commons](/agents/agents-kit-2/genai-for-mx/commons/#microflows) module. + +The **USE_ME** folder contains several subfolders containing operations. The following example microflows have been created for each of these inside the **ExampleImplementations** folder: + +* EXAMPLE_ChatCompletions_FunctionCalling +* EXAMPLE_ChatCompletions_Vision +* EXAMPLE_ChatCompletions_withHistory +* EXAMPLE_ChatCompletions_withoutHistory +* EXAMPLE_Embeddings_ChunkCollection +* EXAMPLE_Embeddings_SingleString +* EXAMPLE_Retrieve +* EXAMPLE_RetrieveAndGenerate +* EXAMPLE_RetrieveAndGenerate_PromptTemplate +* EXAMPLE_ImageGeneration_MultipleImages + +You can also take a look at the [GenAI Showcase Application](https://marketplace.mendix.com/link/component/220475) to get some inspiration on what you can use these operations for. + +For operations that do not depend on the GenAI Commons, you can take a different approach. For example, to list all foundational models, implement the **List Foundation Models** activity by doing the following steps: + +1. In the **App Explorer**, right-click on the name of your module, and then click **Add microflow**. +2. Enter a name for your microflow, for example, *ACT_ListFoundationModels*, and then click **OK**. +3. From the **Toolbox**, drag a **Create Object** activity to your microflow and create an object of type `ListFoundationModelsRequest`. +4. In the **Toolbox**, in the in the **Amazon Bedrock (other)** section, find the **ListFoundationModels** activity. +5. Drag the **ListFoundationModels** activity onto the work area of your microflow. +6. Double-click the **ListFoundationModels** activity to configure the required parameters. +7. For the **ENUM_Region** parameter, provide a value by using a variable or an expression. This must be of the type `ENUM_Region` of the AWS Authentication connector. +8. For the **Credentials** parameter, provide a **Credentials** object from the AWS Authentication connector: + 1. In the **App Explorer**, in the **AWSAuthentication** > **Operations** section, find the **GetStaticCredentials** or **GetTemporaryCredentials** action. + 2. Drag the one you would like to use to the beginning of your microflow. + 3. Double-click the microflow action to configure the required parameters and provide a value for the AWS Region. For the **ListFoundationModels** parameter, provide the `ListFoundationModelsRequest` created in step 3. +9. The `ListFoundationModelsResponse` object is returned by the **ListFoundationModels** activity. +10. From the **Toolbox**, drag a **Retrieve object(s)** activity to your microflow and place it after the **ListFoundationModels** activity. +11. Double-click the **Retrieve object(s)** activity and make sure **By Association** is selected. +12. Select the **FoundationModelSummary_ListFoundationModelsResponse** association, which will return a list of the type **FoundationModelSummary**. +13. To further use the response information, you can create an implementation module with copies of the `ListFoundationModelsResponse` and `ModelSummary` Entities. This way, you can use your custom user roles and access rules for those entities and keep them when updating the connector. + +You can follow a similar approach to implement any of the other operations in **Amazon Bedrock (other)**. + +### Chatting with Large Language Models using the ChatCompletions Operation + +A common use case of the Amazon Bedrock Connector is the development of chatbots and chat solutions. The **ChatCompletions (without history / with history)** operations offer an easy way to connect to most of the text-generation models available on Amazon Bedrock. The ChatCompletions operations are built on top of Bedrock's Converse API, allowing you to talk to different models without the need of a model-specific implementation. For more information on the ChatCompletion operations, see [GenAI Commons: Chat Completions](/agents/agents-kit-2/genai-for-mx/commons/#genai-generate). + +For an overview of supported models and model-specific capabilities and limitations, see [Amazon Bedrock Converse API](https://docs.aws.amazon.com/bedrock/latest/userguide/conversation-inference.html#conversation-inference-supported-models-features) in the AWS documentation. + +To build a simple microflow that uses the ChatCompletions operation to send a single message to the Anthropic Claude 3.5 Sonnet model and show the response on a page, perform the following steps: + +1. Make sure that you [synced models](#sync-models) before running the microflow in the app. +2. Create a new microflow and name it, for example, *AmazonBedrockChatCompletions*. +3. Add a **Call microflow** activity from the **Toolbox** and choose microflow *AmazonBedrockConnector.BedrockDeployedModel_Get* +4. Double-click it to configure its parameters. + 1. For the **ModelID** parameter, enter the model id of the LLM you want to send a message to. The model id of Claude 3.5 Sonnet is *anthropic.claude-3-5-sonnet-20240620-v1:0*. + 2. Click **OK**. +5. In the **Toolbox**, search for the **Chat Completions (without history)** activity in the *GenAI (Generate)* and drag it onto your microflow. +6. Double click on the activity to see its parameters. + 1. The **OptionalRequest** and **OptionalFileCollection** parameters are not needed for this example, so you can set them to **empty**. + 2. For the **UserPrompt** parameter, enter a string of your choice, for example *Hi, Claude!*. + 3. For the **DeployedModel** parameter, pass the retrieved **BedrockDeployedModel** object. + 4. Click **OK**. +7. Add a **Show Message** activity to the end of the microflow and configure it to show *$Response/ResponseText* +8. Add a button that calls this microflow, run your project, and verify the results. + +{{< figure src="/attachments/appstore/platform-supported-content/modules/aws-bedrock/chat-completions-mf.png" class="no-border" >}} + +You can find several implementation examples for the ChatCompletions operations inside of the [GenAI showcase application](https://marketplace.mendix.com/link/component/220475). + +### Advanced: Invoking Specific Models by Using the InvokeModel Operation + +If you need Bedrock specific capabilities that you cannot execute through the normal [GenAI Commons](/agents/) operations, you can use the operations inside the **Amazon Bedrock** sections. In most cases, the standard procedures outlined in *GenAI Commons Operations* should be sufficient to address your needs. You can use most text models with the **ChatCompletions** operation. For an overview of the supported models and capabilities, see [Supported models and model features](https://docs.aws.amazon.com/bedrock/latest/userguide/conversation-inference.html#conversation-inference-supported-models-features) in the AWS Bedrock documentation. + +To invoke a specific model that is not covered by the ChatCompletions operation, you can make use of the **Invoke Model** operation by performing the following steps: + +1. Choose the model with which you want to interact by using the **Invoke Model** operation. +2. In the [Model Parameters](https://docs.aws.amazon.com/bedrock/latest/userguide/model-parameters.html) section of the Amazon Bedrock user guide, find the request and response JSON structures of the specific model that you want to invoke. +3. Create your domain model inspired by the JSON structures that you found. You can use a tool to visualize Json structures if needed, such as [JSON Crack](https://jsoncrack.com/editor). +4. In Mendix Studio Pro, create a JSON structure by doing the following steps: + 1. Right-click on the target folder. + 2. Click **Add other** > **JSON structure**. + 3. Create a structure for the request JSON. + 4. Repeat the previous steps to create a structure for the response JSON. + 5. Open the created JSON structure. + 6. Paste the request or response JSON into the created structure. + 7. Click **OK**. +5. Generate export and import mappings for the request and response JSON structures. + The export mapping creates a JSON from the request-related objects (specific to the model that you want to invoke). The JSON must be added as the request body of the `InvokeModelRequest` object provided as input parameter to the **Invoke Model** operation. The import mapping maps the response returned by the **Invoke Model** operation to your model-specific response objects. To create import or export mappings, perform the following steps: + 1. Right-click the target folder. + 2. Click **Add other** > **Import/Export mapping**. + 3. In the dialog box, select the **Schema source**. + 4. Click **JSON structure** and select the appropriate request/response JSON structure. + 5. Select the relevant schema elements. + 6. Click **OK**. + 7. Map the relevant elements to the correct attributes by double-clicking the shown entities and choosing the correct entity attributes for the correct elements. +6. Create a microflow that invokes a specific model using the **Invoke Model** operation, such as in the following figure (for Claude v. 2.1): + + {{< figure src="/attachments/appstore/platform-supported-content/modules/aws-bedrock/microflow.png" class="no-border" >}} + +### Invoking an Agent with the InvokeAgent Operation {#invokeagent} + +Agents in Amazon Bedrock can perform certain automated tasks such as API calls and data source interactions. For more information, see [Agents in Bedrock](https://docs.aws.amazon.com/bedrock/latest/userguide/agents.html). + +To invoke a Bedrock agent for your Mendix app, do the following steps: + +1. Create the agent, as described in [Create an agent](https://docs.aws.amazon.com/bedrock/latest/userguide/agents-create.html) in the Amazon Bedrock documentation. +2. Deploy the agent to create an alias, as described in [Deploy your agent](https://docs.aws.amazon.com/bedrock/latest/userguide/agents-deploy.html) in the Amazon Bedrock documentation. +3. In your Mendix app, create a new microflow and add the **InvokeAgent** operation as a microflow step. +4. Either pass an **InvokeAgentRequest** object as a parameter to the flow, or create one within the microflow. Ensure that the following attributes are populated for the request: + + * Agent ID + * Agent Alias ID + * Input text (the prompt to send to the agent). + * The session id (by reusing this value in a subsequent request, it is possible to continue a conversation). + * Make a choice on 'EnableTrace' to enable or disable the tracking of the reasoning process. + * Set 'EndSession' to specify whether or not you want to have the option of continuing the conversation in another request. + +5. Enter the desired region as a value of the **AWSAuthentication.ENUM_Region** type. +6. Select a **Credentials** object. You can put it in scope with the **AWSAuthentication.GetStaticCredentials** or the **AWSAuthentication.GetTemporaryCredentials** microflow. +7. Select a microflow that takes an **AmazonBedrockConnector.InvokeAgentResponse** object as an input and handles that response. + This is necessary because InvokeAgent is an asynchronous operation which means that it will not necessarily finish when the process that it was invoked from finishes. By giving the operation a handler microflow, the response can be handled as soon as it arrives. For an example handler microflow, see **AmazonBedrockConnector.InvokeAgentResponse_Handle** in the connector module. This microflow logs the response, so you can also use it just to investigate the response. + +### Token Usage {#tokenusage} + +[Token usage](/agents/agents-kit-2/genai-for-mx/commons/#token-usage) monitoring is now possible for the following operations: + +* Chat Completions with History +* Chat Completion without History +* Embeddings with Cohere Embed + +For more information about using this feature, refer to the [GenAI commons documentation](/agents/agents-kit-2/genai-for-mx/commons/#token-usage). + +## Technical Reference {#technical-reference} + +The module includes technical reference documentation for the available entities, enumerations, activities, and other items that you can use in your application. You can view the information about each object in context by using the **Documentation** pane in Studio Pro. + +The **Documentation** pane displays the documentation for the currently selected element. To view it, perform the following steps: + +1. In the [View menu](/refguide/view-menu/) of Studio Pro, select **Documentation**. +2. Click on the element for which you want to view the documentation. + + {{< figure src="/attachments/appstore/platform-supported-content/modules/technical-reference/doc-pane.png" class="no-border" >}} + +For additional information about available operations, refer to the sections below. + +### GenAI Commons-Based Operations + +#### ChatCompletions (With History) and ChatCompletions (Without History) {#chat-completions} + +The [ChatCompletions (with history)](/agents/agents-kit-2/genai-for-mx/commons/#chat-completions-with-history) and [ChatCompletions (without history)](/agents/agents-kit-2/genai-for-mx/commons/#chat-completions-without-history) activities can be used with a variety of supported LLMs. + +Some capabilities of the chat completions operations are currently only available for specific models: + +* **Function Calling** - You can use function calling in all chat completions operations. To do this, use a [supported model](https://docs.aws.amazon.com/bedrock/latest/userguide/conversation-inference-supported-models-features.html) by adding a `ToolCollection` with a `Tool` via the [Tools: Add Function to Request](/agents/agents-kit-2/genai-for-mx/commons/#add-function-to-request) operation. You can also first retrieve data from a knowledge base and then call `ChatCompletions` with the information required using the connector's function calling properties. In order to use a function calling pattern with knowledge bases, add a knowledge base to your Request using [Tools: Add Knowledge Base](/agents/agents-kit-2/genai-for-mx/commons/#add-knowledge-base-to-request). Here the collection identifier that needs to be passed is the `KnowledgeBaseID`. +For additional general information about function calling, see [Function Calling](/agents/function-calling/). + +**Function calling microflows**: A microflow used as a tool for function calling must satisfy the following conditions: + +1. At least one of the following: + + * Either none, one, or multiple primitive input parameters (such as Boolean, Datetime, Decimal, Enumeration, Integer and String) + * [Request](/agents/agents-kit-2/genai-for-mx/commons/#request) object + * [Tool](/agents/agents-kit-2/genai-for-mx/commons/#tool) object + +2. Return value of the type String. + +* **Vision** - This operation supports the *vision* capability for [supported models](https://docs.aws.amazon.com/bedrock/latest/userguide/conversation-inference-supported-models-features.html). With vision, you can send image prompts, in addition to the traditional text prompts. You can use vision by adding a `FileCollection` with a `File` to the `Message` using the [Files: Initialize Collection with File](/agents/agents-kit-2/genai-for-mx/commons/#initialize-filecollection) or the [Files: Add to Collection](/agents/agents-kit-2/genai-for-mx/commons/#add-file-to-collection) operation. Make sure to set the `FileType` attribute to **image**. + +* **Document Chat** - This operation supports the ability to chat with documents for [supported models](https://docs.aws.amazon.com/bedrock/latest/userguide/conversation-inference-supported-models-features.html). To send a document to the model add a `FileCollection` with a `System.FileDocument` to the `Message` using the [Files: Initialize Collection with File](/agents/agents-kit-2/genai-for-mx/commons/#initialize-filecollection) or the [Files: Add to Collection](/agents/agents-kit-2/genai-for-mx/commons/#add-file-to-collection) operation. For Document Chat, it is not supported to create a `FileContent` from an URL using the above mentioned operations; Please use the `System.FileDocument` option. Make sure to set the `FileType` attribute to **document**. + +##### Tool Choice + +All [tool choice types](/agents/agents-kit-2/genai-for-mx/commons/#enum-toolchoice) of GenAI Commons for the [Tools: Set Tool Choice](/agents/agents-kit-2/genai-for-mx/commons/#set-toolchoice) action are supported. For API mapping reference, see the table below: + +| GenAI Commons (Mendix) | Amazon Bedrock | +| --- | --- | +| Auto | Auto | +| Any | Any | +| None | Tools removed from request | +| Tool | Tool | + +#### RetrieveAndGenerate {#retrieve-and-generate} + +The `Retrieve and Generate` activity can be used for conversations leveraging Retrieval Augmented Generation through a knowledge base. This operation corresponds to the *RetrieveAndGenerate* microflow. +The input and output for this service are shown in the table below: + +| Input | Output | +| --- | --- | +| `GenAICommons.Request (object)`, `GenAICommons.DeployedModel (object)`| `GenAICommons.Response (object)`| + +The request object passed to this operation must include a KnowledgeBaseTool object, which can be added to the request using the [Request: Add Knowledge Base Tool to Collection](#add-knowledge-base-tool) operation. + +##### Prompt Template {#prompt-template} + +A prompt template is an orchestration mechanism that allows you to customize how Amazon Bedrock generates responses when querying a knowledge base. By leveraging prompt templates, you can influence the tone, structure, and content of responses, enabling more nuanced and context-appropriate interactions with your knowledge. + +###### Prompt Templates and System Prompts {#prompt-templates-versus-systems-prompts} + +While prompt templates may contain instructions similar to system prompts, they serve a distinct purpose in the query process. System prompts typically provide overall behavioral guidance to a model, whereas prompt templates orchestrate specifically how the retrieved information should be incorporated into responses. Prompt templates are particularly crucial in Amazon Bedrock's Retrieve operations, where system prompts are not supported. This constraint can make it challenging to do the following tasks: + +* Adopt specific tones or personas in responses +* Provide tailored advice based on search results +* Handle edge cases (such as when no relevant information is found) +* Maintain consistent response formatting + +Prompt templates address this constraint by allowing you to include orchestration instructions alongside your Retrieve operations. When creating a prompt template, you can use various tokens to customize the output. The *$searchresult$* token is mandatory in every prompt template, as it indicates where the retrieved information should be inserted. + +For a deeper understanding of prompt templates and their implementation, refer to the [Amazon documentation on prompt templates](https://docs.aws.amazon.com/bedrock/latest/userguide/kb-test-config.html#kb-test-config-sysprompt), which provides comprehensive guidance on their usage and best practices. + +For more information about how to structure your prompts, see [Prompt engineering](/agents/prompt-engineering/). + +#### Chatting with History {#retrieve-and-generate-with-history} + +The `RetrieveAndGenerate` operation only allows a single user message to be part of the request. Unlike the `ChatCompletions` operation, it is not supported to send a history of messages to the model. + +The history can be enabled using the `SessionId` parameter on the RetrieveAndGenerateRequest_Extension entity. By reusing the same `SessionId` value, the model will run in the context of the session. + +#### Image Generation {#image-generation} + +{{% alert color="info" %}} +This activity was introduced in Amazon Bedrock Connector version 3.1.0. +{{% /alert %}} + +The [Generate Image](/agents/agents-kit-2/genai-for-mx/commons/#generate-image) operation can be used to generate one or more images. Currently *Amazon Titan Image Generator G1* is the only supported model for image generation of the Amazon Bedrock Connector. + +`GenAICommons.ImageOptions` can be an empty object. If provided, it allows you to set additional options for Image Generation and can be created by using the [Image: Create Options](/agents/agents-kit-2/genai-for-mx/commons/#imageoptions-create) operation of GenAI Commons. + +To retrieve actual image objects from the response, you can use the [Image: Get Generated Image (Single)](/agents/agents-kit-2/genai-for-mx/commons/#image-get-single) or [Image: Get Generated Images (List)](/agents/agents-kit-2/genai-for-mx/commons/#image-get-list) helper operations from GenAI Commons. + +For Titan Image models, the `Image Generation: Add Titan Image Extension` operation can be used to configure Titan image-specific values (currently only *NegativeText*). + +#### Generate Embeddings (String) {#embeddings-single-string} + +The [Generate Embeddings (String)](/agents/agents-kit-2/genai-for-mx/commons/#embeddings-string) activity can be used to generate an embedding vector for a given input string with one of the Cohere Embed models. + +For Cohere Embed, the request can be associated to their respective EmbeddingsOptions extension object which can be created with the [Embeddings Options: Add Cohere Embed Extension](#add-cohere-embed-extension) operation. Through this extension, it is possible to tailor the operation to more specific needs. + +Currently, embeddings are available for the Cohere Embed family. + +#### Generate Embeddings (Chunk Collection) {#embeddings-chunk-collection} + +The [Generate Embeddings (Chunk Collection)](/agents/agents-kit-2/genai-for-mx/commons/#embeddings-chunk-collection) activity can be used to generate a collection of embedding vectors for a given collection of text chunks with one of the Cohere Embed models. + +For each model family, the request can be associated to an extension of the EmbeddingsOptions object which can be created with either the [Embeddings Options: Add Cohere Embed Extension](#add-cohere-embed-extension) operation. Through this extension, it is possible to tailor the operation to more specific needs. + +Currently, embeddings are available for the Cohere Embed family. + +#### Retrieve {#retrieve} + +The `Retrieve` activity allows you to query a knowledge base and retrieve information from it. This operation corresponds to the *Retrieve* microflow. + +The input and output for this service are shown in the table below: + +| Input | Output | +| --- | --- | +| `GenAICommons.Request (object)`| `GenAICommons.Response (object)`| + +### GenAI Commons Helper Operations + +The following operations are specific to Amazon Bedrock and helpful to configure additional settings when using operations from GenAI Commons. + +#### Add Knowledge Base Tool {#add-knowledge-base-tool} + +Use this microflow to add a new KnowledgeBaseTool object to your request. This is useful for adding additional parameters when using the [Retrieve And Generate](#retrieve-and-generate) operation. + +This operation corresponds to the **Request_AddKnowledgeBaseTool** microflow. + +| Input | Output | +| --- | --- | +| `GenAICommons.Request (object)`, `KnowledgeBaseId (string)` | *none* | + +#### Configure Bedrock Retrieve and Generate (add Knowledge Base) {#add-rag-extension} + +Use this microflow to add a new RetrieveAndGenerateRequest_Extension object to your request. This is required in order to use the [Retrieve And Generate](#retrieve-and-generate) operation successfully. + +This operation corresponds to the **RetrieveAndGenerateRequest_Extension_Create** microflow. + +| Input | Output | +| --- | --- | +| `GenAICommons.Request (object)`, `KnowledgeBaseID (string)`, `KmsKeyARN (string)`, `SessionId (string)`, `Enum_RetrieveAndGenerateType (enumeration)`, `PromptTemplate (string)` | `RetrieveAndGenerateRequest_Extension (object)` | + +`KmsKeyARN`, `SessionId`, `PromptTemplate` and `Enum_RetrieveAndGenerateType` can be empty, in which case they are not sent to the Bedrock API. + +#### Image Generation: Add Titan Image Extension {#add-titan-image-extension} + +{{% alert color="info" %}} +This microflow was introduced in Amazon Bedrock Connector version 3.1.0. +{{% /alert %}} + +Use this microflow to add a new TitanImageOptions_Extension object to your GenAICommons.ImageOptions object. This will allow you to configure the **NegativeText** attribute. + +This operation corresponds to the **TitanImageOptions_Extension_Create** microflow. + +| Input | Output | +| --- | --- | +| `GenAICommons.ImageOptions (object)`, `NegativeText (string)` | `TitanImageOptions_Extension (object)` | + +#### Image Generation: Set Image Size (Titan Image) {#set-titan-image-size} + +{{% alert color="info" %}} +This microflow was introduced in Amazon Bedrock Connector version 3.1.0. +{{% /alert %}} + +Use this microflow to set the **Height** and **Width** attributes of your **GenAICommons.ImageOptions** object to any valid image size supported by Titan Image models. The `ENUM_ImageSize_TitanImage` enumeration contains all valid height-width combinations to choose from. + +This operation corresponds to the **ImageOptions_SetImageSize_TitanImage** microflow. + +| Input | Output | +| --- | --- | +| `GenAICommons.ImageOptions (object)`, `ENUM_ImageSize_TitanImage (enumeration)` | `none` | + +#### Image Generation: Set Randomness {#set-randomness} + +{{% alert color="info" %}} +This microflow was introduced in Amazon Bedrock Connector version 3.1.0. +{{% /alert %}} + +Use this microflow to set the **Seed** and **CfgScale** attributes of your GenAICommons.ImageOptions object. These attributes can be used to influence the randomness of the image generation. + +For more information, refer to the specific model documentation such as [Titan Image Generator G1](https://docs.aws.amazon.com/bedrock/latest/userguide/model-parameters-titan-image.html). + +This operation corresponds to the **ImageOptions_SetRandomness** microflow. + +| Input | Output | +| --- | --- | +| `GenAICommons.ImageOptions (object)`, `Seed (integer)`, `CfgScale (decimal)` | `none` | + +`Seed` and `GfgScale` can be empty, in which case they are not sent to the Bedrock API. + +#### Embeddings Options: Add Cohere Embed Extension {#add-cohere-embed-extension} + +Use this microflow to add a new CohereEmbedOptions_Extension object to your `EmbeddingsOptions` object. You can use it to include parameters that are unique to Cohere Embed models. + +This operation corresponds to the **CohereEmbedOptions_Extension_Create** microflow. + +| Input | Output | +| --- | --- | +| `GenAICommons.EmbeddingsOptions (object)`, `InputType (enumeration)`, `EmbeddingTypes (enumeration, optional)`, `Truncate (enumeration, optional)` | `CohereEmbedOptions_Extension (object)`| + +#### Set Bedrock Retrieve Options {#add-r-extension} + +Use this microflow to add a new RetrieveRequest_Extension object to your request. This is required in order to use the [Retrieve](#retrieve) activity. + +To use this activity, you must set up a knowledge base in your Amazon Bedrock Environment. For more information, see [Knowledge Base](#knowledge-base). + +The input and output for this service are shown in the table below: + +| Input | Output | +| --- | --- | +| `Request (object)`, `KnowledgeBaseID (string)`, `NumberOfResults (integer)`, `NextToken (string)` | `RetrieveRequest_Extension (object)` | + +#### Request: Add Additional Request Parameter {#add-request-parameter} + +Use this microflow to add an additional model-specific request parameter to your request. Follow this link to find available additional request parameters: [Inference parameters](https://docs.aws.amazon.com/bedrock/latest/userguide/inference-parameters.html) + +This operation corresponds to the **Request_CreateAdditionalRequestParameter** microflow. + +| Input | Output | +| --- | --- | +| `GenAICommons.Request (object)`, `Key (string)`, `StringValue (string)`, `DecimalValue (decimal)`, `IntegerValue (integer)` | `none` | + +You need to provide a value using either the *StringValue*, *DecimalValue*, or *IntegerValue* parameters. For example, if you are providing a *StringValue* as the value of the parameter, *DecimalValue* and *IntegerValue* should be left **empty**. + +#### Request: Add Additional Response Field {#add-response-field} + +Some models can return additional information that is not part of the `GenAICommons.Response` entity. Use this microflow to add an additional model-specific response field to your request. + +You can retrieve the additional requested response fields using the [Response: Get Requested Response Fields](#get-response-fields) operation. + +This operation corresponds to the **Request_CreateResponseFieldRequest** microflow. + +| Input | Output | +| --- | --- | +| `GenAICommons.Request (object)`, `FieldName (string)`| `none` | + +If the used model supports that response field, it will be returned as a ChatCompletionsResponse object as part of the response. + +#### Response: Get Requested Response Fields {#get-response-fields} + +Use this microflow to retrieve all requested model-specific response fields from the response. + +Some models can return additional information that is not part of the `GenAICommons.Response` entity. You can request additional request parameters using the [Request: Add Additional Response Fields](#add-response-field) operation. + +This operation corresponds to the **Response_GetRequestedResponseFields** microflow. + +| Input | Output | +| --- | --- | +| `GenAICommons.Response (object)`| `RequestedResponseField (list)` | + +#### Response: Get NextToken {#get-next-token} + +Use this microflow to retrieve the NextToken from the response after using the `Retrieve` operation. + +This operation corresponds to the **Response_GetNextToken** microflow. + +| Input | Output | +| --- | --- | +| `GenAICommons.Response (object)`| `NextToken (string)` | + +#### Response: Cast RetrieveAndGenerateResponse {#cast-rag-response} + +Use this microflow to get the RetrieveAndGenerateResponse object from the GenAiCommons.Response that is returned by the `RetrieveAndGenerate` operation. + +The RetrieveAndGenerateResponse object contains the SessionID of the current Session that can be used in a subsequent request to chat within the same session. + +This operation corresponds to the **Response_Cast_RetrieveAndGenerateResponse** microflow. + +| Input | Output | +| --- | --- | +| `GenAICommons.Response (object)`| `RetrieveAndGenerateResponse (object)` | + +### Other Operations + +#### ListFoundationModels {#list-foundation-models} + +The `ListFoundationModels` activity allows you to get all the available foundational models which Amazon Bedrock provides. It requires `ENUM_Region`, `Credentials` and `ListFoundationModelsRequest` as input parameters. + +The input and output for this service are shown in the table below: + +| Input | Output | +| --- | --- | +| `ENUM_Region (enumeration)`, `Credentials (object)`, `ListFoundationModelsRequest (object)` | `ListFoundationModelsResponse (object)`| + +#### InvokeModel {#invoke-model} + +The `InvokeModel` activity allows you to invoke a model from Amazon Bedrock. This activity provides the generic parts that are equal for the invocation of every model. It requires `ENUM_Region`, `Credentials` and `InvokeModelRequest` as input parameters. + +The `InvokeModel` operation provides a versatile interface for integrating with Amazon Bedrock models. Each available model in Amazon Bedrock has its own set of model-specific parameters required to be passed into the `InvokeModelRequest`. The Amazon Bedrock Connector contains two example implementations to showcase how to use the `InvokeModel` operation to invoke specific models. + +The input and output for this service are shown in the table below: + +| Input | Output | +| --- | --- | +| `ENUM_Region (enumeration)`, `Credentials (object)`, `InvokeModelRequest (object)` | `InvokeModelResponse (object)` | + +#### ListKnowledgeBases {#list-knowledge-bases} + +The `ListKnowledgeBases` activity allows you to list the knowledge bases in an account and get information about each of them. It requires `ENUM_Region`, `Credentials`, and `ListKnowledgeBasesRequest` as input parameters. + +To use this activity, you must set up a knowledge base in your Amazon Bedrock Environment. For more information, see [Knowledge Base](#knowledge-base). + +The input and output for this service are shown in the table below: + +| Input | Output | +| --- | --- | +| `ENUM_Region (enumeration)`, `Credentials (object)`, `ListKnowledgeBasesRequest (object)` | `ListKnowledgeBasesResponse (object)` | + +#### StartIngestionJob {#start-ingestion-job} + +The `StartIngestionJob` activity allows you to begin an ingestion job, in which the contents of the data source S3 bucket is preprocessed and synced with the vector database of the knowledge base. It requires `ENUM_Region`, `Credentials` and `StartIngestionJobRequest` as input parameters. + +To use this activity, you must set up a knowledge base in your Amazon Bedrock Environment. For more information, see [Knowledge Base](#knowledge-base). + +The input and output for this service are shown in the table below: + +| Input | Output | +| --- | --- | +| `ENUM_Region (enumeration)`, `Credentials (object)`, `StartIngestionJobRequest (object)` | `StartIngestionJobResponse (object)` | + +#### GetIngestionJob {#get-ingestion-job} + +The `GetIngestionJob` activity allows you to retrieve information about a ingestion job, in which the contents of the data source S3 bucket is preprocessed and synced with the vector database of the knowledge base. It requires `ENUM_Region`, `Credentials` and `GetIngestionJobRequest` as input parameters. + +To use this activity, you must set up a knowledge base in your Amazon Bedrock Environment. For more information, see [Knowledge Base](#knowledge-base). + +The input and output for this service are shown in the table below: + +| Input | Output | +| --- | --- | +| `ENUM_Region (enumeration)`, `Credentials (object)`, `GetIngestionJobRequest (object)` | `GetIngestionJobResponse (object)` | + +#### ListIngestionJobs {#list-ingestion-jobs} + +The `ListIngestionJobs` action allows you to retrieve a list of the data ingestion jobs for a data source. It requires `ENUM_Region`, `Credentials` and `ListIngestionJobsRequest` as input parameters. + +To use this activity, you must set up a knowledge base and a data source in your Amazon Bedrock Environment. + +The input and output for this service are shown in the table below: + +| Input | Output | +| --- | --- | +| `ENUM_Region (enumeration)`, `Credentials (object)`, `ListIngestionJobsRequest (object)` | `ListIngestionJobsResponse (object)` | + +#### CreateDataSource {#create-data-source} + +The `CreateDataSource` action allows you to connect a knowledge base to a data source. This action is currently supported for datasource types S3 and Confluence. It requires `ENUM_Region`, `Credentials` and `CreateDataSourceRequest` as input parameters. + +To use this activity, you must set up a knowledge base in your Amazon Bedrock Environment. + +The input and output for this service are shown in the table below: + +| Input | Output | +| --- | --- | +| `ENUM_Region (enumeration)`, `Credentials (object)`, `CreateDataSourceRequest (object)` | `CreateDataSourceResponse (object)` | + +#### DeleteDataSource {#delete-data-source} + +The `DeleteDataSource` action allows you to delete a data source from a knowledge base. It requires `ENUM_Region`, `Credentials` and `DeleteDataSourceRequest` as input parameters. + +To use this activity, you must set up a knowledge base and a datasource in your Amazon Bedrock Environment. + +The input and output for this service are shown in the table below: + +| Input | Output | +| --- | --- | +| `ENUM_Region (enumeration)`, `Credentials (object)`, `DeleteDataSourceRequest (object)` | `DeleteDataSourceResponse (object)` | + +#### GetDataSource {#get-data-source} + +The `GetDataSource` action allows you to get information about a data source. It requires `ENUM_Region`, `Credentials` and `GetDataSourceRequest` as input parameters. + +To use this activity, you must set up a knowledge base and a datasource in your Amazon Bedrock Environment. + +The input and output for this service are shown in the table below: + +| Input | Output | +| --- | --- | +| `ENUM_Region (enumeration)`, `Credentials (object)`, `GetDataSourceRequest (object)` | `GetDataSourceResponse (object)` | + +#### ListDataSources {#list-data-sources} + +The `ListDataSources` action allows you to list the data sources in a knowledge base and get information about each of them. It requires `ENUM_Region`, `Credentials` and `ListDataSourcesRequest` as input parameters. + +To use this activity, you must set up a knowledge base in your Amazon Bedrock Environment. + +The input and output for this service are shown in the table below: + +| Input | Output | +| --- | --- | +| `ENUM_Region (enumeration)`, `Credentials (object)`, `ListDataSourcesRequest (object)` | `ListDataSourcesResponse (object)` | + +#### InvokeAgent {#invoke-agent} + +The `InvokeAgent` activity allows you to invoke an agent from Amazon Bedrock, so that you can orchestrate tasks involving foundation models and enrich the process with organizational data and user input. It requires `ENUM_Region`, `Credentials`, `InvokeAgentRequest`, a `ResponseHandlerMicroflow` and an `ErrorHandlerMicroflow` as input parameters. The microflow parameters are necessary since `InvokeAgent` is an asynchronous operation. The `ResponseHandlerMicroflow` is required to have exactly one input parameter of the `InvokeAgentResponse` entity type. It is called in a background thread once the response is available. The `ErrorHandlerMicroflow` is required to have exactly one input parameter of type String. It will be called when there is an error during the asynchronous process and the error type will be passed to its string parameter. The Amazon Bedrock Connector includes sample response handler and error handler microflows to help you set up handlers for your implementation. + +For more information, see [Agents for Bedrock](https://docs.aws.amazon.com/bedrock/latest/userguide/agents.html) + +The input and output for this service are shown in the table below: + +| Input | Output | +| --- | --- | +| `ENUM_Region (enumeration)`, `Credentials (object)`, `InvokeModelRequest (object)`, `ResponseHandlerMicroflow (microflow)`, `ErrorHandlerMicroflow (microflow)` | `none` | + +#### Handling the Asynchronous InvokeAgentResponse + +The `InvokeAgentResponse` object is passed as a parameter to the ResponseHandler microflow. This microflow can perform any custom logic with the `InvokeAgentResponse`, for example storing it in the database. The microflow is called in another background thread, so the client is not automatically notified when the response is processed. If you want to display the agent's response to the user of your app, you can use one of the following methods: + +##### Polling + +The easiest way to make sure the client gets a response is to constantly poll for it until it is available. This can be done using the [Microflow Timer Widget](https://marketplace.mendix.com/link/component/27), which allows you to configure a microflow or nanoflow to run every X number of seconds. + +This approach is only recommended for testing and for applications that do not have a large number of concurrent users. It is not preferred for scaling. + +##### Websockets + +WebSockets is a communication protocol that provides full-duplex communication channels over a single, persistent connection. Unlike traditional HTTP connections, which are request-response based and stateless, WebSockets enable real-time, bi-directional communication between a client (such as a Web browser) and a server. + +The open source [EZ Websocket Module](https://marketplace.mendix.com/link/component/205276) from the Mendix Marketplace provides an easy way to implement real-time server-to-client communication using WebSockets without external dependencies. + +##### Pusher + +The platform-supported [Pusher Module](https://marketplace.mendix.com/link/component/107957) is built around the [Pusher Channels](https://pusher.com/channels/) offering. This module requires a Pusher account. Pusher Channels is a paid service, but it also has a [Free Sandbox Plan](https://pusher.com/channels/pricing/). This module allows you to trigger a Notify event on the server to immediately trigger an action in the client application. + +### Working with Action Groups and Lambda Functions + +Without action groups, the agent will still access associated knowledge bases, but will not be able to perform tasks that make agents an extension of simply invoking a model. Action groups are what make agents so powerful. + +For example, it might be beneficial for the agent to dynamically retrieve more information via a REST endpoint or other source, rather than storing all possible information in a knowledge base. To achieve this, a lambda function must first be specified for the REST request and then associated with the agent as part of an action group. + +If you would like to add lambda functions to your agent, refer to the [AWS documentation](https://docs.aws.amazon.com/bedrock/latest/userguide/agents-lambda.html). + +#### ListAgents {#list-agents} + +The `ListAgents` activity allows you to list the agents in an account and get information about each of them. It requires `ENUM_Region`, `Credentials`, and `ListAgentsRequest` as input parameters. + +To use this activity, you must set up an agent in your Amazon Bedrock environment. + +The input and output for this service are shown in the table below: + +| Input | Output | +| --- | --- | +| `ENUM_Region (enumeration)`, `Credentials (object)`, `ListAgentsRequest (object)` | `ListAgentsResponse (object)` | + +#### GetAgent {#get-agent} + +The `GetAgent` activity allows you to retrieve information about an agent. It requires `ENUM_Region`, `Credentials`, and `GetAgentRequest` as input parameters. + +To use this activity, you must set up an agent in your Amazon Bedrock Environment. + +The input and output for this service are shown in the table below: + +| Input | Output | +| --- | --- | +| `ENUM_Region (enumeration)`, `Credentials (object)`, `GetAgentRequest (object)` | `GetAgentResponse (object)` | + +### Operations to Persist Amazon Bedrock Metadata inside the Application + +The Amazon Bedrock Connector offers a range of operations to retrieve and store metadata information in the Mendix app's database. + +This can be useful to, for example, associate a chatbot configuration to an available model by selecting the model via dropdown in runtime. The persistent domain model allows for simple and efficient filtering capabilities on the available metadata. Further, the *SNIP_Settings_Admin_BedrockConfig* Snippet can be used to manage and view the synced data from an administrator perspective. + +The syncing process works the same for all of these operations: + +1. Information about models, knowledge bases, and agents is persistent in the Mendix app's database on the initial sync. +2. For knowledge base and agents, an association to the `AmazonBedrockRegion` object, that represents the AWS region used when syncing, is stored. +3. On a subsequent syncing process, the available data is extended and updated. No data is removed from the app's database, even if it is no longer available on AWS. This is because existing usages of the object in the running application should not be removed. + +The available operations are described in the following sections. + +#### Sync Models {#sync-models} + +The `Sync Models` activity lets you retrieve and store metadata about available models on Amazon Bedrock in your app's database. The model information is persistent in the `BedrockDeployedModel` entity which is mandatory for chat completions or embeddings operations from GenAI Commons. + +Information about the model's input modalities are stored as associations to the `InputModality` entity and its output modality is stored to the `OutputModality` attribute. The input modality describes which form of data can be sent to the model. The output modality describes which form of data the model will return. + +Information about the model's inference type is stored as association to the `ModelInferenceType` entity. The inference type describes how the model can be accessed. *ON Demand* models are accessible by default and charged by usage. + +The input and output for this service are shown in the table below: + +| Input | Output | +| --- | --- | +| `ENUM_Region (enumeration)`, `UseStaticCredentials (boolean)` | `Count (integer)` | + +The operation returns an integer that indicates how many objects were created or changed during the syncing process. + +#### Sync Knowledge Bases {#sync-knowledge-bases} + +The `Sync Knowledge Bases` activity allows you to retrieve and store metadata about available knowledge bases on Amazon Bedrock in your app's database. +The knowledge base information is persistent in the `AmazonBedrockKnowledgeBase` entity. + +The input and output for this service are shown in the table below: + +| Input | Output | +| --- | --- | +| `ENUM_Region (enumeration)`, `UseStaticCredentials (boolean)` | `Count (integer)` | + +The operation returns an integer that indicates how many objects were created or changed during the syncing process. + +#### Sync Agents {#sync-agents} + +The `Sync Agents` activity allows you to retrieve and store metadata about available agents on Amazon Bedrock in your app's database. +The agent information is persistent in the `AmazonBedrockAgent` entity. + +The input and output for this service are shown in the table below: + +| Input | Output | +| --- | --- | +| `ENUM_Region (enumeration)`, `UseStaticCredentials (boolean)` | `Count (integer)` | + +The operation returns an integer that indicates how many objects were created or changed during the syncing process. + +### Knowledge Base {#knowledge-base} + +In Bedrock, a *knowledge base* denotes a substantial storehouse of data and information. This serves as a foundational resource, enabling the AI system to glean insights and effectively comprehend and respond to natural language queries. + +For more information about knowledge bases, see [Knowledge Base](https://docs.aws.amazon.com/bedrock/latest/userguide/knowledge-base.html) in the Amazon Bedrock documentation. + +#### Creating a Knowledge Base + +Setting up knowledge bases is usually a one-time configuration, which can be done with the AWS Console. In order to get the best results, you should consider whether you want to use one of the chunking strategies available on AWS when creating the knowledge base, or whether you want to pre-process the data beforehand. + +*Chunking* is the practice of breaking large chunks of data into smaller segments. Chunking the data allows the embedding algorithm to process the given data in chunks, thus increasing efficiency. Chunking can also introduce a structure that helps the model understand which data belongs to the same context. You can use the default chunking strategy, or create a custom strategy if there is a specific way in which the model data should be split. + +For example, when building a chatbot that gives restaurant recommendations, you should set up the knowledge base with a list of restaurant reviews. In this case, using the default chunking into 300 tokens might result in chunks containing reviews for different restaurants, which is not optimal. You will likely have better results if each chunk corresponds to reviews for one restaurant, as with that strategy it is less likely that the model will then associate a review with the wrong restaurant. You can achieve the required results by pre-processing the data so that there is one file per restaurant, and using the **No chunking** option when setting up the knowledge base. + +For more information about creating the knowledge base, including a list of the available chunking strategies, see [Create a knowledge base](https://docs.aws.amazon.com/bedrock/latest/userguide/knowledge-base-create.html). + +#### Adding Data from Your App + +After a knowledge base has been set up, information from your app can be added in a file to the relevant S3 bucket, and then used during subsequent inquiries. Which information is used and how that information is exported depends on the customer's use case and is up to the Mendix developer to implement. For more information, see [Set up your data for ingestion](https://docs.aws.amazon.com/bedrock/latest/userguide/knowledge-base-setup.html). + +Amazon Bedrock only processes the information that existed during the last sync, so the data source must be synchronized whenever a new file is added to your S3 bucket or the existing files are changed. For more information, see [Sync to ingest your data sources into the knowledge base](https://docs.aws.amazon.com/bedrock/latest/userguide/knowledge-base-ingest.html). + +The sync can be done from the information page of your knowledge base in the Amazon Bedrock Console, or by using the **StartIngestionJob** action in the Amazon Bedrock Connector. + +{{% alert color="info" %}} +The sync can take up to a few minutes and the calls to your knowledge base during this process cannot be handled accurately. To make sure the sync process has ended, you can use the **GetIngestionJob** action in the Amazon Bedrock Connector to retrieve the status of the ingestion job, along with other details. +{{% /alert %}} + +### Safeguards + +AWS has introduced safeguards for Bedrock (currently in preview). When available, there will be two features: Guardrails and Watermark detection. + +The guardrail feature will allow you to: + +* Filter harmful content with configurable thresholds based on your responsible AI policies. +* Determine how to handle personally identifiable information (PII). +* Deny topics. + +The watermark detection feature will make it possible to tell if an image has been created using Amazon Titan. + +More information about guardrails can be found in this [AWS blogpost](https://aws.amazon.com/blogs/aws/guardrails-for-amazon-bedrock-helps-implement-safeguards-customized-to-your-use-cases-and-responsible-ai-policies-preview/) and in the [AWS documentation](https://aws.amazon.com/en/bedrock/guardrails/). + +### Advanced Prompts for Agents + +By default, an agent is configured with the following base prompt templates, one for each step in the agent sequence: + +* Pre-processing +* Orchestration +* Knowledge base response generation +* Post-processing + +By customizing the prompt templates and modifying these configurations, you can fine-tune your agent's accuracy. Additionally, you can provide custom examples for a technique known as few-shot prompting. This involves providing labeled examples for specific tasks, which further enhances the model's performance in targeted areas. For more information about advanced prompts, see [Advanced prompts](https://docs.aws.amazon.com/bedrock/latest/userguide/advanced-prompts.html) in the AWS documentation. + +You can also use placeholder variables in agent prompt templates. For example, in the orchestration prompt template, the *$prompt_session_attributes$* placeholder variable can be used to ingest the information from the `PromptSessionAttribute` entity into the prompt, if it was specified as part of the `InvokeAgentRequest`. For more information about placeholder variables available in agent prompt templates, see [Prompt placeholders](https://docs.aws.amazon.com/bedrock/latest/userguide/prompt-placeholders.html) in the AWS documentation. + +## Troubleshooting + +If you encounter any issues while using the Amazon Bedrock connector, use the following troubleshooting tips to help you solve them. + +### Error Code 400 - Bad Request + +The service returns the error code *400 - Bad Request*. + +#### Cause + +Your AWS organization may not have been granted access to the model which you are trying to invoke. + +#### Solution + +To solve this issue, follow these steps: + +1. In your Amazon Bedrock environment, navigate to **Model Access** for the region in which you would like to work. +2. If the status of a model is **Available**, enable access to this model for your AWS organization by doing the following steps: + 1. In the top-right corner of the overview, click on **Edit**. + 2. Select the check boxes by the models which you want to access with your credential set. + 3. Click **Save Changes**. + +After the status of the models changes to **Access Granted**, you can use it with the Amazon Bedrock connector. + +### Error Code 403 - AccessDeniedException + +When invoking a model, the error code *403 - Access denied* indicates that you do not have access to the targeted resource. + +#### Cause + +Possible root causes for this error include the following: + +* You do not have access to the model in the specified AWS region. + +#### Solution + +To solve this issue, ensure that you have selected an AWS Region where you have model access. You can see an overview of the models accessible to you in the AWS Management Console, in the [Model Access](https://us-west-2.console.aws.amazon.com/bedrock/home?#/modelaccess) section of your Amazon Bedrock environment. + +### Error Code 404 - ResourceNotFoundException + +When invoking a model, the error code *404 - Resource not found* indicates that the targeted resource was not found. + +#### Cause + +Possible root causes for this error include the following: + +* The model which you are trying to invoke is not available in your specified AWS region. +* The model which you are trying to invoke is deprecated. + +#### Solution + +To solve this issue, verify the following: + +1. Ensure that you have selected an AWS Region where the targeted model exists. You can see an overview of the models accessible to you in the AWS Management Console, on the [Overview page](https://us-west-2.console.aws.amazon.com/bedrock/home?#/overview) of your Amazon Bedrock environment. Make sure the region specified in the AWS Console matches the region you have configured in Mendix. +2. Ensure that the model that you have selected is not deprecated and that the *model-id* is currently available in Amazon Bedrock. + +### Attribute or Reference Required Error Message After Upgrade + +If you encounter an error stating that an attribute or a reference is required after an upgrade, first upgrade all modules by right-clicking the error, then upgrade Data Widgets. + +### Conflicted Lib Error After Module Import + +If you encounter an error caused by conflicting Java libraries, such as `java.lang.NoSuchMethodError: 'com.fasterxml.jackson.annotation.OptBoolean com.fasterxml.jackson.annotation.JsonProperty.isRequired()'`, try synchronizing all dependencies (**App** > **Synchronize dependencies**) and then restart your application. diff --git a/content/en/docs/genai/v2/reference-guide/external-platforms/bedrock.md b/content/en/docs/genai/v2/reference-guide/external-platforms/bedrock.md index ded75480afd..5a636bf6eb6 100644 --- a/content/en/docs/genai/v2/reference-guide/external-platforms/bedrock.md +++ b/content/en/docs/genai/v2/reference-guide/external-platforms/bedrock.md @@ -1,8 +1,8 @@ --- title: "Amazon Bedrock" url: /agents/agents-kit-2/reference-guide/external-connectors/bedrock/ -weight: 10 -description: "Agents Kit 2: Describes the Amazon Bedrock GenAI service." +description: "Agents Kit 2: Describes the Amazon Bedrock connector." +weight: 20 aliases: - /agents/reference-guide/external-connectors/bedrock/ - /appstore/modules/genai/bedrock/ @@ -11,12 +11,879 @@ aliases: ## Introduction -[Amazon Bedrock](https://aws.amazon.com/bedrock/) is a fully managed service that makes foundation models (FMs) from Amazon and leading AI startups available through an API. You can choose from various foundation models to find the model that is best suited for your use case. With the Amazon Bedrock serverless experience, you can quickly get started and easily experiment with all kinds of generative AI functionality such as leading large language models, knowledge bases or agents. +The [Amazon Bedrock connector](https://marketplace.mendix.com/link/component/215042) enables you to enrich your Mendix app with generative AI capabilities by connecting it to [Amazon Bedrock](https://aws.amazon.com/bedrock/). -## Available Model Families +Amazon Bedrock is a fully managed service that makes foundation models (FMs) from Amazon and leading AI startups available through an API, so you can choose from various FMs to find the model that is best suited for your use case. With the Amazon Bedrock serverless experience, you can quickly get started, easily experiment with FMs, and seamlessly integrate and deploy them into your applications using AWS tools and capabilities. -For more information about the model families that Amazon Bedrock supports, refer to the list of Model Providers on the [Amazon Bedrock](https://aws.amazon.com/bedrock/) webpage. +### Available Model Families -## Integrating Your Mendix App with Amazon Bedrock +For details about the model families that Amazon Bedrock supports, refer to the list of Model Providers on the [Amazon Bedrock](https://aws.amazon.com/bedrock/) webpage. -To allow your Mendix app to use Amazon Bedrock functionalities, install and configure the [Amazon Bedrock connector](/appstore/modules/aws/amazon-bedrock/). +### Typical Use Cases + +Typical use cases include the following: + +* Chatting with an AI assistant, including a chat history. +* Building an AI agent to answer questions about proprietary data. +* Generating images based on text prompts and displaying them in the Mendix app. +* Generating embedding vectors for text inputs. + +Typical use cases for generative AI are described in the [Typical LLM Use Cases](/agents/get-started/#llm-use-cases). + +### Prerequisites {#prerequisites} + +The Amazon Bedrock connector requires Mendix Studio Pro version 10.24.0 or above. + +To authenticate with Amazon Web Services (AWS), you must install and configure the [AWS Authentication connector version 4.1.1 or above](https://marketplace.mendix.com/link/component/120333). It is crucial for the Amazon Bedrock connector to function correctly. For more information about installing and configuring the AWS Authentication connector, see [AWS Authentication](/appstore/modules/aws/aws-authentication/). + +You must have the latest [GenAI Commons](https://marketplace.mendix.com/link/component/239448) version. To make integration of generative AI capabilities as easy as possible, the Amazon Bedrock connector depends on the generic domain model and operations provided by the GenAI Commons module. + +To ensure that your app can connect to Bedrock, you must also install and configure the [Encryption module](/appstore/modules/encryption/#configuration). + +### Licensing and Cost + +This connector is available as a free download from the Mendix Marketplace, but the AWS service to which it connects may incur a usage cost. For more information, refer to AWS documentation. + +The pricing of the Amazon Bedrock Connectors is dependent on the Foundational Model that you use. You can find information about the pricing in the Foundational Model overview in the AWS console. + +{{% alert color="info" %}} +Most AWS services provide a free tier that allows easy access to most services. To find out if this service is included in the free tier, see [AWS Free Tier](https://aws.amazon.com/free/). To calculate the potential cost of using an AWS service outside of the free tier, use the [AWS Cost calculator](https://calculator.aws/). +{{% /alert %}} + +Depending on your use case, your deployment environment, and the type of app that you want to build, you may also need a license for your Mendix app. For more information, refer to [Licensing Apps](/developerportal/deploy/licensing-apps-outside-mxcloud/). + +## Installation + +Follow the instructions in [How to Use Marketplace Content](/appstore/use-content/) to import the Amazon Bedrock connector into your app. + +## Configuration + +After you install the connector, you can find it in the **App Explorer**, in the **AmazonBedrockConnector** section. The connector provides a [domain model and several activities](#technical-reference) that you can use to connect your app to Amazon Bedrock. Each activity can be implemented by using it in a microflow. To ensure that your app can connect to the AWS service, you must also configure AWS authentication for the connector. + +### Configuring AWS Authentication {#configure-authentication} + +To use the Amazon Bedrock service, you must authenticate with AWS. To do so, you must set up a configuration profile in your Mendix app. After you set up the configuration profile, the connector module handles the authentication internally. + +As of version 3.0.0 of the [AWS Authentication Connector](https://marketplace.mendix.com/link/component/120333), all the resources and logic required to set up authentication are centralized inside the AWS Authentication Connector module. + +The AWS Authentication Connector supports both **static credentials** and **temporary credentials**. For more information and detailed instructions, refer to the [AWS Authentication Connector documentation page](/appstore/modules/aws/aws-authentication/). + +To use other authentication methods, you can create a `Credentials` object programmatically using custom logic and pass it to the operations using the following associations: + +* Chat Completions - Associate the `Credentials` object with the `ChatCompletionsRequest_Extension`. +* Embeddings - Associate the `Credentials` object with the `CohereEmbedOptions_Extension`. +* Image Generation - Associate the `Credentials` object with the `TitanImageOptions_Extension`. +* Retrieve - Associate the `Credentials` object with the `RetrieveRequest_Extension`. +* Retrieve and Generate - Associate the `Credentials` object with the `RetrieveAndGenerateRequest_Extension`. + +### Syncing Available Models, Knowledge Bases, and Agents + +You can use the `SNIP_Settings_Admin_ConfigOverview` snippet under **_USE_ME > SyncBedrockMetadata > ReusableUI** on an administrator page to sync models, knowledge bases, and agents for the selected region at runtime. Admins on the page first need to configure the settings of the [AWS Authentication Connector](#configure-authentication) module by selecting the AWS region and how to authenticate. When saving the settings or when you sync the models for the current region, AWS metadata services are called to create persistent objects in the database that you can view in the tables at the bottom of the snippet. + +{{% alert color="info" %}} +All models are shown, even those which you cannot access. Access to models must be configured in the AWS console. +{{% /alert %}} + +#### Syncing Resources Using Microflows + +If you do not want to configure the resources in an admin user interface, you can use the following actions in your custom microflows: + +* Sync Models & Knowledge Bases & Agents +* [Sync Models](#sync-models) +* [Sync Knowledge Bases](#sync-knowledge-bases) +* [Sync Agents](#sync-agents) + +The following actions only list the available resources as non-persistent objects: + +* List Foundation Models +* List Agents +* List Data Sources + +#### Adding Custom Models to your Bedrock Configuration {#adding-custom-model} + +When syncing your Bedrock configuration, only publicly available foundation models can be retrieved using the `List Foundation Models` action from the Bedrock Connector. To use cross-region inference (CRI) or provisioned-throughput models that you have setup in your AWS console, you must manually add these models to your database. To add a custom model, follow the steps below: + +1. Click **Add model** above the model's table. +2. Fill out the form with the relevant information of your custom model. + + The display name is shown in the model dropdowns while choosing a model to use. The model ID is required to use the model with Amazon Bedrock. The other fields must be filled out according to the functionalities the model can use. + +3. Save the model. + +You can now use the custom model with the actions of your choice. + +### Using Amazon Bedrock Models + +To use Amazon Bedrock models, keep in mind some specific requirements, as listed below. + +#### Model Lifecycle + +Amazon Bedrock models have a lifecycle that consists of the Active, Legacy, and EOL stages. For more information, see [Model lifecycle](https://docs.aws.amazon.com/bedrock/latest/userguide/model-lifecycle.html). Models are no longer available for use after they reach the EOL state. To ensure that your application functions as intended, make sure that you regularly monitor the state of the model that you are using. For example, you may want to use an API call to retrieve the status of the model and alert you once it reaches the Legacy state. To programmatically get information about available models and their lifecycle status, you can use the **ListFoundationModels** operation. + +### Configuring a Microflow for an AWS Service + +After you configure the authentication profile for Amazon Bedrock, you can implement the functions of the connector by using the provided activities in microflows. The most important actions are available in the toolbox or in the [GenAI Commons](/agents/agents-kit-2/genai-for-mx/commons/#microflows) module. + +The **USE_ME** folder contains several subfolders containing operations. The following example microflows have been created for each of these inside the **ExampleImplementations** folder: + +* EXAMPLE_ChatCompletions_FunctionCalling +* EXAMPLE_ChatCompletions_Vision +* EXAMPLE_ChatCompletions_withHistory +* EXAMPLE_ChatCompletions_withoutHistory +* EXAMPLE_Embeddings_ChunkCollection +* EXAMPLE_Embeddings_SingleString +* EXAMPLE_Retrieve +* EXAMPLE_RetrieveAndGenerate +* EXAMPLE_RetrieveAndGenerate_PromptTemplate +* EXAMPLE_ImageGeneration_MultipleImages + +You can also take a look at the [GenAI Showcase Application](https://marketplace.mendix.com/link/component/220475) to get some inspiration on what you can use these operations for. + +For operations that do not depend on the GenAI Commons, you can take a different approach. For example, to list all foundational models, implement the **List Foundation Models** activity by doing the following steps: + +1. In the **App Explorer**, right-click on the name of your module, and then click **Add microflow**. +2. Enter a name for your microflow, for example, *ACT_ListFoundationModels*, and then click **OK**. +3. From the **Toolbox**, drag a **Create Object** activity to your microflow and create an object of type `ListFoundationModelsRequest`. +4. In the **Toolbox**, in the in the **Amazon Bedrock (other)** section, find the **ListFoundationModels** activity. +5. Drag the **ListFoundationModels** activity onto the work area of your microflow. +6. Double-click the **ListFoundationModels** activity to configure the required parameters. +7. For the **ENUM_Region** parameter, provide a value by using a variable or an expression. This must be of the type `ENUM_Region` of the AWS Authentication connector. +8. For the **Credentials** parameter, provide a **Credentials** object from the AWS Authentication connector: + 1. In the **App Explorer**, in the **AWSAuthentication** > **Operations** section, find the **GetStaticCredentials** or **GetTemporaryCredentials** action. + 2. Drag the one you would like to use to the beginning of your microflow. + 3. Double-click the microflow action to configure the required parameters and provide a value for the AWS Region. For the **ListFoundationModels** parameter, provide the `ListFoundationModelsRequest` created in step 3. +9. The `ListFoundationModelsResponse` object is returned by the **ListFoundationModels** activity. +10. From the **Toolbox**, drag a **Retrieve object(s)** activity to your microflow and place it after the **ListFoundationModels** activity. +11. Double-click the **Retrieve object(s)** activity and make sure **By Association** is selected. +12. Select the **FoundationModelSummary_ListFoundationModelsResponse** association, which will return a list of the type **FoundationModelSummary**. +13. To further use the response information, you can create an implementation module with copies of the `ListFoundationModelsResponse` and `ModelSummary` Entities. This way, you can use your custom user roles and access rules for those entities and keep them when updating the connector. + +You can follow a similar approach to implement any of the other operations in **Amazon Bedrock (other)**. + +### Chatting with Large Language Models using the ChatCompletions Operation + +A common use case of the Amazon Bedrock Connector is the development of chatbots and chat solutions. The **ChatCompletions (without history / with history)** operations offer an easy way to connect to most of the text-generation models available on Amazon Bedrock. The ChatCompletions operations are built on top of Bedrock's Converse API, allowing you to talk to different models without the need of a model-specific implementation. For more information on the ChatCompletion operations, see [GenAI Commons: Chat Completions](/agents/agents-kit-2/genai-for-mx/commons/#genai-generate). + +For an overview of supported models and model-specific capabilities and limitations, see [Amazon Bedrock Converse API](https://docs.aws.amazon.com/bedrock/latest/userguide/conversation-inference.html#conversation-inference-supported-models-features) in the AWS documentation. + +To build a simple microflow that uses the ChatCompletions operation to send a single message to the Anthropic Claude 3.5 Sonnet model and show the response on a page, perform the following steps: + +1. Make sure that you [synced models](#sync-models) before running the microflow in the app. +2. Create a new microflow and name it, for example, *AmazonBedrockChatCompletions*. +3. Add a **Call microflow** activity from the **Toolbox** and choose microflow *AmazonBedrockConnector.BedrockDeployedModel_Get* +4. Double-click it to configure its parameters. + 1. For the **ModelID** parameter, enter the model id of the LLM you want to send a message to. The model id of Claude 3.5 Sonnet is *anthropic.claude-3-5-sonnet-20240620-v1:0*. + 2. Click **OK**. +5. In the **Toolbox**, search for the **Chat Completions (without history)** activity in the *GenAI (Generate)* and drag it onto your microflow. +6. Double click on the activity to see its parameters. + 1. The **OptionalRequest** and **OptionalFileCollection** parameters are not needed for this example, so you can set them to **empty**. + 2. For the **UserPrompt** parameter, enter a string of your choice, for example *Hi, Claude!*. + 3. For the **DeployedModel** parameter, pass the retrieved **BedrockDeployedModel** object. + 4. Click **OK**. +7. Add a **Show Message** activity to the end of the microflow and configure it to show *$Response/ResponseText* +8. Add a button that calls this microflow, run your project, and verify the results. + +{{< figure src="/attachments/appstore/platform-supported-content/modules/aws-bedrock/chat-completions-mf.png" class="no-border" >}} + +You can find several implementation examples for the ChatCompletions operations inside of the [GenAI showcase application](https://marketplace.mendix.com/link/component/220475). + +### Advanced: Invoking Specific Models by Using the InvokeModel Operation + +If you need Bedrock specific capabilities that you cannot execute through the normal [GenAI Commons](/agents/) operations, you can use the operations inside the **Amazon Bedrock** sections. In most cases, the standard procedures outlined in *GenAI Commons Operations* should be sufficient to address your needs. You can use most text models with the **ChatCompletions** operation. For an overview of the supported models and capabilities, see [Supported models and model features](https://docs.aws.amazon.com/bedrock/latest/userguide/conversation-inference.html#conversation-inference-supported-models-features) in the AWS Bedrock documentation. + +To invoke a specific model that is not covered by the ChatCompletions operation, you can make use of the **Invoke Model** operation by performing the following steps: + +1. Choose the model with which you want to interact by using the **Invoke Model** operation. +2. In the [Model Parameters](https://docs.aws.amazon.com/bedrock/latest/userguide/model-parameters.html) section of the Amazon Bedrock user guide, find the request and response JSON structures of the specific model that you want to invoke. +3. Create your domain model inspired by the JSON structures that you found. You can use a tool to visualize Json structures if needed, such as [JSON Crack](https://jsoncrack.com/editor). +4. In Mendix Studio Pro, create a JSON structure by doing the following steps: + 1. Right-click on the target folder. + 2. Click **Add other** > **JSON structure**. + 3. Create a structure for the request JSON. + 4. Repeat the previous steps to create a structure for the response JSON. + 5. Open the created JSON structure. + 6. Paste the request or response JSON into the created structure. + 7. Click **OK**. +5. Generate export and import mappings for the request and response JSON structures. + The export mapping creates a JSON from the request-related objects (specific to the model that you want to invoke). The JSON must be added as the request body of the `InvokeModelRequest` object provided as input parameter to the **Invoke Model** operation. The import mapping maps the response returned by the **Invoke Model** operation to your model-specific response objects. To create import or export mappings, perform the following steps: + 1. Right-click the target folder. + 2. Click **Add other** > **Import/Export mapping**. + 3. In the dialog box, select the **Schema source**. + 4. Click **JSON structure** and select the appropriate request/response JSON structure. + 5. Select the relevant schema elements. + 6. Click **OK**. + 7. Map the relevant elements to the correct attributes by double-clicking the shown entities and choosing the correct entity attributes for the correct elements. +6. Create a microflow that invokes a specific model using the **Invoke Model** operation, such as in the following figure (for Claude v. 2.1): + + {{< figure src="/attachments/appstore/platform-supported-content/modules/aws-bedrock/microflow.png" class="no-border" >}} + +### Invoking an Agent with the InvokeAgent Operation {#invokeagent} + +Agents in Amazon Bedrock can perform certain automated tasks such as API calls and data source interactions. For more information, see [Agents in Bedrock](https://docs.aws.amazon.com/bedrock/latest/userguide/agents.html). + +To invoke a Bedrock agent for your Mendix app, do the following steps: + +1. Create the agent, as described in [Create an agent](https://docs.aws.amazon.com/bedrock/latest/userguide/agents-create.html) in the Amazon Bedrock documentation. +2. Deploy the agent to create an alias, as described in [Deploy your agent](https://docs.aws.amazon.com/bedrock/latest/userguide/agents-deploy.html) in the Amazon Bedrock documentation. +3. In your Mendix app, create a new microflow and add the **InvokeAgent** operation as a microflow step. +4. Either pass an **InvokeAgentRequest** object as a parameter to the flow, or create one within the microflow. Ensure that the following attributes are populated for the request: + + * Agent ID + * Agent Alias ID + * Input text (the prompt to send to the agent). + * The session id (by reusing this value in a subsequent request, it is possible to continue a conversation). + * Make a choice on 'EnableTrace' to enable or disable the tracking of the reasoning process. + * Set 'EndSession' to specify whether or not you want to have the option of continuing the conversation in another request. + +5. Enter the desired region as a value of the **AWSAuthentication.ENUM_Region** type. +6. Select a **Credentials** object. You can put it in scope with the **AWSAuthentication.GetStaticCredentials** or the **AWSAuthentication.GetTemporaryCredentials** microflow. +7. Select a microflow that takes an **AmazonBedrockConnector.InvokeAgentResponse** object as an input and handles that response. + This is necessary because InvokeAgent is an asynchronous operation which means that it will not necessarily finish when the process that it was invoked from finishes. By giving the operation a handler microflow, the response can be handled as soon as it arrives. For an example handler microflow, see **AmazonBedrockConnector.InvokeAgentResponse_Handle** in the connector module. This microflow logs the response, so you can also use it just to investigate the response. + +### Token Usage {#tokenusage} + +[Token usage](/agents/agents-kit-2/genai-for-mx/commons/#token-usage) monitoring is now possible for the following operations: + +* Chat Completions with History +* Chat Completion without History +* Embeddings with Cohere Embed + +For more information about using this feature, refer to the [GenAI commons documentation](/agents/agents-kit-2/genai-for-mx/commons/#token-usage). + +## Technical Reference {#technical-reference} + +The module includes technical reference documentation for the available entities, enumerations, activities, and other items that you can use in your application. You can view the information about each object in context by using the **Documentation** pane in Studio Pro. + +The **Documentation** pane displays the documentation for the currently selected element. To view it, perform the following steps: + +1. In the [View menu](/refguide/view-menu/) of Studio Pro, select **Documentation**. +2. Click on the element for which you want to view the documentation. + + {{< figure src="/attachments/appstore/platform-supported-content/modules/technical-reference/doc-pane.png" class="no-border" >}} + +For additional information about available operations, refer to the sections below. + +### GenAI Commons-Based Operations + +#### ChatCompletions (With History) and ChatCompletions (Without History) {#chat-completions} + +The [ChatCompletions (with history)](/agents/agents-kit-2/genai-for-mx/commons/#chat-completions-with-history) and [ChatCompletions (without history)](/agents/agents-kit-2/genai-for-mx/commons/#chat-completions-without-history) activities can be used with a variety of supported LLMs. + +Some capabilities of the chat completions operations are currently only available for specific models: + +* **Function Calling** - You can use function calling in all chat completions operations. To do this, use a [supported model](https://docs.aws.amazon.com/bedrock/latest/userguide/conversation-inference-supported-models-features.html) by adding a `ToolCollection` with a `Tool` via the [Tools: Add Function to Request](/agents/agents-kit-2/genai-for-mx/commons/#add-function-to-request) operation. You can also first retrieve data from a knowledge base and then call `ChatCompletions` with the information required using the connector's function calling properties. In order to use a function calling pattern with knowledge bases, add a knowledge base to your Request using [Tools: Add Knowledge Base](/agents/agents-kit-2/genai-for-mx/commons/#add-knowledge-base-to-request). Here the collection identifier that needs to be passed is the `KnowledgeBaseID`. +For additional general information about function calling, see [Function Calling](/agents/function-calling/). + +**Function calling microflows**: A microflow used as a tool for function calling must satisfy the following conditions: + +1. At least one of the following: + + * Either none, one, or multiple primitive input parameters (such as Boolean, Datetime, Decimal, Enumeration, Integer and String) + * [Request](/agents/agents-kit-2/genai-for-mx/commons/#request) object + * [Tool](/agents/agents-kit-2/genai-for-mx/commons/#tool) object + +2. Return value of the type String. + +* **Vision** - This operation supports the *vision* capability for [supported models](https://docs.aws.amazon.com/bedrock/latest/userguide/conversation-inference-supported-models-features.html). With vision, you can send image prompts, in addition to the traditional text prompts. You can use vision by adding a `FileCollection` with a `File` to the `Message` using the [Files: Initialize Collection with File](/agents/agents-kit-2/genai-for-mx/commons/#initialize-filecollection) or the [Files: Add to Collection](/agents/agents-kit-2/genai-for-mx/commons/#add-file-to-collection) operation. Make sure to set the `FileType` attribute to **image**. + +* **Document Chat** - This operation supports the ability to chat with documents for [supported models](https://docs.aws.amazon.com/bedrock/latest/userguide/conversation-inference-supported-models-features.html). To send a document to the model add a `FileCollection` with a `System.FileDocument` to the `Message` using the [Files: Initialize Collection with File](/agents/agents-kit-2/genai-for-mx/commons/#initialize-filecollection) or the [Files: Add to Collection](/agents/agents-kit-2/genai-for-mx/commons/#add-file-to-collection) operation. For Document Chat, it is not supported to create a `FileContent` from an URL using the above mentioned operations; Please use the `System.FileDocument` option. Make sure to set the `FileType` attribute to **document**. + +##### Tool Choice + +All [tool choice types](/agents/agents-kit-2/genai-for-mx/commons/#enum-toolchoice) of GenAI Commons for the [Tools: Set Tool Choice](/agents/agents-kit-2/genai-for-mx/commons/#set-toolchoice) action are supported. For API mapping reference, see the table below: + +| GenAI Commons (Mendix) | Amazon Bedrock | +| --- | --- | +| Auto | Auto | +| Any | Any | +| None | Tools removed from request | +| Tool | Tool | + +#### RetrieveAndGenerate {#retrieve-and-generate} + +The `Retrieve and Generate` activity can be used for conversations leveraging Retrieval Augmented Generation through a knowledge base. This operation corresponds to the *RetrieveAndGenerate* microflow. +The input and output for this service are shown in the table below: + +| Input | Output | +| --- | --- | +| `GenAICommons.Request (object)`, `GenAICommons.DeployedModel (object)`| `GenAICommons.Response (object)`| + +The request object passed to this operation must include a KnowledgeBaseTool object, which can be added to the request using the [Request: Add Knowledge Base Tool to Collection](#add-knowledge-base-tool) operation. + +##### Prompt Template {#prompt-template} + +A prompt template is an orchestration mechanism that allows you to customize how Amazon Bedrock generates responses when querying a knowledge base. By leveraging prompt templates, you can influence the tone, structure, and content of responses, enabling more nuanced and context-appropriate interactions with your knowledge. + +###### Prompt Templates and System Prompts {#prompt-templates-versus-systems-prompts} + +While prompt templates may contain instructions similar to system prompts, they serve a distinct purpose in the query process. System prompts typically provide overall behavioral guidance to a model, whereas prompt templates orchestrate specifically how the retrieved information should be incorporated into responses. Prompt templates are particularly crucial in Amazon Bedrock's Retrieve operations, where system prompts are not supported. This constraint can make it challenging to do the following tasks: + +* Adopt specific tones or personas in responses +* Provide tailored advice based on search results +* Handle edge cases (such as when no relevant information is found) +* Maintain consistent response formatting + +Prompt templates address this constraint by allowing you to include orchestration instructions alongside your Retrieve operations. When creating a prompt template, you can use various tokens to customize the output. The *$searchresult$* token is mandatory in every prompt template, as it indicates where the retrieved information should be inserted. + +For a deeper understanding of prompt templates and their implementation, refer to the [Amazon documentation on prompt templates](https://docs.aws.amazon.com/bedrock/latest/userguide/kb-test-config.html#kb-test-config-sysprompt), which provides comprehensive guidance on their usage and best practices. + +For more information about how to structure your prompts, see [Prompt engineering](/agents/prompt-engineering/). + +#### Chatting with History {#retrieve-and-generate-with-history} + +The `RetrieveAndGenerate` operation only allows a single user message to be part of the request. Unlike the `ChatCompletions` operation, it is not supported to send a history of messages to the model. + +The history can be enabled using the `SessionId` parameter on the RetrieveAndGenerateRequest_Extension entity. By reusing the same `SessionId` value, the model will run in the context of the session. + +#### Image Generation {#image-generation} + +{{% alert color="info" %}} +This activity was introduced in Amazon Bedrock Connector version 3.1.0. +{{% /alert %}} + +The [Generate Image](/agents/agents-kit-2/genai-for-mx/commons/#generate-image) operation can be used to generate one or more images. Currently *Amazon Titan Image Generator G1* is the only supported model for image generation of the Amazon Bedrock Connector. + +`GenAICommons.ImageOptions` can be an empty object. If provided, it allows you to set additional options for Image Generation and can be created by using the [Image: Create Options](/agents/agents-kit-2/genai-for-mx/commons/#imageoptions-create) operation of GenAI Commons. + +To retrieve actual image objects from the response, you can use the [Image: Get Generated Image (Single)](/agents/agents-kit-2/genai-for-mx/commons/#image-get-single) or [Image: Get Generated Images (List)](/agents/agents-kit-2/genai-for-mx/commons/#image-get-list) helper operations from GenAI Commons. + +For Titan Image models, the `Image Generation: Add Titan Image Extension` operation can be used to configure Titan image-specific values (currently only *NegativeText*). + +#### Generate Embeddings (String) {#embeddings-single-string} + +The [Generate Embeddings (String)](/agents/agents-kit-2/genai-for-mx/commons/#embeddings-string) activity can be used to generate an embedding vector for a given input string with one of the Cohere Embed models. + +For Cohere Embed, the request can be associated to their respective EmbeddingsOptions extension object which can be created with the [Embeddings Options: Add Cohere Embed Extension](#add-cohere-embed-extension) operation. Through this extension, it is possible to tailor the operation to more specific needs. + +Currently, embeddings are available for the Cohere Embed family. + +#### Generate Embeddings (Chunk Collection) {#embeddings-chunk-collection} + +The [Generate Embeddings (Chunk Collection)](/agents/agents-kit-2/genai-for-mx/commons/#embeddings-chunk-collection) activity can be used to generate a collection of embedding vectors for a given collection of text chunks with one of the Cohere Embed models. + +For each model family, the request can be associated to an extension of the EmbeddingsOptions object which can be created with either the [Embeddings Options: Add Cohere Embed Extension](#add-cohere-embed-extension) operation. Through this extension, it is possible to tailor the operation to more specific needs. + +Currently, embeddings are available for the Cohere Embed family. + +#### Retrieve {#retrieve} + +The `Retrieve` activity allows you to query a knowledge base and retrieve information from it. This operation corresponds to the *Retrieve* microflow. + +The input and output for this service are shown in the table below: + +| Input | Output | +| --- | --- | +| `GenAICommons.Request (object)`| `GenAICommons.Response (object)`| + +### GenAI Commons Helper Operations + +The following operations are specific to Amazon Bedrock and helpful to configure additional settings when using operations from GenAI Commons. + +#### Add Knowledge Base Tool {#add-knowledge-base-tool} + +Use this microflow to add a new KnowledgeBaseTool object to your request. This is useful for adding additional parameters when using the [Retrieve And Generate](#retrieve-and-generate) operation. + +This operation corresponds to the **Request_AddKnowledgeBaseTool** microflow. + +| Input | Output | +| --- | --- | +| `GenAICommons.Request (object)`, `KnowledgeBaseId (string)` | *none* | + +#### Configure Bedrock Retrieve and Generate (add Knowledge Base) {#add-rag-extension} + +Use this microflow to add a new RetrieveAndGenerateRequest_Extension object to your request. This is required in order to use the [Retrieve And Generate](#retrieve-and-generate) operation successfully. + +This operation corresponds to the **RetrieveAndGenerateRequest_Extension_Create** microflow. + +| Input | Output | +| --- | --- | +| `GenAICommons.Request (object)`, `KnowledgeBaseID (string)`, `KmsKeyARN (string)`, `SessionId (string)`, `Enum_RetrieveAndGenerateType (enumeration)`, `PromptTemplate (string)` | `RetrieveAndGenerateRequest_Extension (object)` | + +`KmsKeyARN`, `SessionId`, `PromptTemplate` and `Enum_RetrieveAndGenerateType` can be empty, in which case they are not sent to the Bedrock API. + +#### Image Generation: Add Titan Image Extension {#add-titan-image-extension} + +{{% alert color="info" %}} +This microflow was introduced in Amazon Bedrock Connector version 3.1.0. +{{% /alert %}} + +Use this microflow to add a new TitanImageOptions_Extension object to your GenAICommons.ImageOptions object. This will allow you to configure the **NegativeText** attribute. + +This operation corresponds to the **TitanImageOptions_Extension_Create** microflow. + +| Input | Output | +| --- | --- | +| `GenAICommons.ImageOptions (object)`, `NegativeText (string)` | `TitanImageOptions_Extension (object)` | + +#### Image Generation: Set Image Size (Titan Image) {#set-titan-image-size} + +{{% alert color="info" %}} +This microflow was introduced in Amazon Bedrock Connector version 3.1.0. +{{% /alert %}} + +Use this microflow to set the **Height** and **Width** attributes of your **GenAICommons.ImageOptions** object to any valid image size supported by Titan Image models. The `ENUM_ImageSize_TitanImage` enumeration contains all valid height-width combinations to choose from. + +This operation corresponds to the **ImageOptions_SetImageSize_TitanImage** microflow. + +| Input | Output | +| --- | --- | +| `GenAICommons.ImageOptions (object)`, `ENUM_ImageSize_TitanImage (enumeration)` | `none` | + +#### Image Generation: Set Randomness {#set-randomness} + +{{% alert color="info" %}} +This microflow was introduced in Amazon Bedrock Connector version 3.1.0. +{{% /alert %}} + +Use this microflow to set the **Seed** and **CfgScale** attributes of your GenAICommons.ImageOptions object. These attributes can be used to influence the randomness of the image generation. + +For more information, refer to the specific model documentation such as [Titan Image Generator G1](https://docs.aws.amazon.com/bedrock/latest/userguide/model-parameters-titan-image.html). + +This operation corresponds to the **ImageOptions_SetRandomness** microflow. + +| Input | Output | +| --- | --- | +| `GenAICommons.ImageOptions (object)`, `Seed (integer)`, `CfgScale (decimal)` | `none` | + +`Seed` and `GfgScale` can be empty, in which case they are not sent to the Bedrock API. + +#### Embeddings Options: Add Cohere Embed Extension {#add-cohere-embed-extension} + +Use this microflow to add a new CohereEmbedOptions_Extension object to your `EmbeddingsOptions` object. You can use it to include parameters that are unique to Cohere Embed models. + +This operation corresponds to the **CohereEmbedOptions_Extension_Create** microflow. + +| Input | Output | +| --- | --- | +| `GenAICommons.EmbeddingsOptions (object)`, `InputType (enumeration)`, `EmbeddingTypes (enumeration, optional)`, `Truncate (enumeration, optional)` | `CohereEmbedOptions_Extension (object)`| + +#### Set Bedrock Retrieve Options {#add-r-extension} + +Use this microflow to add a new RetrieveRequest_Extension object to your request. This is required in order to use the [Retrieve](#retrieve) activity. + +To use this activity, you must set up a knowledge base in your Amazon Bedrock Environment. For more information, see [Knowledge Base](#knowledge-base). + +The input and output for this service are shown in the table below: + +| Input | Output | +| --- | --- | +| `Request (object)`, `KnowledgeBaseID (string)`, `NumberOfResults (integer)`, `NextToken (string)` | `RetrieveRequest_Extension (object)` | + +#### Request: Add Additional Request Parameter {#add-request-parameter} + +Use this microflow to add an additional model-specific request parameter to your request. Follow this link to find available additional request parameters: [Inference parameters](https://docs.aws.amazon.com/bedrock/latest/userguide/inference-parameters.html) + +This operation corresponds to the **Request_CreateAdditionalRequestParameter** microflow. + +| Input | Output | +| --- | --- | +| `GenAICommons.Request (object)`, `Key (string)`, `StringValue (string)`, `DecimalValue (decimal)`, `IntegerValue (integer)` | `none` | + +You need to provide a value using either the *StringValue*, *DecimalValue*, or *IntegerValue* parameters. For example, if you are providing a *StringValue* as the value of the parameter, *DecimalValue* and *IntegerValue* should be left **empty**. + +#### Request: Add Additional Response Field {#add-response-field} + +Some models can return additional information that is not part of the `GenAICommons.Response` entity. Use this microflow to add an additional model-specific response field to your request. + +You can retrieve the additional requested response fields using the [Response: Get Requested Response Fields](#get-response-fields) operation. + +This operation corresponds to the **Request_CreateResponseFieldRequest** microflow. + +| Input | Output | +| --- | --- | +| `GenAICommons.Request (object)`, `FieldName (string)`| `none` | + +If the used model supports that response field, it will be returned as a ChatCompletionsResponse object as part of the response. + +#### Response: Get Requested Response Fields {#get-response-fields} + +Use this microflow to retrieve all requested model-specific response fields from the response. + +Some models can return additional information that is not part of the `GenAICommons.Response` entity. You can request additional request parameters using the [Request: Add Additional Response Fields](#add-response-field) operation. + +This operation corresponds to the **Response_GetRequestedResponseFields** microflow. + +| Input | Output | +| --- | --- | +| `GenAICommons.Response (object)`| `RequestedResponseField (list)` | + +#### Response: Get NextToken {#get-next-token} + +Use this microflow to retrieve the NextToken from the response after using the `Retrieve` operation. + +This operation corresponds to the **Response_GetNextToken** microflow. + +| Input | Output | +| --- | --- | +| `GenAICommons.Response (object)`| `NextToken (string)` | + +#### Response: Cast RetrieveAndGenerateResponse {#cast-rag-response} + +Use this microflow to get the RetrieveAndGenerateResponse object from the GenAiCommons.Response that is returned by the `RetrieveAndGenerate` operation. + +The RetrieveAndGenerateResponse object contains the SessionID of the current Session that can be used in a subsequent request to chat within the same session. + +This operation corresponds to the **Response_Cast_RetrieveAndGenerateResponse** microflow. + +| Input | Output | +| --- | --- | +| `GenAICommons.Response (object)`| `RetrieveAndGenerateResponse (object)` | + +### Other Operations + +#### ListFoundationModels {#list-foundation-models} + +The `ListFoundationModels` activity allows you to get all the available foundational models which Amazon Bedrock provides. It requires `ENUM_Region`, `Credentials` and `ListFoundationModelsRequest` as input parameters. + +The input and output for this service are shown in the table below: + +| Input | Output | +| --- | --- | +| `ENUM_Region (enumeration)`, `Credentials (object)`, `ListFoundationModelsRequest (object)` | `ListFoundationModelsResponse (object)`| + +#### InvokeModel {#invoke-model} + +The `InvokeModel` activity allows you to invoke a model from Amazon Bedrock. This activity provides the generic parts that are equal for the invocation of every model. It requires `ENUM_Region`, `Credentials` and `InvokeModelRequest` as input parameters. + +The `InvokeModel` operation provides a versatile interface for integrating with Amazon Bedrock models. Each available model in Amazon Bedrock has its own set of model-specific parameters required to be passed into the `InvokeModelRequest`. The Amazon Bedrock Connector contains two example implementations to showcase how to use the `InvokeModel` operation to invoke specific models. + +The input and output for this service are shown in the table below: + +| Input | Output | +| --- | --- | +| `ENUM_Region (enumeration)`, `Credentials (object)`, `InvokeModelRequest (object)` | `InvokeModelResponse (object)` | + +#### ListKnowledgeBases {#list-knowledge-bases} + +The `ListKnowledgeBases` activity allows you to list the knowledge bases in an account and get information about each of them. It requires `ENUM_Region`, `Credentials`, and `ListKnowledgeBasesRequest` as input parameters. + +To use this activity, you must set up a knowledge base in your Amazon Bedrock Environment. For more information, see [Knowledge Base](#knowledge-base). + +The input and output for this service are shown in the table below: + +| Input | Output | +| --- | --- | +| `ENUM_Region (enumeration)`, `Credentials (object)`, `ListKnowledgeBasesRequest (object)` | `ListKnowledgeBasesResponse (object)` | + +#### StartIngestionJob {#start-ingestion-job} + +The `StartIngestionJob` activity allows you to begin an ingestion job, in which the contents of the data source S3 bucket is preprocessed and synced with the vector database of the knowledge base. It requires `ENUM_Region`, `Credentials` and `StartIngestionJobRequest` as input parameters. + +To use this activity, you must set up a knowledge base in your Amazon Bedrock Environment. For more information, see [Knowledge Base](#knowledge-base). + +The input and output for this service are shown in the table below: + +| Input | Output | +| --- | --- | +| `ENUM_Region (enumeration)`, `Credentials (object)`, `StartIngestionJobRequest (object)` | `StartIngestionJobResponse (object)` | + +#### GetIngestionJob {#get-ingestion-job} + +The `GetIngestionJob` activity allows you to retrieve information about a ingestion job, in which the contents of the data source S3 bucket is preprocessed and synced with the vector database of the knowledge base. It requires `ENUM_Region`, `Credentials` and `GetIngestionJobRequest` as input parameters. + +To use this activity, you must set up a knowledge base in your Amazon Bedrock Environment. For more information, see [Knowledge Base](#knowledge-base). + +The input and output for this service are shown in the table below: + +| Input | Output | +| --- | --- | +| `ENUM_Region (enumeration)`, `Credentials (object)`, `GetIngestionJobRequest (object)` | `GetIngestionJobResponse (object)` | + +#### ListIngestionJobs {#list-ingestion-jobs} + +The `ListIngestionJobs` action allows you to retrieve a list of the data ingestion jobs for a data source. It requires `ENUM_Region`, `Credentials` and `ListIngestionJobsRequest` as input parameters. + +To use this activity, you must set up a knowledge base and a data source in your Amazon Bedrock Environment. + +The input and output for this service are shown in the table below: + +| Input | Output | +| --- | --- | +| `ENUM_Region (enumeration)`, `Credentials (object)`, `ListIngestionJobsRequest (object)` | `ListIngestionJobsResponse (object)` | + +#### CreateDataSource {#create-data-source} + +The `CreateDataSource` action allows you to connect a knowledge base to a data source. This action is currently supported for datasource types S3 and Confluence. It requires `ENUM_Region`, `Credentials` and `CreateDataSourceRequest` as input parameters. + +To use this activity, you must set up a knowledge base in your Amazon Bedrock Environment. + +The input and output for this service are shown in the table below: + +| Input | Output | +| --- | --- | +| `ENUM_Region (enumeration)`, `Credentials (object)`, `CreateDataSourceRequest (object)` | `CreateDataSourceResponse (object)` | + +#### DeleteDataSource {#delete-data-source} + +The `DeleteDataSource` action allows you to delete a data source from a knowledge base. It requires `ENUM_Region`, `Credentials` and `DeleteDataSourceRequest` as input parameters. + +To use this activity, you must set up a knowledge base and a datasource in your Amazon Bedrock Environment. + +The input and output for this service are shown in the table below: + +| Input | Output | +| --- | --- | +| `ENUM_Region (enumeration)`, `Credentials (object)`, `DeleteDataSourceRequest (object)` | `DeleteDataSourceResponse (object)` | + +#### GetDataSource {#get-data-source} + +The `GetDataSource` action allows you to get information about a data source. It requires `ENUM_Region`, `Credentials` and `GetDataSourceRequest` as input parameters. + +To use this activity, you must set up a knowledge base and a datasource in your Amazon Bedrock Environment. + +The input and output for this service are shown in the table below: + +| Input | Output | +| --- | --- | +| `ENUM_Region (enumeration)`, `Credentials (object)`, `GetDataSourceRequest (object)` | `GetDataSourceResponse (object)` | + +#### ListDataSources {#list-data-sources} + +The `ListDataSources` action allows you to list the data sources in a knowledge base and get information about each of them. It requires `ENUM_Region`, `Credentials` and `ListDataSourcesRequest` as input parameters. + +To use this activity, you must set up a knowledge base in your Amazon Bedrock Environment. + +The input and output for this service are shown in the table below: + +| Input | Output | +| --- | --- | +| `ENUM_Region (enumeration)`, `Credentials (object)`, `ListDataSourcesRequest (object)` | `ListDataSourcesResponse (object)` | + +#### InvokeAgent {#invoke-agent} + +The `InvokeAgent` activity allows you to invoke an agent from Amazon Bedrock, so that you can orchestrate tasks involving foundation models and enrich the process with organizational data and user input. It requires `ENUM_Region`, `Credentials`, `InvokeAgentRequest`, a `ResponseHandlerMicroflow`, and an `ErrorHandlerMicroflow` as input parameters. The microflow parameters are necessary since `InvokeAgent` is an asynchronous operation. The `ResponseHandlerMicroflow` is required to have exactly one input parameter of the `InvokeAgentResponse` entity type. It is called in a background thread once the response is available. The `ErrorHandlerMicroflow` is required to have exactly one input parameter of type String. It will be called when there is an error during the asynchronous process and the error type will be passed to its string parameter. The Amazon Bedrock Connector includes sample response handler and error handler microflows to help you set up handlers for your implementation. + +For more information, see [Agents for Bedrock](https://docs.aws.amazon.com/bedrock/latest/userguide/agents.html) + +The input and output for this service are shown in the table below: + +| Input | Output | +| --- | --- | +| `ENUM_Region (enumeration)`, `Credentials (object)`, `InvokeModelRequest (object)`, `ResponseHandlerMicroflow (microflow)`, `ErrorHandlerMicroflow (microflow)` | `none` | + +#### Handling the Asynchronous InvokeAgentResponse + +The `InvokeAgentResponse` object is passed as a parameter to the ResponseHandler microflow. This microflow can perform any custom logic with the `InvokeAgentResponse`, for example storing it in the database. The microflow is called in another background thread, so the client is not automatically notified when the response is processed. If you want to display the agent's response to the user of your app, you can use one of the following methods: + +##### Polling + +The easiest way to make sure the client gets a response is to constantly poll for it until it is available. This can be done using the [Microflow Timer Widget](https://marketplace.mendix.com/link/component/27), which allows you to configure a microflow or nanoflow to run every X number of seconds. + +This approach is only recommended for testing and for applications that do not have a large number of concurrent users. It is not preferred for scaling. + +##### Websockets + +WebSockets is a communication protocol that provides full-duplex communication channels over a single, persistent connection. Unlike traditional HTTP connections, which are request-response based and stateless, WebSockets enable real-time, bi-directional communication between a client (such as a Web browser) and a server. + +The open source [EZ Websocket Module](https://marketplace.mendix.com/link/component/205276) from the Mendix Marketplace provides an easy way to implement real-time server-to-client communication using WebSockets without external dependencies. + +##### Pusher + +The platform-supported [Pusher Module](https://marketplace.mendix.com/link/component/107957) is built around the [Pusher Channels](https://pusher.com/channels/) offering. This module requires a Pusher account. Pusher Channels is a paid service, but it also has a [Free Sandbox Plan](https://pusher.com/channels/pricing/). This module allows you to trigger a Notify event on the server to immediately trigger an action in the client application. + +### Working with Action Groups and Lambda Functions + +Without action groups, the agent will still access associated knowledge bases, but will not be able to perform tasks that make agents an extension of simply invoking a model. Action groups are what make agents so powerful. + +For example, it might be beneficial for the agent to dynamically retrieve more information via a REST endpoint or other source, rather than storing all possible information in a knowledge base. To achieve this, a lambda function must first be specified for the REST request and then associated with the agent as part of an action group. + +If you would like to add lambda functions to your agent, refer to the [AWS documentation](https://docs.aws.amazon.com/bedrock/latest/userguide/agents-lambda.html). + +#### ListAgents {#list-agents} + +The `ListAgents` activity allows you to list the agents in an account and get information about each of them. It requires `ENUM_Region`, `Credentials`, and `ListAgentsRequest` as input parameters. + +To use this activity, you must set up an agent in your Amazon Bedrock environment. + +The input and output for this service are shown in the table below: + +| Input | Output | +| --- | --- | +| `ENUM_Region (enumeration)`, `Credentials (object)`, `ListAgentsRequest (object)` | `ListAgentsResponse (object)` | + +#### GetAgent {#get-agent} + +The `GetAgent` activity allows you to retrieve information about an agent. It requires `ENUM_Region`, `Credentials`, and `GetAgentRequest` as input parameters. + +To use this activity, you must set up an agent in your Amazon Bedrock Environment. + +The input and output for this service are shown in the table below: + +| Input | Output | +| --- | --- | +| `ENUM_Region (enumeration)`, `Credentials (object)`, `GetAgentRequest (object)` | `GetAgentResponse (object)` | + +### Operations to Persist Amazon Bedrock Metadata inside the Application + +The Amazon Bedrock Connector offers a range of operations to retrieve and store metadata information in the Mendix app's database. + +This can be useful to, for example, associate a chatbot configuration to an available model by selecting the model via dropdown in runtime. The persistent domain model allows for simple and efficient filtering capabilities on the available metadata. Further, the *SNIP_Settings_Admin_BedrockConfig* Snippet can be used to manage and view the synced data from an administrator perspective. + +The syncing process works the same for all of these operations: + +1. Information about models, knowledge bases, and agents is persistent in the Mendix app's database on the initial sync. +2. For knowledge base and agents, an association to the `AmazonBedrockRegion` object, that represents the AWS region used when syncing, is stored. +3. On a subsequent syncing process, the available data is extended and updated. No data is removed from the app's database, even if it is no longer available on AWS. This is because existing usages of the object in the running application should not be removed. + +The available operations are described in the following sections. + +#### Sync Models {#sync-models} + +The `Sync Models` activity lets you retrieve and store metadata about available models on Amazon Bedrock in your app's database. The model information is persistent in the `BedrockDeployedModel` entity which is mandatory for chat completions or embeddings operations from GenAI Commons. + +Information about the model's input modalities are stored as associations to the `InputModality` entity and its output modality is stored to the `OutputModality` attribute. The input modality describes which form of data can be sent to the model. The output modality describes which form of data the model will return. + +Information about the model's inference type is stored as association to the `ModelInferenceType` entity. The inference type describes how the model can be accessed. *ON Demand* models are accessible by default and charged by usage. + +The input and output for this service are shown in the table below: + +| Input | Output | +| --- | --- | +| `ENUM_Region (enumeration)`, `Credentials (object)` | `Count (integer)` | + +The operation returns an integer that indicates how many objects were created or changed during the syncing process. + +#### Sync Knowledge Bases {#sync-knowledge-bases} + +The `Sync Knowledge Bases` activity allows you to retrieve and store metadata about available knowledge bases on Amazon Bedrock in your app's database. +The knowledge base information is persistent in the `AmazonBedrockKnowledgeBase` entity. + +The input and output for this service are shown in the table below: + +| Input | Output | +| --- | --- | +| `ENUM_Region (enumeration)`, `Credentials (object)` | `Count (integer)` | + +The operation returns an integer that indicates how many objects were created or changed during the syncing process. + +#### Sync Agents {#sync-agents} + +The `Sync Agents` activity allows you to retrieve and store metadata about available agents on Amazon Bedrock in your app's database. +The agent information is persistent in the `AmazonBedrockAgent` entity. + +The input and output for this service are shown in the table below: + +| Input | Output | +| --- | --- | +| `ENUM_Region (enumeration)`, `Credentials (object)` | `Count (integer)` | + +The operation returns an integer that indicates how many objects were created or changed during the syncing process. + +### Knowledge Base {#knowledge-base} + +In Bedrock, a *knowledge base* denotes a substantial storehouse of data and information. This serves as a foundational resource, enabling the AI system to glean insights and effectively comprehend and respond to natural language queries. + +For more information about knowledge bases, see [Knowledge Base](https://docs.aws.amazon.com/bedrock/latest/userguide/knowledge-base.html) in the Amazon Bedrock documentation. + +#### Creating a Knowledge Base + +Setting up knowledge bases is usually a one-time configuration, which can be done with the AWS Console. In order to get the best results, you should consider whether you want to use one of the chunking strategies available on AWS when creating the knowledge base, or whether you want to pre-process the data beforehand. + +*Chunking* is the practice of breaking large chunks of data into smaller segments. Chunking the data allows the embedding algorithm to process the given data in chunks, thus increasing efficiency. Chunking can also introduce a structure that helps the model understand which data belongs to the same context. You can use the default chunking strategy, or create a custom strategy if there is a specific way in which the model data should be split. + +For example, when building a chatbot that gives restaurant recommendations, you should set up the knowledge base with a list of restaurant reviews. In this case, using the default chunking into 300 tokens might result in chunks containing reviews for different restaurants, which is not optimal. You will likely have better results if each chunk corresponds to reviews for one restaurant, as with that strategy it is less likely that the model will then associate a review with the wrong restaurant. You can achieve the required results by pre-processing the data so that there is one file per restaurant, and using the **No chunking** option when setting up the knowledge base. + +For more information about creating the knowledge base, including a list of the available chunking strategies, see [Create a knowledge base](https://docs.aws.amazon.com/bedrock/latest/userguide/knowledge-base-create.html). + +#### Adding Data from Your App + +After a knowledge base has been set up, information from your app can be added in a file to the relevant S3 bucket, and then used during subsequent inquiries. Which information is used and how that information is exported depends on the customer's use case and is up to the Mendix developer to implement. For more information, see [Set up your data for ingestion](https://docs.aws.amazon.com/bedrock/latest/userguide/knowledge-base-setup.html). + +Amazon Bedrock only processes the information that existed during the last sync, so the data source must be synchronized whenever a new file is added to your S3 bucket or the existing files are changed. For more information, see [Sync to ingest your data sources into the knowledge base](https://docs.aws.amazon.com/bedrock/latest/userguide/knowledge-base-ingest.html). + +The sync can be done from the information page of your knowledge base in the Amazon Bedrock Console, or by using the **StartIngestionJob** action in the Amazon Bedrock Connector. + +{{% alert color="info" %}} +The sync can take up to a few minutes and the calls to your knowledge base during this process cannot be handled accurately. To make sure the sync process has ended, you can use the **GetIngestionJob** action in the Amazon Bedrock Connector to retrieve the status of the ingestion job, along with other details. +{{% /alert %}} + +### Safeguards + +AWS has introduced safeguards for Bedrock (currently in preview). When available, there will be two features: Guardrails and Watermark detection. + +The guardrail feature will allow you to: + +* Filter harmful content with configurable thresholds based on your responsible AI policies. +* Determine how to handle personally identifiable information (PII). +* Deny topics. + +The watermark detection feature will make it possible to tell if an image has been created using Amazon Titan. + +More information about guardrails can be found in this [AWS blogpost](https://aws.amazon.com/blogs/aws/guardrails-for-amazon-bedrock-helps-implement-safeguards-customized-to-your-use-cases-and-responsible-ai-policies-preview/) and in the [AWS documentation](https://aws.amazon.com/en/bedrock/guardrails/). + +### Advanced Prompts for Agents + +By default, an agent is configured with the following base prompt templates, one for each step in the agent sequence: + +* Pre-processing +* Orchestration +* Knowledge base response generation +* Post-processing + +By customizing the prompt templates and modifying these configurations, you can fine-tune your agent's accuracy. Additionally, you can provide custom examples for a technique known as few-shot prompting. This involves providing labeled examples for specific tasks, which further enhances the model's performance in targeted areas. For more information about advanced prompts, see [Advanced prompts](https://docs.aws.amazon.com/bedrock/latest/userguide/advanced-prompts.html) in the AWS documentation. + +You can also use placeholder variables in agent prompt templates. For example, in the orchestration prompt template, the *$prompt_session_attributes$* placeholder variable can be used to ingest the information from the `PromptSessionAttribute` entity into the prompt, if it was specified as part of the `InvokeAgentRequest`. For more information about placeholder variables available in agent prompt templates, see [Prompt placeholders](https://docs.aws.amazon.com/bedrock/latest/userguide/prompt-placeholders.html) in the AWS documentation. + +## Troubleshooting + +If you encounter any issues while using the Amazon Bedrock connector, use the following troubleshooting tips to help you solve them. + +### Error Code 400 - Bad Request + +The service returns the error code *400 - Bad Request*. + +#### Cause + +Your AWS organization may not have been granted access to the model which you are trying to invoke. + +#### Solution + +To solve this issue, follow these steps: + +1. In your Amazon Bedrock environment, navigate to **Model Access** for the region in which you would like to work. +2. If the status of a model is **Available**, enable access to this model for your AWS organization by doing the following steps: + 1. In the top-right corner of the overview, click on **Edit**. + 2. Select the check boxes by the models which you want to access with your credential set. + 3. Click **Save Changes**. + +After the status of the models changes to **Access Granted**, you can use it with the Amazon Bedrock connector. + +### Error Code 403 - AccessDeniedException + +When invoking a model, the error code *403 - Access denied* indicates that you do not have access to the targeted resource. + +#### Cause + +Possible root causes for this error include the following: + +* You do not have access to the model in the specified AWS region. + +#### Solution + +To solve this issue, ensure that you have selected an AWS Region where you have model access. You can see an overview of the models accessible to you in the AWS Management Console, in the [Model Access](https://us-west-2.console.aws.amazon.com/bedrock/home?#/modelaccess) section of your Amazon Bedrock environment. + +### Error Code 404 - ResourceNotFoundException + +When invoking a model, the error code *404 - Resource not found* indicates that the targeted resource was not found. + +#### Cause + +Possible root causes for this error include the following: + +* The model which you are trying to invoke is not available in your specified AWS region. +* The model which you are trying to invoke is deprecated. + +#### Solution + +To solve this issue, verify the following: + +1. Ensure that you have selected an AWS Region where the targeted model exists. You can see an overview of the models accessible to you in the AWS Management Console, on the [Overview page](https://us-west-2.console.aws.amazon.com/bedrock/home?#/overview) of your Amazon Bedrock environment. Make sure the region specified in the AWS Console matches the region you have configured in Mendix. +2. Ensure that the model that you have selected is not deprecated and that the *model-id* is currently available in Amazon Bedrock. + +### Attribute or Reference Required Error Message After Upgrade + +If you encounter an error stating that an attribute or a reference is required after an upgrade, first upgrade all modules by right-clicking the error, then upgrade Data Widgets. + +### Conflicted Lib Error After Module Import + +If you encounter an error caused by conflicting Java libraries, such as `java.lang.NoSuchMethodError: 'com.fasterxml.jackson.annotation.OptBoolean com.fasterxml.jackson.annotation.JsonProperty.isRequired()'`, try synchronizing all dependencies (**App** > **Synchronize dependencies**) and then restart your application. diff --git a/content/en/docs/marketplace/platform-supported-content/modules/aws/amazon-bedrock.md b/content/en/docs/marketplace/platform-supported-content/modules/aws/amazon-bedrock.md index 74f1682eb9d..ef9cf609821 100644 --- a/content/en/docs/marketplace/platform-supported-content/modules/aws/amazon-bedrock.md +++ b/content/en/docs/marketplace/platform-supported-content/modules/aws/amazon-bedrock.md @@ -1,7 +1,7 @@ --- title: "Amazon Bedrock" url: /appstore/modules/aws/amazon-bedrock/ -description: "Describes the configuration and usage of the Amazon Bedrock connector from the Mendix Marketplace. Amazon Bedrock is a fully managed service that makes foundation models (FMs) from Amazon and leading AI startups available through an API." +description: "Describes the Amazon Bedrock connector from the Mendix Marketplace. Amazon Bedrock is a fully managed service that makes foundation models (FMs) from Amazon and leading AI startups available through an API." weight: 20 aliases: - /appstore/connectors/aws/amazon-bedrock/ @@ -12,871 +12,8 @@ aliases: The [Amazon Bedrock](https://marketplace.mendix.com/link/component/215042) connector enables you to enrich your Mendix app with generative AI capabilities by connecting it to [Amazon Bedrock](https://aws.amazon.com/bedrock/). -### Typical Use Cases +Amazon Bedrock is a fully managed service that makes foundation models (FMs) from Amazon and leading AI startups available through an API, so you can choose from various FMs to find the model that is best suited for your use case. With the Amazon Bedrock serverless experience, you can quickly get started, easily experiment with FMs, and seamlessly integrate and deploy them into your applications using AWS tools and capabilities. -Amazon Bedrock is a fully managed service that makes foundation models (FMs) from Amazon and leading AI startups available through an API, so you can choose from various FMs to find the model that is best suited for your use case. With the Amazon Bedrock serverless experience, you can quickly get started, easily experiment with FMs, and seamlessly integrate and deploy them into your applications using AWS tools and capabilities. Typical use cases include the following: +## Integrating Your Mendix App with Amazon Bedrock -* Chatting with an AI assistant, including a chat history. -* Building an AI agent to answer questions about proprietary data. -* Generating images based on text prompts and displaying them in the Mendix app. -* Generating embedding vectors for text inputs. - -Typical use cases for generative AI are described in the [Typical LLM Use Cases](/agents/get-started/#llm-use-cases). - -### Prerequisites {#prerequisites} - -The Amazon Bedrock connector requires Mendix Studio Pro version 10.24.0 or above. - -To authenticate with Amazon Web Service (AWS), you must install and configure the [AWS Authentication connector version 4.1.1 or above](https://marketplace.mendix.com/link/component/120333). It is crucial for the Amazon Bedrock connector to function correctly. For more information about installing and configuring the AWS Authentication connector, see [AWS Authentication](/appstore/modules/aws/aws-authentication/). - -You must have the latest [GenAI Commons](https://marketplace.mendix.com/link/component/239448) version. To make integration of generative AI capabilities as easy as possible, the Amazon Bedrock connector depends on the generic domain model and operations provided by the GenAI Commons module. - -To ensure that your app can connect to Bedrock, you must also install and configure the [Encryption module](/appstore/modules/encryption/#configuration). - -### Licensing and Cost - -This connector is available as a free download from the Mendix Marketplace, but the AWS service to which is connects may incur a usage cost. For more information, refer to AWS documentation. - -The pricing of the Amazon Bedrock Connectors is dependent on the Foundational Model that you use. You can find information about the pricing in the Foundational Model overview in the AWS console. - -{{% alert color="info" %}} -Most AWS services provide a free tier that allows easy access to most services. To find out if this service is included in the free tier, see [AWS Free Tier](https://aws.amazon.com/free/). To calculate the potential cost of using an AWS service outside of the free tier, use the [AWS Cost calculator](https://calculator.aws/). -{{% /alert %}} - -Depending on your use case, your deployment environment, and the type of app that you want to build, you may also need a license for your Mendix app. For more information, refer to [Licensing Apps](/developerportal/deploy/licensing-apps-outside-mxcloud/). - -## Installation - -Follow the instructions in [How to Use Marketplace Content](/appstore/use-content/) to import the Amazon Bedrock connector into your app. - -## Configuration - -After you install the connector, you can find it in the **App Explorer**, in the **AmazonBedrockConnector** section. The connector provides a [domain model and several activities](#technical-reference) that you can use to connect your app to Amazon Bedrock. Each activity can be implemented by using it in a microflow. To ensure that your app can connect to the AWS service, you must also configure AWS authentication for the connector. - -### Configuring AWS Authentication {#configure-authentication} - -To use the Amazon Bedrock service, you must authenticate with AWS. To do so, you must set up a configuration profile in your Mendix app. After you set up the configuration profile, the connector module handles the authentication internally. - -As of version 3.0.0 of the [AWS Authentication Connector](https://marketplace.mendix.com/link/component/120333), all the resources and logic required to set up authentication are centralized inside the AWS Authentication Connector module. - -The AWS Authentication Connector supports both **static credentials** and **temporary credentials**. For more information and detailed instructions please refer to the [AWS Authentication Connector documentation page](/appstore/modules/aws/aws-authentication/). - -To use other authentication methods, you can create a `Credentials` object programmatically using custom logic and pass it to the operations using the following associations: - -* Chat Completions - Associate the `Credentials` object with the `ChatCompletionsRequest_Extension`. -* Embeddings - Associate the `Credentials` object with the `CohereEmbedOptions_Extension`. -* Image Generation - Associate the `Credentials` object with the `TitanImageOptions_Extension`. -* Retrieve - Associate the `Credentials` object with the `RetrieveRequest_Extension`. -* Retrieve and Generate - Associate the `Credentials` object with the `RetrieveAndGenerateRequest_Extension`. - -### Syncing Available Models, Knowledge Bases, and Agents - -You can use the `SNIP_Settings_Admin_ConfigOverview` snippet under **_USE_ME > SyncBedrockMetadata > ReusableUI** on an administrator page to sync models, knowledge bases and agents for the selected region at runtime. Admins on the page first need to configure the settings of the [AWS Authentication Connector](#configure-authentication) module, by selecting the AWS region and how to authenticate. When saving the settings or when you sync the models for the current region, AWS metadata services are called to create persistent objects in the database which you can view in the tables at the bottom of the snippet. - -{{% alert color="info" %}} -All models are shown, even those which you cannot access. Access to models must be configured in the AWS console. -{{% /alert %}} - -#### Syncing Resources Using Microflows - -If you do not want to configure the resources in an admin user interface, you can use the following actions in your custom microflows: - -* Sync Models & Knowledge Bases & Agents -* [Sync Models](#sync-models) -* [Sync Knowledge Bases](#sync-knowledge-bases) -* [Sync Agents](#sync-agents) - -The following actions only list the available resources as non-persistent objects: - -* List Foundation Models -* List Agents -* List Data Sources - -#### Adding Custom Models to your Bedrock Configuration {#adding-custom-model} - -When syncing your Bedrock configuration, only publicly available foundation models can be retrieved using the `List Foundation Models` action from the Bedrock Connector. To use cross-region inference (CRI) or provisioned-throughput models that you have setup in your AWS console, you must manually add these models to your database. To add a custom model, follow the steps below: - -1. Click **Add model** above the model's table. -2. Fill out the form with the relevant information of your custom model. - - The display name is shown in the model dropdowns while choosing a model to use. The model ID is required to use the model with Amazon Bedrock. The other fields must be filled out according to the functionalities the model can use. - -3. Save the model. - -You can now use the custom model with the actions of your choice. - -### Using Amazon Bedrock Models - -To use Amazon Bedrock models, keep in mind some specific requirements, as listed below. - -#### Model Lifecycle - -Amazon Bedrock models have a lifecycle that consists of the Active, Legacy, and EOL stages. For more information, see [Model lifecycle](https://docs.aws.amazon.com/bedrock/latest/userguide/model-lifecycle.html). Models are no longer available for use after they reach the EOL state. To ensure that your application functions as intended, make sure that you regularly monitor the state of the model that you are using. For example, you may want to use an API call to retrieve the status of the model and alert you once it reaches the Legacy state. To programmatically get information about available models and their lifecycle status, you can use the **ListFoundationModels** operation. - -### Configuring a Microflow for an AWS Service - -After you configure the authentication profile for Amazon Bedrock, you can implement the functions of the connector by using the provided activities in microflows. The most important actions are available in the toolbox or in the [GenAI Commons](/agents/agents-kit-2/genai-for-mx/commons/#microflows) module. - -The **USE_ME** folder contains several subfolders containing operations. The following example microflows have been created for each of these inside the **ExampleImplementations** folder: - -* EXAMPLE_ChatCompletions_FunctionCalling -* EXAMPLE_ChatCompletions_Vision -* EXAMPLE_ChatCompletions_withHistory -* EXAMPLE_ChatCompletions_withoutHistory -* EXAMPLE_Embeddings_ChunkCollection -* EXAMPLE_Embeddings_SingleString -* EXAMPLE_Retrieve -* EXAMPLE_RetrieveAndGenerate -* EXAMPLE_RetrieveAndGenerate_PromptTemplate -* EXAMPLE_ImageGeneration_MultipleImages - -You can also take a look at the [GenAI Showcase Application](https://marketplace.mendix.com/link/component/220475) to get some inspiration on what you can use these operations for. - -For operations that do not depend on the GenAI Commons, you can take a different approach. For example, to list all foundational models, implement the **List Foundation Models** activity by doing the following steps: - -1. In the **App Explorer**, right-click on the name of your module, and then click **Add microflow**. -2. Enter a name for your microflow, for example, *ACT_ListFoundationModels*, and then click **OK**. -3. From the **Toolbox**, drag a **Create Object** activity to your microflow and create an object of type `ListFoundationModelsRequest`. -4. In the **Toolbox**, in the in the **Amazon Bedrock (other)** section, find the **ListFoundationModels** activity. -5. Drag the **ListFoundationModels** activity onto the work area of your microflow. -6. Double-click the **ListFoundationModels** activity to configure the required parameters. -7. For the **ENUM_Region** parameter, provide a value by using a variable or an expression. This must be of the type `ENUM_Region` of the AWS Authentication connector. -8. For the **Credentials** parameter, provide a **Credentials** object from the AWS Authentication connector: - 1. In the **App Explorer**, in the **AWSAuthentication** > **Operations** section, find the **GetStaticCredentials** or **GetTemporaryCredentials** action. - 2. Drag the one you would like to use to the beginning of your microflow. - 3. Double-click the microflow action to configure the required parameters and provide a value for the AWS Region. For the **ListFoundationModels** parameter, provide the `ListFoundationModelsRequest` created in step 3. -9. The `ListFoundationModelsResponse` object is returned by the **ListFoundationModels** activity. -10. From the **Toolbox**, drag a **Retrieve object(s)** activity to your microflow and place it after the **ListFoundationModels** activity. -11. Double-click the **Retrieve object(s)** activity and make sure **By Association** is selected. -12. Select the **FoundationModelSummary_ListFoundationModelsResponse** association, which will return a list of the type **FoundationModelSummary**. -13. To further use the response information, you can create an implementation module with copies of the `ListFoundationModelsResponse` and `ModelSummary` Entities. This way, you can use your custom user roles and access rules for those entities and keep them when updating the connector. - -You can follow a similar approach to implement any of the other operations in **Amazon Bedrock (other)**. - -### Chatting with Large Language Models using the ChatCompletions Operation - -A common use case of the Amazon Bedrock Connector is the development of chatbots and chat solutions. The **ChatCompletions (without history / with history)** operations offer an easy way to connect to most of the text-generation models available on Amazon Bedrock. The ChatCompletions operations are built on top of Bedrock's Converse API, allowing you to talk to different models without the need of a model-specific implementation. For more information on the ChatCompletion operations, see [GenAI Commons: Chat Completions](/agents/agents-kit-2/genai-for-mx/commons/#genai-generate). - -For an overview of supported models and model-specific capabilities and limitations, see [Amazon Bedrock Converse API](https://docs.aws.amazon.com/bedrock/latest/userguide/conversation-inference.html#conversation-inference-supported-models-features) in the AWS documentation. - -To build a simple microflow that uses the ChatCompletions operation to send a single message to the Anthropic Claude 3.5 Sonnet model and show the response on a page, perform the following steps: - -1. Make sure that you [synced models](#sync-models) before running the microflow in the app. -2. Create a new microflow and name it, for example, *AmazonBedrockChatCompletions*. -3. Add a **Call microflow** activity from the **Toolbox** and choose microflow *AmazonBedrockConnector.BedrockDeployedModel_Get* -4. Double-click it to configure its parameters. - 1. For the **ModelID** parameter, enter the model id of the LLM you want to send a message to. The model id of Claude 3.5 Sonnet is *anthropic.claude-3-5-sonnet-20240620-v1:0*. - 2. Click **OK**. -5. In the **Toolbox**, search for the **Chat Completions (without history)** activity in the *GenAI (Generate)* and drag it onto your microflow. -6. Double click on the activity to see its parameters. - 1. The **OptionalRequest** and **OptionalFileCollection** parameters are not needed for this example, so you can set them to **empty**. - 2. For the **UserPrompt** parameter, enter a string of your choice, for example *Hi, Claude!*. - 3. For the **DeployedModel** parameter, pass the retrieved **BedrockDeployedModel** object - 4. CLick **OK**. -7. Add a **Show Message** activity to the end of the microflow and configure it to show *$Response/ResponseText* -8. Add a button that calls this microflow, run your project, and verify the results. - -{{< figure src="/attachments/appstore/platform-supported-content/modules/aws-bedrock/chat-completions-mf.png" class="no-border" >}} - -You can find several implementation examples for the ChatCompletions operations inside of the [GenAI showcase application](https://marketplace.mendix.com/link/component/220475). - -### Advanced: Invoking Specific Models by Using the InvokeModel Operation - -If you need Bedrock specific capabilities that you cannot execute through the normal [GenAI Commons](/agents/) operations, you can use the operations inside the **Amazon Bedrock** sections. In most cases, the standard procedures outlined in *GenAI Commons Operations* should be sufficient to address your needs. You can use most text models with the **ChatCompletions** operation. For an overview of the supported models and capabilities, see [Supported models and model features](https://docs.aws.amazon.com/bedrock/latest/userguide/conversation-inference.html#conversation-inference-supported-models-features) in the AWS Bedrock documentation. - -To invoke a specific model that is not covered by the ChatCompletions operation, you can make use of the **Invoke Model** operation by performing the following steps: - -1. Choose the model with which you want to interact by using the **Invoke Model** operation. -2. In the [Model Parameters](https://docs.aws.amazon.com/bedrock/latest/userguide/model-parameters.html) section of the Amazon Bedrock user guide, find the request and response JSON structures of the specific model that you want to invoke. -3. Create your domain model inspired by the JSON structures that you found. You can use a tool to visualize Json structures if needed, such as [JSON Crack](https://jsoncrack.com/editor). -4. In Mendix Studio Pro, create a JSON structure by doing the following steps: - 1. Right-click on the target folder. - 2. Click **Add other** > **JSON structure**. - 3. Create a structure for the request JSON. - 4. Repeat the previous steps to create a structure for the response JSON. - 5. Open the created JSON structure. - 6. Paste the request or response JSON into the created structure. - 7. Click **OK**. -5. Generate export and import mappings for the request and response JSON structures. - The export mapping creates a JSON from the request-related objects (specific to the model that you want to invoke). The JSON must be added as the request body of the `InvokeModelRequest` object provided as input parameter to the **Invoke Model** operation. The import mapping maps the response returned by the **Invoke Model** operation to your model-specific response objects. To create import or export mappings, perform the following steps: - 1. Right-click the target folder. - 2. Click **Add other** > **Import/Export mapping**. - 3. In the dialog box, select the **Schema source**. - 4. Click **JSON structure** and select the appropriate request/response JSON structure. - 5. Select the relevant schema elements. - 6. Click **OK**. - 7. Map the relevant elements to the correct attributes by double-clicking the shown entities and choosing the correct entity attributes for the correct elements. -6. Create a microflow that invokes a specific model using the **Invoke Model** operation, such as in the following figure (for Claude v. 2.1): - - {{< figure src="/attachments/appstore/platform-supported-content/modules/aws-bedrock/microflow.png" class="no-border" >}} - -### Invoking an Agent with the InvokeAgent Operation {#invokeagent} - -Agents in Amazon Bedrock can perform certain automated tasks such as API calls and data source interactions. For more information, see [Agents in Bedrock](https://docs.aws.amazon.com/bedrock/latest/userguide/agents.html). - -To invoke a Bedrock agent for your Mendix app, do the following steps: - -1. Create the agent, as described in [Create an agent](https://docs.aws.amazon.com/bedrock/latest/userguide/agents-create.html) in the Amazon Bedrock documentation. -2. Deploy the agent to create an alias, as described in [Deploy your agent](https://docs.aws.amazon.com/bedrock/latest/userguide/agents-deploy.html) in the Amazon Bedrock documentation. -3. In your Mendix app, create a new microflow and add the **InvokeAgent** operation as a microflow step. -4. Either pass an **InvokeAgentRequest** object as a parameter to the flow, or create one within the microflow. Ensure that the following attributes are populated for the request: - - * Agent ID - * Agent Alias ID - * Input text (the prompt to send to the agent). - * The session id (by reusing this value in a subsequent request, it is possible to continue a conversation). - * Make a choice on 'EnableTrace' to enable or disable the tracking of the reasoning process. - * Set 'EndSession' to specify whether or not you want to have the option of continuing the conversation in another request. - -5. Enter the desired region as a value of the **AWSAuthentication.ENUM_Region** type. -6. Select a **Credentials** object. You can put it in scope with the **AWSAuthentication.GetStaticCredentials** or the **AWSAuthentication.GetTemporaryCredentials** microflow. -7. Select a microflow that takes an **AmazonBedrockConnector.InvokeAgentResponse** object as an input and handles that response. - This is necessary because InvokeAgent is an asynchronous operation which means that it will not necessarily finish when the process that it was invoked from finishes. By giving the operation a handler microflow, the response can be handled as soon as it arrives. For an example handler microflow, see **AmazonBedrockConnector.InvokeAgentResponse_Handle** in the connector module. This microflow logs the response, so you can also use it just to investigate the response. - -### Token Usage {#tokenusage} - -[Token usage](/agents/agents-kit-2/genai-for-mx/commons/#token-usage) monitoring is now possible for the following operations: - -* Chat Completions with History -* Chat Completion without History -* Embeddings with Cohere Embed - -For more information about using this feature, refer to the [GenAI commons documentation](/agents/agents-kit-2/genai-for-mx/commons/#token-usage). - -## Technical Reference {#technical-reference} - -The module includes technical reference documentation for the available entities, enumerations, activities, and other items that you can use in your application. You can view the information about each object in context by using the **Documentation** pane in Studio Pro. - -The **Documentation** pane displays the documentation for the currently selected element. To view it, perform the following steps: - -1. In the [View menu](/refguide/view-menu/) of Studio Pro, select **Documentation**. -2. Click on the element for which you want to view the documentation. - - {{< figure src="/attachments/appstore/platform-supported-content/modules/technical-reference/doc-pane.png" class="no-border" >}} - -For additional information about available operations, refer to the sections below. - -### GenAI Commons-Based Operations - -#### ChatCompletions (With History) and ChatCompletions (Without History) {#chat-completions} - -The [ChatCompletions (with history)](/agents/agents-kit-2/genai-for-mx/commons/#chat-completions-with-history) and [ChatCompletions (without history)](/agents/agents-kit-2/genai-for-mx/commons/#chat-completions-without-history) activities can be used with a variety of supported LLMs. - -Some capabilities of the chat completions operations are currently only available for specific models: - -* **Function Calling** - You can use function calling in all chat completions operations. To do this, use a [supported model](https://docs.aws.amazon.com/bedrock/latest/userguide/conversation-inference-supported-models-features.html) by adding a `ToolCollection` with a `Tool` via the [Tools: Add Function to Request](/agents/agents-kit-2/genai-for-mx/commons/#add-function-to-request) operation. You can also first retrieve data from a knowledge base and then call `ChatCompletions` with the information required using the connector's function calling properties. In order to use a function calling pattern with knowledge bases, add a knowledge base to your Request using [Tools: Add Knowledge Base](/agents/agents-kit-2/genai-for-mx/commons/#add-knowledge-base-to-request). Here the collection identifier that needs to be passed is the `KnowledgeBaseID`. -For additional general information about function calling, see [Function Calling](/agents/function-calling/). - -**Function calling microflows**: A microflow used as a tool for function calling must satisfy the following conditions: - -1. At least one of the following: - - * Either none, one, or multiple primitive input parameters (such as Boolean, Datetime, Decimal, Enumeration, Integer and String) - * [Request](/agents/agents-kit-2/genai-for-mx/commons/#request) object - * [Tool](/agents/agents-kit-2/genai-for-mx/commons/#tool) object - -2. Return value of the type String. - -* **Vision** - This operation supports the *vision* capability for [supported models](https://docs.aws.amazon.com/bedrock/latest/userguide/conversation-inference-supported-models-features.html). With vision, you can send image prompts, in addition to the traditional text prompts. You can use vision by adding a `FileCollection` with a `File` to the `Message` using the [Files: Initialize Collection with File](/agents/agents-kit-2/genai-for-mx/commons/#initialize-filecollection) or the [Files: Add to Collection](/agents/agents-kit-2/genai-for-mx/commons/#add-file-to-collection) operation. Make sure to set the `FileType` attribute to **image**. - -* **Document Chat** - This operation supports the ability to chat with documents for [supported models](https://docs.aws.amazon.com/bedrock/latest/userguide/conversation-inference-supported-models-features.html). To send a document to the model add a `FileCollection` with a `System.FileDocument` to the `Message` using the [Files: Initialize Collection with File](/agents/agents-kit-2/genai-for-mx/commons/#initialize-filecollection) or the [Files: Add to Collection](/agents/agents-kit-2/genai-for-mx/commons/#add-file-to-collection) operation. For Document Chat, it is not supported to create a `FileContent` from an URL using the above mentioned operations; Please use the `System.FileDocument` option. Make sure to set the `FileType` attribute to **document**. - -##### Tool Choice - -All [tool choice types](/agents/agents-kit-2/genai-for-mx/commons/#enum-toolchoice) of GenAI Commons for the [Tools: Set Tool Choice](/agents/agents-kit-2/genai-for-mx/commons/#set-toolchoice) action are supported. For API mapping reference, see the table below: - -| GenAI Commons (Mendix) | Amazon Bedrock | -| --- | --- | -| Auto | Auto | -| Any | Any | -| None | Tools removed from request | -| Tool | Tool | - -#### RetrieveAndGenerate {#retrieve-and-generate} - -The `Retrieve and Generate` activity can be used for conversations leveraging Retrieval Augmented Generation through a knowledge base. This operation corresponds to the *RetrieveAndGenerate* microflow. -The input and output for this service are shown in the table below: - -| Input | Output | -| --- | --- | -| `GenAICommons.Request (object)`, `GenAICommons.DeployedModel (object)`| `GenAICommons.Response (object)`| - -The request object passed to this operation must include a KnowledgeBaseTool object, which can be added to the request using the [Request: Add Knowledge Base Tool to Collection](#add-knowledge-base-tool) operation. - -##### Prompt Template {#prompt-template} - -A prompt template is an orchestration mechanism that allows you to customize how Amazon Bedrock generates responses when querying a knowledge base. By leveraging prompt templates, you can influence the tone, structure, and content of responses, enabling more nuanced and context-appropriate interactions with your knowledge. - -###### Prompt Templates and System Prompts {#prompt-templates-versus-systems-prompts} - -While prompt templates may contain instructions similar to system prompts, they serve a distinct purpose in the query process. System prompts typically provide overall behavioral guidance to a model, whereas prompt templates orchestrate specifically how the retrieved information should be incorporated into responses. Prompt templates are particularly crucial in Amazon Bedrock's Retrieve operations, where system prompts are not supported. This constraint can make it challenging to do the following tasks: - -* Adopt specific tones or personas in responses -* Provide tailored advice based on search results -* Handle edge cases (such as when no relevant information is found) -* Maintain consistent response formatting - -Prompt templates address this constraint by allowing you to include orchestration instructions alongside your Retrieve operations. When creating a prompt template, you can use various tokens to customize the output. The *$searchresult$* token is mandatory in every prompt template, as it indicates where the retrieved information should be inserted. - -For a deeper understanding of prompt templates and their implementation, refer to the [Amazon documentation on prompt templates](https://docs.aws.amazon.com/bedrock/latest/userguide/kb-test-config.html#kb-test-config-sysprompt), which provides comprehensive guidance on their usage and best practices. - -For more information about how to structure your prompts, see [Prompt engineering](/agents/prompt-engineering/). - -#### Chatting with History {#retrieve-and-generate-with-history} - -The `RetrieveAndGenerate` operation only allows a single user message to be part of the request. Unlike the `ChatCompletions` operation, it is not supported to send a history of messages to the model. - -The history can be enabled using the `SessionId` parameter on the RetrieveAndGenerateRequest_Extension entity. By reusing the same `SessionId` value, the model will run in the context of the session. - -#### Image Generation {#image-generation} - -{{% alert color="info" %}} -This activity was introduced in Amazon Bedrock Connector version 3.1.0. -{{% /alert %}} - -The [Generate Image](/agents/agents-kit-2/genai-for-mx/commons/#generate-image) operation can be used to generate one or more images. Currently *Amazon Titan Image Generator G1* is the only supported model for image generation of the Amazon Bedrock Connector. - -`GenAICommons.ImageOptions` can be an empty object. If provided, it allows you to set additional options for Image Generation and can be created by using the [Image: Create Options](/agents/agents-kit-2/genai-for-mx/commons/#imageoptions-create) operation of GenAI Commons. - -To retrieve actual image objects from the response, you can use the [Image: Get Generated Image (Single)](/agents/agents-kit-2/genai-for-mx/commons/#image-get-single) or [Image: Get Generated Images (List)](/agents/agents-kit-2/genai-for-mx/commons/#image-get-list) helper operations from GenAI Commons. - -For Titan Image models, the `Image Generation: Add Titan Image Extension` operation can be used to configure Titan image-specific values (currently only *NegativeText*). - -#### Generate Embeddings (String) {#embeddings-single-string} - -The [Generate Embeddings (String)](/agents/agents-kit-2/genai-for-mx/commons/#embeddings-string) activity can be used to generate an embedding vector for a given input string with one of the Cohere Embed models. - -For Cohere Embed, the request can be associated to their respective EmbeddingsOptions extension object which can be created with the [Embeddings Options: Add Cohere Embed Extension](#add-cohere-embed-extension) operation. Through this extension, it is possible to tailor the operation to more specific needs. - -Currently, embeddings are available for the Cohere Embed family. - -#### Generate Embeddings (Chunk Collection) {#embeddings-chunk-collection} - -The [Generate Embeddings (Chunk Collection)](/agents/agents-kit-2/genai-for-mx/commons/#embeddings-chunk-collection) activity can be used to generate a collection of embedding vectors for a given collection of text chunks with one of the Cohere Embed models. - -For each model family, the request can be associated to an extension of the EmbeddingsOptions object which can be created with either the [Embeddings Options: Add Cohere Embed Extension](#add-cohere-embed-extension) operation. Through this extension, it is possible to tailor the operation to more specific needs. - -Currently, embeddings are available for the Cohere Embed family. - -#### Retrieve {#retrieve} - -The `Retrieve` activity allows you to query a knowledge base and retrieve information from it. This operation corresponds to the *Retrieve* microflow. - -The input and output for this service are shown in the table below: - -| Input | Output | -| --- | --- | -| `GenAICommons.Request (object)`| `GenAICommons.Response (object)`| - -### GenAI Commons Helper Operations - -The following operations are specific to Amazon Bedrock and helpful to configure additional settings when using operations from GenAI Commons. - -#### Add Knowledge Base Tool {#add-knowledge-base-tool} - -Use this microflow to add a new KnowledgeBaseTool object to your request. This is useful for adding additional parameters when using the [Retrieve And Generate](#retrieve-and-generate) operation. - -This operation corresponds to the **Request_AddKnowledgeBaseTool** microflow. - -| Input | Output | -| --- | --- | -| `GenAICommons.Request (object)`, `KnowledgeBaseId (string)` | *none* | - -#### Configure Bedrock Retrieve and Generate (add Knowledge Base) {#add-rag-extension} - -Use this microflow to add a new RetrieveAndGenerateRequest_Extension object to your request. This is required in order to use the [Retrieve And Generate](#retrieve-and-generate) operation successfully. - -This operation corresponds to the **RetrieveAndGenerateRequest_Extension_Create** microflow. - -| Input | Output | -| --- | --- | -| `GenAICommons.Request (object)`, `KnowledgeBaseID (string)`, `KmsKeyARN (string)`, `SessionId (string)`, `Enum_RetrieveAndGenerateType (enumeration)`, `PromptTemplate (string)` | `RetrieveAndGenerateRequest_Extension (object)` | - -`KmsKeyARN`, `SessionId`, `PromptTemplate` and `Enum_RetrieveAndGenerateType` can be empty, in which case they are not sent to the Bedrock API. - -#### Image Generation: Add Titan Image Extension {#add-titan-image-extension} - -{{% alert color="info" %}} -This microflow was introduced in Amazon Bedrock Connector version 3.1.0. -{{% /alert %}} - -Use this microflow to add a new TitanImageOptions_Extension object to your GenAICommons.ImageOptions object. This will allow you to configure the **NegativeText** attribute. - -This operation corresponds to the **TitanImageOptions_Extension_Create** microflow. - -| Input | Output | -| --- | --- | -| `GenAICommons.ImageOptions (object)`, `NegativeText (string)` | `TitanImageOptions_Extension (object)` | - -#### Image Generation: Set Image Size (Titan Image) {#set-titan-image-size} - -{{% alert color="info" %}} -This microflow was introduced in Amazon Bedrock Connector version 3.1.0. -{{% /alert %}} - -Use this microflow to set the **Height** and **Width** attributes of your **GenAICommons.ImageOptions** object to any valid image size supported by Titan Image models. The `ENUM_ImageSize_TitanImage` enumeration contains all valid height-width combinations to choose from. - -This operation corresponds to the **ImageOptions_SetImageSize_TitanImage** microflow. - -| Input | Output | -| --- | --- | -| `GenAICommons.ImageOptions (object)`, `ENUM_ImageSize_TitanImage (enumeration)` | `none` | - -#### Image Generation: Set Randomness {#set-randomness} - -{{% alert color="info" %}} -This microflow was introduced in Amazon Bedrock Connector version 3.1.0. -{{% /alert %}} - -Use this microflow to set the **Seed** and **CfgScale** attributes of your GenAICommons.ImageOptions object. These attributes can be used to influence the randomness of the image generation. - -For more information, please refer to the specific model documentation such as [Titan Image Generator G1](https://docs.aws.amazon.com/bedrock/latest/userguide/model-parameters-titan-image.html). - -This operation corresponds to the **ImageOptions_SetRandomness** microflow. - -| Input | Output | -| --- | --- | -| `GenAICommons.ImageOptions (object)`, `Seed (integer)`, `CfgScale (decimal)` | `none` | - -`Seed` and `GfgScale` can be empty, in which case they are not sent to the Bedrock API. - -#### Embeddings Options: Add Cohere Embed Extension {#add-cohere-embed-extension} - -Use this microflow to add a new CohereEmbedOptions_Extension object to your `EmbeddingsOptions` object. You can use it to include parameters that are unique to Cohere Embed models. - -This operation corresponds to the **CohereEmbedOptions_Extension_Create** microflow. - -| Input | Output | -| --- | --- | -| `GenAICommons.EmbeddingsOptions (object)`, `InputType (enumeration)`, `EmbeddingTypes (enumeration, optional)`, `Truncate (enumeration, optional)` | `CohereEmbedOptions_Extension (object)`| - -#### Set Bedrock Retrieve Options {#add-r-extension} - -Use this microflow to add a new RetrieveRequest_Extension object to your request. This is required in order to use the [Retrieve](#retrieve) activity. - -To use this activity, you must set up a knowledge base in your Amazon Bedrock Environment. For more information, see [Knowledge Base](#knowledge-base). - -The input and output for this service are shown in the table below: - -| Input | Output | -| --- | --- | -| `Request (object)`, `KnowledgeBaseID (string)`, `NumberOfResults (integer)`, `NextToken (string)` | `RetrieveRequest_Extension (object)` | - -#### Request: Add Additional Request Parameter {#add-request-parameter} - -Use this microflow to add an additional model-specific request parameter to your request. Please follow this link to find available additional request parameters: [Inference parameters](https://docs.aws.amazon.com/bedrock/latest/userguide/inference-parameters.html) - -This operation corresponds to the **Request_CreateAdditionalRequestParameter** microflow. - -| Input | Output | -| --- | --- | -| `GenAICommons.Request (object)`, `Key (string)`, `StringValue (string)`, `DecimalValue (decimal)`, `IntegerValue (integer)` | `none` | - -You need to provide a value using either the *StringValue*, *DecimalValue* or *IntegerValue* parameters. For example, if you providing a *StringValue* as value of the parameter, *DecimalValue* and *IntegerValue* should be left **empty**. - -#### Request: Add Additional Response Field {#add-response-field} - -Some models can return additional information that is not part of the `GenAICommons.Response` entity. Use this microflow to add an additional model-specific response field to your request. - -You can retrieve the additional requested response fields using the [Response: Get Requested Response Fields](#get-response-fields) operation. - -This operation corresponds to the **Request_CreateResponseFieldRequest** microflow. - -| Input | Output | -| --- | --- | -| `GenAICommons.Request (object)`, `FieldName (string)`| `none` | - -If the used model supports that response field, it will be returned as a ChatCompletionsResponse object as part of the response. - -#### Response: Get Requested Response Fields {#get-response-fields} - -Use this microflow to retrieve all requested model-specific response fields from the response. - -Some models can return additional information that is not part of the `GenAICommons.Response` entity. You can request additional request parameters using the [Request: Add Additional Response Fields](#add-response-field) operation. - -This operation corresponds to the **Response_GetRequestedResponseFields** microflow. - -| Input | Output | -| --- | --- | -| `GenAICommons.Response (object)`| `RequestedResponseField (list)` | - -#### Response: Get NextToken {#get-next-token} - -Use this microflow to retrieve the NextToken from the response after using the `Retrieve` operation. - -This operation corresponds to the **Response_GetNextToken** microflow. - -| Input | Output | -| --- | --- | -| `GenAICommons.Response (object)`| `NextToken (string)` | - -#### Response: Cast RetrieveAndGenerateResponse {#cast-rag-response} - -Use this microflow to get the RetrieveAndGenerateResponse object from the GenAiCommons.Response that is returned by the `RetrieveAndGenerate` operation. - -The RetrieveAndGenerateResponse object contains the SessionID of the current Session that can be used in a subsequent request to chat within the same session. - -This operation corresponds to the **Response_Cast_RetrieveAndGenerateResponse** microflow. - -| Input | Output | -| --- | --- | -| `GenAICommons.Response (object)`| `RetrieveAndGenerateResponse (object)` | - -### Other Operations - -#### ListFoundationModels {#list-foundation-models} - -The `ListFoundationModels` activity allows you to get all the available foundational models which Amazon Bedrock provides. It requires `ENUM_Region`, `Credentials` and `ListFoundationModelsRequest` as input parameters. - -The input and output for this service are shown in the table below: - -| Input | Output | -| --- | --- | -| `ENUM_Region (enumeration)`, `Credentials (object)`, `ListFoundationModelsRequest (object)` | `ListFoundationModelsResponse (object)`| - -#### InvokeModel {#invoke-model} - -The `InvokeModel` activity allows you to invoke a model from Amazon Bedrock. This activity provides the generic parts that are equal for the invocation of every model. It requires `ENUM_Region`, `Credentials` and `InvokeModelRequest` as input parameters. - -The `InvokeModel` operation provides a versatile interface for integrating with Amazon Bedrock models. Each available model in Amazon Bedrock has its own set of model-specific parameters required to be passed into the `InvokeModelRequest`. The Amazon Bedrock Connector contains two example implementations to showcase how to use the `InvokeModel` operation to invoke specific models. - -The input and output for this service are shown in the table below: - -| Input | Output | -| --- | --- | -| `ENUM_Region (enumeration)`, `Credentials (object)`, `InvokeModelRequest (object)` | `InvokeModelResponse (object)` | - -#### ListKnowledgeBases {#list-knowledge-bases} - -The `ListKnowledgeBases` activity allows you to list the knowledge bases in an account and get information about each of them. It requires `ENUM_Region`, `Credentials`, and `ListKnowledgeBasesRequest` as input parameters. - -To use this activity, you must set up a knowledge base in your Amazon Bedrock Environment. For more information, see [Knowledge Base](#knowledge-base). - -The input and output for this service are shown in the table below: - -| Input | Output | -| --- | --- | -| `ENUM_Region (enumeration)`, `Credentials (object)`, `ListKnowledgeBasesRequest (object)` | `ListKnowledgeBasesResponse (object)` | - -#### StartIngestionJob {#start-ingestion-job} - -The `StartIngestionJob` activity allows you to begin an ingestion job, in which the contents of the data source S3 bucket is preprocessed and synced with the vector database of the knowledge base. It requires `ENUM_Region`, `Credentials` and `StartIngestionJobRequest` as input parameters. - -To use this activity, you must set up a knowledge base in your Amazon Bedrock Environment. For more information, see [Knowledge Base](#knowledge-base). - -The input and output for this service are shown in the table below: - -| Input | Output | -| --- | --- | -| `ENUM_Region (enumeration)`, `Credentials (object)`, `StartIngestionJobRequest (object)` | `StartIngestionJobResponse (object)` | - -#### GetIngestionJob {#get-ingestion-job} - -The `GetIngestionJob` activity allows you to retrieve information about a ingestion job, in which the contents of the data source S3 bucket is preprocessed and synced with the vector database of the knowledge base. It requires `ENUM_Region`, `Credentials` and `GetIngestionJobRequest` as input parameters. - -To use this activity, you must set up a knowledge base in your Amazon Bedrock Environment. For more information, see [Knowledge Base](#knowledge-base). - -The input and output for this service are shown in the table below: - -| Input | Output | -| --- | --- | -| `ENUM_Region (enumeration)`, `Credentials (object)`, `GetIngestionJobRequest (object)` | `GetIngestionJobResponse (object)` | - -#### ListIngestionJobs {#list-ingestion-jobs} - -The `ListIngestionJobs` action allows you to retrieve a list of the data ingestion jobs for a data source. It requires `ENUM_Region`, `Credentials` and `ListIngestionJobsRequest` as input parameters. - -To use this activity, you must set up a knowledge base and a data source in your Amazon Bedrock Environment. - -The input and output for this service are shown in the table below: - -| Input | Output | -| --- | --- | -| `ENUM_Region (enumeration)`, `Credentials (object)`, `ListIngestionJobsRequest (object)` | `ListIngestionJobsResponse (object)` | - -#### CreateDataSource {#create-data-source} - -The `CreateDataSource` action allows you to connect a knowledge base to a data source. This action is currently supported for datasource types S3 and Confluence. It requires `ENUM_Region`, `Credentials` and `CreateDataSourceRequest` as input parameters. - -To use this activity, you must set up a knowledge base in your Amazon Bedrock Environment. - -The input and output for this service are shown in the table below: - -| Input | Output | -| --- | --- | -| `ENUM_Region (enumeration)`, `Credentials (object)`, `CreateDataSourceRequest (object)` | `CreateDataSourceResponse (object)` | - -#### DeleteDataSource {#delete-data-source} - -The `DeleteDataSource` action allows you to delete a data source from a knowledge base. It requires `ENUM_Region`, `Credentials` and `DeleteDataSourceRequest` as input parameters. - -To use this activity, you must set up a knowledge base and a datasource in your Amazon Bedrock Environment. - -The input and output for this service are shown in the table below: - -| Input | Output | -| --- | --- | -| `ENUM_Region (enumeration)`, `Credentials (object)`, `DeleteDataSourceRequest (object)` | `DeleteDataSourceResponse (object)` | - -#### GetDataSource {#get-data-source} - -The `GetDataSource` action allows you to get information about a data source. It requires `ENUM_Region`, `Credentials` and `GetDataSourceRequest` as input parameters. - -To use this activity, you must set up a knowledge base and a datasource in your Amazon Bedrock Environment. - -The input and output for this service are shown in the table below: - -| Input | Output | -| --- | --- | -| `ENUM_Region (enumeration)`, `Credentials (object)`, `GetDataSourceRequest (object)` | `GetDataSourceResponse (object)` | - -#### ListDataSources {#list-data-sources} - -The `ListDataSources` action allows you to list the data sources in a knowledge base and get information about each of them. It requires `ENUM_Region`, `Credentials` and `ListDataSourcesRequest` as input parameters. - -To use this activity, you must set up a knowledge base in your Amazon Bedrock Environment. - -The input and output for this service are shown in the table below: - -| Input | Output | -| --- | --- | -| `ENUM_Region (enumeration)`, `Credentials (object)`, `ListDataSourcesRequest (object)` | `ListDataSourcesResponse (object)` | - -#### InvokeAgent {#invoke-agent} - -The `InvokeAgent` activity allows you to invoke an agent from Amazon Bedrock, so that you can orchestrate tasks involving foundation models and enrich the process with organizational data and user input. It requires `ENUM_Region`, `Credentials`, `InvokeAgentRequest`, a `ResponseHandlerMicroflow` and a `ErrorHandlerMicroflow` as input parameters. The microflow parameters are necessary since `InvokeAgent` is an asynchronous operation. The `ResponseHandlerMicroflow` is required to have exactly one input parameter of the `InvokeAgentResponse` entity type. It is called in a background threat once the response is available. The `ErrorHandlerMicroflow` is required to have exactly one input parameter of type String. It will be called when there is an error during the asynchronous process and the error type will be passed to it's string parameter. The Amazon Bedrock Connector includes sample response handler and error handler microflows to help you set up handlers for your implementation. - -For more information, see [Agents for Bedrock](https://docs.aws.amazon.com/bedrock/latest/userguide/agents.html) - -The input and output for this service are shown in the table below: - -| Input | Output | -| --- | --- | -| `ENUM_Region (enumeration)`, `Credentials (object)`, `InvokeModelRequest (object)`, `ResponseHandlerMicroflow (microflow)`, `ErrorHandlerMicroflow (microflow)` | `none` | - -#### Handling the Asynchronous InvokeAgentResponse - -The `InvokeAgentResponse` object is passed as a parameter to the ResponseHandler microflow. This microflow can perform any custom logic with the `InvokeAgentResponse`, for example storing it in the database. The microflow is called in another background thread, so the client is not automatically notified when the response is processed. If you want to display the agent's response to the user of your app, you can use one of the following methods: - -##### Polling - -The easiest way to make sure the client gets a response is to constantly poll for it until it is available. This can be done using the [Microflow Timer Widget](https://marketplace.mendix.com/link/component/27), which allows you to configure a microflow or nanoflow to run every X number of seconds. - -This approach is only recommended for testing and for applications that do not have a large number of concurrent users. It is not preferred for scaling. - -##### Websockets - -WebSockets is a communication protocol that provides full-duplex communication channels over a single, persistent connection. Unlike traditional HTTP connections, which are request-response based and stateless, WebSockets enable real-time, bi-directional communication between a client (such as a Web browser) and a server. - -The open source [EZ Websocket Module](https://marketplace.mendix.com/link/component/205276) from the Mendix Marketplace provides an easy way to implement real-time server-to-client communication using WebSockets without external dependencies. - -##### Pusher - -The platform-supported [Pusher Module](https://marketplace.mendix.com/link/component/107957) is built around the [Pusher Channels](https://pusher.com/channels/) offering. This module requires a Pusher account. Pusher Channels is a paid service, but it also has a [Free Sandbox Plan](https://pusher.com/channels/pricing/). This module allows you to trigger a Notify event on the server to immediately trigger an action in the client application. - -### Working with Action Groups and Lambda Functions - -Without action groups, the agent will still access associated knowledge bases, but will not be able to perform tasks that make agents an extension of simply invoking a model. Action groups are what make agents so powerful. - -For example, it might be beneficial for the agent to dynamically retrieve more information via a REST endpoint or other source, rather than storing all possible information in a knowledge base. To achieve this, a lambda function must first be specified for the REST request and then associated with the agent as part of an action group. - -If you would like to add lambda functions to your agent, please refer to the [AWS documentation](https://docs.aws.amazon.com/bedrock/latest/userguide/agents-lambda.html). - -#### ListAgents {#list-agents} - -The `ListAgents` activity allows you to list the agents in an account and get information about each of them. It requires `ENUM_Region`, `Credentials`, and `ListAgentsRequest` as input parameters. - -To use this activity, you must set up an agent in your Amazon Bedrock environment. - -The input and output for this service are shown in the table below: - -| Input | Output | -| --- | --- | -| `ENUM_Region (enumeration)`, `Credentials (object)`, `ListAgentsRequest (object)` | `ListAgentsResponse (object)` | - -#### GetAgent {#get-agent} - -The `GetAgent` activity allows you to retrieve information about an agent. It requires `ENUM_Region`, `Credentials`, and `GetAgentRequest` as input parameters. - -To use this activity, you must set up an agent in your Amazon Bedrock Environment. - -The input and output for this service are shown in the table below: - -| Input | Output | -| --- | --- | -| `ENUM_Region (enumeration)`, `Credentials (object)`, `GetAgentRequest (object)` | `GetAgentResponse (object)` | - -### Operations to Persist Amazon Bedrock Metadata inside the Application - -The Amazon Bedrock Connector offers a range of operations to retrieve and store metadata information in the Mendix app's database. - -This can be useful to, for example, associate a chatbot configuration to an available model by selecting the model via dropdown in runtime. The persistent domain model allows for simple and efficient filtering capabilities on the available metadata. Further, the *SNIP_Settings_Admin_BedrockConfig* Snippet can be used to manage and view the synced data from an administrator perspective. - -The syncing process works the same for all of these operations: - -1. Information about models, knowledge bases, and agents is persistent in the Mendix app's database on the initial sync. -2. For knowledge base and agents, an association to the `AmazonBedrockRegion` object, that represents the AWS region used when syncing, is stored. -3. On a subsequent syncing process, the available data is extended and updated. No data is removed from the app's database, even if it is no longer available on AWS. This is because existing usages of the object in the running application should not be removed. - -The available operations are described in the following sections. - -#### Sync Models {#sync-models} - -The `Sync Models` activity lets you retrieve and store metadata about available models on Amazon Bedrock in your app's database. The model information is persistent in the `BedrockDeployedModel` entity which is mandatory for chat completions or embeddings operations from GenAI Commons. - -Information about the model's input modalities are stored as associations to the `InputModality` entity and its output modality is stored to the `OutputModality` attribute. The input modality describes which form of data can be sent to the model. The output modality describes which form of data the model will return. - -Information about the model's inference type is stored as association to the `ModelInferenceType` entity. The inference type describes how the model can be accessed. *ON Demand* models are accessible by default and charged by usage. - -The input and output for this service are shown in the table below: - -| Input | Output | -| --- | --- | -| `ENUM_Region (enumeration)`, `Credentials (object)` | `Count (integer)` | - -The operation returns an integer that indicates how many objects were created or changed during the syncing process. - -#### Sync Knowledge Bases {#sync-knowledge-bases} - -The `Sync Knowledge Bases` activity allows you to retrieve and store metadata about available knowledge bases on Amazon Bedrock in your app's database. -The knowledge base information is persistent in the `AmazonBedrockKnowledgeBase` entity. - -The input and output for this service are shown in the table below: - -| Input | Output | -| --- | --- | -| `ENUM_Region (enumeration)`, `Credentials (object)` | `Count (integer)` | - -The operation returns an integer that indicates how many objects were created or changed during the syncing process. - -#### Sync Agents {#sync-agents} - -The `Sync Agents` activity allows you to retrieve and store metadata about available agents on Amazon Bedrock in your app's database. -The agent information is persistent in the `AmazonBedrockAgent` entity. - -The input and output for this service are shown in the table below: - -| Input | Output | -| --- | --- | -| `ENUM_Region (enumeration)`, `Credentials (object)` | `Count (integer)` | - -The operation returns an integer that indicates how many objects were created or changed during the syncing process. - -### Knowledge Base {#knowledge-base} - -In Bedrock, a *knowledge base* denotes a substantial storehouse of data and information. This serves as a foundational resource, enabling the AI system to glean insights and effectively comprehend and respond to natural language queries. - -For more information about knowledge bases, see [Knowledge Base](https://docs.aws.amazon.com/bedrock/latest/userguide/knowledge-base.html) in the Amazon Bedrock documentation. - -#### Creating a Knowledge Base - -Setting up knowledge bases is usually a one-time configuration, which can be done with the AWS Console. In order to get the best results, you should consider whether you want to use one of the chunking strategies available on AWS when creating the knowledge base, or whether you want to pre-process the data beforehand. - -*Chunking* is the practice of breaking large chunks of data into smaller segments. Chunking the data allows the embedding algorithm to process the given data in chunks, thus increasing efficiency. Chunking can also introduce a structure that helps the model understand which data belongs to the same context. You can use the default chunking strategy, or create a custom strategy if there is a specific way in which the model data should be split. - -For example, when building a chatbot that gives restaurant recommendations, you should set up the knowledge base with a list of restaurant reviews. In this case, using the default chunking into 300 tokens might result in chunks containing reviews for different restaurants, which is not optimal. You will likely have better results if each chunk corresponds to reviews for one restaurant, as with that strategy it is less likely that the model will then associate a review with the wrong restaurant. You can achieve the required results by pre-processing the data so that there is one file per restaurant, and using the **No chunking** option when setting up the knowledge base. - -For more information about creating the knowledge base, including a list of the available chunking strategies, see [Create a knowledge base](https://docs.aws.amazon.com/bedrock/latest/userguide/knowledge-base-create.html). - -#### Adding Data from Your App - -After a knowledge base has been set up, information from your app can be added in a file to the relevant S3 bucket, and then used during subsequent inquiries. Which information is used and how that information is exported depends on the customer's use case and is up to the Mendix developer to implement. For more information, see [Set up your data for ingestion](https://docs.aws.amazon.com/bedrock/latest/userguide/knowledge-base-setup.html). - -Amazon Bedrock only processes the information that existed during the last sync, so the data source must be synchronized whenever a new file is added to your S3 bucket or the existing files are changed. For more information, see [Sync to ingest your data sources into the knowledge base](https://docs.aws.amazon.com/bedrock/latest/userguide/knowledge-base-ingest.html). - -The sync can be done from the information page of your knowledge base in the Amazon Bedrock Console, or by using the **StartIngestionJob** action in the Amazon Bedrock Connector. - -{{% alert color="info" %}} -The sync can take up to a few minutes and the calls to your knowledge base during this process cannot be handled accurately. To make sure the sync process has ended, you can use the **GetIngestionJob** action in the Amazon Bedrock Connector to retrieve the status of the ingestion job, along with other details. -{{% /alert %}} - -### Safeguards - -AWS has introduced safeguards for Bedrock (currently in preview). When available, there will be two features: Guardrails and Watermark detection. - -The guardrail feature will allow you to: - -* Filter harmful content with configurable thresholds based on your responsible AI policies. -* Determine how to handle personally identifiable information (PII). -* Deny topics. - -The watermark detection feature will make it possible to tell if an image has been created using Amazon Titan. - -More information about guardrails can be found in this [AWS blogpost](https://aws.amazon.com/blogs/aws/guardrails-for-amazon-bedrock-helps-implement-safeguards-customized-to-your-use-cases-and-responsible-ai-policies-preview/) and in the [AWS documentation](https://aws.amazon.com/en/bedrock/guardrails/). - -### Advanced Prompts for Agents - -By default, an agent is configured with the following base prompt templates, one for each step in the agent sequence: - -* Pre-processing -* Orchestration -* Knowledge base response generation -* Post-processing - -By customizing the prompt templates and modifying these configurations, you can fine-tune your agent's accuracy. Additionally, you can provide custom examples for a technique known as few-shot prompting. This involves providing labeled examples for specific tasks, which further enhances the model's performance in targeted areas. For more information about advanced prompts, see [Advanced prompts](https://docs.aws.amazon.com/bedrock/latest/userguide/advanced-prompts.html) in the AWS documentation. - -You can also use placeholder variables in agent prompt templates. For example, in the orchestration prompt template, the *$prompt_session_attributes$* placeholder variable can be used to ingest the information from the `PromptSessionAttribute` entity into the prompt, if it was specified as part of the `InvokeAgentRequest`. For more information about placeholder variables available in agent prompt templates, see [Prompt placeholders](https://docs.aws.amazon.com/bedrock/latest/userguide/prompt-placeholders.html) in the AWS documentation. - -## Troubleshooting - -If you encounter any issues while using the Amazon Bedrock connector, use the following troubleshooting tips to help you solve them. - -### Error Code 400 - Bad Request - -The service returns the error code *400 - Bad Request*. - -#### Cause - -Your AWS organization may not have been granted access to the model which you are trying to invoke. - -#### Solution - -To solve this issue, follow these steps: - -1. In your Amazon Bedrock environment, navigate to **Model Access** for the region in which you would like to work. -2. If the status of a model is **Available**, enable access to this model for your AWS organization by doing the following steps: - 1. In the top-right corner of the overview, click on **Edit**. - 2. Select the check boxes by the models which you want to access with your credential set. - 3. Click **Save Changes**. - -After the status of the models changes to **Access Granted**, you can use it with the Amazon Bedrock connector. - -### Error Code 403 - AccessDeniedException - -When invoking a model, the error code *403 - Access denied* indicates that you do not have access to the targeted resource. - -#### Cause - -Possible root causes for this error include the following: - -* You do not have access to the model in the specified AWS region. - -#### Solution - -To solve this issue, ensure that you have selected an AWS Region where you have model access. You can see an overview of the models accessible to you in the AWS Management Console, in the [Model Access](https://us-west-2.console.aws.amazon.com/bedrock/home?#/modelaccess) section of your Amazon Bedrock environment. - -### Error Code 404 - ResourceNotFoundException - -When invoking a model, the error code *404 - Resource not found* indicates that the targeted resource was not found. - -#### Cause - -Possible root causes for this error include the following: - -* The model which you are trying to invoke is not available in your specified AWS region. -* The model which you are trying to invoke is deprecated. - -#### Solution - -To solve this issue, verify the following: - -1. Ensure that you have selected an AWS Region where the targeted model exists. You can see an overview of the models accessible to you in the AWS Management Console, on the [Overview page](https://us-west-2.console.aws.amazon.com/bedrock/home?#/overview) of your Amazon Bedrock environment. Make sure the region specified in the AWS Console matches the region you have configured in Mendix. -2. Ensure that the model that you have selected is not deprecated and that the *model-id* is currently available in Amazon Bedrock. - -### Attribute or Reference Required Error Message After Upgrade - -If you encounter an error stating that an attribute or a reference is required after an upgrade, first upgrade all modules by right-clicking the error, then upgrade Data Widgets. - -### Conflicted Lib Error After Module Import - -If you encounter an error caused by conflicting Java libraries, such as `java.lang.NoSuchMethodError: 'com.fasterxml.jackson.annotation.OptBoolean com.fasterxml.jackson.annotation.JsonProperty.isRequired()'`, try synchronizing all dependencies (**App** > **Synchronize dependencies**) and then restart your application. +To allow your Mendix app to use Amazon Bedrock functionalities, install and configure the [Amazon Bedrock connector](/agents/reference-guide/external-connectors/bedrock/). \ No newline at end of file From 7afca38fb98b4416ae4073ec6f197a822d88e6a4 Mon Sep 17 00:00:00 2001 From: Dana Breseman Date: Mon, 22 Jun 2026 13:04:35 +0200 Subject: [PATCH 5/5] Update cross-links --- content/en/docs/genai/_index.md | 2 +- content/en/docs/genai/concepts/_index.md | 4 ++-- content/en/docs/genai/concepts/tool-calling.md | 2 +- content/en/docs/genai/v1/_index.md | 2 +- content/en/docs/genai/v1/how-to/_index.md | 2 +- .../en/docs/genai/v1/how-to/creating-agents/shared-setup.md | 2 +- .../en/docs/genai/v1/how-to/integrate_function_calling.md | 2 +- .../en/docs/genai/v1/how-to/prompt_engineering-runtime.md | 2 +- content/en/docs/genai/v1/how-to/start_from_a_starter_app.md | 2 +- content/en/docs/genai/v1/how-to/start_from_blank_app.md | 4 ++-- content/en/docs/genai/v1/reference-guide/agent-commons.md | 4 ++-- .../en/docs/genai/v1/reference-guide/conversational-ui.md | 6 +++--- content/en/docs/genai/v1/reference-guide/genai-commons.md | 2 +- content/en/docs/genai/v2/_index.md | 2 +- content/en/docs/genai/v2/how-to/_index.md | 2 +- .../en/docs/genai/v2/how-to/creating-agents/shared-setup.md | 2 +- .../en/docs/genai/v2/how-to/integrate_function_calling.md | 2 +- .../en/docs/genai/v2/how-to/prompt_engineering-runtime.md | 2 +- content/en/docs/genai/v2/how-to/start_from_a_starter_app.md | 2 +- content/en/docs/genai/v2/how-to/start_from_blank_app.md | 4 ++-- content/en/docs/genai/v2/reference-guide/agent-commons.md | 4 ++-- .../en/docs/genai/v2/reference-guide/conversational-ui.md | 6 +++--- content/en/docs/genai/v2/reference-guide/genai-commons.md | 2 +- 23 files changed, 32 insertions(+), 32 deletions(-) diff --git a/content/en/docs/genai/_index.md b/content/en/docs/genai/_index.md index 10b5ff1022f..22872c7e4cb 100644 --- a/content/en/docs/genai/_index.md +++ b/content/en/docs/genai/_index.md @@ -92,6 +92,6 @@ For a list of all Bedrock Models, see [Models at a glance](https://docs.aws.amaz In addition to the models listed above, you can also connect to other models by implementing one of the following options: -* To connect to other [foundation models](https://docs.aws.amazon.com/bedrock/latest/userguide/models-features.html) and implement them in your app, use the [Amazon Bedrock connector](/appstore/modules/aws/amazon-bedrock/). +* To connect to other [foundation models](https://docs.aws.amazon.com/bedrock/latest/userguide/models-features.html) and implement them in your app, use the [Amazon Bedrock connector](/agents/reference-guide/external-connectors/bedrock/). * To connect to [Snowflake Cortex LLM](https://docs.snowflake.com/en/sql-reference/functions/complete-snowflake-cortex) functions, [configure the Snowflake AI Data Connector for Snowflake Cortex Analyst](/appstore/connectors/snowflake/snowflake-ai-data-connector/#cortex-analyst). * To implement your own connector that is compatible with the other components, use the [GenAI Commons](/agents/agents-kit-2/genai-for-mx/commons/) interface and see [How to Build Your Own GenAI Connector](/agents/agents-kit-2/how-to/byo-connector/). diff --git a/content/en/docs/genai/concepts/_index.md b/content/en/docs/genai/concepts/_index.md index afb1b748d9e..f709e274e06 100644 --- a/content/en/docs/genai/concepts/_index.md +++ b/content/en/docs/genai/concepts/_index.md @@ -42,7 +42,7 @@ For example, you can use an LLM to do: * Translate languages * Simulate characters for games -Some LLMs, such as [Anthropic Claude](/appstore/modules/aws/amazon-bedrock/) and [GPT-4o](/agents/agents-kit-2/reference-guide/external-connectors/openai/), can also use one or more images as input, allowing you to ask questions about images for use cases such as object recognition, image to text (OCR), and validating whether an image is as intended. +Some LLMs, such as [Anthropic Claude](/agents/reference-guide/external-connectors/bedrock/) and [GPT-4o](/agents/agents-kit-2/reference-guide/external-connectors/openai/), can also use one or more images as input, allowing you to ask questions about images for use cases such as object recognition, image to text (OCR), and validating whether an image is as intended. #### Embeddings Generation @@ -145,4 +145,4 @@ The agent concept combines prompts, RAG (Retrieval Augmented Generation), and Re For an overview of the components that help you get started, refer to [Agents Kit Components](/agents/agents-kit-2/#components). -In addition, you can integrate agentic behavior in a Mendix app by leveraging external agents through cloud infrastructure providers. In this case, the Mendix app does not store the agent definition. Instead, it only calls the external agent. For example, [Agents for Amazon Bedrock](https://aws.amazon.com/bedrock/agents/) provides this functionality for Amazon Bedrock. You can find out how to use this in your Mendix application in [Invoking an Agent with the InvokeAgent Operation](/appstore/modules/aws/amazon-bedrock/#invokeagent) section of the *Amazon Bedrock* module documentation. +In addition, you can integrate agentic behavior in a Mendix app by leveraging external agents through cloud infrastructure providers. In this case, the Mendix app does not store the agent definition. Instead, it only calls the external agent. For example, [Agents for Amazon Bedrock](https://aws.amazon.com/bedrock/agents/) provides this functionality for Amazon Bedrock. You can find out how to use this in your Mendix application in [Invoking an Agent with the InvokeAgent Operation](/agents/reference-guide/external-connectors/bedrock/#invokeagent) section of the *Amazon Bedrock* module documentation. diff --git a/content/en/docs/genai/concepts/tool-calling.md b/content/en/docs/genai/concepts/tool-calling.md index 4552719808b..d39e9c955e7 100644 --- a/content/en/docs/genai/concepts/tool-calling.md +++ b/content/en/docs/genai/concepts/tool-calling.md @@ -37,7 +37,7 @@ Sometimes, tool calls should not be executed immediately, and should first requi ## Tool Calling with the GenAI Commons Module and the LLM Connectors {#llm-connector} -All platform-supported connectors ([Mendix Cloud GenAI](/agents/agents-kit-2/mx-cloud-genai/mxgenai-connector/), [OpenAI](/agents/agents-kit-2/reference-guide/external-connectors/openai/), and [Amazon Bedrock Connector](/appstore/modules/aws/amazon-bedrock/)) support tool calling by leveraging the [GenAI Commons module](/agents/agents-kit-2/genai-for-mx/commons/). Tool calling is supported for all chat completions operations. All entity, attribute, and activity names in this section refer to the GenAI Commons module. +All platform-supported connectors ([Mendix Cloud GenAI](/agents/agents-kit-2/mx-cloud-genai/mxgenai-connector/), [OpenAI](/agents/agents-kit-2/reference-guide/external-connectors/openai/), and [Amazon Bedrock Connector](/agents/reference-guide/external-connectors/bedrock/)) support tool calling by leveraging the [GenAI Commons module](/agents/agents-kit-2/genai-for-mx/commons/). Tool calling is supported for all chat completions operations. All entity, attribute, and activity names in this section refer to the GenAI Commons module. Functions in Mendix are essentially microflows that can be registered within the request to the LLM​. The LLM connector takes care of handling the tool call response as well as executing the function microflows until the LLM returns the final assistant's response. Function microflows can have none, a single, or multiple primitive input parameters such as Boolean, Datetime, Decimal, Enumeration, Integer or String. Additionally, they may accept the [Request](/agents/agents-kit-2/genai-for-mx/commons/#request) or [Tool](/agents/agents-kit-2/genai-for-mx/commons/#tool) objects as inputs. The microflow can only return a String value. diff --git a/content/en/docs/genai/v1/_index.md b/content/en/docs/genai/v1/_index.md index 1a0f1cfa99f..fedb78f2925 100644 --- a/content/en/docs/genai/v1/_index.md +++ b/content/en/docs/genai/v1/_index.md @@ -56,7 +56,7 @@ All connectors depend on GenAI Commons and can be used with the other [core modu | Asset | Description | Release Version | | --- | --- | --- | -| [Amazon Bedrock Connector](/appstore/modules/aws/amazon-bedrock/) | Connect to Amazon Bedrock. | 9.0.0 | +| [Amazon Bedrock Connector](/agents/agents-kit-1/reference-guide/external-connectors/bedrock/) | Connect to Amazon Bedrock. | 9.0.0 | | [Google Gemini Connector](/agents/agents-kit-1/reference-guide/external-connectors/gemini/) | Connect to Google Gemini. Available in Studio Pro 10.24.13 and above. | 1.0.0 | | [Mendix Cloud GenAI Connector](/agents/agents-kit-1/mx-cloud-genai/mxgenai-connector/) | Connect to Mendix Cloud and use Mendix Cloud GenAI resource packs directly within your Mendix application. | 5.0.0 | | [Mistral Connector](/agents/agents-kit-1/reference-guide/external-connectors/mistral/) | Connect to Mistral AI. | 1.0.0 | diff --git a/content/en/docs/genai/v1/how-to/_index.md b/content/en/docs/genai/v1/how-to/_index.md index 0c63a240736..42908b2203a 100644 --- a/content/en/docs/genai/v1/how-to/_index.md +++ b/content/en/docs/genai/v1/how-to/_index.md @@ -40,7 +40,7 @@ Generative Artificial Intelligence (GenAI) transforms business applications, emp * The [ConversationalUI](/agents/agents-kit-1/genai-for-mx/conversational-ui/) module that offers UI snippets for chat, token consumption monitoring and prompt management. * The [Mendix Cloud GenAI Resource Packs](/agents/mx-cloud-genai/resource-packs/) to learn how to quickly access GenAI capabilities from a Mendix app. * The [OpenAI](/agents/agents-kit-1/reference-guide/external-connectors/openai/) provides essential information about the OpenAI connector. -* The [Amazon Bedrock](/appstore/modules/aws/amazon-bedrock/) provides key information about the AWS Bedrock connector. +* The [Amazon Bedrock](/agents/agents-kit-1/reference-guide/external-connectors/bedrock/) provides key information about the AWS Bedrock connector. * The [MCP Server Module](/agents/agents-kit-1/mcp-modules/mcp-server/) provides reusable operations to create and initialize an MCP server within a Mendix app to expose tools and prompts to external clients. * The [PGVector Knowledge Base](/agents/agents-kit-1/reference-guide/external-connectors/pgvector/) offers the option for a private knowledge base outside of the LLM infrastructure. diff --git a/content/en/docs/genai/v1/how-to/creating-agents/shared-setup.md b/content/en/docs/genai/v1/how-to/creating-agents/shared-setup.md index 2f29cf0588a..3f3917ae520 100644 --- a/content/en/docs/genai/v1/how-to/creating-agents/shared-setup.md +++ b/content/en/docs/genai/v1/how-to/creating-agents/shared-setup.md @@ -39,7 +39,7 @@ Before you build an agent in your app, make sure your scenario meets the followi ## Setting Up Your Application {{% alert color="info" %}} -This guide uses the Mendix Cloud GenAI Connector for text generation. You can also use alternative [supported connectors](/agents/agents-kit-1/#connectors), such as [Amazon Bedrock](/appstore/modules/aws/amazon-bedrock/) or [OpenAI](/agents/agents-kit-1/reference-guide/external-connectors/openai/). For knowledge base operations, this guide uses the Mendix Cloud Knowledge Base, but the [pgVector Knowledge Base](/agents/agents-kit-1/reference-guide/external-connectors/pgvector/) is also supported. As long as you configure access to a provider and knowledge base according to the connector documentation, and the knowledge base supports inserting chunks from a microflow, the remaining steps in this guide apply. +This guide uses the Mendix Cloud GenAI Connector for text generation. You can also use alternative [supported connectors](/agents/agents-kit-1/#connectors), such as [Amazon Bedrock](/agents/agents-kit-1/reference-guide/external-connectors/bedrock/) or [OpenAI](/agents/agents-kit-1/reference-guide/external-connectors/openai/). For knowledge base operations, this guide uses the Mendix Cloud Knowledge Base, but the [pgVector Knowledge Base](/agents/agents-kit-1/reference-guide/external-connectors/pgvector/) is also supported. As long as you configure access to a provider and knowledge base according to the connector documentation, and the knowledge base supports inserting chunks from a microflow, the remaining steps in this guide apply. {{% /alert %}} If you are using a GenAI starter app such as the Blank GenAI Starter App, you can skip ahead to [Creating the Agent's Functional Prerequisites](#creating-functional-prerequisites) because the following setup steps are completed by default. Otherwise, follow these steps to add the required modules and configuration to your app: diff --git a/content/en/docs/genai/v1/how-to/integrate_function_calling.md b/content/en/docs/genai/v1/how-to/integrate_function_calling.md index 5af2c4f153d..715fed8a12d 100644 --- a/content/en/docs/genai/v1/how-to/integrate_function_calling.md +++ b/content/en/docs/genai/v1/how-to/integrate_function_calling.md @@ -51,7 +51,7 @@ Selecting the infrastructure for integrating GenAI into your Mendix application * [OpenAI](/agents/agents-kit-1/reference-guide/external-connectors/openai/): The [OpenAI Connector](https://marketplace.mendix.com/link/component/220472) supports both OpenAI's platform and Microsoft Foundry. -* [Amazon Bedrock](/appstore/modules/aws/amazon-bedrock/): The [Amazon Bedrock Connector](https://marketplace.mendix.com/link/component/215042) allows you to leverage Amazon Bedrock’s fully managed service to integrate foundation models from Amazon and leading AI providers. +* [Amazon Bedrock](/agents/agents-kit-1/reference-guide/external-connectors/bedrock/): The [Amazon Bedrock Connector](https://marketplace.mendix.com/link/component/215042) allows you to leverage Amazon Bedrock’s fully managed service to integrate foundation models from Amazon and leading AI providers. * Your Own Connector: Optionally, if you prefer a custom connector, you can integrate your chosen infrastructure. However, this document focuses on the Mendix Cloud GenAI, OpenAI, and Amazon Bedrock connectors, as they offer comprehensive support and ease of use to get started. diff --git a/content/en/docs/genai/v1/how-to/prompt_engineering-runtime.md b/content/en/docs/genai/v1/how-to/prompt_engineering-runtime.md index f2fab5551c4..181c30209af 100644 --- a/content/en/docs/genai/v1/how-to/prompt_engineering-runtime.md +++ b/content/en/docs/genai/v1/how-to/prompt_engineering-runtime.md @@ -61,7 +61,7 @@ If you are using a GenAI starter app such as the Blank GenAI app, you can skip a To enable generative AI capabilities, install and configure the [Mendix Cloud GenAI Connector](/agents/agents-kit-1/mx-cloud-genai/mxgenai-connector/) and its dependencies from the Mendix Marketplace. If you are using a GenAI starter app such as the Blank GenAI app, you can skip the installation and just follow the [configuration instructions](/agents/agents-kit-1/mx-cloud-genai/mxgenai-connector/#configuration). {{% alert color="info" %}} -This example uses the Mendix Cloud GenAI Connector. Alternatively, you can install and configure an [external connector](/agents/agents-kit-1/reference-guide/connectors/) for any provider with a connector that is compatible with [GenAICommons](/agents/agents-kit-1/genai-for-mx/commons/). This includes [OpenAI](/agents/agents-kit-1/reference-guide/external-connectors/openai/) and [Amazon Bedrock](/appstore/modules/aws/amazon-bedrock/). +This example uses the Mendix Cloud GenAI Connector. Alternatively, you can install and configure an [external connector](/agents/agents-kit-1/reference-guide/connectors/) for any provider with a connector that is compatible with [GenAICommons](/agents/agents-kit-1/genai-for-mx/commons/). This includes [OpenAI](/agents/agents-kit-1/reference-guide/external-connectors/openai/) and [Amazon Bedrock](/agents/agents-kit-1/reference-guide/external-connectors/bedrock/). {{% /alert %}} ## Verifying Setup {#verification} diff --git a/content/en/docs/genai/v1/how-to/start_from_a_starter_app.md b/content/en/docs/genai/v1/how-to/start_from_a_starter_app.md index a014cf5694f..f1299723785 100644 --- a/content/en/docs/genai/v1/how-to/start_from_a_starter_app.md +++ b/content/en/docs/genai/v1/how-to/start_from_a_starter_app.md @@ -49,7 +49,7 @@ Selecting the infrastructure for integrating GenAI into your Mendix application * [OpenAI](/agents/agents-kit-1/reference-guide/external-connectors/openai/): The [OpenAI Connector](https://marketplace.mendix.com/link/component/220472) supports OpenAI's platform and Microsoft Foundry. -* [Amazon Bedrock](/appstore/modules/aws/amazon-bedrock/): The [Amazon Bedrock Connector](https://marketplace.mendix.com/link/component/215042) allows you to leverage Amazon Bedrock’s fully managed service to integrate foundation models from Amazon and leading AI providers. +* [Amazon Bedrock](/agents/agents-kit-1/reference-guide/external-connectors/bedrock/): The [Amazon Bedrock Connector](https://marketplace.mendix.com/link/component/215042) allows you to leverage Amazon Bedrock’s fully managed service to integrate foundation models from Amazon and leading AI providers. * Your Own Connector: Optionally, if you prefer a custom connector, you can integrate your chosen infrastructure. However, this document focuses on the platform-supported connectors, as they offer comprehensive support and ease of use to get started. diff --git a/content/en/docs/genai/v1/how-to/start_from_blank_app.md b/content/en/docs/genai/v1/how-to/start_from_blank_app.md index c1ef8bbcc16..c5f47e38ba9 100644 --- a/content/en/docs/genai/v1/how-to/start_from_blank_app.md +++ b/content/en/docs/genai/v1/how-to/start_from_blank_app.md @@ -58,7 +58,7 @@ Selecting the infrastructure for integrating GenAI into your Mendix application * [Mendix Cloud GenAI Resources Packs](/agents/mx-cloud-genai/resource-packs/): The [Mendix Cloud GenAI Connector](https://marketplace.mendix.com/link/component/239449) integrates LLMs by dragging and dropping common operations from its toolbox in Studio Pro. * [OpenAI](/agents/agents-kit-1/reference-guide/external-connectors/openai/): The [OpenAI Connector](https://marketplace.mendix.com/link/component/220472) supports both OpenAI's platform and Microsoft Foundry. -* [Amazon Bedrock](/appstore/modules/aws/amazon-bedrock/): The [Amazon Bedrock Connector](https://marketplace.mendix.com/link/component/215042) allows you to leverage Amazon Bedrock’s fully managed service to integrate foundation models from Amazon and leading AI providers. +* [Amazon Bedrock](/agents/agents-kit-1/reference-guide/external-connectors/bedrock/): The [Amazon Bedrock Connector](https://marketplace.mendix.com/link/component/215042) allows you to leverage Amazon Bedrock’s fully managed service to integrate foundation models from Amazon and leading AI providers. * Your Own Connector: Optionally, if you prefer a custom connector, you can integrate your chosen infrastructure. However, this document focuses on the OpenAI and Bedrock connectors, as they offer comprehensive support and ease of use to get started. @@ -177,7 +177,7 @@ Follow the steps below to configure Amazon Bedrock for your application: * Click **Save & Sync Data** to ensure your changes are applied. {{% alert color="info" %}} -If you encounter any issues while using the Amazon Bedrock connector, see the [Troubleshooting](/appstore/modules/aws/amazon-bedrock/#troubleshooting) section of the *Amazon Bedrock*. +If you encounter any issues while using the Amazon Bedrock connector, see the [Troubleshooting](/agents/agents-kit-1/reference-guide/external-connectors/bedrock/#troubleshooting) section of the *Amazon Bedrock*. {{% /alert %}} ## Testing and Troubleshooting diff --git a/content/en/docs/genai/v1/reference-guide/agent-commons.md b/content/en/docs/genai/v1/reference-guide/agent-commons.md index 2b71525dd84..7ae93ffb808 100644 --- a/content/en/docs/genai/v1/reference-guide/agent-commons.md +++ b/content/en/docs/genai/v1/reference-guide/agent-commons.md @@ -98,7 +98,7 @@ For example, download and run the [Agent Builder Starter App](https://marketplac To interact with LLMs using Agent Commons, you need at least one GenAI connector that adheres to the GenAI Commons principles. To test agent behavior, you must configure at least one [Deployed Model](/agents/agents-kit-1/genai-for-mx/commons/#deployed-model) for your chosen connector. Refer to the specific connector’s documentation for detailed instructions on setting up the Deployed Model. * For [Mendix Cloud GenAI](https://marketplace.mendix.com/link/component/239449), importing the **Key** from the Mendix portal automatically creates a MxCloud Deployed Model. This is part of the [configuration](/agents/agents-kit-1/mx-cloud-genai/mxgenai-connector/#configuration). -* For [Amazon Bedrock](https://marketplace.mendix.com/link/component/215042), the creation of Bedrock Deployed Models is part of the [model synchronization mechanism](/appstore/modules/aws/amazon-bedrock/#sync-models). +* For [Amazon Bedrock](https://marketplace.mendix.com/link/component/215042), the creation of Bedrock Deployed Models is part of the [model synchronization mechanism](/agents/agents-kit-1/reference-guide/external-connectors/bedrock/#sync-models). * For [OpenAI](https://marketplace.mendix.com/link/component/220472), the configuration of OpenAI Deployed Models is part of the [configuration](/agents/agents-kit-1/reference-guide/external-connectors/openai/#general-configuration). ### Defining the Agent {#define-agent} @@ -149,7 +149,7 @@ Besides microflow tools, tools exposed by MCP servers are also supported. To add For supported knowledge bases registered in your app, you can connect them to agents to enable autonomous retrievals. Refer to the documentation of the connector provided by your selected knowledge base provider. Follow the instructions to configure the knowledge bases in your app, so that they can be linked to your agents. Mendix provides the following platform-supported connectors that support knowledge base integrations with agents: * [Mendix Cloud GenAI Connector](/agents/agents-kit-1/mx-cloud-genai/mxgenai-connector/#configuration) -* [Amazon Bedrock Connector](/appstore/modules/aws/amazon-bedrock/#sync-models) +* [Amazon Bedrock Connector](/agents/agents-kit-1/reference-guide/external-connectors/bedrock/#sync-models) * [OpenAI Connector](/agents/agents-kit-1/reference-guide/external-connectors/openai/#azure-ai-search) * [PgVector Knowledge Base](/agents/agents-kit-1/reference-guide/external-connectors/pgvector/#general-configuration) diff --git a/content/en/docs/genai/v1/reference-guide/conversational-ui.md b/content/en/docs/genai/v1/reference-guide/conversational-ui.md index d8d5ded82d6..e57cb9f998c 100644 --- a/content/en/docs/genai/v1/reference-guide/conversational-ui.md +++ b/content/en/docs/genai/v1/reference-guide/conversational-ui.md @@ -64,7 +64,7 @@ You must also ensure you have the other prerequisite modules that Conversational * [Nanoflow Commons](https://marketplace.mendix.com/link/component/109515) * [Web Actions](https://marketplace.mendix.com/link/component/114337) -Finally, you must also set up a connector that is compatible with [GenAI Commons](/agents/agents-kit-1/genai-for-mx/commons/). One option is to use the [Mendix Cloud GenAI connector](https://marketplace.mendix.com/link/component/239449). For more information on how to configure this connector, see the [Configuration](/agents/agents-kit-1/mx-cloud-genai/mxgenai-connector/#configuration) section of *Mendix Cloud GenAI connector*. Additionally, Mendix offers platform-supported integration with [(Azure) OpenAI](/agents/agents-kit-1/reference-guide/external-connectors/openai/) and [Amazon Bedrock](/appstore/modules/aws/amazon-bedrock/). If desired, you need to download these integrations manually from the Marketplace. Alternatively, you can integrate with custom models by creating your own connector and making its operations and object structure compatible with the [GenAI Commons](/agents/agents-kit-1/genai-for-mx/commons/) `Request` and `Response`. +Finally, you must also set up a connector that is compatible with [GenAI Commons](/agents/agents-kit-1/genai-for-mx/commons/). One option is to use the [Mendix Cloud GenAI connector](https://marketplace.mendix.com/link/component/239449). For more information on how to configure this connector, see the [Configuration](/agents/agents-kit-1/mx-cloud-genai/mxgenai-connector/#configuration) section of *Mendix Cloud GenAI connector*. Additionally, Mendix offers platform-supported integration with [(Azure) OpenAI](/agents/agents-kit-1/reference-guide/external-connectors/openai/) and [Amazon Bedrock](/agents/agents-kit-1/reference-guide/external-connectors/bedrock/). If desired, you need to download these integrations manually from the Marketplace. Alternatively, you can integrate with custom models by creating your own connector and making its operations and object structure compatible with the [GenAI Commons](/agents/agents-kit-1/genai-for-mx/commons/) `Request` and `Response`. ## Installation {#installation} @@ -177,7 +177,7 @@ If the `ChatContext`, however, already exists and a new `ProviderConfig` needs t ### Defining and Setting the Action Microflow {#action-microflow} -The `Action Microflow` stored on a `ProviderConfig` is executed when the user clicks the **Send** button. This microflow handles the interaction between the LLM connectors and the Conversational UI entities. The **USE_ME > ConversationalUI > Action microflow examples** folder included in the Conversational UI module contains an example action microflow that is compatible with all connectors that follow GenAI Commons principles (such as [Mendix Cloud GenAI Connector](/agents/agents-kit-1/mx-cloud-genai/mxgenai-connector/), [OpenAI](/agents/agents-kit-1/reference-guide/external-connectors/openai/), and [Amazon Bedrock](/appstore/modules/aws/amazon-bedrock/)). You can copy and modify the microflow or use it directly. +The `Action Microflow` stored on a `ProviderConfig` is executed when the user clicks the **Send** button. This microflow handles the interaction between the LLM connectors and the Conversational UI entities. The **USE_ME > ConversationalUI > Action microflow examples** folder included in the Conversational UI module contains an example action microflow that is compatible with all connectors that follow GenAI Commons principles (such as [Mendix Cloud GenAI Connector](/agents/agents-kit-1/mx-cloud-genai/mxgenai-connector/), [OpenAI](/agents/agents-kit-1/reference-guide/external-connectors/openai/), and [Amazon Bedrock](/agents/agents-kit-1/reference-guide/external-connectors/bedrock/)). You can copy and modify the microflow or use it directly. Add the action microflow to an existing `ProviderConfig` by using the **Set Chat Action** toolbox action. Note that this action does not commit the object, so you must add a step to commit it afterward. @@ -204,7 +204,7 @@ The following operations can be found in the toolbox for changing the [ChatConte * `Set Topic` sets the `Topic` of the `ChatContext`. This attribute can be used in the **History** sidebar while making historical chats visible to users. * `Default Preprocessing` sets a default `Topic` for `ChatContext` and creates a sample [Request](/agents/agents-kit-1/genai-for-mx/commons/#request). -* `Set ConversationID` sets the ConversationID on the `ChatContext`. Storing the ConversationID is needed for a chat with history within [Retrieve and Generate with Amazon Bedrock](/appstore/modules/aws/amazon-bedrock/#retrieve-and-generate). +* `Set ConversationID` sets the ConversationID on the `ChatContext`. Storing the ConversationID is needed for a chat with history within [Retrieve and Generate with Amazon Bedrock](/agents/agents-kit-1/reference-guide/external-connectors/bedrock/#retrieve-and-generate). ##### Request Operations {#request-operations} diff --git a/content/en/docs/genai/v1/reference-guide/genai-commons.md b/content/en/docs/genai/v1/reference-guide/genai-commons.md index 0535b67a37e..ad4e4fb60eb 100644 --- a/content/en/docs/genai/v1/reference-guide/genai-commons.md +++ b/content/en/docs/genai/v1/reference-guide/genai-commons.md @@ -36,7 +36,7 @@ If you start from a blank app or have an existing app where you want to include ## Implementation {#implementation} -GenAI Commons is the foundation of large language model implementations within the [Mendix Cloud GenAI Connector](/agents/agents-kit-1/mx-cloud-genai/mxgenai-connector/), [OpenAI connector](/agents/agents-kit-1/reference-guide/external-connectors/openai/), and the [Amazon Bedrock connector](/appstore/modules/aws/amazon-bedrock/). You can also use it to build other GenAI service implementations by reusing the provided domain model and exposed actions. +GenAI Commons is the foundation of large language model implementations within the [Mendix Cloud GenAI Connector](/agents/agents-kit-1/mx-cloud-genai/mxgenai-connector/), [OpenAI connector](/agents/agents-kit-1/reference-guide/external-connectors/openai/), and the [Amazon Bedrock connector](/agents/agents-kit-1/reference-guide/external-connectors/bedrock/). You can also use it to build other GenAI service implementations by reusing the provided domain model and exposed actions. GenAI Commons defines additional capabilities typically found in chat completion APIs, such as image processing (vision) and tools (function calling). Whether these capabilities are implemented and supported by the LLM depends on the connector module you choose. To learn which additional capabilities a connector supports and for which models these can be used, refer to the documentation of that connector. diff --git a/content/en/docs/genai/v2/_index.md b/content/en/docs/genai/v2/_index.md index a989ab8f950..2b2d219a97b 100644 --- a/content/en/docs/genai/v2/_index.md +++ b/content/en/docs/genai/v2/_index.md @@ -54,7 +54,7 @@ All connectors depend on GenAI Commons and can be used with the other [core modu | Asset | Description | Release Version | | --- | --- | --- | -| [Amazon Bedrock Connector](/appstore/modules/aws/amazon-bedrock/) | Connect to Amazon Bedrock. | TBD | +| [Amazon Bedrock Connector](/agents/agents-kit-2/reference-guide/external-connectors/bedrock/) | Connect to Amazon Bedrock. | TBD | | [Google Gemini Connector](/agents/agents-kit-2/reference-guide/external-connectors/gemini/) | Connect to Google Gemini. | TBD | | [Mendix Cloud GenAI Connector](/agents/agents-kit-2/mx-cloud-genai/mxgenai-connector/) | Connect to Mendix Cloud and use Mendix Cloud GenAI resource packs directly within your Mendix application. | TBD | | [Mistral Connector](/agents/agents-kit-2/reference-guide/external-connectors/mistral/) | Connect to Mistral AI. | TBD | diff --git a/content/en/docs/genai/v2/how-to/_index.md b/content/en/docs/genai/v2/how-to/_index.md index 27b798a1e27..153c427dc65 100644 --- a/content/en/docs/genai/v2/how-to/_index.md +++ b/content/en/docs/genai/v2/how-to/_index.md @@ -41,7 +41,7 @@ Generative Artificial Intelligence (GenAI) transforms business applications, emp * The [ConversationalUI](/agents/agents-kit-2/genai-for-mx/conversational-ui/) module that offers UI snippets for chat, token consumption monitoring and prompt management. * The [Mendix Cloud GenAI Resource Packs](/agents/mx-cloud-genai/resource-packs/) to learn how to quickly access GenAI capabilities from a Mendix app. * The [OpenAI](/agents/agents-kit-2/reference-guide/external-connectors/openai/) provides essential information about the OpenAI connector. -* The [Amazon Bedrock](/appstore/modules/aws/amazon-bedrock/) provides key information about the AWS Bedrock connector. +* The [Amazon Bedrock](/agents/agents-kit-2/reference-guide/external-connectors/bedrock/) provides key information about the AWS Bedrock connector. * The [MCP Server Module](/agents/agents-kit-2/mcp-modules/mcp-server/) provides reusable operations to create and initialize an MCP server within a Mendix app to expose tools and prompts to external clients. * The [PGVector Knowledge Base](/agents/agents-kit-2/reference-guide/external-connectors/pgvector/) offers the option for a private knowledge base outside of the LLM infrastructure. diff --git a/content/en/docs/genai/v2/how-to/creating-agents/shared-setup.md b/content/en/docs/genai/v2/how-to/creating-agents/shared-setup.md index 4a7eca8756c..170dbbeaa38 100644 --- a/content/en/docs/genai/v2/how-to/creating-agents/shared-setup.md +++ b/content/en/docs/genai/v2/how-to/creating-agents/shared-setup.md @@ -40,7 +40,7 @@ Before you build an agent in your app, make sure your scenario meets the followi ## Setting Up Your Application {{% alert color="info" %}} -This guide uses the Mendix Cloud GenAI Connector for text generation. You can also use alternative [supported connectors](/agents/agents-kit-2/#connectors), such as [Amazon Bedrock](/appstore/modules/aws/amazon-bedrock/) or [OpenAI](/agents/agents-kit-2/reference-guide/external-connectors/openai/). For knowledge base operations, this guide uses the Mendix Cloud Knowledge Base, but the [pgVector Knowledge Base](/agents/agents-kit-2/reference-guide/external-connectors/pgvector/) is also supported. As long as you configure access to a provider and knowledge base according to the connector documentation, and the knowledge base supports inserting chunks from a microflow, the remaining steps in this guide apply. +This guide uses the Mendix Cloud GenAI Connector for text generation. You can also use alternative [supported connectors](/agents/agents-kit-2/#connectors), such as [Amazon Bedrock](/agents/agents-kit-2/reference-guide/external-connectors/bedrock/) or [OpenAI](/agents/agents-kit-2/reference-guide/external-connectors/openai/). For knowledge base operations, this guide uses the Mendix Cloud Knowledge Base, but the [pgVector Knowledge Base](/agents/agents-kit-2/reference-guide/external-connectors/pgvector/) is also supported. As long as you configure access to a provider and knowledge base according to the connector documentation, and the knowledge base supports inserting chunks from a microflow, the remaining steps in this guide apply. {{% /alert %}} If you are using a GenAI starter app such as the Blank GenAI Starter App, you can skip ahead to [Creating the Agent's Functional Prerequisites](#creating-functional-prerequisites) because the following setup steps are completed by default. Otherwise, follow these steps to add the required modules and configuration to your app: diff --git a/content/en/docs/genai/v2/how-to/integrate_function_calling.md b/content/en/docs/genai/v2/how-to/integrate_function_calling.md index a4d6b84b8c4..5f03eb23eab 100644 --- a/content/en/docs/genai/v2/how-to/integrate_function_calling.md +++ b/content/en/docs/genai/v2/how-to/integrate_function_calling.md @@ -52,7 +52,7 @@ Selecting the infrastructure for integrating GenAI into your Mendix application * [OpenAI](/agents/agents-kit-2/reference-guide/external-connectors/openai/): The [OpenAI Connector](https://marketplace.mendix.com/link/component/220472) supports both OpenAI's platform and Microsoft Foundry. -* [Amazon Bedrock](/appstore/modules/aws/amazon-bedrock/): The [Amazon Bedrock Connector](https://marketplace.mendix.com/link/component/215042) allows you to leverage Amazon Bedrock’s fully managed service to integrate foundation models from Amazon and leading AI providers. +* [Amazon Bedrock](/agents/agents-kit-2/reference-guide/external-connectors/bedrock/): The [Amazon Bedrock Connector](https://marketplace.mendix.com/link/component/215042) allows you to leverage Amazon Bedrock’s fully managed service to integrate foundation models from Amazon and leading AI providers. * Your Own Connector: Optionally, if you prefer a custom connector, you can integrate your chosen infrastructure. However, this document focuses on the Mendix Cloud GenAI, OpenAI, and Amazon Bedrock connectors, as they offer comprehensive support and ease of use to get started. diff --git a/content/en/docs/genai/v2/how-to/prompt_engineering-runtime.md b/content/en/docs/genai/v2/how-to/prompt_engineering-runtime.md index 63ebbcdbb70..878f299f934 100644 --- a/content/en/docs/genai/v2/how-to/prompt_engineering-runtime.md +++ b/content/en/docs/genai/v2/how-to/prompt_engineering-runtime.md @@ -62,7 +62,7 @@ If you are using a GenAI starter app such as the Blank GenAI app, you can skip a To enable generative AI capabilities, install and configure the [Mendix Cloud GenAI Connector](/agents/agents-kit-2/mx-cloud-genai/mxgenai-connector/) and its dependencies from the Mendix Marketplace. If you are using a GenAI starter app such as the Blank GenAI app, you can skip the installation and just follow the [configuration instructions](/agents/agents-kit-2/mx-cloud-genai/mxgenai-connector/#configuration). {{% alert color="info" %}} -This example uses the Mendix Cloud GenAI Connector. Alternatively, you can install and configure an [external connector](/agents/agents-kit-2/reference-guide/connectors/) for any provider with a connector that is compatible with [GenAICommons](/agents/agents-kit-2/genai-for-mx/commons/). This includes [OpenAI](/agents/agents-kit-2/reference-guide/external-connectors/openai/) and [Amazon Bedrock](/appstore/modules/aws/amazon-bedrock/). +This example uses the Mendix Cloud GenAI Connector. Alternatively, you can install and configure an [external connector](/agents/agents-kit-2/reference-guide/connectors/) for any provider with a connector that is compatible with [GenAICommons](/agents/agents-kit-2/genai-for-mx/commons/). This includes [OpenAI](/agents/agents-kit-2/reference-guide/external-connectors/openai/) and [Amazon Bedrock](/agents/agents-kit-2/reference-guide/external-connectors/bedrock/). {{% /alert %}} ## Verifying Setup {#verification} diff --git a/content/en/docs/genai/v2/how-to/start_from_a_starter_app.md b/content/en/docs/genai/v2/how-to/start_from_a_starter_app.md index 27bf91021b5..d65cbe8eaa8 100644 --- a/content/en/docs/genai/v2/how-to/start_from_a_starter_app.md +++ b/content/en/docs/genai/v2/how-to/start_from_a_starter_app.md @@ -50,7 +50,7 @@ Selecting the infrastructure for integrating GenAI into your Mendix application * [OpenAI](/agents/agents-kit-2/reference-guide/external-connectors/openai/): The [OpenAI Connector](https://marketplace.mendix.com/link/component/220472) supports OpenAI's platform and Microsoft Foundry. -* [Amazon Bedrock](/appstore/modules/aws/amazon-bedrock/): The [Amazon Bedrock Connector](https://marketplace.mendix.com/link/component/215042) allows you to leverage Amazon Bedrock’s fully managed service to integrate foundation models from Amazon and leading AI providers. +* [Amazon Bedrock](/agents/agents-kit-2/reference-guide/external-connectors/bedrock/): The [Amazon Bedrock Connector](https://marketplace.mendix.com/link/component/215042) allows you to leverage Amazon Bedrock’s fully managed service to integrate foundation models from Amazon and leading AI providers. * Your Own Connector: Optionally, if you prefer a custom connector, you can integrate your chosen infrastructure. However, this document focuses on the platform-supported connectors, as they offer comprehensive support and ease of use to get started. diff --git a/content/en/docs/genai/v2/how-to/start_from_blank_app.md b/content/en/docs/genai/v2/how-to/start_from_blank_app.md index f5d32a1c949..7a256d8a322 100644 --- a/content/en/docs/genai/v2/how-to/start_from_blank_app.md +++ b/content/en/docs/genai/v2/how-to/start_from_blank_app.md @@ -59,7 +59,7 @@ Selecting the infrastructure for integrating GenAI into your Mendix application * [Mendix Cloud GenAI Resources Packs](/agents/mx-cloud-genai/resource-packs/): The [Mendix Cloud GenAI Connector](https://marketplace.mendix.com/link/component/239449) integrates LLMs by dragging and dropping common operations from its toolbox in Studio Pro. * [OpenAI](/agents/agents-kit-2/reference-guide/external-connectors/openai/): The [OpenAI Connector](https://marketplace.mendix.com/link/component/220472) supports both OpenAI's platform and Microsoft Foundry. -* [Amazon Bedrock](/appstore/modules/aws/amazon-bedrock/): The [Amazon Bedrock Connector](https://marketplace.mendix.com/link/component/215042) allows you to leverage Amazon Bedrock’s fully managed service to integrate foundation models from Amazon and leading AI providers. +* [Amazon Bedrock](/agents/agents-kit-2/reference-guide/external-connectors/bedrock/): The [Amazon Bedrock Connector](https://marketplace.mendix.com/link/component/215042) allows you to leverage Amazon Bedrock’s fully managed service to integrate foundation models from Amazon and leading AI providers. * Your Own Connector: Optionally, if you prefer a custom connector, you can integrate your chosen infrastructure. However, this document focuses on the OpenAI and Bedrock connectors, as they offer comprehensive support and ease of use to get started. @@ -178,7 +178,7 @@ Follow the steps below to configure Amazon Bedrock for your application: * Click **Save & Sync Data** to ensure your changes are applied. {{% alert color="info" %}} -If you encounter any issues while using the Amazon Bedrock connector, see the [Troubleshooting](/appstore/modules/aws/amazon-bedrock/#troubleshooting) section of the *Amazon Bedrock*. +If you encounter any issues while using the Amazon Bedrock connector, see the [Troubleshooting](/agents/agents-kit-2/reference-guide/external-connectors/bedrock/#troubleshooting) section of the *Amazon Bedrock*. {{% /alert %}} ## Testing and Troubleshooting diff --git a/content/en/docs/genai/v2/reference-guide/agent-commons.md b/content/en/docs/genai/v2/reference-guide/agent-commons.md index 78118e19379..15c8089f081 100644 --- a/content/en/docs/genai/v2/reference-guide/agent-commons.md +++ b/content/en/docs/genai/v2/reference-guide/agent-commons.md @@ -99,7 +99,7 @@ For example, download and run the [Agent Builder Starter App](https://marketplac To interact with LLMs using Agent Commons, you need at least one GenAI connector that adheres to the GenAI Commons principles. To test agent behavior, you must configure at least one [Deployed Model](/agents/agents-kit-2/genai-for-mx/commons/#deployed-model) for your chosen connector. Refer to the specific connector’s documentation for detailed instructions on setting up the Deployed Model. * For [Mendix Cloud GenAI](https://marketplace.mendix.com/link/component/239449), importing the **Key** from the Mendix portal automatically creates a MxCloud Deployed Model. This is part of the [configuration](/agents/agents-kit-2/mx-cloud-genai/mxgenai-connector/#configuration). -* For [Amazon Bedrock](https://marketplace.mendix.com/link/component/215042), the creation of Bedrock Deployed Models is part of the [model synchronization mechanism](/appstore/modules/aws/amazon-bedrock/#sync-models). +* For [Amazon Bedrock](https://marketplace.mendix.com/link/component/215042), the creation of Bedrock Deployed Models is part of the [model synchronization mechanism](/agents/agents-kit-2/reference-guide/external-connectors/bedrock/#sync-models). * For [OpenAI](https://marketplace.mendix.com/link/component/220472), the configuration of OpenAI Deployed Models is part of the [configuration](/agents/agents-kit-2/reference-guide/external-connectors/openai/#general-configuration). ### Defining the Agent {#define-agent} @@ -150,7 +150,7 @@ Besides microflow tools, tools exposed by MCP servers are also supported. To add For supported knowledge bases registered in your app, you can connect them to agents to enable autonomous retrievals. Refer to the documentation of the connector provided by your selected knowledge base provider. Follow the instructions to configure the knowledge bases in your app, so that they can be linked to your agents. Mendix provides the following platform-supported connectors that support knowledge base integrations with agents: * [Mendix Cloud GenAI Connector](/agents/agents-kit-2/mx-cloud-genai/mxgenai-connector/#configuration) -* [Amazon Bedrock Connector](/appstore/modules/aws/amazon-bedrock/#sync-models) +* [Amazon Bedrock Connector](/agents/agents-kit-2/reference-guide/external-connectors/bedrock/#sync-models) * [OpenAI Connector](/agents/agents-kit-2/reference-guide/external-connectors/openai/#azure-ai-search) * [PgVector Knowledge Base](/agents/agents-kit-2/reference-guide/external-connectors/pgvector/#general-configuration) diff --git a/content/en/docs/genai/v2/reference-guide/conversational-ui.md b/content/en/docs/genai/v2/reference-guide/conversational-ui.md index 617928e3c9b..84adaf61e0e 100644 --- a/content/en/docs/genai/v2/reference-guide/conversational-ui.md +++ b/content/en/docs/genai/v2/reference-guide/conversational-ui.md @@ -65,7 +65,7 @@ You must also ensure you have the other prerequisite modules that Conversational * [Nanoflow Commons](https://marketplace.mendix.com/link/component/109515) * [Web Actions](https://marketplace.mendix.com/link/component/114337) -Finally, you must also set up a connector that is compatible with [GenAI Commons](/agents/agents-kit-2/genai-for-mx/commons/). One option is to use the [Mendix Cloud GenAI connector](https://marketplace.mendix.com/link/component/239449). For more information on how to configure this connector, see the [Configuration](/agents/agents-kit-2/mx-cloud-genai/mxgenai-connector/#configuration) section of *Mendix Cloud GenAI connector*. Additionally, Mendix offers platform-supported integration with [(Azure) OpenAI](/agents/agents-kit-2/reference-guide/external-connectors/openai/) and [Amazon Bedrock](/appstore/modules/aws/amazon-bedrock/). If desired, you need to download these integrations manually from the Marketplace. Alternatively, you can integrate with custom models by creating your own connector and making its operations and object structure compatible with the [GenAI Commons](/agents/agents-kit-2/genai-for-mx/commons/) `Request` and `Response`. +Finally, you must also set up a connector that is compatible with [GenAI Commons](/agents/agents-kit-2/genai-for-mx/commons/). One option is to use the [Mendix Cloud GenAI connector](https://marketplace.mendix.com/link/component/239449). For more information on how to configure this connector, see the [Configuration](/agents/agents-kit-2/mx-cloud-genai/mxgenai-connector/#configuration) section of *Mendix Cloud GenAI connector*. Additionally, Mendix offers platform-supported integration with [(Azure) OpenAI](/agents/agents-kit-2/reference-guide/external-connectors/openai/) and [Amazon Bedrock](/agents/agents-kit-2/reference-guide/external-connectors/bedrock/). If desired, you need to download these integrations manually from the Marketplace. Alternatively, you can integrate with custom models by creating your own connector and making its operations and object structure compatible with the [GenAI Commons](/agents/agents-kit-2/genai-for-mx/commons/) `Request` and `Response`. ## Installation {#installation} @@ -178,7 +178,7 @@ If the `ChatContext`, however, already exists and a new `ProviderConfig` needs t ### Defining and Setting the Action Microflow {#action-microflow} -The `Action Microflow` stored on a `ProviderConfig` is executed when the user clicks the **Send** button. This microflow handles the interaction between the LLM connectors and the Conversational UI entities. The **USE_ME > ConversationalUI > Action microflow examples** folder included in the Conversational UI module contains an example action microflow that is compatible with all connectors that follow GenAI Commons principles (such as [Mendix Cloud GenAI Connector](/agents/agents-kit-2/mx-cloud-genai/mxgenai-connector/), [OpenAI](/agents/agents-kit-2/reference-guide/external-connectors/openai/), and [Amazon Bedrock](/appstore/modules/aws/amazon-bedrock/)). You can copy and modify the microflow or use it directly. +The `Action Microflow` stored on a `ProviderConfig` is executed when the user clicks the **Send** button. This microflow handles the interaction between the LLM connectors and the Conversational UI entities. The **USE_ME > ConversationalUI > Action microflow examples** folder included in the Conversational UI module contains an example action microflow that is compatible with all connectors that follow GenAI Commons principles (such as [Mendix Cloud GenAI Connector](/agents/agents-kit-2/mx-cloud-genai/mxgenai-connector/), [OpenAI](/agents/agents-kit-2/reference-guide/external-connectors/openai/), and [Amazon Bedrock](/agents/agents-kit-2/reference-guide/external-connectors/bedrock/)). You can copy and modify the microflow or use it directly. Add the action microflow to an existing `ProviderConfig` by using the **Set Chat Action** toolbox action. Note that this action does not commit the object, so you must add a step to commit it afterward. @@ -205,7 +205,7 @@ The following operations can be found in the toolbox for changing the [ChatConte * `Set Topic` sets the `Topic` of the `ChatContext`. This attribute can be used in the **History** sidebar while making historical chats visible to users. * `Default Preprocessing` sets a default `Topic` for `ChatContext` and creates a sample [Request](/agents/agents-kit-2/genai-for-mx/commons/#request). -* `Set ConversationID` sets the ConversationID on the `ChatContext`. Storing the ConversationID is needed for a chat with history within [Retrieve and Generate with Amazon Bedrock](/appstore/modules/aws/amazon-bedrock/#retrieve-and-generate). +* `Set ConversationID` sets the ConversationID on the `ChatContext`. Storing the ConversationID is needed for a chat with history within [Retrieve and Generate with Amazon Bedrock](/agents/agents-kit-2/reference-guide/external-connectors/bedrock/#retrieve-and-generate). ##### Request Operations {#request-operations} diff --git a/content/en/docs/genai/v2/reference-guide/genai-commons.md b/content/en/docs/genai/v2/reference-guide/genai-commons.md index 12f54315ffd..0a1c9fc73c4 100644 --- a/content/en/docs/genai/v2/reference-guide/genai-commons.md +++ b/content/en/docs/genai/v2/reference-guide/genai-commons.md @@ -37,7 +37,7 @@ If you start from a blank app or have an existing app where you want to include ## Implementation {#implementation} -GenAI Commons is the foundation of large language model implementations within the [Mendix Cloud GenAI Connector](/agents/agents-kit-2/mx-cloud-genai/mxgenai-connector/), [OpenAI connector](/agents/agents-kit-2/reference-guide/external-connectors/openai/), and the [Amazon Bedrock connector](/appstore/modules/aws/amazon-bedrock/). You can also use it to build other GenAI service implementations by reusing the provided domain model and exposed actions. +GenAI Commons is the foundation of large language model implementations within the [Mendix Cloud GenAI Connector](/agents/agents-kit-2/mx-cloud-genai/mxgenai-connector/), [OpenAI connector](/agents/agents-kit-2/reference-guide/external-connectors/openai/), and the [Amazon Bedrock connector](/agents/agents-kit-2/reference-guide/external-connectors/bedrock/). You can also use it to build other GenAI service implementations by reusing the provided domain model and exposed actions. GenAI Commons defines additional capabilities typically found in chat completion APIs, such as image processing (vision) and tools (function calling). Whether these capabilities are implemented and supported by the LLM depends on the connector module you choose. To learn which additional capabilities a connector supports and for which models these can be used, refer to the documentation of that connector.