Create Grounded Model connections with Google Vertex AI RAG capability

Use the Google Vertex AI RAG (Retrieval Augmented Generation) capability to create Grounded by data store Model connections to generate accurate and contextually relevant information referenced from Google Data Source.

Important:

To use Google Vertex AI Grounded models for On-Premises, document chunking must be disabled in the data store.

In Cloud, the option to use Google Vertex AI Grounded by data store is currently unavailable. With the upcoming patch release, data stores with enabled document chunking will be supported to ensure optimal results in automation executions.

We are now offering you the option to create Grounded by data store Model connections using the Google Agent Builder service. A search query on the Google Data Store retrieves relevant content from large-data-sets and feeds it to the model to generate accurate response.

During an automation execution, using Model connections created with Grounded by data store retrieves the response by referencing the Google Data Source in Google Data Store. This ensures an optimized response from the relevant content with higher accuracy. Grounding is an important aspect of using a foundational model as it returns grounded responses against organizational data and prevents response inaccuracies and model hallucinations.
Note: To use the Google Vertex AI RAG capability in AI Agent Studio, you would first create a data source in Google Agent Builder. Then create a Model connection using the Grounded by data store option.

See: Data stores in Google Vertex AI.

Prerequisites

The Automation Admin requires these roles and permissions to create and manage Model connections for their business organization.
  • Role: AAE_Basic, Automation Admin custom role
  • Permission: Attended Bot Runner
  • Settings: AI Data Management must be enabled by the Automation Admin and the check box selected for Allow users to disable logs on AI Skills option.

See Roles and permissions for AI Tools for the Automation Admin custom role permissions.

Other requirements:
  • As mentioned earlier, you would first create a Google Data Source to create a Grounded by data store Model connection and use it successfully in an AI Skill.
  • If you want to store authentication details in a credential vault, have that information handy. See Secure credential store through Credential Vault.
  • To test a Model connection, you must be connected to a Bot Agent 22.60.10 and later. As part of the test, you would have to run the bot on your desktop. Hence ensure the Bot Agent is configured to your user. For this task, if you have to switch connection to a different Control Room, see: Switch device registration between Control Room instances.
  • You would need access to the Recorder package and the AI Skills package to test the connection successfully. A test Prompt would be executed to test the Model connection.

Procedure

  1. In your Control Room environment, navigate to AI > Model connections > Create model connection.
  2. In the Create model connection screen you would configure these Connection settings:
    Create Grounded Model connections with Google Vertex AI RAG capability
    You can manually enter the model name in the Choose a model or create a custom one field. The name you enter will be used for creating the Model connection.
    1. Model connection name: Provide a name for easy identification of the Model connection.
    2. Description (optional): Add a meaningful short description defining the connection.
    3. Choose a vendor: Choose a foundational model vendor from the supported list of vendors. For creating a Grounded by data store Model connection with Google Vertex AI, select Google Vertex AI from the drop-down list.
    4. Choose a type: Choose Grounded by data store for using the RAG capability for Google Vertex AI.
    5. Choose a model or create a custom one: Choose a model from the drop-down list.
      Additionally, we also support other models available from Google Data Store, which are not available in the drop-down list. If you want to add a model from the Google Data Store, you would have to enter the model name and version with the complete URI of the model. For example: If the model is Gemini 1.5 Flash 001, the format would be gemini-1.5-001/answer_gen/v1.
    6. Click Next to proceed to the Authentication details section.
  3. In the Authentication details section, configure these settings:
    1. Project Name: This is the Google Cloud account project.
    2. Region: Select a region from the drop-down list to connect for authenticating the Model connection. You can also add your own region that you configured when you created a data source in Google Agent Builder.
      See Create a data store.
    3. Control Room OAuth Connection: Create an OAuth 2.0 Client ID. A client ID is used to identify a single application to Google's OAuth Servers.
    4. After setting up the authentication details, confirm and click Next to proceed to the Test connection section to test the Model connection.
    Note: For details on setting up the Google Cloud Project and Control Room OAuth Connection for Google Vertex AI, see Vertex AI: Connect action
  4. Click Test connection to make sure all connection details have been defined correctly and check if the connection is working.
    This is a desktop operation using a Bot Agent. Use Bot Agent 22.60.10 and later for successful testing.
    • If the connection works as expected, the system will process the request and you will get a system generated success message.
    • If the connection does not work as expected, you will get a system-generated message stating the reason for the connection failure. For example, if you have not downloaded the supported foundational model package to your workspace, you would get an error message. You would have to download the package and then retest the Model connection.
    • If the testing a Model connection is unsuccessful or if you leave the task incomplete, the Model connection will not get saved and you will have to restart the process of creating the Model connection.
  5. Click Next to proceed to the Invite roles section to begin assigning custom roles to users.
    The Automation Admin would create custom roles and assign the Model connections to the role, which can then be assigned to users. Only users assigned to this custom role can use this Model connection.
  6. Assign the Model connection to the custom role (using RBAC) so users assigned to the role can access it.
  7. Click Create model connection to complete creating the Model connection.
    After successfully creating the Model connection, the Pro Developer would use it to create an AI Skill.

    See: Create AI Skills with Grounded by data store Model connections

    .

Next steps

After creating and testing the Model connection, you would assign it to the Pro Developers, who would use this connection to create AI Skills.

As the next step in your sequence of tasks, go to Create AI Skills with Grounded by data store Model connections and create an AI Skill and connect to a Grounded by data store Model connection to use it in an automation.