Create Grounded Model connections with Azure OpenAI RAG capability

Create Grounded by AI Search Model connections using the native Retrieval Augmented Generation(RAG) capability from Azure OpenAI to build rich search experiences that combine large language models with enterprise data from Azure AI Search.

You can now integrate an Azure AI Search service with the Azure OpenAI service to create a RAG solution. This allows the LLMs to provide more informed and contextually relevant responses by retrieving information from your own data.

AI Agent Studio Azure AI Search RAG

Note: You must setup the following within the Azure OpenAI portal:
  • Azure AI Search Service Setup: This involves creating the AI Search service in the Azure OpenAI portal. This includes setting up the service endpoint URL, API keys, and creating the index. For more information on creating an Azure AI Search search, see Create an Azure AI Search service in the Azure portal.
  • Data Ingestion and Indexing: Documents are uploaded to a data source, like a blob storage, and then the index is created using the files in the storage. The documents are split into chunks, and the content is vectorized using the embedding model if vector search is enabled.

    When setting up your Azure AI Search service for integration with Automation 360 Grounded by AI Search Model connections, it is essential to configure your data ingestion and indexing process to create vector embeddings. WhileAzure AI Search supports other content types and configurations (like just text or just semantic), the Automation Anywhere integration is optimized for and primarily supports the use of vector embeddings to enable semantic understanding and retrieval.

Prerequisites

The Automation Admin requires these roles and permissions to create and manage Model connections for their business organization.
  • Role: AAE_Basic, Automation Admin custom role
  • Permission: Attended Bot Runner
  • Settings: AI Data Management must be enabled by the Automation Admin and the check box selected for Allow users to disable logs on AI Skills. Allow users with the Bot Creator license to disable data logging when using AI Skills to enable the Data logging toggle in the AI Skills screen.

See Roles and permissions for AI Tools for the Automation Admin custom role permissions.

Other requirements:

  • As mentioned earlier, you would first create an Azure AI Search to create a Grounded by AI Search Model connection and use it successfully in an AI Skill.
  • If you want to store authentication details in a credential vault, have that information handy. See Secure credential store through Credential Vault.
  • To test a Model connection, you must be connected to a Bot Agent 22.60.10 and later. As part of the test, you would have to run the bot on your desktop. Hence ensure the Bot Agent is configured to your user. For this task, if you have to switch connection to a different Control Room, see: Switch device registration between Control Room instances.
  • You would need access to the AI Skills package to test the connection successfully. A test Prompt would be executed to test the Model connection.

Procedure

  1. In your Control Room environment, navigate to AI > Model connections > Create model connection.
  2. In the Create model connection screen you would configure these Connection settings:
    Create Grounded Model connections with Azure OpenAI RAG capability
    1. Model connection name: Provide a name for easy identification of the Model connection.
    2. Description (optional): Add a meaningful short description defining the connection.
    3. Choose a vendor: Choose a foundational model vendor from the supported list of vendors. For creating a Grounded by AI Search Model connection with Azure OpenAI, you would select Azure OpenAI from the drop-down list.
    4. Choose a type: Choose Grounded by AI Search for using the RAG capability.
    5. Choose a model or create a custom one: Choose a model from the drop-down list of validated models from Azure OpenAI.
    6. Click Next to proceed to the Authentication details section.
  3. In the Authentication details section, configure these settings:
    1. Azure OpenAI Resource Name: Enter the name of your Azure OpenAI resource.
    2. Deployment ID: Enter the deployment ID of the model you want to use.
    3. API Key: Enter the API key for your Azure OpenAI service.
    4. AI Search service URL: The URL of your deployed Azure AI Search service. This is the endpoint through which the search service is accessed.
    5. AI Search index name: The name of the index within the Azure AI Search service that contains your data. This is where the documents to be searched are stored.
    6. AI Search API key:The API key for your Azure AI Search service. This key is needed to authenticate access to the search service.
    7. AI Search embedding model deployment name: The name of the embedding model that has been deployed in Azure. This model is used to convert text into vector representations which are used for semantic search.
      Note:
      1. Embedding model: An embedding model is a type of machine learning model designed to transform data (like text, images, or even code) into numerical representations called vector embeddings.

        An embedding model is crucial as vector embeddings of your data are necessary for the Grounded by AI Search functionality to effectively find semantically relevant information.

      2. Vector embeddings: These vectors are lists of numbers that capture the semantic meaning of the data. Similar pieces of data will have vectors that are close together in a high-dimensional space.

        Ensure that your Azure AI Search index is populated with vector embeddings generated from your data. This is fundamental for the AI Search types supported by Automation Anywhere. Refer to Azure AI Search documentation for detailed guidance on creating embeddings during the data ingestion process.

      3. Semantic understanding: The key is that these embeddings capture the meaning and context of the data, not just keywords. This allows Azure AI Search to understand the relationships between different pieces of information.
    8. Click Next to proceed to the Test connection section.
  4. Click Test connection to make sure all connection details have been defined correctly and check if the connection is working.
    This is a desktop operation using a Bot Agent. Use Bot Agent 22.60.10 and later for successful testing.
    • If the connection works as expected, the system will process the request and you will get a system generated success message.
    • If the connection does not work as expected, you will get a system-generated message stating the reason for the connection failure. For example, if you have not downloaded the supported foundational model package to your workspace, you would get an error message. You would have to download the package and then retest the Model connection.
    • If the testing a Model connection is unsuccessful or if you leave the task incomplete, the Model connection will not get saved and you will have to restart the process of creating the Model connection.
  5. Click Next to proceed to the Invite roles section to begin assigning custom roles to users.
    The Automation Admin would create custom roles and assign the Model connections to the role, which can then be assigned to users. Only users assigned to this custom role can use this Model connection.
  6. Assign access to the Pro Developer via custom role, for using this Model connection to create an AI Skill.
  7. Click Create model connection to complete creating the Model connection.
    After successfully creating the Model connection, the Pro Developer would use it to create an AI Skill.

Next steps

After creating and testing the Model connection, you would assign it to the Pro Developers, who would use this connection to create AI Skills. See Create AI Skills with Grounded by AI Search Model connections.
Note: When you create or test an AI Skill in the AI Skill screen, the success or failure details along with the model responses can be viewed in these navigation screens:
  • Administration > AI governance > AI Prompt log
  • Administration > AI governance > Event log
  • Administration > Audit log

See AI Governance.