Create Grounded Model connections with Azure OpenAI RAG capability

Create Grounded by AI Search Model connections using the native RAG (Retrieval Augmented Generation) capability from Azure OpenAI to build rich search experiences that combine large language models with enterprise data from Azure AI Search.

You can now integrate an Azure AI Search service with the Azure OpenAI service to create a RAG solution. This allows the LLMs to provide more informed and contextually relevant responses by retrieving information from a your own data.

Prerequisites

You must setup the following within the Azure OpenAI portal:
  • Azure AI Search Service Setup: This involves creating the AI Search service in the Azure OpenAI portal. This includes setting up the service endpoint URL, API keys, and creating the index.
  • Data Ingestion and Indexing: Documents are uploaded to a data source, like a blob storage, and then the index is created using the files in the storage. The documents are split into chunks, and the content is vectorized using the embedding model if vector search is enabled.

Prerequisites

The Automation Admin requires these roles and permissions to create and manage Model connections for their business organization.
  • Role: AAE_Basic, Automation Admin custom role
  • Permission: Attended Bot Runner
  • Settings: AI Data Management must be enabled by the Automation Admin and the check box selected for Allow users to disable logs on AI Skills. Allow users with the Bot Creator license to disable data logging when using AI Skills to enable the Data logging toggle in the AI Skills screen.

See Roles and permissions for AI Tools for the Automation Admin custom role permissions.

Other requirements:

  • As mentioned earlier, you would first create an Azure AI Search to create a Grounded by AI Search Model connection and use it successfully in an AI Skill.
  • If you want to store authentication details in a credential vault, have that information handy. See Secure credential store through Credential Vault.
  • To test a Model connection, you must be connected to a Bot Agent 22.60.10 and later. As part of the test, you would have to run the bot on your desktop. Hence ensure the Bot Agent is configured to your user. For this task, if you have to switch connection to a different Control Room, see: Switch device registration between Control Room instances.
  • You would need access to the AI Skills package to test the connection successfully. A test Prompt would be executed to test the Model connection.

Procedure

  1. In your Control Room environment, navigate to AI > Model connections > Create model connection.
  2. In the Create model connection screen you would configure these Connection settings:
    Create Grounded Model connections with Azure OpenAI RAG capability
    You can manually enter the model name in the Choose a model or create a custom one field. The name you enter will be used for creating the Model connection.
    1. Model connection name: Provide a name for easy identification of the Model connection.
    2. Description (optional): Add a meaningful short description defining the connection.
    3. Choose a vendor: Choose a foundational model vendor from the supported list of vendors. For creating a Grounded by AI Search Model connection with Azure OpenAI, you would select Azure OpenAI from the drop-down list.
    4. Choose a type: Choose Grounded by AI Search for using the RAG capability.
    5. Choose a model or create a custom one: Choose a model from the drop-down list of validated models from Azure OpenAI.
      Additionally, we also support other models available from Azure OpenAI, which are not available in the drop-down list.
      For a complete list of supported models for each foundational model vendor, see General FAQs.
    6. Click Next to proceed to the Authentication details section.
  3. In the Authentication details section, configure these settings:
    1. Azure OpenAI Resource Name: Enter the name of your Azure OpenAI resource.
    2. Deployment ID: Enter the deployment ID of the model you want to use.
    3. API Key: Enter the API key for your Azure OpenAI service.
    4. AI Search service URL: The URL of your deployed Azure AI Search service. This is the endpoint through which the search service is accessed.
    5. AI Search index name: The name of the index within the Azure AI Search service that contains your data. This is where the documents to be searched are stored.
    6. AI Search API key:The API key for your Azure AI Search service. This key is needed to authenticate access to the search service.
    7. AI Search embedding model deployment name: The name of the embedding model that has been deployed in Azure. This model is used to convert text into vector representations which are used for semantic search.
    8. Click Next to proceed to the Test connection section.
  4. Click Test connection to make sure all connection details have been defined correctly and check if the connection is working.
    This is a desktop operation using a Bot Agent. Use Bot Agent 22.60.10 and later for successful testing.
    • If the connection works as expected, the system will process the request and you will get a system generated success message.
    • If the connection does not work as expected, you will get a system-generated message stating the reason for the connection failure. For example, if you have not downloaded the supported foundational model package to your workspace, you would get an error message. You would have to download the package and then retest the Model connection.
    • If the testing a Model connection is unsuccessful or if you leave the task incomplete, the Model connection will not get saved and you will have to restart the process of creating the Model connection.
  5. Click Next to proceed to the Invite roles section to begin assigning custom roles to users.
    The Automation Admin would create custom roles and assign the Model connections to the role, which can then be assigned to users. Only users assigned to this custom role can use this Model connection.
  6. Assign access to the Pro Developer via custom role (using RBAC), for using this Model connection to create an AI Skill.
  7. Click Create model connection to complete creating the Model connection.
    After successfully creating the Model connection, the Pro Developer would use it to create an AI Skill.

Next steps

After creating and testing the Model connection, you would assign it to the Pro Developers, who would use this connection to create AI Skills.
Note: When you create or test an AI Skill in the AI Skill screen, the success or failure details along with the model responses can be viewed in these navigation screens:
  • Administration > AI governance > AI Prompt log
  • Administration > AI governance > Event log
  • Administration > Audit log

See AI Governance.