Create AI Skills with Grounded by AI Search Model connections

This is the next logical step after creating a Grounded by AI Search Model connection. You would create an AI Skill and connect it to a Grounded Model connection from Azure OpenAI.

Azure OpenAI Grounded by AI Search Model connections is available in Automation 360 on Cloud and On-Premises.

A Pro Developer creates AI Skills so the Bot Creators can use these in their automations and save time and effort.

AI Skills are created by connecting to Model connections the Pro Developer has access to, and fine-tuning prompts by testing them with different foundational models to find the best response that addresses the business ask. These AI Skills are made available to developers for use and reuse to help accelerate creating automations across solutions.

Prerequisites

A Pro Developer requires these roles and permissions to create and test AI Skills.
  • Role: AAE_Basic, Pro Developer Custom role
  • Permission: Bot Creator. See Roles and permissions for AI Tools
  • Other requirements:Besides the roles and permissions the Pro Developer must be connected to a Bot Agent 22.60.10 and later. As part of testing the Model connection, you would have to run the bot on your desktop. Hence ensure the Bot Agent is configured to your user. If you have to switch connection to a different Control Room, see: Switch device registration between Control Room instances.

Procedure

  1. Log in to the Control Room and navigate to Automation > Create new or ‘+’ icon and choose AI Skills.
  2. Provide a name and description and click Create & edit to display a template outline.
  3. In the AI Skills screen, click Choose model connection to choose from the available list of Model connections you have access to. You would choose a Model connection that was created using the Grounded by AI Search type option from Azure OpenAI. For more information on creating a Model connections using the Grounded by AI Search, see Create Grounded Model connections with Azure OpenAI RAG capability.
    These Model connections are created by the Automation Admin and assigned to your user with a custom role.
  4. After selecting a Model connection, the AI Skills is set up with the default parameter settings that is optimal for the chosen model. You can change the settings as required.
    The AI Skill editor displays with default parameter values set by the model vendor which you can configure as required. These values can be configured when creating a data store in Agent Builder.
  5. Select the AI Search type.
    1. Hybrid (vector + keyword): This combines the strengths of full-text and vector search in a single query. Azure AI Search performs both a keyword-based search and a vector-based search simultaneously. It then merges the results to provide a more comprehensive set of relevant documents.
    2. Vector: This type of search leverages the power of AI embeddings to find documents that are semantically related to your query, even if they do not contain the exact keywords. You use an embedding model (like those from Azure OpenAI) to convert your documents and queries into vector embeddings. Azure AI Search stores these embeddings in your index. When you search, it compares the embedding of your query to the embeddings of the documents to find those that are most similar.
    3. Hybrid + semantic: This combines full-text, vector, and semantic ranking and you get the most comprehensive and relevant search results. The semantic ranker can identify subtle relationships between the query and the documents, leading to more accurate results.
      • Semantic Ranking: The Hybrid + semantic option enhances search results by applying semantic ranking. This advanced ranking capability in Azure AI Search understands the intent and context of the query to provide more relevant results.
      • Configuration: To use semantic ranking, you need to configure semantic search in your Azure Azure AI Search. Refer to the official Azure AI Search documentation for detailed steps on how to set up semantic configurations:Azure AI Search Semantic Search Configuration.
      • Cost Considerations: Note that Azure AI Search has a separate pricing model for using semantic ranking features. It is recommended to review the Azure AI Search pricing page for detailed information on the costs associated with semantic search: Azure AI Search Pricing.
      Note: While Azure AI Search offers various search types, ensure your underlying Azure AI Search index is configured with vector embeddings. The Vector and Hybrid (vector + keyword) options directly leverage these embeddings. The Hybrid + semantic option also benefits from vector embeddings for its initial retrieval phase.
  6. Select the Strictness. It controls how strict the search engine is when filtering and selecting documents to answer a question. Higher strictness leads to higher precision but lower recall (you might miss some potentially relevant answers). Lower strictness leads to higher recall (you will get more potential answers) but lower precision (some answers might be less relevant or even incorrect).
  7. Select the Document count. It determines how many of the top-ranked documents Azure AI Search should provide to the language model to generate an answer. Higher document count gives the language model more context to generate a more comprehensive and accurate answer. Lower document count results in less comprehensive or accurate answers, as the language model has less information to work with.
  8. Next, add an Azure Filter condition, which is optional. This field supports a string format for entering the filter value. Adding a filter helps narrow down the model's search to the specific files within the storage.

    You can make sure that the AI Skill grounds the response using information in a specific set of documents in the Azure OpenAI portal. This narrows down the scope of response and makes it more accurate.

  9. Now you can start creating an AI Skill and add prompt inputs, as required. Let us use an example to walk you through the steps.
  10. In the System prompt and User prompt fields enter your Prompt text along with the input variables, if required.

    For example:

    System prompt: You are the code expert in Java.

    User prompt: Write a sample code to calculate area of rectangle and circle using OOD principles.

    The response for the Prompt text will be referenced from documents in the Azure portal.

  11. Click out of the prompt entry fields.
  12. Click Get response to get a response from the model based on your prompts.
    Note: Prompt data details could contain PHI, PII or other sensitive data you choose to enter in the System prompt or User prompt fields. We recommend being mindful of this when testing and executing prompts.
  13. The Grounded by AI Search Model connection returns a response in the Response field, and additionally displays a Citations field displaying all citation references.

    Citations are chunks of information stating, which section of a document stored in the Grounded by AI Search, the response is referenced from. You can see the document title of the referenced data store from the Azure OpenAI.