Review what's new and changed, and the fixes and limitations in AI Agent Studio for the v.36 release.

What's new

AI Guardrails

This release introduces AI Guardrails, a new feature protecting sensitive data and promoting responsible AI usage. AI Guardrails protect sensitive data by masking PII, PHI, and PCI within prompts sent to LLMs. Additionally, they monitor prompts and responses for potentially harmful language. Administrators can configure data masking rules and assign them to specific folders, ensuring consistent data protection across automations.

Important: The AI Guardrails offering is available on Automation 360 Cloud, and will be available for use with AI Guardrail (Number of LLM Prompts) license along with the Enterprise Platform license. For details about this license, see Enterprise Platform.

AI Guardrails
System prompt in AI Skills

Within AI Skills, a new feature called System Prompt will be introduced. This optional field, visible to anyone with access to view or edit the AI Skill, allows you to provide initial instructions or context to the underlying model. The System prompt helps to orient or ground the model, influencing its responses to user prompts and ensuring more relevant and accurate outputs. The System prompt field is displayed for all models. For models that support System prompt, the provided text will be sent as a System prompt. For models that do not support System prompts, the text provided in the System prompt field will be placed before the User prompt.

System prompt
Enable or disable AI Guardrails policies globally within the Control Room

Administrators can now enable the AI Guardrails in the Control Room with a cloud license by navigating to Administration > Settings > AI Guardrails.

When this setting is enabled, Professional Developers can create AI Guardrails to enforce rules for protecting sensitive information and promote responsible AI usage.

Assign roles and permission to enable AI Guardrails
Support for RAG capability in AI Agent Studio to create grounded Model connections using Azure OpenAI Grounded by AI Search

Create Grounded by AI Search Model connections using the native RAG (retrieval-augmented generation) capability from Azure OpenAI to retrieve information from Azure AI Search indexes for more accurate and relevant responses.

Create Grounded Model connections with Azure OpenAI RAG capability
Test with AI guardrail in AI Skills

Enhance the reliability and safety of your AI Skills by testing them against AI Guardrails before deployment. This new feature allows you to simulate the effects of your chosen guardrails on prompts and model responses. This update provides toxicity and data masking results. You can click the view data masking execution option to reveal the tokenized masked values of sensitive data. Gain valuable insights into how your AI Skill will perform in a production environment with guardrails enabled. Access this functionality within the AI Skills editor and review the AI Governance logs for a detailed record of each test.

Create AI Skills
Custom model definitions - Authenticate with AWS Signature authentication

This release introduces support for AWS Signature authentication when defining custom models in AI Agent Studio. This enhancement allows you to seamlessly integrate with a wider range of AWS-based AI/ML services that use this authentication method. You can now set the authType to AWS SIGNATURE when defining a new custom model. For example: "authType" : "AWS_SIGNATURE_V4". This will require providing the necessary AWS credentials - Access Key ID, Secret Access Key, Session Key (optional), Region.

For example:

{
   "name":"Bedrock - Claude2.1",
   "description":"string",
   "version":"string",
   "auth_action":{
      "auth_type":"AWS_SIGNATURE_V4",
      "awsSignatureV4":{
         "accessKey":{
            "location":"header",
            "keyName":"aws_sign_access_key"
         },
         "secretkey":{
            "location":"header",
            "keyName":"aws_sign_access_key"
         },
         "sessionkey":{
            "location":"header",
            "keyName":"aws_sign_session_key"
         }
      }
   },
   "api_type":"REST",
   "actions":[
      {
         .... 
      }
   ]
}

Custom model definitions
Custom model definitions - Update existing model

This release introduces the ability to modify existing custom model definitions using the newly introduced PUT and PATCH methods. Previously, modifications to an existing custom model were not supported.

PUT (Replace/Update entire model): Use the PUT endpoint to completely replace or update an existing custom model definition. This method is used when you need to update multiple properties of the model. 
PUT https://{ControlRoom}/gai/prompttools/v1/custommodel
Note: This action is restricted to custom models that have no associated Model connections.
PATCH (Partial update - Name and Description): Use the PATCH endpoint to update only the name and description of an existing custom model definition. 
PATCH https://{ControlRoom}/gai/prompttools/v1/custommodel

Both endpoints require the vendorName and modelName to identify the custom model to be updated and will return a 200 OK response with the updated model details upon successful execution.

Custom model definitions
Custom model definitions - List custom models API, can be filtered by vendor name

This release introduces the ability to filter the list of custom models by vendor name and sort the results based on specific criteria.

POST https://{ControlRoom}/gai/prompttools/v1/custommodel/list
Request body:
{
   "sort":[
      {
         "field":"name",
         "direction":"asc"
      }
   ],
   "filter":{
      "operator":"and",
      "operands":[
         {
            "operator":"eq",
            "field":"name",
            "value":"CustomVendor"
         }
      ]
   }
}

AI Agent Studio API
Support for RAG capability in AI Agent Studio to create grounded Model connections using Azure OpenAI Grounded by AI Search

Create Grounded by AI Search Model connections using the native RAG (retrieval-augmented generation) capability from Azure OpenAI to build rich search experiences that combine large language models with enterprise data from Azure AI Search.

Create Grounded Model connections with Azure OpenAI RAG capability
Introducing GenAI model calls widget

The new GenAI model calls widget within the AI Governance dashboard in the home screen displays the top 5 GenAI models used in the automations. Clicking on a model name drills down to the AI Governance Event Log, pre-filtered for that model.

Using AI Governance dashboard widgets

What's changed

Custom Model Integration Enhancements (Service Cloud Case ID: 02147228)

AI Agent Studio now supports the integration of custom large language models (LLMs) that you have developed or deployed. This feature now includes APIs for full lifecycle management of these custom models, including defining, retrieving, modifying, and removing them, giving you the ability to use specialized AI models within the AI Agent Studio.

Custom model definitions
Search and add roles in create Model connections wizard

You can now search and add roles while creating Model connections.

Create and manage Model connections
Enhanced AI Governance Logging for Parent/Child Bot Execution

AI Governance logs now provide enhanced visibility into bot execution by accurately reflecting parent bot details when initiating child bots, applicable to both Generative AI Command packages and automations leveraging AI Skills. Previously, logs displayed child bot information, creating inconsistency. This enhancement ensures logs now include the parent bot ID, name, and folder path for improved audit trails. For example, if parent bot ID 103 triggers a child bot, logs will correctly show 103, not the child bot's ID. This improvement impacts both prompt and event logs within AI Governance.

AI Governance
AI Governance AI prompt log and Event log tabs: New AI guardrail column

A new column, AI guardrail, has been added to the AI Governance AI prompt log and Event log screens. A new field AI guardrail is also included within the session details or events when you open individual prompt or event logs. This column provides information about the specific guardrails applied to each prompts and events.

Monitor AI prompt log interactions | Monitor Event log interactions
Enhancement to AI Governance AI prompt log : New Toxicity analysis and System prompt/User prompt
  1. Overall Toxicity: Toxicity scores are now displayed for both the system and user prompts.
  2. Response Toxicity: The toxicity level of the LLM's response is also included, enabling better evaluation of generated content.
  3. Dual prompt display: Both the System prompt and User prompt are now clearly shown. This clarifies the context of the interaction, especially with the introduction of system prompts alongside user prompts. The previous Prompt field has been renamed to User prompt.
  4. Model Connection Type: The AI prompt log now displays the Model Connection Type detailing how models are connected, such as Standard, Fine tuned or various grounded options.

Monitor AI prompt log interactions
Enhancement to AI Governance Event log : New Toxicity analysis and System prompt/User prompt
  1. Overall Toxicity: Toxicity scores are now displayed for both the system and user prompts.
  2. Response Toxicity: The toxicity level of the LLM's response is also included, enabling better evaluation of generated content.
  3. Dual prompt display: Both the System prompt and User prompt are now clearly shown. This clarifies the context of the interaction, especially with the introduction of system prompts alongside user prompts. The previous Prompt field has been renamed to User prompt.
  4. Model Connection Type: The Event log now displays the Model Connection Type detailing how models are connected, such as Standard, Fine tuned or various grounded options.

Monitor Event log interactions

Fixes

The Anthropic Claude 2.1 and Anthropic Claude 3.5 models, which are not supported on Amazon Bedrock as a fine-tuned models, have been removed from the available model options within the Amazon Bedrock fine-tuned types. Previously, these models were incorrectly displayed in the model selection when selecting Fine-tuned as the type under Amazon Bedrock.
Fix for an issue where AI Governance logs were not supported for API Tasks run in real-time mode (attended automation). This limitation has been resolved in this release.

Limitations

Assigning folders to AI Guardrails
  • AI Guardrails can only be assigned to parent folders. Child folders within the parent folder do not inherit or have specific guardrail assignments.
  • When a parent bot invokes a child bot residing in a different folder, the guardrail policy associated with the parent bot's folder will be applied.
When clicking on a model name containing special characters (such as "*", "?", "+", "=", "&&", "||", ">", "<", "!", "(", ")", "{", "}", "[", "]", "^", "~", ":", or words enclosed within double quotes) within the AI Governance dashboard in the home screen of the Most used models widget, an error message is displayed similar to the following screen shot. This occurs because the search functionality within the AI Governance dashboard does not currently support these special characters.

Error Message when clicking Model name containing special characters
The Folder field within the File Properties of an AI Skill or an AI Skill template does not currently function as intended. Changing the folder location in this screen does not actually move the AI Skill or the AI Skill template to the specified location when the changes are saved.
Test with AI guardrail is unavailable for System prompt in AI Skills

Test with AI guardrail is currently not support for system prompts within the AI Skills editor. While guardrails will function correctly for system prompts when used within automations and executed at run time. This means that you cannot directly test the application of guardrails to system prompts during the AI Skill development phase. This limitation only affects the testing with AI guardrail.
AI Skill API Task logging issue

AI Skill executions triggered via on-demand API Tasks do not log responses to the AI Governance Event log. Adding a Delay action of at least 1 second after the AI Skill: Execute action for all on-demand API Task executions is a current temporary solution to resolve this logging issue.


AI Skill Execution Logging Issue with On-Demand API Tasks