AI Agent Studio v.37 release

Review what's new and the fixes and limitations in AI Agent Studio for the v.37 release.

What's new

New Skills page for AI Skills management

Introducing the new Skills page, found under the AI section in the left navigation. This centralized view allows you to easily access and manage all your AI Skills. The page is split into two sections: the Skills tab for managing your existing skills and the Templates tab to browse and create skills from available templates, offering an alternative to the previous creation method via the Automation.

Fähigkeiten

Enhanced AI Guardrails with toxicity monitoring and blocking

The AI Guardrails creation wizard now inclues a dedicated Toxicity section. This section enables administrators to control language in AI interactions. This enhancement analyzes both input prompts and LLM-generated responses, classifying toxicity levels as low, moderate, or high. Going beyond simple monitoring, administrators can now configure guardrails to actively block prompts containing specified levels of toxic language before they reach the LLM. Similarly, guardrails can block LLM responses exceeding defined toxicity thresholds before presenting them to the user. Blocked prompts trigger error messages and prevent LLM processing. Blocked responses result in empty outputs and error notifications. The system meticulously logs all blocking actions for comprehensive AI Governance. By default, toxicity checks are set to Allow all for audit purposes, and the system records the toxicity score in AI Governance in observe only mode.

AI Guardrails

AI Guardrails Testing

The AI Guardrails creation wizard now features a dedicated Testing section, enabling you to test the guardrails with the configured settings. You can now input sample prompts and click Test guardrail to see how the guardrail responds based on configured toxicity/data masking settings.

AI Guardrails erstellen und verwalten

Source of prompts displayed in AI Governance

The source for each prompt is displayed in the prompt and event logs in AI Governance. This offers admin more insight into how AI capabilities are being used and the performance of various products.

Fixes

AI Skill API Task logging issue fixed:

AI Skill executions triggered via on-demand API Tasks now correctly log responses to the AI Governance Event log. Previously, a limitation prevented these responses from being logged; the previous temporary workaround, which requires adding a delay action of at least 1 second after the AI Skill: Execute action for all on-demand API Task executions, is no longer necessary.

Special character limitation in AI Governance dashboard fixed:

When clicking on model names containing special characters within the AI Governance dashboard tab in the home screen now triggers an exact search for that specific model name. Previously, the underlying search functionality did not fully support special characters such as "*", "?", "+", "=", "&&", "||", ">", "<", "!", "(", ")", "{", "}", "[", "]", "^", "~", ":", or words enclosed within double quotes. Consequently, clicking on a model name containing any of these characters would result in an error.

Limitations

Limited visibility of System prompt toxicity failures

The AI Guardrails can block highly toxic content, but there are some limitations specifically for the System prompt, which will be addressed in future releases. These known limitations include:

  • When the System prompt fails due to toxicity, the Send prompt event in the logs does not explicitly show this failure, as it combines the display of user and system prompts.
  • Furthermore, AI Governance logs lack a dedicated error field for Send prompt failures, leading to error messages being embedded within the User prompt, which can be confusing. Similarly, Session logs concatenate System prompt error messages with the User prompt, potentially creating a poor experience, especially with longer prompts.
Using RAG Model connections

System prompt are not considered when using RAG based Model connections. Additionally, the data masking is not applied when a RAG model interacts with data from a knowledge base.

AI Skills landing page displays an error message when no skills are available

Admins with permissions for Model connections and AI Skills will see a File not found error when accessing the AI Skills page if no skills have been configured.

Guardrails landing page File not found error fixed

You can now successfully view the Guardrails landing page, even if you do not have View content permissions for every folder where Guardrails are created or assigned by other users. Previously, you might have encountered a File not found error on this page if you lacked View content access to any folder that has a Guardrail assigned, which incorrectly impeded general navigation and management of Guardrails.

This fix ensures you can access Guardrails relevant to your permissions. The Folders assigned to column on this page will now only display folders for which you have View content permissions. You also require a View AI Guardrail permission to access this page. Users with the AAE_Admin role will continue to have implicit permissions to view all assigned folders and Guardrails, providing comprehensive oversight.

Updates to the interface

Skills
A new navigation menu is introduced to access AI Skills:

New navigation to access AI Skills

Fähigkeiten