Using AI governance dashboard widgets
- Updated: 2024/09/26
Using AI governance dashboard widgets
Enhance your AI governance dashboard experience by reviewing data for Top Automations using AI, Most used models, and Token consumption widgets to view, monitor, and analyze AI data derived from model interactions during automation development and automations executions across your enterprise.
Permissions
Top Automations using AI widget
This dashboard widget displays a bar graph showing the top ten frequently run automations that uses AI interactions. You can see the name of the top ten automations and their number of executions in the Last 24 hours. Hovering over an automation bar in the widget shows you a blurb with the automation name and the total number of execution runs.
Drill-down:
Click a bar graph to drill-down and view additional details. Clicking an automation bar navigates you to the tab in the Control Room where you can view a list of all the executions the automation ran listed by sessions. Clicking each sessions displays the details. You can change the time filter in this screen.
See, Monitor AI prompt log interactions.
Most used models widget
The GRC Lead and Automation Lead would use this widget to monitor and track the Model utilization for the Most used models within automations to effectively manage subscriptions. This will help them implement the use of approved and supported models from the hyperscaler vendors.
The Most used models widget gives visibility into the most popularly used top five models within automations using the Generative AI Package and AI Skills Packageto execute automations in the Production environment.
- The automation name with the package used, whether it used the Generative AI Packages or the AI Skills package
- The model used from the hyperscaler vendor, for example: GPT-4 from OpenAI
Hover over a bar to see a summary of the details.
Drill-down:
Click a bar graph to drill-down and view additional details in addition to the entire list of the most used models from hyperscaler vendors. Clicking the bar navigates you to the tab in the Control Room where you can view a list of logs filtered by the model name. The logs are displayed by Event type. Click an Event type to view additional details for the model interactions that took place within the automation execution.
See Monitor Event logs.
- In addition to the drill-down option from the bar graph, click the ellipses at the top-right corner to select View details for a detail view.
- The AI governance dashboard screen displays the Model utilization tab details.
The Model utilization details view for the Most used models displays the top five automations in the visual, in addition to the entire list of automation details in the table as follows. You can change the time filter in this screen.
Export reports based on the data represented in the widget by clicking or on your screen.
| Column header | Description |
|---|---|
| Model name | This is the model used in a specific automation such as
GPT-4 from OpenAI. You can click
the model to go to tab in the Control Room
where you can view all logs filtered by the model
name. See Monitor Event logs. |
| Publisher | Supported hyperscaler vendor such as Amazon Bedrock, Google Vertex AI, Azure OpenAI, or OpenAI. |
| Automations | Number of generative AIcommand actions used in automation executions using the Generative AI Package and AI Skills Package. |
| AI Skills | Number of calls automations executed using AI Skills. |
| Model calls | Number of calls made to the model. |
| Devices | Number of devices used during the model interaction. |
Token consumption widget
The Token consumption widget provides visibility into your top automations consuming most number of tokens aggregated per model. To derive the total count for a model's Token consumption, the metrics is calculated based on the Automation name, Folder path, and the model name from the hyperscaler vendor. Hover over a bar in the widget to see the Token consumption details of a specific automation.
By default, the Token consumption widget shows data for the Last 24 hours which can be changed to a different time filter in the widget details screen.
The GRC Leads and Automation Leads would use this widget to monitor and track the token usage of the models during their interactions and execution within automations to effectively manage cost and subscription. This will help them project Token consumption based on model preference.
- Navigate to the ellipses at the top-right corner of each widget and click to select View details for a detail view.
- The AI governance dashboard screen displays the Token consumption tab details.
The Token consumption details view displays the top five automations in the visual, in addition to the entire list of automation details in the table as follows. You can change the time filter in this screen.
Export reports based on the data represented in the widget by clicking or on your screen.
| Column header | Description |
|---|---|
| Automation name | Name of the automation that was executed. You can click the name to go to tab in the Control Room where you can view all logs filtered by the automation name. |
| Folder path | The folder location of the automation from where it was executed. |
| Publisher | Supported hyperscaler vendor such as Amazon Bedrock, Google Vertex AI, Azure OpenAI, or OpenAI. |
| Model name | This is the model used in a specific automation such as GPT-4 from OpenAI. |
| Users | Number of users. |
| Workspace | This can be the Private or Public folder from where the automation was executed. |
| AI Skills | Number of AI Skills used in the automation. |
| Tokens | Total token count used by the model. |
You can export reports based on the data represented in the widget by clicking or on your screen.