AI prompt log provides a consolidated summary view of all sessions occurring within an automation execution for model interactions.

There could be interactions with multiple models within a session. Every automation execution is assigned a Session ID which allows aggregation of all the events into a single session ID.

The AI governance > AI prompt log > Session ID link in the Session ID column lets you to drill-down and view all model interaction details that occurred within a session for that Session ID. The detailed view includes prompts, responses, and other parameters exchanged with the foundational models. You can refine your search by Session ID to view additional details specific to the selected session.

Note: Use your cursor to roll over the action icons to identify specific functions for each.

AI prompt log table details

The AI prompt log table sorts and displays details as per these columns:

AI prompt log updates

one
Duration: Shows the start and end time of a session execution.
Note: This information is not available if the session is in progress.
two

Start time: Shows the start time of the session execution.
three

Session ID: Shows the session ID for an automation execution. A session can consist of multiple model interactions, which are part of the same automation execution. Click the Session ID link to view details of all model interactions for that session.
four

Source: Shows where in the product this prompt was entered.
five
Prompt type: Shows the type of prompt used in the automation. The types include:
  • Free Form (when Generative AI packages are used).
  • AI Skill ( when AI Skills packages are used.)
six

Automation name: Shows the automation name that used the prompt.
seven
Device name: Shows ID of the device on which the automation was executed.
Note: This information is not available if the action was performed in the Control Room.
eight

User: Shows the ID of the user who ran the automation.
nine

AI guardrail: Shows the specific Guardrail applied to each AI session.
ten

Folder path: The folder location of the automation from where it was executed. When an AI Skill is integrated with an automation (parent or child), the folder path displayed is the same as the folder path of the parent automation.

AI prompt log Session and model interaction details

From the AI prompt log table, you can click the Session ID link to drill-down further to view details of a session.

You can view the additional model interaction session details in the same screen by expanding the foundational model name displayed for that interaction. Click the prompt or action in the details section to view more details of the prompt or model responses.

Note: The Automation type field shows the model-type for which the foundational model was used in the model interaction session. The remaining fields are same as the AI prompt log table details.

Model Interaction details

Table item Description
AI guardrail assigned Displays the specific Guardrail applied to each AI session.
Toxicity Displays the toxicity level of the prompt and response.
Prompt Shows the prompt-text user entered in the User prompt field when defining the prompt.
Response Shows the response received from the model, for the prompt-request.
When a prompt is blocked by an AI guardrail, this event is recorded in the AI prompt log. The logs include details such as the prompt content, the assessed toxicity level, and the action taken. It is important to note that if a prompt is blocked by the guardrail policy, it does not reach the LLM.
  • Allowed Prompts: If the prompt's toxicity score falls within acceptable limits, the log will include the model's response, with any sensitive data masked. The response's toxicity score and the guardrail action taken on the response will also be recorded.
  • Blocked Prompts: If a prompt is blocked by a guardrail (before reaching the LLM) or due to its toxicity score, the log will explicitly state the guardrail action taken, such as Blocked by guardrail.
  • Blocked Responses: If the model's generated response is flagged by a guardrail, the log will show the masked response, its toxicity score, and the guardrail action taken on it, such as Blocked by guardrail.

Typically, the Automation Lead and the GRC Lead would have permission to view the additional details for model interactions in each session, as they monitor the logs for compliance and governance to ensure policies are adhered to when using foundation models within automations.

Click More details to view additional parameters for the model interaction:
Note: This additional drill-down is available to Pro Developers only if the relevant permission is enabled by the Automation Admin ( Administration > Roles > AI governance > View AI prompt logs > View AI prompt details). Else they are able to view only the first level session details.
Table item Description
Session duration Shows the start and end times.
Publisher Shows the foundational model provider name.
Model Shows the foundational model version name such as: GPT 3.5 Turbo for Azure OpenAI, VertexAI for Google Vertex AI, Claude for Amazon Bedrock and others.
External session ID Shows the session ID of a specific model interaction captured within a session. A single session can consist of multiple model interactions within a single automation execution. You can use this information to keep track of every model interaction occurring in a session.
Response configurations Shows the parameter values used for the prompt, such as: Temperature, Top P, Max Tokens, Presence Penalty and others. These parameters and their values vary based on the model provider.
Response attributes Shows the attributes with the values from the response.
Model Connection Type Specifies the type of model connection. The Model Connection Type can be one of the following:
  • Standard
  • Fine tuned
  • Grounded by knowledge base
  • Grounded by data store
  • Grounded by enterprise knowledge
  • Grounded by AI search
Tip: For an automation execution, with multiple model interactions, you would see a separate expandable panel for each model call in the session log . This is a typical use case for automations executed for generative AI Packages.