Monitor AI prompt log interactions

AI prompt log provides a consolidated summary view of all sessions occurring within an automation execution for model interactions.

There could be interactions with multiple models within a session. Every automation execution is assigned a Session ID which allows aggregation of all the events into a single session ID.

The AI governance > AI prompt log > Session ID link in the Session ID column lets you to drill-down and view all model interaction details that occurred within a session for that Session ID. The detailed view includes prompts, responses, and other parameters exchanged with the foundational models. You can refine your search by Session ID to view additional details specific to the selected session.

Note: Use your cursor to roll over the action icons to identify specific functions for each.

AI prompt log table details

The AI prompt log table sorts and displays details as per these columns:

AI prompt log table details

one
Duration: Shows the start and end time of a session execution.
Note: This information is not available if the session is in progress.
two

Start time: Shows the start time of the session execution.

three

Session ID: Shows the session ID for an automation execution. A session can consist of multiple model interactions, which are part of the same automation execution. Click the Session ID link to view details of all model interactions for that session.

four

Prompt type: Shows the type of prompt used in the automation. For example, Free Form is relevant when Generative AI packages are used or Prompt Template is relevant when Generative AI Prompt Template packages are used.

five

Automation name: Shows the automation name that used the prompt.

six
Device name: Shows ID of the device on which the automation was executed.
Note: This information is not available if the action was performed in the Control Room.
seven

User: Shows the ID of the user who ran the automation.

eight

Folder path: The folder location of the automation from where it was executed.

AI prompt log session details

From the AI prompt log table, you can click the Session ID link to drill-down further to view details of a session.

AI prompt log Session ID details

Note: The Automation type field shows the model-type for which the foundational model was used in the model interaction session. The remaining fields are same as the AI prompt log table details.

Model interaction details

You can view the additional model interaction session details in the same screen by expanding the foundational model name displayed for that interaction. Click the prompt or action in the details section to view more details of the prompt or model responses.

AI prompt log additional session details

Table item Description
Prompt Shows the prompt-text user entered in the Prompt field when defining the prompt.
Response Shows the response received from the model, for the prompt-request.

Typically, the Automation Lead and the GRC Lead would have permission to view the additional details for model interactions in each session, as they monitor the logs for compliance and governance to ensure policies are adhered to when using foundation models within automations.

Click More details to view additional parameters for the model interaction:
Note: This additional drill-down is available to Pro Developers only if the relevant permission is enabled by the Automation Admin ( Administration > Roles > AI governance > View AI prompt logs > View AI prompt details). Else they are able to view only the first level session details.
Table item Description
Session duration Shows the start and end times.
Vendor Shows the foundational model provider name.
External session ID Shows the session ID of a specific model interaction captured within a session. A single session can consist of multiple model interactions within a single automation execution. You can use this information to keep track of every model interaction occurring in a session.
Parameters Shows the parameter values used for the prompt, such as: Temperature, Top P, Max Tokens, Presence Penalty and others. These parameters and their values vary based on the model provider.
Action Shows the model provider action used in the prompt.
Model Shows the foundational model version name such as: GPT 3.5 Turbo for Azure OpenAI, VertexAI for Google Vertex AI, Claude for Amazon Bedrock and others.
Line number Shows the line number where this action occurred in the automation. Use this information to quickly identify the action in the automation.

For an automation execution, with multiple model interactions, you would see a separate expandable panel for each model call in the session log . This is a typical use case for automations executed for generative AI Packages.

AI prompt log session with multiple model interaction logs