Monitor AI prompt log interactions
- Updated: 2024/09/26
Monitor AI prompt log interactions
AI prompt log provides a consolidated summary view of all sessions occurring within an automation execution for model interactions.
There could be interactions with multiple models within a session. Every automation execution is assigned a Session ID which allows aggregation of all the events into a single session ID.
The link in the Session ID column lets you to drill-down and view all model interaction details that occurred within a session for that Session ID. The detailed view includes prompts, responses, and other parameters exchanged with the foundational models. You can refine your search by Session ID to view additional details specific to the selected session.
AI prompt log table details
The AI prompt log table sorts and displays details as per these columns:
 |
Duration: Shows the start and end time of a session
execution.
Note: This information
is not available if the session is in
progress. |
 |
Start time: Shows the start time of the session execution. |
 |
Session ID: Shows the session ID for an automation execution. A session can consist of multiple model interactions, which are part of the same automation execution. Click the Session ID link to view details of all model interactions for that session. |
 |
Prompt type: Shows the type of prompt used in the automation. For example, Free Form is relevant when Generative AI packages are used or Prompt Template is relevant when Generative AI Prompt Template packages are used. |
 |
Automation name: Shows the automation name that used the prompt. |
 |
Device name: Shows ID of the device on which the
automation was executed.
Note: This
information is not available if the action was performed
in the Control Room. |
 |
User: Shows the ID of the user who ran the automation. |
 |
Folder path: The folder location of the automation from where it was executed. |
AI prompt log session details
From the AI prompt log table, you can click the Session ID link to drill-down further to view details of a session.
Model interaction details
You can view the additional model interaction session details in the same screen by expanding the foundational model name displayed for that interaction. Click the prompt or action in the details section to view more details of the prompt or model responses.
| Table item | Description |
|---|---|
| Prompt | Shows the prompt-text user entered in the Prompt field when defining the prompt. |
| Response | Shows the response received from the model, for the prompt-request. |
Typically, the Automation Lead and the GRC Lead would have permission to view the additional details for model interactions in each session, as they monitor the logs for compliance and governance to ensure policies are adhered to when using foundation models within automations.
| Table item | Description |
|---|---|
| Session duration | Shows the start and end times. |
| Vendor | Shows the foundational model provider name. |
| External session ID | Shows the session ID of a specific model interaction captured within a session. A single session can consist of multiple model interactions within a single automation execution. You can use this information to keep track of every model interaction occurring in a session. |
| Parameters | Shows the parameter values used for the prompt, such as: Temperature, Top P, Max Tokens, Presence Penalty and others. These parameters and their values vary based on the model provider. |
| Action | Shows the model provider action used in the prompt. |
| Model | Shows the foundational model version name such as: GPT 3.5 Turbo for Azure OpenAI, VertexAI for Google Vertex AI, Claude for Amazon Bedrock and others. |
| Line number | Shows the line number where this action occurred in the automation. Use this information to quickly identify the action in the automation. |
For an automation execution, with multiple model interactions, you would see a separate expandable panel for each model call in the session log . This is a typical use case for automations executed for generative AI Packages.
