Licensing model for AI Evaluations - Understand how AI Evaluations consume credits based on token usage across AI Skills and AI Agents evaluation scenarios.

Licensing model for AI Evaluations

Automation AI Credits

Automation AI Credits are consumed when an automatic AI Evaluations completes successfully. Manual evaluations do not consume credits.

Larger datasets can trigger more model calls or AI Agent executions, resulting in higher token and credit consumption.

Key interpretation

• Token-based billing model:

  AI Evaluations consumption is driven entirely by token usage (input + output) rather than execution count.

• Cost hierarchy:

  • AI Skills < AI Agents

  • Input tokens < Output tokens

• Why the difference:

  • AI Skills evaluations typically involve single-step model responses

  • AI Agents evaluations involve multi-step reasoning, tool use, and orchestration, increasing token utilization and cost.

• Practical implication:

  The total credit consumption for an evaluation can be estimated as:

  Total Credits = 
    (Input Tokens × Input Rate) + 
    (Output Tokens × Output Rate)
  

  Rates vary by evaluation type (Skills vs Agents).

• Also consumes credits for Agent executions when executed from public workspace, and credits consumed for using out of the box model connections. This consumption varies based on the size of the dataset.

Grace policy

If purchased credits are exhausted, up to 10% additional credits can be consumed as a grace buffer to prevent disruption of evaluation workflows.

Visibility