> ## Documentation Index
> Fetch the complete documentation index at: https://docs.retab.com/llms.txt
> Use this file to discover all available pages before exploring further.
# Get Experiment Run Metrics
> Get metrics for an experiment run.
Requires the `run_id` query parameter. Use `view` to choose the breakdown
(`summary`, `by_document`, `by_target`, or `votes`), and narrow with
`document_id` or `target_path`. By default each score-bearing row also
carries a `prior_score` from the previous completed run; pass
`include_prior=false` to omit it or `prior_run_id` to compare against a
specific run.
Get metrics for a specific experiment run. Metrics are consensus likelihoods on
the `[0.0, 1.0]` scale where `0.0` is low agreement and `1.0` is total
agreement.
The `view` query parameter selects one of four successful response shapes. In
responses, branch on `kind`; it is the shared discriminator for the response
shape. Because this endpoint is scoped to a concrete `run_id`, it returns
historical metrics for that run even if the experiment definition has since
changed. Use the experiment's `freshness` / `run_plan_mode` fields from
`experiments.get` or `experiments.list` to decide whether to create a newer run.
| View | Use it to |
| ------------- | ------------------------------------------------------------------------------------------------------------- |
| `summary` | Read the overall score plus block-specific aggregates. Start here. |
| `by_document` | Drill into one document and see all its targets, sorted ascending. Requires `document_id`. |
| `by_target` | Drill into one target and see its score across every document. Requires `target_path`. |
| `votes` | See the per-voter consensus rows for one document/target cell. Requires both `document_id` and `target_path`. |
Pass `include_prior=false` to omit prior-run comparison fields, or
`prior_run_id=...` to override which run is treated as the prior.
```python Python theme={null}
from retab import Retab
client = Retab()
summary = client.workflows.experiments.metrics.get(
"exprun_2",
view="summary",
)
target_view = client.workflows.experiments.metrics.get(
"exprun_2",
view="by_target",
target_path="line_items.*.unit_price",
)
```
```typescript TypeScript theme={null}
import { Retab } from "@retab/node";
const client = new Retab({ apiKey: process.env.RETAB_API_KEY });
const summary = await client.workflows.experiments.metrics.get({
runId: "exprun_2",
view: "summary",
});
const targetView = await client.workflows.experiments.metrics.get({
runId: "exprun_2",
view: "by_target",
targetPath: "line_items.*.unit_price",
});
```
```go Go theme={null}
package main
import (
"context"
"fmt"
"log"
retab "github.com/retab-dev/retab/clients/go"
)
func ptr[T any](v T) *T { return &v }
func main() {
ctx := context.Background()
client, err := retab.NewClient("")
if err != nil {
log.Fatal(err)
}
summary, err := client.Workflows.Experiments.Metrics.Get(
ctx,
&retab.ExperimentRunMetricsGetParams{
RunID: "exprun_2",
View: ptr(retab.ExperimentRunMetricsViewSummary),
},
)
if err != nil {
log.Fatal(err)
}
targetView, err := client.Workflows.Experiments.Metrics.Get(
ctx,
&retab.ExperimentRunMetricsGetParams{
RunID: "exprun_2",
View: ptr(retab.ExperimentRunMetricsViewByTarget),
TargetPath: ptr("line_items.*.unit_price"),
},
)
if err != nil {
log.Fatal(err)
}
fmt.Println(summary, targetView)
}
```
```ruby Ruby theme={null}
require 'retab'
client = Retab::Client.new(api_key: ENV['RETAB_API_KEY'])
summary = client.workflows.experiments.metrics.get(
run_id: 'exprun_2',
view: 'summary',
)
target_view = client.workflows.experiments.metrics.get(
run_id: 'exprun_2',
view: 'by_target',
target_path: 'line_items.*.unit_price',
)
```
```rust Rust theme={null}
use retab::enums::ExperimentRunMetricsView;
use retab::resources::experiment_run_metrics::GetParams;
use retab::Retab;
#[tokio::main]
async fn main() -> Result<(), Box> {
let client = Retab::new(std::env::var("RETAB_API_KEY")?);
let mut summary_params = GetParams::new("exprun_2");
summary_params.view = Some(ExperimentRunMetricsView::Summary);
let _summary = client.workflows().experiments().metrics().get(summary_params).await?;
let mut target_params = GetParams::new("exprun_2");
target_params.view = Some(ExperimentRunMetricsView::ByTarget);
target_params.target_path = Some("line_items.*.unit_price".into());
let _target_view = client.workflows().experiments().metrics().get(target_params).await?;
Ok(())
}
```
```php PHP theme={null}
workflows()->experiments()->metrics()->get(
runId: 'run_abc123',
);
print_r($result);
```
```csharp C# theme={null}
using Retab;
using RetabClient = Retab.Retab;
var apiKey = Environment.GetEnvironmentVariable("RETAB_API_KEY")!;
var client = new RetabClient(apiKey);
var result = await client.Workflows.Experiments.Metrics.GetAsync(new ExperimentRunMetricsGetOptions());
Console.WriteLine(result);
```
```java Java theme={null}
import com.retab.RetabClient;
public final class Example {
public static void main(String[] args) throws Exception {
RetabClient client = new RetabClient(System.getenv("RETAB_API_KEY"));
var result = client.workflows().experiments().metrics().get("run_abc123", null, null, null, null, null);
System.out.println(result);
}
}
```
```curl cURL theme={null}
curl -X 'GET' \
'https://api.retab.com/v1/workflows/experiments/metrics?run_id=exprun_2&view=summary' \
-H 'Authorization: Bearer '
curl -X 'GET' \
'https://api.retab.com/v1/workflows/experiments/metrics?run_id=exprun_2&view=by_target&target_path=total' \
-H 'Authorization: Bearer '
```
```json 200 theme={null}
{
"experiment_id": "exp_abc",
"run_id": "exprun_2",
"kind": "summary",
"view": "summary",
"block_execution_fingerprint": "deadbeef",
"block_kind": "extract",
"score": 0.83,
"prior_score": 0.79,
"prior_run_id": "exprun_1",
"documents": [
{
"id": "expdoc_1",
"filename": "a.pdf",
"score": 0.91,
"prior_score": 0.85
}
],
"aggregate": {
"likelihoods": { "total": 0.92, "vendor.name": 0.78 }
}
}
```
## OpenAPI
````yaml GET /v1/workflows/experiments/metrics
openapi: 3.1.0
info:
title: FastAPI
version: 0.1.0
servers:
- url: https://api.retab.com
security: []
paths:
/v1/workflows/experiments/metrics:
get:
tags:
- Workflows
- Workflow Experiments
summary: Get Experiment Metrics For Run
description: >-
Get metrics for an experiment run.
Requires the `run_id` query parameter. Use `view` to choose the
breakdown
(`summary`, `by_document`, `by_target`, or `votes`), and narrow with
`document_id` or `target_path`. By default each score-bearing row also
carries a `prior_score` from the previous completed run; pass
`include_prior=false` to omit it or `prior_run_id` to compare against a
specific run.
operationId: get_experiment_metrics_for_run
parameters:
- in: query
name: run_id
required: true
schema:
type: string
title: Run Id
- in: query
name: view
schema:
enum:
- summary
- by_document
- by_target
- votes
type: string
default: summary
title: View
required: false
- in: query
name: document_id
schema:
anyOf:
- type: string
- type: 'null'
title: Document Id
required: false
- in: query
name: target_path
schema:
anyOf:
- type: string
- type: 'null'
title: Target Path
required: false
- in: query
name: include_prior
schema:
type: boolean
default: true
title: Include Prior
required: false
- in: query
name: prior_run_id
schema:
anyOf:
- type: string
- type: 'null'
title: Prior Run Id
required: false
responses:
'200':
description: Successful Response
content:
application/json:
schema:
oneOf:
- $ref: '#/components/schemas/ExperimentSummaryMetricsResponse'
- $ref: '#/components/schemas/ExperimentByDocumentMetricsResponse'
- $ref: '#/components/schemas/ExperimentByTargetMetricsResponse'
- $ref: '#/components/schemas/ExperimentVotesMetricsResponse'
- $ref: '#/components/schemas/ExperimentMetricsStaleError'
- $ref: '#/components/schemas/ExperimentMetricsMissingError'
discriminator:
propertyName: kind
mapping:
summary:
$ref: '#/components/schemas/ExperimentSummaryMetricsResponse'
by_document:
$ref: '#/components/schemas/ExperimentByDocumentMetricsResponse'
by_target:
$ref: '#/components/schemas/ExperimentByTargetMetricsResponse'
votes:
$ref: '#/components/schemas/ExperimentVotesMetricsResponse'
stale_metrics:
$ref: '#/components/schemas/ExperimentMetricsStaleError'
no_metrics:
$ref: '#/components/schemas/ExperimentMetricsMissingError'
title: >-
Response Get Experiment Metrics For Run V1 Workflows
Experiments Metrics Get
'422':
description: Validation Error
content:
application/json:
schema:
$ref: '#/components/schemas/HTTPValidationError'
security:
- HTTPBearer: []
components:
schemas:
ExperimentSummaryMetricsResponse:
properties:
experiment_id:
type: string
title: Experiment Id
run_id:
type: string
title: Run Id
kind:
type: string
const: summary
title: Kind
default: summary
view:
type: string
const: summary
title: View
default: summary
block_execution_fingerprint:
anyOf:
- type: string
- type: 'null'
title: Block Execution Fingerprint
block_type:
type: string
enum:
- extract
- classifier
- split
- for_each
title: Block Type
score:
anyOf:
- type: number
- type: 'null'
title: Score
prior_score:
anyOf:
- type: number
- type: 'null'
title: Prior Score
documents:
items:
$ref: '#/components/schemas/ExperimentSummaryMetricDocument'
type: array
title: Documents
default: []
aggregate:
anyOf:
- $ref: '#/components/schemas/ExperimentExtractSummaryAggregate'
- $ref: '#/components/schemas/ExperimentConfusionSummaryAggregate'
- type: 'null'
title: Aggregate
prior_run_id:
anyOf:
- type: string
- type: 'null'
title: Prior Run Id
type: object
required:
- block_type
- experiment_id
- run_id
title: ExperimentSummaryMetricsResponse
description: |-
Run-level summary plus block-specific diagnostics.
`prior_run_id` + `prior_score` populate when the request opts into
prior-comparison and a completed prior run exists.
ExperimentByDocumentMetricsResponse:
properties:
run_id:
type: string
title: Run Id
kind:
type: string
const: by_document
title: Kind
default: by_document
view:
type: string
const: by_document
title: View
default: by_document
document:
$ref: '#/components/schemas/ExperimentMetricDocumentRef'
score:
anyOf:
- type: number
- type: 'null'
title: Score
prior_score:
anyOf:
- type: number
- type: 'null'
title: Prior Score
confusion:
anyOf:
- $ref: '#/components/schemas/ExperimentDocumentConfusionMetric'
- type: 'null'
targets:
items:
$ref: '#/components/schemas/ExperimentByDocumentTargetMetric'
type: array
title: Targets
default: []
type: object
required:
- document
- run_id
title: ExperimentByDocumentMetricsResponse
description: One document's compact per-target breakdown.
ExperimentByTargetMetricsResponse:
properties:
run_id:
type: string
title: Run Id
kind:
type: string
const: by_target
title: Kind
default: by_target
view:
type: string
const: by_target
title: View
default: by_target
target:
type: string
title: Target
score:
anyOf:
- type: number
- type: 'null'
title: Score
prior_score:
anyOf:
- type: number
- type: 'null'
title: Prior Score
confusion:
anyOf:
- $ref: '#/components/schemas/ExperimentTargetConfusionMetric'
- type: 'null'
documents:
items:
$ref: '#/components/schemas/ExperimentByTargetDocumentMetric'
type: array
title: Documents
default: []
type: object
required:
- run_id
- target
title: ExperimentByTargetMetricsResponse
description: One target's compact per-document distribution.
ExperimentVotesMetricsResponse:
properties:
run_id:
type: string
title: Run Id
kind:
type: string
const: votes
title: Kind
default: votes
view:
type: string
const: votes
title: View
default: votes
document:
$ref: '#/components/schemas/ExperimentVotesMetricDocument'
target:
type: string
title: Target
score:
anyOf:
- type: number
- type: 'null'
title: Score
prior_score:
anyOf:
- type: number
- type: 'null'
title: Prior Score
rows:
items:
$ref: '#/components/schemas/ExperimentVoteRow'
type: array
title: Rows
default: []
type: object
required:
- document
- run_id
- target
title: ExperimentVotesMetricsResponse
description: One document/target vote matrix with flat consensus rows.
ExperimentMetricsStaleError:
properties:
kind:
type: string
const: stale_metrics
title: Kind
default: stale_metrics
error:
type: string
const: stale_metrics
title: Error
default: stale_metrics
experiment_id:
type: string
title: Experiment Id
stale_reasons:
items:
type: string
type: array
title: Stale Reasons
default: []
last_run:
$ref: '#/components/schemas/MetricsStaleErrorLastRun'
current_block_execution_fingerprint:
anyOf:
- type: string
- type: 'null'
title: Current Block Execution Fingerprint
message:
type: string
title: Message
type: object
required:
- experiment_id
- last_run
- message
title: ExperimentMetricsStaleError
description: |-
Returned when the latest run's config or document set has drifted
from the current draft, so its metrics no longer reflect the
experiment's definition.
ExperimentMetricsMissingError:
properties:
kind:
type: string
const: no_metrics
title: Kind
default: no_metrics
error:
type: string
const: no_metrics
title: Error
default: no_metrics
experiment_id:
type: string
title: Experiment Id
message:
type: string
title: Message
type: object
required:
- experiment_id
- message
title: ExperimentMetricsMissingError
description: Returned when the experiment has no runs at all.
HTTPValidationError:
properties:
detail:
items:
$ref: '#/components/schemas/ValidationError'
type: array
title: Detail
default: []
type: object
title: HTTPValidationError
ExperimentSummaryMetricDocument:
properties:
id:
type: string
title: Id
filename:
type: string
title: Filename
score:
anyOf:
- type: number
- type: 'null'
title: Score
prior_score:
anyOf:
- type: number
- type: 'null'
title: Prior Score
type: object
required:
- filename
- id
title: ExperimentSummaryMetricDocument
description: Lightweight document row returned by the summary view.
ExperimentExtractSummaryAggregate:
properties:
likelihoods:
additionalProperties:
type: number
type: object
title: Likelihoods
default: {}
type: object
title: ExperimentExtractSummaryAggregate
description: Extract-only diagnostics attached to the summary response.
ExperimentConfusionSummaryAggregate:
properties:
diag:
additionalProperties:
type: number
type: object
title: Diag
default: {}
flows:
items:
$ref: '#/components/schemas/ExperimentConfusionFlowMetric'
type: array
title: Flows
default: []
type: object
title: ExperimentConfusionSummaryAggregate
description: Split/classifier diagnostics attached to the summary response.
ExperimentMetricDocumentRef:
properties:
id:
type: string
title: Id
filename:
type: string
title: Filename
type: object
required:
- filename
- id
title: ExperimentMetricDocumentRef
description: Document identity used by compact metric responses.
ExperimentDocumentConfusionMetric:
properties:
diag:
additionalProperties:
type: number
type: object
title: Diag
default: {}
flows:
items:
$ref: '#/components/schemas/ExperimentConfusionFlowMetric'
type: array
title: Flows
default: []
type: object
title: ExperimentDocumentConfusionMetric
description: Document-local confusion summary for split/classifier blocks.
ExperimentByDocumentTargetMetric:
properties:
path:
type: string
title: Path
score:
anyOf:
- type: number
- type: 'null'
title: Score
prior_score:
anyOf:
- type: number
- type: 'null'
title: Prior Score
value:
anyOf:
- {}
- type: 'null'
title: Value
type: object
required:
- path
title: ExperimentByDocumentTargetMetric
description: Compact target score returned by the per-document metrics view.
ExperimentTargetConfusionMetric:
properties:
self:
anyOf:
- type: number
- type: 'null'
title: Self
flow_from:
additionalProperties:
type: number
type: object
title: Flow From
default: {}
flow_to:
additionalProperties:
type: number
type: object
title: Flow To
default: {}
type: object
title: ExperimentTargetConfusionMetric
description: Directional confusion slice for one split/classifier target.
ExperimentByTargetDocumentMetric:
properties:
id:
type: string
title: Id
filename:
type: string
title: Filename
score:
anyOf:
- type: number
- type: 'null'
title: Score
prior_score:
anyOf:
- type: number
- type: 'null'
title: Prior Score
value:
anyOf:
- {}
- type: 'null'
title: Value
type: object
required:
- filename
- id
title: ExperimentByTargetDocumentMetric
description: Compact document score returned by the per-target metrics view.
ExperimentVotesMetricDocument:
properties:
id:
type: string
title: Id
filename:
type: string
title: Filename
type: object
required:
- filename
- id
title: ExperimentVotesMetricDocument
description: Document identity for the vote matrix response.
ExperimentVoteRow:
properties:
consensus:
anyOf:
- {}
- type: 'null'
title: Consensus
votes:
items: {}
type: array
title: Votes
default: []
score:
anyOf:
- type: number
- type: 'null'
title: Score
row_presence_score:
anyOf:
- type: number
- type: 'null'
title: Row Presence Score
present_voter_count:
anyOf:
- type: integer
- type: 'null'
title: Present Voter Count
total_voter_count:
anyOf:
- type: integer
- type: 'null'
title: Total Voter Count
type: object
title: ExperimentVoteRow
description: Consensus and flat voter values for one selected target row.
MetricsStaleErrorLastRun:
properties:
run_id:
type: string
title: Run Id
block_execution_fingerprint:
anyOf:
- type: string
- type: 'null'
title: Block Execution Fingerprint
score:
anyOf:
- type: number
- type: 'null'
title: Score
created_at:
anyOf:
- type: string
format: date-time
- type: 'null'
title: Created At
type: object
required:
- run_id
title: MetricsStaleErrorLastRun
ValidationError:
properties:
loc:
items:
anyOf:
- type: string
- type: integer
type: array
title: Location
msg:
type: string
title: Message
type:
type: string
title: Error Type
input:
title: Input
default: null
ctx:
type: object
title: Context
default: {}
type: object
required:
- loc
- msg
- type
title: ValidationError
ExperimentConfusionFlowMetric:
properties:
source:
type: string
title: Source
target:
type: string
title: Target
score:
type: number
title: Score
type: object
required:
- score
- source
- target
title: ExperimentConfusionFlowMetric
description: One non-diagonal confusion flow between two labels.
securitySchemes:
HTTPBearer:
type: http
scheme: bearer
````