Experiments

What are Experiments?

Experiments are controlled, block-level evaluations for workflow blocks. They run the same block with multiple consensus passes over the same set of documents and use agreement between those passes as the quality signal. Use experiments when you want to answer questions like:

Did this schema change make extraction fields more stable?
Which invoice documents are causing low agreement?
Which split category or classifier category is ambiguous?
Did a prompt, category, or split-definition change improve the block?

Experiments do not require ground-truth labels. They are consensus evals: Retab asks the block to produce several independent candidate outputs, compares those candidate outputs, and reports where the candidates agree or disagree. A higher score means stronger agreement. A low score points to an unstable document, field, category, subdocument type, or split-by-key partition.

Experiments vs Evals

Evals and experiments both help you keep workflow changes under control, but they answer different questions.

Tool	Best for	Signal
Evals	Checking a specific expected output	Pass, fail, or blocked against an assertion
Experiments	Measuring output stability across documents	Consensus score and disagreement details

Use an eval when you know what the output should be. Use an experiment when you want to find weak spots, compare block configurations, or inspect whether a block is internally consistent before you write stricter assertions.

Supported Blocks

Experiments are currently supported for:

Block	What Retab measures
Extract	Field-level agreement for extracted JSON values
Split	Agreement on subdocument/page assignments
Classifier	Agreement on routing category decisions
For Each	Key-level agreement when the block is configured as split-by-key

Other workflow blocks can still be covered with workflow evals, but they do not currently produce experiment metrics.

How Experiments Work

An experiment is attached to one block in one workflow. It stores:

The block under test - the workflow block id and block kind.
A fixed document set - materialized block inputs captured from completed workflow runs or from files uploaded while creating the experiment.
A consensus count - 3, 5, or 7 independent passes.

When you run the experiment, Retab freezes the current block configuration and replays the selected block for each document. Each document execution becomes an experiment result under the parent experiment run. The result stores the canonical artifact produced by the block:

Block	Artifact
Extract	Extraction
Split	Split
Classifier	Classification
For Each split-by-key	Partition

Experiment-run metrics are normalized across the same shape for every supported block:

document x target x voter

The target depends on the block type:

Block	Target
Extract	Field
Split	Subdocument
Classifier	Category
For Each split-by-key	Key

This lets the dashboard show the same core views for different block types: overall summary, by-document scores, by-target scores, and voter-level disagreement.

Creating an Experiment

Open a workflow in the dashboard.
Go to Console -> Experiments.
Click Experiment.
Name the experiment.
Choose the number of consensus passes: 3, 5, or 7.
Select the block to evaluate.
Select files from completed runs, or upload files for the experiment.
Create the experiment.

When you create an experiment from the dashboard, Retab creates the experiment definition and immediately starts an experiment run for it. Metrics are always scoped to that run, so the experiment page updates while the run is pending or running.

Reading Results

The experiment detail page has three sections:

Section	Purpose
Config	Review or edit the block configuration being evaluated.
Data	Inspect per-document outputs and the underlying artifacts.
Metrics	Analyze experiment-run scores, weak targets, document-level failures, and voter disagreements.

The metrics views help you move from broad signal to specific evidence:

Summary shows the overall experiment score, target averages, document averages, and previous-run delta when available.
By document shows which files are least stable.
By target shows which fields, categories, subdocuments, or keys are least stable across the document set.
Votes shows the individual candidate outputs from an experiment run for one document-target cell, including the agreed value and disagreements.

Split and classifier experiments also expose specialized visualizations, such as confusion-style views, to make routing and page-assignment ambiguity easier to inspect.

Staleness and Re-runs

Experiment metrics belong to a specific experiment run, block configuration, and document set, and are read from the experiment-run metrics API. If you edit the block or change the experiment documents, Retab marks the latest run’s metrics as stale. If the output schema changes, Retab can also report schema drift. When an experiment is stale, run it again to refresh the score against the current workflow draft. Retab keeps run history, so you can compare the latest score with earlier runs and see whether a configuration change improved or degraded the block.

Recommended Workflow

Run the workflow with representative documents.
Create an experiment for an Extract, Split, Classifier, or split-by-key For Each block.
Start with 3 consensus passes while iterating quickly.
Inspect the lowest-scoring documents and targets.
Adjust the schema, prompt, categories, or split definitions.
Re-run the experiment and compare the score with the previous run.
Add workflow evals for outputs that should now be protected with explicit assertions.

Experiments work best as a discovery and comparison tool. They tell you where a block is uncertain; evals then lock in the behaviors you decide are correct.

Using the SDK

The dashboard flow above maps onto a small set of SDK calls. The same calls back the MCP tool surface, so anything you can do interactively or through an agent you can do programmatically.

Create an experiment

Pick a supported block, give the experiment a name and a document set, and choose how many consensus passes per document (3, 5, or 7). Documents come from prior workflow runs (via document_captures) or as explicit handle inputs.

from retab import Retab

client = Retab()

experiment = client.workflows.experiments.create(
    workflow_id="wf_abc123",
    block_id="extract-invoice",
    name="Q1 invoices",
    document_captures=[
        {"run_id": "wfrun_1"},
        {"run_id": "wfrun_2", "step_id": "for_each-0"},
    ],
    n_consensus=5,
)

import { Retab } from "@retab/node";

const client = new Retab({ apiKey: process.env.RETAB_API_KEY });

const experiment = await client.workflows.experiments.create("wf_abc123", "extract-invoice", [
    { runId: "wfrun_1" },
    { runId: "wfrun_2", stepId: "for_each-0" },
  ], undefined, 5, "Q1 invoices");

package main

import (
	"context"
	"fmt"
	"log"

	retab "github.com/retab-dev/retab/clients/go"
)

func main() {
	ctx := context.Background()
	client, err := retab.NewClient("")
	if err != nil {
		log.Fatal(err)
	}

	blockID := "extract-invoice"
	name := "Q1 invoices"
	stepID := "for_each-0"
	nConsensus := retab.CreateExperimentRequestNConsensus5
	experiment, err := client.Workflows.Experiments.Create(ctx, &retab.WorkflowExperimentsCreateParams{
		WorkflowID: "wf_abc123",
		BlockID:    &blockID,
		Name:       &name,
		DocumentCaptures: []*retab.ExperimentDocumentCaptureRequest{
			{RunID: "wfrun_1"},
			{RunID: "wfrun_2", StepID: &stepID},
		},
		NConsensus: &nConsensus,
	})
	if err != nil {
		log.Fatal(err)
	}

	fmt.Println(experiment.ID)
}

require 'retab'

client = Retab::Client.new(api_key: ENV['RETAB_API_KEY'])

experiment = client.workflows.experiments.create(
  workflow_id: 'wf_abc123',
  block_id: 'extract-invoice',
  name: 'Q1 invoices',
  document_captures: [
    { run_id: 'wfrun_1' },
    { run_id: 'wfrun_2', step_id: 'for_each-0' },
  ],
  n_consensus: 5,
)

puts experiment.id

<?php
require 'vendor/autoload.php';

use Retab\Client;
use Retab\Resource\ExperimentDocumentCaptureRequest;
use Retab\Resource\NConsensusValue;

$client = new Client();

$experiment = $client->workflows()->experiments()->create(
    workflowId: 'wf_abc123',
    blockId: 'extract-invoice',
    name: 'Q1 invoices',
    documentCaptures: [
        new ExperimentDocumentCaptureRequest(runId: 'wfrun_1'),
        new ExperimentDocumentCaptureRequest(runId: 'wfrun_2', stepId: 'for_each-0'),
    ],
    nConsensus: NConsensusValue::Value5,
);

using System;
using System.Collections.Generic;
using System.Threading.Tasks;
using Retab;
using RetabClient = Retab.Retab;

var client = new RetabClient("YOUR_API_KEY");

var experiment = await client.Workflows.Experiments.CreateAsync(
    new WorkflowExperimentsCreateOptions
    {
        WorkflowId = "wf_abc123",
        BlockId = "extract-invoice",
        Name = "Q1 invoices",
        DocumentCaptures = new List<ExperimentDocumentCaptureRequest>
        {
            new ExperimentDocumentCaptureRequest { RunId = "wfrun_1" },
            new ExperimentDocumentCaptureRequest { RunId = "wfrun_2", StepId = "for_each-0" },
        },
        // n_consensus accepts 3, 5, or 7
        NConsensus = (CreateExperimentRequestNConsensus)5,
    }
);

Console.WriteLine(experiment.Id);

use retab::{
    enums::CreateExperimentRequestNConsensus,
    models::{CreateExperimentRequest, ExperimentDocumentCaptureRequest},
    resources::workflow_experiments,
    Retab,
};

let client = Retab::new(std::env::var("RETAB_API_KEY")?);
let mut request = CreateExperimentRequest::new("wf_abc123");
request.block_id = Some("extract-invoice".to_string());
request.name = Some("Q1 invoices".to_string());
request.document_captures = Some(vec![
    ExperimentDocumentCaptureRequest::new("wfrun_1"),
    ExperimentDocumentCaptureRequest {
        run_id: "wfrun_2".to_string(),
        step_id: Some("for_each-0".to_string()),
    },
]);
request.n_consensus = Some(CreateExperimentRequestNConsensus::V5);

let experiment = client
    .workflows()
    .experiments()
    .create(workflow_experiments::CreateParams::new(request))
    .await?;
println!("{}", experiment.id);

import com.retab.RetabClient;

public final class Example {
  public static void main(String[] args) throws Exception {
    RetabClient client = new RetabClient(System.getenv("RETAB_API_KEY"));

    var result = client.workflows().experiments().create("wf_abc123", "block_abc123", null, null, null, "Invoice Processing", null);
    System.out.println(result);
  }
}

Creating an experiment does NOT trigger a run — the document set is registered but no metrics exist yet.

Run the experiment

Trigger consensus runs against the current draft block config. This is async; the SDK returns a run resource that you inspect through the experiment runs API.

run = client.workflows.experiments.runs.create(
    workflow_id="wf_abc123",
    experiment_id=experiment.id,
)

print(run.id, run.lifecycle.status)

const run = await client.workflows.experiments.runs.create(experiment.id, "wf_abc123");

console.log(run.id, run.lifecycle.status);

package main

import (
	"context"
	"fmt"
	"log"

	retab "github.com/retab-dev/retab/clients/go"
)

func main() {
	client, err := retab.NewClient("")
	if err != nil {
		log.Fatal(err)
	}

	workflowID := "wf_abc123"
	run, err := client.Workflows.Experiments.Runs.Create(
		context.Background(),
		&retab.ExperimentRunsCreateParams{
			WorkflowID:    &workflowID,
			ExperimentID: "exp_abc123",
		},
	)
	if err != nil {
		log.Fatal(err)
	}

	fmt.Println(run.ID, run.Lifecycle.Status())
}

require 'retab'

client = Retab::Client.new(api_key: ENV['RETAB_API_KEY'])

run = client.workflows.experiments.runs.create(experiment_id: experiment.id)

puts "#{run.id} #{run.lifecycle.status}"

<?php
$run = $client->workflows()->experiments()->runs()->create(
    experimentId: $experiment->id,
    workflowId: 'wf_abc123',
);

echo $run->id . ' ' . $run->lifecycle->status . PHP_EOL;

using System;
using System.Threading.Tasks;
using Retab;
using RetabClient = Retab.Retab;

static string LifecycleStatus(object? lifecycle)
{
    return lifecycle?.GetType().GetProperty("Status")?.GetValue(lifecycle)?.ToString() ?? "unknown";
}

var run = await client.Workflows.Experiments.Runs.CreateAsync(
    new ExperimentRunsCreateOptions
    {
        ExperimentId = experiment.Id,
        WorkflowId = "wf_abc123",
    }
);

Console.WriteLine($"{run.Id} {LifecycleStatus(run.Lifecycle)}");

use retab::{models::CreateExperimentRunRequest, resources::experiment_runs, Retab};

let client = Retab::new(std::env::var("RETAB_API_KEY")?);
let experiment_id = "wexp_abc123";
let mut request = CreateExperimentRunRequest::new(experiment_id);
request.workflow_id = Some("wf_abc123".to_string());
let run = client
    .workflows()
    .experiments()
    .runs()
    .create(experiment_runs::CreateParams::new(request))
    .await?;

println!("{} {:?}", run.id, run.lifecycle);

import com.retab.RetabClient;

public final class Example {
  public static void main(String[] args) throws Exception {
    RetabClient client = new RetabClient(System.getenv("RETAB_API_KEY"));

    var result = client.workflows().experiments().runs().create("exp_abc123", "wf_abc123", null);
    System.out.println(result);
  }
}

Runs use the experiment’s stored n_consensus and document set.

Inspect the run and results

Poll the run until lifecycle.status is terminal, then read the per-document results produced by that run.

run = client.workflows.experiments.runs.get(run.id)

results = client.workflows.experiments.runs.results.list(run.id)
for result in results.data:
    print(result.document_id, result.lifecycle.status)

result = client.workflows.experiments.runs.results.get(
    run.id,
    document_id="expdoc_xyz",
)
print(result.artifact)

const currentRun = await client.workflows.experiments.runs.get(run.id);
console.log(currentRun.id, currentRun.lifecycle.status);

const results = await client.workflows.experiments.runs.results.list({
  runId: run.id,
});
for (const result of results.data) {
  console.log(result.documentId, result.lifecycle.status);
}

const result = await client.workflows.experiments.runs.results.get({
  runId: run.id,
  documentId: "expdoc_xyz",
});
console.log(result.artifact);

package main

import (
	"context"
	"fmt"
	"log"

	retab "github.com/retab-dev/retab/clients/go"
)

func main() {
	ctx := context.Background()
	client, err := retab.NewClient("")
	if err != nil {
		log.Fatal(err)
	}

	run, err := client.Workflows.Experiments.Runs.Get(ctx, "exprun_2")
	if err != nil {
		log.Fatal(err)
	}
	fmt.Println(run.ID, run.Lifecycle.Status())

	limit := 20
	results, err := client.Workflows.Experiments.Results.List(ctx, &retab.ExperimentRunResultsListParams{
		RunID: "exprun_2",
		PaginationParams: retab.PaginationParams{
			Limit: &limit,
		},
	})
	if err != nil {
		log.Fatal(err)
	}
	for _, result := range results.Data {
		fmt.Println(result.DocumentID, result.Lifecycle.Status())
	}

	result, err := client.Workflows.Experiments.Results.Get(ctx, "expdoc_xyz")
	if err != nil {
		log.Fatal(err)
	}
	fmt.Println(result.Artifact)
}

require 'retab'

client = Retab::Client.new(api_key: ENV['RETAB_API_KEY'])

run = client.workflows.experiments.runs.get(run_id: run.id)

results = client.workflows.experiments.results.list(run_id: run.id)
results.data.each do |result|
  puts "#{result.document_id} #{result.lifecycle.status}"
end

result = client.workflows.experiments.results.get(
  run_id: run.id,
  document_id: 'expdoc_xyz',
)
puts result.artifact

<?php
$run = $client->workflows()->experiments()->runs()->get($run->id);

$results = $client->workflows()->experiments()->results()->list(runId: $run->id);
foreach ($results->data as $result) {
    echo $result->documentId . ' ' . ($result->lifecycle->status ?? '') . PHP_EOL;
}

$result = $client->workflows()->experiments()->results()->get('expdoc_xyz');
print_r($result->artifact);

using System;
using System.Threading.Tasks;
using Retab;
using RetabClient = Retab.Retab;

static string LifecycleStatus(object? lifecycle)
{
    return lifecycle?.GetType().GetProperty("Status")?.GetValue(lifecycle)?.ToString() ?? "unknown";
}

var currentRun = await client.Workflows.Experiments.Runs.GetAsync(run.Id);
Console.WriteLine($"{currentRun.Id} {LifecycleStatus(currentRun.Lifecycle)}");

var results = await client.Workflows.Experiments.Results.ListAsync(
    new ExperimentRunResultsListOptions { RunId = run.Id }
);
await foreach (var item in results)
{
    Console.WriteLine($"{item.DocumentId} {LifecycleStatus(item.Lifecycle)}");
}

var result = await client.Workflows.Experiments.Results.GetAsync("expdoc_xyz");
Console.WriteLine(result.Artifact);

use retab::{resources::experiment_run_results, Retab};

let client = Retab::new(std::env::var("RETAB_API_KEY")?);
let run_id = "exprun_abc123";
let current_run = client.workflows().experiments().runs().get(run_id).await?;
println!("{} {:?}", current_run.id, current_run.lifecycle);

let results = client
    .workflows().experiments().results()
    .list(experiment_run_results::ListParams::new(run_id))
    .await?;
for result in results.data {
    println!("{} {:?}", result.document_id, result.lifecycle);
}

let result = client.workflows().experiments().results().get("expdoc_xyz").await?;
println!("{:#?}", result.artifact);

import com.retab.RetabClient;

public final class Example {
  public static void main(String[] args) throws Exception {
    RetabClient client = new RetabClient(System.getenv("RETAB_API_KEY"));

    var result = client.workflows().experiments().runs().get("run_abc123");
    System.out.println(result);
  }
}

Read metrics

Metrics live under the experiment run with four views — start at summary, drill into by_target on low-scoring fields, then into votes to see voter disagreement on a specific cell. These are experiment-run metric views under /v1/workflows/experiments/metrics?run_id={run_id}. Metric responses use kind as the shared discriminator across successful views. Successful payloads also echo view for compatibility. Because this endpoint is scoped to a concrete run_id, it returns the historical metrics for that run even if the experiment definition has since changed. Check the experiment’s freshness / run_plan_mode fields from experiments.get or experiments.list to decide whether to create a newer run.

summary = client.workflows.experiments.runs.metrics.get(
    run.id,
    view="summary",
)

# A weak field surfaces in summary.aggregate.likelihoods - drill in

target_view = client.workflows.experiments.runs.metrics.get(
    run.id,
    view="by_target",
    target_path="line_items.*.unit_price",
)

# To see what each voter said for one document/target cell

votes = client.workflows.experiments.runs.metrics.get(
    run.id,
    view="votes",
    document_id="expdoc_xyz",
    target_path="line_items.*.unit_price",
)

const summary = await client.workflows.experiments.runs.metrics.get({
  runId: run.id,
  view: "summary",
});

// A weak field surfaces in summary.aggregate.likelihoods - drill in
const targetView = await client.workflows.experiments.runs.metrics.get({
  runId: run.id,
  view: "by_target",
  targetPath: "line_items.*.unit_price",
});

// To see what each voter said for one document/target cell
const votes = await client.workflows.experiments.runs.metrics.get({
  runId: run.id,
  view: "votes",
  documentId: "expdoc_xyz",
  targetPath: "line_items.*.unit_price",
});

package main

import (
	"context"
	"fmt"
	"log"

	retab "github.com/retab-dev/retab/clients/go"
)

func main() {
	client, err := retab.NewClient("")
	if err != nil {
		log.Fatal(err)
	}

	summaryView := retab.ExperimentRunMetricsViewSummary
	summary, err := client.Workflows.Experiments.Metrics.Get(
		context.Background(),
		&retab.ExperimentRunMetricsGetParams{RunID: "exprun_2", View: &summaryView},
	)
	if err != nil {
		log.Fatal(err)
	}
	fmt.Println(summary)

	// A weak field surfaces in summary.aggregate.likelihoods - drill in
	targetView := retab.ExperimentRunMetricsViewByTarget
	targetPath := "line_items.*.unit_price"
	target, err := client.Workflows.Experiments.Metrics.Get(
		context.Background(),
		&retab.ExperimentRunMetricsGetParams{
			RunID:      "exprun_2",
			View:       &targetView,
			TargetPath: &targetPath,
		},
	)
	if err != nil {
		log.Fatal(err)
	}
	fmt.Println(target)

	// To see what each voter said for one document/target cell
	votesView := retab.ExperimentRunMetricsViewVotes
	documentID := "expdoc_xyz"
	votes, err := client.Workflows.Experiments.Metrics.Get(
		context.Background(),
		&retab.ExperimentRunMetricsGetParams{
			RunID:      "exprun_2",
			View:       &votesView,
			DocumentID: &documentID,
			TargetPath: &targetPath,
		},
	)
	if err != nil {
		log.Fatal(err)
	}
	fmt.Println(votes)
}

require 'retab'

client = Retab::Client.new(api_key: ENV['RETAB_API_KEY'])

summary = client.workflows.experiments.metrics.get(
  run_id: run.id,
  view: 'summary',
)

# A weak field surfaces in summary.aggregate.likelihoods - drill in
target_view = client.workflows.experiments.metrics.get(
  run_id: run.id,
  view: 'by_target',
  target_path: 'line_items.*.unit_price',
)

# To see what each voter said for one document/target cell
votes = client.workflows.experiments.metrics.get(
  run_id: run.id,
  view: 'votes',
  document_id: 'expdoc_xyz',
  target_path: 'line_items.*.unit_price',
)

<?php
use Retab\Resource\ExperimentRunMetricsView;

$summary = $client->workflows()->experiments()->metrics()->get(
    runId: $run->id,
    view: ExperimentRunMetricsView::Summary,
);

// A weak field surfaces in summary.aggregate.likelihoods - drill in
$targetView = $client->workflows()->experiments()->metrics()->get(
    runId: $run->id,
    view: ExperimentRunMetricsView::ByTarget,
    targetPath: 'line_items.*.unit_price',
);

// To see what each voter said for one document/target cell
$votes = $client->workflows()->experiments()->metrics()->get(
    runId: $run->id,
    view: ExperimentRunMetricsView::Votes,
    documentId: 'expdoc_xyz',
    targetPath: 'line_items.*.unit_price',
);

using System;
using System.Threading.Tasks;
using Retab;
using RetabClient = Retab.Retab;

var summary = await client.Workflows.Experiments.Metrics.GetAsync(
    new ExperimentRunMetricsGetOptions
    {
        RunId = run.Id,
        View = ExperimentRunMetricsView.Summary,
    }
);

// A weak field surfaces in summary.aggregate.likelihoods - drill in
var targetView = await client.Workflows.Experiments.Metrics.GetAsync(
    new ExperimentRunMetricsGetOptions
    {
        RunId = run.Id,
        View = ExperimentRunMetricsView.ByTarget,
        TargetPath = "line_items.*.unit_price",
    }
);

// To see what each voter said for one document/target cell
var votes = await client.Workflows.Experiments.Metrics.GetAsync(
    new ExperimentRunMetricsGetOptions
    {
        RunId = run.Id,
        View = ExperimentRunMetricsView.Votes,
        DocumentId = "expdoc_xyz",
        TargetPath = "line_items.*.unit_price",
    }
);

use retab::{
    enums::ExperimentRunMetricsView,
    resources::experiment_run_metrics,
    Retab,
};

let client = Retab::new(std::env::var("RETAB_API_KEY")?);
let run_id = "exprun_abc123";
let summary = client
    .workflows()
    .experiments()
    .metrics()
    .get(experiment_run_metrics::GetParams::new(run_id))
    .await?;

let mut target_params = experiment_run_metrics::GetParams::new(run_id);
target_params.view = Some(ExperimentRunMetricsView::ByTarget);
target_params.target_path = Some("line_items.*.unit_price".to_string());
let target_view = client.workflows().experiments().metrics().get(target_params).await?;

let mut votes_params = experiment_run_metrics::GetParams::new(run_id);
votes_params.view = Some(ExperimentRunMetricsView::Votes);
votes_params.document_id = Some("expdoc_xyz".to_string());
votes_params.target_path = Some("line_items.*.unit_price".to_string());
let votes = client.workflows().experiments().metrics().get(votes_params).await?;

println!("{summary:#?} {target_view:#?} {votes:#?}");

import com.retab.RetabClient;

public final class Example {
  public static void main(String[] args) throws Exception {
    RetabClient client = new RetabClient(System.getenv("RETAB_API_KEY"));

    var metrics = client.workflows().experiments().metrics().get("run_abc123", null, null, null, null, null);
    System.out.println(metrics);
  }
}

If an experiment is stale relative to the current block config, consensus count, or document set, experiments.get(...) and experiments.list(...) return freshness.status: "stale" and run_plan_mode: "run". Call runs.create(...) to recompute and then inspect the new run’s metrics. Existing run metrics remain readable as historical evidence.

Update / delete

# Change the document set or n_consensus — invalidates existing metrics
client.workflows.experiments.update(
    experiment_id=experiment.id,
    n_consensus=7,
)

client.workflows.experiments.delete(
    experiment_id=experiment.id,
)

await client.workflows.experiments.update(experiment.id, undefined, undefined, 7);

await client.workflows.experiments.delete(experiment.id);

package main

import (
	"context"
	"fmt"
	"log"

	retab "github.com/retab-dev/retab/clients/go"
)

func main() {
	ctx := context.Background()
	client, err := retab.NewClient("")
	if err != nil {
		log.Fatal(err)
	}

	// Change n_consensus - invalidates existing metrics
	nConsensus := 7
	nConsensusValue := retab.UpdateExperimentRequestNConsensus(nConsensus)
	updated, err := client.Workflows.Experiments.Update(ctx, "exp_abc123", &retab.WorkflowExperimentsUpdateParams{
		NConsensus: &nConsensusValue,
	})
	if err != nil {
		log.Fatal(err)
	}
	fmt.Println(updated.ID)

	// Delete the experiment
	if err := client.Workflows.Experiments.Delete(ctx, "exp_abc123"); err != nil {
		log.Fatal(err)
	}
}

require 'retab'

client = Retab::Client.new(api_key: ENV['RETAB_API_KEY'])

# Change the document set or n_consensus - invalidates existing metrics
client.workflows.experiments.update(
  experiment_id: experiment.id,
  n_consensus: 7,
)

client.workflows.experiments.delete(experiment_id: experiment.id)

<?php
use Retab\Resource\NConsensusValue;

// Change n_consensus — invalidates existing metrics
$client->workflows()->experiments()->update(
    experimentId: $experiment->id,
    nConsensus: NConsensusValue::Value7,
);

$client->workflows()->experiments()->delete($experiment->id);

using System;
using System.Threading.Tasks;
using Retab;
using RetabClient = Retab.Retab;

// Change n_consensus — invalidates existing metrics
await client.Workflows.Experiments.UpdateAsync(
    experiment.Id,
    new WorkflowExperimentsUpdateOptions
    {
        NConsensus = (CreateExperimentRequestNConsensus)7,
    }
);

await client.Workflows.Experiments.DeleteAsync(experiment.Id);

use retab::{
    enums::UpdateExperimentRequestNConsensus,
    models::UpdateExperimentRequest,
    resources::workflow_experiments,
    Retab,
};

let client = Retab::new(std::env::var("RETAB_API_KEY")?);
let experiment_id = "wexp_abc123";
let mut request = UpdateExperimentRequest::default();
request.n_consensus = Some(UpdateExperimentRequestNConsensus::V7);
client
    .workflows()
    .experiments()
    .update(
        experiment_id,
        workflow_experiments::UpdateParams::new(request),
    )
    .await?;
client.workflows().experiments().delete(experiment_id).await?;

import com.retab.RetabClient;

public final class Example {
  public static void main(String[] args) throws Exception {
    RetabClient client = new RetabClient(System.getenv("RETAB_API_KEY"));

    var result = client.workflows().experiments().update("exp_abc123", null, null, null, "Invoice Processing");
    System.out.println(result);
  }
}

For the full method reference (including async variants under AsyncRetab.workflows.experiments), see the API reference.

Overview

Primitives

Workflows

Core Concepts

Enterprise

What are Experiments?

Experiments vs Evals

Supported Blocks

How Experiments Work

Creating an Experiment

Reading Results

Staleness and Re-runs

Recommended Workflow

Using the SDK

Create an experiment

Run the experiment

Inspect the run and results

Read metrics

Update / delete

​What are Experiments?

​Experiments vs Evals

​Supported Blocks

​How Experiments Work

​Creating an Experiment

​Reading Results

​Staleness and Re-runs

​Recommended Workflow

​Using the SDK

​Create an experiment

​Run the experiment

​Inspect the run and results

​Read metrics

​Update / delete

What are Experiments?

Experiments vs Evals

Supported Blocks

How Experiments Work

Creating an Experiment

Reading Results

Staleness and Re-runs

Recommended Workflow

Using the SDK

Create an experiment

Run the experiment

Inspect the run and results

Read metrics

Update / delete