Guides / A/B testing / What is A/B testing

A/B testing API version 3

On this page

Changes compared to the A/B testing API version 2

  • Timeseries endpoint: analyze how your chosen metrics evolve over time while the test runs.
    • New in v3: GET /3/abtests/{id}/timeseries with date range and metric selection.
  • Create A/B test: choose which metrics the experiment should measure, and add statistical controls for evaluation.
    • v2: create a test by naming it, defining its variants, and setting an end date.
    • v3: in addition, you explicitly select the metrics to analyze (for example, conversion rate) and can configure statistical options such as minimum detectable effect (MDE), and multiple-testing correction.
  • Estimation endpoint: estimate required sample sizes and duration to detect your target effect.
    • v2: basic estimator.
    • v3: uses your selected metrics and desired effect size/MDE for more tailored estimates.

  • Listing and retrieval: same operations with more flexible sorting/filtering and responses that include your selected metrics and evaluation settings.

  • Multivariant testing:
    • v2: mixed setups such as A/A/B are rejected. You must either use the same index for all variants (A/A/…) or use distinct indices for all variants (A/B/C/…). Mixed shapes fail with an invalid configuration error.
    • v3: supports mixed shapes (for example, A/A/B).

For details, see the API references:

When to migrate

Migrate if you need any of the following:

  • Timeseries analysis of test metrics.
  • Explicit metric selection per test.
  • Minimum detectable effect (MDE), population filters, or multiple-testing correction.

If your v2 usage is stable and you don’t need these new features, you can keep using v2.

How to migrate

  • Do this in all cases:
    • When creating a test, include the metrics to measure and any optional evaluation settings.
    • For planning, use the estimator with your desired effect size (MDE).
    • Use the timeseries endpoint if you need trends over time.
  • If you call the REST API directly:
    • Change base paths from /2/abtests to /3/abtests and adapt payloads.
  • If you use an API client:
Did you find this page helpful?