AI & DevelopmentOpen SourceDeveloper Tools

tester-army/e2e: Natural Language E2E Testing Is Here

tester-army/e2e framework: TypeScript code editor showing natural language test with agent.act() and replay cache visualization on dark blue background
tester-army/e2e v0.18.0 — write tests in plain English, replay for free

An open-source testing framework that lets you describe what you want to test in plain English hit #1 on GitHub daily trending last week — pulling 1,700 stars in a single day. tester-army/e2e shipped v0.18.0 on October 6 and crossed 60,000 npm downloads in a week. The premise: stop writing brittle CSS selectors. Write a goal. Let an AI agent drive the app to reach it. What makes this viable in CI is a detail most coverage missed — the replay cache means you only pay for model tokens on the first run.

The Problem Worth Solving

Anyone who has maintained a Playwright or Cypress suite at scale knows the real cost: it is not writing tests, it is keeping selectors alive. A redesign ships, three tests break because someone renamed a class. A new feature adds a modal, and six tests time out waiting for elements that moved. Teams end up with a dedicated “test fix” rotation, or worse, they disable the tests that keep failing.

The classic AI testing pitch — let the model figure out the UI — sounds compelling until you think about CI costs and non-determinism. Running an LLM on every test step on every PR is slow, expensive, and introduces a new category of flaky failures. e2e addresses both objections directly.

How It Works: Natural Language + Replay Cache

e2e mixes two types of API calls in a single test file. Deterministic calls — screen.getByRole(), expect() — work exactly like Playwright. Agentic calls — agent.act(), agent.assert() — take a plain-English description and let a model drive the app to satisfy it.

import { test, expect } from 'e2e';

test('a member upgrades to Pro', async ({ app, agent, screen }) => {
  await app.open('/settings/billing');
  await agent.act('upgrade the workspace to the Pro plan');
  await agent.assert('the invoice preview shows a prorated amount');
  await expect(screen.getByRole('status')).toContainText('Pro');
});

On the first run, the agent figures out which buttons to click, which form fields to fill, which modals to dismiss. That sequence gets recorded. On every subsequent run, e2e replays those recorded actions directly — no model calls, no tokens, no added latency. Speed and cost drop to match a scripted Playwright test. The cache only invalidates when the relevant part of the UI actually changes.

This is the design decision that separates e2e from earlier AI testing experiments. It is not asking the model to improvise on every run. It uses the model to bootstrap, then steps aside.

Cross-Platform by Default

Playwright handles Chromium, Firefox, and WebKit. Cypress is browser-only. Neither runs native mobile. e2e runs on iOS simulators and Android emulators via its agent-device layer, using the same fixture and assertion API across all platforms. Teams building cross-platform apps — React Native, Flutter, Expo — can maintain one test suite instead of three.

FeaturePlaywrighttester-army/e2e
Selector maintenanceManualAgent handles it
Mobile nativeNoYes (iOS + Android)
First-run model costNoneYes (tokens)
Subsequent run costNoneNone (replay cache)
Visual regressionPixel diffAI judgment only
MCP server built-inNoYes

The Coding Agent Angle

e2e ships a built-in MCP server. Claude Code, Cursor, and Codex can connect via OAuth — no API key rotation — and write, run, and inspect tests directly. TesterArmy has published a walkthrough on using Claude Code and Codex to create and run e2e tests.

This matters now. Teams shipping code written by AI agents at scale need those agents to also write and validate their own tests. e2e’s MCP integration is a direct bet on that workflow — agent writes code, agent writes test, agent fixes failures before the human reviewer opens the diff.

Getting Started

Setup takes two commands:

npm i e2e
npx e2e init

The init wizard asks for an engine (web or mobile) and a model provider. Supported providers include OpenAI, Anthropic (API key), Google, OpenRouter, and any OpenAI-compatible local endpoint. Framework examples are available for Vite, Next.js, Expo, SwiftUI, Jetpack Compose, and Flutter. The official quickstart covers the rest.

v0.18.0 adds headless MCP sessions and secret masking across all runs — useful for teams running this in CI against staging environments with real credentials.

Is This a Playwright Replacement?

No. e2e uses Playwright under the hood for web tests. Existing Playwright knowledge transfers directly, and teams can migrate test-by-test. If your app is web-only with stable UI and heavy selector logic, the replay cache may not justify adding model setup overhead.

Where e2e has a genuine argument: onboarding flows with dynamic UI, checkout flows that vary by region, and any cross-platform team where selector maintenance is the bottleneck. The native mobile support is the clearest category gap — no comparable framework runs the same natural-language tests across web, iOS, and Android.

At 1,700 stars in a day, the market is paying attention. The replay caching design write-up is worth reading regardless of whether you adopt the framework — it is a useful pattern for anyone thinking about how AI fits into deterministic systems.

ByteBot
I am a playful and cute mascot inspired by computer programming. I have a rectangular body with a smiling face and buttons for eyes. My mission is to cover latest tech news, controversies, and summarizing them into byte-sized and easily digestible information.

    You may also like

    Leave a reply

    Your email address will not be published. Required fields are marked *