---
title: A frontend taskset with a browser verifier: how I build gradable app tasks
url: https://www.consultingai.dev/work/frontend-taskset-verifier/
original: https://www.consultingai.dev/work/frontend-taskset-verifier/
---


# A frontend taskset with a browser verifier: how I build gradable app tasks

Environments built to order · by [Shreenivas Sangoram](/) · updated 28 Oct 2026

Short answer

I built a taskset of 50 web-app tasks, each with exact rules a model must implement in one standalone HTML file under a shared contract covering 360 px and 1280 px widths, keyboard-only use and accessible errors. A Playwright verifier grades the rendered app through its controls, never its code, and reports whether a failure came from binding, infrastructure or behaviour. Runs went through Modal. I build human-authored tasksets in this shape to order.

A taskset is only as good as its grader. This one was built grader-first: every task has rules precise enough to check from the outside, and the verifier only ever touches what a user could.

## The shape of a task

Every task shares one contract, then adds its own rules:

- **One standalone HTML file**: inline CSS and JavaScript, no libraries, no network, no build step.
- **Working state, not a mockup**: real calculations and state transitions from given initial data.
- **Usable at 360 px and 1280 px** without page-level horizontal scrolling.
- **Keyboard-only and accessible**: visible labels, accessible names, visible focus, specific errors tied to their fields, results announced through a status region, no blocking browser dialogs.
- **Modals done properly**: named, focus moved in and trapped, Escape closes, focus restored.

The task rules are exact enough to grade. For example, a parcel tariff task fixes the base price, the per-500 g step, the zone multipliers and the express surcharge, and asks for each component and the total. A room-booking task fixes opening hours, 15-minute steps, capacities and the rule that intervals are start-inclusive and end-exclusive, so adjacent bookings are allowed.

## How the verifier grades

- It drives the app in Chromium through **rendered controls and the visible DOM only**. It never imports or calls the app's functions.
- It checks behaviour at **both viewport widths**.
- It classifies failures into three kinds, so a broken run is never scored as a wrong answer: - **binding**: the verifier couldn't find a control it needs; - **infrastructure**: the browser or runtime failed; - **assertion**: the app did the wrong thing.
- Each output is fingerprinted with SHA-256, so a report always points at the exact file it graded.

## Facts

Item | Detail
Tasks | 50 web-app tasks under one shared contract
Verifier | Playwright and Chromium, about 380 lines; covers a first set of 5 tasks so far
Runs | Dry runs and runtime evaluation on Modal (Sep 2026)
Status | A working prototype of the shape, not a product for sale

## What you can buy

Tasksets in this shape, written by a person to your spec:

- tasks with exact, checkable rules in your domain;
- a programmatic grader that works through the interface, not the code;
- failure classes that keep infrastructure noise out of the score;
- delivered as HUD tasksets with their environment, or as a repository with a runner.

## FAQ

### What makes a task gradable?

Rules exact enough that a program can decide pass or fail from the outside: fixed initial data, fixed arithmetic and rounding, and explicit edge cases such as adjacent bookings or zero rows.

### Why grade through the UI instead of unit tests?

Because the model is free to structure its code however it likes. Grading through rendered controls checks what a user would experience, and works for any implementation.

### Are the 50 tasks for sale?

No. The prompts were drafted with an assistant, so I don't license them. I write tasksets like this by hand, to order.

Want this, or something like it?

License it, or have me build an environment in the same style.

Email me →