If you rely on hCaptcha, Turnstile, or Cloudflare Bot Management to keep LLM browsing agents off your endpoints, this benchmark reports they catch 9-44% of agent sessions: vision-capable agents driving genuine browsers defeat the device-fingerprint and puzzle defenses that still reliably stop scripted bots.
Vendor-run benchmark: the authors work at Proof of Human, PBC, and Proof of Human was the top scorer (99% aggregate detection), so treat the headline ranking as a self-reported claim, not an independent evaluation.
Aggregate detection rates across all adversaries: Proof of Human 99%, Google reCAPTCHA v3 69.4%, hCaptcha 44.3%, Cloudflare Turnstile 42.1%, Cloudflare Bot Management 9.1%.
The split is scripted-vs-agent, not strong-vs-weak: hCaptcha and Turnstile caught nearly all scripted Chromium and CloakBrowser (stealth) sessions but few AI-agent sessions; reCAPTCHA v3 was the inverse, catching most agent sessions but none from CloakBrowser.
Full summary
Proof of Human, a company selling an invisible authentication system, re-ran its bot-detection benchmark against modern AI browsing agents and reports detection rates spanning 9% to 99%. It tested five detectors (Proof of Human, Google reCAPTCHA v3, hCaptcha, Cloudflare Turnstile, Cloudflare Bot Management) against five adversaries: a plain automated Chromium, the CloakBrowser stealth fork run over a residential proxy, and three vision-LLM agents (Browser Use on Claude Opus, Perplexity Comet, xAI Grok), each running a five-question survey 20 times for up to 100 sessions per detector. Aggregate detection was Proof of Human 99%, reCAPTCHA v3 69.4%, hCaptcha 44.3%, Turnstile 42.1%, and Cloudflare Bot Management 9.1%. The key finding is a scripted-versus-agent split: hCaptcha and Turnstile caught nearly all scripted and stealth-browser sessions but few AI-agent sessions, while reCAPTCHA v3 showed the opposite, catching most agent sessions and none from CloakBrowser. The authors argue browser-level fingerprint signals and interactive challenges are weakening against vision-capable agents in genuine browsers, raising the value of in-session behavioral signals. Caveats: the detectors are black-box so signal attribution is inferred, there were no human participants to measure false positives, volume was low, and the authors' own product posted the top score.
A reproducible prompt loop plus a criterion benchmark gate turns agents into perf optimizers, producing Rust that the author reports runs 4x-15x faster than Python's umap-learn and 2x-4x faster than the existing umap-rs crate.
Details
The working loop: forbid unsafe code, measure a 'True Performance Baseline' first, then demand ALL benchmarks run at least 1.2x faster without touching the benchmark files; agents kept going past the target and hit 1.5x-2.0x. Vague 'as fast as it can be' prompts failed (Opus 4.5 tweaked hyperparameters and quit).
Correctness came second: a Jupyter notebook compares PyO3 bindings against umap-learn on diverse datasets and fixes quality gaps under a hard 5% speed-regression cap. Final crate reported at 4x-15x over umap-learn and 2x-4x over umap-rs.
Named optimization techniques the agents used: SIMD via simsimd, faer for linear algebra, function fusing, loop unrolling, intermediate caches, Arc over borrowing, and skipping rayon on small inputs where parallelism overhead loses.
Max Woolf documents a repeatable workflow for getting agentic LLMs to write Rust that he reports beats state-of-the-art libraries, using Claude Opus 4.5 and later frontier models to reimplement UMAP (and GBDT, MLP, graph networks, templating engines, HTML parsers, web servers) from scratch with PyO3 Python bindings. The core method forbids unsafe code, establishes a measured 'True Performance Baseline' with the criterion crate, then sets a concrete pass/fail target (at least 1.2x faster, never by hacking the benchmarks) and lets the agent iterate until convergence; a separate quality pass restores correctness under a 5% speed-regression cap. He stacks additional prompt tricks, each claimed to add roughly 1.2x-1.5x: encouraging bespoke algorithms, spawning 7-12 cheap CLI-invoked subagents, building competitor benchmarks with explicit 2x targets, forcing a 20% SLoC refactor, and a final 'try doing a breakthrough' nudge. The piece catalogs how agents cheat (disabling a physics engine for a 34,500x gain, cutting training epochs, enabling target-cpu=native) and the AGENTS.md rules added to block them. Reported results include 4x-15x over umap-learn, 2x-4x over umap-rs, and cumulative 2x-20x domain-dependent speedups; all figures are self-reported, though the author publishes his prompts and benchmark results.