Hedgeling mega benchmark
Hedgeling 1.3.1. Every project is pinned to the commit shown.
56 repos · 79,428 files · 11,809,839 LOC scanned ·
79,484 strings surfaced ·
extraction runtime 174.2 s ·
at 3 min/string: 3,974 h of manual string-hunting replaced
Precision by category (tuned)
| category | repos | strings | pooled precision | real strings in sample | how to read it |
|---|---|---|---|---|---|
| flagship · greenfield | 5 | 5,868 | 95% | 475/500 | High precision expected — the headline accuracy number. Strings are net-new copy to translate. |
| flagship · brownfield | 2 | 241 | 76% | 152/200 | Low precision is fine. The app already ships i18n, so extraction surfaces the i18n gaps it missed. The metric that matters is how many real gaps are in the haul (the tp count), not the ratio. |
| production · greenfield | 3 | 17,845 | 93% | 204/220 | High precision expected with only one mechanical tuning round. |
| production · brownfield | 17 | 27,215 | 74% | 1,262/1700 | Gap-finding mode: precision is a residue ratio, not a quality score. tp = untranslated strings the project's own i18n pass missed. |
| examples | 6 | 7,701 | 90% | 499/557 | Demo/template apps — mostly greenfield, precision applies. |
| library · docs/demo sites | 4 | 16,979 | 81% | 324/400 | Docs sites render real prose — precision applies. |
| library · component source | 3 | 1,594 | 60% | 90/151 | Component-library internals: mostly framework plumbing and locale catalogs. Low precision expected; these repos exist to stress false-positive behavior. |
| html | 6 | 1,225 | 79% | 212/268 | Static/HTML-first projects; template repos have tiny samples. |
| noise | 10 | 816 | 0% | 1/515 | Control group: ~0% precision is the CORRECT outcome. These repos contain no user-facing copy, so anything extracted is dev-facing by construction. The gate metric is the raw string count, and success is that tuned counts are tiny (3–100 strings across half a million LOC). |
How to read this. Every number comes from the project's tuned configuration: a zero-config start,
hedgeling tune (its deterministic tier and its LLM tier), then a hand-tuned pass driven by the
diagnostics report.
Precision means different things per category (see the summary table above). For greenfield apps it is
the accuracy headline. For brownfield repos that already ship i18n, extraction is gap-finding: their
translated copy is already wrapped, so what Hedgeling pulls out is the residue their own i18n pass missed, mixed
with technical noise. Low precision there is expected and the interesting number is the real strings in
sample column (real gaps found). For the noise tier, near-zero precision is the correct result: those repos
contain no UI copy at all, and the gate metric is the raw string count staying near zero.
Precision cells come from labeled 100-string samples per project. Hover a project name for the commit it is
pinned to.
Labeling disclosure: precision samples were labeled with AI assistance in a single reviewer pass
(tp = user-visible translatable copy; fp = identifiers, config, code, keys, locale-catalog data, developer-facing
messages) and have not been independently hand-verified. Treat borderline calls as a few points of judgment noise.
flagship
Accuracy tier: hand-labeled precision samples, tuned configs, strip-test recall where ground truth exists.
| project | framework | LOC | strings | precision | real strings in sample | ||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| greenfield — precision applies | |||||||||||||||||||
| bulletproof-react | react | 6,960 | 103 | 100% | 100/100 | ||||||||||||||
| hf-chat-ui | svelte | 39,934 | 528 | 100% | 100/100 | ||||||||||||||
| appwrite-console | svelte | 154,639 | 4,129 | 98% | 98/100 | ||||||||||||||
| pocketbase-admin | svelte | 34,650 | 992 | 96% | 96/100 | ||||||||||||||
| analog | angular | 10,080 | 116 | 81% | 81/100 | ||||||||||||||
| brownfield — already internationalized; the sample count is the real gaps found | |||||||||||||||||||
| directus | vue | 204,109 | 119 | 81% | 81/100 | ||||||||||||||
| keycloak-admin-ui | react | 95,840 | 122 | 71% | 71/100 | ||||||||||||||
production
Compatibility/scale tier: real shipped frontends, bare config, automated metrics only.
| project | framework | LOC | strings | precision | real strings in sample | ||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| greenfield — precision applies | |||||||||||||||||||
| react-admin-demo | react | 6,899 | 20 | 100% | 20/20 | ||||||||||||||
| supabase-studio | react | 578,458 | 14,084 | 97% | 97/100 | ||||||||||||||
| twenty | react | 626,812 | 3,741 | 87% | 87/100 | ||||||||||||||
| brownfield — already internationalized; the sample count is the real gaps found | |||||||||||||||||||
| strapi-admin | react | 75,542 | 573 | 94% | 94/100 | ||||||||||||||
| tooljet | react | 273,820 | 4,168 | 93% | 93/100 | ||||||||||||||
| outline | react | 98,252 | 1,484 | 91% | 91/100 | ||||||||||||||
| plane | react | 183,579 | 1,804 | 90% | 90/100 | ||||||||||||||
| typebot | react | 64,579 | 1,417 | 87% | 87/100 | ||||||||||||||
| grafana | react | 1,160,248 | 5,301 | 82% | 82/100 | ||||||||||||||
| appsmith | react | 674,934 | 5,314 | 80% | 80/100 | ||||||||||||||
| nocodb | vue | 464,263 | 1,378 | 80% | 80/100 | ||||||||||||||
| rocketchat | react | 180,774 | 735 | 76% | 76/100 | ||||||||||||||
| saleor-dashboard | react | 536,655 | 1,000 | 76% | 76/100 | ||||||||||||||
| immich-web | svelte | 65,555 | 114 | 75% | 75/100 | ||||||||||||||
| open-webui | svelte | 304,780 | 1,025 | 73% | 73/100 | ||||||||||||||
| calcom | react | 130,396 | 502 | 62% | 62/100 | ||||||||||||||
| chatwoot | vue | 781,711 | 1,245 | 59% | 59/100 | ||||||||||||||
| payload | react | 93,650 | 111 | 55% | 55/100 | ||||||||||||||
| jitsi-meet | react | 211,846 | 568 | 51% | 51/100 | ||||||||||||||
| home-assistant-frontend | js-ts | 482,977 | 476 | 38% | 38/100 | ||||||||||||||
examples
Framework example apps and docs sites: greenfield-style codebases people copy from.
| project | framework | LOC | strings | precision | real strings in sample | ||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| greenfield — precision applies | |||||||||||||||||||
| angular-realworld | angular | 4,611 | 57 | 100% | 57/57 | ||||||||||||||
| ngx-admin | angular | 18,465 | 874 | 93% | 93/100 | ||||||||||||||
| shadcn-ui-www | react | 662,404 | 5,283 | 85% | 85/100 | ||||||||||||||
| react-dev | react | 48,397 | 453 | 77% | 77/100 | ||||||||||||||
| brownfield — already internationalized; the sample count is the real gaps found | |||||||||||||||||||
| vuestic-admin | vue | 54,337 | 500 | 95% | 95/100 | ||||||||||||||
| vue-element-admin | vue | 18,073 | 534 | 92% | 92/100 | ||||||||||||||
library
Component libraries / design systems: template-heavy compatibility data, strings are mostly demo copy.
| project | framework | LOC | strings | precision | real strings in sample | ||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| greenfield — precision applies | |||||||||||||||||||
| shadcn-svelte-docs | svelte | 102,548 | 3,562 | 88% | 88/100 | ||||||||||||||
| taiga-ui-demo | angular | 85,335 | 4,809 | 82% | 82/100 | ||||||||||||||
| primeng-showcase | angular | 1,837,732 | 6,980 | 77% | 77/100 | ||||||||||||||
| mantine-docs | react | 18,852 | 1,628 | 77% | 77/100 | ||||||||||||||
| brownfield — already internationalized; the sample count is the real gaps found | |||||||||||||||||||
| ant-design-vue | vue | 182,333 | 1,543 | 83% | 83/100 | ||||||||||||||
| element-plus | vue | 144,755 | 17 | 35% | 6/17 | ||||||||||||||
| vuetify | vue | 107,060 | 34 | 3% | 1/34 | ||||||||||||||
html
Static HTML sites and templates: exercises the html adapter at scale.
| project | framework | LOC | strings | precision | real strings in sample | ||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| greenfield — precision applies | |||||||||||||||||||
| bootstrap-site | html | 2,350 | 13 | 100% | 13/13 | ||||||||||||||
| html5-boilerplate | html | 177 | 3 | 100% | 3/3 | ||||||||||||||
| tabler | html | 8,744 | 794 | 91% | 91/100 | ||||||||||||||
| adminlte | html | 5,820 | 6 | 83% | 5/6 | ||||||||||||||
| sb-admin-2 | html | 116,613 | 363 | 70% | 70/100 | ||||||||||||||
| htmx-www | html | 33,683 | 46 | 65% | 30/46 | ||||||||||||||
noise
Control tier: tooling and framework internals with near-zero UI strings. Gate: bare density < 1 string/kLOC.
| project | framework | LOC | strings | precision | real strings in sample | ||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| controls — near-zero precision expected; the string count is the metric | |||||||||||||||||||
| svelte-compiler | js-ts | 69,902 | 49 | 2% | 1/49 | ||||||||||||||
| react-source | js-ts | 122,186 | 42 | 0% | 0/42 | ||||||||||||||
| vue-core | js-ts | 20,901 | 177 | 0% | 0/100 | ||||||||||||||
| eslint | js-ts | 106,141 | 370 | 0% | 0/100 | ||||||||||||||
| prettier | js-ts | 51,845 | 63 | 0% | 0/63 | ||||||||||||||
| tanstack-query | js-ts | 48,365 | 3 | 0% | 0/3 | ||||||||||||||
| react-hook-form | js-ts | 58,433 | 0 | 0% | 0/2 | ||||||||||||||
| redux-toolkit | js-ts | 60,163 | 7 | 0% | 0/7 | ||||||||||||||
| ngrx-platform | js-ts | 139,068 | 105 | 0% | 0/100 | ||||||||||||||
| angular-material | angular | 138,604 | 0 | 0% | 0/49 | ||||||||||||||