73%
Mean cost overrun on IT projects. 18% of them overrun by more than 50%.
Approved budget100
Final cost, on average173
← AgentsFlyvbjerg et al., Project Management Journal, 2026 — 11,011 projects, 126 countries ↗ PMI, Pulse of the Profession 2026 — 2,534 respondents, 35 countries ↗ Wellingtone, State of Project Management 2026 — more than 150 organisations ↗ Wellingtone, State of Project Management 2026 — more than 150 organisations ↗
Applied AI · Open source
Extend your PMO’s capacity
Vera, Samuel and Alba join your project office: they process every project’s documentation, compute variances and alerts, and prepare the report, the minutes and the committee pack, so your team spends its time deciding rather than consolidating.
What it costs, measured by others
31%
Of complex projects fail to achieve the benefits they set out to deliver.
31 of every 100 complex projects
1 in 3
Projects have no baseline at all. Half of organisations have no real-time indicators, and 22% still plan in Excel.
One in every three projects
72%
Spend half a day or more every month collating reports by hand.
72 of every 100 organisations
Third-party figures, each with its source, its year and its sample. The agents’ own are further down, in the test evidence.
Criterio is open source. If it creates value for you, help fund testing, documentation and the next capabilities.
Vera, Samuel and Alba are the project governance family of Criterio, an open marketplace of agents. See the marketplace
Criterio · PMO family
Three agents and a server for the project office
They process what your organisation already wrote, prepare the project office’s work and tell you what changed, what contradicts itself and what no document supports.
CRITERIO · PMOREADY
VERA
The portfolio
SAMUEL
One project
ALBA
One product
ROSTRUM
The server
THE PROJECT RECORDThe only contract. The agents never talk to each other: they talk through it.
PRINTER · CRITERIOSHEET 1
CRITERIO · PMO FAMILY
Where the family stands: tested round after round on test data from financial entities (banks and cooperatives), retail and mining in South America, built to be more demanding than those organisations’ usual documentation. The next step is its first use on a specific organisation’s documentation.
The four, one at a time
What each one does, what it never does and what its work looks like
Each tab stands on its own: you can install one agent without the others. The screenshots are Rostrum, the family’s server, publishing what the agents found in round 1’s mining test data.
criterio-portfolio
From verus, the true: she says what the documents say, not what gets reported.
Processes the whole portfolio’s documentation: builds one record per project with a citation for every data point, computes variances and alerts, and prepares the portfolio report and the committee pack.
What only she does
Checks what every manager declares against what their documents support, across the whole portfolio at once. A project reporting green with five weeks without a document is only visible to someone who looks at all forty with the same yardstick.
What she answers
- What changed since the last run
- What contradicts itself between two documents of the same project
- What has been silent for weeks
- How many greens have evidence the green does not explain
- What needs a committee decision, and what happens if nobody decides
- How much of the budget is committed, and what a supplier invoiced without delivering
- Who is affected if a project moves
Never
Declares a project’s status or prioritises demand.
Rostrum · Portafolio de proyectos mineros
- Extends
- The PMO manager and analysts
- Instances
- One per PMO
- Setup
- 15 min · 5 questions
- Cadence
- Daily sweep; report ahead of the committee
- Needs from you
- Someone who chases what the report asks for
Set up with/criterio-portfolio:portfolio-setup
Its commands, by group
Start and rhythm3 commands+
portfolio-setupportfolio-wakeportfolio-serverReading documents2 commands+
document-indexportfolio-scanReports3 commands+
portfolio-reportstatus-reportsteering-packOne project in depth6 commands+
health-checkproject-historyraid-logchange-controlproject-charterproject-closureMoney and suppliers2 commands+
budget-trackingvendor-trackingProducts1 command+
product-view17 commands · 10 skills
What it costs you today
The work Vera, Samuel and Alba prepare, priced with your own numbers
Each agent extends one person: Vera the project office, Samuel each project manager, Alba each product manager. Enter how many people you have, what they cost and how many hours a week each task takes them. Nothing you type leaves your browser.
I work in
Write the costs in the currency you choose. There is no conversion: changing the currency clears the costs, because an exchange rate written into a page is wrong within weeks.
The people
PMO analysts
Loaded monthly cost per personFully loaded: salary, benefits and overhead.
USD
Hours per week on each task
Reading project documentation to build a status/portfolio-scan · /document-index
h
Consolidating the portfolio report and each project’s status/portfolio-report · /status-report
h
Preparing the committee pack/steering-pack
h
Keeping risks, assumptions, issues and dependencies current/raid-log
h
Reconciling budget and supplier deliverables/budget-tracking · /vendor-tracking
h
Reviewing changes, charters, closures and diagnostics/change-control · /project-charter · /project-closure · /health-check
h
Hours are per person, per week. The starting people and hours are examples: replace them with yours. The cost is yours to write: we do not print one we cannot source.
Your numbers · per year
Vera
PMO analysts · 15 h a week each (38% of the week)
1,380 ha year
—a year
Samuel
Project managers · 7 h a week each (18% of the week)
1,932 ha year
—a year
Alba
Product managers · 8.5 h a week each (21% of the week)
1,173 ha year
—a year
Annual cost of the work the three prepare—
Hours a year4,485 h
Annual cost of the people included—
Share of that cost—
Write the monthly cost of each group to see what it costs.
Rostrum, the server, adds no hours here. It publishes the report where the team reads it and answers people who never open a folder. Its effect is fewer “how is the project going?” emails, and we do not measure that yet.
What this number is and what it is not
It is the cost of the hours your team spends today on tasks the agents prepare: reading, consolidating, drafting. It is not a saving. The person still reviews, decides and declares the status.
How many of those hours come back is something we will publish in the evidence section, with each week’s tests. Until that figure is measured, it does not go here.
Test evidence · updated with every round
This is what is already measured, on what sample and with what margin.
Every figure comes from a real run of the repository, with its count and its 95% confidence margin.
How it is tested
4 industries
Financial entities (banks and cooperatives), retail and mining in South America. Test data written by the repository to be more demanding than usual documentation: mixed formats, contradictions, moved commitments, deliveries with no receipt and a healthy control project in each industry.
37 commands per round
The three agents in the order of a PMO, in every industry: Alba defines the product, Samuel runs a project, Vera consolidates the portfolio and Rostrum publishes it.
532 documents · 52 projects
Charters, schedules, minutes, reports and spreadsheets in text, Word, PDF, Excel, CSV and JSON, processed in the round.
116 million tokens
The language model’s (LLM) work in the round: 2 h 51 min of processing, about 43 minutes per industry.
A key they never see
What is expected is derived from the material’s facts, and every session is audited: no agent read it.
95% margin
Every figure comes with its confidence margin: it says how much the sample supports, which a percentage alone does not.
Latest cut: 2026-10-18 · commit 1bcb73f8
Vera, Samuel and Alba process the PMO’s documentation; Rostrum shows it
Round 9 · 4 industriesIn each round the three agents run their 37 commands in the order of a PMO, on the documentation of one organisation per industry. What they find is compared with a key they never see.
Vera
The portfolio
99.6%
reliability of her alerts
266 of the 267 alerts she raised were correct.
Measured
Coverage: she found 266 of 266 of the alerts that were there. 4 of 4 healthy projects with no false alarms.
She found the 266 alerts that were there. The extra one is a code defect this round found: when the manager writes «No aplica» as the product, the contrast between the two records read it as a product (D056, fixed after 0.4.0).
95% confidence margin: 97.9–100%
Samuel
A project
100%
commitments and close processed correctly
108 of 108: the commitments from committee minutes and unstructured meeting notes —with their current date, their reschedules and whether they are met— and the current close date he publishes to the PMO.
Measured
108 of 108 for the third round in a row.
95% confidence margin (Wilson): 96.5–100%
Alba
A product
100%
alerts on her product detected
128 of the 128 alerts the material held about her product.
Measured
Reliability: 128 of 128 alerts she raised were right.
128 of 128, with no extra alert.
95% confidence margin (Wilson): 97–100%
Rostrum
The server
100%
checks on what it shows
2170 of 2170: every page answers, nothing technical reaches the report, it fits a phone and a desktop, it saves requests and leaves the records alone.
Measured
2170 of 2170: every industry's report landed in the folder Rostrum serves (D054 verified).
95% confidence margin (Wilson): 99.8–100%
148 of 148 commands finished and 148 kept their contract · 4 industries · no reads of the key · about 43 minutes per industry
How Vera has improved from one version to the next
Each dot is one industry in its first run with that version; the line is its 95% confidence margin. The axis starts at the lowest value published.
BanksSavings and credit cooperativesRetailMining
v0.3.41
v0.3.52
Round 1 · v0.3.63
Round 2 · v0.3.84
Round 3 · v0.3.95
Round 4 · v0.3.106
Round 5 · v0.3.117
Round 6 · v0.3.128
Round 7 · v0.3.139
Round 8 · v0.3.1310
Round 9 · v0.4.011
Round 10 · v0.5.012
- 1 · First contact with documents: only Vera’s setup and sweep, on hand-written cases, banks only.
- 2 · Fixes how silence and progress against the plan are computed. Four industries; still only Vera’s setup and sweep.
- 3 · First complete round: the three agents’ 37 commands, in order, with documents in Word, Excel and PDF.
- 4 · Alba builds her product record from what is written, each agent keeps its state in its own folder and accepts the terms of its version.
- 5 · One rule per field, citations that exist and Rostrum walked page by page; samples of about a hundred cases per agent.
- 6 · Samuel reads every minute when set up; the report has one place.
- 7 · Alba cites the interview, Rostrum opens products with a name and Vera's seal checks only her own citations.
- 8 · The key models the contrast between the two records; Samuel publishes the current close date and the charter closes where the schedule does.
- 9 · The text cache has one place, a run page fits a phone and the calculation tells Samuel what to fix in his record.
- 10 · The same version on new material, with verified assumptions carrying their document.
- 11 · The first public version: the report always goes to the folder Rostrum serves.
- 12 · The launch version: the contrast between the two records reads the product with its rule.
Defects the tests found · 60
Every error is logged with its origin and the commit that fixes it. It is only closed when a new run verifies it.
- 57 Fixed and verified
- 3 Fixed, to verify
- 0 Open
Where they came from
- the code22
- the agent’s instructions20
- the test data10
- the answer key4
- the test harness3
- the agent’s reading1
For auditing
The full detail of every run. Each block opens on its own.
Results by industryround 9 · 4 industries
| Test data | Commandsfinished | Documents | Vera · reliabilitycorrect alerts of those raised | Vera · coveragealerts found of those there | Samuel · commitmentscorrect of those graded | Alba · productalerts detected of those there | Minutes |
|---|---|---|---|---|---|---|---|
| Banks | 37 / 3737 kept their contract | 128 | 58 of 58 | 58 of 58 | 27 of 27 | 36 of 36 | 40 |
| Savings and credit cooperatives | 37 / 3737 kept their contract | 126 | 52 of 52 | 52 of 52 | 27 of 27 | 25 of 25 | 41 |
| Retail | 37 / 3737 kept their contract | 146 | 90 of 90 | 90 of 90 | 27 of 27 | 36 of 36 | 45 |
| Mining | 37 / 3737 kept their contract | 132 | 66 of 67 | 66 of 66 | 27 of 27 | 31 of 31 | 45 |
Reliability: of the alerts Vera raised, how many were correct according to the key. Coverage: of the alerts the key held, how many she found. Samuel: commitments taken from minutes and unstructured meeting notes, with the current date and the number of reschedules right, and from round 6 the current close date in his record. Alba: alerts on her product that the material held and she detected.
The round on 0.4.0, the first public version: all 148 commands kept their contract, Samuel, Alba and Rostrum at 100% and Vera found every alert that was there. Her only extra alert is a code defect this round found, already fixed (D056). D054 and D055 verified.
Reliability by alert type16 of 17 types with no errors
First runs of the round, all industries together. It is what decides whether someone acts directly on an alert.
| Alert | Reliability · correct of those raised | Coverage · found of those there |
|---|---|---|
| The manager and the PMO read it differentlyThe alert is right about the material: the minutes and the schedule carry different closing dates. The key does not yet model this cross-agent reading and counts it as an error (D012). | 0 of 1 | 0 of 0 |
| Milestone declared closed, no evidence | 83 of 83 | 83 of 83 |
| Overdue commitment | 39 of 39 | 39 of 39 |
| Green the evidence does not support | 30 of 30 | 30 of 30 |
| Declared progress the schedule does not support | 28 of 28 | 28 of 28 |
| Commitment with no date | 16 of 16 | 16 of 16 |
| Overdue supplier deliverable | 13 of 13 | 13 of 13 |
| Rescheduled commitment | 11 of 11 | 11 of 11 |
| Cost variance | 11 of 11 | 11 of 11 |
| Schedule variance | 9 of 9 | 9 of 9 |
| Deliverable accepted with no receipt | 7 of 7 | 7 of 7 |
| Overdue milestone with no evidence | 6 of 6 | 6 of 6 |
| Unauthorised rebaseline | 5 of 5 | 5 of 5 |
| Project gone silent | 3 of 3 | 3 of 3 |
| Approved change with no baseline | 2 of 2 | 2 of 2 |
| Invoiced above what was accepted | 2 of 2 | 2 of 2 |
| Invoiced with no delivery | 1 of 1 | 1 of 1 |
The defects, one by one60 logged
- D060El anexo de verificación decía «verificado con la entrevista con doce clientes», y esa entrevista no estaba en la carpeta; Alba no lo dio por verificado, con razón. El anexo contiene ahora toda su evidencia y no remite a otro documento.Origin: the test dataFix: 1bcb73f8
Verified: pending - D059El índice contaba como citas rotas las de la ficha del gerente (pm_record), que citan con las rutas de su propio estado: 18 en PRY-203. citas_rotas ya las excluía; el índice no.Origin: the codeFix: 77c3e6ad
Verified: pending - D058En el setup Vera dejó sin compromisos el proyecto de Samuel porque estaban en la ficha que él publicó; la ficha quedó sellada, el barrido no la releyó y faltaron 7 alertas de compromisos en PRY-203. Las dos fichas nunca se fusionan: ahora lo dicen el setup y el barrido, y compute lo señala en record_issues.Origin: the agent’s instructionsFix: 77c3e6ad
Verified: pending - D057La clave del contraste contó «no aplica» del gerente como un producto y esperaba ese pm_vs_pmo; la primera calificación de minería lo dio como acierto. Corregida antes de publicar: es una alerta de más (D056).Origin: the answer keyFix: 94addcf8
Verified: ronda-10 · la clave del contraste no esperó ningún pm_vs_pmo por «No aplica» - D056pm_vs_pmo comparaba el producto con value() y no con producto(): Samuel escribió «no aplica» como producto, la ficha de Vera no tenía ninguno, y sonó un contraste contra nada en PRY-211. El esquema dice que «No aplica» es vacío.Origin: the codeFix: 219a4469
Verified: ronda-10 · las cuatro industrias sobre 0.5.0 - D055D022 quedó a medias: en minería el informe de avance todavía podía decir al azar «El proveedor entregó los equipos en el campamento, pendientes de aceptación». Vera lo registró y el motor dio vendor_accepted_without_evidence en PRY-201, correcto sobre el material; la clave no lo modela. La frase se reemplazó.Origin: the test dataFix: 43cd7c8f
Verified: ronda-9 · las cuatro industrias sobre 0.4.0 - D054El informe salió en estado/reportes/ porque informe.py resolvía paths.reports contra la carpeta donde estaba el shell del agente, y Rostrum respondió 404 en las páginas del informe (394 de 409). Ahora se cuenta desde la raíz del espacio.Origin: the codeFix: 2631bbcc
Verified: ronda-9 · las cuatro industrias sobre 0.4.0 - D053D052 a medias: la definición ya decía con qué se verificó el supuesto, pero ese documento no estaba en la carpeta, y Alba no lo dio por verificado. El generador escribe ahora el anexo de verificación en la carpeta del producto.Origin: the test dataFix: 7597895d
Verified: ronda-8 · las cuatro industrias: ninguna alerta de más de Alba - D052La definición generada decía «verificado» sin decir con qué, y Alba, aplicando assumption-tracking (verificado solo contra algo que se pueda citar), levantó assumption_unverified. La clave lo daba por verificado. El material dice ahora con qué y cuándo se verificó.Origin: the test dataFix: 3aa2d272
Verified: ronda-8 · las cuatro industrias: ninguna alerta de más de Alba - D051D042 otra vez: Samuel dejó la primera fecha de un compromiso «por definir» como reprogramación en la ficha (1 de 104). El conteo de la alerta era correcto. compute devuelve ahora record_issues y los comandos de Samuel corrigen la ficha antes de terminar.Origin: the agent’s readingFix: e2b36df0
Verified: ronda-7 · las cuatro industrias - D050La página de una corrida de product-setup medía 616 px en un teléfono de 390: la tabla ya se desplazaba por dentro, pero su bloque no. El bloque se desplaza ahora él mismo.Origin: the codeFix: 82c35aa0
Verified: ronda-7 · las cuatro industrias - D049portfolio-scan y health-check dejaron el caché de texto en <estado>/texto, fuera de su contrato, aunque document-intake dice <estado>/cache/texto. La regla pasa al script: texto.py rechaza cualquier otra carpeta.Origin: the agent’s instructionsFix: 82c35aa0
Verified: ronda-7 · las cuatro industrias - D048pm-setup y pm-publish no decían de dónde sale plan.end_date, y el ejemplo de project-record ponía en value el documento más viejo, contra su propia regla. Ahora es la fecha vigente, del cronograma más reciente, y el acta va en la línea base.Origin: the agent’s instructionsFix: cb0d57b2
Verified: ronda-6 · las cuatro industrias - D047El generador escribía en el acta un cierre que el cronograma del mismo día no alcanzaba: dos documentos de la misma fecha con dos cierres. Samuel tomaba el del acta y Vera el último hito, y sonaba pm_vs_pmo en las cuatro industrias.Origin: the test dataFix: cb0d57b2
Verified: ronda-6 · las cuatro industrias - D046El motor hacía sonar commitment_rescheduled en un compromiso reprogramado tres veces y después cumplido con acta. La señal es para un bloqueo abierto; la clave del simulador ya lo excluía.Origin: the codeFix: 88e4386
Verified: selftest de portafolio.py: un compromiso cumplido no suena como reprogramado · ronda-6 · las cuatro industrias - D045Samuel escribió la cita de un proveedor con dos rutas («contrato.md; factura: …»), el error de D026 que el sello de Vera atrapa y el de Samuel no tiene. project-record ahora dice que una cita es una sola ruta y que la factura va en invoiced.Origin: the agent’s instructionsFix: 0bd6787
Verified: ronda-5 · cuatro industrias sobre 0.3.11: ninguna cita con dos rutas; 148 de 148 contratos - D044La ficha de Samuel traía una cita con dos rutas; como la de Vera incluye la de Samuel para el contraste, su sello se negó y la línea base del cronograma no se completó: rebaseline_unauthorized y variance_time no salieron. El sello de Vera ahora revisa solo sus propias citas.Origin: the codeFix: 0bd6787
Verified: ronda-5 · cuatro industrias sobre 0.3.11: el sello de Vera no se frena por citas ajenas; cobertura 100 % - D043Las páginas de los productos con nombre («tarjeta débito», «despacho de flota») respondían 404: la guarda de rutas solo aceptaba letras sin tilde, números, punto y guion. Ahora acepta letras de cualquier alfabeto y espacios, nunca una barra ni «..»; los enlaces además van codificados (31496ee).Origin: the codeFix: 0bd6787
Verified: ronda-5 · cuatro industrias sobre 0.3.11: Rostrum 2155 de 2155, los productos con nombre abren - D042Samuel contó como reprogramación la primera fecha de cuatro compromisos prometidos «por definir», el mismo error de Vera en D038. La regla estaba solo en el skill; ahora está en sus comandos y la aritmética no cuenta esa primera fecha.Origin: the agent’s instructionsFix: 5ae06a6
Verified: ronda-7 · las cuatro industrias - D041D028 volvió: Alba citó el acta del comité como evidencia de demanda y 11 evidence_stale no salieron. La corrección de la 0.3.9 estaba en demand-evidence, pero quien extrae los requerimientos es product-setup, que solo decía «la evidencia con su source_date». Ahora lo dice ahí.Origin: the agent’s instructionsFix: a46261d
Verified: ronda-5 · cuatro industrias sobre 0.3.11: Alba cita la entrevista; 150 de 150 alertas encontradas - D040Alba escribió el doliente como la cadena not_found y el motor la leyó como un nombre: requirement_without_owner no sonó. Es el mismo error de D018; ahora value() lo trata como vacío en los tres agentes.Origin: the codeFix: df7598f
Verified: ronda-5 · cuatro industrias sobre 0.3.11: Alba 150 de 150, ningún doliente sin señal - D039El barrido de pm-wake escribe compromisos en la ficha, como dice su diseño, pero su contrato solo declaraba cadencia.json; además dejó una copia .bak junto a la ficha.Origin: the agent’s instructionsFix: 7d48c2a
Verified: ronda-4 · cuatro industrias sobre 0.3.10: pm-wake cumple su contrato en las cuatro industrias - D038Vera contó como reprogramación la primera fecha de un compromiso prometido «por definir», y un compromiso movido dos veces sonó como commitment_rescheduled. commitment-tracking no decía que fechar no es reprogramar.Origin: the agent’s instructionsFix: 7d48c2a
Verified: ronda-4 · cuatro industrias sobre 0.3.10: ningún commitment_rescheduled falso en Vera - D037portfolio-report decía --salida <carpeta> sin decir cuál; Vera escribió el informe en una subcarpeta con fecha y Rostrum, que sirve paths.reports, respondió 404 en cada página del informe (366 de 380). Ahora el informe va siempre a paths.reports y el script rechaza otra carpeta.Origin: the agent’s instructionsFix: f1ac133
Verified: ronda-4 · cuatro industrias sobre 0.3.10: el informe queda en paths.reports y Rostrum lo sirve en las cuatro - D036Una ruta de archivo larga dentro de código no se partía y la página de una corrida medía 436 px en un teléfono de 390.Origin: the codeFix: 14f78e6
Verified: la página regenerada con 14f78e6 mide 390 px en un teléfono de 390 - D035pm-setup leía solo la última reunión y ofrecía leer las anteriores; en una sesión sin nadie que respondiera, Samuel nunca las leyó y su ficha tuvo 3 de los 26 compromisos del proyecto. En cooperativas, retail y minería los recuperó pm-commitments y acertó 77 de 78: dependía de la sesión. Ahora pm-setup lee todas las minutas en orden.Origin: the agent’s instructionsFix: 4ba01bd
Verified: ronda-4 · cuatro industrias sobre 0.3.10: Samuel lee todas las minutas en las cuatro; 100 de 104 compromisos - D034La prueba de Rostrum de cada ronda pedía siete rutas y miraba solo que respondieran 200, y corría el servidor del repositorio en lugar del instalado. Por eso D030 a D033 pasaron la ronda 1 y la 2 sin que nada fallara. Ahora recorre cada enlace con la versión instalada, busca rutas, palabras del esquema y claves de señal, prueba una petición y revisa que no escriba fichas.Origin: the test harnessFix: fa65257
Verified: ronda.py rostrum sobre las cuatro industrias de la ronda 2 - D033En un teléfono (390 px) las tablas empujaban la página más allá de la pantalla en 62 vistas: proyectos, productos, cada proyecto, cada producto y las páginas de corrida. D017 había resuelto los bloques, no las tablas sueltas. Ahora cada tabla se desplaza dentro de su caja.Origin: the codeFix: fa65257
Verified: ronda-2 · bancos regenerado con fa65257, navegador en la nube: 0 de 147 rutas se salen de la pantalla a 390 y 1280 px - D032La historia de cada caso, lo que se arrastra y la vista de caso mostraban la clave de la señal (milestone_met_without_evidence) en lugar de su nombre. Ahora usan los nombres del informe, y un selftest comprueba que toda señal del motor tenga uno.Origin: the codeFix: fa65257
Verified: ronda-2 · las cuatro industrias regeneradas con fa65257: 0 rutas rotas y 0 palabras crudas en el informe - D031El informe nombraba los campos sin dato por su ruta: identity.phase en los 52 proyectos, declared.*, pm_record.* y vendors[].invoiced. El léxico se había quedado atrás del esquema; ahora lo cubre entero y un selftest lee schema.yaml para que no vuelva a pasar. La vista de cortes también nombra los campos.Origin: the codeFix: fa65257
Verified: ronda-2 · las cuatro industrias regeneradas con fa65257: 0 rutas rotas y 0 palabras crudas en el informe - D030El menú de Rostrum enlaza «Qué cambió entre dos cortes» y, con un solo corte —la primera corrida—, la página respondía 404 en las cuatro industrias. Ahora dice que la comparación llega con la próxima corrida.Origin: the codeFix: fa65257
Verified: ronda-2 · las cuatro industrias regeneradas con fa65257: 0 rutas rotas y 0 palabras crudas en el informe - D029La clave de las notas contó las fechas del simulador (dos) y en la carpeta quedó una sola minuta con una fecha: esperaba dos reprogramaciones donde el material sostiene una. Samuel registró una; su lectura fue la correcta.Origin: the answer keyFix: 7b5ab31
Verified: armar cooperativas --ronda 2 sobre 7b5ab31: la clave espera 1 reprogramación, la que muestra el material - D028Alba citó el acta del comité como evidencia de demanda en lugar de la entrevista: la evidencia quedó con la fecha de la reunión y cinco evidence_stale no salieron. demand-evidence no decía que el acta que acepta es la decisión, no la demanda.Origin: the agent’s instructionsFix: 7b5ab31
Verified: ronda-3 · cuatro industrias sobre 0.3.9: evidence_stale encontrado en las cuatro industrias; Alba 142 de 143 - D027Nueve requerimientos aceptados en un acta de comité salieron como decision_without_source: en bancos la cita quedó en un campo que el esquema no tiene; en minería Alba dejó decision vacía porque el acta era una revisión posterior. El skill no decía que la fuente del estado va en decision.Origin: the agent’s instructionsFix: 7b5ab31
Verified: ronda-3 · cuatro industrias sobre 0.3.9: Alba 142 de 142 alertas correctas, cero decision_without_source falsos - D026Una ficha juntó dos rutas en un solo source («contrato.md; factura: …») y sellar la dio por leída: nadie que abra la carpeta puede seguir esa cita. sellar no revisaba que las citas fueran documentos que existen.Origin: the codeFix: 7b5ab31
Verified: selftest de portafolio.py (una cita con dos rutas no se sella) y las fichas reales de la ronda 2: solo esa cita sale rota - D025D014 volvió en retail: Samuel escribió la promesa original como una reprogramación. El skill commitment-tracking ya decía que la original no es una entrada, pero pm-minutes y pm-commitments seguían diciendo que «la fecha anterior entra en su historial».Origin: the agent’s instructionsFix: 7b5ab31
Verified: ronda-3 · cuatro industrias sobre 0.3.9: ningún compromiso con la promesa original contada como reprogramación - D024El barrido convirtió los documentos con texto.py y dejó el caché en estado/texto/, fuera de lo que declara su contrato: ningún comando ni skill decía dónde va el caché. Es la única falla de contrato de bancos.Origin: the agent’s instructionsFix: 7b5ab31
Verified: ronda-3 · cuatro industrias sobre 0.3.9: el caché queda en <estado>/cache/texto/ y el barrido cumple su contrato en las cuatro - D023vendor-control decía que un entregable sin documento es unknown, no delivered; project-record y el motor dicen que state es lo declarado y evidence lo probado. Dos skills del mismo agente daban reglas opuestas para el mismo campo.Origin: the agent’s instructionsFix: 7b5ab31
Verified: ronda-3 · cuatro industrias sobre 0.3.9: cero falsos positivos de proveedor en Vera (252 de 256) - D022El generador escribe al azar en el informe de avance «El proveedor entregó el componente de integración, pendiente de aceptación», sin acta y sin modelarlo en la clave. Vera lo registró como entregado sin evidencia en cuatro proyectos y el motor dio vendor_accepted_without_evidence: la señal es correcta sobre el material; la clave no la lista.Origin: the test dataFix: 7b5ab31
Verified: ronda-3 · cuatro industrias sobre 0.3.9: sin la frase al azar, cero vendor_accepted_without_evidence fuera de la clave - D021La guía de aceptación describía un bloque YAML en «un archivo local de la empresa»; lo real era JSON en .criterio/<agente>/config.json, sin huella ni comprobante. Ahora describe lo que el código hace: comprobante en Markdown y PDF, archivo .sha256, bitácora encadenada y registro de versiones.Origin: the agent’s instructionsFix: 45d62ff
Verified: validate_plugins y coherencia en verde con la versión 1.1 - D020El descargo remitía a TERMS.md, que estaba en la raíz del repositorio y no dentro de cada plugin: quien instalaba un agente no tenía garantizado el texto que aceptaba. Samuel y Alba además decían «muestra el descargo corto» sin el texto. Ahora los términos viajan byte a byte en cada plugin y los tres muestran la misma sección «En corto».Origin: the agent’s instructionsFix: 45d62ff
Verified: puerta de sincronización: los términos y el registro viajan idénticos en los tres plugins - D019Los términos prometían volver a pedir aceptación cuando cambiaran, pero la verificación de la configuración solo revisaba que hubiera una versión y una fecha: nunca las comparaba con los términos vigentes ni con la versión instalada. Ahora la puerta es código, en config y en corrida-inicio: una aceptación por versión de la familia, amarrada al texto por su huella SHA-256 y a su comprobante.Origin: the codeFix: 45d62ff
Verified: selftest de terminos.py: una aceptación vieja, otra versión o un comprobante tocado no abren la puerta - D018En minería Vera escribió «not_found» como valor de la autoridad del gerente y de la etapa, además de marcarlo como no encontrado, y la página de cada proyecto imprimió la palabra. El informe ahora lee el estado: un campo no encontrado dice «no está dicho en ninguna parte».Origin: the codeFix: 5295fa0
Verified: ronda-1 · minería (informe regenerado, PRY-208) - D017En la historia de corridas de Rostrum la tabla de diez columnas se salía de su bloque por la derecha y cortaba las últimas columnas. Los bloques ahora se desplazan de lado dentro de su borde.Origin: the codeFix: 4b15b73
Verified: ronda-1 · minería (página de corridas) - D016La vista por producto de Rostrum mostró «No aplica» como un producto construido por siete proyectos: el acta dice que el proyecto no sirve a un producto, y la ficha copió las palabras. El código ahora lee esas palabras como un campo vacío y esos proyectos pasan a «proyectos que no dicen qué producto construyen».Origin: the codeFix: 4b15b73
Verified: ronda-1 · minería (informe regenerado: 6 productos, sin «No aplica») - D015El calificador leía el estado de cada agente en una carpeta fija. En retail Vera guardó su portafolio en estado-portafolio, como le permite su configuración, y la calificación dio 0 de 36 sin que el agente hubiera fallado. Ahora el calificador sigue la configuración de cada agente, igual que ellos.Origin: the test harnessFix: eb76825
Verified: ronda-1 · retail (2026-10-14-retail-vera-9) - D014Samuel escribió la promesa original como una reprogramación más: un compromiso dicho dos veces quedó con dos reprogramaciones. En bancos lo había escrito sin la original. El esquema no decía cuál de las dos formas es la correcta, y el conteo que dispara la alerta de compromiso reprogramado depende de eso.Origin: the agent’s instructionsFix: 5e81ef7
Verified: ronda-3 · cuatro industrias sobre 0.3.9: Samuel escribe la promesa original aparte; 77 de 78 compromisos con sus reprogramaciones bien contadas donde leyó las minutas - D013La clave de las notas quedó dentro de la carpeta del agente. La auditoría de las herramientas que llamó cada sesión no encontró ninguna lectura en la corrida válida; desde cooperativas va fuera, y la auditoría de fugas corre en cada industria.Origin: the test harnessFix: 286ac45
Verified: ronda-1 · cooperativas (auditoría de fugas: 0) - D012pm_vs_pmo en PRY-202: el acta y el cronograma traen fechas de cierre distintas y cada agente leyó una. La alerta es correcta; la clave no modela lo que cruzan dos agentes.Origin: the answer keyFix: cb0d57b2
Verified: ronda-6 · las cuatro industrias - D011Las notas crudas se fecharon el mismo día que la minuta del simulador y con otra fecha para el mismo compromiso; la clave esperaba una reprogramación. Samuel señaló el conflicto entre las dos fuentes y no contó una reprogramación sin promesa anterior: su lectura fue la correcta.Origin: the test dataFix: 286ac45
Verified: ronda-1 · cooperativas - D010Alba guardó su estado en estado/, la misma carpeta que usa Vera: comparten corridas.json y cadencia.json. Una ficha, un escritor: cada agente necesita su propia carpeta de estado.Origin: the agent’s instructionsFix: db80a49
Verified: ronda-2 · cuatro industrias: Alba guarda su estado en estado-producto/<código>, separado del de Vera - D009La cadena de Alba nunca construye la ficha del producto (producto.json) ni el registro de requerimientos desde los documentos: los requerimientos están en el acta del comité y ningún comando los registra. Los 11 comandos trabajaron sobre un estado vacío y Alba encontró 0 de las 10 alertas de su producto.Origin: the agent’s instructionsFix: db80a49
Verified: ronda-2 · cuatro industrias: Alba construye producto y requerimientos y encuentra 38 de 43 (0 de 32 en la ronda 1) - D008Un hito con estado ambiguo se contaba como cerrado y disparó progress_vs_plan en el control negativo.Origin: the codeFix: 8c84264
Verified: 2026-10-14-cooperativas-vera-8 - D007El generador escribía avances al azar que daban por cerrado un hito que el cronograma tenía en curso.Origin: the test dataFix: 8c84264
Verified: 2026-10-14-bancos-vera-7 - D006El generador escribía actas de recibo con fecha futura; el agente no las aceptó como evidencia y la clave sí.Origin: the test dataFix: 8c84264
Verified: 2026-10-14-bancos-vera-7 - D005Los proyectos de control se llamaban «Control negativo»: el agente podía leer la respuesta en el nombre de la carpeta.Origin: the test dataFix: 39f574b
Verified: 2026-10-14-retail-vera-5 - D004La clave escrita a mano estaba desactualizada: tres fechas son dos reprogramaciones, un compromiso que el material daba por cumplido, y alertas que el material plantaba y la clave no listaba.Origin: the answer keyFix: 9fd41a8
Verified: 2026-10-14-bancos-vera-2 - D003El hash de texto devolvía vacío justo para Word y Excel, y la línea base solo se reconstruía desde CSV: un cronograma en Excel no dejaba línea base.Origin: the codeFix: 39f574b
Verified: puerta formatos.py - D002progress_vs_plan medía el avance contra los hitos cerrados (un punto) y no contra el tramo posible: disparó en 6 de 10 proyectos, incluido el control negativo.Origin: the codeFix: 9fd41a8
Verified: 2026-10-14-bancos-vera-2 - D001La alerta de silencio nunca se calculaba: ninguna ficha traía activity y el esquema dice que la calcula el script. El proyecto con 52 días sin documentos salió sin alerta.Origin: the codeFix: 9fd41a8
Verified: 2026-10-14-bancos-vera-2
Every graded run46 runs
| Test data | Run | Reliability · correct alerts of those raised | Coverage · alerts found of those there | Negative control clean | Errors by originagent · code · key |
|---|---|---|---|---|---|
| Retail13 projects · 2026-10-18 | First runv0.5.0 · round 10 | 63 of 63 · 100%95% CI 94.3–100% | 63 of 63 · 100%95% CI 94.3–100% | 1 of 1 | — |
| Mining13 projects · 2026-10-18 | First runv0.5.0 · round 10 | 47 of 47 · 100%95% CI 92.4–100% | 47 of 54 · 87%95% CI 62.8–100% | 1 of 1 | — |
| Savings and credit cooperatives13 projects · 2026-10-18 | First runv0.5.0 · round 10 | 65 of 65 · 100%95% CI 94.4–100% | 65 of 65 · 100%95% CI 94.4–100% | 1 of 1 | — |
| Banks13 projects · 2026-10-18 | First runv0.5.0 · round 10 | 67 of 67 · 100%95% CI 94.6–100% | 67 of 67 · 100%95% CI 94.6–100% | 1 of 1 | — |
| Mining13 projects · 2026-10-17 | First runv0.4.0 · round 9 | 66 of 67 · 98.5%95% CI 92–100% | 66 of 66 · 100%95% CI 94.5–100% | 1 of 1 | — |
| Retail13 projects · 2026-10-17 | First runv0.4.0 · round 9 | 90 of 90 · 100%95% CI 95.9–100% | 90 of 90 · 100%95% CI 95.9–100% | 1 of 1 | — |
| Savings and credit cooperatives13 projects · 2026-10-17 | First runv0.4.0 · round 9 | 52 of 52 · 100%95% CI 93.1–100% | 52 of 52 · 100%95% CI 93.1–100% | 1 of 1 | — |
| Banks13 projects · 2026-10-17 | First runv0.4.0 · round 9 | 58 of 58 · 100%95% CI 93.8–100% | 58 of 58 · 100%95% CI 93.8–100% | 1 of 1 | — |
| Mining13 projects · 2026-10-17 | First runv0.3.13 · round 8 | 51 of 52 · 98.1%95% CI 89.9–100% | 51 of 51 · 100%95% CI 93–100% | 1 of 1 | — |
| Retail13 projects · 2026-10-17 | First runv0.3.13 · round 8 | 73 of 73 · 100%95% CI 95–100% | 73 of 73 · 100%95% CI 95–100% | 1 of 1 | — |
| Savings and credit cooperatives13 projects · 2026-10-17 | First runv0.3.13 · round 8 | 69 of 69 · 100%95% CI 94.7–100% | 69 of 69 · 100%95% CI 94.7–100% | 1 of 1 | — |
| Banks13 projects · 2026-10-17 | First runv0.3.13 · round 8 | 60 of 60 · 100%95% CI 94–100% | 60 of 60 · 100%95% CI 94–100% | 1 of 1 | — |
| Mining13 projects · 2026-10-17 | First runv0.3.13 · round 7 | 86 of 86 · 100%95% CI 95.7–100% | 86 of 86 · 100%95% CI 95.7–100% | 1 of 1 | — |
| Retail13 projects · 2026-10-17 | First runv0.3.13 · round 7 | 61 of 61 · 100%95% CI 94.1–100% | 61 of 61 · 100%95% CI 94.1–100% | 1 of 1 | — |
| Savings and credit cooperatives13 projects · 2026-10-17 | First runv0.3.13 · round 7 | 66 of 66 · 100%95% CI 94.5–100% | 66 of 66 · 100%95% CI 94.5–100% | 1 of 1 | — |
| Banks13 projects · 2026-10-17 | First runv0.3.13 · round 7 | 67 of 67 · 100%95% CI 94.6–100% | 67 of 67 · 100%95% CI 94.6–100% | 1 of 1 | — |
| Retail13 projects · 2026-10-16 | First runv0.3.12 · round 6 | 76 of 76 · 100%95% CI 95.2–100% | 76 of 76 · 100%95% CI 95.2–100% | 1 of 1 | — |
| Mining13 projects · 2026-10-16 | First runv0.3.12 · round 6 | 60 of 60 · 100%95% CI 94–100% | 60 of 60 · 100%95% CI 94–100% | 1 of 1 | — |
| Savings and credit cooperatives13 projects · 2026-10-16 | First runv0.3.12 · round 6 | 52 of 52 · 100%95% CI 93.1–100% | 52 of 52 · 100%95% CI 93.1–100% | 1 of 1 | — |
| Banks13 projects · 2026-10-16 | First runv0.3.12 · round 6 | 78 of 78 · 100%95% CI 95.3–100% | 78 of 78 · 100%95% CI 95.3–100% | 1 of 1 | — |
| Mining13 projects · 2026-10-16 | First runv0.3.11 · round 5 | 56 of 58 · 96.6%95% CI 88.3–100% | 56 of 56 · 100%95% CI 93.6–100% | 1 of 1 | 0 · 1 · 1 |
| Retail13 projects · 2026-10-16 | First runv0.3.11 · round 5 | 53 of 54 · 98.1%95% CI 90.2–100% | 53 of 53 · 100%95% CI 93.2–100% | 1 of 1 | 0 · 0 · 1 |
| Savings and credit cooperatives13 projects · 2026-10-16 | First runv0.3.11 · round 5 | 65 of 66 · 98.5%95% CI 91.9–100% | 65 of 65 · 100%95% CI 94.4–100% | 1 of 1 | 0 · 0 · 1 |
| Banks13 projects · 2026-10-16 | First runv0.3.11 · round 5 | 49 of 50 · 98%95% CI 89.5–100% | 49 of 49 · 100%95% CI 92.7–100% | 1 of 1 | 0 · 0 · 1 |
| Mining13 projects · 2026-10-16 | First runv0.3.10 · round 4 | 51 of 51 · 100%95% CI 93–100% | 51 of 53 · 96.2%95% CI 87.2–100% | 1 of 1 | 0 · 2 · 0 |
| Retail13 projects · 2026-10-16 | First runv0.3.10 · round 4 | 55 of 56 · 98.2%95% CI 90.6–100% | 55 of 55 · 100%95% CI 93.5–100% | 1 of 1 | 0 · 0 · 1 |
| Savings and credit cooperatives13 projects · 2026-10-16 | First runv0.3.10 · round 4 | 63 of 64 · 98.4%95% CI 91.7–100% | 63 of 63 · 100%95% CI 94.3–100% | 1 of 1 | 0 · 0 · 1 |
| Banks13 projects · 2026-10-16 | First runv0.3.10 · round 4 | 59 of 60 · 98.3%95% CI 91.1–100% | 59 of 59 · 100%95% CI 93.9–100% | 1 of 1 | 0 · 0 · 1 |
| Mining13 projects · 2026-10-15 | First runv0.3.9 · round 3 | 61 of 63 · 96.8%95% CI 89.1–100% | 61 of 61 · 100%95% CI 94.1–100% | 1 of 1 | 1 · 0 · 1 |
| Retail13 projects · 2026-10-15 | First runv0.3.9 · round 3 | 60 of 61 · 98.4%95% CI 91.3–100% | 60 of 60 · 100%95% CI 94–100% | 1 of 1 | 0 · 0 · 1 |
| Savings and credit cooperatives13 projects · 2026-10-15 | First runv0.3.9 · round 3 | 64 of 64 · 100%95% CI 94.3–100% | 64 of 64 · 100%95% CI 94.3–100% | 1 of 1 | — |
| Banks13 projects · 2026-10-15 | First runv0.3.9 · round 3 | 67 of 68 · 98.5%95% CI 92.1–100% | 67 of 67 · 100%95% CI 94.6–100% | 1 of 1 | 0 · 0 · 1 |
| Mining13 projects · 2026-10-15 | First runv0.3.8 · round 2 | 43 of 44 · 97.7%95% CI 88.2–100% | 43 of 43 · 100%95% CI 91.8–100% | 1 of 1 | 0 · 0 · 1 |
| Retail13 projects · 2026-10-15 | First runv0.3.8 · round 2 | 60 of 60 · 100%95% CI 94–100% | 60 of 60 · 100%95% CI 94–100% | 1 of 1 | — |
| Savings and credit cooperatives13 projects · 2026-10-15 | First runv0.3.8 · round 2 | 58 of 59 · 98.3%95% CI 91–100% | 58 of 58 · 100%95% CI 93.8–100% | 1 of 1 | 0 · 0 · 1 |
| Banks13 projects · 2026-10-15 | First runv0.3.8 · round 2 | 62 of 67 · 92.5%95% CI 83.7–97.1% | 62 of 62 · 100%95% CI 94.2–100% | 1 of 1 | 0 · 0 · 5 |
| Mining13 projects · 2026-10-14 | First runv0.3.6 · round 1 | 37 of 39 · 94.9%95% CI 83.1–100% | 37 of 37 · 100%95% CI 90.6–100% | 1 of 1 | 0 · 0 · 2 |
| Retail13 projects · 2026-10-14 | First runv0.3.6 · round 1 | 36 of 37 · 97.3%95% CI 86.2–100% | 36 of 36 · 100%95% CI 90.4–100% | 1 of 1 | 0 · 0 · 1 |
| Savings and credit cooperatives13 projects · 2026-10-14 | First runv0.3.6 · round 1 | 60 of 60 · 100%95% CI 94–100% | 60 of 60 · 100%95% CI 94–100% | 1 of 1 | — |
| Banks13 projects · 2026-10-14 | First runv0.3.6 · round 1 | 53 of 54 · 98.1%95% CI 90.2–100% | 53 of 53 · 100%95% CI 93.2–100% | 1 of 1 | 0 · 0 · 1 |
| Mining13 projects · 2026-10-14 | First runv0.3.5 | 53 of 54 · 98.1%95% CI 90.2–100% | 53 of 53 · 100%95% CI 93.2–100% | 1 of 1 | 0 · 0 · 1 |
| Retail13 projects · 2026-10-14 | First runv0.3.5 | 49 of 49 · 100%95% CI 92.7–100% | 49 of 49 · 100%95% CI 92.7–100% | 1 of 1 | 0 · 0 · 0 |
| Savings and credit cooperatives13 projects · 2026-10-14 | First runv0.3.5 | 34 of 39 · 87.2%95% CI 66.7–100% | 34 of 34 · 100%95% CI 89.8–100% | 0 of 1 | 0 · 3 · 2 |
| Banks13 projects · 2026-10-14 | First runv0.3.5 | 51 of 53 · 96.2%95% CI 86–100% | 51 of 51 · 100%95% CI 93–100% | 1 of 1 | 0 · 0 · 2 |
| Banks10 projects · hand-written cases · 2026-10-14 | Corrected after seeing itv0.3.5 | 29 of 31 · 93.5%95% CI 79.3–100% | 29 of 29 · 100%95% CI 88.3–100% | 1 of 1 | 0 · 0 · 2 |
| Banks10 projects · hand-written cases · 2026-10-14 | First runv0.3.4 | 21 of 33 · 63.6%95% CI 40–84.8% | 21 of 25 · 84%95% CI 65.3–100% | 0 of 1 | 0 · 10 · 6 |
A first run is one by a version that had not seen that material: it is the measurement that counts. A corrected run is the same family of material after fixing the code while looking at it; it is not independent, it is published to show the effect of the fix and it is never added to the first runs.
The margin is the wider of Wilson’s interval and a per-project bootstrap: the alerts of one project fail together, so they are not counted as independent.
The origin of each error is classified by a person reviewing it: the model’s reading, the code that computes, or the answer key. It is the only figure that does not come from a script.
The engine that computesRuns on every change · latest 2026-10-15: 5 of 5 cuts
Arithmetic, dates, thresholds and alerts, on well-built records and without the model. A deterministic computation has no random error to estimate: the rule matches the key or it does not. So it is reported as regression —how often it matched— and not as an error rate.
Runs on every code verification. Latest run: 2026-10-15, on commit c97cd8e: 5 of 5 cuts match the key. The first one was run by hand, on commit 56a1478.
5 / 5
Cuts matching the key
Five weeks of the same organisation, written a different way each time.
117
Cases
51 projects and 66 products. Weeks repeat cases, so they are not added up.
2 / 2
Clean negative controls
Healthy cases that must not raise a single alert, in every cut.
3.1
ms per document
The script alone, without the model.
Weekly cuts of the longitudinal test
A synthetic organisation of 51 projects and 66 products. The same material is written a different way every week and the expected answer does not change.
Cut 1
ordered
✓ 100%
461 docs
Cut 2
flat
✓ 100%
526 docs
Cut 3
shuffled
✓ 100%
569 docs
Cut 4
no folders
✓ 100%
616 docs
Cut 5
ordered
✓ 100%
641 docs
The output did not change with the shape of the folder. Median: 3.1 ms per document. ✓ negative controls at zero.
Code verification
Full run of 2026-10-18
- gates green
- 33/33
- gates green
- checks
- 738
- checks
- corpora with a negative control
- 14
- corpora with a negative control
- seconds for the whole run
- 31.9
- seconds for the whole run
Checks by owner
Vera227
Samuel147
Alba107
Rostrum67
La familia172
It shows where verification sits; the sizes are not comparable with each other.
What comes next
- The first use on a specific organisation’s documentation: the step that decides version 1.0.
- Each industry has thirteen projects per round, and every figure is published with its confidence margin.
- Graded against a key: Vera’s alerts, the commitments Samuel takes from minutes and raw notes, with the close date he publishes, and the alerts on Alba’s product. The other commands are checked against their contract —what they must write and must not—, not against a key.
- The model’s writing (minutes, reports, interview synthesis) is not measured by a deterministic grader; the arithmetic behind its figures is.
A results page that only says what passed is advertising. What makes it auditable is what it says is not known yet.
How they work
Five steps and one boundary. The language model (LLM) reads and extracts; the code computes. It is generative AI plus a deterministic engine, applied to the project office.
- 01
Your documents
Charters, schedules, minutes, status reports, spreadsheets, emails. However they are, wherever they are.
It does not need the folder to be tidy.
The model extracts - 02
Text
Each document is turned into text: Word, Excel, PowerPoint, PDF and email. Whatever it cannot read, it declares, with the reason.
Two fingerprints per document: only what changed is read again.
The model extracts - 03
The record
One per project. Every data point with a citation to the document it came from and its date. Declared and evidenced go in separate fields.
"Not found" is a valid answer.
The model extracts - 04
Arithmetic
Variance against the original and the current baseline, days of silence, committed against executed, overdue commitments, who is affected if a project moves.
Deterministic and tested.
The code computes - 05
Signals
Only what crossed a threshold: what changed, what contradicts itself, what no document supports. If nothing crossed one, nothing is said.
Silence is the feature.
The code computes
No number in a report comes from an estimate. And what is stated nowhere is declared, not filled in.
Why this boundary matters →Seven invariants that hold them up
01
The agents never talk to each other
They talk through the project record. Two agents that talk directly are two agents that have to be deployed together.
02
Rostrum does not write
The server publishes the report and takes questions. It never touches the record.
03
The source can change; the record does not
If the documents come from another system tomorrow, a new adapter fills the same fields.
04
Declared and evidenced never merge
What the manager states and what the documents support live in separate fields. The gap between the two is usually the most valuable finding.
05
The model extracts, the code computes
Reading a date is reading. Subtracting, projecting and adding is arithmetic, and arithmetic lives in code, where it can be tested.
06
No agent writes the declared status
A person writes it. If an agent did, comparing what is declared against the evidence would compare the system with itself. It is a restriction on the write path, and it fails if tried.
07
One record, one writer
If Samuel and Vera read the same project, they write two records and never merge them. What would have been a write conflict becomes the signal.
Installation, setup and terms
It installs in minutes. Before the first analysis, you accept the terms.
Three steps, and none of them asks you to edit a file. You install only the agent for the role you have; if you install more than one, they find each other through the project record: there is nothing to connect.
Step 1
Install
- 01Customize → Browse plugins → Personal.
- 02Press + → Add marketplace from GitHub.
- 03Enter josenanez/criterio.
- 04Install the agent for your role: Vera if you run the portfolio, Samuel if you run a project, Alba if you define a product.
What it needs from you
- A folder with the documentation, as it is. A messy folder is the normal case.
- A paid Claude plan, and the desktop app if the documents live on your machine.
- Fifteen minutes for Vera; ten for Samuel or Alba.
Step 2
Set up
Each agent comes with its own setup command. It looks at your folders, asks a few questions and shows you a first result on your own documents.
/criterio-portfolio:portfolio-setup
- 1Where the documents are
- 2Who you are
- 3When your committee meets
- 4Who receives the report, and in what form
- 5The terms
How it asks
- One question at a time. Never a form.
- Every question comes with a suggested answer, and “I don’t know” is a valid answer.
- It does not ask what it can find out by looking at the folder.
- Nobody edits the configuration by hand: if you want to change something, you say so in the conversation.
None of them schedules itself. Each comes with the command a clock calls (portfolio-wake, pm-wake, product-wake), and someone sets it once as a scheduled task. If your organisation does not want unattended runs, they are run by hand.
/criterio-portfolio:portfolio-setupWhat Vera’s setup looks like
VeraI am Vera, Criterio’s PMO agent. I process the documentation you already have and say what does not hold up. Where are your project documents?
An illustrative walk-through on the repository’s test folder: 6 projects, 28 documents. Nothing leaves your browser.
Step 3
The terms
Nobody reads a disclaimer in a repository. So the warning reaches the person who runs the agent and the person who later receives the document, in four layers that all three agents implement.
- 1
One acceptance per version
Every version of the family is accepted again and leaves on your machine a receipt in Markdown and PDF, with the SHA-256 fingerprint of the terms and of the receipt itself. Until the one for the installed version is recorded, the agent answers questions but produces no full analysis. It never assumes it or infers it from you continuing to talk.
- 2
On every report
Every report carries a footer saying it is a draft, based on which documents, how many fields had no data and how many contradictions remain open. The document travels by email and reaches a committee: the warning travels with it.
- 3
At the start of every run
Before the first finding it says which records it loaded, when they were refreshed, and whether any is stale or missing.
- 4
In the legal notice
The full terms, the legal notice and the version register —what changed in each version, which terms govern it and their fingerprint— are published on this site, and every receipt points to them.
What you agree to, in plain words
- You get working drafts ready for review, and every version arrives certified: its evidence is public.
- Every data point carries its citation: whoever decides checks it in the original document in seconds.
- It works with what is written; what was agreed outside the documents comes from your team.
- Your documents are processed on the AI platform where you run it; your organisation decides which folders go in.
- Warranty and liability: those of the Apache 2.0 licence.
- Every version is accepted again and leaves a receipt in Markdown and PDF. If anyone modifies it, its SHA-256 fingerprint stops matching and it is void.
Terms version 1.1, of 2026-10-15, for version 0.5.0 of the family. Every new version asks for acceptance again.
Legal notice and version register →Apache 2.0
The method, the record schema, the code that computes and the acceptance criteria, in full and with no conditions. Use it inside your organisation without asking, adapt it, and build the capability you are missing: the agent for a function should be written by someone who knows the work.
What it does not do
- It produces working drafts, not decisions.
- It knows what was written, not what was said outside the documents.
- All three were tested round after round on test data from four industries, more demanding than usual documentation; their first use on a specific organisation’s documentation is the next step.
Frequently asked questions
What people usually ask when they reach the family
What it is
What do Vera, Samuel, Alba and Rostrum do?
They process what the project office has already written. Vera builds one record per project with the citation of every data point and consolidates the portfolio; Samuel keeps the commitments from each meeting of a project; Alba checks a product's definition against the evidence of demand; Rostrum publishes the report for whoever never opens a folder.
Do they replace the PMO?
No. They prepare working drafts and say what changed, what contradicts what and what no document supports. No agent declares a project's status: a person does.
What do I need to use them?
A paid Claude plan, the desktop app if the documents live on your computer, and a folder with the documentation as it is. A messy folder is the normal case.
How it is tested
What data was it tested on?
On test data from four industries —financial institutions (banks and credit unions), retail and mining in South America—, built to be more demanding than those organisations' usual documentation: over 500 documents per round in Markdown, Word, PDF, Excel and CSV; contradictions between documents; commitments moved up to six times; milestones declared met with no receipt; projects gone silent; vendor deliveries with no evidence; and in each industry one healthy project that must raise no alert.
How do you know an agent got it right?
Each round comes with a key of what the material holds, derived from the facts the generator wrote. No agent sees it: every session is audited and the number of reads is published (zero in all of them). Every figure is published with its count and its 95% confidence margin.
Where is the evidence?
In the repository, under tests/evidencias/: what each command showed in each round, the key, the grading, and every defect with its origin, the commit that fixes it and the round that verified it. The family's page summarises it.
Has it been used in an organisation yet?
Not yet on a specific organisation's documentation: that is the next step, and what will decide version 1.0. If your PMO wants to be among the first, open an issue in the repository.
Data and privacy
Do my documents leave my organisation?
They are processed on the AI platform where the agent runs, like any conversation with it. The plugins do not connect to the internet and send us nothing.
What is stored, and where?
On your computer: the records, the reports, the log of each run and the receipt for accepting the terms. None of it travels anywhere.
Do you collect usage statistics?
No. Today the only way back is the one you choose: an issue on GitHub. Sharing findings to improve the family will always be a question, and nothing is ever sent without a yes.
Versions and licence
Why is the version not 1.0 yet?
Because the record schema may change with the first use on real documentation, and a 1.0 promises it will not. 1.0 comes when the schema survives that use.
How much does it cost?
Nothing. It is open source under Apache 2.0: use it in your organisation without asking permission, and adapt it. The contribution on the page is voluntary and unlocks nothing.
What do I accept before using it?
The terms of each version. The agent shows them, asks for an explicit yes and leaves a receipt in Markdown and PDF with the SHA-256 fingerprint of the terms. Every new version is accepted again.
Help
How do I report an error?
With an issue in the repository and the document that shows it, with nobody's real data. Security flaws go through GitHub's private report, never in a public issue.
Can I build an agent for another function?
Yes. The repository explains the method and the acceptance criteria, and the finance and legal families are declared and not yet built: a function's agent should be written by someone who knows the work.
The same answers live in the repository, in families/criterio-pmo/PREGUNTAS.md.
Support the work
If Criterio created value for you, help keep it moving.
Criterio remains open source. A voluntary contribution helps fund testing, documentation and the next capabilities without changing access to the software.
This is a voluntary contribution. It does not unlock features, purchase consulting services or create an advisory relationship.
Support Criterio
Choose an amount and how you want to contribute.
Choose your country to see the payment methods available to you.
dLocal Go currently covers Latin America, Indonesia, Malaysia, Kenya and Nigeria. Global coverage will be added separately.
The point
The evidence was always there. Nobody had read it together.
An agent that processes your portfolio will not hand you a governance you never built. It will show you, in an afternoon, exactly where it is missing — and that is the only honest place to start.
Principles
- Approach
- Strategy and execution.
- Commitment
- Generating, sharing and preserving knowledge is a professional obligation.
- What it is not
- It does not replace your PMO or decide for it: it processes what your team has already written, prepares the work and says what does not hold up.
- Licence
- Apache 2.0.
- What comes next
- More capabilities to improve the workplace.
Criterio is open source under Apache 2.0. The project governance family (Vera, Samuel, Alba and Rostrum) is built and tested on test data from financial entities (banks and cooperatives), retail and mining in South America; the CFO and CLO capabilities are declared and not yet built. If you practise one of them and want to build its agent, the repository explains how.
Legal notice and version register →José Ñáñez