CatchBench :: PRE board (offline; no GRADE bridge, no corpus download)

PRE over_privilege: 1187 configs across 6 corpora {'crewai': 298, 'injecagent': 340, 'mcp': 144, 'n8n': 219, 'sweagent': 130, 'synthetic': 56}

[PRE] pre_over_privilege :: multi
  method                          precision    recall        f1  coverage
  flag_all                            0.430     1.000     0.601     1.000
  flag_none                           0.000     0.000     0.000     1.000
  flag_risky_perms                    0.418     0.564     0.480     1.000
  owasp_excess_permissions            0.504     0.506     0.505     1.000
  owasp_excess_functionality          0.538     0.796     0.642     1.000
  owasp_privilege_escalation          0.811     0.010     0.020     1.000
  unrequested_high_impact             0.633     0.148     0.240     1.000
  sensitive_access                    0.763     0.016     0.030     1.000
  owasp_asi_combined                  0.511     0.910     0.654     1.000
  oracle_privilege_diff               1.000     1.000     1.000     1.000
  llm_judge_needed(llama-3.3-70b)     0.594     0.839     0.695     0.996

[PRE] pre_over_privilege :: F1 by source
  method                                crewai          n8n          mcp   injecagent     sweagent    synthetic      overall
  flag_all                               0.388        0.154        0.654        0.750        0.574        0.763        0.601
  flag_none                              0.000        0.000        0.000        0.000        0.000        0.000        0.000
  flag_risky_perms                       0.326        0.095        0.575        0.827        0.025        0.803        0.480
  owasp_excess_permissions               0.327        0.052        0.566        0.801        0.007        0.825        0.505
  owasp_excess_functionality             0.451        0.514        0.632        0.957        0.574        0.539        0.642
  owasp_privilege_escalation             0.000        0.000        0.018        0.065        0.000        0.000        0.020
  unrequested_high_impact                0.066        0.041        0.211        0.605        0.000        0.248        0.240
  sensitive_access                       0.014        0.000        0.058        0.000        0.000        0.000        0.030
  owasp_asi_combined                     0.448        0.411        0.644        0.961        0.570        0.842        0.654
  oracle_privilege_diff                  1.000        1.000        1.000        1.000        1.000        1.000        1.000
  llm_judge_needed(llama-3.3-70b)        0.518        0.362        0.744        0.990        0.467        0.972        0.695
  label source per column: crewai n=298 (llm_judge), n8n n=219 (llm_judge), mcp n=144 (llm_judge), injecagent n=340 (roster_relabel), sweagent n=130 (declared_minus_used), synthetic n=56 (synthetic_inject)
  pooled F1 mixes these label sources; read per source, not just overall.
  abstained (scored on fewer configs, not comparable cell to cell): llm_judge_needed(llama-3.3-70b): n8n 215/219, mcp 143/144

Reading: flag_all is the floor a method must beat to earn its false alarms, and the per-source columns are the result. The pooled row mixes four label processes, so read it last. Run 'catchbench' (or 'python run.py' from a checkout) for the POST, LIVE, and Gold boards; those need the GRADE checkout bridge and download their corpora on first use.