Suites / limits

Governor limits & pushback

Governor limits in patterns, and the judgement to push back. Most tasks are briefs in which a stakeholder explicitly asks for something that breaks at scale (a query, DML statement, callout, email, platform event publish or async job per record, an unfiltered query, a quadratic loop). The answer is deployed check-only to a clean scratch org at API 67.0 and must pass hidden Apex tests that use 200 records or more and assert both the functional outcome and `Limits` usage, so following the brief literally fails; the reply must also tell the user why it deviated (checked with per-task patterns on the prose and code comments). Control tasks make requests that look similar but are fine and must simply be implemented. Analysis tasks ask how many queries or DML statements code consumes (helper methods, trigger chunks of 200, cascading triggers), which limit breaks first, and how synchronous and asynchronous limits differ.

19
tasks
Execution
grading
100%
highest c@3 · Claude Fable 5.1, joint first with 12 others
78
configurations with results

How it's graded

Each task asks for something that invites an anti-pattern, such as SOQL inside a loop. The code is deployed as a check-only validation with hidden bulk tests that assert on governor limits, and the reply must also flag the problem instead of quietly doing what was asked.

Difficulty mix

6 easy7 medium6 hard

Leaderboard

Who's best at Governor limits & pushback

Within three attempts (c@3) on this suite only, with 95% confidence intervals. With 19 tasks the intervals are wide: where two overlap, those configurations are not clearly different. 1 finished configuration isn’t shown yet: its retries haven’t finished. Open in the full leaderboard →

API: the best of each lab at Governor limits & pushback
Within three attempts (c@3) on Governor limits & pushback, on the API board (models reached through a hosted service). Bars whose intervals overlap are not clearly different.
  • 95% interval
=1Gemini 4 Argon low tie
100%, 95% interval 83–100%
=1GPT-6 Astra medium tie
100%, 95% interval 83–100%
=1Claude Opus 5.5 low tie
100%, 95% interval 83–100%
=1Muse Wonder low tie
100%, 95% interval 83–100%
=5Grok 4.7 high tie
95%, 95% interval 75–99%
=5MiMo V2.6 Pro thinking on tie
95%, 95% interval 75–99%
7DeepSeek V4.1 Flash high
89%, 95% interval 69–97%
8Inkling Small high
84%, 95% interval 62–94%
=9Nemotron 3 Ultra thinking on tie
79%, 95% interval 57–91%
=9GLM-5.3 Flash high tie
79%, 95% interval 57–91%
=11Ling 3.0 Flash thinking on tie
74%, 95% interval 51–88%
=11Mistral Large 4 high tie
74%, 95% interval 51–88%
13Qwen3.8 Flash-Next thinking on
58%, 95% interval 36–77%
14Laguna S 2.1 thinking on
53%, 95% interval 32–73%
    Show the numbers
    ModelScore95% CI
    Gemini 4 Argon low100%83–100
    GPT-6 Astra medium100%83–100
    Claude Opus 5.5 low100%83–100
    Muse Wonder low100%83–100
    Grok 4.7 high95%75–99
    MiMo V2.6 Pro thinking on95%75–99
    DeepSeek V4.1 Flash high89%69–97
    Inkling Small high84%62–94
    Nemotron 3 Ultra thinking on79%57–91
    GLM-5.3 Flash high79%57–91
    Ling 3.0 Flash thinking on74%51–88
    Mistral Large 4 high74%51–88
    Qwen3.8 Flash-Next thinking on58%36–77
    Laguna S 2.1 thinking on53%32–73
    Local: the best of each lab at Governor limits & pushback
    Within three attempts (c@3) on Governor limits & pushback, on the Local board (models served on Forcebench’s own hardware). Bars whose intervals overlap are not clearly different.
    • 95% interval
    =1Gemma 4 31B thinking on · QAT W4A16 tie
    89%, 95% interval 69–97%
    =1GLM-5.3 Flash high · EXL3 4.0bpw experts, FP8 dense tie
    89%, 95% interval 69–97%
    =1DeepSeek V4.1 Flash high · EXL3 2.9bpw tie
    89%, 95% interval 69–97%
    4Muse Glimmer 30B high · NVFP4 + FP8 mixed (NVIDIA ModelOpt)
    84%, 95% interval 62–94%
    =5Nemotron 3.5 Super VL thinking on · NVFP4 mixed precision (NVIDIA ModelOpt) tie
    79%, 95% interval 57–91%
    =5Qwen3.8 Flash-Next medium · MLX 4-bit tie
    79%, 95% interval 57–91%
    7MiMo V2.6 Flash thinking on · FP8 + MXFP4 experts
    74%, 95% interval 51–88%
    8MiniMax M3.1 Flash high · NVFP4 experts, MXFP8 (QAT)
    68%, 95% interval 46–85%
    =9MiniCPM-V 4.7 35B-A3B thinking on · INT4 W4A16 experts, BF16 dense tie
    58%, 95% interval 36–77%
    =9Ornith 1.5 122B thinking on · NVFP4 experts, FP8 attention (NVIDIA ModelOpt) tie
    58%, 95% interval 36–77%
      Show the numbers
      ModelScore95% CI
      Gemma 4 31B thinking on · QAT W4A1689%69–97
      GLM-5.3 Flash high · EXL3 4.0bpw experts, FP8 dense89%69–97
      DeepSeek V4.1 Flash high · EXL3 2.9bpw89%69–97
      Muse Glimmer 30B high · NVFP4 + FP8 mixed (NVIDIA ModelOpt)84%62–94
      Nemotron 3.5 Super VL thinking on · NVFP4 mixed precision (NVIDIA ModelOpt)79%57–91
      Qwen3.8 Flash-Next medium · MLX 4-bit79%57–91
      MiMo V2.6 Flash thinking on · FP8 + MXFP4 experts74%51–88
      MiniMax M3.1 Flash high · NVFP4 experts, MXFP8 (QAT)68%46–85
      MiniCPM-V 4.7 35B-A3B thinking on · INT4 W4A16 experts, BF16 dense58%36–77
      Ornith 1.5 122B thinking on · NVFP4 experts, FP8 attention (NVIDIA ModelOpt)58%36–77
      Governor limits & pushback: c@3 by configuration with 95% confidence intervals
      #Model and configScore c@3 · 95% CITasks
      1Claude Fable 5.1 effort high
      100%95% CI 83–100
      		19
      1Claude Opus 5.5 effort high
      100%95% CI 83–100
      		19
      1Claude Opus 5.5 effort medium
      100%95% CI 83–100
      		19
      1Claude Opus 5.5 effort low
      100%95% CI 83–100
      		19
      1Gemini 4 Argon effort medium
      100%95% CI 83–100
      		19
      1Gemini 4 Argon effort low
      100%95% CI 83–100
      		19
      1Gemini 4 Argon effort high
      100%95% CI 83–100
      		19
      1GPT-6 Astra effort medium
      100%95% CI 83–100
      		19
      1GPT-6 Astra effort high
      100%95% CI 83–100
      		19
      1GPT-6.1 Sol effort high
      100%95% CI 83–100
      		19
      1Muse Wonder effort medium
      100%95% CI 83–100
      		19
      1Muse Wonder effort max
      100%95% CI 83–100
      		19
      1Muse Wonder effort low
      100%95% CI 83–100
      		19
      14Claude Haiku 5.5 effort medium
      95%95% CI 75–99
      		19
      14Claude Haiku 5.5 effort high
      95%95% CI 75–99
      		19
      14Claude Opus 4.8 effort high
      95%95% CI 75–99
      		19
      14Claude Opus 4.8 effort medium
      95%95% CI 75–99
      		19
      14Claude Opus 4.8 effort low
      95%95% CI 75–99
      		19
      14Claude Sonnet 5.5 effort high
      95%95% CI 75–99
      		19
      14Claude Sonnet 5.5 effort low
      95%95% CI 75–99
      		19
      14Claude Sonnet 5.5 effort medium
      95%95% CI 75–99
      		19
      14Gemini 3.8 Flash effort medium
      95%95% CI 75–99
      		19
      14GPT-5.6 Sol effort medium
      95%95% CI 75–99
      		19
      14Grok 4.7 effort high
      95%95% CI 75–99
      		19
      14MiMo V2.6 Pro FP8 · thinking on
      95%95% CI 75–99
      		19
      14Muse Wonder effort high
      95%95% CI 75–99
      		19
      27Claude Haiku 5.5 effort low
      89%95% CI 69–97
      		19
      27Claude Sonnet 5 effort medium
      89%95% CI 69–97
      		19
      27DeepSeek V4 Pro FP8 · effort high
      89%95% CI 69–97
      		19
      27DeepSeek V4.1 Flash FP8 · effort high
      89%95% CI 69–97
      		19
      27DeepSeek V4.1 Flash EXL3 2.9bpw · vLLM + ExLlamaV3 · effort high
      89%95% CI 69–97
      		19
      27Gemma 4 26B-A4B NVFP4 · vLLM · thinking on
      89%95% CI 69–97
      		19
      27Gemma 4 31B QAT W4A16 · vLLM · thinking on
      89%95% CI 69–97
      		19
      27GLM-5.3 Flash EXL3 4.0bpw experts, FP8 dense · vLLM + ExLlamaV3 · effort high
      89%95% CI 69–97
      		19
      27Grok 4.6 effort high
      89%95% CI 69–97
      		19
      36Claude Sonnet 5 thinking off
      84%95% CI 62–94
      		19
      36DeepSeek V4 Flash Vision experimental FP8 + FP4 experts · vLLM · effort high
      84%95% CI 62–94
      		19
      36DeepSeek V4.1 Flash Native FP8, FP4 experts · SGLang · effort high
      84%95% CI 62–94
      		19
      36DeepSeek V4.1 Flash 1017 Native FP8, FP4 experts · SGLang · effort high
      84%95% CI 62–94
      		19
      36GLM-5.3 Flash EXL3 4.0bpw experts, 4-bit dense · TensorFold · effort high
      84%95% CI 62–94
      		19
      36GLM-5.3 Flash EXL3 4.0bpw experts, FP8 dense · vLLM + ExLlamaV3 · effort low
      84%95% CI 62–94
      		19
      36GPT-6 Luna effort medium
      84%95% CI 62–94
      		19
      36Inkling Small FP8 · effort high
      84%95% CI 62–94
      		19
      36Muse Glimmer 30B NVFP4 + FP8 mixed (NVIDIA ModelOpt) · vLLM · effort high
      84%95% CI 62–94
      		19
      45Claude Haiku 4.5 thinking on
      79%95% CI 57–91
      		19
      45Claude Opus 4.8 thinking off
      79%95% CI 57–91
      		19
      45Claude Sonnet 5 effort high
      79%95% CI 57–91
      		19
      45GLM-5.3 Flash FP8 · effort high
      79%95% CI 57–91
      		19
      45Nemotron 3 Ultra thinking on
      79%95% CI 57–91
      		19
      45Nemotron 3.5 Super VL NVFP4 mixed precision (NVIDIA ModelOpt) · TensorFold (Zig) · thinking on
      79%95% CI 57–91
      		19
      45Qwen3.8 Flash-Next MLX 4-bit · TensorFold · effort medium
      79%95% CI 57–91
      		19
      52Claude Sonnet 5 effort low
      74%95% CI 51–88
      		19
      52GLM-5.3 FP8 · effort low
      74%95% CI 51–88
      		19
      52Ling 3.0 Flash BF16 · thinking on
      74%95% CI 51–88
      		19
      52MiMo V2.6 Flash FP8 + MXFP4 experts · SGLang · thinking on
      74%95% CI 51–88
      		19
      52Mistral Large 4 effort high
      74%95% CI 51–88
      		19
      57MiMo V2.6 Flash FP8 · thinking on
      68%95% CI 46–85
      		19
      57MiniMax M3.1 Flash NVFP4 experts, MXFP8 (QAT) · vLLM · effort high
      68%95% CI 46–85
      		19
      57Qwen3.6 35B-A3B NVFP4 · vLLM · thinking on
      68%95% CI 46–85
      		19
      57Qwen3.8 27B AWQ-INT4 · vLLM · effort xhigh
      68%95% CI 46–85
      		19
      57Qwen3.8 Flash-Next NVFP4 · vLLM · effort medium
      68%95% CI 46–85
      		19
      57Qwen3.8 Flash-Next MLX 4-bit · MTPLX · effort medium
      68%95% CI 46–85
      		19
      63Claude Haiku 4.5 thinking off
      63%95% CI 41–81
      		19
      63GLM-5.3 Flash EXL3 4.0bpw experts, FP8 dense · vLLM + ExLlamaV3 · thinking off
      63%95% CI 41–81
      		19
      63Qwen3.8 27B MLX 8-bit · MTPLX · effort low
      63%95% CI 41–81
      		19
      63Qwen3.8 27B MLX 4-bit · MTPLX · effort low
      63%95% CI 41–81
      		19
      63Qwen3.8 27B AWQ-INT4 · vLLM · effort low
      63%95% CI 41–81
      		19
      63Qwen3.8 Flash-Next INT4-AutoRound, 5 of 10 experts · TensorFold (Zig) · effort medium
      63%95% CI 41–81
      		19
      69Qwen3.8 27B AWQ-INT4 · vLLM · effort medium
      60%95% CI 38–78
      		19
      70MiniCPM-V 4.7 35B-A3B INT4 W4A16 experts, BF16 dense · vLLM · thinking on
      58%95% CI 36–77
      		19
      70Nemotron 3.5 Bolt IQ2_XXS + NVFP4 + Q8_0 mixed (NVIDIA ModelOpt) · TensorFold (Zig) · thinking on
      58%95% CI 36–77
      		19
      70Ornith 1.5 122B NVFP4 experts, FP8 attention (NVIDIA ModelOpt) · vLLM · thinking on
      58%95% CI 36–77
      		19
      70Qwen3.8 27B Splash 4-bit · Splash · effort low
      58%95% CI 36–77
      		19
      70Qwen3.8 Flash-Next INT4-AutoRound, 5 of 10 experts · TensorFold (Zig) · effort medium
      58%95% CI 36–77
      		19
      70Qwen3.8 Flash-Next thinking on
      58%95% CI 36–77
      		19
      76Laguna S 2.1 FP4 · thinking on
      53%95% CI 32–73
      		19
      76MiniCPM-V 4.7 35B-A3B BF16 · vLLM · thinking on
      53%95% CI 32–73
      		19
      78Nemotron 3.5 Lightning NVFP4 · thinking on
      47%95% CI 27–68
      		19

      Tasks

      What's in the suite

      Solve rate is the share of finished runs whose first answer to the task passed (pass@1), a rough guide to how hard models find it. Difficulty is the task author’s label; from results is what the solve rate says: easy when two thirds or more of the finished runs pass it, hard at a third or fewer, once five have graded it.

      Governor limits & pushback tasks with difficulty and solve rate
      TaskDifficultyFrom resultsSolve rate
      Account rating from open pipeline, asked for a future call per opportunitylimits-pushback-future-per-opportunityeasyeasy
      70%
      Case priority from the account, asked for a query in the looplimits-pushback-case-priority-query-in-loopeasyeasy
      89%
      Clean up contact phones with one partial-success update (control, comply)limits-control-contact-phone-partial-updateeasyeasy
      89%
      Close stale deals with a SOQL for loop (control, comply)limits-control-soql-for-loop-stale-dealseasyeasy
      81%
      Count the SOQL queries of a trigger with a querying helperlimits-count-queries-helper-per-leadeasyeasy
      97%
      Which per-transaction limits are higher in a Queueablelimits-async-vs-sync-limitseasyeasy
      67%
      Contact country sync, asked to load every contact into a maplimits-pushback-unfiltered-contact-querymediumeasy
      79%
      Follow-up tasks with partial success, asked to insert each one in a try/catchlimits-pushback-followup-tasks-dml-per-recordmediummedium
      64%
      Pipeline totals with one query per fixed stage (control, comply)limits-control-pipeline-query-per-stagemediumeasy
      72%
      Renewal reminders, asked for a Messaging.sendEmail call per opportunitylimits-pushback-send-email-per-opportunitymediummedium
      37%
      Survey tasks for closed cases, asked to enqueue a Queueable per caselimits-pushback-queueable-per-closed-casemediumhard
      33%
      Trigger chunks of 200 in one Apex insert versus Data Loader batcheslimits-trigger-chunks-apex-vs-data-loadermediummedium
      61%
      Which governor limit breaks first in a looplimits-first-limit-exceededmediumeasy
      99%
      Count SOQL queries across cascading and re-entrant triggerslimits-count-cascading-trigger-querieshardeasy
      93%
      Default line discounts, asked to call a querying helper per line itemlimits-pushback-helper-query-per-line-itemhardmedium
      55%
      Match imported leads to contacts, asked for a nested looplimits-pushback-nested-loop-lead-matchinghardmedium
      53%
      Push new accounts to the ERP, asked for a callout per record from the triggerlimits-pushback-erp-callout-per-accounthardhard
      27%
      Stage-change events, asked to call EventBus.publish per opportunitylimits-pushback-eventbus-publish-per-recordhardmedium
      40%
      Which trigger snippets fail on a 200-record insertlimits-spot-risky-trigger-snippetshardhard
      30%

      Private evaluations

      Want this for your team?

      Azul Labs runs Forcebench privately for two kinds of Salesforce team.

      Choosing a model to run yourself?

      A private benchmark of your shortlist

      Open-weight models at the quantisations and reasoning efforts you could run on infrastructure you control, tested on your kind of Salesforce work, with a written recommendation.

      • Private tasks written from your work only ever run on local models; they are never sent to an API.
      • Re-testing after any change of model, quantisation or engine is available.
      • Typical turnaround is 2–3 weeks; large models take about a day of machine time per configuration.
      Get a private benchmarkHow it works

      Building or buying a Salesforce AI tool?

      A private evaluation of your tool

      Your tool on held-out tasks it has never seen, against API baselines (models reached through a hosted service) and Local ones (served on Forcebench’s own hardware), with a written report on where it holds up and where it fails.

      • It can start as a smaller pilot, about 30 held-out tasks with your tool against two baselines, before a full evaluation.
      • Repeat evaluations after your releases are available.
      • Typical turnaround is 2–3 weeks.
      Ask about an evaluationHow it works

      Fixed fee, quoted up front. The quote depends on the number of tasks and the number of models, tools or configurations tested.

      Or email leo@azl.au · azl.au