Suites / fflib

fflib Enterprise Patterns

Apex Enterprise Patterns with the open-source fflib libraries (fflib-apex-common and fflib-apex-mocks): Selector, Domain, Service and Unit of Work layers, the Application factory class, and unit tests with ApexMocks. Code is deployed to a scratch org with the pinned fflib commits installed and must pass hidden Apex tests; test-writing tasks are graded by mutation testing (the model's tests must pass against the correct code and fail against seeded bugs).

18
tasks
Execution
grading
100%
highest c@3 · Claude Fable 5.1, joint first with 7 others
78
configurations with results

How it's graded

Code is deployed as a check-only validation to a scratch org with the pinned fflib libraries installed and must pass hidden Apex tests. ApexMocks test-writing tasks are graded by mutation testing: the model’s tests must pass against the correct code and fail against seeded bugs.

Difficulty mix

5 easy7 medium6 hard

Leaderboard

Who's best at fflib Enterprise Patterns

Within three attempts (c@3) on this suite only, with 95% confidence intervals. With 18 tasks the intervals are wide: where two overlap, those configurations are not clearly different. 1 finished configuration isn’t shown yet: its retries haven’t finished. Open in the full leaderboard →

API: the best of each lab at fflib Enterprise Patterns
Within three attempts (c@3) on fflib Enterprise Patterns, on the API board (models reached through a hosted service). Bars whose intervals overlap are not clearly different.
  • 95% interval
=1Gemini 4 Argon medium tie
100%, 95% interval 82–100%
=1Claude Opus 5.5 low tie
100%, 95% interval 82–100%
=1GPT-6.1 Sol high tie
100%, 95% interval 82–100%
=4Grok 4.6 high tie
94%, 95% interval 74–99%
=4Muse Wonder max tie
94%, 95% interval 74–99%
6DeepSeek V4.1 Flash high
67%, 95% interval 44–84%
7Mistral Large 4 high
50%, 95% interval 29–71%
=8GLM-5.3 low tie
39%, 95% interval 20–61%
=8MiMo V2.6 Pro thinking on tie
39%, 95% interval 20–61%
10Nemotron 3 Ultra thinking on
33%, 95% interval 16–56%
11Ling 3.0 Flash thinking on
22%, 95% interval 9–45%
=12Laguna S 2.1 thinking on tie
17%, 95% interval 6–39%
=12Inkling Small high tie
17%, 95% interval 6–39%
=12Qwen3.8 Flash-Next thinking on tie
17%, 95% interval 6–39%
    Show the numbers
    ModelScore95% CI
    Gemini 4 Argon medium100%82–100
    Claude Opus 5.5 low100%82–100
    GPT-6.1 Sol high100%82–100
    Grok 4.6 high94%74–99
    Muse Wonder max94%74–99
    DeepSeek V4.1 Flash high67%44–84
    Mistral Large 4 high50%29–71
    GLM-5.3 low39%20–61
    MiMo V2.6 Pro thinking on39%20–61
    Nemotron 3 Ultra thinking on33%16–56
    Ling 3.0 Flash thinking on22%9–45
    Laguna S 2.1 thinking on17%6–39
    Inkling Small high17%6–39
    Qwen3.8 Flash-Next thinking on17%6–39
    Local: the best of each lab at fflib Enterprise Patterns
    Within three attempts (c@3) on fflib Enterprise Patterns, on the Local board (models served on Forcebench’s own hardware). Bars whose intervals overlap are not clearly different.
    • 95% interval
    1DeepSeek V4.1 Flash 1017 high · Native FP8, FP4 experts
    56%, 95% interval 34–75%
    2MiMo V2.6 Flash thinking on · FP8 + MXFP4 experts
    44%, 95% interval 25–66%
    3GLM-5.3 Flash low · EXL3 4.0bpw experts, FP8 dense
    33%, 95% interval 16–56%
    4Qwen3.8 Flash-Next medium · MLX 4-bit
    22%, 95% interval 9–45%
    =5Gemma 4 31B thinking on · QAT W4A16 tie
    17%, 95% interval 6–39%
    =5Muse Glimmer 30B high · NVFP4 + FP8 mixed (NVIDIA ModelOpt) tie
    17%, 95% interval 6–39%
    =5Nemotron 3.5 Super VL thinking on · NVFP4 mixed precision (NVIDIA ModelOpt) tie
    17%, 95% interval 6–39%
    8Ornith 1.5 122B thinking on · NVFP4 experts, FP8 attention (NVIDIA ModelOpt)
    11%, 95% interval 3–33%
    9MiniCPM-V 4.7 35B-A3B thinking on · BF16
    6%, 95% interval 1–26%
    10MiniMax M3.1 Flash high · NVFP4 experts, MXFP8 (QAT)
    0%, 95% interval 0–18%
      Show the numbers
      ModelScore95% CI
      DeepSeek V4.1 Flash 1017 high · Native FP8, FP4 experts56%34–75
      MiMo V2.6 Flash thinking on · FP8 + MXFP4 experts44%25–66
      GLM-5.3 Flash low · EXL3 4.0bpw experts, FP8 dense33%16–56
      Qwen3.8 Flash-Next medium · MLX 4-bit22%9–45
      Gemma 4 31B thinking on · QAT W4A1617%6–39
      Muse Glimmer 30B high · NVFP4 + FP8 mixed (NVIDIA ModelOpt)17%6–39
      Nemotron 3.5 Super VL thinking on · NVFP4 mixed precision (NVIDIA ModelOpt)17%6–39
      Ornith 1.5 122B thinking on · NVFP4 experts, FP8 attention (NVIDIA ModelOpt)11%3–33
      MiniCPM-V 4.7 35B-A3B thinking on · BF166%1–26
      MiniMax M3.1 Flash high · NVFP4 experts, MXFP8 (QAT)0%0–18
      fflib Enterprise Patterns: c@3 by configuration with 95% confidence intervals
      #Model and configScore c@3 · 95% CITasks
      1Claude Fable 5.1 effort high
      100%95% CI 82–100
      		18
      1Claude Opus 5.5 effort high
      100%95% CI 82–100
      		18
      1Claude Opus 5.5 effort medium
      100%95% CI 82–100
      		18
      1Claude Opus 5.5 effort low
      100%95% CI 82–100
      		18
      1Gemini 3.8 Flash effort medium
      100%95% CI 82–100
      		18
      1Gemini 4 Argon effort medium
      100%95% CI 82–100
      		18
      1GPT-6 Astra effort high
      100%95% CI 82–100
      		18
      1GPT-6.1 Sol effort high
      100%95% CI 82–100
      		18
      9Claude Opus 4.8 effort high
      94%95% CI 74–99
      		18
      9Claude Opus 4.8 effort medium
      94%95% CI 74–99
      		18
      9Claude Opus 4.8 effort low
      94%95% CI 74–99
      		18
      9Claude Sonnet 5.5 effort low
      94%95% CI 74–99
      		18
      9Claude Sonnet 5.5 effort medium
      94%95% CI 74–99
      		18
      9Gemini 4 Argon effort low
      94%95% CI 74–99
      		18
      9Gemini 4 Argon effort high
      94%95% CI 74–99
      		18
      9GPT-6 Astra effort medium
      94%95% CI 74–99
      		18
      9Grok 4.6 effort high
      94%95% CI 74–99
      		18
      9Grok 4.7 effort high
      94%95% CI 74–99
      		18
      9Muse Wonder effort max
      94%95% CI 74–99
      		18
      20Claude Opus 4.8 thinking off
      89%95% CI 67–97
      		18
      20Claude Sonnet 5.5 effort high
      89%95% CI 67–97
      		18
      20Muse Wonder effort low
      89%95% CI 67–97
      		18
      20Muse Wonder effort high
      89%95% CI 67–97
      		18
      24Muse Wonder effort medium
      83%95% CI 61–94
      		18
      25Claude Sonnet 5 effort high
      67%95% CI 44–84
      		18
      25Claude Sonnet 5 effort medium
      67%95% CI 44–84
      		18
      25Claude Sonnet 5 effort low
      67%95% CI 44–84
      		18
      25DeepSeek V4.1 Flash FP8 · effort high
      67%95% CI 44–84
      		18
      29Claude Haiku 5.5 effort medium
      61%95% CI 39–80
      		18
      29Claude Sonnet 5 thinking off
      61%95% CI 39–80
      		18
      29GPT-5.6 Sol effort medium
      61%95% CI 39–80
      		18
      29GPT-6 Luna effort medium
      61%95% CI 39–80
      		18
      33Claude Haiku 5.5 effort high
      56%95% CI 34–75
      		18
      33Claude Haiku 5.5 effort low
      56%95% CI 34–75
      		18
      33DeepSeek V4 Pro FP8 · effort high
      56%95% CI 34–75
      		18
      33DeepSeek V4.1 Flash 1017 Native FP8, FP4 experts · SGLang · effort high
      56%95% CI 34–75
      		18
      37DeepSeek V4.1 Flash EXL3 2.9bpw · vLLM + ExLlamaV3 · effort high
      50%95% CI 29–71
      		18
      37DeepSeek V4.1 Flash Native FP8, FP4 experts · SGLang · effort high
      50%95% CI 29–71
      		18
      37Mistral Large 4 effort high
      50%95% CI 29–71
      		18
      40Claude Haiku 4.5 thinking on
      44%95% CI 25–66
      		18
      40Claude Haiku 4.5 thinking off
      44%95% CI 25–66
      		18
      40MiMo V2.6 Flash FP8 + MXFP4 experts · SGLang · thinking on
      44%95% CI 25–66
      		18
      43GLM-5.3 FP8 · effort low
      39%95% CI 20–61
      		18
      43GLM-5.3 Flash FP8 · effort high
      39%95% CI 20–61
      		18
      43MiMo V2.6 Pro FP8 · thinking on
      39%95% CI 20–61
      		18
      46GLM-5.3 Flash EXL3 4.0bpw experts, FP8 dense · vLLM + ExLlamaV3 · effort high
      33%95% CI 16–56
      		18
      46GLM-5.3 Flash EXL3 4.0bpw experts, FP8 dense · vLLM + ExLlamaV3 · effort low
      33%95% CI 16–56
      		18
      46Nemotron 3 Ultra thinking on
      33%95% CI 16–56
      		18
      49GLM-5.3 Flash EXL3 4.0bpw experts, 4-bit dense · TensorFold · effort high
      28%95% CI 13–51
      		18
      50GLM-5.3 Flash EXL3 4.0bpw experts, FP8 dense · vLLM + ExLlamaV3 · thinking off
      22%95% CI 9–45
      		18
      50Ling 3.0 Flash BF16 · thinking on
      22%95% CI 9–45
      		18
      50MiMo V2.6 Flash FP8 · thinking on
      22%95% CI 9–45
      		18
      50Qwen3.8 Flash-Next MLX 4-bit · TensorFold · effort medium
      22%95% CI 9–45
      		18
      54DeepSeek V4 Flash Vision experimental FP8 + FP4 experts · vLLM · effort high
      17%95% CI 6–39
      		18
      54Gemma 4 31B QAT W4A16 · vLLM · thinking on
      17%95% CI 6–39
      		18
      54Inkling Small FP8 · effort high
      17%95% CI 6–39
      		18
      54Laguna S 2.1 FP4 · thinking on
      17%95% CI 6–39
      		18
      54Muse Glimmer 30B NVFP4 + FP8 mixed (NVIDIA ModelOpt) · vLLM · effort high
      17%95% CI 6–39
      		18
      54Nemotron 3.5 Super VL NVFP4 mixed precision (NVIDIA ModelOpt) · TensorFold (Zig) · thinking on
      17%95% CI 6–39
      		18
      54Qwen3.6 35B-A3B NVFP4 · vLLM · thinking on
      17%95% CI 6–39
      		18
      54Qwen3.8 Flash-Next NVFP4 · vLLM · effort medium
      17%95% CI 6–39
      		18
      54Qwen3.8 Flash-Next MLX 4-bit · MTPLX · effort medium
      17%95% CI 6–39
      		18
      54Qwen3.8 Flash-Next INT4-AutoRound, 5 of 10 experts · TensorFold (Zig) · effort medium
      17%95% CI 6–39
      		18
      54Qwen3.8 Flash-Next INT4-AutoRound, 5 of 10 experts · TensorFold (Zig) · effort medium
      17%95% CI 6–39
      		18
      54Qwen3.8 Flash-Next thinking on
      17%95% CI 6–39
      		18
      66Ornith 1.5 122B NVFP4 experts, FP8 attention (NVIDIA ModelOpt) · vLLM · thinking on
      11%95% CI 3–33
      		18
      66Qwen3.8 27B MLX 4-bit · MTPLX · effort low
      11%95% CI 3–33
      		18
      66Qwen3.8 27B AWQ-INT4 · vLLM · effort medium
      11%95% CI 3–33
      		18
      69Gemma 4 26B-A4B NVFP4 · vLLM · thinking on
      6%95% CI 1–26
      		18
      69MiniCPM-V 4.7 35B-A3B BF16 · vLLM · thinking on
      6%95% CI 1–26
      		18
      69MiniCPM-V 4.7 35B-A3B INT4 W4A16 experts, BF16 dense · vLLM · thinking on
      6%95% CI 1–26
      		18
      69Nemotron 3.5 Lightning NVFP4 · thinking on
      6%95% CI 1–26
      		18
      69Qwen3.8 27B Splash 4-bit · Splash · effort low
      6%95% CI 1–26
      		18
      74MiniMax M3.1 Flash NVFP4 experts, MXFP8 (QAT) · vLLM · effort high
      0%95% CI 0–18
      		18
      74Nemotron 3.5 Bolt IQ2_XXS + NVFP4 + Q8_0 mixed (NVIDIA ModelOpt) · TensorFold (Zig) · thinking on
      0%95% CI 0–18
      		18
      74Qwen3.8 27B MLX 8-bit · MTPLX · effort low
      0%95% CI 0–18
      		18
      74Qwen3.8 27B AWQ-INT4 · vLLM · effort low
      0%95% CI 0–18
      		18
      74Qwen3.8 27B AWQ-INT4 · vLLM · effort xhigh
      0%95% CI 0–18
      		18

      Tasks

      What's in the suite

      Solve rate is the share of finished runs whose first answer to the task passed (pass@1), a rough guide to how hard models find it. Difficulty is the task author’s label; from results is what the solve rate says: easy when two thirds or more of the finished runs pass it, hard at a third or fewer, once five have graded it.

      fflib Enterprise Patterns tasks with difficulty and solve rate
      TaskDifficultyFrom resultsSolve rate
      ApexMocks unit test with a stubbed selectorfflib-apexmocks-stub-selectoreasyhard
      29%
      Contact selector with fflib_SObjectSelectorfflib-selector-contactseasymedium
      42%
      Create accounts and contacts in one unit of workfflib-uow-account-contactseasymedium
      44%
      Opportunity domain class with insert defaultsfflib-domain-opportunity-defaultseasymedium
      56%
      Static service facade over the Application service factoryfflib-service-facadeeasyeasy
      76%
      ApexMocks test verifying unit of work interactionsfflib-apexmocks-verify-uowmediumhard
      29%
      Application class with the four fflib factoriesfflib-application-factoriesmediummedium
      49%
      Custom selector query with its own ordering and limitfflib-selector-query-factorymediumhard
      30%
      Domain trigger logic delegating to a service once per triggerfflib-domain-calls-servicemediummedium
      49%
      Domain unit test with fflib's mock trigger databasefflib-domain-unit-test-mock-dbmediumhard
      30%
      Insert and update validation in an fflib domain classfflib-domain-validationmediumhard
      5%
      Mockable service using the Application unit of work and selectorfflib-service-case-escalationmediummedium
      66%
      ApexMocks test for a service's dependency failuresfflib-apexmocks-exception-pathshardhard
      8%
      ApexMocks test for records created and related through the unit of workfflib-apexmocks-argument-captorhardhard
      15%
      ApexMocks test that mocks a domain and verifies call orderfflib-apexmocks-mock-domainhardhard
      0%
      Mockable domain on the fflib_SObjects structurefflib-domain-sobjects-structurehardhard
      14%
      Publish platform events only after a successful unit of work commitfflib-uow-platform-eventshardhard
      24%
      User-mode selector with a field set and a child subselectfflib-selector-user-mode-fieldsethardhard
      23%

      Private evaluations

      Want this for your team?

      Azul Labs runs Forcebench privately for two kinds of Salesforce team.

      Choosing a model to run yourself?

      A private benchmark of your shortlist

      Open-weight models at the quantisations and reasoning efforts you could run on infrastructure you control, tested on your kind of Salesforce work, with a written recommendation.

      • Private tasks written from your work only ever run on local models; they are never sent to an API.
      • Re-testing after any change of model, quantisation or engine is available.
      • Typical turnaround is 2–3 weeks; large models take about a day of machine time per configuration.
      Get a private benchmarkHow it works

      Building or buying a Salesforce AI tool?

      A private evaluation of your tool

      Your tool on held-out tasks it has never seen, against API baselines (models reached through a hosted service) and Local ones (served on Forcebench’s own hardware), with a written report on where it holds up and where it fails.

      • It can start as a smaller pilot, about 30 held-out tasks with your tool against two baselines, before a full evaluation.
      • Repeat evaluations after your releases are available.
      • Typical turnaround is 2–3 weeks.
      Ask about an evaluationHow it works

      Fixed fee, quoted up front. The quote depends on the number of tasks and the number of models, tools or configurations tested.

      Or email leo@azl.au · azl.au