Suites / api

Salesforce APIs

Integration work against Salesforce's APIs: REST sObject, query, composite, composite graph and sObject Collections resources, Bulk API 2.0 ingest and query jobs, Tooling API, User Interface API, platform events, custom Apex REST endpoints and OAuth 2.0 token requests. Answers are raw HTTP requests, checked deterministically for method, path, query, headers and body (JSON, form-encoded or CSV), plus a few questions on API behaviour and choice.

18
tasks
Deterministic validators
grading
100%
highest c@3 · GPT-6 Astra
78
configurations with results

How it's graded

Answers are raw HTTP requests, checked deterministically for method, path, query, headers and body (JSON, form-encoded or CSV). A few questions cover API behaviour and choosing the right API.

Difficulty mix

6 easy7 medium5 hard

Leaderboard

Who's best at Salesforce APIs

Within three attempts (c@3) on this suite only, with 95% confidence intervals. With 18 tasks the intervals are wide: where two overlap, those configurations are not clearly different. 1 finished configuration isn’t shown yet: its retries haven’t finished. Open in the full leaderboard →

API: the best of each lab at Salesforce APIs
Within three attempts (c@3) on Salesforce APIs, on the API board (models reached through a hosted service). Bars whose intervals overlap are not clearly different.
  • 95% interval
1GPT-6 Astra high
100%, 95% interval 82–100%
=2Gemini 4 Argon medium tie
94%, 95% interval 74–99%
=2Claude Opus 4.8 low tie
94%, 95% interval 74–99%
=2Muse Wonder medium tie
94%, 95% interval 74–99%
5Grok 4.7 high
89%, 95% interval 67–97%
6DeepSeek V4.1 Flash high
83%, 95% interval 61–94%
=7GLM-5.3 Flash high tie
72%, 95% interval 49–88%
=7MiMo V2.6 Pro thinking on tie
72%, 95% interval 49–88%
9Nemotron 3 Ultra thinking on
61%, 95% interval 39–80%
10Inkling Small high
56%, 95% interval 34–75%
11Mistral Large 4 high
39%, 95% interval 20–61%
=12Laguna S 2.1 thinking on tie
33%, 95% interval 16–56%
=12Ling 3.0 Flash thinking on tie
33%, 95% interval 16–56%
=12Qwen3.8 Flash-Next thinking on tie
33%, 95% interval 16–56%
    Show the numbers
    ModelScore95% CI
    GPT-6 Astra high100%82–100
    Gemini 4 Argon medium94%74–99
    Claude Opus 4.8 low94%74–99
    Muse Wonder medium94%74–99
    Grok 4.7 high89%67–97
    DeepSeek V4.1 Flash high83%61–94
    GLM-5.3 Flash high72%49–88
    MiMo V2.6 Pro thinking on72%49–88
    Nemotron 3 Ultra thinking on61%39–80
    Inkling Small high56%34–75
    Mistral Large 4 high39%20–61
    Laguna S 2.1 thinking on33%16–56
    Ling 3.0 Flash thinking on33%16–56
    Qwen3.8 Flash-Next thinking on33%16–56
    Local: the best of each lab at Salesforce APIs
    Within three attempts (c@3) on Salesforce APIs, on the Local board (models served on Forcebench’s own hardware). Bars whose intervals overlap are not clearly different.
    • 95% interval
    1DeepSeek V4.1 Flash high · Native FP8, FP4 experts
    89%, 95% interval 67–97%
    =2GLM-5.3 Flash thinking off · EXL3 4.0bpw experts, FP8 dense tie
    56%, 95% interval 34–75%
    =2Qwen3.8 Flash-Next medium · INT4-AutoRound, 5 of 10 experts tie
    56%, 95% interval 34–75%
    =4Gemma 4 31B thinking on · QAT W4A16 tie
    50%, 95% interval 29–71%
    =4Nemotron 3.5 Super VL thinking on · NVFP4 mixed precision (NVIDIA ModelOpt) tie
    50%, 95% interval 29–71%
    =6Muse Glimmer 30B high · NVFP4 + FP8 mixed (NVIDIA ModelOpt) tie
    39%, 95% interval 20–61%
    =6MiMo V2.6 Flash thinking on · FP8 + MXFP4 experts tie
    39%, 95% interval 20–61%
    =8MiniCPM-V 4.7 35B-A3B thinking on · BF16 tie
    33%, 95% interval 16–56%
    =8MiniMax M3.1 Flash high · NVFP4 experts, MXFP8 (QAT) tie
    33%, 95% interval 16–56%
    10Ornith 1.5 122B thinking on · NVFP4 experts, FP8 attention (NVIDIA ModelOpt)
    28%, 95% interval 13–51%
      Show the numbers
      ModelScore95% CI
      DeepSeek V4.1 Flash high · Native FP8, FP4 experts89%67–97
      GLM-5.3 Flash thinking off · EXL3 4.0bpw experts, FP8 dense56%34–75
      Qwen3.8 Flash-Next medium · INT4-AutoRound, 5 of 10 experts56%34–75
      Gemma 4 31B thinking on · QAT W4A1650%29–71
      Nemotron 3.5 Super VL thinking on · NVFP4 mixed precision (NVIDIA ModelOpt)50%29–71
      Muse Glimmer 30B high · NVFP4 + FP8 mixed (NVIDIA ModelOpt)39%20–61
      MiMo V2.6 Flash thinking on · FP8 + MXFP4 experts39%20–61
      MiniCPM-V 4.7 35B-A3B thinking on · BF1633%16–56
      MiniMax M3.1 Flash high · NVFP4 experts, MXFP8 (QAT)33%16–56
      Ornith 1.5 122B thinking on · NVFP4 experts, FP8 attention (NVIDIA ModelOpt)28%13–51
      Salesforce APIs: c@3 by configuration with 95% confidence intervals
      #Model and configScore c@3 · 95% CITasks
      1GPT-6 Astra effort high
      100%95% CI 82–100
      		18
      2Claude Opus 4.8 effort medium
      94%95% CI 74–99
      		18
      2Claude Opus 4.8 effort low
      94%95% CI 74–99
      		18
      2Claude Opus 5.5 effort medium
      94%95% CI 74–99
      		18
      2Gemini 4 Argon effort medium
      94%95% CI 74–99
      		18
      2Gemini 4 Argon effort high
      94%95% CI 74–99
      		18
      2Muse Wonder effort medium
      94%95% CI 74–99
      		18
      8Claude Haiku 5.5 effort medium
      89%95% CI 67–97
      		18
      8Claude Haiku 5.5 effort high
      89%95% CI 67–97
      		18
      8Claude Opus 4.8 thinking off
      89%95% CI 67–97
      		18
      8Claude Sonnet 5 effort high
      89%95% CI 67–97
      		18
      8Claude Sonnet 5.5 effort high
      89%95% CI 67–97
      		18
      8Claude Sonnet 5.5 effort low
      89%95% CI 67–97
      		18
      8Claude Sonnet 5.5 effort medium
      89%95% CI 67–97
      		18
      8DeepSeek V4.1 Flash Native FP8, FP4 experts · SGLang · effort high
      89%95% CI 67–97
      		18
      8Gemini 3.8 Flash effort medium
      89%95% CI 67–97
      		18
      8Gemini 4 Argon effort low
      89%95% CI 67–97
      		18
      8GPT-6 Astra effort medium
      89%95% CI 67–97
      		18
      8GPT-6.1 Sol effort high
      89%95% CI 67–97
      		18
      8Grok 4.7 effort high
      89%95% CI 67–97
      		18
      21Claude Opus 4.8 effort high
      83%95% CI 61–94
      		18
      21Claude Opus 5.5 effort high
      83%95% CI 61–94
      		18
      21Claude Opus 5.5 effort low
      83%95% CI 61–94
      		18
      21Claude Sonnet 5 effort medium
      83%95% CI 61–94
      		18
      21DeepSeek V4.1 Flash FP8 · effort high
      83%95% CI 61–94
      		18
      21Muse Wonder effort max
      83%95% CI 61–94
      		18
      21Muse Wonder effort low
      83%95% CI 61–94
      		18
      28Claude Haiku 5.5 effort low
      78%95% CI 55–91
      		18
      28Claude Sonnet 5 thinking off
      78%95% CI 55–91
      		18
      28DeepSeek V4.1 Flash EXL3 2.9bpw · vLLM + ExLlamaV3 · effort high
      78%95% CI 55–91
      		18
      28GPT-5.6 Sol effort medium
      78%95% CI 55–91
      		18
      28Grok 4.6 effort high
      78%95% CI 55–91
      		18
      28Muse Wonder effort high
      78%95% CI 55–91
      		18
      34Claude Sonnet 5 effort low
      72%95% CI 49–88
      		18
      34DeepSeek V4.1 Flash 1017 Native FP8, FP4 experts · SGLang · effort high
      72%95% CI 49–88
      		18
      34GLM-5.3 Flash FP8 · effort high
      72%95% CI 49–88
      		18
      34GPT-6 Luna effort medium
      72%95% CI 49–88
      		18
      34MiMo V2.6 Pro FP8 · thinking on
      72%95% CI 49–88
      		18
      39DeepSeek V4 Pro FP8 · effort high
      67%95% CI 44–84
      		18
      40Nemotron 3 Ultra thinking on
      61%95% CI 39–80
      		18
      41GLM-5.3 FP8 · effort low
      56%95% CI 34–75
      		18
      41GLM-5.3 Flash EXL3 4.0bpw experts, FP8 dense · vLLM + ExLlamaV3 · effort high
      56%95% CI 34–75
      		18
      41GLM-5.3 Flash EXL3 4.0bpw experts, FP8 dense · vLLM + ExLlamaV3 · thinking off
      56%95% CI 34–75
      		18
      41Inkling Small FP8 · effort high
      56%95% CI 34–75
      		18
      41Qwen3.8 Flash-Next INT4-AutoRound, 5 of 10 experts · TensorFold (Zig) · effort medium
      56%95% CI 34–75
      		18
      46DeepSeek V4 Flash Vision experimental FP8 + FP4 experts · vLLM · effort high
      50%95% CI 29–71
      		18
      46Gemma 4 31B QAT W4A16 · vLLM · thinking on
      50%95% CI 29–71
      		18
      46Nemotron 3.5 Super VL NVFP4 mixed precision (NVIDIA ModelOpt) · TensorFold (Zig) · thinking on
      50%95% CI 29–71
      		18
      49Claude Haiku 4.5 thinking on
      44%95% CI 25–66
      		18
      49Claude Haiku 4.5 thinking off
      44%95% CI 25–66
      		18
      49GLM-5.3 Flash EXL3 4.0bpw experts, FP8 dense · vLLM + ExLlamaV3 · effort low
      44%95% CI 25–66
      		18
      49MiMo V2.6 Flash FP8 · thinking on
      44%95% CI 25–66
      		18
      49Qwen3.8 27B MLX 8-bit · MTPLX · effort low
      44%95% CI 25–66
      		18
      49Qwen3.8 27B Splash 4-bit · Splash · effort low
      44%95% CI 25–66
      		18
      49Qwen3.8 Flash-Next NVFP4 · vLLM · effort medium
      44%95% CI 25–66
      		18
      49Qwen3.8 Flash-Next MLX 4-bit · MTPLX · effort medium
      44%95% CI 25–66
      		18
      57Claude Fable 5.1 effort high
      39%95% CI 20–61
      		18
      57Gemma 4 26B-A4B NVFP4 · vLLM · thinking on
      39%95% CI 20–61
      		18
      57MiMo V2.6 Flash FP8 + MXFP4 experts · SGLang · thinking on
      39%95% CI 20–61
      		18
      57Mistral Large 4 effort high
      39%95% CI 20–61
      		18
      57Muse Glimmer 30B NVFP4 + FP8 mixed (NVIDIA ModelOpt) · vLLM · effort high
      39%95% CI 20–61
      		18
      57Qwen3.6 35B-A3B NVFP4 · vLLM · thinking on
      39%95% CI 20–61
      		18
      57Qwen3.8 27B AWQ-INT4 · vLLM · effort medium
      39%95% CI 20–61
      		18
      57Qwen3.8 27B AWQ-INT4 · vLLM · effort low
      39%95% CI 20–61
      		18
      57Qwen3.8 Flash-Next MLX 4-bit · TensorFold · effort medium
      39%95% CI 20–61
      		18
      57Qwen3.8 Flash-Next INT4-AutoRound, 5 of 10 experts · TensorFold (Zig) · effort medium
      39%95% CI 20–61
      		18
      67GLM-5.3 Flash EXL3 4.0bpw experts, 4-bit dense · TensorFold · effort high
      33%95% CI 16–56
      		18
      67Laguna S 2.1 FP4 · thinking on
      33%95% CI 16–56
      		18
      67Ling 3.0 Flash BF16 · thinking on
      33%95% CI 16–56
      		18
      67MiniCPM-V 4.7 35B-A3B BF16 · vLLM · thinking on
      33%95% CI 16–56
      		18
      67MiniMax M3.1 Flash NVFP4 experts, MXFP8 (QAT) · vLLM · effort high
      33%95% CI 16–56
      		18
      67Nemotron 3.5 Lightning NVFP4 · thinking on
      33%95% CI 16–56
      		18
      67Qwen3.8 27B MLX 4-bit · MTPLX · effort low
      33%95% CI 16–56
      		18
      67Qwen3.8 27B AWQ-INT4 · vLLM · effort xhigh
      33%95% CI 16–56
      		18
      67Qwen3.8 Flash-Next thinking on
      33%95% CI 16–56
      		18
      76Ornith 1.5 122B NVFP4 experts, FP8 attention (NVIDIA ModelOpt) · vLLM · thinking on
      28%95% CI 13–51
      		18
      77MiniCPM-V 4.7 35B-A3B INT4 W4A16 experts, BF16 dense · vLLM · thinking on
      22%95% CI 9–45
      		18
      77Nemotron 3.5 Bolt IQ2_XXS + NVFP4 + Q8_0 mixed (NVIDIA ModelOpt) · TensorFold (Zig) · thinking on
      22%95% CI 9–45
      		18

      Tasks

      What's in the suite

      Solve rate is the share of finished runs whose first answer to the task passed (pass@1), a rough guide to how hard models find it. Difficulty is the task author’s label; from results is what the solve rate says: easy when two thirds or more of the finished runs pass it, hard at a third or fewer, once five have graded it.

      Salesforce APIs tasks with difficulty and solve rate
      TaskDifficultyFrom resultsSolve rate
      Close an opportunity and delete a duplicate contactapi-sobject-update-deleteeasyeasy
      96%
      Create a Case with the REST APIapi-sobject-create-caseeasyeasy
      97%
      Pick the API for a nightly 5-million-row loadapi-choose-bulk-for-large-loadeasyeasy
      100%
      Pre-sync checks with describe and limitsapi-describe-and-limitseasyeasy
      85%
      Publish a platform event from an external systemapi-platform-event-publisheasyeasy
      89%
      Query Apex code coverage with the Tooling APIapi-tooling-coverage-queryeasymedium
      59%
      Call a managed-package Apex REST endpointapi-apexrest-namespaced-postmediummedium
      65%
      Create and update records with sObject Collectionsapi-collections-create-updatemediummedium
      53%
      Fetch a record with UI API fields and optionalFieldsapi-uiapi-record-optional-fieldsmediummedium
      47%
      Page through a delta query that includes deleted recordsapi-query-paging-queryallmediummedium
      49%
      Request a token with the OAuth 2.0 client credentials flowapi-oauth-client-credentialsmediumeasy
      92%
      Run anonymous Apex through the Tooling APIapi-tooling-execute-anonymousmediummedium
      37%
      Upsert an account by external ID with its parentapi-upsert-external-idmediummedium
      66%
      Bulk API 2.0 export including deleted recordsapi-bulk2-query-exporthardmedium
      54%
      Bulk API 2.0 upsert with a parent external IDapi-bulk2-ingest-upserthardmedium
      43%
      Composite request with an implicit dependencyapi-composite-implicit-dependencyhardhard
      8%
      Independent all-or-nothing orders in one callapi-composite-graph-ordershardmedium
      41%
      Interpret external-ID upsert responsesapi-upsert-response-semanticshardmedium
      35%

      Private evaluations

      Want this for your team?

      Azul Labs runs Forcebench privately for two kinds of Salesforce team.

      Choosing a model to run yourself?

      A private benchmark of your shortlist

      Open-weight models at the quantisations and reasoning efforts you could run on infrastructure you control, tested on your kind of Salesforce work, with a written recommendation.

      • Private tasks written from your work only ever run on local models; they are never sent to an API.
      • Re-testing after any change of model, quantisation or engine is available.
      • Typical turnaround is 2–3 weeks; large models take about a day of machine time per configuration.
      Get a private benchmarkHow it works

      Building or buying a Salesforce AI tool?

      A private evaluation of your tool

      Your tool on held-out tasks it has never seen, against API baselines (models reached through a hosted service) and Local ones (served on Forcebench’s own hardware), with a written report on where it holds up and where it fails.

      • It can start as a smaller pilot, about 30 held-out tasks with your tool against two baselines, before a full evaluation.
      • Repeat evaluations after your releases are available.
      • Typical turnaround is 2–3 weeks.
      Ask about an evaluationHow it works

      Fixed fee, quoted up front. The quote depends on the number of tasks and the number of models, tools or configurations tested.

      Or email leo@azl.au · azl.au