Models / deepseek-v4-pro

DeepSeek V4 Pro

DeepSeek's model, with open weights. Forcebench has run it in 1 configuration on the API board. API is models reached through a hosted service; Local, models served on Forcebench’s own hardware.

On the API board

Models reached through a hosted service. DeepSeek V4 Pro is ranked by its one configuration: the 16th-highest score of 26 models, not clearly different from 6 other models. Scores are within 3 attempts (c@3) with 95% intervals; each configuration is compared with the highest-scoring one by whether their 95% intervals overlap. Every configuration on the board → Cost and speed →

69%
within 3 attempts (c@3), 95% CI 65–74 · high · 16th-highest score of 26 models, not clearly different from 6 other models
11k
output tokens per task · median 7.8k, p90 27k
127 s
median per task, every attempt counted, including network and queueing · 90th percentile 659 s
6%
no answer within the 32,768-token budget
$0.052
per task at the list price of 10 October 2026, every attempt counted
$0.075
per solved task: the total cost over the tasks solved · cost and speed
Every configuration of DeepSeek V4 Pro on the API board
ConfigurationRank all configsWithin 3 attempts c@3 · 95% CITokens mean · median · p90No answerLatency median per task, every attempt countedCost per taskCost per solved task
OpenRouter · FP8 · highhighest score28
69%95% CI 65–74
11k7.8k · 27k6%127 s$0.052$0.075
Suites, at its highest-scoring configuration
high · OpenRouter · FP8: within 3 attempts (c@3) on each suite with 95% intervals, against the median of the API board's models, each at its highest-scoring configuration.
  • 95% interval
  • API board median
Apex
80%, 95% interval 58–92%
Salesforce APIs
67%, 95% interval 44–84%
CI/CD
88%, 95% interval 66–97%
Salesforce CLI
90%, 95% interval 70–97%
Salesforce docs
18%, 95% interval 6–41%
fflib Enterprise Patterns
56%, 95% interval 34–75%
Flow
17%, 95% interval 6–39%
Governor limits & pushback
89%, 95% interval 69–97%
Lightning Web Components
100%, 95% interval 82–100%
Nonprofit Success Pack
72%, 95% interval 49–88%
Packaging
50%, 95% interval 29–71%
Permissions & access
87%, 95% interval 62–96%
Scratch org definitions
39%, 95% interval 20–61%
SOQL
100%, 95% interval 84–100%
Trigger Actions Framework
89%, 95% interval 67–97%
    Show the numbers
    Per-suite scores of DeepSeek V4 Pro's highest-scoring configuration on the API board, with the board's median
    Suitec@3 95% CIAPI medianTasks
    Apex
    80%95% CI 58–92
    		90%20
    Salesforce APIs
    67%95% CI 44–84
    		72%18
    CI/CD
    88%95% CI 66–97
    		88%17
    Salesforce CLI
    90%95% CI 70–97
    		90%20
    Salesforce docs
    18%95% CI 6–41
    		29%17
    fflib Enterprise Patterns
    56%95% CI 34–75
    		61%18
    Flow
    17%95% CI 6–39
    		69%18
    Governor limits & pushback
    89%95% CI 69–97
    		89%19
    Lightning Web Components
    100%95% CI 82–100
    		100%18
    Nonprofit Success Pack
    72%95% CI 49–88
    		81%18
    Packaging
    50%95% CI 29–71
    		69%18
    Permissions & access
    87%95% CI 62–96
    		87%15
    Scratch org definitions
    39%95% CI 20–61
    		50%18
    SOQL
    100%95% CI 84–100
    		100%20
    Trigger Actions Framework
    89%95% CI 67–97
    		86%18

    In the studies

    Private evaluations

    Want this for your team?

    Azul Labs runs Forcebench privately for two kinds of Salesforce team.

    Choosing a model to run yourself?

    A private benchmark of your shortlist

    Open-weight models at the quantisations and reasoning efforts you could run on infrastructure you control, tested on your kind of Salesforce work, with a written recommendation.

    • Private tasks written from your work only ever run on local models; they are never sent to an API.
    • Re-testing after any change of model, quantisation or engine is available.
    • Typical turnaround is 2–3 weeks; large models take about a day of machine time per configuration.
    Get a private benchmarkHow it works

    Building or buying a Salesforce AI tool?

    A private evaluation of your tool

    Your tool on held-out tasks it has never seen, against API baselines (models reached through a hosted service) and Local ones (served on Forcebench’s own hardware), with a written report on where it holds up and where it fails.

    • It can start as a smaller pilot, about 30 held-out tasks with your tool against two baselines, before a full evaluation.
    • Repeat evaluations after your releases are available.
    • Typical turnaround is 2–3 weeks.
    Ask about an evaluationHow it works

    Fixed fee, quoted up front. The quote depends on the number of tasks and the number of models, tools or configurations tested.

    Or email leo@azl.au · azl.au