Suites / flow
Flow
Salesforce Flow engineering: record-triggered flows (before-save fast field updates, after-save related-record automation, before-delete), entry conditions and $Record__Prior, decisions and formulas, Custom Error, fault paths, subflows, autolaunched flows called from Apex, invocable Apex actions, scheduled paths, and choosing between flows and Apex triggers. Answers are Flow metadata files (API 67.0) deployed check-only to a clean scratch org, where hidden Apex tests do DML (including 200-record bulk tests that assert on Limits) and check the outcome.
How it's graded
Flow metadata is deployed as a check-only validation to a clean scratch org, where hidden Apex tests do DML (including 200-record bulk tests that assert on governor limits) and check the outcome. Decision-guide questions (flow or Apex?) are multiple choice.
Difficulty mix
Leaderboard
Who's best at Flow
pass@1 on this suite only, with 95% confidence intervals. With 18 tasks the intervals are wide, so read overlapping bars as ties. Open in the full leaderboard →
| # | Model and config | Score pass@1 · 95% CI | Tasks |
|---|---|---|---|
| 1 | DeepSeek V4.1 Flash EXL3 2.9bpw · vLLM + ExLlamaV3 · effort high | 28%95% CI 11–50 | 18 |
| 2 | Qwen3.8 Flash-Next NVFP4 · vLLM · effort medium | 22%95% CI 6–44 | 18 |
| 3 | Qwen3.8 27B AWQ-INT4 · vLLM · effort medium | 20%95% CI 6–39 | 18 |
| 4 | Gemma 4 26B-A4B NVFP4 · vLLM · effort on | 17%95% CI 0–33 | 18 |
| 4 | Qwen3.8 27B MLX 8-bit · MTPLX · effort low | 17%95% CI 0–33 | 18 |
| 4 | Qwen3.8 27B MLX 4-bit · MTPLX · effort low | 17%95% CI 0–33 | 18 |
| 4 | Qwen3.8 Flash-Next MLX 4-bit · MTPLX · effort medium | 17%95% CI 0–33 | 18 |
| 8 | Gemma 4 31B QAT W4A16 · vLLM · effort on | 11%95% CI 0–28 | 18 |
| 8 | Qwen3.6 35B-A3B NVFP4 · vLLM · effort on | 11%95% CI 0–28 | 18 |
| 8 | Qwen3.8 27B Splash 4-bit · Splash · effort low | 11%95% CI 0–28 | 18 |
| 8 | Qwen3.8 27B AWQ-INT4 · vLLM · effort xhigh | 11%95% CI 0–28 | 18 |
| 12 | Qwen3.8 27B AWQ-INT4 · vLLM · effort low | 6%95% CI 0–17 | 18 |
| – | GLM-5.3 Flash partialEXL3 4.0bpw · vLLM + ExLlamaV3 · effort highRun so far: 14 of 15 suites · 219 of 272 tasks graded | 25%95% CI 0–50 | 12 |
Tasks
What's in the suite
Solve rate is the mean pass@1 on that task across every finished run, a rough guide to how hard models find it.
| Task | Difficulty | Solve rate |
|---|---|---|
| Autolaunched shipping quote flow called from Apexflow-shipping-quote-autolaunched | easy | 8% |
| Choose the tool for a reminder relative to a record dateflow-choice-close-date-reminder | easy | 100% |
| Choose the tool for a same-record field updateflow-choice-same-record-update | easy | 92% |
| Escalate breakdown cases with a fast field updateflow-case-breakdown-escalation | easy | 8% |
| Welcome-call task for new customer accountsflow-account-customer-welcome-task | easy | 0% |
| Automation when a record is restored from the Recycle Binflow-choice-undelete-restore | medium | 58% |
| Block closing a case that still has open tasksflow-case-close-open-tasks | medium | 0% |
| Call an invocable Apex credit check from a record-triggered flowflow-account-credit-check-action | medium | 0% |
| Close-date reminder on a scheduled pathflow-opportunity-close-reminder-scheduled | medium | 0% |
| Create a delivery project when an opportunity is wonflow-opportunity-won-project | medium | 0% |
| Flag large opportunity amount swings using $Record__Priorflow-opportunity-amount-swing | medium | 0% |
| Set a case SLA due date/time from its priorityflow-case-sla-due | medium | 0% |
| Cascade an account discount to its open opportunities in bulkflow-account-discount-cascade | hard | 0% |
| Fault path that logs failed shipment creationflow-order-shipment-fault-path | hard | 0% |
| Fix a flow that syncs account addresses to contactsflow-fix-account-address-sync | hard | 0% |
| Invocable Apex contact matcher orchestrated by a flowflow-web-signup-contact-match | hard | 0% |
| Record-Triggered Automation decision guide principlesflow-choice-guide-statements | hard | 14% |
| Reusable onboarding subflow called from a record-triggered flowflow-account-onboarding-subflow | hard | 0% |
Work with Leo
Want this for your team?
Leo helps Salesforce teams run AI they own: open-weight models on infrastructure you control, tested on your kind of work before you rely on them. A private benchmark of your shortlist is a fixed fee.
Get a private benchmarkOr email leo@azl.au · azl.au