Suites / npsp
Nonprofit Success Pack
Engineering on the Nonprofit Success Pack (NPSP) managed packages (npsp, npe01, npo02, npe03, npe4, npe5): the Household Account model, gifts and payments, GAU allocations, soft credits, Enhanced Recurring Donations, affiliations and relationships, TDTM trigger handlers and the BDI data import API. Code answers are deployed to a scratch org with NPSP installed and must pass hidden Apex tests that exercise NPSP's own automation; SOQL answers run against seeded NPSP data; knowledge questions cover NPSP architecture and configuration.
How it's graded
Code answers are deployed as a check-only validation to a scratch org with NPSP installed and must pass hidden Apex tests that exercise NPSP’s own automation. SOQL answers run against seeded NPSP data; knowledge questions cover NPSP architecture and configuration.
Difficulty mix
Leaderboard
Who's best at Nonprofit Success Pack
pass@1 on this suite only, with 95% confidence intervals. With 18 tasks the intervals are wide, so read overlapping bars as ties. Open in the full leaderboard →
| # | Model and config | Score pass@1 · 95% CI | Tasks |
|---|---|---|---|
| 1 | DeepSeek V4.1 Flash EXL3 2.9bpw · vLLM + ExLlamaV3 · effort high | 44%95% CI 22–67 | 18 |
| 2 | Qwen3.8 27B AWQ-INT4 · vLLM · effort xhigh | 22%95% CI 6–44 | 18 |
| 2 | Qwen3.8 Flash-Next MLX 4-bit · MTPLX · effort medium | 22%95% CI 6–44 | 18 |
| 4 | Gemma 4 26B-A4B NVFP4 · vLLM · effort on | 17%95% CI 0–33 | 18 |
| 4 | Gemma 4 31B QAT W4A16 · vLLM · effort on | 17%95% CI 0–33 | 18 |
| 4 | Qwen3.6 35B-A3B NVFP4 · vLLM · effort on | 17%95% CI 0–33 | 18 |
| 4 | Qwen3.8 27B MLX 8-bit · MTPLX · effort low | 17%95% CI 0–33 | 18 |
| 4 | Qwen3.8 27B MLX 4-bit · MTPLX · effort low | 17%95% CI 0–33 | 18 |
| 4 | Qwen3.8 27B Splash 4-bit · Splash · effort low | 17%95% CI 0–33 | 18 |
| 4 | Qwen3.8 27B AWQ-INT4 · vLLM · effort low | 17%95% CI 0–33 | 18 |
| 4 | Qwen3.8 Flash-Next NVFP4 · vLLM · effort medium | 17%95% CI 0–33 | 18 |
| 12 | Qwen3.8 27B AWQ-INT4 · vLLM · effort medium | 15%95% CI 0–31 | 18 |
| – | GLM-5.3 Flash partialEXL3 4.0bpw · vLLM + ExLlamaV3 · effort highRun so far: 14 of 15 suites · 219 of 272 tasks graded | 45%95% CI 18–73 | 11 |
Tasks
What's in the suite
Solve rate is the mean pass@1 on that task across every finished run, a rough guide to how hard models find it.
| Task | Difficulty | Solve rate |
|---|---|---|
| NPSP versus Nonprofit Cloudnpsp-vs-nonprofit-cloud | easy | 100% |
| Query pledge payments still duenpsp-soql-unpaid-payments-due | easy | 8% |
| Record a donation the NPSP waynpsp-record-gift | easy | 0% |
| Set donors' current employer with NPSP affiliationsnpsp-affiliation-primary-employer | easy | 0% |
| Which NPSP object API names are real?npsp-namespace-objects | easy | 8% |
| Annual-report totals per GAUnpsp-soql-gau-totals | medium | 0% |
| Create one NPSP household for several new contactsnpsp-household-create-members | medium | 0% |
| Predict NPSP's household formal greetingnpsp-household-formal-greeting | medium | 8% |
| Promote NPSP Customizable Rollups to productionnpsp-crlp-promote-to-production | medium | 97% |
| Record parent relationships with NPSP reciprocalsnpsp-relationship-parent-child | medium | 0% |
| Replace a pledge's open payments with an installment plannpsp-payment-installments | medium | 8% |
| Skip an NPSP trigger handler for one integration usernpsp-trigger-handler-exclude-user | medium | 100% |
| Split a gift across GAUs by percentagenpsp-gau-percent-allocations | medium | 17% |
| Bulk partial soft credits linked to contact rolesnpsp-partial-soft-credits | hard | 0% |
| Create monthly Enhanced Recurring Donationsnpsp-enhanced-rd-monthly | hard | 0% |
| Import web gifts through NPSP Data Importer (BDI)npsp-bdi-gift-import | hard | 0% |
| Major gift follow-up as an NPSP TDTM handlernpsp-tdtm-major-gift-handler | hard | 0% |
| Testing a custom TDTM handler in a subscriber orgnpsp-tdtm-test-registration | hard | 8% |
Work with Leo
Want this for your team?
Leo helps Salesforce teams run AI they own: open-weight models on infrastructure you control, tested on your kind of work before you rely on them. A private benchmark of your shortlist is a fixed fee.
Get a private benchmarkOr email leo@azl.au · azl.au