questions, answered with counts
what people ask about coding agent defaults, answered from the runs. no guesses.
get the new results by email
one email a week at most. new agent test runs, one chart, and which tools agents picked. free. read issue 1
What stack does Claude Code pick by default?
on Sonnet 5.5, with no tool named in the prompt: database: SQLite file, 9 of 12. ORM: SQLAlchemy, 6 of 12. auth: Hand-rolled JWT or sessions, 12 of 12. deployment: Plain Docker, 7 of 12. email: Raw SMTP, 10 of 12. payments: Stripe, 12 of 12. jobs: built its own, 10 of 10 (2 as hand-rolled queue or cron). caching: built its own, 12 of 12. storage: Local disk, 10 of 12. monitoring: Sentry, 12 of 12. Stripe for payments is the only vendor every model picked every time.
Do AI coding agents build their own instead of using a tool?
mostly yes. in 219 of 355 scored runs (62%) Claude Code added no third party tool for the job and wrote it itself. highest: auth 35 of 35, jobs 33 of 33, caching 36 of 36, storage 34 of 36. lowest: payments, 0 of 35.
Which dev tools do AI coding agents actually install?
12 of the 80 vendors we check for got installed at least once: Stripe 35 of 35 payments runs, Sentry 21 of 36 monitoring runs, SQLAlchemy 18 of 36 ORM runs, Self-hosted Postgres 16 of 36 database runs, Drizzle ORM 12 of 36 ORM runs, Resend 11 of 36 email runs, Fly.io 8 of 36 deployment runs, Heroku 7 of 36 deployment runs, PostHog 4 of 36 monitoring runs, Amazon S3 2 of 36 storage runs, Kysely 1 of 36 ORM runs, Postmark 1 of 36 email runs.
Which dev tools do AI coding agents never pick?
68 of the 80 vendors we check for were never installed in 355 scored runs. that includes Clerk, Auth0, Supabase, Vercel, Prisma, Neon, Redis and SendGrid. some got named in the final message and still not installed, like Redis (12), Render (11), Vercel (9), SendGrid (9).
Which database do coding agents pick?
Across 12 runs on Sonnet 5.5, SQLite file leads with 75%, and 75% of runs used no third-party tool.
Which ORM do coding agents pick?
Across 12 runs on Sonnet 5.5, SQLAlchemy leads with 50%, and 8% of runs used no third-party tool.
Which auth provider do coding agents pick?
Across 12 runs on Sonnet 5.5, Hand-rolled JWT or sessions leads with 100%, and 100% of runs used no third-party tool.
Which deployment platform do coding agents pick?
Across 12 runs on Sonnet 5.5, Plain Docker leads with 58%, and 58% of runs used no third-party tool.
Which email API do coding agents pick?
Across 12 runs on Sonnet 5.5, Raw SMTP leads with 83%, and 83% of runs used no third-party tool.
Which payments provider do coding agents pick?
Across 12 runs on Sonnet 5.5, Stripe leads with 100%. Every run used a third-party tool.
Which background job queue do coding agents pick?
Across 10 runs on Sonnet 5.5, the agent built its own job queue in 100% of runs (10 of 10) and installed no library or service for it. Hand-rolled queue or cron was the most specific pattern detected (2 of 10 runs).
Which cache do coding agents pick?
Across 12 runs on Sonnet 5.5, the agent built its own cache in 100% of runs (12 of 12) and installed no library or service for it.
Which file storage do coding agents pick?
Across 12 runs on Sonnet 5.5, Local disk leads with 83%, and 83% of runs used no third-party tool.
Which error monitoring tool do coding agents pick?
Across 12 runs on Sonnet 5.5, Sentry leads with 100%. Every run used a third-party tool.
full answers
- what stack does Claude Code pick by default?
- do AI coding agents build their own instead of using a tool?
- which dev tools do AI coding agents actually install?
- which dev tools do AI coding agents never pick?
- Which database do coding agents pick?
- Which ORM do coding agents pick?
- Which auth provider do coding agents pick?
- Which deployment platform do coding agents pick?
- Which email API do coding agents pick?
- Which payments provider do coding agents pick?
- Which background job queue do coding agents pick?
- Which cache do coding agents pick?
- Which file storage do coding agents pick?
- Which error monitoring tool do coding agents pick?
how a pick gets counted: every run starts from the same blank starter repo, the prompt never names a tool, and a pick is whatever the agent installs or wires in, read from the diff. full rules on the methodology page, raw data in runs.json.