Vibe Coding & AI IDEs · Risk Score 78 · Severe
Is Devin (Cognition AI) a Scam or Legit? Here's What the Record Shows.
Autonomous AI software engineer claiming to complete full engineering tasks end-to-end
Short answer: Not technically — but it's complicated.
Our read
Devin (Cognition AI) isn't a "scam" in the criminal sense — it's a real company with real software — but the complaint record raises serious questions about its sales practices, billing, and the gap between what's advertised and what's delivered.
The strongest "this feels like a scam" signals tend to come from one of two places: marketing claims that don't match reality, or billing practices that make leaving harder than joining. Below are the 2 flags in those two categories we've documented for Devin (Cognition AI).
Marketing & billing complaints
Misleading Marketing
CRITICAL
Source: News
Updated 20d ago
Launch demo benchmark claims found to be misleading by independent researchers
The viral March 2024 demo claimed Devin achieved '13.86% on SWE-bench' — but independent researchers including those at Princeton found the statistic was from a non-standard subset of the benchmark, with unreported human assistance. The claim was not corrected in the marketing and continued to be cited in media coverage.
"We replicated the evaluation and found the 13.86% figure used a non-verified subset with human-in-the-loop assistance not disclosed in the original announcement."
Billing Problems
HIGH
Source: Reddit
Updated 20d ago
Pricing at $500/month for capabilities that fall short of the demo
Early adopters describe paying $500/month and finding Devin reliable only for narrow, well-scoped tasks — not the end-to-end autonomous engineering in the launch demo. The price-to-actual-productivity ratio is the most common complaint in professional developer communities.
"It's an impressive tool but at $500/month I expected something closer to the demo. In reality I'm still reviewing and fixing almost everything it touches."
Our AI scanner searches Reddit, Trustpilot, BBB, and news sources for fresh complaints from the past year, paraphrases what it finds, and adds anything new to this page. Takes up to 90 seconds.
Everything else worth knowing
For the full record — lawsuits, churn signals, support failures, and BBB filings — see the complete complaints page.