We asked Claude Code to build an enterprise MCP server. It fell short.
MCP has changed how AI agents access data. AI coding tools have changed how software gets built. We wanted to know if you could combine the two to build a connector you'd trust in production.
After testing across eight dimensions critical to enterprise MCP reliability (e.g., OAuth token lifecycle, large dataset behavior), we found a meaningful difference between connecting to data and delivering it dependably. Claude Code cleared the first bar. It didn't clear the second.
What we found
Each test dimension maps to a real-world failure mode. The cost of falling short isn't just a failed query—it's an AI agent returning incomplete data silently, a business decision made on a stale record, and an engineering team discovering the problem weeks later.
correct implementation requiring no human intervention
functional but with gaps
incorrect or missing implementation requiring human diagnosis and rewrite
Where it broke
Claude Code is optimized to produce software that works. Enterprise production environments require software that keeps working—under conditions the initial build never encountered. That's where the gap appears.
A list with 13 lookup columns was queried. The server silently dropped one column from the response. No warning. No error. The agent had no way to know its data was incomplete.
Pagination hallucination
Asked to retrieve all items from a 6,500-row list, the agent reported 500 valid results and stopped—incorrectly concluding it had entered a retrieval loop. Proxy inspection confirmed all eight pages were fetched correctly. The agent's count was wrong.
Unusable field data
Hyperlink columns returned raw Java object references instead of usable data. Filter and sort parameters appeared functional but were silently ignored. Every call returned the full unfiltered dataset regardless of what was passed.
The pattern: the distance between code that compiles and code that holds up in production is wider than it appears.
Methodology
9 sessions, 2 phases Phase 1: vanilla Claude Code, minimal prompting. Phase 2: experienced developer, structured workflow, CLAUDE.md, plan-first implementation.
8 evaluation dimensions Each mapped to a production-grade connector standard: correctness, semantic resolution, resilience at scale, authorization fidelity, depth, adaptability.
Pass / Partial / Fail Pass: correct with no human intervention.
Partial: gaps found under real conditions.
Fail: requires human diagnosis and rewrite.
Real data, realistic scale Production SharePoint environment. Lists ranging from three to 40,000+ items. No synthetic test data.
Get the research
Executive Summary
Key findings, evaluation grid with real-world implications, failure pattern analysis, and conclusion.
Full Research Report
Complete methodology, dimension-by-dimension results for both phases, and implications for teams evaluating build vs. buy.
Companion Analysis
The natural follow-on question: how does the Claude Code-generated connector compare to CData Connect AI on those same eight dimensions? Side-by-side evaluation with screenshots, log excerpts, and tool call traces.