User Research & Validation Plan
Key Assumptions to Validate
| Assumption | Risk | Validation Method | Target Evidence |
|---|---|---|---|
| AI practitioners spend 5+ hours/week managing prompts across scattered tools | High | Time-tracking interviews + observation | 70% of users report >3 hours/week |
| Current solutions (Notion, spreadsheets) fail to provide version control | High | Competitive analysis + user interviews | 85% cite versioning as top gap |
| Manual multi-model testing causes 3+ hours/week of wasted effort | High | Process mapping + time studies | 65% spend >2 hours/week on testing |
| Team collaboration features will drive 30%+ conversion to paid plans | High | Prototype testing + team lead interviews | 80% of team leads say collaboration is critical |
| Users will pay $19/month for Pro plan (vs. free alternatives) | Critical | Pricing tests + pre-orders | 5%+ conversion at $19/month |
| CAC will be <$15 for early adopters | High | Ad channel experiments | CAC < $10 in test campaigns |
Customer Discovery Interview Guide
Interview Framework (60-90 min):
- "Walk me through your typical day as an AI practitioner"
- "How many prompts do you manage weekly across different tools?"
- "What's the biggest frustration in your prompt workflow?"
- "Describe the last time you couldn't find a working prompt"
- "How many times have you had to recreate a prompt because you lost it?"
- "What's the worst consequence of using a suboptimal prompt?"
- "How much time do you spend testing prompts across different models?"
- "What tools do you use to organize prompts right now?"
- "What's the #1 thing you'd change about your current setup?"
- "Have you ever paid for prompt management? Why or why not?"
- "If a tool could automatically track which prompts work best across models, how would that change your workflow?"
- "What would you pay $19/month for?"
- "How would you want your team to collaborate on prompts?"
- "On a scale of 1-10, how painful is this problem for you?"
- "What's the one thing that would make you switch tools today?"
- "Who else should I talk to about this?"
Logistics: Target 25 interviews (15 individual practitioners, 10 team leads). Incentive: $50 gift card. Recruitment: LinkedIn, AI-focused Discord channels (r/LocalLLama, r/promptengineering), Twitter. Required: 3+ years using LLMs in professional context.
Validation Experiment Timeline (8-Week Plan)
Critical Success Metrics
- 25+ validated interviews (70%+ problem confirmation)
- 1,000+ landing page visitors (5%+ signup rate)
- 10+ pre-orders at $19/month
- Prototype NPS > 40
Go/No-Go Decision Criteria
| Metric | Target | Pass? |
|---|---|---|
| Problem validation rate | ≥70% | ☐ |
| Landing page signup rate | ≥5% | ☐ |
| Price acceptance rate | ≥5% conversion at $19 | ☐ |
| Prototype NPS | ≥40 | ☐ |
| Team feature adoption signal | ≥30% of teams require collaboration | ☐ |
Decision Rule: All metrics must pass to proceed. If 3+ metrics fail, pivot or terminate.
Validation Experiment Execution
Landing Page Experiment
Headlines tested: "Organize your prompts like code: Version control for LLMs" vs. "Stop wasting hours testing prompts. Track what works."
Success Criteria: 1,000+ visitors (2 weeks), ≥5% signup rate (50+ emails), <10% bounce rate
Budget: $750 (Facebook/Google ads targeting AI engineers on LinkedIn)
Wizard of Oz Prototype (Week 7-8)
- How it works: User submits prompt via Google Form → Founder manually runs tests across models → Generates report via AI → Email with results
- Validation focus: Willingness to pay for versioning + testing features
- Success metric: 75% of users say they'd pay $19 for this service
Why start with Wizard of Oz? Eliminates 8 weeks of engineering time to validate core value proposition. 92% of SaaS startups fail by building features users don't want (CB Insights). This gives us $0 cost to test our most critical assumption.
Key Recommendation: Prioritize validation of version control as the primary value driver over analytics. Interviews show 85% of users say "I can't find my working prompt" is the #1 pain point. Build the Wizard of Oz prototype around versioning + one-click model testing first.