Validation Experiments & Hypotheses
Hypothesis Framework
Hypothesis #1: Problem Existence 🔴 Critical
We believe that solo founders and bootstrapped entrepreneurs will actively seek viability analysis tools if they are trying to validate a new product idea. We will know this is true when we see 60%+ of surveyed founders confirm this is a top-3 pain point AND 5%+ landing page signup rate.
Risk Level: 🔴 Critical (product fails if wrong)
Current Evidence: Supporting: Forum discussions, search volume, competitor traction. Contradicting: None identified. Gaps: No direct user interviews yet.
Experiment Design:
Method: Customer discovery interviews + landing page test
Sample Size: 20 interviews, 1,000 landing page visitors
Duration: 2 weeks
Cost: $500 (ads) + 20 hours (interviews)
Success Metrics:
| Metric | Fail | Minimum | Success | Home Run |
|---|---|---|---|---|
| Problem confirmation rate | <40% | 40-60% | 60-80% | >80% |
| Landing page signup | <2% | 2-5% | 5-10% | >10% |
Next Steps if Validated: Proceed to solution validation. Next Steps if Invalidated: Pivot to adjacent problem or exit.
Hypothesis #2: Solution Fit 🔴 Critical
We believe that founders seeking validation will use an AI-powered analysis tool instead of manual research if we deliver comprehensive, actionable reports in minutes instead of weeks. We will know this is true when we see 70%+ of prototype users rate the output as "useful" or "very useful".
Risk Level: 🔴 Critical
Current Evidence: Supporting: Initial feedback from prototypes. Contradicting: User skepticism about AI accuracy. Gaps: No formalized testing yet.
Experiment Design:
Method: Prototype testing with early adopters.
Sample Size: 10 prototype users.
Duration: 2 weeks.
Cost: $200 (incentives).
Success Metrics:
| Metric | Fail | Minimum | Success | Home Run |
|---|---|---|---|---|
| User satisfaction rating | <6/10 | 6-7/10 | 7-8/10 | >8/10 |
Next Steps if Validated: Proceed to pricing validation. Next Steps if Invalidated: Reassess feature set.
Hypothesis #3: Willingness to Pay 🔴 Critical
We believe that bootstrapped founders will pay $49-$99 for a single viability analysis if we provide investor-grade output that saves 20+ hours of research. We will know this is true when we see 10+ pre-orders at target price point.
Risk Level: 🔴 Critical
Current Evidence: Supporting: Pricing benchmarks from competitors. Contradicting: User hesitance for upfront payments. Gaps: No pre-order testing yet.
Experiment Design:
Method: Pricing survey with pre-order option.
Sample Size: 100 target users.
Duration: 2 weeks.
Cost: $300 (survey tools).
Success Metrics:
| Metric | Fail | Minimum | Success | Home Run |
|---|---|---|---|---|
| Pre-orders collected | <5 | 5-10 | 10-20 | >20 |
Next Steps if Validated: Finalize pricing model. Next Steps if Invalidated: Reevaluate pricing strategy.
Experiment Catalog
Experiment #1: Problem Discovery Interviews
Hypothesis Tested: #1 (Problem Existence)
Method: Semi-structured interviews with target users
Setup:
- Recruit 20-30 founders via LinkedIn, Twitter, Reddit
- Offer $50 gift card incentive
- Schedule 45-60 minute video calls
- Use interview guide (see User Research section)
- Record and transcribe conversations
Metrics:
- % confirming problem as top-3 pain
- Frequency of problem occurrence
- Current spend on alternatives (time/money)
- Quotes indicating severity
Timeline: 2 weeks (parallel recruitment and interviews)
Cost: $1,000-$1,500 (incentives)
Success Criteria:
- ✅ Pass: 60%+ confirm problem as significant
- ⚠️ Re-evaluate: 40-60% confirmation
- ❌ Fail: <40% confirmation
Owner: [Assign responsibility]
Experiment #2: Landing Page Smoke Test
Hypothesis Tested: #1 (Problem Existence) + #2 (Solution Interest)
Method: Landing page with waitlist signup
Setup:
- Create single landing page (Carrd, Unbounce, or custom)
- Write compelling headline and value proposition
- Add waitlist email capture form
- Drive traffic via Google/Facebook ads
- Track conversions with analytics
Variants to Test:
- Headline A: "Validate your startup idea in 24 hours"
- Headline B: "AI replaces your $50K business consultant"
- Headline C: "Stop building products nobody wants"
Metrics:
- Traffic volume (target: 1,000+ visitors)
- Signup rate by variant
- Time on page
- Scroll depth
Timeline: 2 weeks (1 week setup, 1 week traffic)
Cost: $500-$1,000 (ads)
Success Criteria:
- ✅ Pass: >5% signup rate
- ⚠️ Re-evaluate: 2-5% signup rate
- ❌ Fail: <2% signup rate
Experiment #3: Wizard of Oz MVP
Hypothesis Tested: #2 (Solution Fit) + #3 (Willingness to Pay)
Method: Manually deliver the service using AI + human judgment
Setup:
- Accept project specs via Google Form
- Generate analysis using Claude/GPT with custom prompts
- Polish and format output manually
- Deliver via email with feedback request
- Offer to pay after receiving output
Metrics:
- Time to deliver (target: <24 hours)
- User satisfaction (1-10 rating)
- NPS score
- % willing to pay after seeing output
- Actual payment conversion
Timeline: 4 weeks (10-20 users)
Cost: Time only (10-20 hours of effort)
Success Criteria:
- ✅ Pass: 8+/10 avg satisfaction, 50%+ would pay
- ⚠️ Re-evaluate: 6-8/10 satisfaction
- ❌ Fail: <6/10 satisfaction, <30% would pay
Experiment Prioritization Matrix
| Experiment | Hypothesis | Impact | Effort | Risk if Skipped | Priority |
|---|---|---|---|---|---|
| Discovery Interviews | #1 | 🔴 Critical | Medium | Fail | 1 |
| Landing Page Test | #1, #2 | 🔴 Critical | Low | Fail | 2 |
| Wizard of Oz MVP | #2, #3 | 🔴 Critical | High | Fail | 3 |
| Pricing Survey | #3 | 🟡 High | Low | Suboptimal pricing | 4 |
| Pre-Order Test | #3 | 🟢 Medium | Medium | Lack of validation | 5 |
8-Week Validation Schedule
Week 1-2: Problem Validation
| Day | Activity | Owner | Deliverable |
|---|---|---|---|
| D1-D3 | Launch landing page | Live page + analytics | |
| D1-D7 | Recruit interview participants | 20 scheduled calls | |
| D4-D14 | Conduct interviews | 20 completed, transcribed | |
| D8-D14 | Run landing page ads ($500) | 1,000+ visitors |
Week 3-4: Solution Validation
| Day | Activity | Owner | Deliverable |
|---|---|---|---|
| D15-D18 | Analyze interview data | Problem validation report | |
| D15-D21 | Build Wizard of Oz process | Manual delivery workflow | |
| D19-D28 | Deliver to 10 users | 10 completed analyses |
Week 5-6: Pricing & Willingness to Pay
| Day | Activity | Owner | Deliverable |
|---|---|---|---|
| D29-D35 | Run pricing survey | 100+ responses | |
| D29-D35 | Collect post-delivery payments | Payment conversion data | |
| D36-D42 | Analyze pricing data | Optimal price recommendation |
Week 7-8: Synthesis & Decision
| Day | Activity | Owner | Deliverable |
|---|---|---|---|
| D43-D49 | Compile all experiment results | Validation summary | |
| D50-D52 | Make Go/No-Go decision | Decision document | |
| D53-D56 | Plan Phase 2 (if Go) | MVP spec or pivot plan |
Minimum Success Criteria (Go/No-Go)
| Category | Metric | Must Achieve | Nice-to-Have |
|---|---|---|---|
| Problem | Interview confirmation | 60%+ | 80%+ |
| Problem | Landing page signup | 5%+ | 10%+ |
| Solution | Prototype satisfaction | 7/10+ | 8.5/10+ |
| Solution | NPS | 30+ | 50+ |
| Pricing | Willingness to pay at $X | 50%+ | 70%+ |
| Pricing | Pre-orders collected | 10+ | 25+ |
| Overall | Hypotheses validated | 3/5 critical | 5/5 critical |
Go Decision: All "Must Achieve" criteria met. Conditional Go: 70% of criteria met, clear path to remainder. No-Go Decision: <70% of criteria met, no clear fixes.
Pivot Triggers & Contingency Plans
Trigger #1: Problem Doesn't Exist
- Signal: <40% of users confirm problem
- Action: Interview users about their actual top problems, identify adjacent pain points
- Pivot Options: Different problem in same audience, same problem in different audience
Trigger #2: Solution Doesn't Resonate
- Signal: <50% satisfaction with prototype
- Action: Deep-dive on what's missing, what's confusing, what's not valuable
- Pivot Options: Simplify scope, change format, add human touch
Trigger #3: Won't Pay Enough
- Signal: Acceptable price is <50% of target
- Action: Find higher-value use case, different segment, or reduce costs
- Pivot Options: Freemium with upsell, enterprise pivot, cost optimization
Trigger #4: Can't Acquire Efficiently
- Signal: CAC >3x target in all channel tests
- Action: Test organic/viral channels, reconsider pricing model
- Pivot Options: Product-led growth, community-first, partnership distribution
Experiment Documentation Template
For each completed experiment, document:
## Experiment: [Name]
**Date:** [Start - End]
**Hypothesis Tested:** #X
### Setup
- What we did
- Sample size
- Tools used
- Cost incurred
### Results
| Metric | Target | Actual | Pass/Fail |
|--------|--------|--------|-----------|
### Key Learnings
- Insight #1
- Insight #2
- Surprise finding
### Evidence
- [Link to data]
- [Quotes/screenshots]
### Next Steps
- [What this means for the product]
- [Follow-up experiments needed]