A/B Test Designer

Design an A/B test for: [DESCRIBE WHAT YOU WANT TO TEST] Current metrics: - Conversion rate for this element: [X]% - Daily visitors/traffic to this page: [X] - Current revenue from this page: $[X]/month (if known) Provide: 1. Hypothesis: What you expect to happen and why 2. Minimum detectable effect: Smallest improvement worth detecting (based on revenue impact) 3. Sample size calculation: visitors needed per variant (95% confidence, 80% power) 4. Test duration estimate given my traffic 5. Control: Current version (describe what exists) 6. Variant: Specific changes with design rationale (grounded in conversion psychology) 7. Primary metric: The one number that determines success 8. Secondary metrics: Additional data to collect 9. Guardrail metrics: Metrics that must NOT decline (e.g., AOV, return rate) 10. Decision framework: What result means "ship," "iterate," or "kill"

**Test: Product Page Layout — Stacked vs. Side-by-Side Image/Details** **Hypothesis:** A layout that places product images in a larger left column (60%) with a sticky add-to-cart panel on the right will increase add-to-cart rate by reducing scroll distance to the CTA and keeping the purchase action always visible, based on Fitts's Law and sticky-element conversion research. **Current State (Control):** - Add-to-cart rate: 8.2% - Daily PDP visitors: 1,400 - AOV: $85 - Estimated monthly PDP revenue: 1,400 x 30 x 8.2% x 85 x (checkout rate) = ~$292K **Minimum Detectable Effect (MDE):** - Target: 1.0 percentage point absolute improvement (8.2% to 9.2%) - Revenue impact of 1pp: 1,400 x 30 x 1.0% x $85 = ~$35,700/month — well worth testing - A 1pp MDE balances statistical rigor with business significance **Sample Size Calculation:** - Baseline rate: 8.2% - MDE: 1.0pp (relative lift of ~12.2%) - Confidence level: 95% (alpha = 0.05, two-tailed) - Statistical power: 80% - Required: ~14,900 visitors per variant (29,800 total) - At 1,400/day: **~21 days minimum test duration** - Recommended: run for 28 days to capture a full weekly cycle x4 **Control (A): Current Layout** - Standard e-commerce PDP: image carousel top, details below on mobile; side-by-side on desktop - "Add to Cart" button scrolls out of view as user reads reviews - Reviews section is below the fold, requiring scroll commitment **Variant (B): Sticky Panel Layout** Changes: 1. Larger image gallery (60% width on desktop) with thumbnail strip 2. Sticky right panel that follows scroll: contains price, variant selector, "Add to Cart" button, and shipping estimate 3. Social proof snippet ("1,247 customers love this") pinned below CTA 4. Reviews moved higher — summary stars + 3 featured reviews visible with one scroll Rationale: The CTA is always accessible. Social proof is closer to the decision point. Larger images address the top objection in DTC skincare (wanting to see the product). **Metrics:** | Type | Metric | Notes | |------|--------|-------| | Primary | Add-to-cart rate | The ONE metric that decides the test | | Secondary | PDP-to-purchase rate | Captures downstream conversion effect | | Secondary | Time on PDP | Longer could mean engagement or confusion | | Secondary | Scroll depth | Did they see more product info? | | Guardrail | AOV | Must not decline >3% (layout changes can shift mix) | | Guardrail | Return rate (30-day) | Better images should not increase returns | **Decision Framework:** | Result | Action | |--------|--------| | Variant wins at >=95% confidence, AOV stable | Ship the variant | | Variant directionally positive (85-95% confidence) | Extend test 7 more days | | Variant positive but AOV drops >3% | Investigate product mix shift. Do not ship without understanding | | Variant flat or negative after 28 days | Kill. Test a different hypothesis (e.g., CTA copy, urgency elements) | **Pre-Test Checklist:** - QA the variant on mobile, tablet, and desktop (sticky behavior differs) - Ensure analytics fire correctly on the new layout (add-to-cart event, scroll depth) - Exclude bot traffic and internal IPs - Do not run other PDP tests simultaneously