P-Value Calculator

The p-value is the probability of observing data as extreme (or more extreme) than what you got, assuming the null hypothesis is true. It's used to make decisions in hypothesis testing. **The Z-Test Formula**: z = (x̄ - μ₀) / (σ/√n) Where: - x̄ = sample mean - μ₀ = hypothesized population mean (null hypothesis) - σ = population standard deviation - n = sample size - σ/√n = standard error of the mean **P-Value Interpretation**: | z-score | P-value (two-tailed) | Interpretation | |---------|----------------------|----------------| | 1.0 | 0.317 | Not significant | | 1.5 | 0.134 | Not significant | | 1.645 | 0.10 | Significant at α=0.10 | | 1.96 | 0.05 | Significant at α=0.05 | | 2.0 | 0.046 | Significant at α=0.05 | | 2.5 | 0.012 | Very significant | | 2.576 | 0.010 | Significant at α=0.01 | | 3.0 | 0.003 | Highly significant | | 3.291 | 0.001 | Significant at α=0.001 | **Common Significance Levels (α)**: - **α = 0.10**: Loosest, exploratory - **α = 0.05**: Most common standard - **α = 0.01**: Stricter - **α = 0.001**: Very strict - **α = 5 × 10⁻⁸**: GWAS (genomics) **Hypothesis Testing Steps**: 1. **State hypotheses**: - H₀ (null): No effect (μ = μ₀) - H₁ (alternative): Effect exists (μ ≠ μ₀, or μ > μ₀, or μ < μ₀) 2. **Choose α**: Usually 0.05 3. **Calculate test statistic**: Use appropriate test (z, t, etc.) 4. **Find p-value**: Probability of observed statistic under H₀ 5. **Decision**: - If p < α: Reject H₀ (significant) - If p ≥ α: Fail to reject H₀ (not significant) 6. **Interpret**: In context of the research question **One-tailed vs Two-tailed**: - **One-tailed**: Testing direction (x̄ > μ or x̄ < μ) - **Two-tailed**: Testing inequality (x̄ ≠ μ) One-tailed tests have smaller p-values but require directional hypothesis beforehand. Two-tailed is the default in most cases. **Example**: New drug claims to lower blood pressure from 140 mmHg Test on 100 patients, average = 133 mmHg, SD = 20 mmHg H₀: μ = 140 (no effect) H₁: μ < 140 (drug works) z = (133 - 140) / (20/√100) = -7/2 = -3.5 One-tailed p-value ≈ 0.0002 Conclusion: p << 0.05, reject H₀. Drug significantly lowers blood pressure. **Common Mistakes**: 1. **P-value is NOT the probability that H₀ is true** - It's the probability of data given H₀ is true - P(data|H₀) ≠ P(H₀|data) 2. **p = 0.05 is arbitrary** - There's no magical threshold - 0.049 and 0.051 are essentially identical 3. **Large p-value doesn't prove H₀** - 'Fail to reject' ≠ 'accept' - Could be underpowered study 4. **Statistical vs practical significance** - Tiny effect with huge n can be 'significant' but meaningless - Always report effect size alongside p-value 5. **Multiple testing problem** - 20 tests at α=0.05 will average 1 false positive - Use Bonferroni, FDR, or other corrections **Z-Test vs T-Test**: - **Z-test**: Use when population SD is known OR n ≥ 30 - **T-test**: Use when population SD is unknown AND n < 30 T-distribution has heavier tails for small samples, accounting for extra uncertainty. **Types of Errors**: - **Type I Error** (α): Rejecting true H₀ (false positive) - **Type II Error** (β): Failing to reject false H₀ (false negative) - **Power** (1-β): Ability to detect true effects Lower α → Lower Type I, but higher Type II (less power) **Effect Size**: P-values don't measure effect size. Report both: - **Cohen's d**: Standardized mean difference - **r**: Correlation coefficient - **Odds ratio**: For categorical data - **Relative risk**: Epidemiology **Typical Effect Sizes (Cohen's d)**: - Small: 0.2 - Medium: 0.5 - Large: 0.8 **The Replication Crisis**: Many psychology and biology studies fail to replicate. Reasons: - P-hacking (running multiple tests, reporting significant one) - Small samples - Publication bias (non-significant results rarely published) - Low-powered studies Modern recommendations: 1. Pre-register analyses 2. Report all results (significant or not) 3. Replicate findings 4. Focus on effect sizes, not just p-values 5. Consider Bayesian alternatives **American Statistical Association Statement (2016)**: Key points: - P-values do NOT measure the probability of hypotheses - Scientific conclusions shouldn't rest solely on p-values - Proper inference requires full reporting and transparency - A p-value doesn't measure effect size or importance - Without context, p-values are not very informative **Bayes Factors (Alternative)**: Measures relative support for H₀ vs H₁: - BF > 1: Supports H₁ over H₀ - BF < 1: Supports H₀ over H₁ - BF = 1: Equal support Some statisticians prefer Bayes Factors for being more interpretable.