Two-sample procedures compare the means of two independent populations or treatment groups. The uncertainty in the estimated difference comes from both samples, and each sample contributes according to its variability and size.
Define the difference before calculating
Let be the population mean difference, estimated by State which group is first and what the measured quantity means. Reversing group order reverses the sign but not the strength of a two-sided comparison.
For independent samples, the estimated standard error is
Variances add for a difference of independent estimates. Standard deviations themselves do not add or subtract directly. Both sample sizes matter; using only the larger sample's size understates uncertainty from the smaller group.
Check independence and distribution shape
The groups should come from independent random samples or an appropriate randomized experiment. Within each sample, observations should be independent, with the usual 10% condition for sampling without replacement when applicable.
For small samples, examine each population's approximate normality and the samples' outliers or strong skewness. Larger samples improve robustness, but severe problems can still matter. The two population standard deviations need not be equal for the usual Welch two-sample procedure.
If observations are naturally paired, such as the same subjects before and after treatment, use a paired procedure on differences instead. Independent-sample formulas discard the matching and use the wrong uncertainty model for paired data.
Example: construct an interval
Suppose independent samples have and The estimated mean difference is 7 points, and
Welch's degrees of freedom are approximately 36.9. Using a 95% critical value near 2.026 gives or approximately points.
We are 95% confident that population 1's mean exceeds population 2's mean by between about 1.3 and 12.7 points, subject to the design conditions. The interval estimates a difference of means, not the range of individual score differences.
Degrees of freedom reflect both variance estimates
Software commonly uses the Welch-Satterthwaite approximation, which depends on both contributions and sample sizes. Its degrees of freedom can be noninteger. In a table-based introductory calculation, using is a conservative alternative if that method is specified or permitted.
Do not automatically use That belongs to a pooled-variance procedure with an additional equal-population-variance assumption. Welch's method avoids requiring that assumption.
Example: test equality of means
For versus a two-sided alternative, the statistic for the example is With Welch degrees of freedom near 36.9, the two-sided P-value is about 0.018.
At the 5% level, reject equality. This agrees with the 95% interval excluding zero when the matching procedure is used. If the alternative were prespecified as group 1 having a greater mean, the corresponding one-sided P-value would be about half the two-sided value because the observed direction matches that alternative.
The general test numerator subtracts the hypothesized difference: Equality tests use but the formula can assess another stated difference with the appropriate null.
Interpret what the design permits
Random assignment supports a causal treatment comparison; random sampling supports population generalization. Observational groups can differ in confounding characteristics, so a significant mean difference alone does not establish a causal effect.
Also consider practical importance. The interval's width and effect-size units show whether the data distinguish a meaningful difference, rather than merely whether the P-value crosses a threshold.
Try it yourself
Two independent sample means differ by 4. Their variance contributions and are 9 and 7. Find the standard error and equality-test statistic.
Answer: so A P-value still requires degrees of freedom and the alternative's direction.
What to remember
Use independent groups, add variance contributions, and use an appropriate reference distribution. Keep group order, paired-versus-independent design, and causal interpretation explicit.