An experiment deliberately assigns treatments to study their effects. A strong design makes the treatment groups comparable, measures outcomes consistently, and uses enough independent experimental units to distinguish a treatment effect from ordinary variation.
Identify the parts of the experiment
Experimental units are the objects receiving treatments; when they are people, they are often called subjects. A factor is an explanatory variable manipulated by the researcher, and its levels are the settings used. A treatment is a specific combination of factor levels.
If a plant study uses two fertilizers and three watering amounts, there are six treatment combinations. The response variable might be plant height after four weeks. Define the response and measurement time before analyzing results, rather than choosing whichever outcome looks most favorable afterward.
Control, randomization, and replication
Control means keeping other conditions comparable and providing a meaningful comparison treatment. A control group might receive no treatment, a placebo, or the current standard treatment. It is not always an untreated group.
Random assignment uses chance to allocate experimental units to treatments. It tends to balance known and unknown confounding variables across groups, supporting a cause-and-effect conclusion when the experiment is well conducted.
Replication means applying each treatment to enough independent experimental units. Repeatedly measuring one plant does not provide the same replication as treating many plants. More measurements of one unit can improve measurement precision without creating more independent units.
Example: a completely randomized design
A researcher compares two study methods using 80 students. Label the students 1 through 80, randomly select 40 distinct labels for method A, and assign the remaining 40 to method B. Use the same study time, test, testing conditions, and scoring procedure.
Compare the groups' test-score distributions or mean scores. Random assignment makes preexisting differences less likely to systematically favor one method. It does not guarantee perfectly equal groups in this particular randomization, but it provides the basis for assessing chance variation.
If the students volunteered from one school, the experiment can support a causal conclusion for the studied setting, but broad generalization to all students is limited. Random assignment and random sampling solve different problems: causation and population representativeness.
Blocking accounts for an important source of variation
A block is a group of similar units formed before random assignment using a characteristic expected to affect the response. Randomize treatments separately within each block, then compare treatments while accounting for blocks.
For the study-method experiment, prior achievement could define blocks. Students with similar initial achievement are compared under different methods, reducing unexplained variation. Blocking does not replace randomization; it organizes where randomization occurs.
Do not form blocks using the response measured after treatment. A blocking variable must be available before treatment assignment and should not itself be caused by the treatment.
Example: matched pairs
To compare two running shoes, each runner can test both shoes under comparable conditions, with the order randomized. Each runner forms a block of two measurements, and analysis uses the within-runner difference.
Randomizing order helps reduce learning, fatigue, and weather-order effects. A washout or rest period may be needed if one treatment's effect carries into the next trial. Alternatively, pair two similar runners and randomly assign opposite treatments within each pair.
The paired measurements are dependent, so they should not be analyzed as two independent samples. Pairing is valuable because each difference removes some person-to-person variation.
Blinding reduces expectations and measurement influence
Subjects who do not know their treatment are less likely to change behavior because of expectations. Evaluators who do not know assignments are less likely to score outcomes differently. A double-blind experiment conceals assignments from both relevant groups.
A placebo helps separate the treatment's active effect from expectations associated with receiving treatment. Blinding is not always practical, but standardized objective measurement can still reduce bias. Ethical consent and appropriate comparison treatments remain part of a responsible design.
Try it yourself
A gardener assigns fertilizer A to all sunny plots and fertilizer B to all shaded plots. Why can the resulting growth difference not be attributed clearly to fertilizer? Propose a repair.
Answer: Sunlight is confounded with fertilizer: its effect cannot be separated from the treatment effect. Form sunny and shaded blocks, then randomly assign both fertilizers within each block, with multiple independent plots for each treatment.
What to remember
Randomize within a controlled, replicated design; block important preexisting differences; use paired analysis for matched pairs; and distinguish causal evidence from permission to generalize to a wider population.