There are various styles of confidence intervals for different kinds of populations [normal or not] and different parameters.
Examples:
Parametric confidence intervals assume that you know the distribution type of the population.
If you have data from a normal population in which the mean $\mu$ and standard deviation are unknown a 95% confidence interval for $\mu$ is $\bar X \pm t^* \frac{S}{\sqrt{n}},$ where $bar X$ estimates $\mu,$ $S$ estimates $\sigma$ and $t^*$ cuts probability $0.025$ from the upper tail of the [symmetrical] Student t distribution with $\nu = n-1$ degrees of freedom.
set.seed[2021]
x = rnorm[100, 50, 15]
mean[x]; sd[x]
[1] 47.38273
[1] 15.42939
CI = mean[x] + qt[c[.025,.975], 99]*sd[x]/sqrt[100]; CI
[1] 44.32121 50.44426 # CI [55.32, 50,44] includes 50--just barely
In the same circumstances, a 95% CI for $\sigma$ has endpoints $\sqrt{\frac{[n-1]S^2}{U}}$ and $\sqrt{\frac{[n-1]S^2}{L}},$ where $L$ and $U$ cut probabilities $0.025$ from the lower and upper tails, respectively, of the distribution $\mathsf{Chisq}[\nu = n-1].$
CI = sqrt[99*var[x]/qchisq[c[.975,.025], 99]]; CI
[1] 13.54711 17.92394 # CI [13.44, 17.92] includes 15
If you have data from an exponential population with rate $\lambda$ and mean $\theta = 1/\lambda,$ then a 95% CI for $\theta$ has endpoints $\frac{\bar X}{U}$ and $\frac{\bar X}{L},$ where $\bar X = \frac 1n\sum_{i=1}^n X_i$ estimates $\theta$ and where $L$ and $U$ cut probabilities $0.025$ from the lower and upper tails, respectively, of the distribution $\mathsf{Gamma}[\mathrm{shape}=n,\mathrm{rate}=n].$
set.seed[525]
X = rexp[100, 1/50] # 'X' here is distinct from 'x' above
mean[X]
[1] 49.80193
CI = mean[X]/qgamma[c[.975,.025], 100, 100]; CI
[1] 41.31948 61.20881 # CI [41.32, 61.21] includes 50
If you do not know the distribution family of the population [but you do know that the population mean $\mu$ exists], then you may be able to find a useful nonparametric confidence interval for $\mu$ by bootstrapping. One
nonparametric bootstrap 95% CI can be make by taking $B = 3000$ re-samples of the data x. Re-samples are of size $n$ taken with replacement. We find the mean a.re of each resample, to obtain the bootstrap distribution; then the CI has endpoints at quantiles $0.025$ and $0.975$ of the bootstrap distribution. [There are many styles of bootstrap CIs; this is a simple one, but not always the best.]
set.seed[1234] # bootstrap normal sample `x`
a.re = replicate[3000, mean[sample[x, 100, rep=T]]]
CI = quantile[a.re, c[.025, .975]]; CI
2.5% 7.5%
44.45223 50.41978 # CI [44.45, 50.42] includes 50
set.seed[1235] # bootstrap exponential sample 'X'
A.re = replicate[3000, mean[sample[X, 100, rep=T]]]
CI = quantile[A.re, c[.025, .975]]; CI
2.5% 97.5%
39.70519 61.21473 # CI [39.71. 61.21] includes 50
The figure below shows the two bootstrap distributions and corresponding CIs.
Note: All of the 95% CI's shown above happen to include the target parameter value. We know this because we are using fictitious simulated data. In about 5% of cases real data will not produce a CI that covers the parameter; of course we will not know which samples produce CIs that don't happen to cover the parameter value.
Confidence Intervals
Author:
Lisa Sullivan, PhD
Professor of Biostatistics
Boston University School of Public Health
Introduction
As noted in earlier modules a key goal in applied biostatistics is to make inferences about unknown population parameters based on sample statistics. There are two broad areas of statistical inference, estimation and hypothesis testing. Estimation is the process of determining a likely value for a population parameter [e.g., the true population mean or population proportion] based on a random sample. In practice, we select a sample from the target population and use sample statistics [e.g., the sample mean or sample proportion] as estimates of the unknown parameter. The sample should be representative of the population, with participants selected at random from the population. In generating estimates, it is also important to quantify the precision of estimates from different samples.
Learning Objectives
After completing this module, the student will be able to:
- Define point estimate, standard error, confidence level and margin of error
- Compare and contrast standard error and margin of error
- Compute and interpret confidence intervals for means and proportions
- Differentiate independent and matched or paired samples
- Compute confidence intervals for the difference in means and proportions in independent samples and for the mean difference in paired samples
- Identify the appropriate confidence interval formula based on type of outcome variable and number of samples
Parameter Estimation
There are a number of population parameters of potential interest when one is estimating health outcomes [or "endpoints"]. Many of the outcomes we are interested in estimating are either continuous or dichotomous variables, although there are other types which are discussed in a later module. The parameters to be estimated depend not only on whether the endpoint is continuous or dichotomous, but also on the number of groups being studied. Moreover, when two groups are being compared, it is important to establish whether the groups are independent [e.g., men versus women] or dependent [i.e., matched or paired, such as a before and after comparison]. The table below summarizes parameters that may be important to estimate in health-related studies.
Continuous Variable | Dichotomous Variable | |
One Sample | mean | proportion or rate, e.g., prevalence, cumulative incidence, incidence rate |
Two Independent Samples | difference in means | difference in proportions or rates, e.g., risk difference, rate difference, risk ratio, odds ratio, attributable proportion |
Two Dependent, Matched Samples | mean difference |
There are two types of estimates for each population parameter: the point estimate and confidence interval [CI] estimate. For both continuous variables [e.g., population mean] and dichotomous variables [e.g., population proportion] one first computes the point estimate from a sample. Recall that sample means and sample proportions are unbiased estimates of the corresponding population parameters.
For both continuous and dichotomous variables, the confidence interval estimate [CI] is a range of likely values for the population parameter based on:
- the point estimate, e.g., the sample mean
- the investigator's desired level of confidence [most commonly 95%, but any level between 0-100% can be selected]
- and the sampling variability or the standard error of the point estimate.
Strictly speaking a 95% confidence interval means that if we were to take 100 different samples and compute a 95% confidence interval for each sample, then approximately 95 of the 100 confidence intervals will contain the true mean value [μ]. In practice, however, we select one random sample and generate one confidence interval, which may or may not contain the true mean. The observed interval may over- or underestimate μ. Consequently, the 95% CI is the likely range of the true, unknown parameter. The confidence interval does not reflect the variability in the unknown parameter. Rather, it reflects the amount of random error in the sample and provides a range of values that are likely to include the unknown parameter. Another way of thinking about a confidence interval is that it is the range of likely values of the parameter [defined as the point estimate + margin of error] with a specified level of confidence [which is similar to a probability].
Suppose we want to generate a 95% confidence interval estimate for an unknown population mean. This means that there is a 95% probability that the confidence interval will contain the true population mean. Thus, P[ [sample mean] - margin of error < μ < [sample mean] + margin of error] = 0.95.
The Central Limit Theorem introduced in the module on Probability stated that, for large samples, the distribution of the sample means is approximately normally distributed with a mean:
and a standard deviation [also called the standard error]:
For the standard normal distribution, P[-1.96 < Z < 1.96] = 0.95, i.e., there is a 95% probability that a standard normal variable, Z, will fall between -1.96 and 1.96. The Central Limit Theorem states that for large samples:
By substituting the expression on the right side of the equation:
Using algebra, we can rework this inequality such that the mean [μ] is the middle term, as shown below.
then
and finally
This last expression, then, provides the 95% confidence interval for the population mean, and this can also be expressed as:
Thus, the margin of error is 1.96 times the standard error [the standard deviation of the point estimate from the sample], and 1.96 reflects the fact that a 95% confidence level was selected. So, the general form of a confidence interval is:
point estimate + Z SE [point estimate]
where Z is the value from the standard normal distribution for the selected confidence level [e.g., for a 95% confidence level, Z=1.96].
In practice, we often do not know the value of the population standard deviation [σ]. However, if the sample size is large [n > 30], then the sample standard deviations can be used to estimate the population standard deviation.
Table - Z-Scores for Commonly Used Confidence Intervals
Desired Confidence Interval | Z Score |
90% 95% 99% | 1.645 1.96 2.576 |
In the health-related publications a 95% confidence interval is most often used, but this is an arbitrary value, and other confidence levels can be selected. Note that for a given sample, the 99% confidence interval would be wider than the 95% confidence interval, because it allows one to be more confident that the unknown population parameter is contained within the interval.
Confidence Interval Estimates for Smaller Samples
With smaller samples [n< 30] the Central Limit Theorem does not apply, and another distribution called the t distribution must be used. The t distribution is similar to the standard normal distribution but takes a slightly different shape depending on the sample size. In a sense, one could think of the t distribution as a family of distributions for smaller samples. Instead of "Z" values, there are "t" values for confidence intervals which are larger for smaller samples, producing larger margins of error, because small samples are less precise. t values are listed by degrees of freedom [df]. Just as with large samples, the t distribution assumes that the outcome of interest is approximately normally distributed.
A table of t values is shown in the frame below. Note that the table can also be accessed from the "Other Resources" on the right side of the page.
Confidence Intervals for One Sample: Continuous Outcome
Suppose we wish to estimate the mean systolic blood pressure, body mass index, total cholesterol level or white blood cell count in a single target population. We select a sample and compute descriptive statistics including the sample size [n], the sample mean, and the sample standard deviation [s]. The formulas for confidence intervals for the population mean depend on the sample size and are given below.
Confidence Intervals for μ
- For n > 30
Use the Z table for the standard normal distribution.
- For n 30 and n2 > 30] or small. The standard error of the point estimate will incorporate the variability in the outcome of interest in each of the comparison groups. If we assume equal variances between groups, we can pool the information on variability [sample variances] to generate an estimate of the
population variability. Therefore, the standard error [SE] of the difference in sample means is the pooled estimate of the common standard deviation [Sp] [assuming that the variances in the populations are similar] computed as the weighted average of the standard deviations in the samples, i.e.:
and the pooled estimate of the common standard deviation is
Computing the Confidence Interval for a Difference Between Two Means
If the sample sizes are larger, that is both n1 and n2 are greater than 30, then one uses the z-table.
If either sample size is less than 30, then the t-table is used.
- If n1 > 30 and n2 > 30, we can use the z-table:
Use Z table for standard normal distribution
- If n1 < 30 or n2 < 30, use the t-table:\
Use the t-table with degrees of freedom = n1+n2-2
For both large and small samples Sp is the pooled estimate of the common standard deviation [assuming that the variances in the populations are similar] computed as the weighted average of the standard deviations in the samples.
These formulas assume equal variability in the two populations [i.e., the population variances are equal, or σ12= σ22], meaning that the outcome is equally variable in each of the comparison populations. For analysis, we have samples from each of the comparison populations, and if the sample variances are similar, then the assumption about variability in the populations is reasonable. As a guideline, if the ratio of the sample variances, s12/s22 is between 0.5 and 2 [i.e., if one variance is no more than double the other], then the formulas in the table above are appropriate. If not, then alternative formulas must be used to account for the heterogeneity in variances.3,4
Large Sample Example:
The table below summarizes data n=3539 participants attending the 7th examination of the Offspring cohort in the Framingham Heart Study.
Men
Women
Characteristic
N
s
n
s
Systolic Blood Pressure
1,623
128.2
17.5
1,911
126.5
20.1
Diastolic Blood Pressure
1,622
75.6
9.8
1,910
72.6
9.7
Total Serum Cholesterol
1,544
192.4
35.2
1,766
207.1
36.7
Weight
1,612
194.0
33.8
1,894
157.7
34.6
Height
1,545
68.9
2.7
1,781
63.4
2.5
Body Mass Index
1,545
28.8
4.6
1,781
27.6
5.9
Suppose we want to calculate the difference in mean systolic blood pressures between men and women, and we also want the 95% confidence interval for the difference in means. The sample is large [> 30 for both men and women], so we can use the confidence interval formula with Z. Next, we will check the assumption of equality of population variances. The ratio of the sample variances is 17.52/20.12 = 0.76, which falls between 0.5 and 2, suggesting that the assumption of equality of population variances is reasonable.
First, we need to compute Sp, the pooled estimate of the common standard deviation.
Substituting we get
which simplifies to
Notice that for this example Sp, the pooled estimate of the common standard deviation, is 19, and this falls in between the standard deviations in the comparison groups [i.e., 17.5 and 20.1]. Next we substitute the Z score for 95% confidence, Sp=19, the sample means, and the sample sizes into the equation for the confidence interval.
Substituting
which simplifies to
Therefore, the confidence interval is [0.44, 2.96]
Interpretation: With 95% confidence the difference in mean systolic blood pressures between men and women is between 0.44 and 2.96 units. Our best estimate of the difference, the point estimate, is 1.7 units. The standard error of the difference is 0.641, and the margin of error is 1.26 units. Note that when we generate estimates for a population parameter in a single sample [e.g., the mean [μ]] or population proportion [p]] the resulting confidence interval provides a range of likely values for that parameter. In contrast, when comparing two independent samples in this fashion the confidence interval provides a range of values for the difference. In this example, we estimate that the difference in mean systolic blood pressures is between 0.44 and 2.96 units with men having the higher values. In this example, we arbitrarily designated the men as group 1 and women as group 2. Had we designated the groups the other way [i.e., women as group 1 and men as group 2], the confidence interval would have been -2.96 to -0.44, suggesting that women have lower systolic blood pressures [anywhere from 0.44 to 2.96 units lower than men].
The table below summarizes differences between men and women with respect to the characteristics listed in the first column. The second and third columns show the means and standard deviations for men and women respectively. The fourth column shows the differences between males and females and the 95% confidence intervals for the differences.
Men
Women
Difference
Characteristic
Mean [s]
Mean [s]
95% CI
Systolic Blood Pressure
128.2 [17.5]
126.5 [20.1]
[0.44, 2.96]
Diastolic Blood Pressure
75.6 [9.8]
72.6 [9.7]
[2.38, 3.67]
Total Serum Cholesterol
192.4 [35.2]
207.1 [36.7]
[-17.16, -12.24]
Weight
194.0 [33.8]
157.7 [34.6]
[33.98, 38.53]
Height
68.9 [2.7]
63.4 [2.5]
[5.31, 5.66]
Body Mass Index
28.8 [4.6]
27.6 [5.9]
[0.76, 1.48]
Notice that the 95% confidence interval for the difference in mean total cholesterol levels between men and women is -17.16 to -12.24. Men have lower mean total cholesterol levels than women; anywhere from 12.24 to 17.16 units lower. The men have higher mean values on each of the other characteristics considered [indicated by the positive confidence intervals].
The confidence interval for the difference in means provides an estimate of the absolute difference in means of the outcome variable of interest between the comparison groups. It is often of interest to make a judgment as to whether there is a statistically meaningful difference between comparison groups. This judgment is based on whether the observed difference is beyond what one would expect by chance.
The confidence intervals for the difference in means provide a range of likely values for [μ1-μ2]. It is important to note that all values in the confidence interval are equally likely estimates of the true value of [μ1-μ2]. If there is no difference between the population means, then the difference will be zero [i.e., [μ1-μ2].= 0]. Zero is the null value of the parameter [in this case the difference in means]. If a 95% confidence interval includes the null value, then there is no statistically meaningful or statistically significant difference between the groups. If the confidence interval does not include the null value, then we conclude that there is a statistically significant difference between the groups. For each of the characteristics in the table above there is a statistically significant difference in means between men and women, because none of the confidence intervals include the null value, zero. Note, however, that some of the means are not very different between men and women [e.g., systolic and diastolic blood pressure], yet the 95% confidence intervals do not include zero. This means that there is a small, but statistically meaningful difference in the means. When there are small differences between groups, it may be possible to demonstrate that the differences are statistically significant if the sample size is sufficiently large, as it is in this example.
Small Sample Example:
We previously considered a subsample of n=10 participants attending the 7th examination of the Offspring cohort in the Framingham Heart Study. The following table contains descriptive statistics on the same continuous characteristics in the subsample stratified by sex.
Men
Women
Characteristic
n
Sample Mean
s
n
Sample Mean
s
Systolic Blood Pressure
6
117.5
9.7
4
126.8
12.0
Diastolic Blood Pressure
6
72.5
7.1
4
69.5
8.1
Total Serum Cholesterol
6
193.8
30.2
4
215.0
48.8
Weight
6
196.9
26.9
4
146.0
7.2
Height
6
70.2
1.0
4
62.6
2.3
Body Mass Index
6
28.0
3.6
4
26.2
2.0
Suppose we wish to construct a 95% confidence interval for the difference in mean systolic blood pressures between men and women using these data. We will again arbitrarily designate men group 1 and women group 2. Since the sample sizes are small [i.e., n1< 30 and n2< 30], the confidence interval formula with t is appropriate. However,we will first check whether the assumption of equality of population variances is reasonable. The ratio of the sample variances is 9.72/12.02 = 0.65, which falls between 0.5 and 2, suggesting that the assumption of equality of population variances is reasonable. The solution is shown below.
First, we compute Sp, the pooled estimate of the common standard deviation:
Substituting:
Note that again the pooled estimate of the common standard deviation, Sp, falls in between the standard deviations in the comparison groups [i.e., 9.7 and 12.0]. The degrees of freedom [df] = n1+n2-2 = 6+4-2 = 8. From the t-Table t=2.306. The 95% confidence interval for the difference in mean systolic blood pressures is:
Substituting:
Then simplifying further:
So, the 95% confidence interval for the difference is [-25.07, 6.47]
Interpretation: Our best estimate of the difference, the point estimate, is -9.3 units. The standard error of the difference is 6.84 units and the margin of error is 15.77 units. We are 95% confident that the difference in mean systolic blood pressures between men and women is between -25.07 and 6.47 units. In this sample, the men have lower mean systolic blood pressures than women by 9.3 units. Based on this interval, we also conclude that there is no statistically significant difference in mean systolic blood pressures between men and women, because the 95% confidence interval includes the null value, zero. Again, the confidence interval is a range of likely values for the difference in means. Since the interval contains zero [no difference], we do not have sufficient evidence to conclude that there is a difference. Note also that this 95% confidence interval for the difference in mean blood pressures is much wider here than the one based on the full sample derived in the previous example, because the very small sample size produces a very imprecise estimate of the difference in mean systolic blood pressures.
Confidence Intervals for Matched Samples, Continuous Outcome
The previous section dealt with confidence intervals for the difference in means between two independent groups. There is an alternative study design in which two comparison groups are dependent, matched or paired. Consider the following scenarios:
- A single sample of participants and each participant is measured twice, once before and then after an intervention.
- A single sample of participants and each participant is measured twice under two different experimental conditions [e.g., in a crossover trial].
A goal of these studies might be to compare the mean scores measured before and after the intervention, or to compare the mean scores obtained with the two conditions in a crossover study.
Yet another scenario is one in which matched samples are used. For example, we might be interested in the difference in an outcome between twins or between siblings.
Once again we have two samples, and the goal is to compare the two means. However, the samples are related or dependent. In the first scenario, before and after measurements are taken in the same individual. In the last scenario, measures are taken in pairs of individuals from the same family. When the samples are dependent, we cannot use the techniques in the previous section to compare means. Because the samples are dependent, statistical techniques that account for the dependency must be used. These techniques focus on difference scores [i.e., each individual's difference in measures before and after the intervention, or the difference in measures between twins or sibling pairs].
The Unit of Analysis
This distinction between independent and dependent samples emphasizes the importance of appropriately identifying the unit of analysis, i.e., the independent entities in a study.
- In the one sample and two independent samples applications participants are the units of analysis.
- However, with two dependent samples application,the pair is the unit [and not the number of measurements which is twice the number of units].
The parameter of interest is the mean difference, μd. Again, the first step is to compute descriptive statistics. We compute the sample size [which in this case is the number of distinct participants or distinct pairs], the mean and standard deviation of the difference scores, and we denote these summary statistics as n,
d and sd, respectively. The appropriate formula for the confidence interval for the mean difference depends on the sample size. The formulas are shown in Table 6.5 and are identical to those we presented for estimating the mean of a single sample, except here we focus on difference scores.Computing the Confidence Intervals for μd
- If n > 30
Use Z table for standard normal distribution
- f n < 30
Use t-table with df=n-1
When samples are matched or paired, difference scores are computed for each participant or between members of a matched pair, and "n" is the number of participants or pairs,
is the mean of the difference scores, and Sd is the standard deviation of the difference scoresExample:
In the Framingham Offspring Study, participants attend clinical examinations approximately every four years. Suppose we want to compare systolic blood pressures between examinations [i.e., changes over 4 years]. The data below are systolic blood pressures measured at the sixth and seventh examinations in a subsample of n=15 randomly selected participants. Since the data in the two samples [examination 6 and 7] are matched, we compute difference scores by subtracting the blood pressure measured at examination 7 from that measured at examination 6 or vice versa. [If we subtract the blood pressure measured at examination 6 from that measured at examination 7, then positive differences represent increases over time and negative differences represent decreases over time.]
Subject #
Examination 6
Examination 7
Difference
1
168
141
-27
2
111
119
8
3
139
122
-17
4
127
127
0
5
155
125
-30
6
115
123
8
7
125
113
-12
8
123
106
-17
9
130
131
1
10
137
142
5
11
130
131
1
12
129
135
6
13
112
119
7
14
141
130
-11
15
122
121
-1
Notice that several participants' systolic blood pressures decreased over 4 years [e.g., participant #1's blood pressure decreased by 27 units from 168 to 141], while others increased [e.g., participant #2's blood pressure increased by 8 units from 111 to 119]. We now estimate the mean difference in blood pressures over 4 years. This is similar to a one sample problem with a continuous outcome except that we are now using the difference scores. In this sample, we have n=15, the mean difference score = -5.3 and sd = 12.8, respectively. The calculations are shown below
Subject #
Difference
Difference - Mean Difference
[Difference - Mean Difference]2
1
-27
-21.7
470.89
2
8
13.3
176.89
3
-17
-11.7
136.89
4
0
5.3
28.09
5
-30
-24.7
610.09
6
8
13.3
176.89
7
-12
-6.7
44.89
8
-17
-11.7
136.89
9
1
6.3
39.69
10
5
10.3
106.09
11
1
6.3
39.69
12
6
11.3
127.69
13
7
12.3
151.29
14
-11
-5.7
32.49
15
-1
4.3
18.49
∑ = -79.0
∑ = 0
∑ =2296.95
Therefore,
and
We can now use these descriptive statistics to compute a 95% confidence interval for the mean difference in systolic blood pressures in the population. Because the sample size is small [n=15], we use the formula that employs the t-statistic. The degrees of freedom are df=n-1=14. From the table of t-scores [see Other Resource on the right], t = 2.145. We can now substitute the descriptive statistics on the difference scores and the t value for 95% confidence as follows:
So, the 95% confidence interval for the difference is [-12.4, 1.8].
Interpretation:
We are 95% confident that the mean difference in systolic blood pressures between examinations 6 and 7 [approximately 4 years apart] is between -12.4 and 1.8. The null [or no effect] value of the CI for the mean difference is zero. Therefore, based on the 95% confidence interval we can conclude that there is no statistically significant difference in blood pressures over time, because the confidence interval for the mean difference includes zero.
Crossover Trials
Crossover trials are a special type of randomized trial in which each subject receives both of the two treatments [e.g., an experimental treatment and a control treatment]. Participants are usually randomly assigned to receive their first treatment and then the other treatment. In many cases there is a "wash-out period" between the two treatments. Outcomes are measured after each treatment in each participant. [An example of a crossover trial with a wash-out period can be seen in a study by Pincus et al. in which the investigators compared responses to analgesics in patients with osteoarthritis of the knee or hip.] A major advantage to the crossover trial is that each participant acts as his or her own control, and, therefore, fewer participants are generally required to demonstrate an effect. When the outcome is continuous, the assessment of a treatment effect in a crossover trial is performed using the techniques described here.
Example:
A crossover trial is conducted to evaluate the effectiveness of a new drug designed to reduce symptoms of depression in adults over 65 years of age following a stroke. Symptoms of depression are measured on a scale of 0-100 with higher scores indicative of more frequent and severe symptoms of depression. Patients who suffered a stroke were eligible for the trial. The trial was run as a crossover trial in which each patient received both the new drug and a placebo. Patients were blind to the treatment assignment and the order of treatments [e.g., placebo and then new drug or new drug and then placebo] were randomly assigned. After each treatment, depressive symptoms were measured in each patient. The difference in depressive symptoms was measured in each patient by subtracting the depressive symptom score after taking the placebo from the depressive symptom score after taking the new drug. A total of 100 participants completed the trial and the data are summarized below.
n
Mean Difference
Std. Dev. Difference
Depressive Symptoms After New Drug - Symptoms After Placebo
100
-12.7
8.9
The mean difference in the sample is -12.7, meaning on average patients scored 12.7 points lower on the depressive symptoms scale after taking the new drug as compared to placebo [i.e., improved by 12.7 points on average]. What would be the 95% confidence interval for the mean difference in the population? Since the sample size is large, we can use the formula that employs the Z-score.
Substituting the current values we get
So, the 95% confidence interval is [-14.1, -10.7].
Interpretation: We are 95% confident that the mean improvement in depressive symptoms after taking the new drug as compared to placebo is between 10.7 and 14.1 units [or alternatively the depressive symptoms scores are 10.7 to 14.1 units lower after taking the new drug as compared to placebo]. Because we computed the differences by subtracting the scores after taking the placebo from the scores after taking the new drug and because higher scores are indicative of worse or more severe depressive symptoms, negative differences reflect improvement [i.e., lower depressive symptoms scores after taking the new drug as compared to placebo]. Because the 95% confidence interval for the mean difference does not include zero, we can conclude that there is a statistically significant difference [in this case a significant improvement] in depressive symptom scores after taking the new drug as compared to placebo.
Confidence Interval for Two Independent Samples, Dichotomous Outcome
It is common to compare two independent groups with respect to the presence or absence of a dichotomous characteristic or attribute, [e.g., prevalent cardiovascular disease or diabetes, current smoking status, cancer remission, or successful device implant]. When the outcome is dichotomous, the analysis involves comparing the proportions of successes between the two groups. There are several ways of comparing proportions in two independent groups.
- One can compute a risk difference, which is computed by taking the difference in proportions between comparison groups and is similar to the estimate of the difference in means for a continuous outcome.
- The risk ratio [or relative risk] is another useful measure to compare proportions between two independent populations and it is computed by taking the ratio of proportions.
Generally the reference group [e.g., unexposed persons, persons without a risk factor or persons assigned to the control group in a clinical trial setting] is considered in the denominator of the ratio. The risk ratio is a good measure of the strength of an effect, while the risk difference is a better measure of the public health impact, because it compares the difference in absolute risk and, therefore provides an indication of how many people might benefit from an intervention. An odds ratio is the measure of association used in case-control studies. It is the ratio of the odds or disease in those with a risk factor compared to the odds of disease in those without the risk factor. When the outcome of interest is relatively uncommon [e.g., 30, use
and use the z-table for standard normal distributionIf n < 30, use the t-table with degrees of freedom [df]=n-1
- Confidence interval for a proportion from one sample [p] with a dichotomous outcome
- Confidence interval for a risk difference [RD] calculated from two independent samples
- Confidence interval for a risk ratio [RR] or prevalence ratio from two independent samples
RR = p1/p2
Then take exp[lower limit of Ln[RR]] and exp[upper limit of Ln[RR]] to get the lower and upper limits of the confidence interval for RR.
- Confidence interval for an odds ratio [OR]
Then take exp[lower limit of Ln[OR]] and exp[upper limit of Ln[OR]] to get the lower and upper limits of the confidence interval for OR.
Note that this summary table only provides formulas for larger samples. As noted throughout the modules alternative formulas must be used for small samples.
References
- Newcomb RG. Two-sided confidence intervals for the single proportion: Comparison of seven methods. Statistics in Medicine 1998;17[8]: 857-872.
- StatXact version 7© 2006 by Cytel, Inc., Cambridge, MA .
- D'Agostino RB, Sullivan LM and Beiser A: Introductory Applied Biostatistics. Belmont, CA: Duxbury-Brooks/Cole; 2004
- Rosner B. Fundamentals of Biostatistics. Belmont, CA: Duxbury-Brooks/Cole; 2006.
- Agresti A. Categorical Data Analysis 2nd ed., New York: John Wiley & Sons, 2002.
- Rothman KJ and Greenland S. Modern Epidemiology 2nd ed., Philadelphia. Lippincott-Raven Publishers, 1998
Solutions to Selected Problems
Answer to first problems on page 3
What is the 90% confidence interval for BMI? [Note that Z=1.645 to reflect the 90% confidence level.]
So, the 90% confidence interval is [126.77, 127.83]
=======================================================
Answer to BMI Problem on page 3
Question: Using the subsample in the table above, what is the 90% confidence interval for BMI?
Solution: Once again, the sample size was 10, so we go to the t-table and use the row with 10 minus 1 degrees of freedom [so 9 degrees of freedom]. But now you want a 90% confidence interval, so you would use the column with a two-tailed probability of 0.10. Looking down to the row for 9 degrees of freedom, you get a t-value of 1.833.
Once again you will use this equation:
Plugging in the values for this problem we get the following expression:
Therefore the 90% confidence interval ranges from 25.46 to 29.06.
=======================================================
Answer to Problem at Bottom of Page 4
The table below, from the 5th examination of the Framingham Offspring cohort, shows the number of men and women found with or without cardiovascular disease [CVD]. Estimate the prevalence of CVD in men using a 95% confidence interval.
Free of CVD
Prevalent CVD
Total
Men
1,548
244
1,792
Women
1,872
135
2,007
Total
3,420
379
3,799
The prevalence of cardiovascular disease [CVD] among men is 244/1792=0.1362. The sample size is large and satisfies the requirement that the number of successes is greater than 5 and the number of failures is greater than 5. Therefore, the following formula can be used again.
Substituting, we get
So, the 95% confidence interval is [0.120, 0.152].
With 95% confidence the prevalence of cardiovascular disease in men is between 12.0 to 15.2%.
=======================================================
Answer to Problem on Confidence Interval for Risk Difference on Page 7
The point estimate for the difference in proportions is [0.46-0.22]=0.24. Note that the new treatment group is group 1, and the standard treatment group is group 2. Therefore, 24% more patients reported a meaningful reduction in pain with the new drug compared to the standard pain reliever. Since there are more than 5 events [pain relief] and non-events [absence of pain relief] in each group, the large sample formula using the z-score can be used.
Substituting we get
This further simplifies to
So, the 96% confidence interval for this risk difference is [0.06, 0.42].
Interpretation: Our best estimate is an increase of 24% in pain relief with the new treatment, and with 95% confidence, the risk difference is between 6% and 42%. Since the 95% confidence interval does not contain the null value of 0, we can conclude that there is a statistically significant improvement with the new treatment.
=======================================================
Answer to Problem on Confidence Interval for Relative Risk - Page 8
Consider again the randomized trial that evaluated the effectiveness of a newly developed pain reliever for patients following joint replacement surgery. Using the data in the table below, compute the point estimate for the relative risk for achieving pain relief, comparing those receiving the new drug to those receiving the standard pain reliever. Then compute the 95% confidence interval for the relative risk, and interpret your findings in words.
Treatment Group
n
# with Reduction
of 3+ Points
Proportion with Reduction
of 3+ Points
New Pain Reliever
50
23
0.46
Standard Pain Reliever
50
11
0.22
The point estimate for the relative risk is
Patients receiving the new drug are 2.09 times more likely to report a meaningful reduction in pain compared to those receivung the standard pain reliever. The 95% confidence interval estimate can be computed in two steps as follows:
This is the confidence interval for ln[RR]. To compute the upper and lower limits for the confidence interval for RR we must find the antilog using the [exp] function:
Therefore, we are 95% confident that patients receiving the new pain reliever are between 1.14 and 3.82 times as likely to report a meaningful reduction in pain compared to patients receiving tha standard pain reliever.
===========================================
Answer to Odds Ratio Problem on Page 10
We now ask you to use these data to compute the odds of pain relief in each group, the odds ratio for patients receiving new pain reliever as compared to patients receiving standard pain reliever, and the 95% confidence interval for the odds ratio.
Treatment Group
n
# with Reduction
of 3+ Points
Proportion with Reduction
of 3+ Points
New Pain Reliever
50
23
0.46
Standard Pain Reliever
50
11
0.22
It is easier to solve this problem if the information is organized in a contingency table in this way:
Pain Relief 3+
Less Relief
New Drug
23
27
Standard Drug
11
39
Odds of pain relief 3+ with new drug = 23/27 0.8519
Odds of pain relief 3+ with standard drug = 11/39 = 0.2821
Odds Ratio = 0.8519 / 0.2821 = 3.02
To compute the 95% confidence interval for the odds ratio we use
Substituting we get
Since we used the log [Ln], we now need to take the antilog to get the limits of the confidente interval.
The point estimate of the odds ratio is OR=3.2, and we are 95% confident that the true odds ratio lies between 1.27 and 7.21. This is statistically significant because the 95% confidence interval does not include the null value [OR=1.0].
Note also that the odds rato was greater than the risk ratio for the same problem. For mathematical reasons the odds ratio tends to exaggerate associates when the outcome is more common.
Answer to Pain Reliever Problem - Page 8
Consider again the randomized trial that evaluated the effectiveness of a newly developed pain reliever for patients following joint replacement surgery. Using the data in the table below, compute the point estimate for the relative risk for achieving pain relief, comparing those receiving the new drug to those receiving the standard pain reliever. Then compute the 95% confidence interval for the relative risk, and interpret your findings in words.
Treatment Group
n
# with Reduction
of 3+ Points
Proportion with Reduction
of 3+ Points
New Pain Reliever
50
23
0.46
Standard Pain Reliever
50
11
0.22
What assumptions are necessary to use the confidence interval calculations?
Assumptions behind our Confidence Intervals.We assume the standard deviation of the population [σ] is known..The sample was randomly selected [independence assumption]..The sample size is adequately large. ... .There are no/few outliers [extreme high or low values]..What assumptions and conditions must be checked before finding a confidence interval?
There are three conditions we need to satisfy before we make a one-sample z-interval to estimate a population proportion. We need to satisfy the random, normal, and independence conditions for these confidence intervals to be valid.What assumptions must hold true to use the students t distribution to make a confidence interval for the population mean μ?
The common assumptions made when doing a t-test include those regarding the scale of measurement, random sampling, normality of data distribution, adequacy of sample size, and equality of variance in standard deviation.What assumption S must hold true to use the normal distribution to make a confidence interval for the population proportion P?
The confidence interval is based on a normally distributed sampling distribution, and we can only assume that we have that if these assumptions have been met: Random sample - the method for choosing the members of the sample must be randomized. Independent sample - the sample must be independent.