Power Analysis for Linear Regression: Single Coefficient (T-Test)

Calculates power, sample size or effect size (only one can be NULL at a time) to test a single coefficient in multiple linear regression. By default, the predictor is assumed to be continuous. However, one can calculate power, sample size or effect size for a binary predictor (such as treatment and control groups in an experimental design) by specifying sd.predictor = sqrt(p * (1 - p)) where p is the proportion of subjects in one of the groups. The sample size in each group would be n * p and n * (1 - p).

Minimal effect and equivalence tests are implemented in line with Hodges and Lehmann (1954), Kim and Robinson (2019), Phillips (1990), and Dupont and Plummer (1998).

Formulas are validated using Monte Carlo simulation, G*Power, tables in the PASS documentation, and tables in Bulus (2021).

Usage

power.t.regression(
  beta = NULL,
  null.beta = 0,
  margin = 0,
  req.sign = "+",
  sd.predictor = 1,
  sd.outcome = 1,
  r.squared = NULL,
  k.total = 1,
  n = NULL,
  power = NULL,
  alpha = 0.05,
  alternative = c("two.sided", "one.sided", "two.one.sided"),
  ceil.n = TRUE,
  verbose = 1,
  utf = FALSE
)

Arguments

beta: regression coefficient. One can use standardized regression coefficient, but should keep sd.predictor = 1 and sd.outcome = 1 or leave them out as they are default specifications.
null.beta: regression coefficient under null hypothesis (typically zero). One can use standardized regression coefficient, but should keep sd.predictor = 1 and sd.outcome = 1 or leave them out as they are default specifications.
margin: margin - ignorable beta - null.beta difference.
req.sign: whether estimated beta is smaller or larger than the margin (when minimum detectable beta is of interest).
sd.predictor: standard deviation of the predictor. For a binary predictor, sd.predictor = sqrt(p * (1 - p)) where p is the proportion of subjects in one of the groups.
sd.outcome: standard deviation of the outcome.
r.squared: model R-squared. Please see also Details below.
k.total: integer; total number of predictors, including the predictor of interest.
n: integer; sample size.
power: statistical power, defined as the probability of correctly rejecting a false null hypothesis, denoted as \(1 - \beta\).
alpha: type 1 error rate, defined as the probability of incorrectly rejecting a true null hypothesis, denoted as \(\alpha\).
alternative: character; the direction or type of the hypothesis test: "two.sided", "one.sided", or "two.one.sided".
ceil.n: logical; whether sample size should be rounded up. TRUE by default.
verbose: 1 by default (returns test, hypotheses, and results), if 2 a more detailed output is given (plus key parameters and definitions), if 0 no output is printed on the console.
utf: logical; whether the output should show Unicode characters (if encoding allows for it). FALSE by default.

Value

parms: list of parameters used in calculation.
test: type of the statistical test (T-Test).
df: degrees of freedom.
ncp: non-centrality parameter for the alternative.
null.ncp: non-centrality parameter for the null.
t.alpha: critical value(s).
r.squared: model R-squared.
power: statistical power \((1-\beta)\).
n: sample size.

Details

When it is requested to calculate the effect size (by giving both their parameters n and power), r.squared must be empty and beta can be empty (NULL). By default, a linear regression with one predictor is assumed. If beta, sd.predictor and sd.outcome are given (i.e., in cases where it is not requested to calculate the effect size), r.squared is calculated as follows: r.squared = (beta * sd.predictor / sd.outcome) ^ 2. If the given beta results in an r.squared below this value, a warning will be issued. To calculate beta from r.squared, the following formula can be used: beta = (sqrt(r.squared) * sd.outcome / sd.predictor).
To consider other covariates in the model provide a value greater than the default value for r.squared (see above) along with the argument k.total > 1. However, in such case, the above formula for calculating beta can not be used.
power.t.regression(), pwrss.t.regression(), power.t.reg() and pwrss.t.reg() are the same functions.
NB: The pwrss.z.regression() function and its alias pwrss.z.reg() are deprecated, but they will remain available as a wrapper for the power.t.regression() function during a transition period.

References

Bulus, M. (2021). Sample size determination and optimal design of randomized / non-equivalent pretest-post-test control-group designs. Adiyaman University Journal of Educational Sciences, 11(1), 48-69. https://doi.org/10.17984/adyuebd.941434

Hodges Jr, J. L., & Lehmann, E. L. (1954). Testing the approximate validity of statistical hypotheses. Journal of the Royal Statistical Society Series B: Statistical Methodology, 16(2), 261-268. https://doi.org/10.1111/j.2517-6161.1954.tb00169.x

Kim, J. H., & Robinson, A. P. (2019). Interval-based hypothesis testing and its applications to economics and finance. Econometrics, 7(2), 21. https://doi.org/10.1111/10.3390/econometrics7020021

Phillips, K. F. (1990). Power of the two one-sided tests procedure in bioequivalence. Journal of Pharmacokinetics and Biopharmaceutics, 18(2), 137-144. https://doi.org/10.1007/bf01063556

Dupont, W. D., and Plummer, W. D. (1998). Power and sample size calculations for studies involving linear regression. Controlled Clinical Trials, 19(6), 589-601. https://doi.org/10.1007/10.1016/s0197-2456(98)00037-3

Examples


# continuous predictor x (and 4 covariates)
power.t.regression(beta = 0.20,
            k.total = 5,
            r.squared = 0.30,
            power = 0.80)
#> +--------------------------------------------------+
#> |             SAMPLE SIZE CALCULATION              |
#> +--------------------------------------------------+
#> 
#> Linear Regression Coefficient (T-Test)
#> 
#> ----------------------------------------------------
#> Hypotheses
#> ----------------------------------------------------
#>   H0 (Null)        : beta - null.beta  = 0
#>   H1 (Alternative) : beta - null.beta != 0
#> 
#> ----------------------------------------------------
#> Results
#> ----------------------------------------------------
#>   Target Effect (Std. beta) = 0.200
#>   Sample Size               = 140  <<
#>   Type 1 Error (alpha)      = 0.050
#>   Type 2 Error (beta)       = 0.198
#>   Statistical Power         = 0.802
#> 

# binary predictor x (and 4 covariates)
p <- 0.50 # proportion of subjects in one group
power.t.regression(beta = 0.20,
            sd.predictor = sqrt(p * (1 - p)),
            k.total = 5,
            r.squared = 0.30,
            power = 0.80)
#> +--------------------------------------------------+
#> |             SAMPLE SIZE CALCULATION              |
#> +--------------------------------------------------+
#> 
#> Linear Regression Coefficient (T-Test)
#> 
#> ----------------------------------------------------
#> Hypotheses
#> ----------------------------------------------------
#>   H0 (Null)        : beta - null.beta  = 0
#>   H1 (Alternative) : beta - null.beta != 0
#> 
#> ----------------------------------------------------
#> Results
#> ----------------------------------------------------
#>   Target Effect (Std. beta) = 0.100
#>   Sample Size               = 552  <<
#>   Type 1 Error (alpha)      = 0.050
#>   Type 2 Error (beta)       = 0.200
#>   Statistical Power         = 0.800
#> 

# non-inferiority test with binary predictor x (and 4 covariates)
p <- 0.50 # proportion of subjects in one group
power.t.regression(beta = 0.20, # Cohen's d
            margin = -0.05, # non-inferiority margin in Cohen's d unit
            alternative = "one.sided",
            sd.predictor = sqrt(p*(1-p)),
            k.total = 5,
            r.squared = 0.30,
            power = 0.80)
#> +--------------------------------------------------+
#> |             SAMPLE SIZE CALCULATION              |
#> +--------------------------------------------------+
#> 
#> Linear Regression Coefficient (T-Test)
#> 
#> ----------------------------------------------------
#> Hypotheses
#> ----------------------------------------------------
#>   H0 (Null)        : beta - null.beta <= margin
#>   H1 (Alternative) : beta - null.beta  > margin
#> 
#> ----------------------------------------------------
#> Results
#> ----------------------------------------------------
#>   Target Effect (Std. beta) = 0.100
#>   Sample Size               = 278  <<
#>   Type 1 Error (alpha)      = 0.050
#>   Type 2 Error (beta)       = 0.200
#>   Statistical Power         = 0.800
#> 

# superiority test with binary predictor x (and 4 covariates)
p <- 0.50 # proportion of subjects in one group
power.t.regression(beta = 0.20, # Cohen's d
            margin = 0.05, # superiority margin in Cohen's d unit
            alternative = "one.sided",
            sd.predictor = sqrt(p * (1 - p)),
            k.total = 5,
            r.squared = 0.30,
            power = 0.80)
#> +--------------------------------------------------+
#> |             SAMPLE SIZE CALCULATION              |
#> +--------------------------------------------------+
#> 
#> Linear Regression Coefficient (T-Test)
#> 
#> ----------------------------------------------------
#> Hypotheses
#> ----------------------------------------------------
#>   H0 (Null)        : beta - null.beta <= margin
#>   H1 (Alternative) : beta - null.beta  > margin
#> 
#> ----------------------------------------------------
#> Results
#> ----------------------------------------------------
#>   Target Effect (Std. beta) = 0.100
#>   Sample Size               = 773  <<
#>   Type 1 Error (alpha)      = 0.050
#>   Type 2 Error (beta)       = 0.200
#>   Statistical Power         = 0.800
#> 

# equivalence test with binary predictor x (and 4 covariates)
p <- 0.50 # proportion of subjects in one group
power.t.regression(beta = 0, # Cohen's d
            margin = c(-0.05, 0.05), # equivalence bounds in Cohen's d unit
            alternative = "two.one.sided",
            sd.predictor = sqrt(p * (1 - p)),
            k.total = 5,
            r.squared = 0.30,
            power = 0.80)
#> +--------------------------------------------------+
#> |             SAMPLE SIZE CALCULATION              |
#> +--------------------------------------------------+
#> 
#> Linear Regression Coefficient (T-Test)
#> 
#> ----------------------------------------------------
#> Hypotheses
#> ----------------------------------------------------
#>   H0 (Null)        : beta - null.beta <= min(margin) or
#>                      beta - null.beta >= max(margin)
#>   H1 (Alternative) : beta - null.beta  > min(margin) and
#>                      beta - null.beta  < max(margin)
#> 
#> ----------------------------------------------------
#> Results
#> ----------------------------------------------------
#>   Target Effect (Std. beta) = 0
#>   Sample Size               = 9593  <<
#>   Type 1 Error (alpha)      = 0.050
#>   Type 2 Error (beta)       = 0.200
#>   Statistical Power         = 0.800
#>