Multiple Regression: 22 Solved Exercises and Formulas

Data and methods: The exercises correspond to Problems 7.23–7.44 in the photographed pages 172–180. The demand dataset is explicitly hypothetical; the other small tables serve as instructional examples, not representative population evidence. All new charts and numerical tables below use those supplied values. Calculations retain full precision until the final display, and original explanations replace the printed solution prose.

Multiple regression formulas and notation

Start with a model that contains an intercept and two predictors. Here, Y represents the outcome, X₁ and X₂ represent the predictors, and u represents the population error. Meanwhile, a hat identifies a fitted value or an estimated coefficient.

A slope measures the change in the fitted outcome for a one-unit change in its predictor while the other included predictors stay fixed. Consequently, b₁ is a conditional association, rather than the slope of the simple regression of Y on X₁. Whether that association supports a causal interpretation depends on the research design and identification assumptions.

The word “linear” describes the coefficients, not necessarily the shape of the relationship with every predictor. For instance, a model containing X and X² remains linear in its coefficients. See the NIST explanation of linear least squares for this distinction.

Notation warning: The photographed exercises do not use k consistently across all displayed formulas. Therefore, this article uses q for the number of slopes and p = q + 1 for the number of coefficients. In Exercises 14–17, the supplied k explicitly counts slopes, so k = q.

The formula sheet

These identities let you cross-check the same model from several directions. For example, the residual sum of squares should agree with the value implied by R². Similarly, the square of a slope’s classical t statistic equals the partial F statistic for adding that single predictor.

Partial correlation: two solved exercises

Exercise 1, Problem 7.23: remove the linear influence of a control

Suppose you want the association between Y and X₁ after accounting for X₂. First, fit Y on an intercept and X₂, then keep the residuals eᵧ. Next, fit X₁ on an intercept and X₂, then keep the residuals e₁. Finally, calculate the ordinary Pearson correlation between eᵧ and e₁.

This procedure removes the fitted linear component associated with X₂ from both variables. As a result, the remaining correlation describes their residual linear association. It does not remove every possible nonlinear relationship or every unmeasured source of confounding.

If a control predicts X₁ perfectly, residual X₁ has no variation. Consequently, the partial correlation becomes undefined, rather than zero. This edge case also signals that the full regression cannot identify separate coefficients for perfectly collinear predictors.

Exercise 2, Problem 7.24: calculate two partial correlations

Use the centered cross-products visible in the page: Σx₁y = −28, Σx₂y = 38, Σx₁x₂ = −12, Σx₁² = 60, Σx₂² = 74, and Σy² = 40. First, convert them into the three ordinary correlations.

Therefore, X₂ has the larger absolute partial correlation: 0.7378 versus 0.6332. The substantive ordering agrees with the printed conclusion. However, the photographed answer of approximately 0.8072 for the second partial correlation does not follow from the displayed inputs; full-precision calculation gives 0.7378.

Squaring the partial correlations gives approximately 0.4010 and 0.5444. In each case, the denominator represents outcome variation left after the other predictor, so these percentages describe different remaining variation. Consequently, you should not add them and call the result the model’s R².

Matrix OLS: four solved exercises

Exercise 3, Problem 7.25: why use matrix notation?

Matrix notation collects all observations and coefficients into a single system. Instead of writing one long equation for every observation, write Y = Xβ + u. Moreover, the same notation works whether the model contains two predictors or twenty.

The main advantage is consistency: estimation, prediction, and covariance calculations share the same framework. However, readers must understand transposes, matrix products, rank, and inverses. For practical computation, the mathematical inverse formula also needs a numerically stable implementation.

Exercise 4, Problem 7.26: derive the OLS estimator

OLS chooses b to minimize the sum of squared residuals. First, express that objective as a quadratic form. Then expand the expression and differentiate with respect to the coefficient vector.

The middle expression gives the normal equations. Because XᵀX is positive definite when X has full column rank, this stationary point gives the unique minimum. Otherwise, fitted values can remain unique even though separate coefficient vectors do not.

In software, avoid treating explicit inversion as the default computational strategy. For example, statsmodels OLS.fit supports a pseudoinverse and QR factorization. These methods solve the least-squares problem without requiring the user to form the inverse manually.

Exercise 5, Problem 7.27: construct X and Y

The supplied matrix example contains 15 observations and two predictor columns. Therefore, X has dimensions 15 × 3, while Y has dimensions 15 × 1. The first column below is the intercept column, so every entry equals one.

Keep the outcome in exactly the same observation order as the design matrix. Otherwise, the products remain mathematically valid but describe the wrong pairings. A row count check alone cannot detect that error.

Exercise 6, Problem 7.28: calculate XᵀX and its inverse

Notice that the top-left entry of XᵀX equals 15, the number of observations. Meanwhile, the first row contains the sums of the intercept and both predictors. The remaining diagonal elements contain sums of squared predictor values, not centered sums of squares.

As a numerical check, multiply the unrounded XᵀX by its unrounded inverse. The product equals the identity matrix within floating-point tolerance. However, multiplying rounded inverse entries can produce visible discrepancies because the cross-products include large numbers.

Exercise 7, Problem 7.29: a complete demand regression

Our first full application relates quantity demanded, Y, to price, X₁, and consumers’ income, X₂. The supplied table labels this dataset hypothetical and dates the observations from 1985 through 1999. Therefore, its estimates illustrate a method; they do not measure an actual market or current consumer behavior.

Step 1: center the data and calculate cross-products

The sample means are Ȳ = 70, X̄₁ = 6, and X̄₂ = 1,100. First, subtract each mean from its observations. Then calculate the following sums using the centered variables y, x₁, and x₂.

Holding income fixed, a one-unit price increase corresponds to about 5.106 fewer units of fitted quantity. Meanwhile, a 100-unit income increase corresponds to about 1.669 additional units, holding price fixed. The intercept describes a zero-price, zero-income combination far outside these observations, so it has little practical meaning here.

Step 2: test coefficients and summarize fit

Both slope p-values fall below 0.05, so the classical two-sided tests reject their individual zero-slope null hypotheses. Moreover, the overall F test rejects the joint null that both slopes equal zero. These conclusions rely on the model assumptions behind the classical standard errors.

Thus, the fitted model accounts for about 95.06% of the sample’s centered quantity variation. This percentage describes in-sample fit, not the proportion of demand that price and income causally determine. Also, an excellent historical fit does not establish forecasting accuracy.

Step 3: calculate conditional associations and elasticities

Around the sample means, the fitted linear equation implies that a 1% price increase corresponds to approximately a 0.438% quantity decrease, holding income fixed. Similarly, a 1% income increase corresponds to approximately a 0.262% quantity increase. Because this model uses levels, these elasticities change when you evaluate them at another price-income combination.

Price has the larger absolute partial correlation in this fitted model. However, that comparison does not establish that pricing interventions have greater causal importance than income changes. A real demand study must address simultaneous price and quantity determination, omitted demand shifters, and the time structure of observations.

Step 4: understand the rounding discrepancy

The printed solution reports an intercept near 82.2666, which results from substituting the rounded income slope 0.0167 into the intercept formula. However, the unrounded slope gives 82.2755. The photographed R² of 0.9508 also differs from the full-precision value of 0.950644, which rounds to 0.9506 at four decimal places.

A small R² difference then propagates into SSE, standard errors, and the F statistic. For this reason, carry full precision through intermediate calculations and round only the final report. The table in the correction audit separates these numerical differences from the unchanged significance conclusions.

Multiple regression inference: Exercises 8–21

Next, fourteen exercises share one ten-observation dataset or develop related formulas. Its variables have no substantive labels in the supplied table, so the interpretations below use generic units. Therefore, no industry, currency, or real-world meaning is invented.

Exercise 8, Problem 7.30: estimate the equation

First, calculate the means: Ȳ = 44, X̄₁ = 5, and X̄₂ = 6. The centered cross-products are Σx₁² = 36, Σx₂² = 6, Σx₁x₂ = 11, Σx₁y = 214, and Σx₂y = 71. Therefore, the two-slope denominator is 36 × 6 − 11² = 95.

The source’s 4.76 intercept reflects rounded intermediate coefficients. In contrast, rounding the full-precision intercept to two decimals gives 4.77. This difference is small, but using a consistent precision policy makes the later calculations reproducible.

Exercise 9, Problem 7.31: interpret all three coefficients

The intercept is the fitted outcome when X₁ and X₂ both equal zero. However, the observed predictors never reach that combination, so the intercept mainly anchors the fitted plane. It should not become a confident prediction for an unobserved setting.

Holding X₂ fixed, a one-unit increase in X₁ corresponds to a 5.2947-unit increase in fitted Y. Meanwhile, holding X₁ fixed, a one-unit increase in X₂ corresponds to a 2.1263-unit increase. Comparing these raw slopes as measures of importance would ignore predictor scales and uncertainty.

Exercise 10, Problem 7.32: calculate variances and standard errors

There are ten observations and three fitted coefficients, so the residual degrees of freedom equal seven. First, square and sum the full-precision residuals to obtain SSE = 363.9579. Then divide by seven to estimate the residual variance.

These calculations differ from the printed residual variance of 50 and standard errors of about 1.78 and 4.35. The difference follows from the supplied raw table, not simply from the last displayed decimal. Consequently, all subsequent inference in this article uses the recalculated values.

Exercise 11, Problem 7.33: test each slope at 5%

With seven degrees of freedom, the two-sided 5% critical value is about 2.3646. Therefore, X₁ passes the individual significance test, with p = 0.0223. By contrast, X₂ does not, with p = 0.6465.

Failing to reject β₂ = 0 does not prove that its population effect equals zero. Instead, this small sample provides weak evidence for a nonzero conditional slope under the model. The confidence interval makes that uncertainty more visible.

Exercise 12, Problem 7.34: construct 95% confidence intervals

Because the first interval excludes zero, its conclusion agrees with the significant t test. Meanwhile, the second interval includes both substantial negative and positive values. That wide range signals imprecision rather than evidence that every value in the range is equally plausible.

A frequentist 95% confidence procedure covers the fixed true coefficient in 95% of repeated samples under its assumptions. It does not assign a 95% probability to the fixed parameter after the particular interval has been calculated. For reporting, pair each interval with the coefficient’s units and the relevant model assumptions.

Exercise 13, Problem 7.35: calculate R² and adjusted R²

The model explains about 77.92% of the centered sample variation. However, its adjusted R² falls to 71.61% because the formula accounts for the number of fitted slopes. With only ten observations, each additional parameter uses a meaningful share of the available degrees of freedom.

For comparison, the X₁-only model has R² = 0.77191208 and adjusted R² = 0.74340109. Thus, adding X₂ raises ordinary R² by only 0.00723968, or 0.724 percentage points, while lowering adjusted R². Under an in-sample parsimony criterion, that result favors the simpler specification, although substantive reasons can still justify retaining a control.

For an overview of the difference between the two fit measures, see Penn State’s multiple linear regression lesson. The numerical comparisons here come from our recalculation of the supplied ten-row table.

Exercise 14, Problem 7.36: adjusted R² with one predictor

Given R² = 0.60, n = 10, and k = 1 slope, the model has two coefficients including the intercept. Therefore, residual degrees of freedom equal eight.

The photographed answer of 0.60 does not match these inputs. In fact, with at least one slope and R² below one, adjusted R² is strictly below ordinary R². That inequality provides a quick check before any calculator work.

Exercise 15, Problem 7.37: adjusted R² with two predictors

Keep R² = 0.60 and n = 10, but increase the slope count to two. Now, the model estimates three coefficients and leaves seven residual degrees of freedom.

The source prints 0.55, which instead matches the preceding one-slope case. Consequently, the correct comparison is 0.55 with one slope versus approximately 0.4857 with two slopes when ordinary R² stays fixed at 0.60. An additional predictor must improve fit enough to offset this penalty.

Exercise 16, Problem 7.38: what changes when n increases?

Use R² = 0.60, n = 100, and k = 2 slopes. The larger sample leaves 97 residual degrees of freedom, so the complexity penalty becomes much smaller.

Thus, the adjusted value lies much closer to 0.60 than in the ten-observation case. The printed 0.596 does not follow from the two-slope specification. Always identify whether a symbol counts predictors or includes the intercept before reusing a formula.

Exercise 17, Problem 7.39: can adjusted R² be negative?

Take R² = 0.40, n = 10, and k = 5 slopes. Including the intercept, the model estimates six coefficients and leaves only four residual degrees of freedom.

Yes, adjusted R² can be negative, and the correct result here is −0.35. It means SSE divided by its residual degrees of freedom exceeds SST divided by n − 1. Although some informal summaries replace negative values with zero, retaining the negative value preserves useful information about the poor degrees-of-freedom-adjusted fit.

Exercise 18, Problem 7.40: build the ANOVA table

For the ten-observation model, subtract SSE from SST to find explained variation. Then divide each sum of squares by its own degrees of freedom. The exercise appears to cross-reference Problem 7.36, but a full ANOVA calculation requires the fitted model and data from Problem 7.30; those are the values used here.

Here, the explained mean square is 642.0211, while the residual mean square is 51.9940. Therefore, their ratio equals 12.3480. These values replace the printed mean squares of approximately 649 and 50, which generate its different F statistic.

Exercise 19, Problem 7.41: test overall significance at 5% and 1%

The overall null hypothesis sets both slopes to zero: H₀: β₁ = β₂ = 0. By contrast, the alternative says that at least one slope differs from zero. Importantly, rejecting the joint null does not mean that both individual slopes are significant.

Because 12.3480 exceeds both critical values, reject the overall null at both levels. This decision agrees with the photographed conclusion despite the recalculated statistic. Meanwhile, the weak individual evidence for X₂ remains unchanged.

The overall test and a single-predictor test answer different questions. For a broader treatment of full-versus-reduced models, see Penn State’s model evaluation lesson. Our example demonstrates why a significant joint test can coexist with an insignificant coefficient.

Exercise 20, Problem 7.42: compare partial correlations

The first predictor has the stronger conditional linear association with the outcome. Moreover, its squared partial correlation equals about 0.5495, versus approximately 0.0317 for the second predictor. This ordering agrees with the individual t statistics.

As a further check, this identity offers an independent numerical check using the classical OLS t statistic for a zero slope. However, do not substitute a heteroskedasticity-robust t statistic into this geometric identity. Robust inference changes the standard error without changing the residual correlation.

Exercise 21, Problem 7.43: locate the intercept and coefficient variance

With an intercept, the first column of X consists entirely of ones. If the coefficient order is (b₀, b₁, b₂), the estimated variance of b₁ occupies the second row and second column of the coefficient covariance matrix. In zero-based programming notation, that location is [1, 1].

The off-diagonal elements contain coefficient covariances rather than variances. Consequently, a square root should apply to a diagonal variance when you need a coefficient’s standard error. Taking square roots of every matrix entry has no valid interpretation.

Exercise 22, Problem 7.44: children, intentions, and education

The final example contains 12 couples. Here, Y records children had, X₁ records children wanted at marriage, and X₂ records the wife’s years of education. These small instructional observations support the calculations below, but they do not justify population-wide or causal conclusions about family choices.

Estimate the equation and test the slopes

The means are Ȳ = 2.5, X̄₁ = 2, and X̄₂ = 14. Centering the observations gives Σx₁² = 14, Σx₂² = 78, Σx₁x₂ = −22, Σx₁y = 16, and Σx₂y = −42. Therefore, the common slope denominator equals 14 × 78 − (−22)² = 608.

Holding education fixed, one additional child wanted corresponds to about 0.533 more children in the fitted outcome. Meanwhile, one additional year of education corresponds to about 0.388 fewer children, holding stated intentions fixed. These are conditional associations in the table, not estimates of what would happen if a policy changed education.

Both slope tests reject zero at the 5% level. Moreover, the overall F statistic of 51.4636 indicates a strong joint association under the classical assumptions. The model’s R² of 0.9196 and adjusted R² of 0.9017 describe its sample fit.

Compare partial correlations and inspect a practical failure

Education has the larger absolute partial correlation in this sample. However, the same fitted equation predicts −0.1184 children for X₁ = 0 and X₂ = 18, a combination that actually appears twice in the data. That negative fitted value exposes a limitation that R² does not reveal.

A count outcome cannot take a negative observed value, although ordinary linear regression can produce a negative prediction. Therefore, a substantive analysis should consider whether a count model better represents the outcome. The statsmodels generalized linear model documentation includes Poisson models, but choosing one still requires checking its assumptions and the data-generating process.

In the photographed solution, the equation uses an intercept of 6.90, which follows from the rounded slopes 0.53 and −0.39 and the sample means. However, full-precision calculation gives 6.8684, or 6.87 to two decimals. Recalculating uncertainty also gives t values of 3.0292 and −5.2081 rather than the printed 3.12 and −5.57.

A numerical audit that goes beyond a standard answer key

The following comparison makes the verification process transparent. Every “recalculated” entry uses the raw observations or sufficient statistics shown in the uploaded pages. Although some differences reflect premature rounding, several are too large to explain by the final displayed digit alone.

Partial correlation and semipartial correlation use different denominators

Consider adding X₂ to the ten-observation model that already contains X₁. The reduced model has R² = 0.77191208, while the full model has R² = 0.77915176. Therefore, the increase in ordinary R² equals 0.00723968.

The semipartial quantity says that X₂ adds about 0.724 percentage points to explained total sample variation. By contrast, the partial quantity says that X₂ accounts for about 3.174% of the variation left after X₁. Both statements are correct because they answer different questions with different denominators.

For the general definition, see Penn State’s explanation of partial R-squared. The distinction matters whenever a model already explains most of the outcome: a sizable share of the remaining variation can still be a small share of total variation.

Why adjusted R² can rise even when a predictor fails a 5% test

Let the reduced model have d + 1 residual degrees of freedom and the full model have d. Adding one predictor raises adjusted R² exactly when the full model lowers residual mean square. Rearranging that comparison produces a simple threshold.

Thus, an added predictor can improve adjusted R² without passing a conventional two-sided 5% t test. The adjusted-fit threshold is |t| > 1, while a 5% test usually requires a magnitude around two or more. In our ten-observation example, |t₂| = 0.4790 fails both criteria.

Strong predictor correlation changes uncertainty

Within the demand table, price and income have correlation −0.9181. With only two predictors, their common variance inflation factor equals 1/(1 − r₁₂²). Consequently, each VIF is approximately 6.3660.

Under a fixed residual variance and fixed centered predictor sum of squares, that VIF corresponds to a standard-error inflation factor of about 2.523 relative to the orthogonal-predictor benchmark. It does not mean the coefficients are biased simply because the predictors correlate. For context, see Penn State’s discussion of multicollinearity and regression pitfalls.

The distinction between association strength and precision explains why large simple correlations do not guarantee decisive conditional estimates. Moreover, dropping a scientifically necessary control solely to reduce VIF can change the question the model answers. Diagnose the design and purpose before choosing a remedy.

A practical workflow for checking multiple regression

Check the data before interpreting the output

First, confirm the observation count, variable definitions, units, and row order. Next, inspect ranges and repeated records, because a copied row or misplaced decimal can alter a small model considerably. Also, verify whether missing values have changed the sample across nested specifications.

For the demand example, income rises by exactly 100 in every consecutive year. Therefore, its variation also tracks a deterministic time trend in this hypothetical dataset. A real application with this pattern would require care before interpreting the income coefficient separately from unmodeled changes over time.

Inspect residuals and unusual observations

Plot residuals against fitted values to look for curvature or changing spread. Additionally, preserve observation order when data have a time component, and investigate whether a few observations dominate the fit. NIST illustrates how residual patterns can reveal unequal variance in its initial linear fit example.

The 1996 demand observation has a residual near 8.0049, while the 1999 residual is about −6.8966. These discrepancies identify observations worth inspecting, but residual size alone does not determine influence. Leverage and sensitivity checks address a different part of that question.

Match inference to the assumptions

Normal errors are not necessary to calculate OLS coefficients. However, the exact small-sample t and F reference distributions used here require the classical normal-error model conditional on X, including constant variance and independence. Without those conditions, asymptotic or alternative inferential procedures need their own justification.

The least-squares objective also emphasizes large residuals because it squares them. As NIST notes in its least-squares overview, outliers and extrapolation deserve particular attention. Therefore, statistical output should follow data checks rather than replace them.

Distinguish fitted means from individual predictions

Suppose a new predictor vector is x₀, including its initial one for the intercept. The model gives the fitted mean x₀ᵀb. However, uncertainty about a single new outcome includes both coefficient uncertainty and a new error term, so its prediction interval is wider than the corresponding confidence interval for the mean.

At the demand sample means, the fitted quantity is exactly 70 because the model contains an intercept. The mean-response standard error is approximately 1.1231, while the individual prediction standard error is approximately 4.4923. Consequently, the same fitted center can accompany very different intervals depending on the prediction target.

Reproduce all three fitted models in Python

The following code includes the full instructional datasets. It uses a least-squares solver, derives the classical covariance matrix, and checks the residual orthogonality conditions. In addition, it computes partial correlations directly from residualized variables, which independently checks the t-statistic identity.

import numpy as np
from scipy.stats import t as t_dist, f as f_dist

DATA = {
    "demand": (
        [40,45,50,55,60,70,65,65,75,75,80,100,90,95,85],
        [9,8,9,8,7,6,6,8,5,5,5,3,4,3,4],
        list(range(400, 1900, 100))),
    "ten_rows": (
        [20,28,40,45,37,52,54,43,65,56],
        [2,3,5,4,3,5,7,6,7,8],
        [5,6,6,5,5,7,6,6,7,7]),
    "couples": (
        [4,3,0,4,4,3,0,4,3,1,3,1],
        [3,3,0,2,2,3,0,3,2,1,3,2],
        [12,14,18,10,10,14,18,12,15,16,14,15])
}

def fit_model(y, x1, x2):
    y = np.asarray(y, dtype=float)
    X = np.column_stack([np.ones(len(y)), x1, x2])
    n, p = X.shape
    assert n > p and np.linalg.matrix_rank(X) == p
    b = np.linalg.lstsq(X, y, rcond=None)[0]
    fitted = X @ b
    residual = y - fitted
    sse = residual @ residual
    sst = ((y - y.mean()) ** 2).sum()
    df = n - p
    mse = sse / df
    # QR covariance calculation avoids forming the inverse of X.T @ X.
    _, upper = np.linalg.qr(X, mode="reduced")
    inverse_upper = np.linalg.solve(upper, np.eye(p))
    covariance = mse * inverse_upper @ inverse_upper.T
    se = np.sqrt(np.diag(covariance))
    t_value = b / se
    p_value = 2 * t_dist.sf(np.abs(t_value), df)
    critical = t_dist.ppf(0.975, df)
    interval = np.column_stack([b - critical*se, b + critical*se])
    r2 = 1 - sse/sst
    adjusted = 1 - (1-r2)*(n-1)/df
    F = ((sst-sse)/(p-1))/mse
    partial = []
    for j in [1, 2]:
        controls = X[:, [0, 3-j]]
        ry = y - controls @ np.linalg.lstsq(controls, y, rcond=None)[0]
        rx = X[:, j] - controls @ np.linalg.lstsq(
            controls, X[:, j], rcond=None)[0]
        partial.append(np.corrcoef(ry, rx)[0, 1])
    np.testing.assert_allclose(X.T @ residual, 0, atol=1e-5)
    np.testing.assert_allclose(
        partial, t_value[1:]/np.sqrt(t_value[1:]**2 + df), atol=1e-10)
    return dict(b=b, se=se, t=t_value, p=p_value, ci=interval,
                SSE=sse, SST=sst, MSE=mse, R2=r2,
                adjusted_R2=adjusted, F=F,
                F_p=f_dist.sf(F, p-1, df), partial=partial)

for label, data in DATA.items():
    print(label, fit_model(*data))

For a quick spreadsheet check, the same data can also support a regression routine that includes an intercept. However, verify the coefficient order because software packages sometimes display slopes in a different sequence. Matching values by position without checking labels can reverse their interpretations.

Frequently asked questions about multiple regression

Does a high R² prove that the model is good?

No. A high R² describes in-sample fit relative to the mean-only benchmark when the model contains an intercept. For example, the children model has R² near 0.92 but predicts negative children for observed predictor combinations. Therefore, inspect outcome support, uncertainty, residual behavior, and the intended use.

Can adjusted R² exceed ordinary R²?

Under the standard intercept model with at least one slope, n > q + 1, and ordinary R² between zero and one, adjusted R² cannot exceed R². Equality occurs for a perfect fit. Otherwise, the degrees-of-freedom adjustment lowers the value.

Why can an overall F test be significant when one t test is not?

The F test asks whether the slopes are jointly zero, while each t test examines one slope conditional on the others. Consequently, one useful predictor can make the overall test significant even when another predictor contributes little. The ten-row example demonstrates precisely this pattern.

Is partial correlation the same as a regression coefficient?

No. A coefficient has outcome units per predictor unit, while a partial correlation has no units and lies between −1 and +1 when defined. However, their signs match in the corresponding full-rank OLS model, and the classical zero-slope t statistic links them through r = t/√(t² + df).

Should I remove every predictor with p greater than 0.05?

No. A control can matter for the scientific question or for interpreting another coefficient even when its own estimate is imprecise. Conversely, a small p-value does not guarantee useful prediction. Choose the model with its purpose, design, validation, and substantive assumptions in view.

Why do hand calculations differ from software output?

Premature rounding, inconsistent degrees of freedom, different observations, and different covariance estimators can all change the answer. Therefore, first compare the actual dataset and model specification. Next, retain full precision and check SSE, the coefficient order, and whether the software uses classical or robust standard errors.

Can I interpret the income slope as an income elasticity?

Not directly when the model uses income and quantity in levels. Instead, multiply the slope by the income-to-quantity ratio at the evaluation point. In the hypothetical demand example, the mean-point elasticity is approximately 0.2623, even though the income slope is about 0.01669.

What these 22 exercises teach

Multiple regression becomes easier to interpret when every statistic connects to the same fitted model. Start with the coefficients, then inspect their uncertainty, the model’s fit, and the remaining residual variation. Finally, connect the result to the actual question and the limitations of the data.

The numerical audit adds another lesson: a printed answer deserves the same verification as a software result. In these examples, full-precision calculations preserve several broad conclusions while correcting correlations, adjusted fit measures, and inferential statistics. As a result, you can learn the method and recognize when an apparently authoritative number does not follow from its own inputs.

Sources and further reading

The uploaded textbook pages provide the exercise data and problem numbering; the author, edition, and full book title are not identifiable from these excerpts alone. For further study, the following primary educational and software references support the methodological discussion. All computed findings and charts in this article derive from the supplied examples rather than external population datasets.

1. NIST: Linear Least Squares Regression
   https://www.itl.nist.gov/div898/handbook/pmd/section1/pmd141.htm
2. Penn State STAT 501: Multiple Linear Regression
   https://online.stat.psu.edu/stat501/Lesson05
3. Penn State STAT 501: MLR Model Evaluation
   https://online.stat.psu.edu/stat501/Lesson06
4. Penn State STAT 462: Partial R-squared
   https://online.stat.psu.edu/stat462/node/138/
5. Penn State STAT 501: Multicollinearity and Other Regression Pitfalls
   https://online.stat.psu.edu/stat501/Lesson12
6. NIST: Initial Linear Fit and Residual Diagnostics
   https://www.itl.nist.gov/div898/handbook/pmd/section6/pmd623.htm
7. statsmodels: OLS.fit documentation
   https://www.statsmodels.org/stable/generated/statsmodels.regression.linear_model.OLS.fit.html
8. statsmodels: Generalized Linear Models
   https://www.statsmodels.org/stable/glm.html