ADVANCED REGRESSION • WORKED METHODS & NUMERICAL AUDIT
Advanced regression helps you forecast uncertain outcomes, estimate binary probabilities, model seasonal differences, and trace effects over time. However, the real skill lies in matching each formula to the question—and checking whether the numbers support the interpretation.
This guide works through 27 exercises, numbered 8.16–8.42 in the supplied textbook pages. It combines original explanations, recalculated examples, readable formulas, and historical data tables. Moreover, it separates results that we can reproduce from answers that rely on earlier tables outside the available pages.
Data provenance: supplied Chapter 8 pages 197–205, including Tables 8.17–8.22. The photographs do not establish a complete title, edition, or author citation. Historical figures below reproduce those classroom tables; they are not current economic statistics. All additional calculations are identified as calculations from these inputs.
Advanced regression: choose the method before the formula
A regression coefficient answers a question within a particular model. Consequently, the same number can represent a level change, an elasticity, a change in log-odds, or an immediate dynamic response. Start by identifying the outcome, the scale of each predictor, and the forecast horizon.
| Question | Useful model | Main output | Common mistake |
|---|---|---|---|
| What might one future outcome equal? | Linear regression | Point forecast and prediction interval | Reporting a mean-response interval |
| How likely is a binary event? | Logit or probit | Probability and marginal effect | Calling the coefficient a probability change |
| Does a relationship vary by group? | Indicators and interactions | Intercept or slope differences | Confusing an interaction with a level shift |
| Does an effect persist? | Distributed lags | Immediate and cumulative effects | Interpreting the impact coefficient as the long-run effect |
| Does the response curve? | Logs, reciprocal terms, polynomials | Elasticity or changing slope | Comparing R² across different response scales |
Notation and units
Throughout the guide, ŷ denotes a fitted value, while y denotes an observed outcome. The symbol ℓ means log-likelihood; Λ denotes the logistic cumulative distribution function, and Φ denotes the standard normal cumulative distribution function. In addition, φ denotes the standard normal density.
For linear regression, k counts all estimated coefficients, including the intercept. Therefore, residual degrees of freedom equal n − k. A predictor entered as 15 means fifteen percentage points when the table uses percentage-point units; substituting 0.15 would change the calculation.
Forecasting and prediction intervals
A point forecast gives one central estimate. By contrast, an interval communicates uncertainty around either the conditional mean or a future individual observation. The distinction matters because a future observation includes its own unpredictable error.
Exercise 8.16: Forecasts, conditional forecasts, and forecast uncertainty
A forecast estimates an outcome using information available at the forecast origin. By contrast, a conditional forecast fixes an assumed path or value for the predictors. For example, forecasting sales under a specified advertising budget produces a conditional forecast even when the future budget has not yet become certain.
Errors come from unpredictable shocks, estimated coefficients, uncertain future predictors, and model misspecification. Moreover, structural changes can make a previously reliable relationship fail. A standard regression prediction interval handles only the uncertainty represented by its assumptions.
The extra 1 inside the prediction formula represents the new observation’s error variance. Consequently, a larger sample can reduce uncertainty about the mean without eliminating individual unpredictability. Exact finite-sample t intervals require the classical normal-error model with independent, constant-variance errors and fixed or appropriately conditioned predictors.
Method reference: Penn State STAT 501: mean-response and new-response intervals.
Exercise 8.17: Recalculate two individual forecasts
The supplied inputs are ŷ = 59.13 − 2.60x, n = 15, x̄ = 11, Sxx = 442, and s² = 220.99. Therefore, the residual degrees of freedom equal 13, giving t0.975,13 ≈ 2.16037.
| xF | Forecast | Prediction SE | 95% prediction interval |
|---|---|---|---|
| 15 | 20.13 | 15.6116 | −13.60 to 53.86 |
| 11.5 | 29.23 | 15.3573 | −3.95 to 62.41 |
The second interval is slightly narrower because 11.5 lies closer to x̄. However, the new observation’s random error dominates both widths. The printed solution uses 2.18, which is not the correct critical value for 13 degrees of freedom; the table above uses the correct quantile with the stated rounded inputs.
A negative lower bound may be unsuitable when the outcome cannot fall below zero. In that case, investigate a model that respects the outcome’s support rather than silently clipping the interval and retaining the same coverage claim.
Exercise 8.18: Explain the shape of the interval bands
Both interval types reach their smallest width at xF = x̄ in simple regression. Moving away from that point increases leverage and therefore increases uncertainty. Nevertheless, the individual prediction interval remains wider than the confidence interval for the mean at every predictor value.
The chart below uses the numerical inputs from Exercise 8.17. Its shaded regions represent pointwise 95% intervals; they do not provide a 95% simultaneous guarantee for the entire curve.
Exercise 8.19: Multiple predictors and the covariance term
For several predictors, use the full estimated coefficient covariance matrix. This approach includes every covariance term and avoids a common expansion error.
The predictor vector includes its intercept entry of 1. In a centered two-predictor model, let d₁ = x₁F − x̄₁ and d₂ = x₂F − x̄₂. Under classical OLS, the centered intercept has zero covariance with the slopes, giving:
The supplied exercise reports d₁ = 17, d₂ = 13, s² = 1.95, Var̂(a) = 2.66, slope variances 0.06 and 0.07, and covariance −0.07. Therefore, the correctly expanded variance from those rounded inputs is:
Using the exercise’s rounded critical value of 2.37 gives 82.48 ± 2.37√2.84, or approximately 78.49 to 86.47. However, covariance rounding strongly affects this result because positive and negative components nearly cancel. The full-precision covariance matrix is necessary for an authoritative interval.
The printed calculation omits the factor 2 and reports variance 18.31. In addition, its stated covariance sign conflicts with its substitution. Accordingly, the reported interval 73.34–92.62 should not serve as a verified result.
Method reference: Penn State STAT 501: matrix-based multiple-regression prediction.
Binary choice models: logit and probit
Binary outcomes take values 0 and 1. Examples include whether a customer purchases, a borrower defaults, or a country receives a particular historical classification. Consequently, the fitted probability must remain between zero and one.
Exercise 8.20: Derive the binary likelihood
Let pᵢ = P(Yᵢ = 1 | xᵢ). Under conditional independence, each observation contributes pᵢ when yᵢ = 1 and 1 − pᵢ when yᵢ = 0. Multiplying those contributions gives the Bernoulli likelihood.
For probit, pᵢ = Φ(xᵢ′b). In logit, pᵢ = Λ(xᵢ′b) = 1/[1 + exp(−xᵢ′b)]. Because both distributions are symmetric, an equivalent compact expression uses qᵢ = 2yᵢ − 1:
Numerical software should evaluate these logarithms directly. Otherwise, extremely small probabilities may round to zero before the logarithm calculation.
Method reference: statsmodels: Logit.loglike and its symmetric formulation.
Exercise 8.21: Compare logit with probit
Both models turn a linear index into a smooth probability curve. However, logit uses a logistic distribution, while probit uses a standard normal distribution. Their latent-variable normalizations differ, so raw coefficients do not share a common scale.
| Feature | Logit | Probit |
|---|---|---|
| Probability | Λ(x′b) | Φ(x′b) |
| Continuous marginal effect | bⱼp(1 − p) | bⱼφ(x′b) |
| Direct coefficient interpretation | Change in log-odds | Change in normalized latent index |
| Error variance normalization | π²/3 | 1 |
| Typical practical comparison | Probabilities and marginal effects | Probabilities and marginal effects |
A rough 1.6 scaling factor often aligns coefficients near the center of their probability curves. Specifically, matching slopes at zero gives φ(0)/Λ′(0) ≈ 0.39894/0.25 ≈ 1.596. Nevertheless, this local approximation does not create a universal conversion rule.
Exercise 8.22: Audit the probit log-likelihood grid
Hold the intercept at −2 and evaluate slopes −0.001, 0, 0.001, 0.002, and 0.003. The highest log-likelihood on this grid occurs at 0.001. However, a grid maximum only identifies the best tested value; it does not establish the unrestricted maximum-likelihood estimate.
| Slope | Stable calculation | Printed table |
|---|---|---|
| −0.001 | −449.4877 | −326.9333 |
| 0 | −38.0620 | −38.0620 |
| 0.001 | −6.8652 | −6.8651 |
| 0.002 | −14.9985 | −14.9985 |
| 0.003 | −36.8061 | −36.8047 |
The first discrepancy has a concrete numerical explanation. Hong Kong and Singapore have y = 1 but highly negative indices when the slope equals −0.001. Their true log-probabilities are approximately −174.43 and −110.96, whereas the table limits each contribution near −69.08, corresponding to ln(10−30).
Replacing just those two contributions with that floor produces approximately −326.93. Therefore, the published total reflects a probability floor rather than the unmodified probit likelihood. Smaller discrepancies elsewhere reflect numerical approximation or rounding.
Exercise 8.23: Estimate and interpret the historical logit model
Fitting the visible 20-country observations gives an intercept of approximately −3.604995 and a slope of 0.001795768. The maximized log-likelihood equals approximately −6.766465. Consequently, the rounded index supplied in the exercise provides a close reproduction:
The coefficient 0.0018 is not a constant increase of 0.18 percentage points in probability. Instead, it changes log-odds per unit of X. For a 100-unit increase, the rounded model multiplies the odds by exp(0.18) ≈ 1.197, although the probability change still depends on the starting value.
Using the textbook’s reported standard errors gives Wald statistics of −3.6050/1.6811 ≈ −2.14 and 0.0018/0.0009 = 2.00. These asymptotic checks narrowly exceed 1.96 in absolute value. However, a sample of only 20 observations calls for caution about precision and influential observations.
Exercise 8.24: Separate coefficients, probabilities, labels, and fit statistics
A fitted probability can equal 0.61, but an observed outcome remains either 0 or 1. Next, a classification rule converts the probability into a label, such as ŷclass = 1 when p ≥ 0.50. Consequently, probability estimation and classification are distinct tasks.
For a Bernoulli variable, E(Y | X) equals P(Y = 1 | X). Therefore, a marginal effect describes a change in both the conditional mean and the success probability. By contrast, an odds ratio describes a multiplicative change in odds.
McFadden’s pseudo-R² compares log-likelihoods; it does not measure the percentage of outcome variance explained. Under an unpenalized maximum-likelihood fit that nests the intercept-only model, it lies between zero and one in the usual nondegenerate case. Nevertheless, poorly optimized or noncomparable fits need not obey that interpretation.
Exercise 8.25: Compute marginal effects and all country probabilities
Using the rounded model, the mean predictor equals 3,469.5 and its fitted probability is about 0.9334. Therefore, the marginal effect at the mean differs substantially from the average of the individual marginal effects.
A local 100-unit increase near the mean corresponds to about 1.12 percentage points under the derivative approximation. In comparison, averaging the local derivatives across observations gives about 1.94 percentage points per 100 units. For an actual finite change, calculate p(X + 100) − p(X) instead of treating the derivative approximation as exact.
The intercept-only model assigns probability 0.5 because the sample contains ten ones and ten zeros. Thus, ℓnull = 20 ln(0.5) ≈ −13.862944, and McFadden R² ≈ 0.5119. The likelihood-ratio statistic is 2(−6.766465 + 13.862944) ≈ 14.1930, which is a different quantity from pseudo-R².
| Country | X | Observed Y | Predicted probability | Class at 0.50 |
|---|---|---|---|---|
| Burundi | 569 | 0 | 0.0704 | 0 |
| Chad | 408 | 0 | 0.0536 | 0 |
| Congo | 2,240 | 0 | 0.6052 | 1 |
| Egypt | 1,869 | 0 | 0.4401 | 0 |
| Hong Kong | 16,471 | 1 | 1.0000 | 1 |
| India | 1,282 | 0 | 0.2146 | 0 |
| Indonesia | 2,102 | 1 | 0.5445 | 1 |
| Ivory Coast | 1,104 | 0 | 0.1655 | 0 |
| Kenya | 914 | 0 | 0.1235 | 0 |
| Malaysia | 5,746 | 1 | 0.9988 | 1 |
| Morocco | 2,173 | 1 | 0.5760 | 1 |
| Nigeria | 978 | 0 | 0.1365 | 0 |
| Rwanda | 762 | 0 | 0.0968 | 0 |
| Singapore | 12,653 | 1 | 1.0000 | 1 |
| South Africa | 3,068 | 1 | 0.8718 | 1 |
| Tunisia | 3,075 | 1 | 0.8732 | 1 |
| Uganda | 547 | 1 | 0.0678 | 0 |
| Uruguay | 5,185 | 1 | 0.9968 | 1 |
| Venezuela | 7,082 | 1 | 0.9999 | 1 |
| Zimbabwe | 1,162 | 0 | 0.1804 | 0 |
| Actual outcome | Predicted 0 | Predicted 1 |
|---|---|---|
| 0 | 9 | 1 |
| 1 | 1 | 9 |
The model correctly classifies 18 of 20 observations, or 90%, in the same sample used for estimation. Congo supplies the false positive, while Uganda supplies the false negative. However, this is training accuracy, not evidence of 90% accuracy on new countries or future periods.
In this balanced sample, sensitivity, specificity, and precision also equal 90%. Furthermore, the majority-class baseline achieves only 50%. A proper assessment would still examine held-out performance, calibration, and the consequences of each error type.
Method reference: statsmodels: marginal effects at the mean, averaged effects, and discrete changes.
Functional forms: transformations and interpretation
A model can curve in X while remaining linear in its unknown coefficients. Consequently, ordinary least squares can estimate many nonlinear-looking relationships after constructing appropriate regressors. The error structure still matters, especially when transforming the dependent variable.
Exercise 8.26: Turn nonlinear-looking functions into regression forms
| Original form | Regressor or transformation | Important condition |
|---|---|---|
| Y = b₀ exp(b₁X + u) | ln Y = ln b₀ + b₁X + u | Y > 0 and b₀ > 0 |
| Y = b₀ + b₁ ln X + u | R = ln X; regress Y on 1 and R | X > 0 |
| Y = b₀ − b₁/X + u | Z = 1/X; regress Y on 1 and Z | X ≠ 0; coefficient on Z is −b₁ |
| Y = b₀ + b₁X − b₂X² + b₃X³ + u | Use X, X², and X³ | Linear in coefficients, cubic in X |
Logging an additive-error equation does not automatically produce the first model. For example, ln[b₀ exp(b₁X) + u] does not simplify to ln b₀ + b₁X + u. Therefore, specify whether the original disturbance is additive or multiplicative before choosing the transformation.
Exercise 8.27: Interpret a double-log relationship
The printed answer gives ln Ŷ = 2.64 + 0.72 ln X, with R² = 83.26% on the log-outcome scale. Its slope is an elasticity: a small 1% increase in X corresponds to approximately a 0.72% increase in the fitted outcome.
Exponentiating the fitted log outcome does not generally recover the conditional arithmetic mean. Under a normal log-error model, a mean forecast needs the appropriate variance correction; alternative error distributions require a suitable retransformation method. Moreover, the earlier raw data in Table 6.12 are absent, so this section interprets the printed fit rather than independently reproducing it.
Method reference: Hyndman and Athanasopoulos: back-transformation and bias adjustment.
Exercise 8.28: Interpret a level-log relationship
The supplied fit is Ŷ = 2.62 + 27.12 ln X, with R² = 81.29%. Because Y remains in levels, a 1% increase in X changes the fitted response by approximately 0.2712 units.
The effect per unit of X equals 27.12/X, so it falls as X rises. Accordingly, this model describes diminishing level changes without requiring a turning point. As in Exercise 8.27, the missing Table 6.12 prevents an independent coefficient refit.
Exercise 8.29: Interpret the polynomial and compare fit fairly
The reported polynomial is Ŷ = −2.25 + 13.67X − 0.77X². Its slope changes with X, rather than remaining equal to 13.67.
The negative squared term creates a maximum at approximately X = 8.88. However, that turning point has practical meaning only if the observed range and the subject matter support it.
The printed R² is 80.75%, compared with 81.29% for the level-log model. If both use identical observations and Y in levels, the level-log model has the higher raw R² and fewer coefficients. By contrast, the double-log R² of 83.26% uses ln Y, so it cannot establish superiority on the original Y scale.
For forecasting, compare predictions on a common scale with a common validation sample. Furthermore, examine residual patterns and whether the model extrapolates plausibly. Table 6.12 and the earlier linear-model results are unavailable, so the full original model ranking cannot be independently verified here.
Dummy variables, interactions, and seasonal effects
An indicator distinguishes groups using values zero and one. By comparison, an interaction multiplies that indicator by another predictor, allowing the slope to differ. Therefore, the coefficient on an indicator and the coefficient on its interaction answer different questions.
Exercise 8.30: Find the peacetime and wartime slopes
The reported model is Ŷ = −2.89 + 0.17X − 0.11XD, where D = 1 in wartime. Substituting the indicator gives:
| Period | Equation | Slope |
|---|---|---|
| Peacetime: D = 0 | Ŷ = −2.89 + 0.17X | 0.17 |
| Wartime: D = 1 | Ŷ = −2.89 + 0.06X | 0.06 |
The slope difference is −0.11. Moreover, the reported interaction t statistic of −7.56 indicates strong evidence of a difference under the model’s conventional test assumptions. The underlying Table 8.2 is not visible, so this conclusion uses the reported regression output.
Exercise 8.31: Allow both the intercept and the slope to change
The expanded reported model is Ŷ = −3.34 + 0.17X + 14.59D − 0.18XD. Therefore, peacetime has intercept −3.34 and slope 0.17, while wartime has intercept 11.25 and slope −0.01.
The reported t statistics for the intercept difference and slope difference are 0.67 and −1.64. At a conventional two-sided 5% level, neither provides sufficient evidence individually. However, this does not prove identical relationships, and a joint test of both terms requires their covariance information.
Centering X at a meaningful value can improve interpretation. In particular, the indicator coefficient would then compare the groups at that value rather than at X = 0. The predicted relationships remain unchanged after a consistent reparameterization.
Method reference: Penn State STAT 501: categorical predictors and interactions.
Exercise 8.32: Model quarterly reserves with a trend
The visible data contain 23 quarters from 1995 Q1 through 2000 Q3, measured in millions of dollars. Let T run from 1 to 23 and omit Q1 from the seasonal indicators. Then estimate R = a + bT + c₂D₂ + c₃D₃ + c₄D₄ + u.
| Year | Q1 | Q2 | Q3 | Q4 |
|---|---|---|---|---|
| 1995 | 57,571 | 57,031 | 57,162 | 57,896 |
| 1996 | 54,878 | 53,742 | 51,045 | 51,174 |
| 1997 | 47,551 | 46,606 | 46,060 | 47,919 |
| 1998 | 45,591 | 45,094 | 44,199 | 45,209 |
| 1999 | 43,229 | 42,331 | 41,314 | 41,655 |
| 2000 | 39,752 | 39,217 | 39,257 | Not supplied |
An OLS refit of these displayed values gives:
The refitted R² is approximately 0.95835. Meanwhile, the trend t statistic is about −20.15, while the Q2, Q3, and Q4 statistics are 0.17, 0.25, and 1.84. Thus, at the exercise’s 10% two-sided level with 18 residual degrees of freedom, Q4 crosses the conventional threshold but Q2 and Q3 do not.
The unrestricted quarter-specific intercepts are 57,957.71, 58,095.79, 58,161.71, and 59,529.56. These values refer to the intercept at T = 0; they are not forecasts for any actual quarter without adding the trend term. In addition, a nonsignificant coefficient remains part of the fitted model until an explicit refit removes it.
The printed answer gives different coefficients, including a trend of −956.30 and R² = 0.98. Those values do not reproduce from the visible table under the stated trend coding. Accordingly, this guide reports the transparent refit and treats the printed discrepancy as unresolved.
Exercise 8.33: Interpret regional intercepts correctly
The reported model uses South Atlantic states as the baseline: Ŷ = 8.16 + 0.56X + 0.88DNE + 2.83DMA. Here, Y measures disposable income in thousands of dollars and X measures the percentage of adults with a college degree.
| Region | Intercept in the reported unrestricted model | Slope on X |
|---|---|---|
| South Atlantic | 8.16 | 0.56 |
| New England | 9.04 | 0.56 |
| Mid-Atlantic | 10.99 | 0.56 |
Holding X constant, the model associates Mid-Atlantic membership with $2,830 higher disposable income than the baseline. Similarly, the fitted New England difference is $880, even though its reported t statistic of 0.79 does not establish significance at 10%.
The printed answer assigns New England the baseline intercept after dismissing its nonsignificant dummy. However, insignificance does not change an estimated coefficient to zero. If you remove that term and refit, the other estimates can also change.
This is an observational regional association. Consequently, the coefficient on educational attainment does not by itself identify the causal income effect of earning a degree.
Distributed lag models: effects that unfold over time
Some economic responses take several periods. For instance, a utility company may adjust its investment gradually after demand rises. A distributed lag model represents that delayed adjustment through current and past predictor values.
Exercise 8.34: Compare unrestricted, Koyck, and Almon lags
An unrestricted q-lag model loses the first q usable observations unless earlier predictor values are available. Moreover, adjacent lagged predictors often correlate strongly, which makes individual lag coefficients imprecise.
The Koyck approach imposes geometric weights βⱼ = βλʲ. This restriction saves parameters but forces a particular response pattern. Under the classical infinite-lag derivation, the transformed disturbance ut − λut−1 correlates with lagged Y, so naive OLS can be inconsistent.
An Almon model instead imposes a polynomial on the lag coefficients. Consequently, it permits more flexible shapes while retaining fewer parameters than a fully unrestricted long lag. However, the polynomial degree, maximum lag, and endpoint restrictions require explicit choices.
Exercise 8.35: Recover Koyck parameters and distinguish printed from refitted results
The printed rounded equation is Ŷt = −1.92 + 0.01Xt + 0.40Yt−1. Conditional on that equation, λ = 0.40 and the original infinite-lag intercept is:
These identities follow from the supplied rounded coefficients. However, an OLS refit of the visible Table 8.21, using 1961–1979 and one lag of Y, produces approximately:
Its R² is about 0.99402. The implied descriptive long-run multiplier is approximately 0.02066, and its mean lag is about 2.73 periods. Therefore, these results do not support silently treating the printed 0.40 lag coefficient as a verified refit of the displayed data.
Neither calculation alone resolves the endogeneity issue in the classical Koyck derivation. Furthermore, trending nominal series can produce a high R² even when a model lacks reliable forecasting or causal interpretation.
Exercise 8.36: Estimate a quadratic Almon lag
For current income and three lagged income terms, impose βⱼ = a₀ + a₁j + a₂j² for j = 0, 1, 2, 3. Construct three transformed predictors:
A refit of the visible 1960–1979 consumption and income table uses 17 observations after the three initial lags. Without extra endpoint restrictions, it gives α ≈ −32.54973, a₀ ≈ 0.762362, a₁ ≈ −1.126135, and a₂ ≈ 0.338141.
| Lag j | Recovered coefficient βj |
|---|---|
| 0 | 0.762362 |
| 1 | −0.025631 |
| 2 | −0.137343 |
| 3 | 0.427228 |
The resulting R² is approximately 0.99933, and the sum of the four lag weights is about 1.02662. However, strong fit in trending nominal levels does not establish a stable long-run economic relationship. The negative middle weights also deserve substantive and statistical examination.
A quadratic Almon model also provides a useful internal consistency check on its four recovered weights. Specifically, quadratic weights require β₃ − 3β₂ + 3β₁ − β₀ = 0. Its displayed values yield 0.04 − 3(0.14) + 3(0.77) − 1.94 = −0.01, which could reflect rounding; nevertheless, its large reported contemporaneous coefficient and low printed R² do not match the refit above.
Only one significant individual lag coefficient would not prove poor overall fit. Instead, evaluate the joint restrictions, residual behavior, lag plausibility, and performance on future observations. Alternative endpoint restrictions or different data could change the estimates, but they must be stated explicitly.
Method reference: Hyndman and Athanasopoulos: time-series regression, distributed lags, and spurious regression.
Further forecasting and binary-choice exercises
The final exercises reinforce the same distinction between substitution, estimation, and interpretation. Some refer to earlier examples that the photographs do not include. Accordingly, the following solutions identify what the available information can support.
Exercise 8.37: Interpret a forecast with incomplete supporting inputs
The printed answer gives forecast variance approximately 1.19, a point forecast of 4.78, and a 95% interval of 4.78 ± 2.43. Therefore, the stated interval endpoints are 2.35 and 7.21.
The corresponding standard error is √1.19 ≈ 1.091. Its implied critical multiplier is about 2.23, which is plausible for a small-sample t interval. However, Problem 6.44 is absent, so the original forecast, variance calculation, and degrees of freedom cannot be independently verified.
Exercise 8.38: Identify a forecast inconsistency before using an interval
The displayed equation is Ŷ = 82.27 − 5.11X₁ + 0.02X₂. At X₁ = 2 and X₂ = 1,250, direct substitution gives:
The page reports forecast variance 468.61, so the reported standard error is approximately 21.65. Using its rounded multiplier 2.18 gives an interval near 49.85 to 144.25.
However, the original regression output and its full covariance matrix are absent. The rounded slope variance for X₂ can dominate the result when multiplied by (1,250 − 1,100)². Consequently, the displayed variance should remain a reported textbook value rather than a verified precision estimate.
A reliable recomputation would use xF′V̂(b)xF and the exact residual degrees of freedom. Moreover, it should distinguish a future individual response from a conditional mean before adding the residual variance.
Exercise 8.39: Evaluate logit likelihood at three slopes
Fix b₀ = −3.6 and use the same 20-country observations. A stable calculation evaluates the negative sum of log[1 + exp(−qᵢzᵢ)] without first rounding probabilities.
| Slope | Recalculated log-likelihood | Printed answer |
|---|---|---|
| 0 | −36.5391 | −36.59 |
| 0.001 | −9.6810 | −9.70 |
| 0.002 | −6.9186 | −6.91 |
Among these three candidates, slope 0.002 gives the best likelihood. However, the unrestricted fitted slope near 0.00179577 performs slightly better. The printed numbers are not exact evaluations of the displayed parameter combinations.
Exercise 8.40: Evaluate logit likelihood at three intercepts
Now fix b₁ = 0.0018. Holding one parameter constant makes the comparison easy to interpret, but it does not reoptimize the other coefficient at each point.
| Intercept | Recalculated log-likelihood |
|---|---|
| −3.8 | −6.803885 |
| −3.6 | −6.766637 |
| −3.4 | −6.815540 |
The best of these three intercepts is −3.6. Accordingly, the result agrees closely with the fitted intercept of −3.604995. These are conditional likelihood evaluations, not a profile likelihood, because the slope remains fixed rather than reestimated.
Exercise 8.41: Explain why logit and probit coefficients differ
No rule requires equal coefficients. The latent-variable scales differ, and the two cumulative distribution functions have different shapes. Therefore, compare fitted probabilities or effects on the probability scale before comparing raw coefficient magnitudes.
For the visible dataset, the independently optimized probit coefficients are approximately −1.99418 and 0.00100350. The logit slope divided by the probit slope is roughly 1.79, not exactly 1.6. Thus, this dataset itself illustrates the limitations of a universal scaling shortcut.
Exercise 8.42: Explain when marginal effects can look similar
Logit and probit can produce close marginal effects when they fit similar probability curves over the observed predictor range. However, equality is not guaranteed, particularly in the tails or under model misspecification.
For a binary predictor, compute a discrete probability difference between values zero and one. In addition, report whether an effect represents a particular observation, a representative predictor profile, or an average over the sample. Those choices can matter more than the choice between logit and probit.
Numerical audit: what ordinary summaries often miss
A useful worked solution should explain discrepancies rather than repeat an answer key. The following checks make the calculations transparent and help prevent errors from spreading into reports, exams, or published articles.
| Issue | What the audit found | Practical lesson |
|---|---|---|
| Critical value, 8.17 | t(0.975,13) ≈ 2.16037, not 2.18 | Use exact degrees of freedom |
| Covariance expansion, 8.19 | Cross-product needs 2Cov(b₁,b₂)d₁d₂ | Prefer the full covariance matrix |
| Extreme probit likelihood, 8.22 | Two probability floors explain the largest discrepancy | Compute log-CDF directly |
| Marginal effects, 8.25 | MEM ≈ 0.00011190; AME ≈ 0.00019368 | Label the averaging convention |
| Seasonal refit, 8.32 | Visible table yields a different trend and R² | Check data transcription and coding |
| Regional intercepts, 8.33 | Nonsignificance does not erase the estimated coefficient | Distinguish an unrestricted fit from a refit |
| Lag estimates, 8.35–8.36 | Visible tables do not reproduce the displayed answers | Report the estimator and restrictions |
| Model comparison, 8.29 | R² changes meaning when Y becomes ln Y | Evaluate on a common response scale |
How these calculations were checked
The numerical audit transcribed the visible country, reserve, utility-investment, and consumption tables. It then used stable log-probability functions, numerical likelihood optimization, and least-squares refits. Reported results retain more precision internally than the formatted article displays.
The binary-model audit optimized the intercept and slope jointly after rescaling the predictor for numerical stability. Meanwhile, the lag-regression calculations use only complete observations and explicitly include an intercept. Nevertheless, the photographs provide historical rounded data, so these calculations verify the transcribed tables rather than an unavailable original digital dataset.
Reproduce the most consequential binary-model checks
The following Python example needs NumPy and SciPy. It reproduces the probability table, MEM, AME, and probit likelihood grid without downloading additional data.
import numpy as np
from scipy.special import expit
from scipy.stats import norm, t
x = np.array([569,408,2240,1869,16471,1282,2102,1104,914,5746,
2173,978,762,12653,3068,3075,547,5185,7082,1162])
y = np.array([0,0,0,0,1,0,1,0,0,1,1,0,0,1,1,1,1,1,1,0])
z = -3.605 + 0.0018*x
p = expit(z)
q = 2*y - 1
loglike = -np.logaddexp(0, -q*z).sum()
p_at_mean = expit(-3.605 + 0.0018*x.mean())
mem = 0.0018*p_at_mean*(1-p_at_mean)
ame = np.mean(0.0018*p*(1-p))
print("probabilities:", p)
print("log-likelihood, MEM, AME:", loglike, mem, ame)
for slope in [-0.001, 0, 0.001, 0.002, 0.003]:
print("probit grid:", slope,
norm.logcdf(q*(-2 + slope*x)).sum())
print("correct t critical value:", t.ppf(0.975, df=13))
Historical data behind the lag-model refits
The table below preserves all 20 annual observations used in the two lag-model checks. Utility investment and GNP belong to the Koyck exercise, while consumption and disposable income belong to the Almon exercise. Although each series uses billions of dollars, the columns represent different economic quantities.
| Year | Utility investment Y | GNP X | Consumption Y | Disposable income X |
|---|---|---|---|---|
| 1960 | 5.2 | 506.0 | 324.9 | 349.4 |
| 1961 | 5.0 | 523.3 | 335.9 | 362.9 |
| 1962 | 4.9 | 563.8 | 355.2 | 383.9 |
| 1963 | 5.0 | 594.7 | 374.6 | 402.8 |
| 1964 | 5.5 | 635.7 | 400.4 | 437.0 |
| 1965 | 6.3 | 688.1 | 430.2 | 472.2 |
| 1966 | 7.4 | 753.0 | 464.8 | 510.4 |
| 1967 | 8.7 | 796.3 | 490.4 | 544.5 |
| 1968 | 10.2 | 868.5 | 535.9 | 588.1 |
| 1969 | 11.6 | 935.5 | 579.7 | 630.4 |
| 1970 | 13.1 | 982.4 | 618.8 | 685.9 |
| 1971 | 15.3 | 1,063.4 | 668.2 | 742.8 |
| 1972 | 17.0 | 1,171.1 | 733.0 | 801.3 |
| 1973 | 18.7 | 1,306.6 | 809.9 | 901.7 |
| 1974 | 20.6 | 1,412.9 | 889.6 | 984.6 |
| 1975 | 20.1 | 1,528.8 | 979.1 | 1,086.7 |
| 1976 | 22.3 | 1,702.2 | 1,089.9 | 1,184.5 |
| 1977 | 25.8 | 1,899.5 | 1,210.0 | 1,305.1 |
| 1978 | 29.5 | 2,127.6 | 1,350.8 | 1,458.4 |
| 1979 | 33.2 | 2,368.5 | 1,509.8 | 1,623.2 |
For the Koyck refit, regress utility investment from 1961 onward on an intercept, contemporaneous GNP, and the previous year’s utility investment. By comparison, for the Almon refit, start in 1963 and construct the three polynomial lag sums from current disposable income and its three prior values. Consequently, the first model uses 19 observations and the second uses 17.
The following continuation uses the table’s consumption and income values. It applies no endpoint restrictions; therefore, another Almon implementation may legitimately differ if it imposes such restrictions explicitly.
# Refit the quadratic Almon specification with NumPy.
consumption = np.array([
324.9,335.9,355.2,374.6,400.4,430.2,464.8,490.4,535.9,579.7,
618.8,668.2,733.0,809.9,889.6,979.1,1089.9,1210,1350.8,1509.8])
income = np.array([
349.4,362.9,383.9,402.8,437,472.2,510.4,544.5,588.1,630.4,
685.9,742.8,801.3,901.7,984.6,1086.7,1184.5,1305.1,1458.4,1623.2])
lags = np.column_stack([income[3-j:20-j] for j in range(4)])
j = np.arange(4)
weights_map = np.column_stack([np.ones(4), j, j**2])
Z = lags @ weights_map
A = np.column_stack([np.ones(17), Z])
coef = np.linalg.lstsq(A, consumption[3:], rcond=None)[0]
lag_weights = weights_map @ coef[1:]
residuals = consumption[3:] - A @ coef
r_squared = 1 - (residuals @ residuals) / np.sum(
(consumption[3:] - consumption[3:].mean())**2)
print("intercept and polynomial parameters:", coef)
print("recovered lag coefficients:", lag_weights)
print("R-squared:", r_squared)
A practical advanced regression workflow
- Define the target. Decide whether you need a mean, an individual future outcome, a probability, or a causal effect.
- Record units. State whether money uses dollars, thousands, or billions, and whether rates use fractions or percentage points.
- Choose the information set. For forecasting, exclude predictors unavailable at the forecast origin.
- State the specification. Identify transformations, baseline categories, lag lengths, and restrictions.
- Check numerical stability. Inspect rank, conditioning, convergence, and extreme fitted probabilities.
- Evaluate assumptions. Investigate heteroskedasticity, serial correlation, influential observations, and structural breaks as appropriate.
- Validate outside the estimation sample. Use time-ordered validation for time-series forecasts.
- Communicate uncertainty. Label intervals, marginal-effect conventions, and unresolved data limitations.
Complete or quasi-complete separation can prevent finite maximum-likelihood estimates in binary models. Moreover, a numerical convergence message alone does not establish that every parameter behaves well. The statsmodels pitfalls documentation discusses these identification and convergence issues.
Frequently asked questions about advanced regression
Is a prediction interval the same as a confidence interval?
No. A confidence interval for the conditional mean quantifies uncertainty about the average response at specified predictors. By contrast, an individual prediction interval also includes the new observation’s random variation, so it is wider under the same model and confidence level.
Does a high R² mean the model forecasts well?
Not necessarily. A flexible model can fit historical observations closely while performing poorly on future data. In addition, common trends can create impressive fit statistics without a stable predictive relationship.
Can I compare logit and OLS R² values directly?
No. Conventional OLS R² and McFadden pseudo-R² summarize different objects. Therefore, use prediction quality on an appropriate common target when comparing alternative approaches to the same task.
Why does the average marginal effect differ from the effect at the mean?
The probability transformation is nonlinear. Consequently, evaluating one derivative at the average predictor generally differs from averaging derivatives across all predictor values. The country example produces an AME about 73% larger than the MEM using the rounded model.
Should a nonsignificant dummy coefficient become zero?
No. Its estimate remains part of the unrestricted fitted model. If you choose a restricted model without that term, refit the regression and report the new specification rather than manually changing just one coefficient.
Does a negative lag coefficient prove the theory is wrong?
Not by itself. Correlated lags, small samples, misspecification, and imposed restrictions can generate unstable individual coefficients. However, a counterintuitive lag pattern deserves investigation through joint tests, substantive reasoning, and external validation.
Which model is better, logit or probit?
Neither wins universally. Compare the models using the purpose of the analysis, probability calibration, predictive performance, and interpretability. Moreover, check whether differences actually matter over the range of observations that supports your decision.
Conclusion: make the interpretation as rigorous as the calculation
Advanced regression becomes more useful when every number connects to a clearly defined question. Forecast intervals require the right uncertainty target, binary models require probability-scale interpretation, and dynamic regressions require a distinction between immediate and cumulative effects.
The 27 exercises also show why verification matters. Correcting a critical value, preserving covariance terms, and computing extreme probabilities accurately can change an answer even when the formula appears familiar. Therefore, combine transparent calculations with explicit assumptions before treating a regression result as evidence.
Sources and further reading
- Supplied textbook photographs: Chapter 8, pages 197–205, exercises 8.16–8.42 and Tables 8.17–8.22. Complete bibliographic details were not available in the supplied images.
- Penn State STAT 501 — Simple linear regression estimation and prediction.
- Penn State STAT 501 — Multiple regression estimation and prediction.
- Penn State STAT 501 — Categorical predictors and interactions.
- Hyndman and Athanasopoulos — Forecasting: Principles and Practice, time-series regression.
- Hyndman and Athanasopoulos — Forecasting using transformations.
- statsmodels — Logistic log-likelihood.
- statsmodels — Marginal effects.
- statsmodels — Numerical and statistical pitfalls.
