Mastering Overdispersion: Cameron & Trivedi's Test

In fields like insurance, credit risk, and operational risk modeling, count data models are indispensable. Yet, one subtle pitfall can undermine even the most carefully specified models: overdispersion.

When the variance in your data exceeds what your Poisson or similar count model predicts, ignoring it can lead to biased inference, underreported uncertainty, and flawed decision-making. That's where Cameron and Trivedi's test for overdispersion becomes crucial.

In this article, I explore what this test is, how to compute and interpret it, real-world applications, and the consequences of getting it wrong.

What Is Overdispersion?

In a canonical Poisson model, the mean and variance are assumed to be equal:

Var(Y) = E(Y)

When the actual variance exceeds the mean:

Var(Y) > E(Y)

the data are said to be overdispersed.

Overdispersion often arises from:

Unobserved heterogeneity (missing covariates)
Excess zeros in count data
Model misspecification or omitted structure

Cameron and Trivedi's Test: The Logic

Cameron and Trivedi's test (1986) is built on the idea that if the Poisson model is correctly specified, the conditional variance should equal the conditional mean. The test works by regressing the squared residuals on the predicted mean and testing for a significant relationship.

If the coefficient is significantly positive, it suggests overdispersion.

How to Compute It (Simplified Steps)

Fit a Poisson regression model to your count data.
Obtain the predicted means (μ̂ᵢ) and compute squared Pearson residuals.
Regress (Yᵢ - μ̂ᵢ)² - μ̂ᵢ on μ̂ᵢ (or a function of it).
Test whether the coefficient β is significantly different from zero.

Interpreting the Results

β ≈ 0: No evidence of overdispersion; Poisson model may suffice.
β significantly > 0: Overdispersion present; Poisson assumptions violated.

In such cases, alternative models like Negative Binomial regression or quasi-Poisson are often more appropriate.

Consequences of Ignoring Overdispersion

Underestimated standard errors → false positives (Type I error)
Inflated model confidence → flawed business strategies
Misspecified risk measures → regulatory or capital adequacy implications

For example, in insurance pricing, ignoring overdispersion can lead to underestimated premium risk buffers; in credit risk, it can distort PD/LGD models and capital calculations.

Real-World Applications

Overdispersion testing is critical in insurance claim frequency modeling, credit default count analysis, operational loss event counting, and any domain where Poisson regression is applied to count data with inherent variability.

Best Practices

Always diagnose overdispersion in count data models before deploying.
Combine statistical diagnostics (like Cameron & Trivedi) with domain expertise.
Document overdispersion checks and decisions in your model risk or validation reports.

Final Thought

"The subtle gaps between model assumptions and real-world data can cause the biggest surprises."

Cameron and Trivedi's test offers a rigorous yet intuitive lens to catch one such gap, overdispersion, before it silently undermines your predictions.

Let's Discuss

Have you encountered overdispersion challenges in your models? What strategies did you use to address them? Share your experience below — let's make diagnostics a bigger part of our modeling conversation.