Linear WELLBY: Validity & Reliability

1. Definition and Use-Case

A WELLBY (wellbeing-adjusted life year) is a "common currency" approach: summarize an intervention's welfare impact as the change in subjective wellbeing per person-year, so interventions involving different mechanisms and outcomes(health, income, education, mental health) can be compared on a single scale.[1]

The standard question (OECD prototype): "Overall, how satisfied are you with your life as a whole these days?" with anchors 0 = "not at all satisfied" and 10 = "completely satisfied".[2]

Incremental vs. Level-Based WELLBYs

A crucial distinction: are we summing changes relative to a counterfactual, or levels of wellbeing? These have different requirements.

This second form requires a defined zero point (e.g., death = 0) and is not invariant to shifting the LS scale. The World Happiness Report makes "dead people score 0" an explicit assumption for level-based accounting.[3]

2. When the Neutral Point Matters

There are two different "zeros" people reference:

• Neutral wellbeing: the point where life is neither good nor bad (u = 0)
• Death as zero: treating death as LS = 0 for life-year calculations

For incremental comparisons (program A vs. program B among the living), adding a constant c to everyone's LS doesn't change ΔLS. The neutral point is mostly irrelevant.

But for mortality comparisons—e.g., "WELLBYs from life extension = (life-years gained) × (average wellbeing level)"—the origin matters because a shift changes the integral.

UK Green Book guidance discusses an indifference point around LS ≈ 2 for some respondents, while adopting a working assumption of 1. Peasgood et al. (2018) provide tentative estimates but acknowledge high uncertainty.[4]

3. Assumptions for Validity

Using linear WELLBYs for cross-intervention comparison requires several assumptions. A common overstatement is that "equal scores mean equal welfare"—but this is stronger than most applications need.

Level Comparability vs. Unit-Change Comparability

The key distinction is between two claims:

• Level comparability: LS^A = 7 and LS^B = 7 imply U^A ≈ U^B (rarely needed for incremental WELLBYs)
• Unit-change comparability: A one-point change has approximately the same welfare meaning across people/groups (what most incremental WELLBY applications require)

Formally, suppose each person has a reporting function[rf]A reporting function f_i(·) maps a person's underlying ("true") welfare u to their reported life satisfaction score. It captures how individuals translate internal states into numerical responses—including factors like personality, cultural norms, and interpretation of scale anchors.: LS_it = f_i(u_it) + ε_it. If f_i differs across people, equal reported levels don't mean equal welfare. But it can still work for differences if the heterogeneity is "only a shift."

Shifters vs. Stretchers

A useful decomposition is the affine model:

This distinction is not merely theoretical. Recent work on scale-use heterogeneity explicitly models "shifters" and "stretchers" and proposes calibration-question approaches to adjust for them.[5]

The Four Core Assumptions (Expanded)

Response Shift: A Distinct Threat

Even if baseline scale-use heterogeneity cancels (by randomization), treatment can change the meaning of the respondent's self-evaluation. In quality-of-life research, this is called response shift: changes in internal standards, values, or conceptualization that alter how respondents answer over time.[6]

For wellbeing interventions in LMICs—especially psychosocial programs that may explicitly reframe cognition—response shift is not a corner case. If treatment changes the reporting function f_i(·), observed ΔLS mixes "true welfare change" with "scale change," biasing the WELLBY estimate in either direction.

4. Empirical Evidence

Reliability: Noisy But Not Useless

OECD guidance synthesizes evidence that single-item life evaluations have test-retest correlations around 0.5-0.7 over short windows (1 day to 2 weeks), with country-level means showing much higher stability.[7]

Krueger and Schkade (2008) found life satisfaction measured two weeks apart had correlation ≈ 0.59. Two implications for WELLBY CEA:

• Measurement error attenuates estimated effects (bias toward zero)—small real effects may be undervalued
• Noise inflates uncertainty—credible estimates need adequate sample sizes and good designs

LMIC Evidence: Life Satisfaction Responds to Interventions

These findings show: (1) LS measures have detectable signal in LMIC RCTs, not just noise; (2) effects can persist, meaning "duration" in WELLBY accounting is empirically relevant. They do not solve cross-study comparability—but demonstrate that in at least one setting, SWB is responsive.

External Validity: Numbers Predict Behavior

A strong response to skepticism: even if the numbers seem arbitrary, do they behave like a measurement? Kaiser and Oswald show that single numeric feelings responses have strong predictive power—relationships to later "get-me-out-of-here" actions (changing neighborhoods, jobs, partners) tend to be replicable and close to linear in large longitudinal datasets.[10]

5. Critiques and Responses

The hardest critique is about what the data cannot pin down. Bond and Lang (2019) argue that with ordinal response data, comparing "average happiness" between groups is generally not identified without strong assumptions—monotonic transformations can reverse results.[11]

Clarifications for workshop discussions:

• The critique is most directly aimed at cross-group mean comparisons and latent-variable interpretation
• It does not automatically imply that within-study randomized treatment effects are meaningless
• It implies you should be explicit about what assumptions let you treat reported changes as welfare units

6. Alternatives and Relative Reliability

We cannot stop at "WELLBY is unreliable." Decision-makers must choose something, and the relevant question is comparative. "WELLBY vs. DALY" should be framed as a portfolio comparison problem under measurement uncertainty, not winner-take-all.

DALYs and QALYs: Standardized But Narrower

DALY (Disability-Adjusted Life Year) = Years of Life Lost (YLL) + Years Lived with Disability (YLD).[12]
QALY (Quality-Adjusted Life Year) = years of life × health utility weight (0-1 scale, 1 = perfect health).[13]

UK Green Book guidance explicitly suggests life satisfaction can reflect dimensions beyond standard QALY instruments (EQ-5D), making WELLBYs informative where QALYs are incomplete.[14]

A Hybrid Decision Rule

• Use DALYs/QALYs where the primary pathway is health and the evidence base is deep
• Use WELLBYs where interventions plausibly change welfare through broader channels (especially mental health and social interventions), but require stronger transparency about measurement assumptions
• When using WELLBYs across heterogeneous contexts, report sensitivity to scale-comparability assumptions

The Measurement Layer Problem

A practical constraint: many LMIC mental health studies report depression scales or symptom indices, not standard 0-10 life satisfaction. GiveWell's assessment of StrongMinds explicitly highlights uncertainty in translating depression improvements into life satisfaction gains.[15]

Even if you accept WELLBY as the target unit, the measurement layer forces choices:

• Use DALYs/QALYs (more standard in health evaluation) even if they miss non-health welfare
• Use life satisfaction directly, but only where trials collect it
• Use mapping models (depression → LS), but carry mapping uncertainty explicitly

7. WELLBY Calculator

This simple calculator assumes constant effect size over the duration. Real applications should account for effect decay, discounting, and uncertainty ranges.

8. Practical Recommendations

For Funders Comparing Interventions

• Treat WELLBY estimates as one input among several, not the final answer
• Conduct sensitivity analyses across different approaches and assumptions
• Report how rankings change if you assume different neutral points or scale comparability
• Communicate uncertainty honestly rather than false precision

For Researchers Designing Studies

• Use standardized life satisfaction questions (OECD prototype) to enable cross-study comparison
• Include long-term follow-up to capture duration effects
• Consider calibration questions/vignettes if comparing across populations
• Pre-register analysis plans including functional form choices

Conditions for Stronger Inference

• Large effect sizes that overwhelm measurement error
• Within-person designs where each person serves as their own control
• Multiple wellbeing measures showing the same directional effect
• Triangulation with behavioral or objective outcomes
• Robustness checks across functional forms (log vs. linear)

9. Open Questions (Research Agenda)

These questions represent high-value areas for future research that could meaningfully improve the reliability of WELLBY-based comparisons:

1. Neutral point estimation: What is the actual neutral point on the 0-10 scale for different populations? How stable is it across contexts?
2. Scale-use heterogeneity mapping: How do shifters vs. stretchers vary across LMIC populations, and can we predict which matters more in a given context?
3. Cheap calibration methods: Can vignettes, anchoring questions, or other calibration approaches work in low-resource settings without excessive respondent burden?
4. WELLBY-DALY relationship: What's the true mapping between WELLBYs and DALYs, and is it linear? How much does it vary by health condition?
5. Demand effects and response shift: How do experimenter demand effects and response shift vary by intervention type? Are psychosocial interventions particularly vulnerable?
6. Instrument standardization: What survey instruments and protocols minimize cross-study comparability problems while remaining feasible in diverse settings?