Linear WELLBYs for Comparing Interventions

1. The decision problem

Organizations comparing interventions—especially in low- and middle-income countries (LMICs)—face a measurement problem: interventions change different things (mortality, morbidity, consumption, mental health, social cohesion). The WELLBY approach proposes translating these into a common unit based on subjective wellbeing, enabling "welfare impact per dollar" comparisons.[1]Frijters, Clark, Krekel & Layard (2020). "A Happy Choice: Wellbeing as the Goal of Government."

A focal question for this workshop: How reliably can we compare interventions, especially in LMICs, by aggregating changes in reported wellbeing (the WELLBY approach), especially across different studies and contexts? This is one of several comparison frameworks; others include DALY/QALY-based approaches, capability approaches, and direct monetary valuation. The workshop examines the linear WELLBY's reliability relative to these alternatives, not in isolation.

The measurement-to-decision pipeline illustrates why comparing interventions requires multiple translation steps. Each box represents a stage where methodological choices affect final conclusions:

2. Definitions and key concepts

Standard Life Satisfaction Questions

OECD single-item: "Overall, how satisfied are you with your life as a whole these days?" (0 = "not at all satisfied" to 10 = "completely satisfied")[2]OECD Guidelines on Measuring Subjective Well-being (2013/2024). Question modules available at oecd.org.

Cantril ladder (Gallup): "Please imagine a ladder, with steps numbered from 0 at the bottom to 10 at the top. The top of the ladder represents the best possible life for you, and the bottom represents the worst possible life for you. On which step of the ladder would you say you personally feel you stand at this time?"

These two framings—satisfaction vs. ladder position—are often used interchangeably, but may capture subtly different constructs. Cross-study comparisons should note which instrument (survey question format) was used.

Incremental vs. Level-Based Frameworks

Incremental WELLBYs[18]"Incremental" vs "level-based" is descriptive language used in this document. The literature typically just refers to "WELLBYs" with context clarifying whether the focus is on changes (differences) or absolute levels. Incremental WELLBYs are captured by observing differences between comparable treated and untreated populations—useful when mortality effects are negligible or balanced across groups.:

Where $LS_{it}$ is person $i$'s life satisfaction at time $t$ (on a 0-10 scale), $\delta$ is a discount factor for future periods, $k$ indexes the intervention, and $0$ is the counterfactual (no intervention). The sum runs over all individuals $i$ and time periods $t$. See notation key below for details.

Level-based WELLBYs (relevant for comparing interventions that affect mortality or, in some conceptual frameworks, birth rates):

3. Core assumptions

Using linear WELLBYs for cross-intervention comparison requires assumptions.[B]Benjamin, Cooper, Heffetz & Kimball (2024). "From Happiness Data to Economic Conclusions." Annual Review of Economics. Articulates four key assumptions for SWB-based welfare analysis. See also Frijters & Krekel (2021), A Handbook for Wellbeing Policy-Making, and UK Green Book supplementary guidance on wellbeing (HM Treasury, 2021/2024). The claim "equal scores mean equal welfare" is stronger than most applications need—what matters for intervention comparison is whether changes in scores reflect comparable welfare changes.

Note: The Benjamin et al. framework offers limited direct focus on comparing interventions in LMICs using RCT data—the central context for our workshop. There may be a research gap here, though HLI has been doing significant applied work in this space—more discussion is needed on how their research connects to these conceptual frameworks. (Tessa Peasgood kindly shared this paper with workshop participants.)

Time structure and discounting

Why this matters for intervention comparison: WELLBY calculations aggregate wellbeing gains over time. If Intervention A produces a 0.5-point LS boost for 2 years while Intervention B produces a 0.3-point boost for 5 years, B yields more total WELLBYs—but only if the effects actually persist as assumed. The time dimension is often the largest source of uncertainty.

Most studies measure outcomes at baseline and one or two follow-ups;[15]LMIC examples: Haushofer & Shapiro (2016, 2018) measure SWB at 9 months and 3 years after Kenya cash transfers; StrongMinds studies typically follow up at 3-6 months post-treatment; GiveDirectly studies (Kenya, Uganda) measure at 1-3 years; BRAC graduation programs measure at 2+ years. Most LMIC wellbeing RCTs have 1-2 follow-up points with limited long-term tracking. extrapolating beyond measured timepoints requires assumptions about persistence (does the effect last?) and discounting (are future wellbeing gains worth less than present ones?).

Response Shift: a distinct threat

Even if baseline scale-use heterogeneity cancels out[20]"Cancels out" means: in a randomized trial, individual differences in scale use (shifters) are equally distributed between treatment and control groups, so when we compare average changes, these individual differences subtract out. This is similar to how fixed effects absorb level differences. (by randomization), treatment can change the meaning of the respondent's self-evaluation. This is called response shift: changes in internal standards, values, or conceptualization that alter how respondents answer over time.[rs]Sprangers & Schwartz (1999). "Integrating response shift into health-related quality of life research." Social Science & Medicine.

For wellbeing interventions—especially psychosocial programs that may explicitly reframe cognition—response shift is a genuine concern, not merely a theoretical possibility. If treatment changes the reporting function $f_i(\cdot)$, observed $\Delta LS$ mixes "true welfare change" with "scale change," potentially biasing the WELLBY estimate in either direction.

4. A key critique: identification and transformations

Bond & Lang (2019) argue that with ordinal happiness data, comparing average wellbeing between groups (e.g., "are Germans happier than Americans?") is not identified without strong assumptions—monotonic transformations can reverse conclusions.[3]Bond, T.N. & Lang, K. (2019). "The Sad Truth about Happiness Scales." JPE, 127(4). However, our focus is on intervention effects, where randomization mitigates some (but not all) of these concerns—see box below.

Why this matters for intervention comparison

• Cross-study synthesis: Comparing treatment effects measured on different scales can inherit the same vulnerability
• Magnitude-sensitive cost-effectiveness: Even if both Intervention A and B show positive effects, cost-effectiveness depends on how much better. If A costs twice as much but has a 2.5× larger effect, it's more cost-effective—but only if that magnitude comparison is valid.

5. Scale-use heterogeneity: shifters vs. stretchers

When comparing different interventions using WELLBY estimates, a key concern is that different people (or populations) may use the life satisfaction scale differently. A useful framework is the affine model, which separates two types of scale-use differences:[11]The "shifters vs. stretchers" framework derives from Benjamin et al. (2012, 2014, 2023). See also Oswald (2008) and Kaiser & Oswald (2022) on scale-use heterogeneity. This matters especially when comparing intervention effects measured in different study populations—for example, a mental health program in Uganda vs. a cash transfer in Kenya.

Here $u_{it}$ is person $i$'s true welfare at time $t$, and $LS_{it}$ is their reported life satisfaction. The parameters $a_i$ (shift) and $b_i$ (stretch) capture individual scale-use patterns.

Benjamin et al. propose calibration questions to identify and adjust for scale-use heterogeneity—questions designed to have the same objective answer across respondents.[4]Benjamin et al. (2023). "Adjusting for Scale-Use Heterogeneity." NBER WP 31728.

6. Neutral point and mortality

When comparing interventions that affect mortality (e.g., bednets) against those that affect only wellbeing among the living (e.g., mental health programs), the choice of "zero point" becomes critical. Two different "zeros" are relevant (see HLI's "The Elephant in the Bednet" for a comprehensive treatment):

• Neutral wellbeing: the point where life is neither good nor bad
• Death as zero: treating "dead people score 0" for life-year calculations

For incremental comparisons among the living, the neutral point often cancels out—that is, when comparing ΔLS between intervention and control groups, both are measured from the same implicit baseline, so the zero point subtracts from both sides and doesn't affect the difference.[14]Mathematically: (LS_treatment − LS_0) − (LS_control − LS_0) = LS_treatment − LS_control. The neutral point LS_0 cancels. But for mortality comparisons—"WELLBYs from life extension = life-years × average wellbeing"—the origin is load-bearing.

7. Evidence and alternatives

Reliability: noisy but not useless

Single-item life evaluations have test-retest correlations around 0.5-0.7 over short windows.[5]Krueger & Schkade (2008). "The reliability of subjective well-being measures." This means measurement error attenuates estimated effects—small real effects may be undervalued. For relative comparisons: if measurement error is similar across interventions and contexts, attenuation affects absolute magnitudes but may preserve relative rankings. However, if noise levels differ (e.g., different survey modes, cultural contexts, or outcome domains), relative comparisons become less reliable.

Predictive validity

What this means: "Predictive validity" asks whether LS scores relate to real-world behaviors and outcomes in expected ways. If LS contains meaningful information about welfare, lower scores should predict people making costly changes (moving, quitting jobs, leaving relationships) to improve their situation.

Kaiser & Oswald show that single numeric feelings responses do predict these consequential outcomes—relationships tend to be replicable and approximately linear.[6]Kaiser & Oswald (2022). "The Scientific Value of Numerical Measures of Human Feelings." PNAS. See also Plant (2024) for additional evidence that LS measures predict consequential outcomes and respond to changes (like income transfers) we expect to matter. This is notable because a single "made-up feelings integer" turns out to have more predictive power for subsequent costly actions (e.g., moving house, leaving a partner, quitting a job, visiting a hospital) than a collection of standard socioeconomic variables including income, employment, and education.[27]Kaiser & Oswald (2022), Table 2: a single feelings integer outperforms combined socioeconomic predictors for "get-me-out-of-here" actions across UK, German, and Australian panel data. For example, moving from LS=3 to LS=7 is associated with roughly halving the probability of leaving one's neighbourhood in the subsequent period. This suggests LS captures something meaningful, though it doesn't directly prove magnitudes are comparable across people.

Note: Predictive validity is necessary but not sufficient for using WELLBYs. A measure could predict outcomes yet still be noisy, inconsistent across groups, or respond to treatments in ways that don't reflect welfare gains. It establishes that LS isn't pure noise—not that it's ready for precise cross-intervention comparison.

Response times as identification: Liu & Netzer (2023) propose using survey response times to help solve identification problems in ordered response models.[28]Liu, S. & Netzer, N. (2023). "Happy Times: Measuring Happiness Using Response Times." AER, 113(12), 3289-3322. The intuition: if two people both report "7" but one answers instantly while another deliberates, the fast responder is likely further from the boundary between "7" and adjacent categories—their latent happiness is more clearly in the "7" range. This "chronometric effect" provides distributional information that the response alone cannot. The counterintuitive insight is that how quickly someone answers a happiness question contains information about the distribution of their underlying wellbeing—not just their point estimate. Their evidence is mixed: conventional distributional assumptions are rejected in some cases but broadly supported overall.

LMIC evidence: GiveDirectly cash transfers

The outcome translation problem

Many LMIC mental health studies report depression scales or symptom indices, not standard 0-10 life satisfaction. GiveWell's assessment of StrongMinds explicitly highlights uncertainty in translating depression improvements into life satisfaction gains.[gw]GiveWell (2023). "Our Assessment of Happier Lives Institute's Cost-Effectiveness Analysis of StrongMinds." Notes substantial uncertainty in depression-to-LS mapping.

Even if you accept WELLBY as the target unit, outcome translation forces choices: use DALYs/QALYs (more standard in health evaluation) even if they miss non-health welfare, use life satisfaction directly but only where trials collect it, or use mapping models—statistical relationships estimated from observational data that predict life satisfaction from other measures (e.g., "a 1-SD reduction in PHQ-9 depression score corresponds to ~0.4 points higher LS")—but carry mapping uncertainty explicitly.

Comparison with alternatives

8. WELLBY Calculator

This calculator assumes constant effect size. Real applications should account for effect decay, discounting, and uncertainty.

9. Considerations for discussion

The following are practices and considerations found in the literature—not workshop conclusions or endorsements. A key goal of the workshop is to assess which of these, if any, represent genuine improvements over current practice, and what their practical limitations are.

What are funders currently doing?

Some funders and evaluators (e.g., HLI, Founders Pledge) report using approaches such as:

• Sensitivity analyses across different neutral points and scale-comparability assumptions
• Reporting how intervention rankings change under alternative assumptions
• Combining WELLBY estimates with other evidence types (e.g., objective indicators, expert judgment)

Whether these practices adequately address the measurement concerns discussed above is itself a question for discussion.

What methodological improvements have been proposed?

Researchers have proposed various approaches to strengthen WELLBY-based comparisons. Each involves trade-offs between feasibility and rigor:

• Standardized life satisfaction questions (e.g., the OECD recommended module, updated 2025) for cross-study comparability
• Long-term follow-up to capture duration effects and potential hedonic adaptation
• Calibration questions or vignettes when comparing across populations (Benjamin et al., 2023)
• Pre-registration of analysis plans including functional form choices

Under what conditions might WELLBY comparisons be more reliable?

Several conditions have been suggested in the literature as potentially supporting stronger inference, though each carries its own caveats:

• Large effect sizes that overwhelm measurement error
• Within-person designs where each person serves as their own control[19]Caveat: Within-person designs can introduce demand effects—participants may feel motivated to report improvement to please researchers. This is a particular concern for psychosocial interventions with non-blinded delivery.
• Multiple wellbeing measures showing the same directional effect
• Triangulation with behavioral or objective outcomes

10. Open questions (research agenda)

The following questions have been identified by researchers in this area (including workshop participants and authors cited above) as areas where additional evidence could meaningfully improve the reliability of WELLBY-based comparisons:

1. Neutral point estimation: What is the actual neutral point on the 0-10 scale for different populations? How stable is it across contexts?[16]Limited LMIC-specific research exists on neutral point estimation. Peasgood et al. (2018) provide estimates for UK populations; whether these generalize to LMIC contexts is unclear.
2. Scale-use heterogeneity mapping: How do shifters vs. stretchers vary across LMIC populations, and can we predict which matters more?
3. Cheap calibration methods: Can vignettes, anchoring questions, or other calibration approaches work in low-resource settings without excessive burden?
4. WELLBY-DALY relationship: What's the mapping between WELLBYs and DALYs, and is it linear? How much does it vary by health condition?
5. Demand effects and response shift: How do experimenter demand effects and response shift vary by intervention type?

11. Workshop prompts

Neutral prompts for workshop deliberation: