WELLBY Measurement Workshop

Introduction & Framing

The Unjournal · Pivotal Questions Initiative
March 16, 2026

These slides are just visual aids—prepared by Claude in ~10 minutes from my notes.
The content is what matters; the slides themselves are not important.

← Back to Sessions · Google Doc

~5 min

David Reinstein

  • 20 years in academia (economics)
  • Rethink Priorities
  • Co-director, The Unjournal

I'm not an expert in this area—
please don't let me dominate the conversation

Goals of This Workshop

Four interconnected objectives

1 Bridge Researchers & Practitioners

  • Help researchers understand practitioners' highest-value questionsWhat measurement challenges actually affect funding decisions? What trade-offs do practitioners face? and trade-offs
  • Help practitioners understand the most relevant and up-to-date research

2 Foster Communication

  • Get on the same page
  • Agree on terminology
  • Identify points of consensus
  • Identify high-value cruxesPoints of disagreement where resolving the disagreement would change decisions

3 Measure Beliefs Openly

  • Precision and calibrated uncertaintyReporting confidence levels that match actual accuracy—being right 70% of the time when you say you're 70% confident
  • High "value of information" Bayesian updatingRevising beliefs based on evidence, proportional to how surprising the evidence is

4 Drive Better Decisions

  • Measuring impact of interventions in LMICs
  • Using existing measures effectively
  • → Better funding decisions

Focus & Scope

Limiting scope to avoid straw-manning

The Core Questions

"What are the best feasible ways to measure and compare the relative benefits and cost-effectiveness of distinct interventions?"

"How do standard approaches compare?"

Using WELLBY and other metrics to prioritize interventions in LMICs, usually linked to RCTs

What We're NOT Asking

  • Which countries are happier?
  • Does marriage make people happier?
  • Do people adapt to large windfalls?
Academia vs. practice: Policymakers and funders must make choices now—we can't say "we don't know." We're not setting a firm scientific precedent.

Premises

Things we hope we can agree on (or maintain for now)

1 Wellbeing Is Important

Improving psychological well-being and life satisfaction are important goals we should consider in comparing interventions

...to the extent these are measurable

2 Self-Reports Carry Some Information

Stated life-satisfaction & WELLBY are at least somewhat informative

But we may disagree about interpretation, reliability vs. other measures, and how much this matters

"Evidence that important things correlate to WELLBY" is not enough to recommend it

3 Simple Assumptions Violated

Neither extreme will be exactly true:

  • "Exactly homogeneous linear response functions"
  • "LS reports convey only within-individual rankings"

Statistically rejecting one or the other is not highly informative

What Actually Matters

To what extent do assumptions hold?
How much do violations matter?

→ Which measurement approaches lead to better decisions?

Corollary

Core assumption violations are potentially important, but may or may not:

  • Be empirically important
  • Affect comparisons in RCT contexts
    (e.g., see "scale shifters")

Potential Violations to Consider

  • Non-linear scale use — 1→2 vs 5→6 may differ in meaning
  • Heterogeneous scale use — "Shifters": same suffering, different scores
  • Response function shifts — Therapy changes reporting?
  • Cross-domain capture
  • Temporal aggregation
  • Neutral points — Critical for mortality comparisons

How much is a "unit" of wellbeing actually worth?

My Biggest Worry

Reinstein: Non-linear scale use is my biggest worry at the moment. I'd personally like to hear more about correspondence between SWB numbers and ~gold standards like time and person tradeoffsMethods where people trade off years of life or risk of death against health states—revealing their true preferences and the standard gambleAsking people to choose between a certain outcome and a gamble to reveal preference strength.

Question for Participants

What other background should we all agree on?

Workshop Structure

Rules & principles

Collaborative Notes (Google Doc)

  • Put comments within the Google Doc
  • Use blank space — don't add lines if possible
  • Questions in subtabs below each section
  • Discussion continues asynchronously after the event
Note: The Gdoc is publicly accessible and linked to the workshop page. Let me know if you want a private document.

Timing & Coordination

  • Strict timing — enables coordinated drop-ins
  • Zoom chat for procedural things only
  • Two breakout rooms:
    • Room 1: Practical/Applied
    • Room 2: Technical/Academic
  • 'Default sharing/public content' — but we ask before broad dissemination

Good Epistemic Norms

  • ModestyAcknowledging uncertainty and limits of one's expertise
  • Steel-manningRepresenting opposing views in their strongest form / charitable interpretation
  • Scout mindsetSeeking truth rather than defending a position
  • Reasoning transparency — explain the basis for your belief
  • Calibrated uncertainty
  • Track the cruxIdentify the key point of disagreement that would change conclusions
  • Avoid motte-and-baileyDefending a weak claim by retreating to a stronger, obvious claim

For Each Issue, Consider:

"What evidence could change my mind?"

What are my cruxes?

Focus on highest-VOI questions — where will our discussions yield the most value?

Let's Begin

→ Stakeholder Problem Statement

Segment 1 Page · Google Doc · All Sessions