5. Technical Appendix: Quantifying the Impact of Economic Outcomes

In constructing the Index of Economic Outcomes, some methodological and practical challenges will inevitably arise. However, there is no reason to think that they are intrinsically any more problematic than with GDP. Over the past several decades of GDP measurement, statistical agencies have solved many problems that have arisen, and they continue to do so. The result has been, according to Diane Coyle (who is “affectionate” towards GDP [2014]), that “The statistics of the national accounts are extremely complicated, with all kinds of ad hoc assumptions and patches” (2011, pp. 81–82). Certain aspects of real GDP calculation involve the use of judgement, such as the selection of items to include in the “representative” baskets that are needed in the conversion of nominal to real GDP (ONS 2019, 2021).

I begin by briefly reviewing some of the technicalities involved in calculating GDP, as a benchmark for discussing the potential challenges with developing and producing the IEO.

Effective ways of dealing with the various measurement issues that have arisen include:

A more fundamental challenge is population heterogeneity: the assumption that the approved basket applies across the whole population is unlikely to be met. Different sections of the population have differing typical baskets, and in particular, basic necessities play a far larger role in the expenditure of lower income groups. In contrast, the calculation of GDP needs to assume that the basket represents the population as a whole, so that inflation may be more significant for those on a low income if price rises affect mainly necessities, and vice versa.

The index number problem has been a major focus. This arises because comparisons involving GDP, e.g. to assess the rate of growth between two points in time, require adjustment for inflation—the conversion of nominal into real GDP. This involves decomposing the observed values into quantity and price components.

An initial observation that has often been made is that the idea of a separate quantity and price does not apply well to all types of transaction. For example, bank charges, gambling expenditures and life insurance payments are not readily decomposable (Turvey 1989).

More technical issues arise from the problem that typical purchases change over time, as some items become less relevant and others are considered to have become important. A great deal of work has been devoted to this: one index (Laspeyres) takes the initial basket as the basis for comparison, while another (Paasche) focuses on the later basket. They do not necessarily coincide in practice, so some combination can be used, such as their arithmetic mean or their geometric mean. The latter is known as the Fisher ideal index. An alternative is the Walsh index. Those two have generally been found to give similar results, and are known as “superlative” indices because they have desirable mathematical properties (IMF 2004, Chapter 15). Other approaches include the Young and Divisia indices, a discrete approximation to the Divisia index, and chain indices (IMF 2004, Chapter 15). In addition, a stochastic approach has been proposed (IMF 2004, Chapter 16).

This brief list of approaches to the index number problem indicates that there is no single method that can be considered as automatically correct. Some reassurance is gained from the observation that the better price indices tend to give similar answers.

Some aspects of the IEO are simpler than calculation of GDP. There is no problem with population heterogeneity or lack of representativeness, as long as all residents (including people who are not in private households), or a representative sample, are included. Volatility, which is a severe problem with the items in a representative basket—because of changes in tastes and new goods, and also seasonally—is not a feature of economic outcomes of the type proposed. And they usually change slowly, so that the appropriate time interval for monitoring them would be quarterly. Exceptions to this are the rapid deteriorations that can occur with a societal disaster such as a war, financial crash or pandemic. Even in such cases, with something like the Covid-19 pandemic and the ensuing inflation, the major changes were not in the needs themselves, but rather in the extent to which needs were met, such as the threat of evictions and difficulty meeting fuel bills. The same items remained valid, albeit with some shift in priorities, e.g. that sufficient living space and easy access to green space tended to become more highly valued during restrictions on personal mobility.

As previously emphasised, the economic outcome indicators take their importance from the mediating role they play between the economy and the impact on health and wellbeing. Economic activity plays a causal role in bringing about the range of outcomes, and in turn, each of these contributes to health and wellbeing. Candidates for selection are restricted to basic needs that fulfil both these criteria.

Second, inclusion depends on the severity of the impact of each item on physical and mental health and subjective wellbeing. In principle this can be quantified, based on the (mainly academic) literature on the causes of health and wellbeing, and updated as new evidence is obtained. Weighting of each specific item in the construction of the IEO would similarly depend on this severity of impact.

Ideally, the criterion for inclusion and the weight given to each individual indicator would both be based on calculations using a comprehensive evidence base. This is not currently feasible, and development work will be required to achieve that aim. In the short term, reliance will be placed on the appropriate types of expert, as stated earlier. A process is envisaged that starts with expert estimates informed by the existing evidence base, and iteratively moves towards a rigorous, evidence-based calculation. As happened with GDP historically, it may take some time to establish fully. Also, there is a case for retaining some less formal input, rather than having a purely technocratic system.

In principle, the magnitude of the contribution of each item to health, taking account of its prevalence in each particular population, can be calculated following well-established epidemiological principles, and could readily be extended to cover subjective wellbeing, as explained below. The appropriate measure is the population attributable fraction (PAF; other names have also been used). In epidemiology, this assesses the public health impact of population “exposures”—in the current context an exposure would be job insecurity, illiteracy and the other economic outcome indicators.

In the medical context, the PAF is typically used to estimate the contribution of each exposure (e.g. cigarette smoking) to a specific disease (e.g. lung cancer), which can be in terms of incidence (new cases) or deaths; it can also be applied to all-cause mortality (see Mansournia and Altman 2018 for a clear and brief introduction). PAFs can readily be visualised using scaled Venn diagrams, making them easily interpretable by non-technical readers (Eide and Heuch 2001).

Smoking elevates the risk of lung cancer tenfold—in epidemiological parlance, the relative risk is 10 (the relative risk is the ratio of the disease rate in the exposed group to that in the unexposed group). There is a baseline risk, call it ρ, and the risk for smokers is 10ρ. The implication is that among smokers, not all the lung cancer cases are caused by smoking—proportionately, ρ would have happened even without it, leaving 9ρ that are attributable to their smoking exposure, 9/10 of the total cases. The formula for attributable fraction (AF) is therefore:where RR is the relative risk.

This is the fraction attributable to smoking only among smokers. We are interested in the fraction attributable to smoking in the whole population, which is the PAF rather than the AF. For that, we require also the proportion of cases that has a particular exposure. The PAF formula for a single exposure is:where p is the proportion of cases with the exposure. Outside the medical research context, we are not generally able to observe this. We can, however, observe the proportion of the whole population with the exposure: this corresponds to the proportion unfulfilled for each indicator, e.g. the proportion with insecure employment, or the proportion who are illiterate. Call this ϕ.

In the smoking and lung cancer example, the non-smokers have the baseline risk of the disease. If half the population are cigarette smokers, by definition there are the same number of cases among the non-smokers as the number of cases among the smokers that are not attributable to smoking. So the non-smokers have (proportionally) ρ cases, the same as the non-attributable cases among the smokers: there are 11ρ cases in all, 10ρ among the smokers. Therefore p = 10/11. If only a quarter of the population are smokers, then for every 10ρ cases among smokers (9ρ of them being attributable and ρ of them not) there are 3ρ cases among non-smokers, and p = 10/13.

To summarise: in the example with half the population being smokers, ϕ = ½; in the one where a quarter were smokers, ϕ = ¼. It is straightforward to show that in general:

Equation (5.2) then becomes:

In the example of cigarettes and lung cancer, this would be 9/11 when ϕ = ½, and 9/13 when ϕ = ¼.

We obtain the value of ϕ for each population, the proportion who have, for example, job insecurity. We obtain the value of RR from the literature; this is based on the assumption that it is applicable to our population of interest, i.e. (usually) that it is transferable across contexts—this may require extra empirical investigation. To generate an estimate that is comparable across different populations and across time, this should be adjusted to an age-standardised population (Martinez et al. 2019).

Thus, suppose that job insecurity applies to 5% of the population, i.e. ϕ = 0.05, and that this raises the risk of an adverse impact, say moderate anxiety, fivefold (RR = 5). From (5.4), we have that the PAF is equal to (5 – 1)/(5 – 1 + 20) = 4/24, or 1/6. It means that of all the people in the population who have moderate anxiety, one in six is due to job insecurity.

In this calculation, it is crucial that RR, the relative risk, refers to the causal impact of the factor. This requires adjustment for confounding variables (omitted variable bias), because PAF calculation assumes the absence of uncontrolled confounding. In addition, careful attention to causal inference is essential. The various exposures (job insecurity, illiteracy, etc.) also have to be mutually adjusted, to avoid double counting (Klompmaker et al. 2021). This is further discussed below, in the subsection “Additional complications”.

There are some useful extensions of the PAF formula in the epidemiological literature. For multiple exposures, indexed by i, the formula for each one becomeswhere RR_i is the relative risk for exposure level i compared with 0, the unexposed group (implying that RR₀ = 1), and p_i is the proportion of exposure level i among cases.

The sum of k category-specific attributable fractions (Rockhill et al. 1998) is.

This formula can be used for joint exposures, and is also applicable to multiple categories of each item—different levels (severity thresholds) for homelessness, literacy, etc. (Note that the PAFs for the different items do not necessarily sum to one.) An alternative method is to use model-based standardisation, incorporating interaction terms.

These formulae apply to the number of new (“incident”) cases, i.e. they are based on counts; the calculations would be the same if deaths rather than new cases were taken as the endpoint. It is advantageous to instead carry out the analysis on the timing of deaths, using survival analysis (Cox and Oakes 1984), which is conveniently thought of as being the interval by which deaths are brought forward, i.e. life is shortened. This does not commit us to considering only the situations where the exposure has fatal consequences, as will be seen in a moment. The formula is derived by substituting the hazard ratio HR for the relative risk RR, giving:

The hazard ratio is the ratio of the instantaneous hazard rate in each exposure group to the baseline group (unexposed; i = 0). Using this survival-analysis approach makes it possible to calculate the years of life lost (YLL) attributable to each exposure (Martinez et al. 2019), which is a better measure of public health importance than counting numbers of deaths. It corresponds to the reduction in life expectancy due to each exposure.

Traditionally, survival analysis of this type has required the assumption of proportional hazards, i.e. that the ratio between the proportion of survivors in each exposure group to that in the unexposed group is approximately constant over time. Where this is not the case, other methods can be used (Uno et al. 2014), for example a parametric accelerated failure time model (Keiding et al. 1997). Relatedly, the hazard ratio may vary over time, if highly susceptible people have an elevated risk shortly after exposure, and subsequently have an apparently lower than baseline risk because there are fewer of the susceptibles. Various methods are available to handle this issue, including the calculation of adjusted survival curves, and comparison of the distribution of survival times (Hernán 2010); see also Mansournia et al. (2019).

In the IEO context, we are interested not only in fatalities but also in reduction in the quality of life. Staying with just the health impact for the moment, one widely used method is to allow also for loss of functional health, using Disability-Adjusted Life Years (DALYs) (Murray et al. 2012).¹ The formula is:where YLL is the Years of Life Lost due to dying early, and YLD (Years of healthy Life lost due to Disease/Disability) is the loss of functional health due to disability or disease. YLD is the product of the prevalence in the population (number of existing cases) in the year of interest with the disability weight.² The disability weights are measures of health loss, obtained by surveys of respondents from diverse cultural and educational backgrounds, who were asked to rank random pairs of health states using lay descriptions.

The upper bound of the duration of the harmful impact is each individual’s average life expectancy in the case of lifelong impairment. This is taken from the lowest observed mortality rates experienced at any age from populations over 5 million across the world, incorporated into a life table. From this, the values of “remaining standard life expectancy” at any given age are derived, and are used to multiply deaths at any age by the corresponding value. This implies that all countries, subpopulations and years are measured using the same standard, one that is aspirationally low (Global Health Data Exchange 2019). This has comparative and ethical advantages (Devleesschauwer et al. 2020).

The YLD component allows this method to be used for exposures that cause illness or other health problems but that do not necessarily shorten life. The calculation of YLD requires information on the severity of the condition. A comprehensive list of medical conditions has been developed, e.g. diabetic foot is 0.20, and moderate anxiety disorder is 0.133 (Murray et al. 2012; Salomon et al. 2015; GBD 2017 DALYs and HALE Collaborators 2018). In Eq. (5.7), the use of DALYs rather than simply of YLL allows survival analysis to be extended to cover the health-related quality as well as the length of life.

A similar measure has been proposed for subjective wellbeing, the Wellbeing-Adjusted Life Year (WALY, WELBY or WELLBY) (Eckhardt and Wiking 2020; Layard et al. 2020). The epidemiological calculation method can therefore be extended to include subjective wellbeing as well as health, using a combination of the DALY and WALY weights. The YLD and its equivalent in the WALY thus enable the measures of IEO items to be sufficiently sensitive.

At present, the categorisation of types of harm is not necessarily identical in the DALY lists and in the causal literature on the social determinants of health, and is not always standardised in the academic literature. Also, for mental health/subjective wellbeing, the conditions that occur in both the DALY and WALY lists may not have identical weighting. More broadly, these various literatures are not currently ideally adapted for performing the calculations recommended here. In the short term, the task is to mesh these literatures, and find the optimal solution given the current state of the evidence. For example, the DALY and WALY lists could be coordinated—an envisaged situation which I will henceforth designate as D/WALYs. In the longer term, research could be orientated to facilitating these calculations by standardising the categories.

Many of the economic outcome indicators have more than one type of harm—as previously stressed, illiteracy has multiple impacts, as do insecurity of livelihood and housing. The total impact attributable to each individual item would be the sum of their D/WALYs.

The score for each indicator in each population would be calculated as the total number of D/WALYs lost as a proportion of the total in a standard life table, which is an “aspirational” mortality risk, as previously stated. This score would be a dimensionless ratio, designated here as σ.

For aggregating the different indicators, the complements of each of the scores (1 – σ) would be used. So, if σ_i = 0.05 for a particular item i, then (1 – σ_i) = 0.95. The IEO would be calculated as their geometric mean. Thus, if there are n items, indexed by i,the n^th root of the product of n component indices. A similar approach has previously been used in the “beyond GDP” context, in the construction of the Human Development Index (HDI).

One implication of this measurement system is that the IEO has a maximum value of 1 (in practice, use of percentages may be preferable, so it would be 100%). It means that the aim would be to achieve an IEO as close as possible to this theoretical maximum—it is a finite measure, unlike GDP (and such health measures as life expectancy), which are in principle indefinitely expandable.

There is a great deal of evidence that connects the proposed economic outcome variables with health, in the rich literature on the social determinants of health (see, e.g., Marmot 2010; Public Health England 2017; Braveman 2023). However, the evidence on mutually-adjusted causal relative risks is not yet available in sufficient detail to allow these calculations to be carried out.

In the case of subjective wellbeing, a number of suitable measures are available, and well studied, to assess wellbeing itself. They cover life satisfaction, positive and negative affect, and assessment of the meaningfulness of one’s life (eudaemonia) (OECD 2013). On the causal relationship between economic outcome indicators and subjective wellbeing, there is a growing literature (e.g. Boarini et al. 2012), but the coverage is rather limited at present (Frijters et al. 2020, Table 1). Much progress is being made on this agenda, e.g. by the Wellbeing Programme at LSE’s Centre for Economic Performance [CEP n.d.], which will greatly contribute to the development work required in this area.

The causal relationships in both these literatures are likely to be specific to a particular context, e.g. whether in a high-, medium- or low-income economy. It is therefore probably advisable for the relevant evidence to be considered separately for economies at different levels of prosperity—different “divisions” as suggested in Chapter 2.

The methods of data collection, and data quality issues, are not discussed in detail in the present book. Considerable experience has been gained, e.g. by the OECD and many national statistical agencies, with the measurement of most of these economic outcomes.

Some of the problems encountered with traditional methods of data collection are likely to apply here too. For example, survey response rates may be a problem for economic outcome indicators just as they are for existing measures, including GDP. On the other hand, innovative developments in data collection are likely to benefit the measurement of economic outcome indicators as they have for the more traditional work of statistical agencies. Such resourcefulness and innovatory capacity is demonstrated by statistical agencies’ initiatives in developing new methods, such as the use of income tax data, VAT receipts, web-scraped data, supermarket scanner data, credit card data and online data, e.g. a job search website to track the labour market impact of the coronavirus pandemic (Adrjan and Lydon 2020). The use of multiple methods of data collection for each indicator is advisable, as this would allow their findings to be combined with each other in a triangulation exercise.

This exposition has implicitly discussed the IEO and its component items as applying to the individual person. In practice, this is an underestimate of the consequences, because there are ramifications for the wider family.

A more complicated problem is that the various indicators of hardship tend to co-occur in the same individuals and households—clustering of disadvantage. This is true in a static sense, that a person with, say, an insecure livelihood is also more likely to have problems with housing quality, and less financial security with all its consequences. The health/wellbeing impact of each indicator needs to be adjusted for others that may be correlated with it, and careful attention to causal direction is essential.

It is also true in a dynamic sense: over time, one problem leads to others. Problems with access to childcare or social care can mean that the carer (usually a woman) has restricted work opportunities, which itself leads to further hardships. And lack of literacy leads to unemployment, low incomes, etc., which mediate its impact on life expectancy (Gilbert et al. 2018).

Dissecting out the specific causal processes in highly interconnected causal systems may require the further development of epidemiological methods that focus on upstream factors (“causes of causes”) and their mutual adjustment. This would involve mapping the causal relationships between the various economic outcomes, as an integral part of the statistical analysis. It can be done using causal diagrams with analysis of mediation and moderation, in “systems epidemiology” (Joffe et al. 2012).

The presence of circular causation adds further complication. Mutual causation may occur: personal resources and external conditions are not only causes of wellbeing; the degree of wellbeing also influences personal resources and external conditions (New Economics Foundation 2011, p. 13). The same is true of health. Notably, the causal complexity of homelessness involves mutual causation with economic inactivity as well as with mental health (and in some cases criminality)—reinforcing feedback loops (“vicious circles”). The use of the method of calculation described above implies that these cyclical relationships are ignored, potentially leading to underestimation in some cases. In addition, the IEO focus on health and wellbeing as the ultimate criteria means that the instrumental reason for promoting the economic outcomes—consequential gain—is ignored, again leading to underestimation.