Advice for the “nudge unit” – remember rebound effects and design pilots carefully

This is a guest post by Paul Dolan, Professor in the Department of Social Policy at the London School of Economics. It was first published on the LSE’s British Policy and Politics blog.

As an author of the MINDSPACE report and a former member of the Behavioural Insights Team (BIT) in the Cabinet Office, I read the recent BIT report on Behaviour Change and Energy Use. I think it represents a great improvement on the first report on health. But I would like to draw attention to three substantive issues.

The most basic issue relates to compensating behaviours and rebound effects. Remarkably, neither is discussed at all in the report. Yet we know from some (admittedly not entirely robust) evidence that about half of the benefits in energy saving are offset elsewhere. And this is without looking too hard at the other possible offsetting effects, including ‘licencing’ effects. We are gathering evidence across a range of contexts (admittedly, largely from lab studies) that these effects could potentially more than offset all of the gains from intervention. The evidence is flimsy but the issues are important and should be made explicit in the report. Drawing attention to such issues will help in the design of the field studies, which should seek to capture as many spillover effects as possible.

The second issue relates to experimental design. I am delighted that the BIT are planning some field studies. These could be flagship studies so they should be designed as robustly as possible. But there are some concerns. In the social networks study, were Kingston and Merton truly randomised? How will the design mitigate for John Henry and Hawthorne effects? And can the study expect to establish any long term equilibrium effects from a three month study? Overall, for the studies described, there is little discussion of time frames of analyses, the sustainability of effects, and the threats to both internal and external validity. The BIT should also make clear how they will randomise and seek to avoid self-selection effects, as these are obviously crucial to how we interpret the results.

The third issue relates to the quality of evidence. In the collective rewards study, what is the rationale for ignoring robust hedonic price regression analysis in favour of surveys that lack any degree of external (and probably internal) validity? Similarly, for the Energy Performance Certificates (EPC), why care about what people say when house prices data tell you what people do in relation to the EPC (I suspect that 18% claiming that EPC’s influence their house buying decisions is an exaggeration of the real effect).

2 comments

  • Paul Dolan makes important points, especially about the rebound effect. This is a major issue, so far neglected by policymakers. At the Universities of Surrey and Sussex my colleagues Steve Sorrell, Mona Chitnis, Angela Druckman and Tim Jackson are working on detailed modelling of rebound effects in relation to household consumption. This work builds on analysis from the University of Surrey’s RESOLVE programme on sustainable lifestyles and consumption, and is part of a new DEFRA-funded initiative, the Sustainable Lifestyles Research Group (SLRG), which runs from 2010 to 2013.
    So far the team has developed preliminary estimates of the rebound effects following five types of energy efficiency improvement by an average UK household : cavity wall insulation, loft insulation, tank insulation, energy efficient lighting, fuel efficient vehicles and solar thermal. The rebound effects have been estimated over different time periods taking into account anticipated changes in household income and the GHG intensity of electricity generation. These findings were presented at the IAEE International Conference in Stockholm in June 2011. Work in progress includes analysis of rebound effects by income group.
    This project is at an early stage but it is clear that rebound effects are significant and complex, have been overlooked and urgently need to be taken into account in policy design.
    The team has published initial results from their rebound effect modelling in this recent journal paper:
    Druckman, A.; Chitnis, M.; Sorrell, S.; Jackson, T. (2011), ‘Missing carbon reductions: exploring rebound and backfire effects in UK households’, Energy Policy , 39, 3572-3581.

  • I concur with the comments made here about the rebound effect. I would also add that the BIT report on Behaviour Change and Energy Use makes the same error that I see being made by several players in this field, namely: its fixation on “Information” (the word is used over forty times in the 35-page document).

    As someone developing a practical approach to the problem of behaviour change and energy use, my research in this area has led me to conclude that “Consumers need Motivation, not Information.”

    The word “motivation” however, appears only once in the BIT report.

    To paraphrase the (very) late Mr W. Stanley Jevons, this surely represents “a confusion of ideas”.

    Read more here: http://welectricity.com/blog/31

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s