Increasing response rates and improving research design: Learnings from the Smart Energy Research Lab in the United Kingdom

Abstract

Obtaining high-resolution energy consumption data from a large, representative sample of homes is critical for research, but low response rates, sample bias and high recruitment costs form substantial barriers. The widespread installation of smart meters offers a novel route to access such data, but in countries like Great Britain (GB) consent is required from each household; a real barrier to large-scale sampling. In this paper we show how certain study design choices can impact the response rate for energy studies requesting access to half-hourly smart meter data and (optional) survey completion. We used a randomised control trial (RCT) with a 3×2×2 factorial design; 3 (including none) incentive groups ×2 message content/structures ×2 ‘push-to-web’ treatment groups. Up to 4 mailings (letters) were sent to 18,000 addresses, recruiting 1711 participants (9.5% response rate) in England and Wales. The most effective strategy offered a conditional £5 voucher and postal response options in multiple mailings (compared to only once in the push-to-web approach, although at the expense of far fewer online signups). Motivational headlines and message structure were also found to be influential. Reminders increased response but a 4th mailing was not cost effective. Our results and recommendations can be used to help future energy studies to achieve greater response rates and improved representation. UK-based researchers can apply to use our longitudinal smart meter and contextual datasets.

Publication
Energy Research & Social Science 83 (2022) 102312

Obtaining high-resolution energy consumption data from a large, representative sample of homes is critical for research, but low response rates, sample bias and high recruitment costs form substantial barriers. The widespread installation of smart meters offers a novel route to access such data, but in countries like Great Britain (GB) consent is required from each household; a real barrier to large-scale sampling. In this paper we show how certain study design choices can impact the response rate for energy studies requesting access to half-hourly smart meter data and (optional) survey completion. We used a randomised control trial (RCT) with a 3×2×2 factorial design; 3 (including none) incentive groups ×2 message content/structures ×2 ‘push-to-web’ treatment groups. Up to 4 mailings (letters) were sent to 18,000 addresses, recruiting 1711 participants (9.5% response rate) in England and Wales. The most effective strategy offered a conditional £5 voucher and postal response options in multiple mailings (compared to only once in the push-to-web approach, although at the expense of far fewer online signups). Motivational headlines and message structure were also found to be influential. Reminders increased response but a 4th mailing was not cost effective. Our results and recommendations can be used to help future energy studies to achieve greater response rates and improved representation. UK-based researchers can apply to use our longitudinal smart meter and contextual datasets.

Highlights:

  • We test recruitment strategies to improve response to a smart energy meter study.
  • A conditional monetary incentive increases response.
  • A push-to-web approach reduces response but significantly increases online sign up.
  • Multiple reminders are useful but a 4th mailing is unlikely to be cost-effective.
  • Motivational headlines and message structure impact response rates.
comments powered by Disqus