This is a guest post by Adam Scheer (Recurve), Jake Millette and Olivia Patterson (Opinion Dynamics), and Julie Michals (E4TheFuture)

Driven by advancing technologies and by policies that are evolving to both mitigate and adapt to climate change, the energy industry is changing at a breakneck pace. On the demand side, our fundamental challenge is moving beyond siloed programs into scaled demand flexibility to achieve states’ priorities such as grid resilience, resource adequacy, and decarbonization, amidst increasing electrification. Critically, scaling distributed energy resources (DERs) to meet a host of policy goals will require that we leverage limited ratepayer dollars to cultivate as much energy efficiency (EE) and other DER investment as possible. The question is: are cost-effectiveness (CE) testing practices developed decades ago adequate to guide our industry investments today? In our experience, legacy CE practices are inhibiting both innovative program designs and commonsense best practices for putting ratepayer dollars to optimal use.

With demand flexibility ultimately blurring the lines across traditionally siloed DER programs, a consistent valuation framework that appropriately and fairly accounts for relevant impacts is more important than ever.

We suggest three fundamental principles to govern a modern benefit-cost analysis (BCA) framework:

1. DERs should be evaluated using a common BCA framework.
2. Policy priorities should be valued to ensure alignment with legislative and regulatory objectives.
3. A cost-effectiveness test must ensure symmetry in treatment of costs and benefits.

The Total Resource Cost (TRC) test, as applied in many jurisdictions today, violates these commonsense principles [1]. However, a framework aligned with these principles – as provided in the National Standard Practice Manual (NSPM) – will help to avoid unintended barriers to investment in DERs. Such barriers include constraining programs from engaging markets and limiting opportunities to maximize private investment for the benefit of all ratepayers.

Below, we share common sources of injury caused by current BCA practices, with a focus on the asymmetry issue. We then offer a way forward, moving from bandage fixes for CE testing to the operating table. It’s time to stop treating the symptoms, and to address the underlying diseases.

Where We Are Now – Fighting the Asymmetry Disease

Beneficial electrification of both buildings and transportation has emerged as a near-consensus priority for long-term decarbonization. Now imagine if today’s cost-effectiveness policy were applied to an electric vehicle (EV) program that balanced a customer’s full $30,000 investment in an EV against only the carbon benefits sought by a state. This may seem like a technocratic comedy routine, but the consequences of such a CE test would stop any EV program in its tracks. Yet this is exactly how the TRC evaluates EE and demand-side interventions.   

As discussed in a recent whitepaper, by treating investment in a home or business equivalently to utility bill surcharges, the TRC, as largely applied in practice, inhibits demand-side programs from leveraging private capital that customers are willing to spend for a host of non-energy reasons. Consequently, program administrators are pushed into an ever-dwindling box of cheap widget-based measures via programs that are simply incapable of driving scalable decarbonization.

Because all participant costs are typically counted, but not the associated non-energy benefits, jurisdictions that use the TRC (or some modified versions) will face an unfortunate and unnecessary reality: the programs that can best achieve long-term resource and policy objectives at reasonable cost to ratepayers will be judged as “cost-ineffective” and will be non-competitive against other options.

Consider two EE programs from Pacific Gas & Electric’s (PG&E) 2017 portfolio [2]:

For every $1 of ratepayer funding, Advanced Home Upgrade (AHUP) drives $2.85 of private EE investment (net of free-ridership), while Energy Fitness is funded nearly exclusively by ratepayer surcharges. AHUP also generates $1.56 in grid and greenhouse gas benefits per program dollar, more than double that of Energy Fitness. Yet the AHUP TRC clocks in at just 0.40, and the Energy Fitness TRC is nearly 50% higher. 

The TRC troubles of AHUP originate from the cost assumption that all participant investment relates to energy benefits. This is no different than assigning the entire cost of an EV to a customer’s desire to save gas. In fact, recent studies detail that customers invest in EE for a variety of non-energy factors including increased comfort, home resale value, indoor air quality, and better health, among others [3, 4, 5].

The asymmetry wound commonly found in TRC practice is exemplified in this graphic illustration. Research shows that more than half of the value participants place on an energy-saving retrofit is associated with non-energy benefits [3]. This is no secret: program implementers and contractors, as well as equipment and technology manufacturers, commonly market non-energy benefits as primary selling points for energy efficient equipment.  

The asymmetrical treatment of costs and benefits especially injures market-based program models – the very direction we need to effectively scale DERs!

Now consider PG&E’s Non-Incentive On-Bill Financing (OBF-NI) program, which moves purposefully away from incentives funded by ratepayer surcharges. With OBF-NI the only costs to ratepayers are program administration and cost of capital through the loan cycle. Along with lowering costs for ratepayers, the program is also a big win for participants, who avoid up-front project costs and experience no change in their bill for the duration of loan repayment – since loan repayments are designed to be balanced by on-bill energy savings – yet benefit immediately from the facility improvements [6].

With benefits at a fraction of the program cost, the forecasted Program Administrator Cost (PAC, or sometimes referred to as the Utility Cost Test) benefit-cost ratio of OBF-NI is a whopping 7.9! [7] This demonstrates a program model that’s a win-win-win-win for ratepayers, participants, the utility, and regulators. Yet while the OBF-NI model eliminates opportunity costs for participants via the on-bill loan structure, the “broken” TRC counts all project costs, resulting in only a moderately cost-effective program with B/C ratio of 1.6. This shows the significant effect of asymmetrical treatment of costs and benefits. If not remedied, the OBF-NI program faces the same challenge as many other programs in scaling or driving deeper savings.

Financing programs are just one example of a modern, market-based design that is inappropriately balanced within a typical TRC framework. Another area of asymmetry is often apparent in demand-side programs that jurisdictions utilize to achieve a host of non-energy policy objectives (e.g., serving low-income customers and disadvantaged communities, providing workforce education/training, supporting emerging technologies, driving market transformation). If benefits are not assigned, these priorities will either be marginalized or pose a threat to the cost-effectiveness of a full resource portfolio. Similarly, if a jurisdiction’s policies articulate that DER investments should help to achieve, for example, environmental goals, public health benefits and/or economic development, then these associated impacts should be accounted for. Here, the issue of symmetry is also one of common sense: how can a policy mandate have no value? 

These examples highlight the need for symmetry in BCA, the ability to enable modern, innovative program designs, and the need to value policy priorities. What, then, are the remedies?

Modernizing CE Policy Today can Lead to a Healthier DER Industry Tomorrow

The solution to modernizing how we conduct BCA begins with addressing the asymmetry disease, in particular as it applies to participant impacts. In general, a jurisdiction can choose among three cures:

• Account for the full range of participant costs and benefits (energy and non-energy).
• If participant non-energy benefits are excluded, then exclude the corresponding costs.
• Remove participant impacts (both costs and benefits) altogether from the BCA.

In the first option, if a jurisdiction’s policies explicitly require inclusion of participant impacts in their BCA, then both costs and benefits should be included to ensure symmetry, even those non-energy benefits that are hard to quantify. Importantly, this approach is not about adding non-energy benefits in order to make EE programs cost-effective (e.g., in a landscape of lighting standards and decreasing natural gas prices) – as viewed by some in the industry. Rather, this is about applying a fundamental principle that is economically sound and avoids biased or skewed valuation of DERs. 

But often jurisdictions struggle to agree not only on which non-energy benefits to include, but also on what value to apply. Too often this leads to asymmetry simply by inaction. As such, a second option to achieving symmetry is to exclude non-energy related participant costs from the denominator of the benefit-cost equation. This approach may engender less controversy, as there is a clear limit to the adjustment and isolated terms (program and measure costs) being addressed. Further, this approach maintains a resource focus on benefits, an attractive feature to jurisdictions seeking a least-cost combined resource portfolio of supply and demand side resources. However, this approach still requires estimating some portion of participant costs to eliminate (that which reflects the ‘value’ of the non-energy benefits).    

A third option, possibly simpler, is to remove participant impacts altogether. In this case, the test effectively trusts customers to make a participation decision based on however they value the benefits.  Importantly, this option may or may not lead to applying a PAC test in place of the TRC, as this will depend on whether other impacts should be accounted for as articulated by policy goals (e.g., carbon reduction, economic development, improved public health). The NSPM provides pros and cons of accounting for participant impacts.

Wait, why are we investing in DERs?

Improving BCA practices requires that a jurisdiction reflect on why it’s investing in DERs. As the NSPM sets forth, this regulatory perspective is a departure from traditional CE tests (e.g., PAC, TRC, Societal Cost test), and offers the flexibility to develop a test that aligns with the policies specific to the jurisdiction. Along with addressing symmetry, a jurisdiction is likely to make other important changes to its BCA practice.

In fact, we’re already seeing some states taking the fractures of asymmetry and misalignment with policies to the operating table. Several states have applied the NSPM to test their test, as described in case studies. A first step taken by these states is a comprehensive review and documentation of relevant policies supporting investment in EE resources.

In New Hampshire, a recent stakeholder-based report proposes excluding participant impacts from the primary CE test, while accounting for carbon reduction goals as articulated by state policy. A similar proposal has been made in Minnesota. In Arkansas, a commission order directs stakeholders to review treatment of participant impacts to address asymmetry issues found through a stakeholder NSPM review process. In Rhode Island, the state expanded its BCA framework to all DERs and applied the NSPM principles.

Earlier, we shared a definitive example of how the application of symmetry – or its misapplication – can lead to unintended consequences. States applying the NSPM are learning to avoid this mistake and are instead modifying their current practice to align with the fundamental NSPM principles. And to the extent that states aim to ensure symmetry of participant impacts, existing impact evaluation processes (such as those cited herein), can be used to collect data on customer energy and non-energy costs.

With rapid DER deployment under way, it’s essential to use a BCA framework that integrates across DER resources. This requires accounting for underlying policy objectives, as well as gauging the decision-making process as customers consider a program’s full (energy and non-energy) value proposition. DERs should be valued to account for their interaction (e.g., what is the value of efficiency with demand response, storage, PV, or some combination?), which are often operational drivers beyond energy benefits. Further, with customer adoption of resources outpacing program development, research is needed today on both benefits and costs to scale up promising pilots.

Moving to a Common BCA Framework for DERs

As integration of DERs becomes increasingly common, the BCA framework must allow for the benefits and costs of all DERs to be compared on a level playing field. In response to the fast-moving changes in our industry, the original National Standard Practice Manual for Assessing Cost-Effectiveness of Energy Efficiency Resources (NSPM for EE) is being expanded to broaden the common BCA framework to a range of DERs including efficiency, demand response, distributed generation, storage and electrification. National Standard Practice Manual for Benefit-Cost Analysis of Distributed Energy Resources (NSPM for DERs) will address key considerations to help regulators and stakeholders navigate the complexities of DER interaction for technology specific, multiple and integrated DER investments, including temporal and locational applications.

This new NSPM for DERs is building on best practices of existing BCA while mitigating the unintended consequences of siloed frameworks. Core to its framework are the fundamental principles of alignment with state policies, consistency in BCA analyses for all DERs including methods and assumptions, and symmetrical treatment of costs and benefits.

If our industry is to successfully scale DER deployment at a sufficiently rapid pace to meet a host of important goals including grid flexibility, resilience, decarbonization, and reducing energy burden for low/moderate income customers, along with goals such as improved air quality and public health and economic development, we will need private investment in DERs, and a lot of it. To get there, we must get CE testing right. And fortunately, solutions such as those outlined herein can make cost-effectiveness a judicious arbiter of resource deployment.

It’s time to put aside that old first aid kit, and surgically alter the 1990s evaluation paradigm. Just as medical interventions have drastically changed based on learning, new technologies, and improved practices, so should cost-effectiveness evaluation be updated.

To address dramatic changes in the energy landscape, we need appropriate valuation to ensure solutions are designed to support grid and customer objectives, rather than prop up a broken TRC. Absent this paradigm shift to modernize CE testing policies, the industry risks remaining on life support.

We all want to contribute to healthier industry that thrives and achieves state policy priorities. So, let’s be proactive, and rally around commonsense principles and bring new tools to the operating table.

[1] The Database on State Efficiency Screening Practices shows how states often include all participant costs but no/few participant benefits in the application of their TRC test.

[2] Values derived from PG&E’s 2017 program year CEDARS filing.

[3] Impact Evaluation Report: Home Upgrade Program – Residential Program Year 2017, DNV GL, 2019.

[4] PG&E Whole House Program: Marketing and Targeting Analysis. Opinion Dynamics Corporation, 2014. CALMAC ID: PGE0302.05

[5] Energy Upgrade California – Home Upgrade Program Process Evaluation 2014-2015 EMI Consulting, 2015. CALMAC ID: PGE0389.01

[6] California 2010 – 2012 On-Bill Financing Process Evaluation and Market Assessment, Cadmus, 2012. CALMAC ID# CPU0056.01

[7] Second Supplement: PG&E’s 2019 Energy Efficiency Annual Budget Advice Letter in Compliance with Decisions 15-10-028 and 18-05-041