BPA EnergySmart Grocer Program Evaluation

1. BPA EnergySmart Grocer Program Evaluation Deborah Swarts, Josh Arnold, Mary Klos, Roger Hill, Kevin Cooney, and Gary Cullen Summit Blue Consulting

2. Program Description • A BPA third-party program that promotes efficiency improvements in commercial refrigeration. Customers include: • National chain supermarkets • Independent grocers • Corner grocers • Specialty shops (ice cream, butchers, etc.) • Regional chain grocers • Convenience stores • Florists • Liquor stores • Restaurants

3. Measure Categories • Program Measures fall into three categories: • Deemed savings: The costs and savings for these measures have been deemed by the Regional Technical Forum (RTF). • Deemed calculated savings: The RTF has reviewed the EnergySmart Grocer software and determined that given reliable inputs, the software appropriately calculates savings. • Provisionally deemed savings: This designation means that the RTF has uncertainty in these measures and they require more research before a “deemed” status can be approved.

4. Measures Within Categories • Deemed savings measures • Auto-closures • CFLs and general lighting • Gaskets • PC Controls • Deemed calculated savings measures • Energy efficient cases • Doors • Efficient compressor – low temp • Electronically Commutated Motors • Air cooled condenser • Floating head controls • Refrigeration – VFD – Motors • Provisionally deemed measures • Case lighting • Anti sweat controls • Night covers • Vending machine controls

5. Program Savings Information by Year

6. Program Savings by Measure Type

7. Evaluation Objectives • Assess program operation and barriers • Assess opportunities for application to other third-party programs • Estimate measure and program realization rates • Assess the strengths/weaknesses of two evaluation approaches: • Billing analysis • Engineering analysis

8. Process EvaluationGoals • Develop a comprehensive understanding of the program • Identify programmatic or organizational barriers • Make recommendations for possible improvement • Assess opportunities and barriers for potential future third-party program administration

9. Process EvaluationMethodology • Review program materials, including logic model • Interview program stakeholders • Research commercial refrigeration programs for benchmarking and best practices • Submit early findings reports to BPA

10. Process EvaluationStakeholder Interviews • 36 Stakeholders Interviewed (February—April 2009) • Utility representatives • End-use customers • Trade Allies • PECI staff • BPA Energy Efficiency Representatives • BPA Program Staff • Interviews included questions about program roles, communication, delivery, and satisfaction

11. Process Evaluation Major Findings • Program found to be highly regarded by all parties, with some fine-tuning opportunities • There is a stakeholder perception of a need to clarify roles and responsibilities • Economic downturn has affected end-use customer uptake • The program’s use of one-on-one and personal visits (I2I) are critical to program success • Program success provides strong evidence to support adding more third-party programs

12. Process EvaluationRecommendations: Roles and Responsibilities • Review proposed logic model and discuss roles • Clarify communication channels among program stakeholders • Engage PECI to address administrative requirements • Review service agreements with utility representatives

13. Process Evaluation Recommendations: Marketing & Outreach • Field Energy Analysts (FEAs) and Utility Representatives are the most trusted source for program information. • To improve implementation even more: • Develop FEA presentation scripts • FEAs leverage local utility reps • FEAs identify local success stories • Create local case studies for different target markets • Utilize the local utility names and contacts

14. Process Evaluation: OpportunitiesFuture Third-Party Initiatives • History and Vision of BPA • Successful history of innovation provides favorable framework • Structure of BPA • The wide geographic range of both the BPA structure and member utilities along with the diverse population and socio-economic characteristics of the member utilities lend themselves to third-party initiatives • Third-party implementer expertise with current program • Access to Research & Resources • BPA’s internal expertise and access to other research organizations in the NW • Human Resources • BPA and member utilities have experienced staff

15. Process Evaluation: RecommendationsFuture Third-Party Initiatives • Engage BPA member utilities in a voluntary advisory group • Fully Develop “New Initiatives Checklist” • Implement pilot phase with review prior to “open enrollment” • Incorporate workplan development period into contract to leverage third-party implementer expertise

16. Impact Evaluation Approaches • Both an engineering and billing analysis are useful • Comparative results from the two are similar • Billing analysis: • Cannot assess small-scale impact measures • Can assess hard to separate multiple measures thatare installed together • Lower cost • On-site engineering analysis: • Can assess small impact loads • Some difficulty with hard to separate multiple measures • On-site verification of installed measures

17. Billing Analysis Methodology • Fixed-Effects Regression Approach • Pre-installation bills are compared to post-installation bills • Each customer serves as his or her own control • Fixed effects control for differences between customers • Statistically Adjusted Engineering Model • Savings estimates from tracking system were used • Coefficients represented realization rate on the savings

18. Billing Analysis Sample • Collecting billing data was a challenge • Many utilities were involved • Billing data was in a different format from each utility • Had to balance cost of data collection with value of additional cases • Final sample design covered 80% of savings by including 15 utilities

19. Billing Analysis Results • Three measures had both sufficient savings and a sufficient number of cases to estimate statistically significant realization rates at the 90% confidence level. 1.27 1.07 0.77

20. Engineering Analysis Methodology • Assess the EnergySmart Grocer Software • Spot measurements combined with logging of temperatures and current • Use of system data logs where available • Three weeks of monitoring for weather dependent measures • Comparison of results with previous studies commissioned by BPA • Specifications for installed systems used along with measured data

21. Assessment of the EnergySmart Grocer Software • Modeling logic is sound • Fundamental engineering principles with arrays of DOE 2.1 simulations • Quality input data is utilized by the model • User friendly interface • Reasonable results based on review of model inputs • The proprietary nature of the model algorithms leaves uncertainty on how the estimates are derived

22. Engineering Evaluation Sample • Goal was to focus on the largest savings in order to estimate program cost-effectiveness • Ten largest sites by deemed calculated savings were chosen • Represented 35% of program savings and 50% of non-deemed program savings • High efficiency cases and refrigeration controls made up the largest portion of both the sample and non-deemed program savings • Night covers and auto-closers were primary missing measures within the ten largest sites

23. Engineering Site Visit Information Measures not monitored were verified during site visits

24. Site-Monitoring Performed • Floating Head/Suction Pressure Controls and VFD Modulating Compressors and Condensers – nine sites • Pre-installation data not available • Spot measurements of temperature and power factors • Installed data loggers or acquired trend log data from the building automation and control system • Data collection covered minimum of 16 days • Electronically Commutated Motors – seven sites • Pre-installation data not available • On-site measurement of post-installation true power • Installed data loggers • Data collection covered minimum of 16 days

25. Site-Monitoring Performed (cont.) • Anti-Sweat Heater Controls and Low ASHC Doors – two sites • Spot measurements used to verify which breakers operated anti-sweat heaters, the supply voltage, and the range of switching • Multi-week logging employed • Long term weather data from nearest available station for weather dependent measures • The data logging was primarily of single phase current, but with some three phase logging also performed for comparison purposes • The spot measurements used Fluke 43B and Amprobe ACD-31P devices

26. Engineering Analysis Results • Overall program savings corresponded well with predicted savings • ECMs showed substantially higher savings than predicted • High efficiency low temperature reach in cases also exhibited high savings, primarily due to the inclusion of ECMs • VFDs contributed substantially less savings than predicted, probably due to the presence of staged controls previously

27. Summary of Measure Realization Rates • Measure group realization rates ranged from 0.77 for lighting to 3.12 for Electronically Commutated Motors • The overall program realization rate was found to be 1.15. • Recommendation to apply measure-level recommended realization rates to evaluated period. Make program and engineering adjustments in future to align RRs.

28. Lighting and Electronically Commutated Motor Realization Rates • For lighting, there did not appear to be any significant issues with the calculation methodology or assumptions. A few possible reasons for the realization rate include: • The effects of other measures and equipment (interactive effects) • Stores did not actually install the lamp and ballast combinations listed for the retrofit • Stocking hours have been extended since the lighting retrofit. • Emergency lights could be in use and operating 8,760 hours per year • Additional lights have been added since the retrofit • It is our assessment that existing motors are running at higher loads than anticipated by the GrocerySmart model

29. Program Cost-Effectiveness • Using the Total Resource Cost (TRC) test, the program was found to be cost effective with a TRC of 1.42 • Other cost-effectiveness test results in table below

30. Summary • The program level measure realization (1.15) was close to the ex-ante impact estimates. • Based on the TRC test, the program is cost effective (TRC=1.42) • The combination of a billing analysis and engineering analysis proved effective • Engineering approach best for measures with impacts that are relatively small compared to whole building loads • Billing analysis less costly and provides good results for measures with relatively large impacts compared to whole building loads • The program is well received among the member utilities

31. Discussion • Contact Information: • Gary Cullen: gcullen@summitblue.com, 360-718-8392, x 304 • Deborah Swarts: dswarts@summitblue.com, 360-314-6298, x 302