Testing Framework

A large number of tests are automatically run for every code change in the GitHub repository.

The current set of tests include:

  • Successful ERI calculations for all sample files

  • RESNET® HERS® tests (ASHRAE 140, HVAC tests, etc.)

  • EPA ENERGY STAR tests

If you are seeking to develop RESNET Accredited Rating Software, you will need to submit your final software product to RESNET for accreditation.

Running Tests Locally

All RESNET HERS tests can be run using:

openstudio resnet_hers_test.rb

All EPA ENERGY STAR & Zero Energy Ready Homes tests can be run using:

openstudio es_zerh_test.rb

Or individual tests can be run by specifying the name of the test. A couple examples:

openstudio resnet_hers_test.rb --name=test_resnet_ashrae_140
openstudio es_zerh_test.rb --name=test_epa

Test results in CSV format are created at workflow/tests/test_results. For many RESNET tests, the Excel spreadsheet test criteria are also implemented in code to automate the process of checking for test failures. All simulation/HPXML/etc. files generated from running the tests can be found inside the workflow/tests/test_files directory.

At the completion of the test, there will also be output that denotes the number of failures/errors like so:

Finished in 36.067116s, 0.0277 runs/s, 0.9704 assertions/s. 1 runs, 35 assertions, 0 failures, 0 errors, 0 skips

Software developers may find it convenient to export HPXML files with the same name as the test files included in the repository. This allows issuing the same commands above to generate test results.

Official Test Results

The official OpenStudio-ERI test results can be found in any release or any checkout of the code at workflow/tests/base_results. The results are based on using the HPXML files found under workflow/tests.