From 130ab8af83bafecb9ef4069c97224cb2d50e9c35 Mon Sep 17 00:00:00 2001 From: JieXiong9119 <69597597+JieXiong9119@users.noreply.github.com> Date: Wed, 11 Nov 2026 13:04:13 -0700 Subject: [PATCH] generalize baseline/scenario osw runner --- docs/scenario_workflow_translation_plan.md | 32 +++-- lib/BOSS/boss.rb | 42 ++++++- spec/tests/unit/boss_spec.rb | 136 +++++++++++++++++++++ 3 files changed, 198 insertions(+), 12 deletions(-) diff --git a/docs/scenario_workflow_translation_plan.md b/docs/scenario_workflow_translation_plan.md index 9857e0e..b1a2f6e 100644 --- a/docs/scenario_workflow_translation_plan.md +++ b/docs/scenario_workflow_translation_plan.md @@ -247,25 +247,25 @@ Use this roadmap as the tracker. Keep `Status` and `Delivered by` current when a ### PR 13: Generic OSW Step Helpers -- Status: In Progress -- Delivered by: `scenario-workflow-13-generic-osw-step-helpers` branch +- Status: Done +- Delivered by: PR #21, merged into `scenario-workflow` - Scope: Centralize mapped step appending and `OpenStudio::Extension.set_measure_argument` use. - Done when: Tests prove boolean, numeric, and string arguments serialize correctly. - Avoid: Broad rewrites of baseline populator methods or unintended baseline JSON changes. ### PR 14: Generalized Runner -- Status: Planned -- Delivered by: - -- Scope: Run `baseline/in.osw` and `scenarios/**/in.osw` while keeping `run_baseline_osw` intact. +- Status: In Progress +- Delivered by: `scenario-workflow-14-generalized-runner` branch +- Scope: Run `baseline/in.osw` and the `:written` scenario OSWs from a `write_baseline_and_scenario_osws` result while keeping `run_baseline_osw` intact. The writer result, not a `scenarios/**/in.osw` scan, selects scenarios, because the writer does not remove stale OSWs. - Done when: Tests or smoke runs show the correct OSW list and baseline-only compatibility. -- Avoid: Running skipped or missing OSWs, or removing baseline-only behavior. +- Avoid: Running skipped, stale, or missing OSWs, or removing baseline-only behavior. ### PR 15: CLI Command - Status: Planned - Delivered by: - -- Scope: Add a Thor command for baseline plus package scenarios using existing output, weather, standard, and run options. +- Scope: Add a Thor command for baseline plus package scenarios using existing output, weather, standard, and run options. Run mode passes the writer result to `run_baseline_and_scenario_osws`. - Done when: CLI help documents the command, write-only mode creates OSWs, and run mode invokes the generalized runner. - Avoid: Changing existing `write_baseline_osw` or `run_osw` behavior. @@ -447,8 +447,8 @@ Use this section for stable facts future PRs need. Keep entries short and tied t ### PR 13: Generic OSW Step Helpers -- Status: In Progress -- Delivered by: `scenario-workflow-13-generic-osw-step-helpers` branch +- Status: Done +- Delivered by: PR #21, merged into `scenario-workflow` - Handoff: `OSWArgPopulator.append_step(osw, step, before_measure: nil)` deep-copies a complete step, adds an empty `arguments` hash if missing, and inserts it before the first `before_measure` step (or appends). It returns the inserted step and never adds `name` or default arguments. - Handoff: `OSWArgPopulator.set_step_argument(step, name, value)` is the only `OpenStudio::Extension.set_measure_argument` call site. It passes a one-step workflow, so other steps with the same `measure_dir_name` (baseline `SetLightingLoadsByLPD`, repeated `ReduceElectricEquipmentLoadsByPercentage`) are untouched. Keys are symbolized by the extension; values are not coerced. - Handoff: The eight baseline populators only swap their append/lambda setup for `step_argument_setter`; calculations, defaults, logging, and argument order are unchanged. `build_scenario_osw` appends prebuilt mapped steps through `append_step`; mapped arguments still do not go through the setter. @@ -456,6 +456,20 @@ Use this section for stable facts future PRs need. Keep entries short and tied t - Verification: `bundle exec rubocop` on the changed lib files drops from 156 to 100 offenses; none are new. - Not run: Baseline simulations (`write_and_run_osws_spec.rb`); generated baseline JSON is pinned by unit specs. +### PR 14: Generalized Runner + +- Status: In Progress +- Delivered by: `scenario-workflow-14-generalized-runner` branch +- Handoff: `BOSS::Boss.run_baseline_and_scenario_osws(write_result)` takes the symbol-keyed result of `write_baseline_and_scenario_osws`. It submits the baseline first, then every `status: :written` scenario `osw_path` in result order, deduplicated, in one `Runner#run_osws` call. It returns the runner's failure-message array; runner exceptions propagate. +- Handoff: Skipped scenarios never run, even if an older `in.osw` exists. Scenario OSWs on disk but absent from the result are ignored; no directory is scanned. A `:written` entry whose path is missing, blank, or not a file is skipped with a `BOSS.BOSS_logger` warning. +- Handoff: A malformed result raises `ArgumentError`; a missing baseline file logs an OpenStudio error and raises `StandardError`. Neither builds a runner. The input is not mutated. +- Handoff: Baseline-first is submission order only. The runner still runs in parallel with the existing options, and scenarios do not wait on baseline success; each scenario OSW contains its own baseline construction steps. +- Handoff: `run_baseline_osw` keeps its signature, error, runner options, and return value; both runners share the private `run_osw_paths`. The CLI is unchanged. +- Verification: `C:\Ruby32-x64\bin\ruby.exe -S bundle exec rspec spec/tests/unit/boss_spec.rb` - 33 examples, 0 failures. +- Verification: `C:\Ruby32-x64\bin\ruby.exe -S bundle exec rspec spec/tests/unit spec/tests/integration/scenario_osw_generation_spec.rb` - 145 examples, 0 failures. +- Verification: `bundle exec rubocop lib/BOSS/boss.rb spec/tests/unit/boss_spec.rb` adds only the existing `class << self` indentation pattern for the new public and private methods. +- Not run: Baseline and scenario simulations. The runner is mocked in unit specs; scenario simulations are PR 18 scope. + ## Verification Commands Use the narrowest command that proves the current PR. Broaden only when the change justifies it. diff --git a/lib/BOSS/boss.rb b/lib/BOSS/boss.rb index c5c2e60..cabeb5f 100644 --- a/lib/BOSS/boss.rb +++ b/lib/BOSS/boss.rb @@ -146,14 +146,50 @@ def run_baseline_osw(osw_dir) raise StandardError, "BOSS.Boss.run_baseline_osw: #{error_message}" end - runner = OpenStudio::Extension::Runner.new(dirname = Dir.pwd, bundle_without = [], options = { run_simulations: true, verbose: false, num_parallel: 7, max_to_run: Float::INFINITY }) + run_osw_paths([baseline_osw_path]) + end + + # Runs baseline/in.osw plus every :written scenario OSW from a + # write_baseline_and_scenario_osws result. The result, not the output directory, + # decides which scenarios run, so stale OSWs from earlier writes are never picked up. + # + # @param write_result [Hash] return value of write_baseline_and_scenario_osws + # @return [Array] runner failure messages; empty when every OSW succeeded + def run_baseline_and_scenario_osws(write_result) + baseline_osw_path = write_result.is_a?(Hash) ? write_result[:baseline_osw_path] : nil + scenario_results = write_result.is_a?(Hash) ? write_result[:scenario_results] : nil + if !baseline_osw_path.is_a?(String) || baseline_osw_path.strip.empty? || + !scenario_results.is_a?(Array) || !scenario_results.all?(Hash) + raise ArgumentError, 'Expected the result of write_baseline_and_scenario_osws with ' \ + ':baseline_osw_path and :scenario_results.' + end + + if !File.file?(baseline_osw_path) + error_message = "this function requires #{baseline_osw_path}, which does not exist. " \ + 'Create it with `write_baseline_and_scenario_osws` and try again.' + OpenStudio.logFree(OpenStudio::Error, 'BOSS.Boss.run_baseline_and_scenario_osws', error_message) + raise StandardError, "BOSS.Boss.run_baseline_and_scenario_osws: #{error_message}" + end + + scenario_osw_paths = scenario_results.select { |scenario| scenario[:status] == :written }.filter_map do |scenario| + path = scenario[:osw_path] + next path if path.is_a?(String) && File.file?(path) + + BOSS.BOSS_logger.warn('BOSS.Boss: Scenario OSW not found; scenario will not run ' \ + "(scenario_id: #{scenario[:scenario_id].inspect}, osw_path: #{path.inspect})") + nil + end - # run the baseline osm - return runner.run_osws([baseline_osw_path]) + run_osw_paths(([baseline_osw_path] + scenario_osw_paths).uniq) end private + def run_osw_paths(osw_paths) + runner = OpenStudio::Extension::Runner.new(Dir.pwd, [], { run_simulations: true, verbose: false, num_parallel: 7, max_to_run: Float::INFINITY }) + runner.run_osws(osw_paths) + end + # Returns a new, unsaved baseline OSW hash for the given reader. def build_baseline_osw(bsync_reader) # start with an empty baseline workflow diff --git a/spec/tests/unit/boss_spec.rb b/spec/tests/unit/boss_spec.rb index 72141a7..2b54652 100644 --- a/spec/tests/unit/boss_spec.rb +++ b/spec/tests/unit/boss_spec.rb @@ -541,4 +541,140 @@ def read_osw(path) ) end end + + describe 'running OSWs' do + let(:runner) { instance_double(OpenStudio::Extension::Runner, run_osws: []) } + let(:runner_options) { { run_simulations: true, verbose: false, num_parallel: 7, max_to_run: Float::INFINITY } } + + before do + allow(OpenStudio::Extension::Runner).to receive(:new).and_return(runner) + allow(BOSS.BOSS_logger).to receive(:warn) + allow(OpenStudio).to receive(:logFree) + end + + def touch_osw(*parts) + path = File.join(*parts, 'in.osw') + FileUtils.mkdir_p(File.dirname(path)) + File.write(path, '{}') + path + end + + def scenario(id, status, osw_path) + { scenario_id: id, status:, osw_path:, warnings: [] } + end + + describe '.run_baseline_osw' do + it 'runs only baseline/in.osw with the existing runner options and returns failures' do + Dir.mktmpdir do |dir| + baseline = touch_osw(dir, 'baseline') + touch_osw(dir, 'scenarios', 'Scenario1') + allow(runner).to receive(:run_osws).and_return(['failed']) + + expect(described_class.run_baseline_osw(dir)).to eq ['failed'] + expect(OpenStudio::Extension::Runner).to have_received(:new).with(Dir.pwd, [], runner_options) + expect(runner).to have_received(:run_osws).with([baseline]) + end + end + + it 'raises the existing error without building a runner when the baseline is missing' do + Dir.mktmpdir do |dir| + expect { described_class.run_baseline_osw(dir) } + .to raise_error(StandardError, /BOSS\.Boss\.run_baseline_osw: .*write_baseline_osw/) + expect(OpenStudio::Extension::Runner).not_to have_received(:new) + end + end + end + + describe '.run_baseline_and_scenario_osws' do + it 'is public with one argument' do + expect(described_class.method(:run_baseline_and_scenario_osws).arity).to eq 1 + expect(described_class.private_methods).to include(:run_osw_paths) + end + + it 'runs baseline then written scenarios in result order, skipping stale, missing, and duplicate paths' do + Dir.mktmpdir do |dir| + baseline = touch_osw(dir, 'baseline') + s2 = touch_osw(dir, 'scenarios', 'Scenario 2') + s1 = touch_osw(dir, 'scenarios', 'Scenario1') + stale = touch_osw(dir, 'scenarios', 'Stale') + touch_osw(dir, 'scenarios', 'Unlisted') + result = { + baseline_osw_path: baseline, + scenario_results: [ + scenario('Scenario 2', :written, s2), scenario('Stale', :skipped, stale), + scenario('Scenario1', :written, s1), scenario('Dup', :written, s1), + scenario('Gone', :written, File.join(dir, 'scenarios', 'Gone', 'in.osw')), + scenario('Nil', :written, nil), scenario('Dir', :written, File.join(dir, 'scenarios')) + ], + parser_warnings: [] + } + before = Marshal.load(Marshal.dump(result)) + + expect(described_class.run_baseline_and_scenario_osws(result)).to eq [] + expect(runner).to have_received(:run_osws).with([baseline, s2, s1]) + expect(BOSS.BOSS_logger).to have_received(:warn).with(/Gone/) + expect(BOSS.BOSS_logger).to have_received(:warn).with(/Nil/) + expect(BOSS.BOSS_logger).to have_received(:warn).with(/Dir/) + expect(result).to eq before + end + end + + it 'runs only the baseline when no scenario was written' do + Dir.mktmpdir do |dir| + baseline = touch_osw(dir, 'baseline') + stale = touch_osw(dir, 'scenarios', 'Scenario4') + described_class.run_baseline_and_scenario_osws( + baseline_osw_path: baseline, scenario_results: [scenario('Scenario4', :skipped, stale)] + ) + expect(runner).to have_received(:run_osws).with([baseline]) + end + end + + it 'raises before building a runner when the baseline file is missing' do + Dir.mktmpdir do |dir| + s1 = touch_osw(dir, 'scenarios', 'Scenario1') + result = { baseline_osw_path: File.join(dir, 'baseline', 'in.osw'), scenario_results: [scenario('Scenario1', :written, s1)] } + + expect { described_class.run_baseline_and_scenario_osws(result) } + .to raise_error(StandardError, /run_baseline_and_scenario_osws: .*does not exist/) + expect(OpenStudio::Extension::Runner).not_to have_received(:new) + end + end + + it 'rejects malformed write results' do + [nil, {}, { baseline_osw_path: ' ', scenario_results: [] }, { baseline_osw_path: 'x', scenario_results: [nil] }].each do |bad| + expect { described_class.run_baseline_and_scenario_osws(bad) }.to raise_error(ArgumentError) + end + expect(OpenStudio::Extension::Runner).not_to have_received(:new) + end + + it 'propagates runner failures and exceptions' do + Dir.mktmpdir do |dir| + result = { baseline_osw_path: touch_osw(dir, 'baseline'), scenario_results: [] } + allow(runner).to receive(:run_osws).and_raise(RuntimeError, 'boom') + expect { described_class.run_baseline_and_scenario_osws(result) }.to raise_error(RuntimeError, 'boom') + end + end + + it 'runs exactly the baseline and 12 written building_151 scenarios from a real write' do + stub_measure_dirs(local: [LOCAL_MEASURES_DIR], external: [external_measure_dir], discovered: [gem_measure_dir]) + allow(BOSS::ExternalMeasureRepoManager.new).to receive(:resolved_measure_directories) + .with(include_scenario_only: true).and_return([external_measure_dir]) + allow(described_class).to receive(:check_scenario_measures_exist) + + Dir.mktmpdir do |dir| + stale = touch_osw(dir, 'scenarios', 'Scenario4') + unlisted = touch_osw(dir, 'scenarios', 'Unlisted') + result = described_class.write_baseline_and_scenario_osws(xml_path, dir, openstudio_weather_file_path, ASHRAE90_1) + written = result[:scenario_results].select { |s| s[:status] == :written }.map { |s| s[:osw_path] } + + described_class.run_baseline_and_scenario_osws(result) + + expect(written.length).to eq 12 + expect(runner).to have_received(:run_osws).with([result[:baseline_osw_path], *written]) + expect(written).not_to include(stale, unlisted) + end + end + end + end end