Skip to content

Index Variable.measure_id to keep analysis creation fast - #894

Merged
brianlball merged 1 commit into
developfrom
fix/variable-measure-id-index
Nov 2, 2026
Merged

brianlball merged 1 commit into
developfrom
fix/variable-measure-id-index

Conversation

@anchapin

@anchapin anchapin commented Nov 2, 2026

Copy link
Copy Markdown
Contributor

Why do we need this PR

POST /projects/:id/analyses.json gets slower with each analysis on the server. In a 179D run of 3648 batches, it went from about 14 s to about 90 s per batch by batch 1560.

Core change explanation

Measure.create_from_os_json calls measure.variables.include? for each measure argument. That runs Variable.where(measure_id: ...), but variables has no index on measure_id, so each call scans the whole collection (COLLSCAN, 441 ms with 601k variables). This PR adds index: true to belongs_to :measure.

Measured on the live server after the same index was added by hand: the query uses IXSCAN and takes 2 ms, and the request takes about 5 s.

Deploy note: existing databases get the index from rake db:mongoid:create_indexes.

Which files are affected

  • server/app/models/variable.rb
  • server/spec/models/variable_spec.rb (new)

What kind of tests are done

  • New model spec that checks the measure_id index is declared. Not run locally (needs Rails and Mongo); the same index_specifications check ran in the production web pod with rails runner.
  • Live check: explain on the server went from COLLSCAN (441 ms) to IXSCAN (2 ms).

Measure.create_from_os_json calls measure.variables.include? for each measure
argument. Without an index on measure_id, each call scans the whole variables
collection, so creating an analysis gets slower as analyses build up. With
601k variables, POST /projects/:id/analyses.json took about 90 s; with the
index it takes about 5 s.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The new regression spec is not included in CI, so removing the index would go undetected.

1 open issue
What changed in this PR

Adds a Mongoid index for measure-to-variable lookups to keep analysis creation fast as the variables collection grows.

Changes:

  • Declares an index on Variable.measure_id while preserving the optional association.
  • Adds a model spec that checks the index declaration.
File Description
server/​spec/​models/​variable_spec.rb Checks that an ascending measure_id index is declared.
server/​app/​models/​variable.rb Enables indexing on the measure association.

Review effort: Max

Comment thread server/spec/models/variable_spec.rb Resolved
@brianlball
brianlball merged commit b47cdce into develop Nov 2, 2026
6 checks passed
@brianlball
brianlball deleted the fix/variable-measure-id-index branch November 2, 2026 20:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants