Repository navigation
Index Variable.measure_id to keep analysis creation fast - #894
Merged
Merged
Conversation
Measure.create_from_os_json calls measure.variables.include? for each measure argument. Without an index on measure_id, each call scans the whole variables collection, so creating an analysis gets slower as analyses build up. With 601k variables, POST /projects/:id/analyses.json took about 90 s; with the index it takes about 5 s.
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
The new regression spec is not included in CI, so removing the index would go undetected.
1 open issue
What changed in this PR
Adds a Mongoid index for measure-to-variable lookups to keep analysis creation fast as the variables collection grows.
Changes:
- Declares an index on
Variable.measure_idwhile preserving the optional association. - Adds a model spec that checks the index declaration.
| File | Description |
|---|---|
server/spec/models/variable_spec.rb |
Checks that an ascending measure_id index is declared. |
server/app/models/variable.rb |
Enables indexing on the measure association. |
Review effort: Max
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

Why do we need this PR
POST /projects/:id/analyses.jsongets slower with each analysis on the server. In a 179D run of 3648 batches, it went from about 14 s to about 90 s per batch by batch 1560.Core change explanation
Measure.create_from_os_jsoncallsmeasure.variables.include?for each measure argument. That runsVariable.where(measure_id: ...), butvariableshas no index onmeasure_id, so each call scans the whole collection (COLLSCAN, 441 ms with 601k variables). This PR addsindex: truetobelongs_to :measure.Measured on the live server after the same index was added by hand: the query uses
IXSCANand takes 2 ms, and the request takes about 5 s.Deploy note: existing databases get the index from
rake db:mongoid:create_indexes.Which files are affected
server/app/models/variable.rbserver/spec/models/variable_spec.rb(new)What kind of tests are done
measure_idindex is declared. Not run locally (needs Rails and Mongo); the sameindex_specificationscheck ran in the production web pod withrails runner.explainon the server went fromCOLLSCAN(441 ms) toIXSCAN(2 ms).