Skip to content

Harden /webSearch query validation and reject regex operators - #194

Open
afoote-mitre wants to merge 1 commit into
devfrom
af/websearch-query-validation
Open

afoote-mitre wants to merge 1 commit into
devfrom
af/websearch-query-validation

Conversation

@afoote-mitre

Copy link
Copy Markdown
Collaborator

Summary

Validate the complete /webSearch expression before contacting OpenSearch, reject regex operators, and distinguish unsupported client syntax from backend failures. Ordinary multi-term searches, phrases, supported Boolean expressions, wildcards, and standalone date/range searches remain supported.

Why These Changes Were Needed

Change Why It Was Needed Primary Files
Validate every token and reject regex operators anywhere in the expression The previous loop retained only the final token's verdict. Regex syntax could also be classified as general text, while the original expression was passed to query_string. Validation now checks quotes, escapes, and grouping and cannot be overridden by a valid trailing token. utils/webSearchQueryBuilder.js
Construct the executed query from validated tokens Validation previously inspected cleaned text while execution used the original input. Recognized slash-containing URLs and simple relative paths are now serialized as literal phrases; quoted and escaped literals retain their meaning. utils/webSearchQueryBuilder.js
Return HTTP 400 for locally rejected syntax Unsupported expressions were reported as HTTP 502 backend failures. A typed local query error now produces WEB_SEARCH_QUERY_SYNTAX in the existing searchMetadata.errors envelope without calling OpenSearch. Backend failures and timeouts retain their sanitized HTTP 502/504 handling. controllers/searchController.js, utils/errors.js
Describe the complete expression in response notes Notes previously reflected only the last token. They now include each distinct supported term classification and correctly identify quoted phrases. utils/webSearchQueryBuilder.js
Document the accepted syntax and error responses Callers need to distinguish unsupported regex syntax from quoted literal text and understand the HTTP 400 response shape. README and Swagger descriptions, generated OpenAPI, and schema assertions now reflect the implementation. README.md, routes/index.js, api-docs/openapi.json, api-docs/openapi.test.js
Add focused regression coverage Regressions cover rejection in every token position, field-qualified regex, quoting and escaping, malformed grouping, URL/path literals, supported multi-term and wildcard searches, sanitized errors, and ensuring rejected inputs never contact the search provider. utils/webSearchQueryBuilder.test.js, controllers/searchController.test.js, routes/search.integration.test.js, utils/errors.test.js

Scope

  • Changes are limited to /webSearch; /search is unchanged.
  • The endpoint continues to use query_string with an AND default operator. Sorting, pagination, and the response envelope are retained.
  • Leading wildcards remain supported. This change does not introduce field restrictions or additional query-complexity limits, and it does not implement a complete OpenSearch query parser.

@afoote-mitre afoote-mitre self-assigned this Oct 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant