diff --git a/reference/configuration/operations.md b/reference/configuration/operations.md index 75d9ed695..c10e66715 100644 --- a/reference/configuration/operations.md +++ b/reference/configuration/operations.md @@ -16,7 +16,9 @@ For the full list of configurable options, see [Configuration Options](./options ## Set Configuration -Modifies one or more Harper configuration parameters. **Requires a [restart](../operations-api/operations.md#restart) or [restart_service](../operations-api/operations.md#restart_service) to take effect** (a [restart](../operations-api/operations.md#restart) for `threads.v8Flags`). A change to `replication.threads` takes effect only on a full `restart`. +Modifies one or more Harper configuration parameters. **Most changes require a [restart](../operations-api/operations.md#restart) or [restart_service](../operations-api/operations.md#restart_service) to take effect** (a [restart](../operations-api/operations.md#restart) for `threads.v8Flags`). A change to `replication.threads` takes effect only on a full `restart`. + +Full-text request work limits (`storage.fullText.maxRawHitsPerRequest`, `storage.fullText.maxResultsPerRequest` and `storage.fullText.requestBudgetMilliseconds`) take effect in HTTP workers after `restart_service` or a full restart. Changes to `storage.fullText.runtime` require a full Harper process restart; `restart_service` does not replace the native process budget. See [Full-text query execution](../resources/query-optimization.md#full-text-search-work-and-paging). `operation` _(required)_ — must be `set_configuration` @@ -24,6 +26,8 @@ Modifies one or more Harper configuration parameters. **Requires a [restart](../ Additional properties correspond to configuration keys in underscore-separated format (e.g. `logging_level` for `logging.level`, `clustering_enabled` for `clustering.enabled`). +For full-text settings, use individual parameter names such as `storage_fullText_maxRawHitsPerRequest` or `storage_fullText_runtime_maxSearchThreads`. `set_configuration` does not accept a whole `storage_fullText` object; use `storage.fullText` as the section name only in the YAML configuration file. + :::warning Replicate cluster-appropriate parameters only A replicated call sends the exact same values to every node. Do not include node-local parameters — ports, `node.hostname`, `rootPath`, storage/log file paths, TLS certificate or key paths, or `replication.hostname`/`url`/`routes` — as they would overwrite each peer's local values. ::: @@ -42,7 +46,7 @@ A replicated call sends the exact same values to every node. Do not include node ```json { - "message": "Configuration successfully set. You must restart HarperDB for new config settings to take effect." + "message": "Configuration successfully set. You must restart Harper for new config settings to take effect." } ``` diff --git a/reference/configuration/options.md b/reference/configuration/options.md index 961f10592..edf7f438d 100644 --- a/reference/configuration/options.md +++ b/reference/configuration/options.md @@ -232,7 +232,7 @@ logging: level: warn root: ~/hdb/log stdStreams: false - auditLog: false + auditLog: true rotation: interval: 1D maxSize: 100M @@ -244,7 +244,7 @@ logging: - `path` — Explicit log file path (overrides `root`) - `stdStreams` — Write to stdout/stderr; _Default_: `false` - `console` — Include `console.*` output; _Default_: `true` -- `auditLog` — Enable table transaction audit logging; _Default_: `false` +- `auditLog` — Enable table transaction audit logging; _Default_: `true` - `auditRetention` — Audit log retention duration; _Default_: `3d` - `external` — Logging for components using the logger API; sub-options: `level`, `path` - `rotation.enabled` / `rotation.compress` / `rotation.interval` / `rotation.maxSize` / `rotation.path` — Log file rotation (activates when `interval` or `maxSize` is set) @@ -312,6 +312,19 @@ storage: ``` - `engine` — The database storage engine to use for new databases. Currently supported engines are `rocksdb` and `lmdb`. The default is `rocksdb`. Existing databases will use the engine that was used when they were created. +- `fullText` — Full-text search work and native process limits. Sub-options: + - `maxRawHitsPerRequest` — Maximum fetched hits per request, including bounded prefix replay when traversal is needed; default `10000`. + - `maxResultsPerRequest` — Maximum retained offset plus limit. A request must also fit the separate raw-hit budget; default `10000`. + - `requestBudgetMilliseconds` — Execution budget across native batches, source checks and highlights, capped by the native execution limit; default `5000` ms. Reader acquisition and freshness waiting have separate bounds. + - `runtime.maxResidentIndexes` — Maximum resident native indexes; default `128`. + - `runtime.maxIndexingThreads` — Maximum aggregate indexing-thread reservations across resident native indexes; default `128`. + - `runtime.maxSearchThreads` — Maximum aggregate search-thread reservations across resident native indexes; default `256`. + - `runtime.maxWriterMemoryBytes` — Aggregate writer-memory reservations across resident native indexes; default `4294967296` (4 GiB). + - `runtime.maxQueuedBytes` — Aggregate queued-write reservations across resident native indexes; default `8589934592` (8 GiB). + - `runtime.maxExpensiveSearches` — Maximum concurrent expensive native searches; default `8`. + + The runtime budgets apply together and are not hard bounds on total process memory. See [Set Configuration](./operations.md#set-configuration) for when request and native runtime changes take effect. Use the usual underscore-separated parameter names, such as `storage_fullText_maxRawHitsPerRequest`. See [full-text query execution](../resources/query-optimization.md#full-text-search-work-and-paging) for costs; see [readiness and recovery](../resources/query-optimization.md#readiness-and-recovery) for failures. + - `writeAsync` — Disable disk sync for higher throughput (**disables durability guarantees**); _Default_: `false` - `caching` — In-memory record caching; _Default_: `true` - `compression` — LZ4 record compression; _Default_: `true` (enabled by default since v4.3.0). Sub-options: `dictionary`, `threshold` diff --git a/reference/configuration/overview.md b/reference/configuration/overview.md index 112327b24..033c8006c 100644 --- a/reference/configuration/overview.md +++ b/reference/configuration/overview.md @@ -21,7 +21,7 @@ For a complete reference of all available configuration options, see [Configurat ## The Configuration File -To change a configuration value, edit `harper-config.yaml` and save. **Harper must be restarted for changes to take effect.** +To change a configuration value, edit `harper-config.yaml` and save. **Restart Harper to apply file edits.** See [Set Configuration](./operations.md#set-configuration) for settings that require a full process restart. Configuration keys use camelCase (e.g. `operationsApi`). Nested keys use dot notation conceptually (e.g. `operationsApi.network.port`). diff --git a/reference/database/storage-tuning.md b/reference/database/storage-tuning.md index 9cade5b28..bf2a99806 100644 --- a/reference/database/storage-tuning.md +++ b/reference/database/storage-tuning.md @@ -202,7 +202,7 @@ Changing this setting is safe for existing data — records already written keep To pin one table's encoding regardless of the global setting, use the [`@table(randomAccessFields:)`](./schema.md#randomaccessfields) directive when creating the table; editing the directive later does not repin an existing table. Tables that carry the directive ignore this option entirely, which also means a fleet-wide change to this setting will not move them. -The typed dictionary is bounded, and reaching the bound is not an error: records with novel shapes past that point still write and read correctly, they are simply stored without random-access field encoding. [`describe_table`](../operations-api/operations.md#describe_table) reports both the current size and the ceiling in force (`typed_structure_count` against `typed_structure_limit`, 256 by default), which is the reliable way to see whether a table's shapes are staying within it. The classic dictionary is bounded far lower — 32 shared named-record structures — so a table with more than 32 distinct field-name sets sits at that number as a matter of course; it is a normal reading, not a leak. +The typed dictionary is bounded, and reaching the bound is not an error: records with novel shapes past that point still write and read correctly, they are simply stored without random-access field encoding. [`describe_table`](../operations-api/operations.md#record-structure-dictionaries) reports both the current size and the ceiling in force (`typed_structure_count` against `typed_structure_limit`, 256 by default), which is the reliable way to see whether a table's shapes are staying within it. The classic dictionary is bounded far lower — 32 shared named-record structures — so a table with more than 32 distinct field-name sets sits at that number as a matter of course; it is a normal reading, not a leak. ## RocksDB Memory @@ -395,6 +395,16 @@ storage: compression: true ``` +## `storage.fullText` + + + +Type: `object` + +Default: See the [storage option reference](../configuration/options.md#storage). + +The [storage option reference](../configuration/options.md#storage) lists the full-text request and native process defaults. Raise the fetched-hit budget to allow more filtering work; raising it does not raise the separate retained-result limit. Larger limits can increase CPU, source reads and memory use for every permitted query. Native process budgets also share capacity across indexes, so tune them with concurrent workloads rather than a single query. See [full-text query execution](../resources/query-optimization.md#full-text-search-work-and-paging) for paging costs and restart requirements; see [readiness and recovery](../resources/query-optimization.md#readiness-and-recovery) for failures. + ## Related - [Configuration Options](../configuration/options.md) — full list of `storage` options diff --git a/reference/database/transaction.md b/reference/database/transaction.md index baba4612e..f071288ae 100644 --- a/reference/database/transaction.md +++ b/reference/database/transaction.md @@ -23,7 +23,7 @@ On RocksDB (the default storage engine) the transaction log is physically a sing The transaction log is **enabled by default**. To disable it, set [`logging.auditLog`](../logging/configuration.md) to `false` in `harper-config.yaml` and restart Harper. -> The transaction log is required for real-time messaging (WebSocket and MQTT subscriptions) and replication. Do not disable it if real-time features or replication are in use. +> The transaction log is required for real-time messaging (WebSocket and MQTT subscriptions), replication and full-text indexing. Do not disable it while any of these features are in use. ## Operations @@ -134,7 +134,7 @@ Deletes transaction log entries older than the specified timestamp. Deprecated i Deletes transaction log entries older than the specified timestamp. :::warning Database-wide and irreversible -On RocksDB (the default storage engine), deletion is database-wide: all tables in a database share one transaction log, so omit `table` and pass only `database` (or `schema`) and `timestamp`. This purges whole log files whose entries predate the timestamp, removing [`read_audit_log`](#read_audit_log) history for **every table in the database** — there is no per-table survivor and no undo. The only recovery route is a [backup](../backups/overview.md), which restores the transaction log alongside the data. Purging below a lagging replica's catch-up position does not lose data on that replica — the sender detects that the requested start predates its retained history and forces a full base copy instead of incremental catch-up — but that full resync is far more expensive than incremental replication, so avoid purging below your slowest replica's position. +On RocksDB (the default storage engine), deletion is database-wide: all tables in a database share one transaction log, so omit `table` and pass only `database` (or `schema`) and `timestamp`. This purges whole log files whose entries predate the timestamp, removing [`read_audit_log`](#read_audit_log) history for **every table in the database** — there is no per-table survivor and no undo. The only recovery route is a [backup](../backups/overview.md), which restores the transaction log alongside the data. Purging below a lagging replica's catch-up position does not lose data on that replica — the sender detects that the requested start predates its retained history and forces a full base copy instead of incremental catch-up — but that full resync is far more expensive than incremental replication, so avoid purging below your slowest replica's position. A full-text rebuild also needs retained history to replay writes committed during its scan; purging that history can force another full rebuild and extend the search outage. Pause manual cleanup during an upgrade rebuild and size automatic retention for the workload. ::: Parameters: @@ -204,7 +204,7 @@ type Dog @table(audit: true) { } ``` -This overrides the [`logging.auditLog`](../logging/configuration.md) global configuration for that specific table. +This overrides the [`logging.auditLog`](../logging/configuration.md) global configuration for that specific table. New schema tables with `@fullText` must explicitly declare `@table(audit: true)`, even when global audit logging is enabled. Existing audited tables can add an index without restating `audit`. Keep auditing enabled while any full-text index is declared. Setting `@table(audit: false)` while a full-text index is declared is rejected during schema registration; this is a per-table prerequisite, independent of the global default. ## Related Documentation diff --git a/reference/logging/configuration.md b/reference/logging/configuration.md index f204b9325..f5f82ea96 100644 --- a/reference/logging/configuration.md +++ b/reference/logging/configuration.md @@ -102,11 +102,11 @@ Default: `true` Enables transaction logging. When enabled, Harper records every insert, update, and delete to the database's transaction log. This history is read with the [`read_audit_log`](../database/transaction.md#read_audit_log) operation — a legacy name; there is a single transaction log, not a separate audit table. -See [Database / Transaction Logging](../database/transaction.md) for details on the transaction log. +Full-text indexing requires transaction logging. New schema tables with `@fullText` must explicitly declare `@table(audit: true)`, even when this global setting is enabled. Existing audited tables can add an index without restating `audit`; disabling table auditing while an index is declared is rejected. See [Database / Transaction Logging](../database/transaction.md) for details on the transaction log. ```yaml logging: - auditLog: false + auditLog: true ``` ### `logging.auditRetention` @@ -339,7 +339,7 @@ logging: file: true stdStreams: false console: false - auditLog: false + auditLog: true auditRetention: 3d rotation: enabled: true diff --git a/reference/operations-api/operations.md b/reference/operations-api/operations.md index 21815c98a..56a5559f3 100644 --- a/reference/operations-api/operations.md +++ b/reference/operations-api/operations.md @@ -47,7 +47,7 @@ Detailed documentation: [Database Overview](../database/overview.md) ### `describe_all` -Returns the definitions of all databases and tables within the database. Record counts above 5000 records are estimated; the response includes `estimated_record_range` when estimated. To force an exact count (requires full table scan), include `"exact_count": true`. Each table definition includes the [record-structure dictionary fields](#describe_table) described for `describe_table`. +Returns the definitions of all databases and tables within the database. Record counts above 5000 records are estimated; the response includes `estimated_record_range` when estimated. To force an exact count (requires full table scan), include `"exact_count": true`. Each table definition includes the [record-structure dictionary fields](#record-structure-dictionaries) described for `describe_table`. ```json { "operation": "describe_all" } @@ -55,7 +55,7 @@ Returns the definitions of all databases and tables within the database. Record ### `describe_database` -Returns all table definitions within the specified database. Each table definition includes the [record-structure dictionary fields](#describe_table) described for `describe_table`. +Returns all table definitions within the specified database. Each table definition includes the [record-structure dictionary fields](#record-structure-dictionaries) described for `describe_table`. ```json { "operation": "describe_database", "database": "dev" } @@ -63,12 +63,43 @@ Returns all table definitions within the specified database. Each table definiti ### `describe_table` -Returns the definition of a specific table. +Returns the definition of a specific table, including its schema and storage metadata. ```json { "operation": "describe_table", "table": "dog", "database": "dev" } ``` +#### Full-text indexes + + + +A table with native full-text indexes includes `full_text_indexes`; each entry reports its query-only field name, source fields, analyzer options, enabled query modes, and `readiness` (`unknown`, `ready`, `needs-rebuild`, `rebuilding`, or `unavailable`). The readiness object also includes `owner_epoch`, `rebuild_attempts`, and an optional `reason`. Query-only `FullText` fields do not appear in the ordinary `attributes` list. Metadata respects attribute permissions: unreadable source fields are omitted, and an index with no readable source fields is omitted entirely. Use an account with access to every indexed source field when checking upgrade readiness; a missing entry does not prove that no index exists. + +Requesting a table with full-text indexes: + +```json +{ "operation": "describe_table", "table": "Product", "database": "catalog" } +``` + +The response also includes the table's ordinary schema and storage metadata. The relevant full-text fragment is: + +```json +{ + "name": "Product", + "full_text_indexes": [ + { + "name": "catalogSearch", + "fields": [{ "name": "name", "weight": 3 }], + "analyzer": "english@2", + "query_modes": ["any", "all", "fuzzy", "phrase", "prefix", "fuzzy-prefix"], + "readiness": { "state": "ready", "owner_epoch": "4", "rebuild_attempts": 0 } + } + ] +} +``` + +#### Record-structure dictionaries + Alongside the schema, the response carries the size of the table's **record-structure dictionaries** @@ -270,7 +301,7 @@ Returns records with a matching value on any attribute. Supports wildcards (e.g. ### `search_by_conditions` -Returns records matching one or more conditions. Supports `operator` (`and`/`or`), `offset`, `limit`, nested `conditions` groups, and `sort` with multi-level tie-breaking. +Returns records matching one or more conditions. Supports `operator` (`and`/`or`), `offset`, `limit`, nested `conditions` groups, and `sort` with multi-level tie-breaking. Full-text conditions use comparators such as `matches` and `matches_prefix`; see [full-text query execution](../resources/query-optimization.md#full-text-search-work-and-paging) for paging costs and limits. ```json { @@ -1679,16 +1710,16 @@ Operations for reading and updating Harper configuration. Detailed documentation: [Configuration Overview](../configuration/overview.md) -| Operation | Description | Role Required | -| ------------------- | ---------------------------------------------------------------- | ------------- | -| `set_configuration` | Modifies Harper configuration file parameters (requires restart) | super_user | -| `get_configuration` | Returns the current Harper configuration | super_user | +| Operation | Description | Role Required | +| ------------------- | --------------------------------------------- | ------------- | +| `set_configuration` | Modifies Harper configuration file parameters | super_user | +| `get_configuration` | Returns the current Harper configuration | super_user | ### `set_configuration` -Updates configuration parameters in `harper-config.yaml`. A restart (`restart` or `restart_service`) is required for changes to take effect. +Updates configuration parameters in `harper-config.yaml`. Most changes require a restart (`restart` or `restart_service`) to take effect. See [Set Configuration](../configuration/operations.md#set-configuration) for full-text restart requirements. -Supports `"replicated": true` to apply the same change to all cluster nodes in one call; per-node outcomes are returned in the response's `replicated` array. Only send cluster-appropriate parameters when replicating — node-local parameters (ports, `node.hostname`, file paths, TLS material, `replication.hostname`/`url`/`routes`) would overwrite every peer's local values. To apply the change cluster-wide, follow with `restart_service` using `"replicated": true` (which restarts nodes one at a time); `restart_service` restarts only worker threads, though, so a change to [`threads.v8Flags`](../configuration/options.md#applying-v8-flags) takes effect only after a `restart` sent to each node in turn (`restart` does not replicate). See [Configuration Operations](../configuration/operations.md#set-configuration) for details. +Supports `"replicated": true` to apply the same change to all cluster nodes in one call; per-node outcomes are returned in the response's `replicated` array. Only send cluster-appropriate parameters when replicating — node-local parameters (ports, `node.hostname`, file paths, TLS material, `replication.hostname`/`url`/`routes`) would overwrite every peer's local values. To apply the change cluster-wide, follow with `restart_service` using `"replicated": true` (which restarts nodes one at a time); `restart_service` restarts only worker threads, though, so changes to [`threads.v8Flags`](../configuration/options.md#applying-v8-flags) or `storage.fullText.runtime.*` take effect only after a `restart` sent to each node in turn (`restart` does not replicate). See [Configuration Operations](../configuration/operations.md#set-configuration) for details. ```json { diff --git a/reference/resources/query-optimization.md b/reference/resources/query-optimization.md index dfe2cf9d1..3e7cd6bd9 100644 --- a/reference/resources/query-optimization.md +++ b/reference/resources/query-optimization.md @@ -79,6 +79,55 @@ When a secondary index is still being built — for example, just after a new `@ This is a transient condition — the index finishes building in the background, after which the same query succeeds. Clients should retry (with backoff) rather than treating it as an empty result or a permanent failure. On the operations API the error body includes `code` and `retryable`, so callers can branch on `error.code === 'INDEX_REBUILDING'` or `error.retryable`. +## Full-Text Search Work and Paging + + + + + +Full-text and autocomplete queries use Harper's existing `limit` and `offset` options. Each page runs a new query: changed records, permissions or index contents can cause records to repeat or be omitted between pages. There is no retained search across requests. See [REST paging and counts](../rest/querying.md#pagination-and-total-count) for URL syntax and count behavior. + +Within one request, Harper uses one native reader to fetch ranked batches. It checks source versions and record filters before counting records toward the requested offset and limit. In v5.4, prefix and fuzzy-prefix searches can return more than the native ordinary autocomplete window. Before v5.4, prefix and fuzzy-prefix requests whose `offset + limit` exceeded 100 were rejected with 400; requests within that window could still return short pages after filtering. In v5.4, the native search window is not a total-result cutoff for bounded Harper queries. + +For example, with `offset: 20` and `limit: 10`, Harper skips the first 20 eligible records and returns the next 10. Hits rejected by source-version checks or record filters do not count toward those 30 records, but fetching them still consumes the raw-hit budget. Harper fetches additional batches as needed until it fills the page, exhausts the matches or reaches a request limit. + +Use a bounded `limit` for large result sets. An unbounded search that exceeds its native window, fetched-hit limit or retained-result limit throws 400; use a bounded query. Exhausting the execution-time budget instead throws 503. `offset + limit` must not exceed either `storage.fullText.maxResultsPerRequest` or `storage.fullText.maxRawHitsPerRequest`; replayed or rejected hits can exhaust the raw-work budget earlier. Raising only the raw-hit budget allows more filtering work without raising the retained-record target. Offset requests revisit earlier matches and can retain up to `offset + limit` accepted records. Raising work limits can increase CPU, reads and memory use. + +The fetched-hit budget includes the initial batch replay when Harper switches to native traversal. It counts hits returned by the native library, not every record Tantivy visits while scoring. Exhausting that budget throws a 400 error; exhausting the execution budget throws 503. A budget failure is a failed query, not a completed short page. If a REST response has already started streaming, a later query error ends that stream instead of changing its HTTP status. JSON-array clients must check for a terminal object shaped like `{ "error": "ErrorName: message" }` and treat the response as failed. For other formats, detect stream or decoding failures; status 200 and count headers alone do not prove completion. See [REST paging](../rest/querying.md#pagination-and-total-count). When there are no further native matches, Harper returns the eligible records found, which may be an empty result. This differs from reaching a budget while more matches remain. + +Configure the three request limits and six native process budgets under [`storage.fullText`](../configuration/options.md#storage). See [Set Configuration](../configuration/operations.md#set-configuration) for restart requirements. The execution budget is not a hard response-time guarantee: reader acquisition, query construction and some native operations can take additional time. Freshness waiting and reader acquisition use separate bounds. + +### Native runtime configuration + +Harper captures native process budgets before loading application components, or on an earlier main-thread native load or managed worker spawn. Application-created workers inherit that configuration. Create workers from a Harper application component after initialization. A worker created before capture fails on its first native full-text load with `FullTextWorkerInitializationError` (500, non-retryable); recreate that worker after Harper initializes. + +Replacement workers keep the captured budgets after a configuration change. See [Set Configuration](../configuration/operations.md#set-configuration) before changing these budgets. + +### Readiness and recovery + +Full-text indexes require audit logging. New schema tables must explicitly declare `@table(audit: true)` when adding `@fullText`, even if global audit logging is enabled; an existing audited table can add an index without restating `audit`. Disabling audit logging while an index is declared is rejected. A format-incompatible native upgrade rebuilds the local index from stored records, and queries return 503 while it rebuilds. Harper v5.4 uses `@harperfast/fulltext` 0.7.0 with Tantivy 0.27.0. Harper 5.3.1–5.3.4 ship fulltext 0.5.0; those indexes using current field-based declarations, as well as prerelease 0.6 indexes, require a full local rebuild. Harper 5.3.0 ships fulltext 0.3.0 and is outside this qualified upgrade procedure; plan and validate that migration separately. Earlier beta declarations and indexes have no supported migration path. Drain the node's search traffic and stop old processes before starting the new version; do not mix engine versions against one local index. Verify readiness and current coverage before restoring search traffic. Replay after a long rebuild can also reject ordinary local writes with 503, `code: "DERIVED_INDEX_LAGGING"`, and `retryable: true`; retry with bounded backoff while it catches up. Replication and crash-recovery replay continue. Direct downgrade to fulltext 0.5.x is unsupported: that engine refuses opening or resetting a 0.7 index identity. Stop an accidentally started old version and restore the 0.7.0-based Harper version, keeping search traffic drained until readiness and coverage recover. Do not edit native index files. Missing retained transaction history can require a full rebuild, as described below; this is current-engine recovery, not a general Harper downgrade procedure. English stemming also changes: `added` now stems to `add` rather than `ad`, so review matching and ranking against representative searches. + +| Failure | HTTP status | Code and retry guidance | +| ------------------------------------------------------ | ----------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Raw-hit or retained-result budget | 400 | No readiness code. Narrow the query or adjust the named budget. | +| Execution budget | 503 | No readiness code. Narrow the query; the same expensive query can fail again. | +| Native busy | 503 | No readiness code. Reduce concurrency or retry with backoff; inspect process-wide native capacity, including `storage.fullText.runtime.maxExpensiveSearches`, before raising limits. | +| Index unknown, needs-rebuild or rebuilding | 503 | `INDEX_REBUILDING`, with `retryable: true`, for a not-ready readiness check. Retry with backoff while the index rebuilds. | +| Coverage lag, wait timeout or local write backpressure | 503 | `DERIVED_INDEX_LAGGING`, with `retryable: true`. Retry with backoff while the index catches up. Ordinary local writes can be refused after more than 30 seconds of index lag. | +| Index unavailable | 503 | No readiness-specific code or retryable flag. Inspect readiness and logs; fix the reported problem before retrying. | +| Query host becomes non-ready during a coverage wait | 503 | Can return no readiness-specific code or retryable flag. Inspect readiness before deciding whether to retry. | +| Worker loaded native search before runtime capture | 500 | REST reports `code: "FullTextWorkerInitializationError"`; the Operations API can omit that code, so inspect the error message too. Non-retryable; recreate the worker after initialization. | + +A native result that exceeds the response-byte envelope can return 413, including an oversized ordinary batch even when each individual hit would fit. Other native failures and invalid query configuration can return 500; inspect the error and server logs rather than assuming they are retryable. The table lists common failures, not every possible error. + +For read-after-write queries, `maxIndexLagMilliseconds: 0` requires coverage of writes already committed on the queried node. It does not wait for replication to deliver writes from another node. `waitForIndexMilliseconds` lets a query wait for coverage (default `0`, maximum `30000` ms); without an override, the allowed index lag is `3000` ms. These options belong to the full-text condition. They wait for coverage, not for an unavailable index to recover. Readiness is checked first; a query host that becomes rebuilding or unavailable during wait setup can fail immediately with a bare 503. + +Concurrent requests each have their own work budget. These per-request budgets do not cap the number of concurrent Harper queries or prevent brief event-loop delays during synchronous source reads. Measure deep offsets and selective filters under the expected concurrency before raising budgets. + +`INDEX_REBUILDING` and `DERIVED_INDEX_LAGGING` identify retryable readiness or coverage failures. Retry with backoff while the index makes progress. Transaction history retained under [`logging.auditRetention`](../logging/configuration.md#loggingauditretention) (default `3d`) is used for catch-up. Lost history can require a rebuild; retries alone do not repair an index that cannot catch up. Use [`describe_table`](../operations-api/operations.md#describe_table) to inspect `full_text_indexes[].readiness`, including its state and reason. Stop aggressive transaction-log cleanup and correct the reported configuration or storage problem before restarting the same Harper version to retry activation. Keep full-text traffic drained until readiness and required coverage recover; do not edit native files. Budget and native-busy failures do not carry those readiness codes. A native busy response also returns 503; reduce concurrency or retry with backoff. For an execution-budget failure, narrow the query, reduce the offset or adjust the work budget instead of repeatedly submitting the same expensive query. + +Native page, response-byte and highlight limits still apply. The fulltext 0.7.0 library limits each analyzed query leaf to 64 terms and the compiled query to 256 clauses. Native invalid-query validation returns 400 before response headers; the streaming-error guidance above still applies. Raising Harper's work budgets does not raise those native limits. See the [published native limits](https://github.com/HarperFast/fulltext/blob/v0.7.0/docs/limits.md) for page, byte and highlight bounds. + ## Relationships and Joins Harper supports relationship-based queries that join data across tables. See [Schema documentation](../database/schema.md) for how to define relationships. diff --git a/reference/resources/resource-api.md b/reference/resources/resource-api.md index 8412c1b0e..bd2c5d2c3 100644 --- a/reference/resources/resource-api.md +++ b/reference/resources/resource-api.md @@ -1035,6 +1035,8 @@ Array of condition objects to filter records. Each condition: | `conditions` | Nested conditions array | | `operator` | `and` (default) or `or` for the nested `conditions` | +Full-text conditions use `matches`, `matches_all`, `matches_phrase`, `matches_prefix`, `matches_fuzzy`, or `matches_fuzzy_prefix` against a declared full-text index. Each comparator also has a `not_` form: `not_matches`, `not_matches_all`, `not_matches_phrase`, `not_matches_prefix`, `not_matches_fuzzy`, and `not_matches_fuzzy_prefix`. A negated condition requires a positive full-text condition on the same index. See [Full-text query execution](./query-optimization.md#full-text-search-work-and-paging) for paging costs and work limits. + Example with nested conditions: ```javascript diff --git a/reference/rest/querying.md b/reference/rest/querying.md index 45474a83a..8c168717c 100644 --- a/reference/rest/querying.md +++ b/reference/rest/querying.md @@ -55,6 +55,8 @@ Harper uses [FIQL](https://datatracker.ietf.org/doc/html/draft-nottingham-atompu | `=`, `===` | Strict equality (no type conversion) | | `!==` | Strict inequality (no type conversion) | +Full-text indexes support `=matches=`, `=matches_all=`, `=matches_phrase=`, `=matches_prefix=`, `=matches_fuzzy=`, and `=matches_fuzzy_prefix=`. Each has a negated form: `=not_matches=`, `=not_matches_all=`, `=not_matches_phrase=`, `=not_matches_prefix=`, `=not_matches_fuzzy=`, and `=not_matches_fuzzy_prefix=`. Combine a negated condition with a positive full-text condition on the same index; a structured filter alone does not qualify. These use the full-text index rather than ordinary string comparisons. See [Full-text query execution](../resources/query-optimization.md#full-text-search-work-and-paging). + **Examples**: ```http @@ -184,6 +186,10 @@ Use `limit(start,end)` to page through a collection, and opt in to a total match Counting is opt-in: without the header, no count is computed and no count headers are returned. It applies only to `GET`/`HEAD` requests and requires a bounded page — the page size (`end - start`, or `end` in the one-argument form) must be a non-negative integer no larger than **10,000**, and the window end (`offset + page size`) no larger than **1,000,000**. A request with no `limit()`, one outside those bounds, or a non-numeric one is served normally with no count headers, since counting an unbounded page would defeat the point of paging. +Full-text and autocomplete queries use the same paging syntax. The page window counts eligible records after record filtering, rather than raw full-text hits. Harper fetches more ranked batches to fill that window when needed. A page can contain fewer records when no further matches remain; reaching a search work or time limit fails the query. A failure detected before response headers can set an error HTTP status. After streaming starts, the response can retain HTTP 200 and end with an error. For JSON-array responses, check for a terminal object shaped like `{ "error": "ErrorName: message" }` and treat the response as failed; do not render that element as a record. For other formats, detect stream or decoding failures as well. Neither status 200 nor count headers prove that a page completed successfully. Each page request runs a new query, so concurrent changes can cause repeats or omissions between pages. See [Full-text query execution](../resources/query-optimization.md#full-text-search-work-and-paging) for search work limits and recovery guidance. + +The count-header guardrails do not override query-specific limits such as the full-text work budget. + ### Requesting a count Send a `Prefer` header on a `GET` (or `HEAD`) request to a collection: @@ -221,6 +227,8 @@ The response status is always `200` — `Content-Range` is informational (Harper The total is reported as `*` (for example `Content-Range: items 0-24/*`) when it cannot be produced — an exact scan that hits its guardrail (counting the tail past the requested page is bounded by a 1,000,000-row cap and a ~1-second budget, and abandons the total rather than truncating the page), or a query with no cardinality estimate (for example a `!=` or `=ct=` (contains) condition). `Preference-Applied` still echoes the requested mode, so an unavailable total (`.../*`) is distinct from a request that asked for no count. +Full-text and autocomplete queries cannot provide an exact total. With REST exact counting enabled, `Prefer: count=exact` returns an unavailable total (`.../*`) and `Preference-Applied: count=exact`. With exact counting disabled (the default), the request uses estimated mode and `Preference-Applied: count=estimated`; its total is the planner estimate when available, or `*`. An estimated count may use the planner estimate when the query has no opaque row or vector filter. See [Full-text query execution](../resources/query-optimization.md#full-text-search-work-and-paging). + ### Enabling exact counts Because an exact count scans the full matched set, it is **off by default**. Enable it in an application's REST configuration via the [`exactCount` option](./overview.md#configuration): diff --git a/release-notes/v5-lincoln/5.4.md b/release-notes/v5-lincoln/5.4.md index 3b5aa53f6..78e9be6d1 100644 --- a/release-notes/v5-lincoln/5.4.md +++ b/release-notes/v5-lincoln/5.4.md @@ -8,6 +8,10 @@ title: '5.4' All patch release notes for 5.4.x are available on the [releases page](https://github.com/HarperFast/harper/releases?q=v5.4&expanded=true). +## Upgrade Notes + +Full-text indexes using fulltext 0.5.0 with current field-based declarations, or prerelease 0.6, rebuild locally after this upgrade. Plan for search unavailability and review the [Tantivy upgrade requirements](#tantivy-027-and-existing-index-rebuilds) before deploying; direct downgrade to fulltext 0.5.x is unsupported. + ## Nested Relationship Constraints [Honor nested relationship filters and pagination](https://github.com/HarperFast/harper/pull/3279) corrects REST and programmatic nested selections that accepted filters or pagination but returned all related records. Filters, ordering, offset, and limit apply per parent in collection and single-record reads. Parents remain present when the nested selection has no matches. Outer relationship filters also constrain the selected children, so an identical filter does not need to be repeated inside the nested selection. @@ -80,6 +84,20 @@ The new `threads.v8Flags` setting takes a V8 flag, or a list of them, and applie `restore_backup` no longer deletes a database before checking that the backup can be read. It first verifies the checksum of every database file in the backup, checks its transaction logs, opens each table file to confirm this version can read its format and compression, and, for a backup that captured blobs, checks that the blob snapshot is present. A backup that fails is refused and the database is left as it was (one an earlier restore left incomplete stays blocked); previously the restore deleted the database first and then failed, so a backup taken by a newer version, or one with a corrupt file, destroyed the database it was meant to restore. The check reads the whole backup once, so a restore takes longer, and an online restore keeps the database unavailable while it runs; it needs no extra disk space. A restore interrupted while it is still checking the backup leaves the destination's files and any preexisting incomplete-restore marker unchanged; a previously healthy database loads normally on the next start. Blob file contents are not checked. See [`restore_backup`](/reference/v5/backups/operations#restore_backup). +## Full-Text Search + +### Tantivy 0.27 and Existing Index Rebuilds + +Harper now uses `@harperfast/fulltext` 0.7.0 with Tantivy 0.27.0. Harper 5.3.1–5.3.4 ship fulltext 0.5.0. Existing 0.5.0 indexes using current field-based declarations, and prerelease 0.6 indexes, rebuild automatically from authoritative records. Harper 5.3.0 ships fulltext 0.3.0 and is outside this qualified procedure; validate that migration separately. Earlier beta declarations and indexes have no supported migration path. Plan for local full-text unavailability, disk headroom and possible retryable write failures while replay catches up. Stop old processes before starting the new version against the same storage; direct downgrade to fulltext 0.5.x is unsupported. + +English stemming changes despite the unchanged `english@2` analyzer name: `added` now stems to `add` rather than `ad`. Matching and ranking can change. Query modes and schema options are unchanged; benchmark results from the wrapper do not establish application performance parity. Review representative searches and see [readiness and recovery](/reference/v5/resources/query-optimization#readiness-and-recovery) before restoring traffic. + +### Paging and Search Work Limits + +Full-text and autocomplete queries use the existing `limit` and `offset` paging options. Harper reads additional native batches when needed after filtering, so the initial native result window no longer silently cuts off eligible results. The new `storage.fullText.maxRawHitsPerRequest`, `storage.fullText.maxResultsPerRequest` and `storage.fullText.requestBudgetMilliseconds` settings bound fetched hits, retained results and execution time. Requests that exceed those budgets return an error. The `storage.fullText.runtime` section also provides process-wide native-capacity budgets; changes to those budgets require a full Harper process restart. Separate page requests run independently and can see changes to records or index contents. + +See [Full-text query execution](/reference/v5/resources/query-optimization#full-text-search-work-and-paging) and [storage configuration](/reference/v5/configuration/options#storage) for limits, restart requirements and recovery guidance. + ## Native Addons ### Pointer-Compression Runtimes Refuse Incompatible Native Addons