From 82bac04242d522147f2c9d376fe52f3a8ec3bb04 Mon Sep 17 00:00:00 2001 From: glasstiger Date: Fri, 13 Nov 2026 23:50:56 +0000 Subject: [PATCH 1/3] docs(clients): document OIDC and rotating bearer tokens --- documentation/connect/clients/c-and-cpp.md | 342 ++++++++++++++++-- .../connect/clients/connect-string.md | 31 +- documentation/connect/clients/python.md | 252 +++++++++++-- documentation/connect/clients/rust.md | 195 +++++++++- .../wire-protocols/qwp-client-behavior.md | 79 ++-- documentation/security/oidc.mdx | 213 ++++++++++- 6 files changed, 991 insertions(+), 121 deletions(-) diff --git a/documentation/connect/clients/c-and-cpp.md b/documentation/connect/clients/c-and-cpp.md index af32aab4f..c64480658 100644 --- a/documentation/connect/clients/c-and-cpp.md +++ b/documentation/connect/clients/c-and-cpp.md @@ -28,7 +28,7 @@ include(FetchContent) FetchContent_Declare( c_questdb_client_proj GIT_REPOSITORY https://github.com/questdb/c-questdb-client.git - GIT_TAG 7.2.0) # or a newer release tag, or a commit SHA + GIT_TAG 7.3.0) # or a newer release tag, or a commit SHA FetchContent_MakeAvailable(c_questdb_client_proj) target_link_libraries(your_target questdb_client) @@ -56,6 +56,10 @@ Three QWP entry headers, in both a C (`.h`) and a C++ (`.hpp`) flavour: | `questdb/ingress/qwp_sender` | Senders: `questdb_db_borrow_sender` / `pool::borrow_sender()` | | `questdb/egress/qwp_reader` | Readers: `questdb_db_borrow_reader` / `pool::borrow_reader()` | +Authentication adds `questdb/oidc.h` / `.hpp` for device-flow sign-in and +`questdb/token_provider.h` / `.hpp` for application-supplied rotating tokens. +Both are included in the native library; no extra CMake feature is needed. + Both borrow headers include the pool header, so a program that only ingests includes only `qwp_sender`, and one that only queries includes only `qwp_reader` — each gets the pool along with it. Include `client` directly to hold @@ -251,8 +255,8 @@ threads, close it at shutdown. You don't open connections yourself: **borrow** a lease (a sender or reader), use it on one thread, then **return** it (recycles the connection) or **drop** it (retires it). In C++ the borrowed sender wrapper and pooled reader return on -destruction; `drop_on_return()` forces a drop. The pool connects lazily, and -reconnects and fails over on its own. +destruction; `drop_on_return()` forces a drop. The pool reconnects and fails +over on its own. ### Which borrow? @@ -271,14 +275,17 @@ the schema server-side. The connect string uses a QWP/WebSocket scheme: `ws` / `wss`. For auth and TLS keys, see the -[connect string reference](/docs/connect/clients/connect-string/). The pool is -**lazy**: `questdb_db_connect` (C) and the `questdb::pool` constructor (C++) -parse and validate the string but perform no blocking network I/O, so auth / -TLS / connect errors surface from the first borrow, not at construction. The -one exception is the on-disk send queue (`sf_dir`, see -[Durability and backpressure](#durability-and-backpressure)): at connect the -pool reopens any queue directories a previous run left holding unacknowledged -rows, then reconnects and resends them in the background. +[connect string reference](/docs/connect/clients/connect-string/). In client +7.3.0, `questdb_db_connect` (C) and the `questdb::pool` constructor (C++) open +the warm minimum connections eagerly, so authentication, TLS, and connection +errors can surface at construction. Set `lazy_connect=true` to defer +connections: senders queue locally and connect in the background, and readers +connect on first borrow. + +With an on-disk send queue (`sf_dir`, see +[Durability and backpressure](#durability-and-backpressure)), the pool may +also reopen queue directories holding unacknowledged rows from an earlier run +and replay them in the background. @@ -303,7 +310,9 @@ if (!db) { /* read err, handle */ } ## Authentication and TLS Authentication happens during the WebSocket upgrade, before any data is -exchanged. All keys go in the connect string: +exchanged. Put fixed credentials in the connect string, or attach an +[OIDC helper](#oidc-device-flow) or +[rotating token provider](#rotating-bearer-tokens) in code: ```text wss::addr=db.example.com:9000;username=admin;password=quest; # HTTP basic @@ -320,21 +329,296 @@ wss::addr=db.example.com:9000;token=your_bearer_token; # bearer token (E rejects it. - **`auth_timeout_ms`** (default 15000) bounds the WebSocket upgrade. -Because the pool connects lazily, a bad credential surfaces as -`line_sender_error_auth_error` from the **first borrow** on each connection. -Handle it there, not at `connect` (see -[Which errors mean what](#which-errors-mean-what)). +A rejected credential reports `line_sender_error_auth_error` (C++ +`questdb::error_code::auth_error`) from connection setup or an operation that +waits for it. Mutual TLS with client certificates is not supported; use +bearer tokens over `wss` instead. + +### OIDC device flow + +Client **7.3.0 or later** can sign a user in through their identity provider's +browser page without collecting their password. This requires QuestDB +Enterprise with OIDC configured and an IdP application with the device grant +enabled. See [Device authorization flow](/docs/security/oidc/#device-authorization-flow) +for discovery, scopes, and token selection. + +Sign in before constructing the pool, then attach the authentication object, +not a token string. Every pooled sender and reader shares it, so reconnects +can use silently refreshed credentials without rebuilding the pool. Replace +the example QuestDB URL and issuer with your deployment's values. + + + + +```cpp title="Sign in and query with a shared OIDC provider" +#include +#include +#include + +using namespace questdb::ingress::literals; + +int main() +{ + try + { + auto auth = questdb::oidc::builder::from_questdb( + "https://db.example.com:9000") + .issuer("https://idp.example.com/realms/questdb") + .scope("openid offline_access") + .build(); + auth.sign_in(); // Only this call may prompt or open a browser. + + questdb::pool pool{"wss::addr=db.example.com:9000;", auth}; + auto reader = pool.borrow_reader(); + auto cursor = reader.execute("SELECT current_user()"_utf8); + while (auto batch = cursor.next_batch()) + std::cout << batch->row_count() << " row(s)\n"; + } + catch (const questdb::error& error) + { + std::cerr << error.what() << '\n'; + return 1; + } +} +``` + + + + +```c title="Sign in and attach OIDC to a connection pool" +#include +#include +#include +#include + +int main(void) +{ + const char* url = "https://db.example.com:9000"; + const char* issuer = "https://idp.example.com/realms/questdb"; + const char* scope = "openid offline_access"; + const char* conf = "wss::addr=db.example.com:9000;"; + questdb_error* error = NULL; + questdb_oidc_auth* auth = NULL; + questdb_db* db = NULL; + int status = 1; + + questdb_oidc_builder* builder = questdb_oidc_builder_from_questdb( + url, strlen(url), &error); + if (!builder) goto cleanup; + if (!questdb_oidc_builder_issuer( + builder, issuer, strlen(issuer), &error)) goto cleanup; + if (!questdb_oidc_builder_scope( + builder, scope, strlen(scope), &error)) goto cleanup; + auth = questdb_oidc_builder_build(builder, &error); + if (!auth) goto cleanup; + if (!questdb_oidc_auth_sign_in(auth, &error)) goto cleanup; + + questdb_db_connect_options options; + questdb_db_connect_options_init(&options, sizeof options); + options.oidc_auth = auth; + db = questdb_db_connect_ex(conf, strlen(conf), &options, &error); + if (!db) goto cleanup; + + /* Borrow senders and readers from db for your application work. */ + status = 0; + +cleanup: + if (error) + { + size_t len = 0; + const char* message = questdb_error_msg(error, &len); + fprintf(stderr, "%.*s\n", (int)len, message); + } + questdb_db_close(db); + questdb_oidc_auth_free(auth); + questdb_oidc_builder_free(builder); + questdb_error_free(error); + return status; +} +``` + + + + +The default renderer prints a verification URL and code. Disable browser +opening with C++ `builder.open_browser(false)` or C +`questdb_oidc_builder_open_browser`; a custom UI can use `event_handler` / +`questdb_oidc_builder_event_handler`. For clickable links or QR codes, use +only the event's vetted `browser_target`, not its display-only verification +URLs. -### Unsupported auth paths +`offline_access` requests a refresh token; the IdP must actually issue one +for silent renewal to work. An expired token without a usable refresh token +reports `InteractionRequired`. Call `sign_in()` again on your UI thread, +never from a transport callback. Token lookups and reconnects never prompt. -The client supports only HTTP basic auth and static bearer-token auth. The -following are **not** supported: +Credentials stay in memory by default. To opt into persistence, use +`builder.default_file_token_store()` in C++, or +`questdb_oidc_builder_default_file_token_store` in C before building the auth. +The store writes **plaintext credentials**, including refresh tokens, and +currently supports writes only on Unix. See +[Credential storage and shutdown](/docs/security/oidc/#credential-storage-and-shutdown). -| Path | Status | Workaround | -|---|---|---| -| OIDC token acquisition or in-band refresh | Not supported. The client does not negotiate with an identity provider and has no callback to refresh a token mid-session. | QuestDB itself supports OIDC; see [OpenID Connect](/docs/security/oidc/). Acquire an access token out-of-band from your IdP, pass it via `token=...`, and rebuild the pool when the token nears expiry. | -| Mutual TLS (client certificates) | Not supported. The QuestDB server does not negotiate client certificates regardless of client. | Use bearer-token auth over `wss`. See the connect-string reference's [TLS section](/docs/connect/clients/connect-string/#tls). | -| Token rotation mid-session | Not supported. Credentials are presented once during the WebSocket upgrade and are not re-sent. | On token expiry, close the pool and build a fresh one with the new token. | +### Rotating bearer tokens + +For an unattended service, supply a source that obtains tokens from your +identity SDK, OAuth2 client-credentials flow, or secrets manager. A refreshing +provider caches the token and renews it on a dedicated thread. The native +client does not bundle an Azure or other identity SDK. + +These helpers take your application's token source and attach one shared +provider to a pool: + + + + +```cpp title="Use an application token source" +#include +#include +#include +#include + +using namespace questdb::ingress::literals; + +void query_with_tokens( + std::function fetch_token) +{ + auto tokens = questdb::token_provider::refreshing( + std::move(fetch_token)); + { + questdb::pool pool{"wss::addr=db.example.com:9000;", tokens}; + auto reader = pool.borrow_reader(); + auto cursor = reader.execute("SELECT current_user()"_utf8); + while (cursor.next_batch()) {} + } // Close clients before their provider. + tokens.close(); +} +``` + +The source returns `questdb::expiring_token{token, expires_at}`, where +`expires_at` is a `std::chrono::system_clock::time_point`. For a relative +lifetime, add the IdP's `expires_in` to the local time of receipt. Throw +`questdb::token_unavailable(message, retryable)` to classify a fetch failure. +Catch `questdb::error` around calls to this helper. + + + + +```c title="Use a token-source callback" +#include +#include +#include + +/* The caller initializes *error to NULL and frees it after a failure. */ +bool query_with_tokens( + questdb_token_source_fn fetch_token, + void* user_data, + questdb_token_release_fn release, + questdb_error** error) +{ + const char* conf = "wss::addr=db.example.com:9000;"; + questdb_db* db = NULL; + qwp_reader* reader = NULL; + qwp_reader_cursor* cursor = NULL; + bool ok = false; + questdb_token_provider* tokens = questdb_refreshing_token_provider_new( + fetch_token, user_data, release, NULL, error); + if (!tokens) return false; + + questdb_db_connect_options options; + questdb_db_connect_options_init(&options, sizeof options); + options.token_provider = tokens; + db = questdb_db_connect_ex(conf, strlen(conf), &options, error); + if (!db) goto cleanup; + reader = questdb_db_borrow_reader(db, error); + if (!reader) goto cleanup; + cursor = qwp_reader_execute( + reader, QDB_UTF8_LITERAL("SELECT current_user()"), error); + if (!cursor) goto cleanup; + while (qwp_reader_cursor_next_batch(cursor, error) != NULL) {} + ok = *error == NULL; + +cleanup: + qwp_reader_cursor_free(cursor); + qwp_reader_close(reader); + questdb_db_close(db); + questdb_token_provider_close(tokens); + questdb_token_provider_free(tokens); + return ok; +} +``` + +Implement `fetch_token(void* user_data, questdb_token_sink* sink)` in your +application. Report exactly one result per invocation: + +- `questdb_token_sink_set_token(sink, token, len, expires_at_ms, refresh_at_ms)` + copies the raw token. Expiry is Unix epoch **milliseconds**; pass `0` for + `refresh_at_ms` when the platform supplies no refresh hint. +- `questdb_token_sink_set_error(sink, retryable, retry_after_ms, msg, len)` + reports failure. Pass `-1` when there is no retry-after hint. + +After successful provider creation, `release(user_data)` owns cleanup, even +if connecting the pool subsequently fails. If provider creation itself +fails, ownership stays with the caller. Closing does not wait for an active +fetch, so release callback resources in `release`, not immediately after +`questdb_token_provider_close` or `free`. + + + + +The source is called serially on the refresher thread. Bound its network I/O +and do not call back into an attached client. In C++, capture source resources +by value or shared ownership: an in-flight fetch can outlive `close()`. + +By default, refresh happens around half the token lifetime; tokens with +60 seconds or less remaining are not handed out, and a caller with no usable +token waits up to 30 seconds. Adjust `handout_floor` / `cold_wait` in C++ +`refreshing_token_provider_options`, or the `_ms` fields in C +`questdb_refreshing_token_provider_options`, for shorter-lived tokens. +`await_ready()` / `questdb_token_provider_await_ready` can gate startup; +`status()` / `questdb_token_provider_get_status` expose fetch diagnostics. + +If your application already caches credentials, use +`questdb::token_provider::from` with a `bearer_token_provider` implementation, +or `questdb_token_provider_new` with a C vtable. Its `token()` / `get_token` +returns the raw token without `Bearer `; the optional `on_rejected` callback +invalidates a rejected cached token. Unlike a refreshing source, these +callbacks can run concurrently and must be fast and thread-safe. + +### Attachment, failures, and ownership + +Providers work with QWP/WebSocket pools, standalone senders and readers, and +ILP/HTTP senders, not TCP or UDP. The standalone attachment points are: + +| Client | C | C++ | +| --- | --- | --- | +| Sender | `line_sender_opts_oidc_auth` / `line_sender_opts_token_provider` | `opts.oidc_auth(auth)` / `opts.token_provider(tokens)` | +| Reader | `qwp_reader_from_conf_with_oidc` / `qwp_reader_from_conf_with_token_provider` | `reader(conf, auth)` / `reader(conf, tokens)` | + +Use `wss::` or `https::` to protect tokens in transit. Choose exactly one +credential source: OIDC, a token provider, a fixed `token`, or +`username`/`password`. Providers are attached in code; `token_provider` and +the `azure_*` provider keys in connection strings are rejected. + +A provider is consulted on each WebSocket connect/reconnect round and +normally once per ILP/HTTP flush, not per batch on an established WebSocket. +An eligible HTTP 401 can trigger one retry with a changed token; `403` or an +`insufficient_scope` challenge does not. See +[Token refresh and reconnects](/docs/security/oidc/#token-refresh-and-reconnects). + +C errors retain structured details through `questdb_error_oidc_get_view` and +`questdb_error_token_unavailable_get_view`; initialize each output view's +`struct_size` before calling. In C++, direct OIDC and reader operations can +throw `questdb::oidc::error`; sender failures retain `oidc_diagnostic()`. +Provider failures expose `token_unavailable_diagnostic()` on `questdb::error`. +A connection event named `credential_unavailable` means acquisition failed; +`auth_failed` means the server rejected the credential. + +Clients retain their own references. Freeing a C auth/provider handle or +destroying a C++ wrapper releases only that reference. Explicitly closing the +shared provider permanently disables token acquisition for all attached +clients, so close clients first. Cancelling sign-in leaves the auth usable; +clearing credentials does not revoke them at the IdP. ## Headers @@ -1579,6 +1863,12 @@ The pool owns reconnection and connection health so borrowers don't have to: `pool_reap=manual`, call `questdb_db_reap_idle(db)` (C++ `pool::reap_idle()`) yourself; it returns the number of connections it closed. +An established QWP/WebSocket sender retries later HTTP 401/403 rejections +with capped backoff, even with fixed credentials, and retains unacknowledged +rows. `auth_failed` therefore does not always mean the sender stopped. +Initial caller-awaited connection setup still fails on rejection. See +[Authentication failures and token providers](/docs/connect/wire-protocols/qwp-client-behavior/#authentication-failures-and-token-providers). + ### Which errors mean what {#which-errors-mean-what} Dispatch on `line_sender_error_get_code(err)` (C++ @@ -1610,7 +1900,7 @@ keys. | --- | --- | --- | | `sender_pool_min` | 1 | Warm/minimum ingestion connections. Once connections have been opened, this many stay open however long they sit idle. | | `sender_pool_max` | 4 | Hard cap on ingestion auto-grow. | -| `query_pool_min` | 1 | Warm/minimum query connections. | +| `query_pool_min` | 1 (`0` with `lazy_connect=true`) | Warm/minimum query connections. | | `query_pool_max` | 4 | Hard cap on query auto-grow. | | `acquire_timeout_ms` | 5000 | How long a borrow waits for a returned connection once its pool is at its cap, before returning an error. `0` fails fast. | | `idle_timeout_ms` | 60000 | Idle connections above the warm minimum are closed after this long. | @@ -1908,6 +2198,8 @@ The installed headers are the complete reference: | `questdb/ingress/qwp_sender.h` | Sender borrowing, buffers, chunks, and ingestion | | `questdb/ingress/line_sender.h` | Errors, table and column names, UTF-8 helpers | | `questdb/egress/qwp_reader.h` | Reader borrowing, queries, cursors, and Arrow result batches | +| `questdb/oidc.h` | Device-flow sign-in, discovery, rendering, persistence, and OIDC diagnostics | +| `questdb/token_provider.h` | Rotating token sources, providers, refresh settings, and status | Each has a `.hpp` counterpart wrapping the same surface in RAII types that throw `questdb::error` instead of returning `false`. diff --git a/documentation/connect/clients/connect-string.md b/documentation/connect/clients/connect-string.md index c35dda7d5..3a01265ce 100644 --- a/documentation/connect/clients/connect-string.md +++ b/documentation/connect/clients/connect-string.md @@ -242,12 +242,33 @@ the TLS handshake is server-authenticated only. `tls_roots` / ### Token providers {#token-providers} -*Applies to: ingress and egress. Java client 1.3.11 or later only.* +*Applies to: ingress and egress.* -A fixed `token` stops working when it expires. A token provider requests -bearer tokens from an identity provider instead, and renews them in the -background before they expire, so every connection, reconnect, and failover -presents a valid token: +A fixed `token` does not follow credential rotation. A token provider supplies +a current bearer token for each connection, reconnect, and failover without +rebuilding the client. The C, C++, Rust, and Python clients accept providers +through their in-code APIs, not this configuration string: + +- [C and C++](/docs/connect/clients/c-and-cpp/#rotating-bearer-tokens): native + client 7.3.0 or later. +- [Rust](/docs/connect/clients/rust/#rotating-bearer-tokens): `questdb-rs` 7.3.0 + or later. +- [Python](/docs/connect/clients/python/#rotating-bearer-tokens): client 6.1.0 + or later. + +These clients also support interactive +[OIDC device sign-in](/docs/security/oidc/#device-authorization-flow). A +provider cannot be combined with fixed `token`, `username`, or `password` +credentials, or with a second provider. The `token_provider`, `azure_resource`, +`azure_client_id`, and `azure_credential` keys below are rejected by these +clients on every connection-string scheme. Use `wss` for QWP or `https` for +ILP so the bearer token is encrypted in transit. + +#### Java connection-string providers + +The following provider keys are available in **Java client 1.3.11 or later +only**. Java can select an Azure provider from the connection string and +refresh its tokens in the background: ``` wss::addr=node-a:9000,node-b:9000;token_provider=azure;azure_resource=api://;azure_credential=managed_identity; diff --git a/documentation/connect/clients/python.md b/documentation/connect/clients/python.md index 2fcc98a00..ce3e38ebf 100644 --- a/documentation/connect/clients/python.md +++ b/documentation/connect/clients/python.md @@ -91,8 +91,10 @@ db = questdb.connect("ws::addr=localhost:9000;") `connect()` accepts only `ws`/`wss` configuration strings; ILP schemes such as `http::` or `tcp::` belong to the [legacy Sender](#legacy-ilp-clients). -It parses and validates the string without opening a connection: the first -`sender()`, `dataframe()`, or `query()` call opens one, and later calls +In client 6.1.0, it opens the warm minimum connections eagerly, so +connection and authentication errors can surface from `connect()`. Set +`lazy_connect=True` to defer connections: row senders queue locally and +connect in the background, and queries connect on first use. Later operations reuse pooled connections. Servers without QWP support fail during the WebSocket upgrade; see [protocol versioning](/docs/connect/wire-protocols/overview/#versioning). @@ -128,7 +130,10 @@ db = questdb.connect(os.environ["QDB_CLIENT_CONF"]) ## Authentication and TLS -Put credentials and TLS settings in the same configuration string: +Put fixed credentials and TLS settings in the configuration string. For +interactive sign-in or expiring credentials, use an +[OIDC helper](#oidc-device-flow) or +[rotating token provider](#rotating-bearer-tokens) instead: ```python import questdb @@ -143,13 +148,13 @@ token = questdb.connect( ``` Authentication happens during the WebSocket upgrade, before any data is -exchanged. Bad credentials raise `QuestDBErrorCode.AuthError` from the -first operation that needs the connection, not from `connect()`. Queries and -`dataframe()` calls raise it directly. Row senders connect in the -background, so a fire-and-forget `flush()` can return before the upgrade -fails; the error then surfaces from `flush(wait=True)`, `wait()`, or the -next call on the lease — and immediately through the -[connection listener](#failover-and-errors) as an `AuthFailed` event. +exchanged. Rejected credentials raise `QuestDBErrorCode.AuthError` from +connection setup or an operation that waits for it. With lazy connection +setup, a fire-and-forget `flush()` can return before the upgrade fails; +observe `flush(wait=True)`, `wait()`, and the +[connection listener](#failover-and-errors). The listener reports server +rejections as `AuthFailed` and token-acquisition failures as +`CredentialUnavailable`. With `wss`, the default combines the bundled `webpki` root store with the operating-system certificate store. Override it with configuration keys: @@ -164,16 +169,193 @@ operating-system certificate store. Override it with configuration keys: See the [connect string reference](/docs/connect/clients/connect-string/) for the full grammar. -### Unsupported auth paths +Mutual TLS with client certificates is not supported. Use bearer-token +authentication over `wss` instead. -The client supports only HTTP basic auth and static bearer-token auth. The -following are **not** supported: +### OIDC device flow -| Path | Status | Workaround | -| --- | --- | --- | -| OIDC token acquisition or in-band refresh | Not supported. The client does not negotiate with an identity provider and cannot refresh a token mid-session. | QuestDB itself supports OIDC; see [OpenID Connect](/docs/security/oidc/). Acquire an access token out-of-band from your IdP, pass it via `token=...`, and rebuild the handle when the token nears expiry. | -| Mutual TLS (client certificates) | Not supported. The QuestDB server does not negotiate client certificates regardless of client. | Use bearer-token auth over `wss`. | -| Token rotation mid-session | Not supported. The handle keeps the credentials it was built with and presents them on every connection it opens — including reconnects and failover, so an expired token also breaks mid-session reconnection. | On token expiry, close the handle and build a fresh one with the new token. | +Python client **6.1.0 or later** supports browser-based sign-in from scripts, +terminals, and remote Jupyter notebooks through `questdb.auth.OidcDeviceAuth`. +OIDC needs no extra Python package. The server must be QuestDB Enterprise +with OIDC configured, and the IdP application must allow the device grant. + +Call `sign_in()` before connecting, then pass the authentication object, not +a fixed token, to the pool: + +```python title="Sign in and query with OIDC" +import questdb +from questdb.auth import OidcDeviceAuth + +with OidcDeviceAuth.from_questdb( + "https://db.example.com:9000", + issuer="https://idp.example.com/realms/questdb", + scope="openid offline_access", +) as auth: + auth.sign_in() + with questdb.connect( + "wss::addr=db.example.com:9000;", oidc_auth=auth + ) as db: + with db.query("SELECT current_user()") as result: + print(list(result.iter_rows())) +``` + +Replace the QuestDB URL and issuer with your deployment's values. Discovery +reads QuestDB's `/settings`; the issuer pins the IdP and enables discovery of +its device endpoint if QuestDB does not advertise one. To skip QuestDB +discovery, construct `OidcDeviceAuth` with keyword arguments `client_id`, +`device_authorization_endpoint`, `token_endpoint`, and `scope`. Match +`groups_in_token` to the server: `True` selects the ID token, otherwise the +access token. See +[Device authorization flow](/docs/security/oidc/#device-authorization-flow). + +Only `sign_in()` can prompt or open a browser. It displays a terminal code and +URL, or a clickable prompt in Jupyter. Browser opening defaults to off in +Jupyter and on elsewhere; set `open_browser=False` to disable it explicitly. +Optional `qr=True` needs `qrcode` (`qrcode[pil]` for notebook images), and a +custom `renderer=` can own the UI. Use only its vetted `browser_target` for +links or QR codes, and HTML-escape text when rendering HTML. + +Token lookups and reconnects use cached credentials or silent refresh, never +an interactive prompt. `offline_access` requests a refresh token; renewal +works only if the IdP issues one. Without a usable refresh token after expiry, +`OidcInteractionRequired` asks your application to call `sign_in()` again. +For CI or headlessly executed notebooks, prefer machine credentials below; +`interactive=False` makes sign-in fail fast when user interaction is needed. + +Credentials stay in memory by default. Pass +`token_store=FileTokenStore.at_default_location()` to `from_questdb`, importing +`FileTokenStore` from `questdb.auth`, to persist them. This writes plaintext +credentials, including refresh tokens, and currently requires POSIX storage +for writes. Use memory-only authentication on other platforms; custom Python +token stores are not supported. See +[Credential storage and shutdown](/docs/security/oidc/#credential-storage-and-shutdown). + +### Rotating bearer tokens + +For an unattended application, use `RefreshingTokenProvider` with a source +that fetches a token and its expiry from your identity platform. The source +can be a callable or an object with `fetch_token()`, returning: + +- `ExpiringToken(token, expires_at)`, where expiry is Unix epoch seconds or an + aware `datetime`, or `ExpiringToken.expires_in(token, lifetime)` for the + IdP's relative lifetime in seconds. +- A `(token, expires_at)` tuple. +- An SDK token with `token` and `expires_on` attributes, such as Azure's + `AccessTokenInfo`. Its optional `refresh_on` hint is also respected. + +The following example uses a Microsoft Entra ID managed identity. Install +`azure-identity` separately and set `QUESTDB_ENTRA_SCOPE` to your QuestDB +application's scope, such as `api:///.default`. Configure the +server to accept [application tokens](/docs/security/oidc/#managed-identities-and-service-principals). +No cloud SDK is bundled with the QuestDB client. + +```python title="Use a managed identity with a shared token cache" +import os + +from azure.identity import ManagedIdentityCredential +import questdb +from questdb.auth import RefreshingTokenProvider + +credential = ManagedIdentityCredential() +tokens = RefreshingTokenProvider( + lambda: credential.get_token_info(os.environ["QUESTDB_ENTRA_SCOPE"]), + name="market-data-service", +) +try: + with questdb.connect( + "wss::addr=db.example.com:9000;", bearer_token_provider=tokens + ) as db: + with db.query("SELECT current_user()") as result: + print(list(result.iter_rows())) +finally: + tokens.close(wait=True, timeout=30) + credential.close() +``` + +The provider starts fetching immediately and refreshes on its own thread at +roughly half the token lifetime, with jitter and failure backoff. By default, +it hands out tokens only while more than 60 seconds remain and waits up to +30 seconds when no usable token is cached. Adjust `handout_floor` and +`cold_wait` (seconds or `timedelta`) for shorter-lived credentials. +`tokens.await_ready(timeout=30)` gates startup; `tokens.status` exposes expiry, +last success, and last failure without exposing credentials. + +Raise `TokenUnavailableError(message, retryable=True, retry_after=...)` from +your source for transient failures. Use `retryable=False` for configuration +failures; unclassified exceptions count as retryable. Bound the source's +network I/O: closing a provider cannot cancel a callback already running. +`close(wait=True, timeout=...)` drains it or raises `TimeoutError`; the +provider remains closed. Close clients before their provider and before +interpreter exit. + +If you already cache tokens, `bearer_token_provider=` also accepts a callable +returning the raw token string, or an object with `token()` and optional +`on_rejected(token, http_status)`. These callbacks can run on native threads; +keep them fast and thread-safe, and never call back into an attached client. +For an OAuth2 client-credentials source, see the +[token-provider recipe](https://github.com/questdb/py-questdb-client/blob/v6.1.0/docs/token_providers.rst#oauth2-client-credentials). + +### Transport integration and recovery + +`questdb.connect`, `QuestDB.from_conf`, `Sender`, `Sender.from_conf`, and +`Sender.from_env` accept `oidc_auth=` or `bearer_token_provider=`. Choose one; +neither can be combined with fixed `token`, `username`, or `password` +credentials. Providers work with HTTP(S) and QWP/WebSocket, not TCP or UDP. +The `token_provider` and `azure_*` provider keys in configuration strings are +rejected; pass the provider as a Python argument instead. + +Use `https::` or `wss::`: plaintext connections warn but still send bearer +credentials in cleartext. The provider is consulted on every WebSocket +connect/reconnect round and normally once per ILP/HTTP flush, not per batch +on an established WebSocket. An eligible HTTP 401 can trigger one retry with +a changed token; `403` and `insufficient_scope` do not trigger that retry. See +[Token refresh and reconnects](/docs/security/oidc/#token-refresh-and-reconnects). + +`OidcError` and `TokenUnavailableError` are `QuestDBError` subclasses. Catch +`OidcInteractionRequired` before a generic retry handler: if +`acquisition_busy` is true, defer until the other acquisition finishes; +otherwise arrange explicit `sign_in()` outside callbacks. A background +reconnect can report acquisition failure through a `CredentialUnavailable` +connection event rather than a raised exception. + +`cancel_sign_in()` or Ctrl-C cancels only the current sign-in attempt. +`clear()` removes local cached and persisted credentials without revoking +them at the IdP. `close()` permanently disables the shared auth and stops +attached clients on their next token pull; it is not a way to cancel one +login. To recover from closure, create a new provider and new clients. +Do not carry providers or clients across `fork()`; create them in fresh +workers started with `spawn` or `forkserver`. + +### PGWire adapters + +`questdb.auth.sqlalchemy_engine` and `psycopg_connect` accept the same OIDC +or application token providers. Enable +[`acl.oidc.pg.token.as.password.enabled`](/docs/configuration/oidc/#acloidcpgtokenaspasswordenabled) +on the server. The adapters use username `_sso` and the current token as the +password. Install SQLAlchemy and psycopg/psycopg2 separately. + +Inside the lifetime of a signed-in `auth` or an open refreshing provider: + +```python title="Open a PGWire connection with the current token" +from questdb.auth import psycopg_connect + +with psycopg_connect( + auth, + "https://db.example.com:9000", + sslrootcert="/etc/ssl/questdb-ca.pem", +) as connection: + with connection.cursor() as cursor: + cursor.execute("SELECT current_user()") + print(cursor.fetchone()) +``` + +SQLAlchemy resolves a token for each new physical connection, not every +checkout; `psycopg_connect` resolves one at connection time. Existing PGWire +sessions are not reauthenticated mid-session. Neither adapter prompts. Remote +hosts default to `sslmode=verify-full`, so supply a trusted CA as above +(`connect_args={"sslrootcert": ...}` for SQLAlchemy). Dispose SQLAlchemy +engines before closing their provider. See the +[full adapter reference](https://github.com/questdb/py-questdb-client/blob/v6.1.0/docs/auth.rst#pg-wire-and-manual-token-use). ## The pool @@ -183,10 +365,11 @@ borrows a connection, opening one on demand, and returns it when the lease or result closes; when every slot is busy, a borrow waits for a free one (see [Pool settings](#pool-settings)). -`connect()` performs no blocking network I/O. `dataframe()`, `query()`, -and `server_info()` connect on first use; `sender()` connects in the -background, so call `flush(wait=True)` or `wait()` to surface connection and -delivery errors. +By default, `connect()` opens the warm minimum connections. With +`lazy_connect=True`, `dataframe()`, `query()`, and `server_info()` connect on +first use, while row senders connect in the background. Call +`flush(wait=True)` or `wait()` to observe delivery rather than only local +queue acceptance. ### Choose an API @@ -955,7 +1138,7 @@ db.close() | Key | Default | Guidance | | --- | --- | --- | -| `sender_pool_min` / `query_pool_min` | `1` | Warm minimum retained per pool after connections have been opened. Set it near steady concurrent use. | +| `sender_pool_min` / `query_pool_min` | `1` (`query_pool_min=0` with lazy connection setup) | Warm minimum retained per pool after connections have been opened. Set it near steady concurrent use. | | `sender_pool_max` / `query_pool_max` | `4` | Per-pool growth cap. Set it at or above peak concurrent leases. | | `acquire_timeout_ms` | `5000` | How long a borrow waits for a free slot before failing; `0` fails fast. | | `idle_timeout_ms` | `60000` | Idle lifetime for connections above the pool minimum. | @@ -987,12 +1170,21 @@ uses these keys: | `reconnect_initial_backoff_millis` | `100` | | `reconnect_max_backoff_millis` | `5000` | -Authentication failures, protocol-version failures, and terminal data -rejections are not retried. When a `flush(wait=True)` or `wait()` makes no -progress within the budget it raises `QuestDBErrorCode.FailoverRetry`: the -rows remain queued and background delivery continues, so retry the wait or -extend the budget — do not rebuild and flush the same rows, which would -duplicate them once the original rows land. +A caller-awaited initial connection fails on authentication rejection. Once +a QWP/WebSocket sender has connected, later HTTP 401/403 rejections are retried +with capped backoff, including with fixed credentials. Unacknowledged rows +stay queued, and the listener reports `AuthFailed`. A provider that cannot +supply a token instead reports `CredentialUnavailable`. Even a +`TokenUnavailableError` with `retryable=False` does not stop background +reconnects; close the provider or sender from application code, outside its +callbacks, to stop them. + +Protocol-version failures and terminal data rejections are not retried. A +`flush(wait=True)` or `wait()` that times out can retain the underlying +authentication diagnostic; transport-only timeouts report +`QuestDBErrorCode.FailoverRetry`. Rows remain queued and background delivery +continues, so retry the wait rather than flushing the same rows again. See +[Authentication failures and token providers](/docs/connect/wire-protocols/qwp-client-behavior/#authentication-failures-and-token-providers). To observe connection state, pass a listener when connecting. It receives one `ConnectionEvent` per state transition — `kind` @@ -1100,7 +1292,7 @@ Codes you will most often dispatch on: | Code | Raised when | Action | | --- | --- | --- | | `ConfigError` | Malformed configuration string, or a non-`ws`/`wss` scheme passed to `connect()`. | Fix the string. | -| `AuthError` | Credentials rejected during the WebSocket upgrade. | Fix or rotate credentials. | +| `AuthError` | Credentials rejected during the WebSocket upgrade, or a non-retryable token-acquisition failure. | Inspect `OidcError` / `TokenUnavailableError` details where present; sign in, repair the source, or replace the credential. | | `InvalidApiCall` | Client-side misuse: exhausted pool, a lease's previous result still undrained, a closed handle. | Fix the call pattern, or raise the pool caps. | | `InvalidTimestamp` | Bad `at` value (wrong type, `NaT`). | Pass `TimestampNanos`, a timezone-aware `datetime`, or `ServerTimestamp`. | | `FailoverRetry` | `flush(wait=True)` or `wait()` made no progress within the budget. | Retry the wait; do not re-send the rows. | diff --git a/documentation/connect/clients/rust.md b/documentation/connect/clients/rust.md index 226051678..d6232a0cc 100644 --- a/documentation/connect/clients/rust.md +++ b/documentation/connect/clients/rust.md @@ -16,7 +16,7 @@ Add the client: ```text title="Cargo.toml" [dependencies] -questdb-rs = "7" +questdb-rs = "7.3" ``` The default features enable both directions: `sync-sender` for pooled @@ -125,20 +125,21 @@ use questdb::QuestDb; let db = QuestDb::connect("ws::addr=localhost:9000;")?; ``` -`QuestDb::connect` parses the connect string and doesn't normally perform any -blocking network I/O. The exception to that happens when store-and-forward -durability is in use, and the client detects unsent data left over from a -previously crashed client. The pool may reopen the sender directories that -hold that data at construction and start replaying them in the background. +In client 7.3.0, `QuestDb::connect` opens the warm minimum connections eagerly, +so authentication, TLS, and connection errors can surface from `connect()`. +Set `lazy_connect=true` to defer connections: ingestion senders queue data +locally and connect in the background, and readers connect on first borrow. -The ingestion sender creates its local producer on first borrow and connects its -delivery runner in the background, so it can queue data while the server is -temporarily unavailable. Reader errors and direct Arrow/Polars transport errors -surface from the borrow or during operation rather than from `connect()`. +With disk-backed store-and-forward, the pool may also reopen sender directories +holding unsent data from a previous process and replay them in the background. +Later borrows and operations can still fail when opening or reconnecting a +connection. ## Authentication and TLS -Put the credentials and TLS settings in the connect string: +Put fixed credentials and TLS settings in the connect string. For interactive +sign-in or expiring credentials, attach an [OIDC helper](#oidc-device-flow) or +[rotating token provider](#rotating-bearer-tokens) instead: ```rust use questdb::QuestDb; @@ -163,6 +164,155 @@ choices have feature requirements: | `tls_roots_password=...` | Unlocks a JKS or PKCS#12 store named by `tls_roots`. | | `tls_verify=unsafe_off` | Enable `insecure-skip-verify`; use only in controlled tests. | +### OIDC device flow + +Client **7.3.0 or later** provides `OidcDeviceAuth` for interactive sign-in to +QuestDB Enterprise through an identity provider. Enable the optional `oidc` +feature in addition to the default features: + +```toml title="Cargo.toml" +[dependencies] +questdb-rs = { version = "7.3", features = ["oidc"] } +``` + +The IdP application must allow the device grant. The client discovers settings +from QuestDB's `/settings`; pin the issuer so it can also discover the IdP's +device endpoint when QuestDB does not advertise one. Replace both URLs below +with your deployment's values: + +```rust title="Sign in and share OIDC across the pool" +use std::sync::Arc; + +use questdb::{oidc::OidcDeviceAuth, QuestDb}; + +fn main() -> questdb::Result<()> { + let auth = Arc::new( + OidcDeviceAuth::from_questdb("https://db.example.com:9000") + .issuer("https://idp.example.com/realms/questdb") + .scope("openid offline_access") + .build()?, + ); + auth.sign_in()?; // Only this call may prompt or open a browser. + + let db = QuestDb::connect_with_token_provider( + "wss::addr=db.example.com:9000;", + { + let auth = Arc::clone(&auth); + move || auth.token() + }, + )?; + let mut reader = db.borrow_reader()?; + let mut cursor = reader.execute("SELECT current_user()")?; + while let Some(batch) = cursor.next_batch()? { + println!("{} row(s)", batch.row_count()); + } + Ok(()) +} +``` + +The same provider serves every pooled sender and reader. Token lookups never +prompt: they use the cache or refresh silently if the IdP issued a refresh +token, usually requested with `offline_access`. Without a usable refresh +token after expiry, they report `OidcErrorKind::InteractionRequired`; call +`sign_in()` again on a UI thread. See +[Device authorization flow](/docs/security/oidc/#device-authorization-flow) +for scope overrides and access-token versus ID-token selection. + +Use `.open_browser(false)` on the builder for a remote terminal; the code and +verification URL are still printed. A custom `oidc::Renderer` can own the UI. +Credentials stay in memory unless the builder receives +`.token_store(FileTokenStore::at_default_location()?)`, with +`FileTokenStore` imported from `questdb::oidc`. That opt-in writes plaintext +credentials and currently supports mutations only on Unix. For encrypted +storage, implement `oidc::TokenStore`. See +[Credential storage and shutdown](/docs/security/oidc/#credential-storage-and-shutdown). + +### Rotating bearer tokens + +For machine credentials, `RefreshingTokenProvider` wraps an application-defined +`TokenSource` with a shared cache and proactive background refresh. No `oidc` +feature is required for this API, and no identity SDK is bundled. A source can +be a closure returning `questdb::Result` or a type implementing +`TokenSource`: + +```rust title="Attach an application token source" +use questdb::{ + token_provider::{RefreshingTokenProvider, TokenSource}, + QuestDb, +}; + +fn query_with_tokens( + source: impl TokenSource + 'static, +) -> questdb::Result<()> { + let tokens = RefreshingTokenProvider::builder(source) + .name("market-data-service") + .build()?; + { + let db = QuestDb::connect_with_token_provider( + "wss::addr=db.example.com:9000;", + tokens.clone(), + )?; + let mut reader = db.borrow_reader()?; + let mut cursor = reader.execute("SELECT current_user()")?; + while cursor.next_batch()?.is_some() {} + } // Close clients before their provider. + tokens.close(); + Ok(()) +} +``` + +Your source obtains credentials through an identity SDK, OAuth2 flow, or +secrets manager. Return `ExpiringToken::new(token, expires_at)` with a +`SystemTime`, or `ExpiringToken::expires_in(token, lifetime)` with a `Duration` +computed from the IdP's reported lifetime, not an invented expiry. Return +`Error::token_unavailable(retryable, retry_after, message)` for failures. + +The source runs serially on a dedicated thread. Bound its network calls and +do not call into clients attached to the provider. Refresh is scheduled at +roughly half the token lifetime, with jitter and failure backoff. Defaults +include a 60-second `handout_floor` and a 30-second `cold_wait`; reduce the +floor through the builder if your IdP issues tokens that live for 60 seconds +or less. Use `await_ready(Duration)` to gate startup and `status()` to inspect +fetch diagnostics without reading credentials. + +If your application already caches tokens, pass a closure returning +`questdb::Result` directly, or implement `BearerTokenProvider` with +`token()` and optional `on_rejected()`. Return the raw token, without +`Bearer `. These callbacks must be fast and thread-safe. + +### Attachment, failures, and shutdown + +For standalone clients, attach a provider with: + +- `SenderBuilder::http_token_provider` for ILP/HTTP. +- `SenderBuilder::qwp_ws_token_provider` for QWP/WebSocket ingestion. +- `ReaderConfig::token_provider` for QWP/WebSocket queries. + +Use `https::` or `wss::`. Providers are not supported over TCP or UDP, and +cannot be combined with fixed `token`, `username`, or `password` credentials. +The `token_provider` and `azure_*` connection-string keys are rejected; attach +providers through the Rust API. + +Tokens are resolved per WebSocket connect/reconnect round and normally once +per ILP/HTTP flush. An eligible HTTP 401 triggers one retry only if the +provider supplies a different token; `403` and `insufficient_scope` do not +trigger that refresh retry. An established WebSocket is not reauthenticated +for each batch. See +[Token refresh and reconnects](/docs/security/oidc/#token-refresh-and-reconnects). + +Inspect `err.oidc_error()` (with `oidc` enabled) for OIDC details and +`err.as_token_unavailable()` for a provider's retryability and retry-after +hint. `CredentialUnavailable` connection events distinguish acquisition +failures from the server rejecting a credential (`AuthFailed`). `ReaderConfig::upgrade_headers()` now returns +`Result>`, because acquiring the credential can fail. + +Cloned refreshing providers share one cache. Dropping one clone, or an `Arc` +reference to OIDC auth, does not close the others. Explicit `close()` disables +the shared provider permanently, so close clients first. It does not cancel +an in-flight source fetch; that fetch finishes before its source is dropped, +and its result is discarded. Do not carry providers or clients across +`fork()`; create them in fresh worker processes. + ## The pool `QuestDb` owns reusable QWP/WebSocket connections. Create one pool per @@ -870,7 +1020,7 @@ and is not part of that thread-bound borrow list. | --- | --- | --- | | `sender_pool_min` | `1` | Warm minimum of ingestion connections retained once they have been opened. Set it near steady concurrent use. | | `sender_pool_max` | `4` | Ingestion pool growth cap. Set it at or above peak concurrent sender borrows. | -| `query_pool_min` | `1` | Warm minimum of reader connections retained once they have been opened. | +| `query_pool_min` | `1` (`0` with `lazy_connect=true`) | Warm minimum of reader connections retained once they have been opened. | | `query_pool_max` | `4` | Reader pool growth cap. Set it at or above peak concurrent reader borrows. | | `acquire_timeout_ms` | `5000` | How long a borrow waits for a returned connection once its pool is at its cap. `0` fails fast. | | `idle_timeout_ms` | `60000` | Idle lifetime for connections above the warm minimum. | @@ -949,9 +1099,18 @@ ingestion retry budget uses these keys: | `reconnect_initial_backoff_millis` | `100` | | `reconnect_max_backoff_millis` | `5000` | -Authentication failures, protocol-version failures, and terminal data -rejections are not retried. A terminal writer is retired when its borrow is -dropped; there is no pooled `must_close()` or async-error polling API. +A caller-awaited initial connection fails on authentication rejection. Once +a QWP/WebSocket sender has connected, later HTTP 401/403 rejections are retried +with capped backoff, including with fixed credentials; unacknowledged rows +stay queued. A `CredentialUnavailable` event means the provider could not +supply a token, while `AuthFailed` means the server rejected one. A permanent +foreground token-fetch failure does not by itself stop background reconnects; +close the provider or sender to stop them. See +[Authentication failures and token providers](/docs/connect/wire-protocols/qwp-client-behavior/#authentication-failures-and-token-providers). + +Protocol-version failures and terminal data rejections are not retried. A +terminal writer is retired when its borrow is dropped; there is no pooled +`must_close()` or async-error polling API. Reader failover has a separate policy: @@ -1019,6 +1178,7 @@ integrations only when your application uses them: | `ring-crypto` | Yes | The default TLS cryptography backend. Do not combine it with `aws-lc-crypto`. | | `arrow-ingress` / `arrow-egress` | No | Arrow ingestion / Arrow query results. The `arrow` feature enables both. | | `polars-ingress` / `polars-egress` | No | Polars ingestion / Polars query results. The `polars` feature enables both. | +| `oidc` | No | OIDC device-flow sign-in and silent refresh with `OidcDeviceAuth`. Rotating application token providers do not need this feature. | | `ndarray` | No | `Buffer::column_arr` from `ndarray` views. | | `rust_decimal` / `bigdecimal` | No | Row-buffer decimal values from those crates. Decimal strings need neither feature. | | `chrono-timestamp` | No | Timestamp values built from `chrono::DateTime`. | @@ -1027,8 +1187,8 @@ integrations only when your application uses them: | `almost-all-features` | No | Client development and testing with most compatible features. It excludes Arrow and Polars. | `almost-all-features` is intended for client development and CI. It extends -the defaults with QWP/UDP, both TLS root sources, `insecure-skip-verify`, JSON -test helpers, timestamp and array integrations, and both decimal crates. It +the defaults with QWP/UDP, OIDC, both TLS root sources, `insecure-skip-verify`, +JSON test helpers, timestamp and array integrations, and both decimal crates. It selects `ring-crypto`, omits the mutually exclusive `aws-lc-crypto`, and excludes `arrow` and `polars`. Prefer the specific features your application uses. @@ -1051,6 +1211,7 @@ These are the supported pool entry points: | Method | Feature | Purpose | | --- | --- | --- | | `QuestDb::connect(conf: &str)` | `sync-sender-qwp-ws` | Parse the connect string and create the pool. | +| `QuestDb::connect_with_token_provider(conf, provider)` | `sync-sender-qwp-ws` | Share one rotating bearer-token provider across ingestion and query connections. | | `new_buffer()` | `sync-sender-qwp-ws` | Create a caller-owned QWP/WebSocket `Buffer` using the pool's name limit. | | `borrow_sender()` | `sync-sender-qwp-ws` | Borrow a `BorrowedSender` for buffers, chunks, or Arrow batches. | | `borrow_reader()` | `sync-sender-qwp-ws` + `sync-reader-qwp-ws` | Borrow a `BorrowedReader`. | diff --git a/documentation/connect/wire-protocols/qwp-client-behavior.md b/documentation/connect/wire-protocols/qwp-client-behavior.md index 58928774b..8702a6ce4 100644 --- a/documentation/connect/wire-protocols/qwp-client-behavior.md +++ b/documentation/connect/wire-protocols/qwp-client-behavior.md @@ -363,9 +363,11 @@ Exhausting `sf_append_deadline_millis` on `append()` with `sf_max_total_bytes` reached fails only that call; the sender stays alive and keeps retrying. ⚠ The Java client treats an authentication rejection or a durable-ack capability mismatch as terminal only -before the sender first connects, and retries them afterwards. A Node.js -sender that uses store-and-forward or `initial_connect_retry=async` does the -same once it has connected, as do some browser senders: see +before the sender first connects, and retries them afterwards. C, C++, and +Rust 7.3.0 and Python 6.1.0 also retry authentication rejections after the +sender has connected, including with fixed credentials. A Node.js sender that +uses store-and-forward or `initial_connect_retry=async` retries both kinds +once it has connected, as do some browser senders: see [Reconnect and outage handling](#reconnect-and-outage-handling). ### Reconnect and outage handling @@ -400,12 +402,13 @@ and apply back-pressure to the producer rather than dropping data. ::: -⚠ In the Java client, a running sender also retries authentication failures. -So does a Node.js sender that uses store-and-forward or -`initial_connect_retry=async`, and a browser sender that authenticates with -`sessionBootstrap` and was started with `initialConnectMode: "async"`, once it -has connected. A browser sender that authenticates with `auth` cannot see why -an upgrade failed, so once it has connected it retries a rejected credential +⚠ In Java, C/C++ and Rust 7.3.0 or later, and Python 6.1.0 or later, a running +sender also retries authentication failures. So does a Node.js sender that +uses store-and-forward or `initial_connect_retry=async`, and a browser sender +that authenticates with `sessionBootstrap` and was started with +`initialConnectMode: "async"`, once it has connected. A browser sender that +authenticates with `auth` cannot see why an upgrade failed, so once it has +connected it retries a rejected credential like an outage, whatever its mode, and reports `endpoint-attempt-failed` rather than `auth-failed`. Other senders treat authentication failures as terminal. See @@ -414,15 +417,22 @@ terminal. See ### Authentication failures and token providers A client sends its credential only on the WebSocket upgrade request, and never -authenticates an established connection again. A client with a token provider, -such as the [Java client](/docs/connect/clients/java/#token-providers), asks -the provider for the credential at the start of every connect round: the -initial connect, every reconnect and failover, every orphan-drain connect, and -every query client connect. When the provider fails, the round ends without -contacting any endpoint, as a **credential-unavailable** failure, which the -provider classifies as retryable or permanent. An exception from a provider -that the application supplies counts as permanent, unless it is a token -failure marked as retryable. +authenticates an established connection again. The +[Java](/docs/connect/clients/java/#token-providers), +[C/C++](/docs/connect/clients/c-and-cpp/#rotating-bearer-tokens), +[Rust](/docs/connect/clients/rust/#rotating-bearer-tokens), and +[Python](/docs/connect/clients/python/#rotating-bearer-tokens) clients support +token providers. They ask the provider for a credential at the start of every +connect round: the initial connect, every reconnect and failover, every +orphan-drain connect, and every query client connect. When the provider fails, +the round ends without contacting any endpoint, as a +**credential-unavailable** failure, which the provider can classify as +retryable or permanent. + +In Java, an unclassified exception from an application provider counts as +permanent. In C, C++, Rust, and Python, an unclassified token-source failure +counts as retryable. Classify failures explicitly when implementing a source. +A closed provider is terminal regardless of the retry policy. When an upgrade that presented a token from a provider receives `401`, without a Bearer `error` or with `error="invalid_token"`, the client tells the provider @@ -443,20 +453,31 @@ fails, or the token did not change, the phase decides what happens: *Retry indefinitely* means with capped exponential backoff for as long as the sender is open. No setting bounds it, because store-and-forward must not end a -live producer. The client reports each failed round to the error handler as a -retriable `SECURITY_ERROR`, and in its connection health, so an application -that prefers a hard failure can close the sender itself. A query that fails -names the failure class in its error, and the next query connects again. - -⚠ This is the behaviour of the Java client 1.3.11 and later. The browser -client's `auth` option also accepts a function that supplies a credential -before every connection attempt, and some Node.js and browser senders retry a -rejected credential once they have connected, as described in +live producer. Java reports each failed round to the error handler as a +retriable `SECURITY_ERROR`, and in its connection health. C, C++, Rust, and +Python distinguish `credential_unavailable` / `CredentialUnavailable` +connection events from server rejection events (`auth_failed` / `AuthFailed`). +An application that prefers a hard failure can close the sender from its own +code, outside the sender's callbacks. + +The table describes Java 1.3.11 and later. Native clients 7.3.0 and Python +6.1.0 follow the same sender-phase distinction: a foreground failure does not +mean an established sender stopped reconnecting. An OIDC provider that needs +user interaction ends a caller-awaited connection attempt; arrange explicit +sign-in, never a prompt inside the reconnect loop. Their readers apply their +own failover limits to recoverable acquisition failures; after an exhausted +failover, a later query can connect again. A server authentication rejection +still fails the query after any eligible one-time token replacement. + +The browser client's `auth` option also accepts a function that supplies a +credential before every connection attempt, and some Node.js and browser +senders retry a rejected credential once they have connected, as described in [Reconnect and outage handling](#reconnect-and-outage-handling). A browser query session that authenticates with `auth` cannot see the status of a failed upgrade, so instead of failing the query it fails over as it would during an -outage, until its failover limits run out. The other clients send fixed credentials only, -and treat a `401` or `403` as terminal in every phase. +outage, until its failover limits run out. Other clients without these +capabilities send fixed credentials and treat a `401` or `403` as terminal in +every phase. ### Ingest endpoint walk (`addr=a:9000,b:9000,...`) diff --git a/documentation/security/oidc.mdx b/documentation/security/oidc.mdx index 09c618f5c..56a145b6e 100644 --- a/documentation/security/oidc.mdx +++ b/documentation/security/oidc.mdx @@ -1,6 +1,6 @@ --- -title: OpenID Connect (OIDC) Integration -description: Set up OpenID Connect (OIDC) in QuestDB Enterprise for Web Console SSO, service authentication with Entra ID managed identities, token validation, and troubleshooting. +title: OpenID Connect (OIDC) integration +description: Configure OIDC for QuestDB Enterprise, including Web Console SSO, client device-flow sign-in, rotating bearer tokens, and token validation. --- import Screenshot from "@theme/Screenshot"; @@ -15,6 +15,9 @@ QuestDB. It is a convenient way to integrate QuestDB into your enterprise environment, and it provides SSO (Single Sign-On) for the [Web Console](/docs/getting-started/web-console/overview/). +Client applications can use [device-flow sign-in](#device-authorization-flow), +and services can use [rotating bearer-token providers](#rotating-bearer-tokens) +to keep authenticating as credentials expire. Microsoft Active Directory and Azure AD, for example, can be turned into an Identity Provider. @@ -330,6 +333,104 @@ Interactive clients usually fall into one of the following categories: - Non-browser based clients, usually some kind of command line interface (CLI) or a standalone application, such as Microsoft Access +### Device authorization flow + +The [C and C++](/docs/connect/clients/c-and-cpp/#oidc-device-flow), +[Rust](/docs/connect/clients/rust/#oidc-device-flow), and +[Python](/docs/connect/clients/python/#oidc-device-flow) clients implement the +OAuth 2.0 Device Authorization Grant (RFC 8628). Use it for a CLI, desktop +application, or notebook where a person can sign in through a browser, even +if the client runs on a remote machine. It requires native client 7.3.0 or +Python client 6.1.0 or later, QuestDB Enterprise configured for OIDC, and an +IdP application that allows the device grant. + +1. The application calls `sign_in()` explicitly. The client requests a device + code and displays the verification URL and user code. +2. The user opens the URL on any device and completes the IdP's sign-in and + MFA steps. Passwords stay in the browser; the application does not collect + them or need an inbound redirect listener. +3. The client polls the token endpoint until authorization succeeds, is + denied, or the device code expires. +4. Attach the authentication object to the QuestDB pool or sender. It + supplies cached or silently refreshed bearer credentials on later requests + and reconnects without rebuilding clients. + +Only `sign_in()` can prompt. Token reads, ingestion, query failover, and pool +connection setup never launch a device flow. When no cached or silently +refreshable credential is available, they report `InteractionRequired`; +Python exposes this as `OidcInteractionRequired`. Arrange a new explicit +sign-in on your UI thread rather than repeatedly retrying the data operation. + +#### Discovery, scopes, and token selection + +`from_questdb` reads the client ID, scope, endpoints, and groups mode from +QuestDB's public `/settings` endpoint. Pass the IdP's `issuer` explicitly to +pin the provider and discover its device-authorization endpoint from +`/.well-known/openid-configuration` when QuestDB does not advertise one. +Use HTTPS for QuestDB discovery and non-loopback IdP endpoints. + +Explicit client ID, endpoint, scope, audience, and groups-mode arguments +override discovered values. You can also skip QuestDB discovery and supply +the client ID, device-authorization endpoint, token endpoint, and scope +explicitly. The two credential endpoints must share an origin (scheme, host, +and port); explicit endpoints configured together with an issuer must also +match the issuer's origin. + +The device-flow helper chooses the token to send based on QuestDB's mode: + +- User Info mode uses the access token. +- `acl.oidc.groups.encoded.in.token=true` selects the ID token. With explicit + client configuration, set `groups_in_token=true` and include `openid` in + the scope so the IdP issues an ID token. + +Silent refresh requires an IdP-issued refresh token. Most providers require +`offline_access`, for example `scope="openid offline_access"`, and may require +additional consent or IdP configuration. A scope override replaces the whole +scope string; retain any groups or API scopes your deployment needs. The +client does not add `openid` or `offline_access` automatically to an explicit +scope. Without a refresh token, sign in again after the token expires. + +A retryable refresh failure uses bounded backoff. If a dispatched refresh +loses its response, the IdP may already have rotated the refresh token; the +client can discard that credential and require sign-in rather than risk +reusing a rotated refresh token. + +#### Credential storage and shutdown + +Credentials remain in memory unless you opt into `FileTokenStore` (the +`default_file_token_store` builder option in C/C++). Its default directory is +`~/.questdb/oidc-tokens/`; `QUESTDB_CLIENT_OIDC_TOKEN_STORE_DIR` overrides it +with an absolute path. The store coordinates refresh across processes and +uses atomic replacement for rotated credentials. + +:::warning Plaintext token storage + +The file store persists access, ID, and long-lived refresh tokens as plaintext +JSON. On Unix, directories use `0700` and files use `0600`; dedicate the +directory to this store. Mutations currently require Unix/POSIX storage. +Use memory-only auth on other platforms, or a custom Rust `TokenStore` backed +by a keychain or secrets manager. Custom Python token stores are not supported. +Never log tokens, authorization headers, or PGWire connection passwords. + +::: + +Cancellation, credential removal, and shutdown are different operations: + +- `cancel_sign_in()` cancels only the current attempt and leaves the shared + auth usable. Python Ctrl-C has the same effect. +- `clear()` removes cached and persisted local credentials. It does not revoke + them at the IdP or close an established database session. +- `close()` permanently disables the shared provider. Attached clients stop + acquiring tokens on their next lookup; recovery requires a new provider + and new clients. Close clients before closing their provider. +- Releasing a native handle or dropping a Rust reference only releases that + reference; other attached clients retain shared ownership. Python auth + context-manager exit calls `close()`. + +Providers and clients must not be inherited across `fork()`. Construct them +in fresh worker processes, or fork before constructing any providers or +clients. + ### Browser-based clients If the tool is browser based and can handle HTTP redirects, it can implement two @@ -352,6 +453,11 @@ with QuestDB as explained in the [above 7th step](#7-database-access). ### Jupyter notebook +For Python applications, prefer the client's +[device flow](/docs/connect/clients/python/#oidc-device-flow). It displays a +clickable notebook prompt and works when the kernel and browser run on +different machines, without collecting the user's password. + JupyterHub can integrate with OAuth2 providers using OAuthenticator, as described in its [documentation](https://jupyterhub.readthedocs.io/en/stable/explanation/oauth.html). @@ -515,8 +621,11 @@ with pg.connect(conn_str, autocommit=True) as connection: ### CLI, standalone applications -When using CLI tools, such as `psql`, or standalone applications like Microsoft -Access, the best option may be the Resource Owner Password Credentials flow. +Applications built with the C, C++, Rust, or Python clients can use the +[device flow](#device-authorization-flow). For other tools such as `psql` or +Microsoft Access, token-as-password authentication is available through +[PGWire](#oidc-for-the-pgwire-endpoint). The legacy Resource Owner Password +Credentials flow is another option when the tool cannot acquire a token. The user logs in with their SSO credentials, and the server validates the details with the OAuth2 provider: @@ -533,12 +642,79 @@ testldap=> ## Non-interactive clients -Non-interactive clients are usually jobs or standalone applications, such as a -client for ingesting data. It is practical to manage their credentials via an -OAuth2 provider too. +For services without a user present, use a managed identity, service +principal, OAuth2 client-credentials flow, or a QuestDB service-account token. +Interactive device sign-in is not a substitute for unattended credentials. + +### Rotating bearer tokens + +A fixed `token=` is captured once and cannot follow credential rotation. A +token provider supplies a current bearer token without recreating the pool, +sender, or reader. Client examples: + +- [C and C++ 7.3.0 or later](/docs/connect/clients/c-and-cpp/#rotating-bearer-tokens). +- [Rust 7.3.0 or later](/docs/connect/clients/rust/#rotating-bearer-tokens). +- [Python 6.1.0 or later](/docs/connect/clients/python/#rotating-bearer-tokens). +- [Java token providers](/docs/connect/clients/java/#token-providers). + +The C, C++, Rust, and Python refreshing providers wrap a token source that +your application supplies. Fetch a token and its expiry using your identity +SDK, OAuth2 flow, or secrets manager; the provider handles caching, +background refresh, concurrent acquisitions, expiry safeguards, and backoff. +No Azure or other identity SDK is bundled with these clients. If your +application already maintains a cache, attach an application provider instead. + +Refreshing providers start fetching immediately. They normally refresh around +half the token lifetime with jitter, honor earlier refresh hints, and retry +failures with bounded backoff. By default, a token is handed out only while +more than 60 seconds remain, and a caller without a usable token waits up to +30 seconds. Lower the handout floor for tokens whose full lifetime is at most +60 seconds. These proactive refresh rules apply to `RefreshingTokenProvider`, +not the device-flow helper, which silently refreshes on demand. + +Attach providers in code, not connection strings. In these four clients, +`token_provider`, `azure_resource`, `azure_client_id`, and `azure_credential` +are rejected connection-string keys. Choose one credential source: OIDC, +a bearer-token provider, a fixed `token`, or `username`/`password`. + +#### Token refresh and reconnects + +Providers work with ILP/HTTP senders and QWP/WebSocket senders, readers, and +pools, not TCP or UDP. Use `https::` or `wss::`; plaintext transports warn but +do not prevent bearer-token exposure. + +- ILP/HTTP normally resolves a token once per flush and reuses it for ordinary + HTTP retries. A transient acquisition failure or eligible HTTP 401 can + cause another lookup. +- QWP/WebSocket resolves a token for each connect/reconnect round, including + query failover, not for each batch on an established connection. Tokens are + not replaced in-band on an open WebSocket. +- An HTTP 401 with no Bearer error or `error="invalid_token"` notifies the + provider and allows one retry of the request or handshake if the token + changed. A `403` or a challenge such as `insufficient_scope` does not trigger + this refresh retry. +- A caller-awaited initial connection still fails on rejected credentials. + An established QWP sender keeps reconnecting with capped backoff and retains + unacknowledged data, including when fixed credentials are rejected. A + failure to obtain credentials is reported as `credential_unavailable`, + separately from the server rejecting them (`auth_failed`). + +Retryability governs foreground retry decisions; it does not by itself stop +background reconnects. Close the sender or provider if the application wants +to give up. Closing a provider stops later token pulls but does not delete +queued disk-backed store-and-forward data, which a later process can replay. +A query has its own failure and replay policy; see +[Authentication failures and token providers](/docs/connect/wire-protocols/qwp-client-behavior/#authentication-failures-and-token-providers). + +For Microsoft Entra ID server configuration and permissions, see +[Managed identities and service principals](#managed-identities-and-service-principals). + +### Legacy password-grant clients -As seen in the Jupyter notebook examples, the clients can request a token -themselves and then use it to authorise data ingestion: +Where the legacy Resource Owner Password Credentials flow is already enabled, +a client can request a token itself and then use it for ingestion. The +following examples capture fixed credentials; use a provider above for a +long-running application: ```python import json @@ -596,10 +772,9 @@ with Sender.from_conf(conf) as sender: A token that a client requests itself expires, and the client keeps sending it unchanged. A client that runs longer than its token lives must switch to a new token before QuestDB stops accepting the old one, as described in -[Sessions and token expiry](#sessions-and-token-expiry). The -[Java client](/docs/connect/clients/java/#token-providers) can request and -renew its tokens itself with a token provider. For Azure services, see -[Managed identities and service principals](#managed-identities-and-service-principals). +[Sessions and token expiry](#sessions-and-token-expiry). Use a +[rotating bearer-token provider](#rotating-bearer-tokens) instead of copying +one token into a long-lived client's configuration. ## OIDC for the PGWire endpoint @@ -610,6 +785,12 @@ This method works wherever a Postgres client library is available, including jup Token authentication for the PGWire endpoint should be enabled by adding the `acl.oidc.pg.token.as.password.enabled=true` setting to the server configuration. +The Python client's [PGWire adapters](/docs/connect/clients/python/#pgwire-adapters) +pass a current token to psycopg or to each new SQLAlchemy physical connection. +They accept both the device-flow helper and application token providers; +connection setup never prompts. With other drivers, resolve the token in your +pool's connection factory rather than capturing it when the pool is created. + The token should be sent in the password field, while the username field should contain the string `_sso`, or left empty if that is an option: ```python @@ -889,8 +1070,10 @@ expected audience, and logs an advisory at startup. QuestDB checks a token when a request or a new connection presents it. An established connection, such as a QWP or PGWire connection, stays open after its token expires. Every new connection and reconnect presents a token again, -so a client that runs longer than its tokens live must present new ones, as -the [Java client does with a token provider](/docs/connect/clients/java/#token-providers). +so a client that runs longer than its tokens live must present new ones. Use +[device-flow authentication](#device-authorization-flow) for interactive +applications or [rotating bearer-token providers](#rotating-bearer-tokens) +for services. The Web Console opens an HTTP session when the user logs in, identified by the `qdb_session` cookie, and also sends its token with every request. In QuestDB From 7bd20a9d6bbe1fc6751392a98974171cf6f2249c Mon Sep 17 00:00:00 2001 From: glasstiger Date: Sat, 14 Nov 2026 17:26:36 +0000 Subject: [PATCH 2/3] docs(clients): fix OIDC lifecycle and recovery guidance --- documentation/connect/clients/c-and-cpp.md | 61 ++++++++++---- .../connect/clients/connect-string.md | 79 ++++++++++++------- documentation/connect/clients/python.md | 32 ++++++-- documentation/connect/clients/rust.md | 45 ++++++++--- .../wire-protocols/qwp-client-behavior.md | 17 +++- .../wire-protocols/qwp-ingress-websocket.md | 38 +++++---- .../client-failover/concepts.md | 30 ++++--- documentation/security/oidc.mdx | 76 +++++++++++++++--- 8 files changed, 276 insertions(+), 102 deletions(-) diff --git a/documentation/connect/clients/c-and-cpp.md b/documentation/connect/clients/c-and-cpp.md index c64480658..670c998aa 100644 --- a/documentation/connect/clients/c-and-cpp.md +++ b/documentation/connect/clients/c-and-cpp.md @@ -364,7 +364,6 @@ int main() auto auth = questdb::oidc::builder::from_questdb( "https://db.example.com:9000") .issuer("https://idp.example.com/realms/questdb") - .scope("openid offline_access") .build(); auth.sign_in(); // Only this call may prompt or open a browser. @@ -395,7 +394,6 @@ int main(void) { const char* url = "https://db.example.com:9000"; const char* issuer = "https://idp.example.com/realms/questdb"; - const char* scope = "openid offline_access"; const char* conf = "wss::addr=db.example.com:9000;"; questdb_error* error = NULL; questdb_oidc_auth* auth = NULL; @@ -407,8 +405,6 @@ int main(void) if (!builder) goto cleanup; if (!questdb_oidc_builder_issuer( builder, issuer, strlen(issuer), &error)) goto cleanup; - if (!questdb_oidc_builder_scope( - builder, scope, strlen(scope), &error)) goto cleanup; auth = questdb_oidc_builder_build(builder, &error); if (!auth) goto cleanup; if (!questdb_oidc_auth_sign_in(auth, &error)) goto cleanup; @@ -447,10 +443,23 @@ opening with C++ `builder.open_browser(false)` or C only the event's vetted `browser_target`, not its display-only verification URLs. -`offline_access` requests a refresh token; the IdP must actually issue one -for silent renewal to work. An expired token without a usable refresh token -reports `InteractionRequired`. Call `sign_in()` again on your UI thread, -never from a transport callback. Token lookups and reconnects never prompt. +The examples preserve the server's configured scopes. To request a refresh +token, include `offline_access` in the server's `acl.oidc.scope`, or supply a +complete scope override that also retains required `profile`, groups, and +API scopes. The IdP must actually issue a refresh token for silent renewal to +work. An expired token without a usable refresh token reports +`InteractionRequired`. Call `sign_in()` again on your UI thread, never from +a transport callback. Token lookups and reconnects never prompt. + +For private CAs, use C++ `builder.ca_bundle("/etc/questdb/oidc-ca.pem")` or C +`questdb_oidc_builder_ca_bundle` before building the auth. This PEM bundle +covers discovery and IdP requests; the pool's `tls_roots` does not configure +it. The separate OIDC request timeout uses milliseconds: C++ +`builder.timeout_ms(10000)` or C `questdb_oidc_builder_timeout_ms` (default +`30000`, maximum `120000`). A token lookup waiting behind another thread's +silent refresh can take up to six times that timeout, independently of +sender timeouts. See +[TLS trust and timeouts](/docs/security/oidc/#tls-trust-and-timeouts). Credentials stay in memory by default. To opt into persistence, use `builder.default_file_token_store()` in C++, or @@ -615,10 +624,18 @@ A connection event named `credential_unavailable` means acquisition failed; `auth_failed` means the server rejected the credential. Clients retain their own references. Freeing a C auth/provider handle or -destroying a C++ wrapper releases only that reference. Explicitly closing the -shared provider permanently disables token acquisition for all attached -clients, so close clients first. Cancelling sign-in leaves the auth usable; -clearing credentials does not revoke them at the IdP. +destroying a C++ wrapper releases only that reference. Explicitly closing an +OIDC auth object or a refreshing token provider permanently disables token +acquisition for all attached clients, so close clients first. Cancelling +sign-in leaves the auth usable; clearing credentials does not revoke them +at the IdP. + +For application-defined providers created with `questdb_token_provider_new` +or `questdb::token_provider::from`, `questdb_token_provider_close` and C++ +`token_provider::close()` are no-ops. Implement a shared shutdown state in +your provider if token acquisition must be disabled, or stop all attached +clients. Do not free callback resources while clients can still invoke them; +use the release callback or shared ownership. ## Headers @@ -2018,9 +2035,13 @@ between plain return and drop. ## Production shape: TLS, token, multi-host, retry -A production-shaped configuration: TLS with OS roots, bearer token, two -endpoints, a store-and-forward spool, durable acks, and a retry-bounded -borrow: +A production-shaped, ingestion-only configuration: TLS with OS roots, bearer +token, two endpoints, a store-and-forward spool, durable acks, and an explicit +two-minute initial-connect retry budget. Pool construction eagerly opens the +sender with `initial_connect_retry=on`; merely setting `reconnect_*` does not +enable startup retry for native pools. `query_pool_min=0` avoids prewarming +readers, whose initial connections do not use the ingestion retry policy. +Later borrows also retry if they need to open a replacement connection: @@ -2039,12 +2060,15 @@ int main() "wss::addr=db-primary.example.com:9000,db-replica.example.com:9000;" "token=YOUR_BEARER_TOKEN;" "tls_ca=os_roots;" // trust the OS store (corporate CAs) + "initial_connect_retry=on;" + "reconnect_max_duration_millis=120000;" + "query_pool_min=0;" // ingestion only; no reader prewarm "sf_dir=/var/spool/questdb;" "sender_id=ingest-01;" // Enterprise; the connect fails against a server without replication. "request_durable_ack=on;"}; - // First borrow opens the connection: give it the failover budget. + // Prewarm already connected; retry if a replacement is needed. auto sender = pool.borrow_sender_with_retry(pool.reconnect_max_duration_ms()); @@ -2086,6 +2110,9 @@ int main(void) "wss::addr=db-primary.example.com:9000,db-replica.example.com:9000;" "token=YOUR_BEARER_TOKEN;" "tls_ca=os_roots;" /* trust the OS store (corporate CAs) */ + "initial_connect_retry=on;" + "reconnect_max_duration_millis=120000;" + "query_pool_min=0;" /* ingestion only; no reader prewarm */ "sf_dir=/var/spool/questdb;" "sender_id=ingest-01;" /* Enterprise; the connect fails against a server without replication. */ @@ -2093,7 +2120,7 @@ int main(void) db = questdb_db_connect(conf, strlen(conf), &err); if (!db) goto on_error; - /* First borrow opens the connection: give it the failover budget. */ + /* Prewarm already connected; retry if a replacement is needed. */ sender = questdb_db_borrow_sender_with_retry( db, questdb_db_reconnect_max_duration_ms(db), &err); if (!sender) goto on_error; diff --git a/documentation/connect/clients/connect-string.md b/documentation/connect/clients/connect-string.md index 3a01265ce..74da65025 100644 --- a/documentation/connect/clients/connect-string.md +++ b/documentation/connect/clients/connect-string.md @@ -149,12 +149,20 @@ wss::addr=node-a:443,node-b:443;target=replica;zone=eu-west-1a; ### Tolerate a slow or restarting server at startup ``` -ws::addr=node-a:9000;reconnect_max_duration_millis=120000; +ws::addr=node-a:9000;initial_connect_retry=on;reconnect_max_duration_millis=120000; ``` -The 2-minute reconnect budget covers both the *first* connect and any -subsequent reconnect: setting any explicit `reconnect_*` key implicitly -turns on `initial_connect_retry`. See +The 2-minute budget bounds the blocking initial **ingestion** connection. +Once connected, the sender retries transport outages indefinitely. Set +`initial_connect_retry` explicitly: `reconnect_*` keys alone do not enable +startup retry for native 7.3.0 and Python 6.1.0 or later pools. + +Pools also prewarm readers when `query_pool_min` is nonzero; their initial +connections do not use this ingestion retry policy. Set `query_pool_min=0` +to defer reader connections until first use. For background ingestion startup +instead of blocking retry, use `lazy_connect=true`, which also defaults +`query_pool_min` to `0` in native/Python pools; do not combine it with an +explicit blocking initial-connect mode. See [Ingress reconnect](#reconnect-keys). ## Recipes {#recipes} @@ -182,7 +190,7 @@ caveats), follow the section links from the [Key index](#key-index). | Orphan recovery for crashed senders | ingress | `drain_orphans=on` | `max_background_drainers` | | Confirm rows are on the server's disk | ingress | `request_durable_ack=local` | `durable_ack_keepalive_interval_millis`; the server needs adaptive durability mode | | Confirm rows reached replication storage | ingress | `request_durable_ack=replicated` | `on` for QuestDB Enterprise 4.1.0 and earlier; `durable_ack_keepalive_interval_millis` | -| Tune ingress reconnect backoff | ingress | — | `reconnect_initial_backoff_millis`, `reconnect_max_backoff_millis`, `reconnect_max_duration_millis` (any of these also implies `initial_connect_retry=on`) | +| Tune ingress reconnect backoff | ingress | — | `reconnect_initial_backoff_millis`, `reconnect_max_backoff_millis`, `reconnect_max_duration_millis`; set `initial_connect_retry` explicitly for portable startup behavior | | Force fail-fast on initial connect | ingress | `initial_connect_retry=off` | overrides the implicit promotion from any explicit `reconnect_*` key | | Retry initial connect in background | ingress | `initial_connect_retry=async` | `reconnect_*` | | Fast `close()` without drain | ingress | `close_flush_timeout_millis=0` | — | @@ -721,16 +729,13 @@ architecture is that a producer survives an arbitrarily long outage. - `reconnect_initial_backoff_millis` — initial wait between reconnect attempts. Backoff grows exponentially up to `reconnect_max_backoff_millis`. - Default: `100`. Setting this enables `initial_connect_retry=on` implicitly; - see below. + Default: `100`. - `reconnect_max_backoff_millis` — cap on per-attempt backoff. - Default: `5000` (5 s). Setting this enables - `initial_connect_retry=on` implicitly; see below. + Default: `5000` (5 s). - `reconnect_max_duration_millis` — time budget for the **blocking sync initial connect** (`initial_connect_retry=on`): once exceeded, the constructor gives up and returns the error. The running loop and the `async` initial connect never consult it. Default: `300000` (5 min). - Setting this enables `initial_connect_retry=on` implicitly; see below. - `initial_connect_retry` — whether the client retries the initial connect attempt on failure. - `off` (default, alias `false`) — fail fast on initial connect failure. @@ -740,13 +745,16 @@ architecture is that a producer survives an arbitrarily long outage. the background indefinitely, surfacing only genuine terminal failures (auth reject, durable-ack mismatch) via the error inbox. - **Implicit promotion.** Setting any explicit `reconnect_*` key without - also choosing an `initial_connect_retry` mode promotes - `initial_connect_retry` to `on` automatically, so the budget also covers - the *first* connect attempt — without the promotion, - `reconnect_max_duration_millis` would be inert. To keep fail-fast - behaviour on the first connect while still tuning the backoff, set - `initial_connect_retry=off` explicitly; the explicit setting wins. + **Implicit promotion is client-specific.** Some clients, including Java + and standalone native senders, promote the mode to `on` when an explicit + `reconnect_*` key is set without an `initial_connect_retry` mode. Pools in + C, C++, and Rust 7.3.0 or later, and Python 6.1.0 or later, do **not** apply + that promotion: they honor only an explicitly set mode and otherwise fail + fast. Set `initial_connect_retry=on` explicitly for blocking ingestion + startup retry, or `off` to retain fail-fast behavior while tuning backoff. + Neither `on` nor `async` changes reader prewarming: a nonzero + `query_pool_min` still creates readers synchronously and can fail pool + construction. - `close_flush_timeout_millis` — `close()` blocks up to this many milliseconds waiting for buffered rows to drain. Set to `0` or `-1` for fast close (skip the drain). **The default differs by client**: `60000` @@ -757,20 +765,31 @@ architecture is that a producer survives an arbitrarily long outage. entirely: in memory-only mode, un-ACKed batches are lost on every clean shutdown; in SF mode they stay in the sender directory for the next sender. -In most clients, rejections of the upgrade request during reconnect -(authentication rejected, version mismatch, durable-ack mismatch, non-101 -upgrade without a role hint) are immediately terminal: the loop does not -retry them. The Java client treats a version mismatch like a transport -failure, to ride out a rolling upgrade, and the other rejections as terminal -only before the sender first connects. A Java sender that has connected keeps -retrying all of them, and reports each failed attempt in its connection -health. It also reports the rejections other than a version mismatch to its -error handler. A Node.js sender that uses store-and-forward or -`initial_connect_retry=async` also keeps retrying all of them once it has -connected, and reports each failed attempt as a connection event. See -[How the client handles token failures](/docs/connect/clients/java/#how-the-client-handles-token-failures) -and, for orphan drainers, +Upgrade rejections have client- and phase-specific retry policies: + +- In C, C++, and Rust **7.3.0 or later**, and Python **6.1.0 or later**, a + caller-awaited initial connection fails on HTTP `401` / `403`. A sender + that has connected keeps retrying those rejections with capped backoff, + even with fixed credentials, retaining unacknowledged rows. Each rejection + ends the current endpoint walk, not the sender's reconnect loop, and is + reported as `auth_failed` / `AuthFailed`. This exception does **not** cover + protocol-version failures, which remain terminal in these clients. +- Java treats a version mismatch like a transport failure to ride out a + rolling upgrade. Other rejections (authentication, durable-ack mismatch, + non-101 upgrade without a role hint) are terminal only before the sender + first connects. Once connected, it retries all of them and reports each + failed attempt in its connection health; rejections other than a version + mismatch also reach its error handler. +- A Node.js sender using store-and-forward or `initial_connect_retry=async` + also retries these rejections once it has connected, reporting each failed + attempt as a connection event. + +Do not assume that an authentication exception makes all upgrade failures +retryable. For the full phase table, browser behavior, and orphan drainers, +see [Authentication failures and token providers](/docs/connect/wire-protocols/qwp-client-behavior/#authentication-failures-and-token-providers). +For Java, see also +[How the client handles token failures](/docs/connect/clients/java/#how-the-client-handles-token-failures). ### Egress failover {#egress-failover} diff --git a/documentation/connect/clients/python.md b/documentation/connect/clients/python.md index ce3e38ebf..434e47ee6 100644 --- a/documentation/connect/clients/python.md +++ b/documentation/connect/clients/python.md @@ -189,7 +189,6 @@ from questdb.auth import OidcDeviceAuth with OidcDeviceAuth.from_questdb( "https://db.example.com:9000", issuer="https://idp.example.com/realms/questdb", - scope="openid offline_access", ) as auth: auth.sign_in() with questdb.connect( @@ -201,13 +200,22 @@ with OidcDeviceAuth.from_questdb( Replace the QuestDB URL and issuer with your deployment's values. Discovery reads QuestDB's `/settings`; the issuer pins the IdP and enables discovery of -its device endpoint if QuestDB does not advertise one. To skip QuestDB -discovery, construct `OidcDeviceAuth` with keyword arguments `client_id`, +its device endpoint if QuestDB does not advertise one. Leaving `scope` unset +preserves the server's configured scopes. To skip QuestDB discovery, construct +`OidcDeviceAuth` with keyword arguments `client_id`, `device_authorization_endpoint`, `token_endpoint`, and `scope`. Match `groups_in_token` to the server: `True` selects the ID token, otherwise the access token. See [Device authorization flow](/docs/security/oidc/#device-authorization-flow). +For private CAs, pass `ca_bundle="/etc/questdb/oidc-ca.pem"` to the auth +constructor or `from_questdb`. This PEM bundle covers discovery and IdP +requests; the pool's `tls_roots` does not configure it. The separate OIDC +`timeout` argument is in seconds (default `30`, maximum `120`). A token lookup +waiting behind another thread's silent refresh can take up to six times that +value, independently of sender timeouts. See +[TLS trust and timeouts](/docs/security/oidc/#tls-trust-and-timeouts). + Only `sign_in()` can prompt or open a browser. It displays a terminal code and URL, or a clickable prompt in Jupyter. Browser opening defaults to off in Jupyter and on elsewhere; set `open_browser=False` to disable it explicitly. @@ -216,8 +224,10 @@ custom `renderer=` can own the UI. Use only its vetted `browser_target` for links or QR codes, and HTML-escape text when rendering HTML. Token lookups and reconnects use cached credentials or silent refresh, never -an interactive prompt. `offline_access` requests a refresh token; renewal -works only if the IdP issues one. Without a usable refresh token after expiry, +an interactive prompt. To request a refresh token, include `offline_access` +in the server's `acl.oidc.scope`, or supply a complete `scope=` override that +also retains required `profile`, groups, and API scopes. Renewal works only +if the IdP issues a refresh token. Without a usable refresh token after expiry, `OidcInteractionRequired` asks your application to call `sign_in()` again. For CI or headlessly executed notebooks, prefer machine credentials below; `interactive=False` makes sign-in fail fast when user interaction is needed. @@ -1176,8 +1186,16 @@ with capped backoff, including with fixed credentials. Unacknowledged rows stay queued, and the listener reports `AuthFailed`. A provider that cannot supply a token instead reports `CredentialUnavailable`. Even a `TokenUnavailableError` with `retryable=False` does not stop background -reconnects; close the provider or sender from application code, outside its -callbacks, to stop them. +reconnects. + +To stop a standalone `Sender`, close it. For a pool, release all leases and +close the `QuestDB` handle (`db.close()`). `PooledSender.close()` only returns +the lease: its underlying connection can keep delivering and reconnecting. +Explicitly closing a shared `OidcDeviceAuth` or `RefreshingTokenProvider` +disables future token acquisition for all attached clients, but an +application-defined provider must implement its own shutdown policy. Perform +shutdown from application code, outside callbacks, and normally close clients +before their provider. Protocol-version failures and terminal data rejections are not retried. A `flush(wait=True)` or `wait()` that times out can retain the underlying diff --git a/documentation/connect/clients/rust.md b/documentation/connect/clients/rust.md index d6232a0cc..2949fe48d 100644 --- a/documentation/connect/clients/rust.md +++ b/documentation/connect/clients/rust.md @@ -189,7 +189,6 @@ fn main() -> questdb::Result<()> { let auth = Arc::new( OidcDeviceAuth::from_questdb("https://db.example.com:9000") .issuer("https://idp.example.com/realms/questdb") - .scope("openid offline_access") .build()?, ); auth.sign_in()?; // Only this call may prompt or open a browser. @@ -210,14 +209,25 @@ fn main() -> questdb::Result<()> { } ``` -The same provider serves every pooled sender and reader. Token lookups never -prompt: they use the cache or refresh silently if the IdP issued a refresh -token, usually requested with `offline_access`. Without a usable refresh -token after expiry, they report `OidcErrorKind::InteractionRequired`; call -`sign_in()` again on a UI thread. See +The example preserves the server's configured scopes. To request a refresh +token, include `offline_access` in the server's `acl.oidc.scope`, or supply a +complete `.scope(...)` override that also retains required `profile`, groups, +and API scopes. The same provider serves every pooled sender and reader. +Token lookups never prompt: they use the cache or refresh silently if the IdP +issued a refresh token. Without a usable refresh token after expiry, they +report `OidcErrorKind::InteractionRequired`; call `sign_in()` again on a UI +thread. See [Device authorization flow](/docs/security/oidc/#device-authorization-flow) for scope overrides and access-token versus ID-token selection. +For private CAs, set `.ca_bundle("/etc/questdb/oidc-ca.pem")` on the OIDC +builder. This PEM bundle covers discovery and IdP requests, independently of +the pool's `tls_roots`. Configure the separate request timeout with +`.timeout(std::time::Duration::from_secs(10))` (default 30 seconds, maximum +120). A token lookup waiting behind another thread's silent refresh can take +up to six times that timeout; sender timeouts do not cap it. See +[TLS trust and timeouts](/docs/security/oidc/#tls-trust-and-timeouts). + Use `.open_browser(false)` on the builder for a remote terminal; the code and verification URL are still printed. A custom `oidc::Renderer` can own the UI. Credentials stay in memory unless the builder receives @@ -1027,8 +1037,15 @@ and is not part of that thread-bound borrow list. | `pool_reap` | `auto` | Use `manual` only when your application will call `reap_idle()`. | | `initial_connect_retry` | `off` | `on`/`sync` retries initial connection synchronously; `async` starts background retry. | -Setting an ingress `reconnect_*` key without explicitly setting -`initial_connect_retry` promotes the initial mode to synchronous retry. See the +In 7.3.0 and later, pools honor only an explicitly set `initial_connect_retry`: +setting `reconnect_*` alone does not change the default fail-fast startup. +Implicit promotion to synchronous retry applies to standalone senders, not +pools. Use `initial_connect_retry=on` to retry eager ingestion connections +within `reconnect_max_duration_millis`. Readers prewarmed by `query_pool_min` +still connect synchronously without that retry policy; set `query_pool_min=0` +to defer them until first use. For background ingestion startup instead, use +`lazy_connect=true`, which also defaults `query_pool_min` to `0`; do not +combine it with an explicit blocking initial-connect mode. See the [connect string reference](/docs/connect/clients/connect-string/) for the full grammar and all limits. @@ -1104,8 +1121,16 @@ a QWP/WebSocket sender has connected, later HTTP 401/403 rejections are retried with capped backoff, including with fixed credentials; unacknowledged rows stay queued. A `CredentialUnavailable` event means the provider could not supply a token, while `AuthFailed` means the server rejected one. A permanent -foreground token-fetch failure does not by itself stop background reconnects; -close the provider or sender to stop them. See +foreground token-fetch failure does not by itself stop background reconnects. + +Dropping a standalone sender stops it. Dropping a `BorrowedSender` normally +returns its connection to the pool, where delivery and reconnects continue. +To retire that connection, call `sender.drop_on_return()` before dropping the +borrow; to shut down the pool, release all borrows and call `db.close()`. +Explicitly closing a shared OIDC or refreshing provider disables future token +acquisition for all attached clients; an application-defined provider needs +its own shutdown policy. Shut down from application code, outside callbacks, +and normally close clients before their provider. See [Authentication failures and token providers](/docs/connect/wire-protocols/qwp-client-behavior/#authentication-failures-and-token-providers). Protocol-version failures and terminal data rejections are not retried. A diff --git a/documentation/connect/wire-protocols/qwp-client-behavior.md b/documentation/connect/wire-protocols/qwp-client-behavior.md index 8702a6ce4..37c707731 100644 --- a/documentation/connect/wire-protocols/qwp-client-behavior.md +++ b/documentation/connect/wire-protocols/qwp-client-behavior.md @@ -143,6 +143,14 @@ knob silently turns startup into a multi-minute **blocking** retry. If no `reconnect_*` knob is set either, the mode is `OFF`. Always set `initial_connect_retry` explicitly to avoid this ([sharp edge #1](#known-sharp-edges)). +**Native/Python pool exception:** C, C++, and Rust 7.3.0 or later, and Python +6.1.0 or later, apply this implicit promotion only to standalone senders, +not pools. Their eager pools honor an explicitly set `initial_connect_retry` +and otherwise fail fast even when `reconnect_*` keys are set. Reader prewarm +is always synchronous and is not covered by the ingestion retry mode; use +`query_pool_min=0` to defer it. `lazy_connect=true` selects background +ingestion and defaults `query_pool_min` to `0` in these pools. + ### Facade prewarm `QuestDBBuilder.build()` validates both configs (without connecting), then @@ -457,8 +465,13 @@ live producer. Java reports each failed round to the error handler as a retriable `SECURITY_ERROR`, and in its connection health. C, C++, Rust, and Python distinguish `credential_unavailable` / `CredentialUnavailable` connection events from server rejection events (`auth_failed` / `AuthFailed`). -An application that prefers a hard failure can close the sender from its own -code, outside the sender's callbacks. +An application that prefers a hard failure can close a standalone sender +from its own code, outside callbacks. With native or Python pools, returning +a sender lease (including Python's `PooledSender.close()`) can leave its +background delivery running. Release all leases and close the pool to stop +it, or explicitly retire a borrowed native connection instead of recycling +it. Closing a shared OIDC or refreshing provider also disables future token +pulls; an application-defined provider needs its own shutdown policy. The table describes Java 1.3.11 and later. Native clients 7.3.0 and Python 6.1.0 follow the same sender-phase distinction: a foreground failure does not diff --git a/documentation/connect/wire-protocols/qwp-ingress-websocket.md b/documentation/connect/wire-protocols/qwp-ingress-websocket.md index 000ab481d..55bb4a95f 100644 --- a/documentation/connect/wire-protocols/qwp-ingress-websocket.md +++ b/documentation/connect/wire-protocols/qwp-ingress-websocket.md @@ -1220,26 +1220,32 @@ Key behaviors: only. The `zone=` connect-string key is accepted but silently ignored, so a connect string shared with egress clients works unchanged on ingress. - **Authentication errors end the endpoint walk** at any host (`401`/`403`): - the client does not try the other hosts. Startup fails. In most clients, - the reconnect loop does not continue past them either. A sender that has - connected keeps retrying instead in the Java client, in the Node.js client - when it uses store-and-forward or was started with `async`, and in the - browser client when it authenticates with `sessionBootstrap` and was started - with `async`. A browser cannot see the status of a failed upgrade, so a - credential sent with the browser client's `auth` option does not end the - walk when QuestDB rejects it: it looks like an unreachable host, and a - sender that has connected keeps retrying it whatever its mode. A Java client - with a token provider first retries the same host once after a `401`, with - a new token. + after any eligible one-time token replacement, the client does not try the + other hosts. Caller-awaited startup fails, but ending the walk does not + necessarily stop the reconnect loop. A sender that has connected keeps + retrying with backoff in Java, C/C++ and Rust 7.3.0 or later, and Python + 6.1.0 or later, including with fixed credentials. The native/Python clients + retain unacknowledged rows and report `auth_failed` / `AuthFailed`. The + Node.js client also retries when it uses store-and-forward or was started + with `async`, as does a browser client using `sessionBootstrap` and `async`. + A browser cannot see the status of a failed upgrade, so a credential sent + with its `auth` option does not end the walk when QuestDB rejects it: it + looks like an unreachable host, and a sender that has connected keeps + retrying it whatever its mode. Java and native/Python clients with token + providers can first retry the same host once after an eligible `401` if + the provider supplies a different token; `403` does not trigger that retry. See [Authentication failures and token providers](/docs/connect/wire-protocols/qwp-client-behavior/#authentication-failures-and-token-providers). - **`421 + X-QuestDB-Role`** is a role reject: transient if the role is `PRIMARY_CATCHUP`, topology-level otherwise. -- **All other upgrade errors are transient** and feed into the reconnect loop, - including `404`, `426`, `503`, generic 4xx/5xx, TCP/TLS failures, - mid-stream send/recv errors, and an upgrade response that advertises a QWP - version outside the client's supported range (per-endpoint, so a host on a - rolling upgrade does not lock the client out of compatible peers). +- **In the Java reference client, other upgrade errors are transient** and + feed into the reconnect loop, including `404`, `426`, `503`, generic + 4xx/5xx, TCP/TLS failures, mid-stream send/recv errors, and an upgrade + response that advertises a QWP version outside the client's supported + range (per-endpoint, so a host on a rolling upgrade does not lock the client + out of compatible peers). The native/Python authentication-retry exception + above does not extend to protocol-version failures: those remain terminal + in these clients. :::note Enterprise diff --git a/documentation/high-availability/client-failover/concepts.md b/documentation/high-availability/client-failover/concepts.md index 3d76d894e..4f7779e45 100644 --- a/documentation/high-availability/client-failover/concepts.md +++ b/documentation/high-availability/client-failover/concepts.md @@ -236,20 +236,32 @@ when at least one peer is healthy." ## Authentication is cluster-wide -A `401` or `403` on the HTTP upgrade is terminal — the client does not retry -other hosts. The assumption is that auth credentials are configured -identically across the cluster, so a credential failure against one node is -a credential failure against all of them. Retrying would spam every peer's -audit log without recovering. - -The Java client relaxes this in two cases. With a token provider, a `401` -first earns one retry of the same host with a new token. A Java sender that -has connected before also treats a rejected credential like an outage: it +A `401` or `403` on the HTTP upgrade ends the current endpoint walk, after +any eligible one-time token replacement: the client does not retry other +hosts. This does not necessarily terminate a sender's reconnect loop. The +assumption is that auth credentials are configured identically across the +cluster, so a credential failure against one node is a credential failure +against all of them. Trying every peer would spam their audit logs without +recovering. + +In Java, a token provider can supply a new token for one same-host retry +after an eligible `401`. A Java sender that has connected before also +treats a rejected credential like an outage: it keeps retrying with backoff, buffering rows, and reports each failed round to its error handler, so ingestion resumes once QuestDB accepts the credential again, such as after a token renewal. See [Authentication failures and token providers](/docs/connect/wire-protocols/qwp-client-behavior/#authentication-failures-and-token-providers). +C, C++, and Rust **7.3.0 or later**, and Python **6.1.0 or later**, also retry +HTTP `401` / `403` with capped backoff once a sender has connected, including +with fixed credentials. They retain unacknowledged rows and report +`auth_failed` / `AuthFailed` connection events. A caller-awaited initial +connection still fails on rejected credentials. With a token provider, an +eligible `401` can first trigger one same-host retry if the token changes; +a `403` cannot. These authentication exceptions do not extend to +protocol-version failures, which remain terminal in the native/Python +clients. + The Node.js client also treats a rejected credential like an outage, for a sender that has connected before and uses store-and-forward or was started with `async`: it keeps retrying with backoff, buffering rows, and reports each diff --git a/documentation/security/oidc.mdx b/documentation/security/oidc.mdx index 56a145b6e..e3e8ca18b 100644 --- a/documentation/security/oidc.mdx +++ b/documentation/security/oidc.mdx @@ -384,17 +384,55 @@ The device-flow helper chooses the token to send based on QuestDB's mode: the scope so the IdP issues an ID token. Silent refresh requires an IdP-issued refresh token. Most providers require -`offline_access`, for example `scope="openid offline_access"`, and may require -additional consent or IdP configuration. A scope override replaces the whole -scope string; retain any groups or API scopes your deployment needs. The -client does not add `openid` or `offline_access` automatically to an explicit -scope. Without a refresh token, sign in again after the token expires. +`offline_access` and may require additional consent or IdP configuration. +The client examples leave `scope` unset to preserve the scopes discovered +from `acl.oidc.scope`. Add `offline_access` to that server setting, or pass a +complete scope override that retains required `openid`, `profile`, groups, +and API scopes. For example, a deployment using `openid profile` needs +`scope="openid profile offline_access"`, not just `openid offline_access`. +An override replaces the whole scope string; the client does not add scopes +automatically. Without a refresh token, sign in again after the token expires. A retryable refresh failure uses bounded backoff. If a dispatched refresh loses its response, the IdP may already have rotated the refresh token; the client can discard that credential and require sign-in rather than risk reusing a rotated refresh token. +#### TLS trust and timeouts + +The native and Python OIDC helpers use their own HTTPS client for QuestDB +`/settings` discovery and all IdP requests. Configure its trust separately +from the pool or sender: a pool's `tls_roots` does not configure the auth +helper, and the helper's `ca_bundle` does not configure database connections. + +For private CAs, supply a PEM bundle before building the authentication +object. By default, the helper trusts the bundled Mozilla roots together with +the OS trust store. A supplied bundle replaces both, so include the CAs needed +for **both** QuestDB discovery and the IdP. Use an expanded absolute path; +the native builders do not expand `~`. + +| Language | OIDC CA bundle | Per-request timeout example | +| --- | --- | --- | +| Python | `ca_bundle="/etc/questdb/oidc-ca.pem"` on the constructor or `from_questdb` | `timeout=10` (seconds) | +| Rust | `.ca_bundle("/etc/questdb/oidc-ca.pem")` on the builder | `.timeout(std::time::Duration::from_secs(10))` | +| C++ | `builder.ca_bundle("/etc/questdb/oidc-ca.pem")` | `builder.timeout_ms(10000)` | +| C | `questdb_oidc_builder_ca_bundle` | `questdb_oidc_builder_timeout_ms` (milliseconds) | + +The OIDC request timeout defaults to **30 seconds**, must be positive, and +has a **120-second maximum**. It bounds each discovery or IdP HTTP request, +not the whole interactive sign-in, which can last until the device code +expires. + +A token lookup uses a valid cached token immediately. Otherwise it can make +a silent refresh request or wait behind another thread already refreshing +the shared provider. That wait is bounded by **six times the OIDC timeout**: +three minutes by default, twelve minutes at the maximum. The sender's +`request_timeout`, `retry_timeout`, `connect_timeout`, and reconnect budget +**do not cap this acquisition wait**, which happens before the database +request. Choose the OIDC timeout to fit the stalls your application can +accept, and call `sign_in()` before connecting to establish the initial +credentials. Transport operations never prompt for sign-in. + #### Credential storage and shutdown Credentials remain in memory unless you opt into `FileTokenStore` (the @@ -700,9 +738,16 @@ do not prevent bearer-token exposure. separately from the server rejecting them (`auth_failed`). Retryability governs foreground retry decisions; it does not by itself stop -background reconnects. Close the sender or provider if the application wants -to give up. Closing a provider stops later token pulls but does not delete -queued disk-backed store-and-forward data, which a later process can replay. +background reconnects. To give up, close a standalone sender, or release all +leases and close the pool. Returning a native or Python sender lease +(including `PooledSender.close()`) does not stop its underlying connection's +background delivery. Explicitly closing a shared OIDC or refreshing provider +stops later token pulls for all attached clients; application-defined +providers must implement their own shutdown policy. In particular, native +`token_provider::close()` / `questdb_token_provider_close` is a no-op for +application-defined providers. Shut down outside callbacks, normally closing +clients before their provider. Provider shutdown does not delete queued +disk-backed store-and-forward data, which a later process can replay. A query has its own failure and replay policy; see [Authentication failures and token providers](/docs/connect/wire-protocols/qwp-client-behavior/#authentication-failures-and-token-providers). @@ -1963,9 +2008,18 @@ try (QuestDB db = QuestDB.connect( } ``` -Other clients take a fixed token: request it with the Azure Identity library -for the scope `api:///.default`, and create a new client with -a new token before the old one expires. +[Python 6.1.0 or later](/docs/connect/clients/python/#rotating-bearer-tokens) +and the [C/C++](/docs/connect/clients/c-and-cpp/#rotating-bearer-tokens) and +[Rust](/docs/connect/clients/rust/#rotating-bearer-tokens) clients from 7.3.0 +can also follow token rotation without recreating clients. Supply an +application token source that requests the scope +`api:///.default` using Azure Identity or your identity SDK, +and attach a shared refreshing provider. These clients do not bundle an +Azure SDK or accept Java's `token_provider=azure` connection-string keys. + +For clients or versions without provider support, request a fixed token with +the Azure Identity library and create a new client with a new token before +the old one expires. QuestDB reads the roles from the token, so a role that you assign to or remove from a service takes effect when the service presents a new token. Azure From d65ee40ebc30ceefba99c98f30bc651ac27f07ec Mon Sep 17 00:00:00 2001 From: glasstiger Date: Sun, 15 Nov 2026 13:52:42 +0000 Subject: [PATCH 3/3] docs(clients): clarify OIDC trust and recovery guidance --- documentation/changelog.mdx | 2 + .../connect/clients/connect-string.md | 4 +- documentation/connect/clients/python.md | 58 +++++++++++++++---- .../client-failover/concepts.md | 28 ++++++--- documentation/security/oidc.mdx | 13 +++-- 5 files changed, 81 insertions(+), 24 deletions(-) diff --git a/documentation/changelog.mdx b/documentation/changelog.mdx index fc76299d9..a4b36621c 100644 --- a/documentation/changelog.mdx +++ b/documentation/changelog.mdx @@ -16,6 +16,7 @@ This page tracks significant updates to the QuestDB documentation. ### New +- [OIDC device flow](/docs/security/oidc/#device-authorization-flow) and [rotating bearer tokens](/docs/security/oidc/#rotating-bearer-tokens) - Sign in interactively or use expiring application credentials with the C, C++, and Rust 7.3.0+ clients and Python 6.1.0+. Covers discovery, TLS trust, token storage, refresh, recovery, and the [Python PGWire adapters](/docs/connect/clients/python/#pgwire-adapters) - [Passthrough materialized views](/docs/concepts/materialized-views/#passthrough-views) - A materialized view no longer has to aggregate: a query without `SAMPLE BY` copies base rows one-for-one, so the view is a live copy of some columns or rows of one table. Covers what it inherits from the base table (designated timestamp, partitioning, symbol indexes), why its columns are fixed at creation, [which queries qualify](/docs/concepts/materialized-views/#which-queries-are-passthrough), and that `REFRESH PERIOD` is rejected - [dbt connector](/docs/connect/connectors/dbt/) - New page for the QuestDB Enterprise dbt adapter, which builds, tests, and documents SQL models over QWP. Covers installation from the release archive, a TLS profile for the bundled trades project, SQL transformations from staging to joined and aggregated models, views, tables, incremental upserts, materialized views with timer and period refresh, snapshots, reading ingested tables such as the Databento connector's as dbt sources, connection and model settings, and limitations - [Expiring rows (`EXPIRE ROWS`)](/docs/concepts/expire-rows/) and [ALTER MATERIALIZED VIEW SET EXPIRE](/docs/query/sql/alter-mat-view-set-expire/) - Keep only some rows in a materialized view. Set it when you create the view (see [CREATE MATERIALIZED VIEW](/docs/query/sql/create-mat-view/)) or add it later. Queries hide expired rows right away, and a background job frees their disk space for the policies where that is safe. @@ -38,6 +39,7 @@ This page tracks significant updates to the QuestDB documentation. ### Updated +- [Native and Python client startup](/docs/connect/clients/connect-string/#reconnect-keys) and [authentication failures](/docs/high-availability/client-failover/concepts/#authentication-is-cluster-wide) - Pools require an explicit initial-connect retry mode rather than inferring it from reconnect settings. Documents eager reader prewarming, established senders retrying rejected credentials, and terminal protocol-version failures in native 7.3.0+/Python 6.1.0+ - [REINDEX](/docs/query/sql/reindex/) - Explains that `LOCK EXCLUSIVE` is required and takes the table's writer lock, and that `REINDEX` rebuilds native partitions only: it skips Parquet partitions and rejects a Parquet partition named in `PARTITION` - [CREATE MATERIALIZED VIEW](/docs/query/sql/create-mat-view/#limit-and-window-function-restrictions) - Breaking change in QuestDB 10.2.0 and QuestDB Enterprise 4.2.0: the part of a materialized view's query that reads its base table (this includes a subquery on the base table) can no longer use `LIMIT`, or a window function nested inside an expression, such as `row_number() OVER (ORDER BY timestamp) + 1`. A window function that makes up a whole column was already rejected. A `LIMIT` in a subquery over a *different* table is still allowed - [ALTER TABLE SET TYPE](/docs/query/sql/alter-table-set-type/) - A conversion to non-WAL applies the WAL first, and is refused while it cannot diff --git a/documentation/connect/clients/connect-string.md b/documentation/connect/clients/connect-string.md index 74da65025..c2751c774 100644 --- a/documentation/connect/clients/connect-string.md +++ b/documentation/connect/clients/connect-string.md @@ -191,7 +191,7 @@ caveats), follow the section links from the [Key index](#key-index). | Confirm rows are on the server's disk | ingress | `request_durable_ack=local` | `durable_ack_keepalive_interval_millis`; the server needs adaptive durability mode | | Confirm rows reached replication storage | ingress | `request_durable_ack=replicated` | `on` for QuestDB Enterprise 4.1.0 and earlier; `durable_ack_keepalive_interval_millis` | | Tune ingress reconnect backoff | ingress | — | `reconnect_initial_backoff_millis`, `reconnect_max_backoff_millis`, `reconnect_max_duration_millis`; set `initial_connect_retry` explicitly for portable startup behavior | -| Force fail-fast on initial connect | ingress | `initial_connect_retry=off` | overrides the implicit promotion from any explicit `reconnect_*` key | +| Force fail-fast on initial connect | ingress | `initial_connect_retry=off` | prevents implicit promotion in clients that support it; native 7.3.0+/Python 6.1.0+ pools already default to fail-fast | | Retry initial connect in background | ingress | `initial_connect_retry=async` | `reconnect_*` | | Fast `close()` without drain | ingress | `close_flush_timeout_millis=0` | — | | Disable per-query egress failover | egress | `failover=off` | — | @@ -1059,7 +1059,7 @@ description and behaviour notes. | `failover_max_attempts` | int | `8` | [Egress failover](#reconnect-keys) | | `failover_max_duration_ms` | int (ms) | `30000` | [Egress failover](#reconnect-keys) | | `init_buf_size` | size | `65536` (64 KiB) | [Buffer sizing](#buffer) | -| `initial_connect_retry` | enum (`off` / `on` / `async`) | `off` (auto-promoted to `on` when any explicit `reconnect_*` key is set) | [Ingress reconnect](#reconnect-keys) | +| `initial_connect_retry` | enum (`off` / `on` / `async`) | `off`; implicit promotion is client-specific, not applied by native 7.3.0+/Python 6.1.0+ pools | [Ingress reconnect](#reconnect-keys) | | `initial_credit` | int (bytes) | `0` (unbounded) | [Query client keys](#egress-keys) | | `housekeeper_interval_ms` | int (ms) | `5000` | [Connection pool](#pool-keys) | | `idle_timeout_ms` | int (ms) | `60000` (`0` ⇒ infinite) | [Connection pool](#pool-keys) | diff --git a/documentation/connect/clients/python.md b/documentation/connect/clients/python.md index 434e47ee6..13dbc2b0c 100644 --- a/documentation/connect/clients/python.md +++ b/documentation/connect/clients/python.md @@ -189,6 +189,7 @@ from questdb.auth import OidcDeviceAuth with OidcDeviceAuth.from_questdb( "https://db.example.com:9000", issuer="https://idp.example.com/realms/questdb", + open_browser=False, ) as auth: auth.sign_in() with questdb.connect( @@ -218,10 +219,11 @@ value, independently of sender timeouts. See Only `sign_in()` can prompt or open a browser. It displays a terminal code and URL, or a clickable prompt in Jupyter. Browser opening defaults to off in -Jupyter and on elsewhere; set `open_browser=False` to disable it explicitly. -Optional `qr=True` needs `qrcode` (`qrcode[pil]` for notebook images), and a -custom `renderer=` can own the UI. Use only its vetted `browser_target` for -links or QR codes, and HTML-escape text when rendering HTML. +Jupyter and on elsewhere; pass `open_browser=False` to the constructor or +`from_questdb`, as above, to disable it explicitly. `sign_in()` takes no +arguments. Optional `qr=True` needs `qrcode` (`qrcode[pil]` for notebook +images), and a custom `renderer=` can own the UI. Use only its vetted +`browser_target` for links or QR codes, and HTML-escape text when rendering HTML. Token lookups and reconnects use cached credentials or silent refresh, never an interactive prompt. To request a refresh token, include `offline_access` @@ -321,12 +323,48 @@ on an established WebSocket. An eligible HTTP 401 can trigger one retry with a changed token; `403` and `insufficient_scope` do not trigger that retry. See [Token refresh and reconnects](/docs/security/oidc/#token-refresh-and-reconnects). -`OidcError` and `TokenUnavailableError` are `QuestDBError` subclasses. Catch -`OidcInteractionRequired` before a generic retry handler: if -`acquisition_busy` is true, defer until the other acquisition finishes; -otherwise arrange explicit `sign_in()` outside callbacks. A background -reconnect can report acquisition failure through a `CredentialUnavailable` -connection event rather than a raised exception. +`OidcError` and `TokenUnavailableError` are `QuestDBError` subclasses. Import +`OidcInteractionRequired` from `questdb.auth` and catch it before a generic +retry handler. If `exc.acquisition_busy` is true, defer until the other +acquisition finishes; otherwise arrange explicit `sign_in()` outside callbacks. + +For a query made after sign-in, call this helper on the application's main or +UI thread, while the `auth` object from the device-flow example is still open. +It covers errors from eager connection creation and from consuming the query +result, and retries this read-only query at most once after sign-in: + +```python title="Recover a query that needs a new sign-in" +import questdb +from questdb.auth import OidcInteractionRequired + + +def query_current_user(auth): + for attempt in range(2): + try: + with questdb.connect( + "wss::addr=db.example.com:9000;", oidc_auth=auth + ) as db: + with db.query("SELECT current_user()") as result: + return list(result.iter_rows()) + except OidcInteractionRequired as exc: + if exc.acquisition_busy or attempt == 1: + raise + auth.sign_in() +``` + +Call `query_current_user(auth)` instead of the unguarded query in the +sign-in example. A busy acquisition is re-raised so the application can +reschedule the query after the existing acquisition finishes, keeping the +shared provider open. Do not start another sign-in or spin in a retry loop +in that case. Other errors, including sign-in failures and failures of the +second query attempt, propagate to the caller. + +A background reconnect can instead report acquisition failure through a +`CredentialUnavailable` [connection event](#failover-and-errors). Have the +listener notify application code rather than call `sign_in()` itself; the +event alone does not establish that interactive sign-in is needed. See the +[OIDC error-handling reference](https://github.com/questdb/py-questdb-client/blob/v6.1.0/docs/auth.rst#error-handling) +for other authentication errors and HTTP ingestion recovery. `cancel_sign_in()` or Ctrl-C cancels only the current sign-in attempt. `clear()` removes local cached and persisted credentials without revoking diff --git a/documentation/high-availability/client-failover/concepts.md b/documentation/high-availability/client-failover/concepts.md index 4f7779e45..35ed8e853 100644 --- a/documentation/high-availability/client-failover/concepts.md +++ b/documentation/high-availability/client-failover/concepts.md @@ -177,8 +177,9 @@ hosts when `zone=` is set. ## Error classification -Every error the client encounters falls into one of three buckets, which drives -the loop's response: +Errors fall into three buckets, which drive the loop's response. The +classification depends on the client and whether it is making an initial +connection, reconnecting an established sender, or running a query. ### Terminal — bypass failover @@ -187,9 +188,16 @@ will not help. | Condition | Why terminal | |---|---| -| HTTP `401` / `403` on upgrade | Credentials are cluster-wide; retrying floods server logs without recovery. | +| HTTP `401` / `403` on a caller-awaited initial connection or query | After any eligible one-time token replacement, rejected credentials end the endpoint walk because credentials are cluster-wide. | +| Protocol-version failure in C, C++, and Rust 7.3.0+, or Python 6.1.0+ | Terminal during initial connection and established sender reconnects; authentication retries do not make version failures retryable. | | Server-status reject (SF) | Application-layer reject; replay reproduces the same response. | +An established C, C++, or Rust 7.3.0+ sender, or Python 6.1.0+ sender, instead +retries HTTP `401` / `403` with capped backoff. The rejection ends the current +endpoint walk, not the reconnect loop. See +[Authentication is cluster-wide](#authentication-is-cluster-wide) for the +phase rules and other clients' behavior. + ### Topology — handled inside the round The host is demoted in the priority lattice; the client walks to the next host @@ -205,11 +213,15 @@ fails the current `execute()` call. ### Transient — enter backoff -Everything else: TCP/TLS errors, `auth_timeout_ms` expiry, mid-stream send or -receive failures, `404` / `426` / `503` on upgrade, version mismatches -(per-endpoint — a rolling upgrade in flight does not lock out compatible -peers), and generic errors decoding server messages. The client records -`TransportError` and walks to the next host. +TCP/TLS errors, `auth_timeout_ms` expiry, mid-stream send or receive failures, +`404` / `426` / `503` on upgrade, and retryable errors decoding server messages +are treated as transport failures. The client records `TransportError` and +walks to the next host. + +Java also treats protocol-version mismatches as transport failures so that +a rolling upgrade does not lock out compatible peers. This is not a shared +retry rule: protocol-version failures remain terminal in the native/Python +clients listed above. The Go query client doesn't resubmit after a server response it can't read that may have ended the statement; see diff --git a/documentation/security/oidc.mdx b/documentation/security/oidc.mdx index e3e8ca18b..4fe60eae8 100644 --- a/documentation/security/oidc.mdx +++ b/documentation/security/oidc.mdx @@ -405,11 +405,16 @@ The native and Python OIDC helpers use their own HTTPS client for QuestDB from the pool or sender: a pool's `tls_roots` does not configure the auth helper, and the helper's `ca_bundle` does not configure database connections. +The C/C++ and Python distributions trust the bundled Mozilla roots together +with the OS trust store by default. Rust follows its enabled Cargo features: +the default features, including when `oidc` is added, trust only the bundled +Mozilla roots. Enable `tls-native-certs` as well to include the OS store, for +example `features = ["oidc", "tls-native-certs"]` with default features enabled. + For private CAs, supply a PEM bundle before building the authentication -object. By default, the helper trusts the bundled Mozilla roots together with -the OS trust store. A supplied bundle replaces both, so include the CAs needed -for **both** QuestDB discovery and the IdP. Use an expanded absolute path; -the native builders do not expand `~`. +object. A supplied bundle replaces the default roots, so include the CAs +needed for **both** QuestDB discovery and the IdP. Use an expanded absolute +path; the native builders do not expand `~`. | Language | OIDC CA bundle | Per-request timeout example | | --- | --- | --- |