From ba307dd912c6120bfa1f6d6b5cd562a9078bbf45 Mon Sep 17 00:00:00 2001 From: wadii Date: Thu, 15 Oct 2026 18:32:49 +0200 Subject: [PATCH 1/3] docs: add ready-to-use coding agent prompts for local setup and experiments --- docs/docs/experimentation/index.md | 7 + .../set-up-with-a-coding-agent.mdx | 121 ++++++++++++++++++ .../run-locally-with-a-coding-agent.mdx | 98 ++++++++++++++ 3 files changed, 226 insertions(+) create mode 100644 docs/docs/experimentation/set-up-with-a-coding-agent.mdx create mode 100644 docs/docs/getting-started/run-locally-with-a-coding-agent.mdx diff --git a/docs/docs/experimentation/index.md b/docs/docs/experimentation/index.md index 8ee24c4afb35..18109747c2b0 100644 --- a/docs/docs/experimentation/index.md +++ b/docs/docs/experimentation/index.md @@ -19,6 +19,13 @@ Experiments are scoped to an [**environment**](/flagsmith-concepts/data-model#en ## Set up and run an experiment +:::tip + +Using Claude Code or another coding agent? [One prompt](/experimentation/set-up-with-a-coding-agent) creates the flag, +metric and experiment, and instruments your application. + +::: + 1. **[Connect a warehouse](/experimentation/connect-a-warehouse)**: enable the managed Flagsmith warehouse, or connect your own ClickHouse or Databricks. Self-hosted Flagsmith uses your own. Done once per environment. 2. **[Create metrics](/experimentation/create-metrics)**: define the outcomes you measure, computed from the events your diff --git a/docs/docs/experimentation/set-up-with-a-coding-agent.mdx b/docs/docs/experimentation/set-up-with-a-coding-agent.mdx new file mode 100644 index 000000000000..f09f00b8ab4c --- /dev/null +++ b/docs/docs/experimentation/set-up-with-a-coding-agent.mdx @@ -0,0 +1,121 @@ +--- +title: Set Up an Experiment with a Coding Agent +sidebar_label: Set Up with an Agent +sidebar_position: 1 +description: A copy-paste prompt that has a coding agent create an experiment and instrument your application to run it. +--- + +# Set Up an Experiment with a Coding Agent + +:::info Beta + +Experimentation is in beta. + +::: + +Run the prompt below from your application's repository in [Claude Code](https://claude.com/claude-code), or any coding +agent that can run shell commands and edit code. It takes you from an idea to a running A/B test: + +1. Asks what you want to test and what counts as success, and finds the matching places in your code. +2. Checks that your stack can send events. +3. Creates a [multivariate flag](/managing-flags/core-management) with a keyed variation, a + [metric](/experimentation/create-metrics) and an [experiment](/experimentation/create-an-experiment), and starts it. +4. Shows you the planned code changes and, once you confirm, adds the SDK calls that + [record exposures and conversion events](/experimentation/run-an-experiment). +5. Optionally sends test traffic and refreshes the results, so you can see the experiment working within minutes. + +No application to hand? The agent can build a small demo page or script instead. + +## Prerequisites + +- A Flagsmith instance with a [connected warehouse](/experimentation/connect-a-warehouse): either Flagsmith Cloud, or + a local instance set up with the [local setup prompt](/getting-started/run-locally-with-a-coding-agent). +- An application using the JavaScript SDK (`@flagsmith/flagsmith` 12.1 or later, including React) or the Python SDK + (`flagsmith` 6.3 or later). Other SDKs, including the Node.js server SDK, can't send events yet. +- [Node.js](https://nodejs.org/) and npm, to install the [Flagsmith CLI](/integrating-with-flagsmith/CLI). +- Claude Code or another coding agent. + +## The prompt + +```text title="Prompt: set up an experiment" +Help me run an A/B test in this codebase with Flagsmith Experimentation. Work step by step and verify each step. +If something fails, stop and show me the exact error; don't guess or work around it. + +1. Ask me these questions first and wait for my answers: + - What do you want to test? (For example, "the copy on the checkout button".) Find the component or file that + renders it. + - What counts as success? (For example, "a completed purchase".) Find where that happens in the code; the + conversion event goes there. + - Which Flagsmith are we using? Flagsmith Cloud, or a local instance (API http://localhost:8000, events + http://localhost:8080, CLI credentials in the `flagsmith.env` file written by the local setup)? + If there is no app to change, offer to build a tiny demo instead (a single HTML page or a script) that simulates + users, so results show up within minutes. + +2. Check the stack. Events need the JavaScript SDK `@flagsmith/flagsmith` 12.1 or later (browser or React) or the + Python SDK `flagsmith` 6.3 or later. The Node.js server SDK (`flagsmith-nodejs`) and other SDKs can't send events + yet: if that's all the app uses, tell me and stop. Find how the app identifies users: exposures and conversions + are joined per identity, so both must use the same identifier. + +3. Connect the Flagsmith CLI. Install it with `npm install -g @flagsmith/cli` (v2 or later). For Cloud, ask me to + run `flagsmith login`; for a local instance, load `flagsmith.env`. Ask me which project and environment to use + (`flagsmith project list`, `flagsmith environment list`) and bind this repository to them with + `flagsmith init --project --environment `. Make the API calls below with + `flagsmith api ` (`--input -` reads a JSON body from stdin). Don't use the Flagsmith MCP server: it can't + start experiments. The environment must have a warehouse: if + `GET api/v1/environments//warehouse-connections/` returns none, send me to + https://docs.flagsmith.com/experimentation/connect-a-warehouse and stop. + +4. Create a multivariate flag: `flagsmith feature create --value control --enabled`, then add each variation + with `flagsmith feature variant add --value --key --weight 0`. Every variation needs a key: + one without a key records no exposures. `control` is reserved for the flag's own value, and `--variants` on + `feature create` can't set keys. Check with `flagsmith feature get --json`. + +5. Create the metric: `POST api/v1/environments//experiment-metrics/` with + {"name": "", "aggregation": "occurrence", "direction": "up", "definition": {"version": 1, "event": ""}} + The event name is case-sensitive and must match what the code sends. + +6. Create the experiment: `POST api/v1/environments//experiments/` with + {"feature": , "name": "", "hypothesis": "", + "metrics": [{"metric": , "expected_direction": "increase"}], + "experiment_rollout": {"enabled": true, "rollout_percentage": 100, + "feature_state_value": {"type": "string", "value": "control"}, + "multivariate_feature_state_values": [{"multivariate_feature_option": , "percentage_allocation": 50}]}} + Each variation gets its `percentage_allocation` and control gets the rest. Start it with + `POST api/v1/environments//experiments//start/` and check its status is `running`. + +7. Plan the code changes and show them to me. Don't edit anything until I confirm. Then: + - Install or upgrade the SDK and initialise it with events on. JavaScript: `enableEvents: true`; for a local + instance also `api: 'http://localhost:8000/api/v1/'` and + `eventProcessorConfig: { eventsApiUrl: 'http://localhost:8080/' }`. Python: `enable_events=True` and a + server-side key (create one with `POST api/v1/environments//api-keys/` and `{"name"}`); for a + local instance also `api_url='http://localhost:8000/api/v1/'` and + `event_processor_config=EventProcessorConfig(events_api_url='http://localhost:8080/')`. + - Identify the user, then call `getExperimentFlag('')` (React: `useExperiment('')`; Python: + `get_experiment_flag('', identifier=...)`) where the user actually sees the variation, and render by its + value. + - Call `trackEvent('')` (Python: `track_event('', identifier=...)`) where success happens. + Keep keys out of source control, following how the app already handles configuration. + +8. Offer to send test traffic with a short script that simulates a few hundred users. In Node.js, create one + instance per user with `createFlagsmithInstance()` from `@flagsmith/flagsmith/isomorphic`, pass `identity` and + `fetch: globalThis.fetch` to `init`, call `getExperimentFlag`, track the event for some users and await + `flushEvents()`. In Python, keep the process alive for longer than the 10-second flush interval before it exits. + About a minute later, call `POST api/v1/environments//experiments//results/refresh/` + (allowed once every 5 minutes) and read `GET .../experiments//results/`. Finish with a link to the experiment: + /project//environment//experiments/, where the dashboard is + https://app.flagsmith.com on Cloud or http://localhost:8000 locally. +``` + +## After it runs + +Open the experiment in the dashboard to watch exposures arrive and read the results; see +[Analyse an Experiment](/experimentation/analyse-an-experiment). The flag is locked while the experiment runs. + +If the agent stops on an error, the most common causes are: + +- **No exposures recorded**: a variation has no key, the user isn't identified before `getExperimentFlag` is called, + or the flag is disabled in the environment. +- **Conversions not counted**: the event name doesn't exactly match the metric's event, or the event was sent with a + different identifier from the exposure. +- **Events not arriving on a local instance**: the SDK is still sending to Flagsmith Cloud. Set `eventsApiUrl` / + `events_api_url` to `http://localhost:8080/`. diff --git a/docs/docs/getting-started/run-locally-with-a-coding-agent.mdx b/docs/docs/getting-started/run-locally-with-a-coding-agent.mdx new file mode 100644 index 000000000000..19d764999fba --- /dev/null +++ b/docs/docs/getting-started/run-locally-with-a-coding-agent.mdx @@ -0,0 +1,98 @@ +--- +title: Run Flagsmith Locally with a Coding Agent +sidebar_label: Run Locally with an Agent +sidebar_position: 2.5 +description: + A copy-paste prompt that has a coding agent run Flagsmith with experimentation on your machine using Docker. +--- + +# Run Flagsmith Locally with a Coding Agent + +Paste the prompt below into [Claude Code](https://claude.com/claude-code), or any coding agent that can run shell +commands, and it will bring up a complete Flagsmith instance on your machine with Docker: + +1. Checks that Docker is installed and running, and that the ports Flagsmith needs are free. +2. Downloads the Flagsmith [Docker Compose file](/deployment-self-hosting/hosting-guides/docker), starts it and waits + for it to become healthy. +3. Creates your admin account, an organisation, a project, and Development and Production environments. +4. Connects the bundled ClickHouse warehouse to both environments, so you can run [experiments](/experimentation) + straight away. +5. Installs the [Flagsmith CLI](/integrating-with-flagsmith/CLI) and authenticates it against your instance. +6. Prints a summary with your login, environment keys and URLs. + +## Prerequisites + +- [Docker](https://docs.docker.com/get-docker/) with Docker Compose v2, and at least 4 GB of memory allocated to + Docker. +- [Node.js](https://nodejs.org/) and npm, to install the Flagsmith CLI. +- Ports 8000, 8001 and 8080 free on your machine. +- Claude Code or another coding agent. + +## The prompt + +```text title="Prompt: run Flagsmith locally" +Set up a local Flagsmith instance with experimentation for me, using Docker. Work step by step. After each step, +verify that it worked. If something fails, stop and show me the exact error; don't guess or work around it. + +1. Prerequisites. Check that Docker is running (`docker info`) and Docker Compose v2 is available + (`docker compose version`), and that ports 8000, 8001 and 8080 are free. If anything is missing or a port is + taken, tell me how to fix it and stop. Never stop or kill anything that is already running. + +2. Start Flagsmith. In a new `flagsmith` directory, download + https://raw.githubusercontent.com/Flagsmith/flagsmith/main/docker-compose.yml and run `docker compose up -d`. + The first run pulls several images and can take a few minutes. Wait until + http://localhost:8000/health/liveness/ and http://localhost:8080/health return 200, and `docker compose ps -a` + shows the one-off services (`migrate-db`, `kafka-init`, `clickhouse-init`) exited with code 0. If a service + fails, show me its last log lines. + +3. Create the admin account. Ask me for my name and email address (or use Admin User / admin@example.com if I + don't mind) and generate a strong password. There is no CLI bootstrap: the dashboard's onboarding screen does + this while `GET http://localhost:8000/version/` reports `self_hosted_data.has_users: false`, and you do the same + through the API: + - `POST /api/v1/auth/users/` with `{"email", "password", "re_password", "first_name", "last_name", + "superuser": true}`. `superuser` is only accepted while no users exist. The response's `key` authenticates + the next calls as `Authorization: Token `. If `has_users` is already true, ask me for the existing + credentials and get a key from `POST /api/v1/auth/login/` with `{"email", "password"}` instead. + - Create an organisation (`POST /api/v1/organisations/` with `{"name"}`), a project (`POST /api/v1/projects/` + with `{"name", "organisation"}`) and two environments, Development and Production + (`POST /api/v1/environments/` with `{"name", "project"}`). Reuse any that already exist. + +4. Connect the bundled ClickHouse warehouse to each environment with + `POST /api/v1/environments//warehouse-connections/` and this body: + {"warehouse_type": "clickhouse", "config": {"host": "clickhouse", "port": 8123, "database": "flagsmith_exp", + "username": "flagsmith", "secure": false}, "credentials": {"password": "password"}} + Each should come back with `"status": "connected"`. To check settings without saving, POST the same body to + `/api/v1/environments//warehouse-connections/test-warehouse-connection/`. If `/version/` reports + `experimentation_enabled: false`, skip this step and tell me. + +5. Set up the Flagsmith CLI. Install it with `npm install -g @flagsmith/cli` (v2 or later). Authenticate it with an + organisation API key, not `flagsmith login`: create one with `POST /api/v1/organisations//master-api-keys/` + and `{"name": "Flagsmith CLI", "is_admin": true}`; its `key` is only returned once. Against a self-hosted + instance the CLI only reads host-scoped keys, so write these two lines to `flagsmith/flagsmith.env`, with file + permissions 600: + export FLAGSMITH_API_URL=http://localhost:8000 + export FLAGSMITH_API_KEY_localhost_8000= + Then load the file and check that `flagsmith auth status` and `flagsmith project list` work. + +6. Finish with a summary: the dashboard URL (http://localhost:8000) with my email and password; the organisation, + project and environment names with each environment's key; the event ingestion URL for SDKs + (http://localhost:8080); how to load the CLI credentials (`source flagsmith/flagsmith.env`); and how to stop + Flagsmith (`docker compose down`) or delete everything (`docker compose down -v`). +``` + +## What you get + +| Service | URL | +| ------------------- | ----------------------- | +| Dashboard and API | `http://localhost:8000` | +| Task processor | `http://localhost:8001` | +| Event ingestion API | `http://localhost:8080` | + +The instance stores its data in Docker volumes, so it survives restarts until you run `docker compose down -v`. Treat it +as a local playground: the Compose file uses default secrets that you should change before exposing it anywhere. For +production deployments, see [Deployment and Self-hosting](/deployment-self-hosting). + +## Next steps + +To run your first A/B test against this instance, open your application's repository and use the +[experiment set-up prompt](/experimentation/set-up-with-a-coding-agent). From 37842c2b138f986108ac3473e0f3c4f5d4f3ae69 Mon Sep 17 00:00:00 2001 From: wadii Date: Fri, 16 Oct 2026 11:11:54 +0200 Subject: [PATCH 2/3] docs: align agent prompts with the experimentation SDK docs --- .../set-up-with-a-coding-agent.mdx | 73 ++++++++++--------- .../run-locally-with-a-coding-agent.mdx | 21 +++--- 2 files changed, 49 insertions(+), 45 deletions(-) diff --git a/docs/docs/experimentation/set-up-with-a-coding-agent.mdx b/docs/docs/experimentation/set-up-with-a-coding-agent.mdx index f09f00b8ab4c..d5f190519940 100644 --- a/docs/docs/experimentation/set-up-with-a-coding-agent.mdx +++ b/docs/docs/experimentation/set-up-with-a-coding-agent.mdx @@ -28,12 +28,14 @@ No application to hand? The agent can build a small demo page or script instead. ## Prerequisites -- A Flagsmith instance with a [connected warehouse](/experimentation/connect-a-warehouse): either Flagsmith Cloud, or - a local instance set up with the [local setup prompt](/getting-started/run-locally-with-a-coding-agent). -- An application using the JavaScript SDK (`@flagsmith/flagsmith` 12.1 or later, including React) or the Python SDK - (`flagsmith` 6.3 or later). Other SDKs, including the Node.js server SDK, can't send events yet. -- [Node.js](https://nodejs.org/) and npm, to install the [Flagsmith CLI](/integrating-with-flagsmith/CLI). -- Claude Code or another coding agent. +- A Flagsmith instance with a [connected warehouse](/experimentation/connect-a-warehouse): either Flagsmith Cloud, or a + local instance set up with the [local setup prompt](/getting-started/run-locally-with-a-coding-agent). +- An application using an SDK that supports experimentation: JavaScript or React (`@flagsmith/flagsmith` 12.4.0+), + Python (`flagsmith` 6.3.0+), Java (`com.flagsmith:flagsmith-java-client` 8.2.0+), Node.js (`@flagsmith/nodejs` + 9.1.0+), Go (`flagsmith-go-client/v5` v5.3.0+) or Flutter (`flagsmith` 6.2.0+). See + [Run an Experiment](/experimentation/run-an-experiment). +- [Node.js](https://nodejs.org/) and npm, to install the [Flagsmith CLI](/integrating-with-flagsmith/CLI). +- Claude Code or another coding agent. ## The prompt @@ -46,15 +48,18 @@ If something fails, stop and show me the exact error; don't guess or work around renders it. - What counts as success? (For example, "a completed purchase".) Find where that happens in the code; the conversion event goes there. - - Which Flagsmith are we using? Flagsmith Cloud, or a local instance (API http://localhost:8000, events - http://localhost:8080, CLI credentials in the `flagsmith.env` file written by the local setup)? + - Which Flagsmith are we using? Flagsmith Cloud, or a local instance (API http://localhost:8000, event ingestion + service http://localhost:8080, CLI credentials in the `flagsmith.env` file written by the local setup)? If there is no app to change, offer to build a tiny demo instead (a single HTML page or a script) that simulates users, so results show up within minutes. -2. Check the stack. Events need the JavaScript SDK `@flagsmith/flagsmith` 12.1 or later (browser or React) or the - Python SDK `flagsmith` 6.3 or later. The Node.js server SDK (`flagsmith-nodejs`) and other SDKs can't send events - yet: if that's all the app uses, tell me and stop. Find how the app identifies users: exposures and conversions - are joined per identity, so both must use the same identifier. +2. Check the stack. Read https://docs.flagsmith.com/experimentation/run-an-experiment and follow its tab for this + app's language from here on; it has the install, initialise, exposure and conversion code for each SDK. The SDKs + that can send events are JavaScript and React (`@flagsmith/flagsmith` 12.4.0+), Python (`flagsmith` 6.3.0+), Java + (`com.flagsmith:flagsmith-java-client` 8.2.0+), Node.js (`@flagsmith/nodejs` 9.1.0+; the older `flagsmith-nodejs` + package can't), Go (`flagsmith-go-client/v5` v5.3.0+) and Flutter (`flagsmith` 6.2.0+). If the app uses none of + them, tell me and stop. Find how the app identifies users: exposures and conversions are joined per identity, so + both must use the same identifier. 3. Connect the Flagsmith CLI. Install it with `npm install -g @flagsmith/cli` (v2 or later). For Cloud, ask me to run `flagsmith login`; for a local instance, load `flagsmith.env`. Ask me which project and environment to use @@ -84,24 +89,22 @@ If something fails, stop and show me the exact error; don't guess or work around `POST api/v1/environments//experiments//start/` and check its status is `running`. 7. Plan the code changes and show them to me. Don't edit anything until I confirm. Then: - - Install or upgrade the SDK and initialise it with events on. JavaScript: `enableEvents: true`; for a local - instance also `api: 'http://localhost:8000/api/v1/'` and - `eventProcessorConfig: { eventsApiUrl: 'http://localhost:8080/' }`. Python: `enable_events=True` and a - server-side key (create one with `POST api/v1/environments//api-keys/` and `{"name"}`); for a - local instance also `api_url='http://localhost:8000/api/v1/'` and - `event_processor_config=EventProcessorConfig(events_api_url='http://localhost:8080/')`. - - Identify the user, then call `getExperimentFlag('')` (React: `useExperiment('')`; Python: - `get_experiment_flag('', identifier=...)`) where the user actually sees the variation, and render by its - value. - - Call `trackEvent('')` (Python: `track_event('', identifier=...)`) where success happens. + - Install or upgrade the SDK and initialise it with events enabled, using the environment key the app already + uses for flags. For a local instance, also set the page's two self-hosted options: the API URL + http://localhost:8000/api/v1/ and the event ingestion service URL http://localhost:8080/. + - Identify the user, then get the experiment flag (for example `getExperimentFlag`, or `useExperiment` in React) + where the user actually sees the variation, and render by its value. + - Track the conversion event where success happens. + - Send buffered events before the app exits, as the page shows for this SDK. Keep keys out of source control, following how the app already handles configuration. -8. Offer to send test traffic with a short script that simulates a few hundred users. In Node.js, create one - instance per user with `createFlagsmithInstance()` from `@flagsmith/flagsmith/isomorphic`, pass `identity` and - `fetch: globalThis.fetch` to `init`, call `getExperimentFlag`, track the event for some users and await - `flushEvents()`. In Python, keep the process alive for longer than the 10-second flush interval before it exits. - About a minute later, call `POST api/v1/environments//experiments//results/refresh/` - (allowed once every 5 minutes) and read `GET .../experiments//results/`. Finish with a link to the experiment: +8. Offer to send test traffic with a short script that simulates a few hundred users, each with its own identifier: + get the experiment flag for each one and track the event for some of them. The SDKs send events in batches, so + send what is left before the script exits: JavaScript `await flushEvents()`, Python + `EventProcessorConfig(max_buffer_items=1)`, Node.js `await close()`, Java `close()`, Go `FlushEvents(ctx)`, + Flutter `await flushEvents()`. About a minute later, call + `POST api/v1/environments//experiments//results/refresh/` (allowed once every 5 minutes) and + read `GET .../experiments//results/`. Finish with a link to the experiment: /project//environment//experiments/, where the dashboard is https://app.flagsmith.com on Cloud or http://localhost:8000 locally. ``` @@ -113,9 +116,11 @@ Open the experiment in the dashboard to watch exposures arrive and read the resu If the agent stops on an error, the most common causes are: -- **No exposures recorded**: a variation has no key, the user isn't identified before `getExperimentFlag` is called, - or the flag is disabled in the environment. -- **Conversions not counted**: the event name doesn't exactly match the metric's event, or the event was sent with a - different identifier from the exposure. -- **Events not arriving on a local instance**: the SDK is still sending to Flagsmith Cloud. Set `eventsApiUrl` / - `events_api_url` to `http://localhost:8080/`. +- **No exposures recorded**: a variation has no key, the user isn't identified before the experiment flag is requested, + or the process exited before buffered events were sent. +- **Conversions not counted**: the event name doesn't exactly match the metric's event, or the event was sent with a + different identifier from the exposure. +- **Events not arriving on a local instance**: the SDK is still sending to Flagsmith Cloud. Set the event ingestion + service URL to `http://localhost:8080/`. + +See [Troubleshooting](/experimentation/run-an-experiment#troubleshooting) for more checks. diff --git a/docs/docs/getting-started/run-locally-with-a-coding-agent.mdx b/docs/docs/getting-started/run-locally-with-a-coding-agent.mdx index 19d764999fba..8a91913eac82 100644 --- a/docs/docs/getting-started/run-locally-with-a-coding-agent.mdx +++ b/docs/docs/getting-started/run-locally-with-a-coding-agent.mdx @@ -22,11 +22,10 @@ commands, and it will bring up a complete Flagsmith instance on your machine wit ## Prerequisites -- [Docker](https://docs.docker.com/get-docker/) with Docker Compose v2, and at least 4 GB of memory allocated to - Docker. -- [Node.js](https://nodejs.org/) and npm, to install the Flagsmith CLI. -- Ports 8000, 8001 and 8080 free on your machine. -- Claude Code or another coding agent. +- [Docker](https://docs.docker.com/get-docker/) with Docker Compose v2, and at least 4 GB of memory allocated to Docker. +- [Node.js](https://nodejs.org/) and npm, to install the Flagsmith CLI. +- Ports 8000, 8001 and 8080 free on your machine. +- Claude Code or another coding agent. ## The prompt @@ -75,18 +74,18 @@ verify that it worked. If something fails, stop and show me the exact error; don Then load the file and check that `flagsmith auth status` and `flagsmith project list` work. 6. Finish with a summary: the dashboard URL (http://localhost:8000) with my email and password; the organisation, - project and environment names with each environment's key; the event ingestion URL for SDKs + project and environment names with each environment's key; the event ingestion service URL for SDKs (http://localhost:8080); how to load the CLI credentials (`source flagsmith/flagsmith.env`); and how to stop Flagsmith (`docker compose down`) or delete everything (`docker compose down -v`). ``` ## What you get -| Service | URL | -| ------------------- | ----------------------- | -| Dashboard and API | `http://localhost:8000` | -| Task processor | `http://localhost:8001` | -| Event ingestion API | `http://localhost:8080` | +| Service | URL | +| ----------------------- | ----------------------- | +| Dashboard and API | `http://localhost:8000` | +| Task processor | `http://localhost:8001` | +| Event ingestion service | `http://localhost:8080` | The instance stores its data in Docker volumes, so it survives restarts until you run `docker compose down -v`. Treat it as a local playground: the Compose file uses default secrets that you should change before exposing it anywhere. For From 48ebb3012567d0f58eecdef0aa41f19308ad4413 Mon Sep 17 00:00:00 2001 From: wadii Date: Fri, 16 Oct 2026 14:32:50 +0200 Subject: [PATCH 3/3] docs: address review on coding agent prompts --- .../set-up-with-a-coding-agent.mdx | 26 ++++++++++++------- .../run-locally-with-a-coding-agent.mdx | 13 +++++----- 2 files changed, 23 insertions(+), 16 deletions(-) diff --git a/docs/docs/experimentation/set-up-with-a-coding-agent.mdx b/docs/docs/experimentation/set-up-with-a-coding-agent.mdx index d5f190519940..750017a600a2 100644 --- a/docs/docs/experimentation/set-up-with-a-coding-agent.mdx +++ b/docs/docs/experimentation/set-up-with-a-coding-agent.mdx @@ -19,9 +19,9 @@ agent that can run shell commands and edit code. It takes you from an idea to a 1. Asks what you want to test and what counts as success, and finds the matching places in your code. 2. Checks that your stack can send events. 3. Creates a [multivariate flag](/managing-flags/core-management) with a keyed variation, a - [metric](/experimentation/create-metrics) and an [experiment](/experimentation/create-an-experiment), and starts it. + [metric](/experimentation/create-metrics) and an [experiment](/experimentation/create-an-experiment). 4. Shows you the planned code changes and, once you confirm, adds the SDK calls that - [record exposures and conversion events](/experimentation/run-an-experiment). + [record exposures and conversion events](/experimentation/run-an-experiment) and starts the experiment. 5. Optionally sends test traffic and refreshes the results, so you can see the experiment working within minutes. No application to hand? The agent can build a small demo page or script instead. @@ -34,7 +34,7 @@ No application to hand? The agent can build a small demo page or script instead. Python (`flagsmith` 6.3.0+), Java (`com.flagsmith:flagsmith-java-client` 8.2.0+), Node.js (`@flagsmith/nodejs` 9.1.0+), Go (`flagsmith-go-client/v5` v5.3.0+) or Flutter (`flagsmith` 6.2.0+). See [Run an Experiment](/experimentation/run-an-experiment). -- [Node.js](https://nodejs.org/) and npm, to install the [Flagsmith CLI](/integrating-with-flagsmith/CLI). +- The [Flagsmith CLI](/integrating-with-flagsmith/CLI) v2 or later. The agent installs it if it's missing. - Claude Code or another coding agent. ## The prompt @@ -49,7 +49,8 @@ If something fails, stop and show me the exact error; don't guess or work around - What counts as success? (For example, "a completed purchase".) Find where that happens in the code; the conversion event goes there. - Which Flagsmith are we using? Flagsmith Cloud, or a local instance (API http://localhost:8000, event ingestion - service http://localhost:8080, CLI credentials in the `flagsmith.env` file written by the local setup)? + service http://localhost:8080)? For a local instance, ask me for the path to the `flagsmith.env` file the local + setup wrote; it holds the CLI credentials. If there is no app to change, offer to build a tiny demo instead (a single HTML page or a script) that simulates users, so results show up within minutes. @@ -61,9 +62,12 @@ If something fails, stop and show me the exact error; don't guess or work around them, tell me and stop. Find how the app identifies users: exposures and conversions are joined per identity, so both must use the same identifier. -3. Connect the Flagsmith CLI. Install it with `npm install -g @flagsmith/cli` (v2 or later). For Cloud, ask me to - run `flagsmith login`; for a local instance, load `flagsmith.env`. Ask me which project and environment to use - (`flagsmith project list`, `flagsmith environment list`) and bind this repository to them with +3. Connect the Flagsmith CLI. If `flagsmith --version` doesn't report v2 or later, install it with + `curl -fsSL https://get.flagsmith.com | sh` or `brew install Flagsmith/tap/flagsmith`; use + `npm install -g @flagsmith/cli` only as a fallback. For Cloud, ask me to run `flagsmith login`; for a local + instance, run `source `. Don't copy that file or its admin API key into this repository, + and never print the key. Ask me which project and environment to use (`flagsmith project list`, + `flagsmith environment list --project `) and bind this repository to them with `flagsmith init --project --environment `. Make the API calls below with `flagsmith api ` (`--input -` reads a JSON body from stdin). Don't use the Flagsmith MCP server: it can't start experiments. The environment must have a warehouse: if @@ -85,8 +89,8 @@ If something fails, stop and show me the exact error; don't guess or work around "experiment_rollout": {"enabled": true, "rollout_percentage": 100, "feature_state_value": {"type": "string", "value": "control"}, "multivariate_feature_state_values": [{"multivariate_feature_option": , "percentage_allocation": 50}]}} - Each variation gets its `percentage_allocation` and control gets the rest. Start it with - `POST api/v1/environments//experiments//start/` and check its status is `running`. + Each variation gets its `percentage_allocation` and control gets the rest. Don't start it yet: starting locks the + flag and turns on the rollout. 7. Plan the code changes and show them to me. Don't edit anything until I confirm. Then: - Install or upgrade the SDK and initialise it with events enabled, using the environment key the app already @@ -96,7 +100,9 @@ If something fails, stop and show me the exact error; don't guess or work around where the user actually sees the variation, and render by its value. - Track the conversion event where success happens. - Send buffered events before the app exits, as the page shows for this SDK. - Keep keys out of source control, following how the app already handles configuration. + Keep keys out of source control, following how the app already handles configuration. Once the changes are in, + start the experiment with `POST api/v1/environments//experiments//start/` and check its + status is `running`. 8. Offer to send test traffic with a short script that simulates a few hundred users, each with its own identifier: get the experiment flag for each one and track the event for some of them. The SDKs send events in batches, so diff --git a/docs/docs/getting-started/run-locally-with-a-coding-agent.mdx b/docs/docs/getting-started/run-locally-with-a-coding-agent.mdx index 8a91913eac82..434cbce1528a 100644 --- a/docs/docs/getting-started/run-locally-with-a-coding-agent.mdx +++ b/docs/docs/getting-started/run-locally-with-a-coding-agent.mdx @@ -23,7 +23,6 @@ commands, and it will bring up a complete Flagsmith instance on your machine wit ## Prerequisites - [Docker](https://docs.docker.com/get-docker/) with Docker Compose v2, and at least 4 GB of memory allocated to Docker. -- [Node.js](https://nodejs.org/) and npm, to install the Flagsmith CLI. - Ports 8000, 8001 and 8080 free on your machine. - Claude Code or another coding agent. @@ -64,11 +63,13 @@ verify that it worked. If something fails, stop and show me the exact error; don `/api/v1/environments//warehouse-connections/test-warehouse-connection/`. If `/version/` reports `experimentation_enabled: false`, skip this step and tell me. -5. Set up the Flagsmith CLI. Install it with `npm install -g @flagsmith/cli` (v2 or later). Authenticate it with an - organisation API key, not `flagsmith login`: create one with `POST /api/v1/organisations//master-api-keys/` - and `{"name": "Flagsmith CLI", "is_admin": true}`; its `key` is only returned once. Against a self-hosted - instance the CLI only reads host-scoped keys, so write these two lines to `flagsmith/flagsmith.env`, with file - permissions 600: +5. Set up the Flagsmith CLI. If `flagsmith --version` doesn't report v2 or later, install it with + `curl -fsSL https://get.flagsmith.com | sh`, which puts it in `$HOME/.local/bin`, or with + `brew install Flagsmith/tap/flagsmith`; use `npm install -g @flagsmith/cli` only as a fallback. Check the version + again. Authenticate the CLI with an organisation API key, not `flagsmith login`: create one with + `POST /api/v1/organisations//master-api-keys/` and `{"name": "Flagsmith CLI", "is_admin": true}`; its `key` + is only returned once. Against a self-hosted instance the CLI only reads host-scoped keys, so write these two + lines to `flagsmith/flagsmith.env`, with file permissions 600: export FLAGSMITH_API_URL=http://localhost:8000 export FLAGSMITH_API_KEY_localhost_8000= Then load the file and check that `flagsmith auth status` and `flagsmith project list` work.