diff --git a/agents.md b/agents.md index d4ef34b..b737372 100644 --- a/agents.md +++ b/agents.md @@ -26,6 +26,7 @@ uv run mkdocs build # Generate static site to public/ docs/ overview/ Product overview and concepts concepts/ Single-user isolation, devices, apps + self_hosting/ Running a shard on your own machine (prerequisites, known gaps) developer_docs/ Developer documentation for app creators includes/ Reusable markdown snippets (template vars, portal name info) img/ Developer docs images @@ -39,7 +40,7 @@ mkdocs.yml Site config, navigation, theme, plugins ## Conventions ### Navigation -Navigation is explicitly defined in `mkdocs.yml` under the `nav` key. Three top-level sections: Overview, Developer Docs, User Guides. Adding a new page requires adding it to `nav`. +Navigation is explicitly defined in `mkdocs.yml` under the `nav` key. Four top-level sections: Overview, Developer Docs, Self-Hosting, User Guides. Adding a new page requires adding it to `nav`. ### Retired Blog The blog no longer lives here. It moved to `freeshard.net//blog/`, as an Astro content collection in the `landing-page` repo — write new posts there, not here. @@ -67,6 +68,6 @@ Built and deployed via GitHub Pages (`.github/workflows/ci.yml`). Every push bui [Scoped Commits](https://scopedcommits.com/): `: `. The scope is the area of the tree the change touches, never a change type — write `user_guides: document the backup restore flow`, not `fix(user_guides): ...`. Body and trailers are optional; a change's reasoning belongs in the body, not in a code comment. -Scopes for this repo: `blog` `developer_docs` `user_guides` `overview` `theme` `ci` `meta` +Scopes for this repo: `blog` `developer_docs` `self_hosting` `user_guides` `overview` `theme` `ci` `meta` `meta` covers repo-level files (agents.md, README, justfile). For a change spanning several scopes, use a broader one, list two comma-separated, or use `treewide`. Merges and reverts keep their own format. Don't generate a changelog from the commit log — release notes come from merged PRs. diff --git a/docs/self_hosting/index.md b/docs/self_hosting/index.md new file mode 100644 index 0000000..c06ee4d --- /dev/null +++ b/docs/self_hosting/index.md @@ -0,0 +1,158 @@ +# Self-Hosting + +Freeshard's software, Shard Core, can run on a machine of your own instead of a hosted shard. +Its source code is public under the [Functional Source License (FSL-1.1-ALv2)](https://github.com/FreeshardBase/freeshard/blob/main/LICENSE.md), which permits self-hosting. + +This page is the complete guide: what you need, how to try it locally, how to run it on a server, and what does not work yet. + +!!! warning "Early stage" + A self-hosted shard runs, but the setup is not polished yet. + Every known gap is listed under [Known gaps](#known-gaps), with the issue that tracks it. + +## Prerequisites + +* **A Linux machine** with Docker Engine and the Compose plugin (`docker compose`). +* **Free ports and network name.** Ports 80, 443 and 8883 must be free, and there must be no Docker network called `portal`. Both are fixed in the compose file ([#196](https://github.com/FreeshardBase/freeshard/issues/196)). A dedicated machine or VM is recommended, see [Known gaps](#known-gaps). +* **Root or `sudo`.** The containers write the shard's data as root. +* **Memory and disk.** There is no tested minimum yet. For comparison, the smallest hosted shard has 1 vCPU and 2 GB of RAM. Apps are downloaded when they are first opened, and some are large: the Immich images alone are about 3.5 GB. +* **For a local test: `*.localhost` resolution.** The shard is reached at `http://.localhost`. systemd-resolved resolves such names; plain glibc without it and Windows do not. +* **For a server on the internet: a domain with wildcard DNS**, and for now a DNS zone hosted in Azure DNS. See [Certificates](#certificates) before you rent a server for this. + +## Get the files + +A shard needs two files from the [repository](https://github.com/FreeshardBase/freeshard), in one directory of your choice. +Run every command on this page from that directory. + +```shell +mkdir freeshard && cd freeshard +curl -fsSL -o docker-compose.yml https://raw.githubusercontent.com/FreeshardBase/freeshard/main/docker-compose.yml +curl -fsSL -o .env https://raw.githubusercontent.com/FreeshardBase/freeshard/main/.env.template +``` + +The compose file pins the same versions of Shard Core, Traefik, Postgres and the Sundial web interface that hosted shards run. + +## Try it locally + +1. Edit `.env`: + * `FREESHARD_DIR`: an absolute path where all shard data will be stored. It is created on first start and owned by root. + * `DNS_ZONE`: leave it at `localhost`. + * `EMAIL`: can stay as it is, no certificates are requested. + * `DISABLE_SSL`: set it to `true`. +2. Start the shard: + ```shell + docker compose up -d + ``` +3. Read the welcome message: + ```shell + docker compose logs shard_core + ``` + It shows the Freeshard logo, the shard's address `http://.localhost` and, as long as no device is paired, a pairing link that is valid for 10 minutes. +4. Open the pairing link in your browser. If the code is not filled in, type it into the pairing field. + +To pair another device, open **Devices** on a device that is already paired and choose **Pair new device**. +It shows a code and a QR code for the new device. + +If no paired device is left, get a fresh code from inside the shard's network instead: + +```shell +docker run --rm --network portal curlimages/curl -s "http://shard_core/protected/terminals/pairing-code" +``` + +Then open `http://.localhost/#/pair` on the new device and enter the code. + +## Run it on a server + +1. Use a server that is reachable from the internet, with a domain whose DNS you control. +2. Point the domain and every subdomain at the server. Each app gets its own subdomain under the shard's domain, so you need a wildcard record, for example `*.example.org`. The shard's own address is `.example.org`, which the wildcard covers. +3. [Get the files](#get-the-files) onto the server and edit `.env`: + * `FREESHARD_DIR`: an absolute path for all shard data. + * `DNS_ZONE`: your domain, for example `example.org`. + * `EMAIL`: your address, which Let's Encrypt uses for certificate notices. + * `DISABLE_SSL`: `false`. Never set it to `true` on a server reachable from the internet. +4. Set up certificates as described under [Certificates](#certificates). +5. Start the shard with `docker compose up -d`, then read the welcome message and pair your first device as in the local test. The address now starts with `https://`. + +### Certificates + +Every app lives on its own subdomain, so the shard needs a wildcard certificate, and Let's Encrypt issues those only through a DNS challenge. +Traefik answers that challenge by creating a DNS record through your DNS provider's API. + +Today Shard Core configures Traefik for Azure DNS only, and the provider cannot be changed without rebuilding the image ([#292](https://github.com/FreeshardBase/freeshard/issues/292)). +If your zone is in Azure DNS, uncomment the `AZURE_*` variables and `LEGO_AZURE_BYPASS_DEPRECATION` of the `traefik` service in `docker-compose.yml` and fill them in with a service principal that may edit the zone ([Traefik's provider list](https://doc.traefik.io/traefik/v2.11/https/acme/#providers) names the variables). +With any other DNS provider, Traefik falls back to its own self-signed certificate, which browsers reject. + +## Configuration + +All settings live in `.env`, and all four are required. + +| Variable | Meaning | +|---|---| +| `FREESHARD_DIR` | Absolute path for all shard data | +| `DNS_ZONE` | Domain the shard lives under; `localhost` for a local test | +| `EMAIL` | Contact address for Let's Encrypt | +| `DISABLE_SSL` | `true` only for a local test without certificates, otherwise `false` | + +After changing `.env`, run `docker compose up -d` again. + +### The owner's email address + +Set your own email address under **Settings** right after pairing the first device, before you log into any app. +Apps that support it log you in through the shard's built-in login provider, which passes them this address. +Without one, those apps create fresh accounts instead of linking to an existing account by address. + +A self-hosted shard cannot send mail, so the address is not confirmed by a link as on a hosted shard: it counts as soon as you save it. +Check it for typos before saving. + +## What a self-hosted shard still gets from Freeshard + +A self-hosted shard is not self-contained ([#195](https://github.com/FreeshardBase/freeshard/issues/195)). Both halves of the app catalogue come from infrastructure Freeshard operates: + +* the catalogue and the app packages come from an Azure storage container (`apps.app_store` in [`config.toml`](https://github.com/FreeshardBase/freeshard/blob/main/config.toml)), and +* the container images of apps that do not use a public upstream image come from Freeshard's registry, `portalapps.azurecr.io`. + +Shard Core logs in to that registry with credentials that are committed to `config.toml` on purpose. +They belong to a service principal whose only permission anywhere is reading that one registry: it can pull images and read registry metadata, nothing else. +Every shard needs it to install apps. +If you would rather not depend on Freeshard's infrastructure, these two are what you would have to replace. + +The shard also still contacts Freeshard's management service, for example at startup and for the nightly backup. +Those requests fail for a self-hosted shard, which has no account there ([#286](https://github.com/FreeshardBase/freeshard/issues/286)). + +## Known gaps + +* **Certificates only through Azure DNS** ([#292](https://github.com/FreeshardBase/freeshard/issues/292)), see [Certificates](#certificates). +* **No automatic backup.** The nightly backup depends on Freeshard's management service and fails on a self-hosted shard, although the interface says a backup runs every night ([#286](https://github.com/FreeshardBase/freeshard/issues/286)). See [Backups](#backups). +* **The shard assumes it owns the host:** fixed ports, a fixed network name, and Traefik also picks up other containers on the host ([#196](https://github.com/FreeshardBase/freeshard/issues/196)). It works next to other services, but a dedicated machine or VM avoids these clashes. +* **Isolation.** Shard Core and Traefik have access to the Docker socket, which amounts to root access on the host. Run a shard on a machine or VM that does nothing else you care about ([#198](https://github.com/FreeshardBase/freeshard/issues/198)). + +## Backups + +Everything a shard has is in `FREESHARD_DIR`: its identity and private key, the database, the installed apps and all user data. +Back up that directory, ideally with the shard stopped (`docker compose stop`) so the database is consistent. +The repository README describes [how hosted shards back up and restore](https://github.com/FreeshardBase/freeshard#backup-and-restore). + +## Updating + +Download the current `docker-compose.yml` again (see [Get the files](#get-the-files)), keep your `.env`, and run: + +```shell +docker compose up -d +``` + +Docker pulls the new images and recreates the containers. Database migrations run when Shard Core starts. +If you changed `docker-compose.yml` yourself, for example for certificates, carry those changes over. + +## Uninstalling + +`docker compose down` stops and removes the containers. +It does not remove any data, not even with `-v`: everything lives in `FREESHARD_DIR`, including the shard's private key. +To remove the shard completely, delete that directory, which needs `sudo` because the files belong to root: + +```shell +docker compose down +sudo rm -rf +``` + +## Questions + +If something on this page is wrong or out of date, or you get stuck, open an issue on [GitHub](https://github.com/FreeshardBase/freeshard/issues) or write to [contact@freeshard.net](mailto:contact@freeshard.net). diff --git a/mkdocs.yml b/mkdocs.yml index 61ad00a..308ebf1 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -52,6 +52,8 @@ nav: - 'developer_docs/revenue_share.md' - 'developer_docs/existing_apps.md' # - 'developer_docs/app_template.md' + - Self-Hosting: + - 'self_hosting/index.md' - User Guides: - 'user_guides/password_management.md' - 'user_guides/smart_home.md'