Document the two bootstrap playbooks; fix the notebook's inventory name

ansible/README.md

@@ -25,7 +25,9 @@ ansible-galaxy install -r requirements/ansible-galaxy.yml
 ```
 
 In order to gain access to machines you will have to add your public key to the
-`ssh_users` variable inside of `ansible/group_vars/all/vars.yml`.
+`ssh_users` variable inside of `ansible/group_vars/all/vars.yml`, and your username to
+`admin_usernames` or `non_admin_usernames`. Hosts with their own lists in `host_vars/`
+(e.g. the notebook server) need you added there too.
 
 It's recommended you generate an `ed25519` key using the following command:
 ```
@@ -106,7 +108,21 @@ In general there are two classes of playbooks:
 
 Some notable playbooks or roles are:
 
-The bootstrap playbook is in `playbook-bootstrap.yml` and is a playbook that should be run once when a new host is created.
+There are two bootstrap playbooks:
+
+* `playbook-bootstrap.yml`: only for a newly created host. It logs in as root (its
+  `remote_user: root` overrides `-u`) and doesn't use `become`. On a host that is already set
+  up it fails with `root@<host>: Permission denied (publickey)`, because root isn't in `AllowUsers`.
+* `deploy-bootstrap.yml`: for hosts that are already set up, including user changes. It logs
+  in as you and uses `sudo`. To apply user changes only, someone who can already log in to the
+  host runs:
+
+  ```
+  ./play -i inventory deploy-bootstrap.yml -l <inventory name> -t ssh_users
+  ```
+
+  Add `-u <your username>` if your local username (or the `User` in `~/.ssh/config`) isn't
+  your username on the host, e.g. inside a container where you are root.
 
 The nftables firewall is configured to read every `.nft` file under
 `/etc/ooni/nftables/` and `/etc/ooni/nftables/`. This allows roles to
@@ -125,7 +141,8 @@ This can be avoided by selecting only the required tags using `-t <tagname>`.
 
 Ideally the root user should be disabled after succesfully creating user accounts. See role
 [ssh_users](https://github.com/ooni/devops/blob/main/ansible/roles/ssh_users/tasks/main.yml#L62)
-which adds AllowUsers to /etc/sshd_config.d/00-ansible_system_role.conf and disables root login.
+which writes `/etc/ssh/sshd_config` with an `AllowUsers` line listing only the host's `admin_usernames`
+and `non_admin_usernames`: root, and anyone missing from those lists, can't log in.
 
 #### Roles layout
 

ansible/playbook-bootstrap.yml

@@ -1,5 +1,8 @@
 # This playbook is to be run on hosts that don't support bootstrapping the base
 # OS setup with something other than ansible (eg. cloud-init)
+#
+# Only for a newly created host: it logs in as root, which hosts that are already set up refuse.
+# For those, use deploy-bootstrap.yml.
 - name: Bootstrap the ssh_users on target host
   hosts: all
   remote_user: root

docs/Runbooks.md

@@ -1187,22 +1187,7 @@ PRIMARY KEY (measurement_uid, observation_idx) ORDER BY (measurement_uid, observ
 
 ### Adding users to notebook server
 
-In order to add new users to the jupyterlab notebook server hosted
-`notebook.ooni.org` the steps are:
-
-1. Open a PR adding an entry to this list:
-   https://github.com/ooni/devops/blob/main/ansible/host_vars/notebook.ooni.org (we need ssh key, email and username)
-2. An OONI core team member will run:
-```
-./play deploy-bootstrap.yml -l notebook.ooni.org --diff -i inventory
-```
-3. An OONI core team member logs into `notebook.ooni.org` and runs:
-```
-sudo passwd NEWUSERNAME
-```
-and sets the password of the new user to a randomly generated password prefixed with the string `CHANGEME`
-4. This password is sent privately to the new user and they are told to login via ssh using their key to update the password
-5. This user is now able to login with the password they have chosen via https://notebook.ooni.org/
+See [Adding External Users to the Notebooks Server](#adding-external-users-to-the-notebooks-server).
 
 ### Reprocessing data
 
@@ -1240,21 +1225,29 @@ The notebooks server uses the system’s actual user accounts to authenticate ac
 4. Ask them to log in once to update their password.
 5. They can then log in to [notebook.ooni.org](https://notebook.ooni.org) with their new password.
 
+`notebook.ooni.org` is the server's public name; in Ansible (inventory, host vars, `-l`) the same
+machine is `notebook1.htz-fsn.prod.ooni.nu`.
+
 ### Procedure
 
 0. Ask the person requesting a new account for the following details: **name**, **username** (used to log in to the server), and **SSH key**.
-1. Create a new user entry in the [notebook server’s host vars](https://github.com/ooni/devops/blob/main/ansible/host_vars/notebook.ooni.org). Remember to also add the username to the `non_admin_usernames` variable
-2. In the `ansible` directory, run:
+1. Open a PR adding a user entry to the [notebook server’s host vars](https://github.com/ooni/devops/blob/main/ansible/host_vars/notebook1.htz-fsn.prod.ooni.nu), with their key. Remember to also add the username to the `non_admin_usernames` variable.
+2. Once it's merged, an OONI core team member who can log in to the server runs, in the `ansible` directory:
 
    ```
-   ./play -i inventory deploy-bootstrap.yml -l notebook.ooni.org --diff
+   ./play -i inventory deploy-bootstrap.yml -l notebook1.htz-fsn.prod.ooni.nu -t ssh_users
    ```
+
+   Add `-u <your username>` if needed (see the bootstrap playbooks in `ansible/README.md`).
+   Don't use `playbook-bootstrap.yml`: it logs in as root, which the server refuses.
 3. Log in to `notebook.ooni.org` via SSH.
-4. Set the default password for the new user using:
+4. Set the default password for the new user to a randomly generated one prefixed with `CHANGEME`, using:
 
    ```
    sudo passwd <username>
    ```
+
+   and send it to them privately.
 5. Expire the password using:
 
    ```