# Automate Soju Upstream SASL Auth to Ergo

## Context

When a user creates a bouncer, web-api spawns `add_default_upstream()` immediately after
creating the SojuBouncer CR. This races the soju-operator, which hasn't provisioned the
database or deployed soju yet. The background task fails silently, leaving new bouncers
with no upstream network to ergo -- or no SASL credentials on the network.

A `backfill_sasl_if_needed()` function exists and runs on every login, but it has a bug:
when the ergo account already exists, `register_ergo_account()` returns Ok without setting
the new password on ergo, then backfill UPDATEs soju's network with a mismatched password.

Result: all bouncers show as "unauthenticated" on net.irc.now.

## Approach

Replace the racing fire-and-forget spawn with a kube.rs watcher in web-api that reacts
to SojuBouncer status changes. When a bouncer transitions to `Ready=True` and hasn't been
set up yet (tracked via annotation), run the upstream setup.

## Changes

### 1. Fix ergo password handling for existing accounts

**File:** `crates/web-api/src/ergo_admin.rs`

- Add `ensure_ergo_account()` that wraps register + password reset:
  1. Try `NS SAREGISTER {username} {password}`
  2. If "account already exists", send `NS SAPASSWD {username} {password}` to force-set
     the password via oper privileges
  3. Return `Ok(password)` on success
- This fixes the bug where backfill sets SASL credentials that don't match ergo.

### 2. Create bouncer watcher module

**File (new):** `crates/web-api/src/bouncer_watcher.rs`

```
pub async fn watch_bouncers(state: AppState)
```

- Uses `kube::runtime::watcher(api, Config::default())` to watch SojuBouncer CRs
- On each `Applied` event, check:
  - `status.conditions` has `Ready=True`
  - Annotation `irc.now/upstream-configured` is NOT present
- When both conditions met:
  1. Read `{name}-db` secret, connect to soju's PostgreSQL
  2. Wait for `User` table to have a row (soju may still be initializing schema)
  3. Call `ensure_ergo_account()` with the soju username
  4. UPSERT the `irc.now` network with SASL PLAIN credentials:
     ```sql
     INSERT INTO "Network" ("user", name, addr, nick, enabled, sasl_mechanism,
                             sasl_plain_username, sasl_plain_password)
     SELECT id, 'irc.now', 'irc+insecure://irc-now-net:6667', username, true,
            'PLAIN', username, $1
     FROM "User" LIMIT 1
     ON CONFLICT (name, "user") DO UPDATE
       SET sasl_mechanism = 'PLAIN',
           sasl_plain_username = EXCLUDED.sasl_plain_username,
           sasl_plain_password = EXCLUDED.sasl_plain_password
     ```
  5. Patch the CR with annotation `irc.now/upstream-configured: "true"`
- On error: log warning, do NOT annotate (watcher will retry on next event/requeue)
- Wrap the stream in a loop so it restarts on disconnect

### 3. Remove racing spawn from bouncer creation

**File:** `crates/web-api/src/routes/bouncer.rs`

- Remove the `tokio::spawn(add_default_upstream(...))` block (lines 151-157)
- Remove `add_default_upstream()` function (lines 383-431) -- superseded by watcher

### 4. Remove login-time backfill

**File:** `crates/web-api/src/routes/auth.rs`

- Remove the `tokio::spawn(backfill_sasl_if_needed(...))` block (lines 200-205)

**File:** `crates/web-api/src/ergo_admin.rs`

- Remove `backfill_sasl_if_needed()` function (lines 143-237) -- superseded by watcher

### 5. Spawn watcher at startup

**File:** `crates/web-api/src/main.rs`

- Add `mod bouncer_watcher;`
- Spawn: `tokio::spawn(bouncer_watcher::watch_bouncers(state.clone()));`
- Place alongside the existing business_metrics spawns

## Existing bouncers

The watcher handles migration automatically. On first startup it receives `Applied`
events for all existing SojuBouncer CRs. Those that are `Ready=True` without the
annotation get the upstream setup. This fixes `josie` and any other pre-existing bouncers.

After setup, the annotation prevents the watcher from ever touching the network again.
Users own their soju DB state from that point forward.

## Key files

- `crates/web-api/src/ergo_admin.rs` -- ergo account registration, password reset
- `crates/web-api/src/bouncer_watcher.rs` -- new watcher module
- `crates/web-api/src/routes/bouncer.rs` -- remove racing spawn
- `crates/web-api/src/routes/auth.rs` -- remove login-time backfill
- `crates/web-api/src/main.rs` -- spawn watcher
- `crates/web-api/src/k8s.rs` -- SojuBouncer type (read-only, already has status)

## Verification

1. `cargo build -p irc-now-web-api` -- confirm it compiles
2. `cargo test -p irc-now-web-api` -- existing tests pass
3. Deploy to cluster, check web-api logs for watcher activity
4. Verify existing bouncers get annotated and SASL credentials set:
   ```
   oc get sojubouncers -o jsonpath='{.items[*].metadata.annotations}'
   oc exec soju-db-1 -- psql -U postgres -d soju_josie \
     -c "SELECT name, sasl_mechanism, sasl_plain_username FROM \"Network\" WHERE name='irc.now'"
   ```
5. Create a new bouncer via my.irc.now, wait for Ready, verify upstream appears with SASL
6. `/whois josie` on chat.irc.now should show authenticated