# OpenShift Certificate Renewal Process - josie.lol
Date: 2025-08-27

## Issue Discovery
- External certificate for josie.lol was expired (expired Aug 25, 2025)
- Internal OpenShift certificate was newer and valid until Oct 24, 2025
- Mismatch between internal cert-manager certificate and what was served externally

## Initial Investigation
1. Checked OpenShift cluster connection: ✓ Connected
2. Listed certificates across all namespaces:
   - josie-lol-tls in josiedotlol namespace: Status "Ready", valid until Oct 24, 2025
   - container.mom certificates: All valid until Oct/Nov 2025
3. External test revealed expired certificate being served:
   ```
   curl https://josie.lol
   # SSL certificate problem: certificate has expired
   ```

## Certificate Renewal Process
1. **Forced certificate renewal:**
   ```bash
   oc annotate certificate josie-lol-tls -n josiedotlol cert-manager.io/issue-temporary-certificate="true" --overwrite
   oc delete secret josie-lol-tls-secret -n josiedotlol
   ```

2. **Verified internal certificate renewal:**
   - New certificate generated: Aug 27 07:20:17 2025 - Nov 25 07:20:16 2025
   - Certificate status: "Ready" in OpenShift

3. **Route Configuration Issues:**
   - Found route was using old certificate in spec.tls section
   - Route: route-josie-lol in josiedotlol namespace
   - Route was not automatically picking up renewed certificate

## Attempted Solutions
1. **Manual certificate injection into route:** Tried updating route TLS spec directly
2. **Router pod restart:** Deleted router pod to force reload
3. **Route recreation:** Deleted and recreated route without custom certificate
4. **Cert-manager annotations:** Added cert-manager.io/cluster-issuer=letsencrypt-prod

## Current Status
- Internal certificate: ✓ Renewed and valid
- External access: ✗ Still serving expired certificate
- Route integration: ✗ cert-manager not automatically injecting certificate into route

## Discovered Integration Method
- OpenShift has feature gates for cert-manager route integration
- Need to investigate openshift-routes controller configuration
- Route controller manager found: openshift-route-controller-manager namespace

## Next Steps (Interrupted)
- Check route controller feature gate settings
- Investigate if openshift-routes controller is properly configured
- Ensure cert-manager can automatically manage route certificates

## Key Commands Used
```bash
# Check certificates
oc get certificates -A
oc get secret [secret-name] -n [namespace] -o jsonpath='{.data.tls\.crt}' | base64 -d | openssl x509 -noout -dates

# Force renewal  
oc annotate certificate [cert-name] cert-manager.io/issue-temporary-certificate="true" --overwrite
oc delete secret [secret-name]

# Route management
oc get route -n [namespace]
oc annotate route [route-name] cert-manager.io/cluster-issuer=letsencrypt-prod

# Test external access
curl -v https://josie.lol
echo | openssl s_client -connect josie.lol:443 -servername josie.lol 2>/dev/null | openssl x509 -noout -dates
```

## Lessons Learned
- cert-manager successfully renewed the internal certificate
- Route integration with cert-manager requires specific configuration
- External certificate serving depends on proper route TLS configuration
- Feature gates may be required for automatic cert-manager route integration