# Getting Started with Karpenter IBM Cloud Provider

This guide walks you through setting up Karpenter IBM Cloud Provider from installation to your first auto-scaled workload.

## Prerequisites

Before starting, ensure you have:

### Required Access
- **IBM Cloud Account** with VPC Infrastructure Services access
- **Kubernetes Cluster** (IKS or self-managed on IBM Cloud VPC)
- **kubectl** configured for your cluster
- **Helm 3** for installation

### Required Tools
```bash
# Verify tools are available
kubectl version --client
helm version
ibmcloud version

# Install IBM Cloud CLI if needed
curl -fsSL https://clis.cloud.ibm.com/install/linux | sh
ibmcloud plugin install vpc-infrastructure
```

## IBM Cloud Setup

### Step 1: Create Service ID and API Keys
For production environments, use Service IDs for better security:

```bash
# Login to IBM Cloud
ibmcloud login

# Create Service ID for Karpenter
ibmcloud iam service-id-create karpenter-provider \
  --description "Service ID for Karpenter IBM Cloud Provider"

# Get Service ID
SERVICE_ID=$(ibmcloud iam service-ids --output json | jq -r '.[] | select(.name=="karpenter-provider") | .id')

# Assign VPC Infrastructure Services role
ibmcloud iam service-policy-create $SERVICE_ID \
  --roles "VPC Infrastructure Services" \
  --service-name is

# Create API keys
ibmcloud iam service-api-key-create karpenter-general $SERVICE_ID \
  --description "General IBM Cloud API access for Karpenter"

ibmcloud iam service-api-key-create karpenter-vpc $SERVICE_ID \
  --description "VPC-specific API access for Karpenter"
```

**Save the API keys securely - they won't be shown again!**

### Step 2: Gather Required Resource Information

```bash
# Set your target region
export REGION=us-south
ibmcloud target -r $REGION

# List available VPCs
ibmcloud is vpcs --output json

# Choose your VPC and list subnets
export VPC_ID="your-vpc-id"
ibmcloud is subnets --vpc $VPC_ID --output json

# List security groups in your VPC
ibmcloud is security-groups --vpc $VPC_ID --output json

# List available images
ibmcloud is images --visibility public --status available | grep ubuntu
```

**Collect the following information:**
- **VPC ID**: `vpc-12345678` (from VPC list)
- **Subnet ID**: `subnet-12345678` (choose one per zone you want to use)
- **Security Group ID**: `sg-12345678` (existing or create new)
- **Image ID**: `r006-12345678` (Ubuntu 20.04 recommended)
- **Region**: `us-south` (or your preferred region)
- **Zone**: `us-south-1` (subnet's availability zone)

## Installation

### Step 1: Install Helm Chart

Install directly with API keys as helm values:

```bash
# Add Helm repository
helm repo add karpenter-ibm https://pfeifferj.github.io/karpenter-provider-ibm-cloud
helm repo update

# Install with your API keys
helm install karpenter karpenter-ibm/karpenter \
  --namespace karpenter \
  --create-namespace \
  --set credentials.region="us-south" \
  --set credentials.ibm_api_key="your-general-api-key" \
  --set credentials.vpc_api_key="your-vpc-api-key"
```

### Step 2: Verify Installation
```bash
# Check if pods are running
kubectl get pods -n karpenter

# Check controller logs for startup
kubectl logs -n karpenter deployment/karpenter -f

# Look for successful controller startup messages
# Expected: "Starting Controller" for nodepool, nodeclaim, nodeclass controllers
```

**Expected Output:**
```
Starting Controller {"controller": "nodepool", "controllerGroup": "karpenter.sh"}
Starting Controller {"controller": "nodeclaim", "controllerGroup": "karpenter.sh"}
Starting Controller {"controller": "nodeclass.hash", "controllerGroup": "karpenter.ibm.sh"}
Starting Controller {"controller": "pricing", "controllerGroup": "karpenter.ibm.sh"}
```

### Step 3: Create Your First NodeClass

Choose the configuration that matches your environment:

**Replace the placeholder values with your actual resource IDs:**

```bash
# Create basic NodeClass
cat <<EOF | kubectl apply -f -
apiVersion: karpenter.ibm.sh/v1alpha1
kind: IBMNodeClass
metadata:
  name: default-nodeclass
  annotations:
    karpenter.ibm.sh/description: "Basic NodeClass for getting started"
spec:
  # REQUIRED: Replace with your actual values
  region: us-south                    # Your IBM Cloud region
  zone: us-south-1                    # Target availability zone
  vpc: vpc-12345678                   # Your VPC ID
  image: r006-12345678                # Ubuntu 20.04 image ID
  
  # REQUIRED: Security and access
  securityGroups:
  - sg-12345678                       # Your security group ID
  
  # OPTIONAL: Specific subnet (auto-selected if not specified)
  subnet: subnet-12345678             # Your subnet ID
  
  # OPTIONAL: SSH access (recommended for troubleshooting)
  sshKeys:
  - key-12345678                      # Your SSH key ID
  
  # Bootstrap mode - fully automatic
  # No userData required - Karpenter handles everything!
EOF
```

### Step 4: Create NodePool for Auto-Scaling

```bash
cat <<EOF | kubectl apply -f -
apiVersion: karpenter.sh/v1
kind: NodePool
metadata:
  name: default-nodepool
  annotations:
    karpenter.sh/description: "General purpose NodePool"
spec:
  # Template for nodes created by this pool
  template:
    metadata:
      labels:
        provisioner: karpenter
        environment: getting-started
    spec:
      # Reference to our NodeClass
      nodeClassRef:
        apiVersion: karpenter.ibm.sh/v1alpha1
        kind: IBMNodeClass
        name: default-nodeclass
      
      # Instance requirements
      requirements:
      - key: node.kubernetes.io/instance-type
        operator: In
        values: ["bx2-2x8", "bx2-4x16", "cx2-2x4", "cx2-4x8"]
      - key: kubernetes.io/arch
        operator: In
        values: ["amd64"]
      - key: karpenter.sh/capacity-type
        operator: In
        values: ["on-demand"]
  
  # Resource limits (adjust based on your needs)
  limits:
    cpu: 1000    # Max 1000 vCPUs across all nodes in this pool
    memory: 1000Gi
  
  # Disruption settings for cost optimization
  disruption:
    consolidationPolicy: WhenEmpty
    consolidateAfter: 30s
    # Remove nodes when they're empty for 30 seconds
EOF
```

### Step 5: Verify Configuration

```bash
# Check if NodeClass is ready
kubectl get ibmnodeclasses
kubectl describe ibmnodeclass default-nodeclass

# Check if NodePool is ready  
kubectl get nodepools
kubectl describe nodepool default-nodepool

# Expected STATUS: Ready for both resources
```

### Step 6: Test Auto-Scaling

Deploy a workload that requires node provisioning:

```bash
cat <<EOF | kubectl apply -f -
apiVersion: apps/v1
kind: Deployment
metadata:
  name: auto-scaling-test
  labels:
    app: scaling-test
spec:
  replicas: 3
  selector:
    matchLabels:
      app: scaling-test
  template:
    metadata:
      labels:
        app: scaling-test
    spec:
      containers:
      - name: resource-consumer
        image: busybox:1.35
        command: ["sleep", "3600"]
        resources:
          requests:
            cpu: "2"      # Request 2 vCPUs per pod
            memory: 4Gi   # Request 4GB RAM per pod
          limits:
            cpu: "2"
            memory: 4Gi
      # Ensure pods spread across different nodes
      affinity:
        podAntiAffinity:
          preferredDuringSchedulingIgnoredDuringExecution:
          - weight: 100
            podAffinityTerm:
              labelSelector:
                matchLabels:
                  app: scaling-test
              topologyKey: kubernetes.io/hostname
EOF
```

### Step 7: Monitor Auto-Scaling

Watch Karpenter provision nodes automatically:

```bash
# Watch NodeClaims being created
kubectl get nodeclaims -w

# Watch nodes being added to cluster
kubectl get nodes -w

# Check pod scheduling
kubectl get pods -o wide

# Monitor Karpenter controller logs
kubectl logs -n karpenter deployment/karpenter -f
```

**Expected Behavior:**
1. Pods will initially show as `Pending` (no suitable nodes)
2. Karpenter detects unschedulable pods
3. Creates NodeClaims based on pod requirements
4. Provisions IBM Cloud VPC instances
5. Instances join cluster automatically (no manual setup!)
6. Pods get scheduled on new nodes

**Typical Timeline:**
- NodeClaim creation: ~10 seconds
- IBM Cloud instance provisioning: 2-3 minutes
- Bootstrap and cluster join: 1-2 minutes
- **Total**: 3-5 minutes for new nodes to be ready
