CI/CD Pipeline Prompt Templates

## CI/CD Pipeline Design

### Pipeline Overview

```
Push to branch → Build → Test → Security Scan
 ↓
Push to main → Build → Test → Security Scan → Deploy Staging
 ↓
Manual approval → Deploy Production
```

### GitHub Actions Configuration

```yaml
#.github/workflows/ci-cd.yml
name: CI/CD Pipeline

on:
 push:
 branches: [main, 'feature/**']
 pull_request:
 branches: [main]

env:
 NODE_VERSION: '20'
 AWS_REGION: us-east-1
 ECR_REPOSITORY: my-api
 ECS_CLUSTER: my-cluster
 ECS_SERVICE_STAGING: api-staging
 ECS_SERVICE_PROD: api-production

jobs:
 # ==================== BUILD ====================
 build:
 runs-on: ubuntu-latest
 steps:
 - uses: actions/checkout@v4

 - name: Setup Node.js
 uses: actions/setup-node@v4
 with:
 node-version: ${{ env.NODE_VERSION }}
 cache: 'npm'

 - name: Install dependencies
 run: npm ci

 - name: Build
 run: npm run build

 - name: Upload build artifacts
 uses: actions/upload-artifact@v4
 with:
 name: dist
 path: dist/
 retention-days: 1

 # ==================== TEST ====================
 test-unit:
 runs-on: ubuntu-latest
 needs: build
 steps:
 - uses: actions/checkout@v4
 - uses: actions/setup-node@v4
 with:
 node-version: ${{ env.NODE_VERSION }}
 cache: 'npm'
 - run: npm ci
 - name: Run unit tests
 run: npm run test:unit -- --coverage
 - name: Upload coverage
 uses: codecov/codecov-action@v3
 with:
 files: coverage/lcov.info

 test-integration:
 runs-on: ubuntu-latest
 needs: build
 services:
 postgres:
 image: postgres:15
 env:
 POSTGRES_PASSWORD: postgres
 POSTGRES_DB: test
 ports:
 - 5432:5432
 options: >-
 --health-cmd pg_isready
 --health-interval 10s
 --health-timeout 5s
 --health-retries 5
 steps:
 - uses: actions/checkout@v4
 - uses: actions/setup-node@v4
 with:
 node-version: ${{ env.NODE_VERSION }}
 cache: 'npm'
 - run: npm ci
 - name: Run database migrations
 run: npm run db:migrate
 env:
 DATABASE_URL: postgres://postgres:postgres@localhost:5432/test
 - name: Run integration tests
 run: npm run test:integration
 env:
 DATABASE_URL: postgres://postgres:postgres@localhost:5432/test

 # ==================== SECURITY ====================
 security:
 runs-on: ubuntu-latest
 needs: build
 steps:
 - uses: actions/checkout@v4
 - name: Run npm audit
 run: npm audit --audit-level=high
 - name: Run Snyk security scan
 uses: snyk/actions/node@master
 continue-on-error: true # Don't block on warnings
 env:
 SNYK_TOKEN: ${{ secrets.SNYK_TOKEN }}

 # ==================== DOCKER BUILD ====================
 docker:
 runs-on: ubuntu-latest
 needs: [test-unit, test-integration, security]
 if: github.ref == 'refs/heads/main'
 outputs:
 image-tag: ${{ steps.meta.outputs.tags }}
 steps:
 - uses: actions/checkout@v4

 - name: Configure AWS credentials
 uses: aws-actions/configure-aws-credentials@v4
 with:
 aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
 aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
 aws-region: ${{ env.AWS_REGION }}

 - name: Login to Amazon ECR
 id: login-ecr
 uses: aws-actions/amazon-ecr-login@v2

 - name: Build and push Docker image
 id: meta
 env:
 ECR_REGISTRY: ${{ steps.login-ecr.outputs.registry }}
 IMAGE_TAG: ${{ github.sha }}
 run: |
 docker build -t $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG.
 docker push $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG
 echo "tags=$ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG" >> $GITHUB_OUTPUT

 # ==================== DEPLOY STAGING ====================
 deploy-staging:
 runs-on: ubuntu-latest
 needs: docker
 environment: staging
 steps:
 - name: Configure AWS credentials
 uses: aws-actions/configure-aws-credentials@v4
 with:
 aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
 aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
 aws-region: ${{ env.AWS_REGION }}

 - name: Deploy to ECS Staging
 run: |
 aws ecs update-service \
 --cluster ${{ env.ECS_CLUSTER }} \
 --service ${{ env.ECS_SERVICE_STAGING }} \
 --force-new-deployment

 - name: Wait for deployment
 run: |
 aws ecs wait services-stable \
 --cluster ${{ env.ECS_CLUSTER }} \
 --services ${{ env.ECS_SERVICE_STAGING }}

 # ==================== DEPLOY PRODUCTION ====================
 deploy-production:
 runs-on: ubuntu-latest
 needs: deploy-staging
 environment: production # Requires manual approval
 steps:
 - name: Configure AWS credentials
 uses: aws-actions/configure-aws-credentials@v4
 with:
 aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
 aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
 aws-region: ${{ env.AWS_REGION }}

 - name: Deploy to ECS Production
 run: |
 aws ecs update-service \
 --cluster ${{ env.ECS_CLUSTER }} \
 --service ${{ env.ECS_SERVICE_PROD }} \
 --force-new-deployment

 - name: Wait for deployment
 run: |
 aws ecs wait services-stable \
 --cluster ${{ env.ECS_CLUSTER }} \
 --services ${{ env.ECS_SERVICE_PROD }}

 - name: Notify Slack
 uses: slackapi/slack-github-action@v1
 with:
 payload: |
 {
 "text": "Deployed to production: ${{ github.sha }}"
 }
 env:
 SLACK_WEBHOOK_URL: ${{ secrets.SLACK_WEBHOOK }}
```

### Secrets Required

| Secret | Purpose |
|--------|--------|
| AWS_ACCESS_KEY_ID | AWS deployment |
| AWS_SECRET_ACCESS_KEY | AWS deployment |
| SNYK_TOKEN | Security scanning |
| SLACK_WEBHOOK | Deployment notifications |

### Environments Setup

In GitHub repo settings → Environments:

1. **staging**: No protection rules (auto-deploy)
2. **production**: Required reviewers (1 approval needed)

## Pipeline Optimization Analysis

### Current Bottlenecks

| Stage | Time | Issue |
|-------|------|-------|
| npm install | 4 min | No caching |
| Docker build | 8 min | No layer caching |
| E2E tests | 10 min | Sequential, single runner |
| **Total** | **25 min** | Sequential execution |

### Improved Pipeline

**Target: 25 min → 10 min**

```yaml
name: Improved CI/CD

on:
 push:
 branches: [main]
 pull_request:

jobs:
 # Cache dependencies once, reuse everywhere
 setup:
 runs-on: ubuntu-latest
 outputs:
 cache-key: ${{ steps.cache.outputs.cache-hit }}
 steps:
 - uses: actions/checkout@v4
 - uses: actions/setup-node@v4
 with:
 node-version: '20'
 cache: 'npm'
 - name: Cache node_modules
 id: cache
 uses: actions/cache@v4
 with:
 path: node_modules
 key: ${{ runner.os }}-node-${{ hashFiles('**/package-lock.json') }}
 - if: steps.cache.outputs.cache-hit != 'true'
 run: npm ci

 # Run tests in parallel
 test-unit:
 needs: setup
 runs-on: ubuntu-latest
 steps:
 - uses: actions/checkout@v4
 - uses: actions/setup-node@v4
 with:
 node-version: '20'
 cache: 'npm'
 - uses: actions/cache@v4
 with:
 path: node_modules
 key: ${{ runner.os }}-node-${{ hashFiles('**/package-lock.json') }}
 - run: npm run test:unit

 test-integration:
 needs: setup
 runs-on: ubuntu-latest
 #... (parallel with unit tests)

 # Split E2E tests across multiple runners
 test-e2e:
 needs: setup
 runs-on: ubuntu-latest
 strategy:
 fail-fast: false
 matrix:
 shard: [1, 2, 3, 4] # 4 parallel runners
 steps:
 - uses: actions/checkout@v4
 - uses: actions/setup-node@v4
 with:
 node-version: '20'
 cache: 'npm'
 - uses: actions/cache@v4
 with:
 path: node_modules
 key: ${{ runner.os }}-node-${{ hashFiles('**/package-lock.json') }}
 - name: Run E2E shard ${{ matrix.shard }}
 run: npm run test:e2e -- --shard=${{ matrix.shard }}/4

 # Docker with BuildKit caching
 docker:
 needs: [test-unit, test-integration, test-e2e]
 runs-on: ubuntu-latest
 steps:
 - uses: actions/checkout@v4
 
 - name: Set up Docker Buildx
 uses: docker/setup-buildx-action@v3

 - name: Login to ECR
 uses: aws-actions/amazon-ecr-login@v2

 - name: Build and push
 uses: docker/build-push-action@v5
 with:
 context:.
 push: true
 tags: ${{ env.ECR_REGISTRY }}/${{ env.REPO }}:${{ github.sha }}
 cache-from: type=gha
 cache-to: type=gha,mode=max
```

### Optimization Summary

| Technique | Before | After | Savings |
|-----------|--------|-------|--------|
| npm cache | 4 min | 30 sec | 3.5 min |
| Parallel tests | 14 min | 4 min | 10 min |
| E2E sharding (4x) | 10 min | 3 min | 7 min |
| Docker BuildKit cache | 8 min | 2 min | 6 min |
| **Total** | **25 min** | **~8 min** | **17 min** |

### Dockerfile Optimization

```dockerfile
# Multi-stage with better layer caching
FROM node:20-alpine AS deps
WORKDIR /app
COPY package*.json./
RUN npm ci --only=production

FROM node:20-alpine AS build
WORKDIR /app
COPY package*.json./
RUN npm ci
COPY..
RUN npm run build

FROM node:20-alpine AS runtime
WORKDIR /app
COPY --from=deps /app/node_modules./node_modules
COPY --from=build /app/dist./dist
CMD ["node", "dist/index.js"]
```

## Deployment Strategy: Blue-Green with Canary

### Why This Strategy

- **Zero downtime**: New version runs alongside old
- **Fast rollback**: Instant switch back to old version
- **Payment safety**: Test with real traffic before full cutover

### Architecture

```
 ┌─────────────────────┐
 │ Ingress/ALB │
 └──────────┬──────────┘
 │
 ┌────────────────┼────────────────┐
 │ │ │
 ▼ ▼ ▼
 ┌──────────┐ ┌──────────┐ ┌──────────┐
 │ Blue │ │ Green │ │ Canary │
 │ (stable) │ │ (new) │ │ (test) │
 │ 90% │ │ 0% │ │ 10% │
 └──────────┘ └──────────┘ └──────────┘
```

### Kubernetes Implementation

```yaml
# deployment-blue.yaml (current stable)
apiVersion: apps/v1
kind: Deployment
metadata:
 name: api-blue
 labels:
 app: api
 version: blue
spec:
 replicas: 5
 selector:
 matchLabels:
 app: api
 version: blue
 template:
 metadata:
 labels:
 app: api
 version: blue
 spec:
 containers:
 - name: api
 image: myregistry/api:v1.2.3 # Current version
 ports:
 - containerPort: 3000
 readinessProbe:
 httpGet:
 path: /health
 port: 3000
 initialDelaySeconds: 5
 periodSeconds: 5
 resources:
 requests:
 memory: "256Mi"
 cpu: "200m"

---
# deployment-green.yaml (new version)
apiVersion: apps/v1
kind: Deployment
metadata:
 name: api-green
 labels:
 app: api
 version: green
spec:
 replicas: 5
 selector:
 matchLabels:
 app: api
 version: green
 template:
 metadata:
 labels:
 app: api
 version: green
 spec:
 containers:
 - name: api
 image: myregistry/api:v1.2.4 # New version
 #... same config as blue

---
# service with traffic split
apiVersion: v1
kind: Service
metadata:
 name: api
spec:
 selector:
 app: api
 # No version selector - both receive traffic
 ports:
 - port: 80
 targetPort: 3000

---
# Istio VirtualService for traffic control
apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
 name: api
spec:
 hosts:
 - api
 http:
 - route:
 - destination:
 host: api
 subset: blue
 weight: 90
 - destination:
 host: api
 subset: green
 weight: 10 # Canary traffic
```

### Deployment Procedure

**Phase 1: Deploy Green (0% traffic)**
```bash
# Deploy new version, not receiving traffic yet
kubectl apply -f deployment-green.yaml

# Wait for all pods ready
kubectl rollout status deployment/api-green

# Verify health
kubectl exec -it deploy/api-green -- curl localhost:3000/health
```

**Phase 2: Canary (10% traffic)**
```bash
# Route 10% of traffic to green
kubectl apply -f virtualservice-canary.yaml

# Monitor for 15 minutes
# - Error rate: Should stay < 0.1%
# - Latency p99: Should stay < 200ms
# - Payment success rate: Should stay > 99.5%
```

**Phase 3: Gradual Rollout**
```bash
# If canary healthy: 10% → 50% → 100%
kubectl patch virtualservice api --type=merge -p '
 {"spec":{"http":[{"route":[
 {"destination":{"host":"api","subset":"blue"},"weight":50},
 {"destination":{"host":"api","subset":"green"},"weight":50}
 ]}]}}'

# Monitor, then...
kubectl patch virtualservice api --type=merge -p '
 {"spec":{"http":[{"route":[
 {"destination":{"host":"api","subset":"green"},"weight":100}
 ]}]}}'
```

**Phase 4: Cleanup**
```bash
# After 24 hours of stability, remove blue
kubectl delete deployment api-blue

# Rename green to blue for next deployment
kubectl label deployment api-green version=blue --overwrite
```

### Rollback Procedure

**Instant rollback (< 30 seconds):**
```bash
# Switch all traffic back to blue
kubectl patch virtualservice api --type=merge -p '
 {"spec":{"http":[{"route":[
 {"destination":{"host":"api","subset":"blue"},"weight":100}
 ]}]}}'
```

### Database Migration Safety

```
1. Migrations must be backward compatible
2. Deploy order:
 a. Run migration (additive only: new columns, tables)
 b. Deploy new code that uses new schema
 c. After stable, clean up old columns (separate deploy)
 
Never: Drop columns, rename columns, or change types in same deploy as code
```

## Pipeline Failure Analysis

### Root Cause

The `EACCES` error is caused by Husky's postinstall script trying to run `git config` and modify `.git/hooks`, which fails because:

1. GitHub Actions runs `npm ci` as root
2. Husky tries to set up git hooks
3. Git hooks setup requires git config access that's restricted

### Error Breakdown

```
npm ERR! code EACCES
npm ERR! syscall mkdir
npm ERR! path /github/workspace/.git/hooks
npm ERR! errno -13
```

Husky's postinstall runs `husky install` which tries to create `.git/hooks` directory with restricted permissions.

### Immediate Fix

**Option 1: Skip Husky in CI (Recommended)**

```yaml
#.github/workflows/ci.yml
steps:
 - uses: actions/checkout@v4
 - uses: actions/setup-node@v4
 with:
 node-version: '20'
 - name: Install dependencies
 run: npm ci
 env:
 HUSKY: 0 # Disables husky postinstall
```

**Option 2: Update package.json**

```json
{
 "scripts": {
 "postinstall": "is-ci || husky install"
 },
 "devDependencies": {
 "is-ci": "^3.0.1"
 }
}
```

**Option 3: Update Husky config (Husky v9+)**

```json
// package.json
{
 "scripts": {
 "prepare": "husky || true"
 }
}
```

The `|| true` ensures the script doesn't fail if husky can't initialize.

### Prevention

1. **Test CI locally** before pushing:
 ```bash
 act push # Run GitHub Actions locally with 'act'
 ```

2. **Add CI check to postinstall scripts**:
 ```json
 {
 "scripts": {
 "postinstall": "node -e \"process.env.CI || require('husky').install()\""
 }
 }
 ```

3. **Document CI-specific behavior**:
 ```markdown
 ## CI Notes
 - Husky is disabled in CI (HUSKY=0)
 - Git hooks run locally only, lint-staged enforced by CI jobs
 ```

### Why This Works Locally But Not in CI

| Environment | User | Git hooks needed? |
|-------------|------|-------------------|
| Local | Your user | Yes (pre-commit checks) |
| CI | root | No (separate lint job) |

Git hooks are for local development workflow. CI should run the same checks via dedicated pipeline jobs, not git hooks.