Billing & Payments Runbook

[!WARNING] The prices, allowances and rates in this document are NOT current and must not be quoted to a customer. The single source of truth for tier pricing is lib/types/pricing.ts; the public shelf is app/(public)/pricing/page.tsx.

A 2026-09-05 sweep found eight mutually incompatible tier tables across this repo, seven of them in documents marked "Production Ready", "Complete", or carrying no status line at all. Numbers here were correct for some earlier draft of the product and were never revised. They are left in place because the surrounding architecture and flow descriptions are still accurate and useful — only the figures are stale.

If you need a real number, read lib/types/pricing.ts. Do not copy one from here.

This runbook covers operational procedures for billing and payment processing including Stripe integration, subscription management, credit wallet system, and webhook handling.

Owner: Platform Team Version: 1.0.0 Last Updated: 2025-12-27 Related Docs: PLATFORM_INTEGRATIONS.md

---

Table of Contents

1. Prerequisites
2. Health Check Commands
3. Common Failure Modes
4. Verification Commands
5. Rollback Steps
6. Escalation

---

Prerequisites

Required Services

Service	Purpose	Health Check
PostgreSQL (Supabase)	Subscription data	pnpm db:health
Stripe API	Payment processing	Stripe Dashboard
Redis (Upstash)	Webhook idempotency	Upstash Dashboard
Stripe Connect	Marketplace payments	Stripe Connect Dashboard

Required Environment Variables

# Database
DATABASE_URL="postgresql://..."

# Stripe Platform
STRIPE_SECRET_KEY="sk_live_..."
STRIPE_PUBLISHABLE_KEY="pk_live_..."
STRIPE_WEBHOOK_SECRET="whsec_..."

# Stripe Connect (for marketplace)
STRIPE_CONNECT_WEBHOOK_SECRET="whsec_..."

# Pricing Configuration
STRIPE_STARTER_PRICE_ID="price_..."
STRIPE_GROWTH_PRICE_ID="price_..."
STRIPE_SCALE_PRICE_ID="price_..."
STRIPE_ENTERPRISE_PRICE_ID="price_..."

Key Files

Component	File Path
Credit Wallet Service	lib/services/billing/credit-wallet-service.ts
Stripe Webhook Handlers	lib/services/billing/stripe-webhook-handlers.ts
Stripe Connect Handlers	lib/services/billing/stripe-connect-webhook-handlers.ts
Processing Fee Service	lib/services/billing/processing-fee-service.ts
Processing Fee Reports	lib/services/billing/processing-fee-reports.ts
Webhook Retry Service	lib/services/billing/webhook-retry-service.ts
Billing Middleware	lib/middleware/billing/
Platform Webhook Route	app/api/webhooks/stripe/route.ts
Connect Webhook Route	app/api/webhooks/stripe/connect/route.ts

Pricing Tiers

Tier	Features	Monthly Price
Starter	Basic features	$29/mo
Growth	+ Advanced features	$99/mo
Scale	+ Priority support	$299/mo
Enterprise	Custom	Contact sales

---

Health Check Commands

Quick Health Check

# Check Stripe webhook endpoint
curl https://your-domain.com/api/webhooks/stripe
# Expected: {"status":"ok","webhook":"stripe"}

# Check Stripe Connect webhook endpoint
curl https://your-domain.com/api/webhooks/stripe/connect
# Expected: {"status":"ok","webhook":"stripe-connect"}

Stripe API Health Check

# Check Stripe API connectivity
curl https://api.stripe.com/v1/balance \
  -u "$STRIPE_SECRET_KEY:"

# List recent events
curl https://api.stripe.com/v1/events?limit=5 \
  -u "$STRIPE_SECRET_KEY:"

# Check webhook endpoint status
curl https://api.stripe.com/v1/webhook_endpoints \
  -u "$STRIPE_SECRET_KEY:"

Database Health

# Check subscription status distribution
psql $DATABASE_URL -c "
SELECT
  subscription_status,
  COUNT(*) as count,
  ROUND(COUNT(*)::numeric / SUM(COUNT(*)) OVER() * 100, 2) as percentage
FROM \"Portal\"
WHERE subscription_status IS NOT NULL
GROUP BY subscription_status
ORDER BY count DESC;
"

# Check MRR by tier
psql $DATABASE_URL -c "
SELECT
  subscription_tier,
  COUNT(*) as subscriptions,
  SUM(
    CASE subscription_tier
      WHEN 'starter' THEN 29
      WHEN 'growth' THEN 99
      WHEN 'scale' THEN 299
      ELSE 0
    END
  ) as mrr
FROM \"Portal\"
WHERE subscription_status = 'active'
GROUP BY subscription_tier
ORDER BY mrr DESC;
"

# Check credit wallet balances
psql $DATABASE_URL -c "
SELECT
  COUNT(*) as total_wallets,
  SUM(balance) as total_balance,
  AVG(balance) as avg_balance,
  MIN(balance) as min_balance,
  MAX(balance) as max_balance
FROM \"CreditWallet\";
"

# Check recent payment failures
psql $DATABASE_URL -c "
SELECT
  portal_id,
  subscription_tier,
  last_payment_error,
  payment_failed_at
FROM \"Portal\"
WHERE payment_failed_at > NOW() - INTERVAL '7 days'
ORDER BY payment_failed_at DESC
LIMIT 20;
"

Webhook Processing Health

# Check webhook processing stats
psql $DATABASE_URL -c "
SELECT
  event_type,
  COUNT(*) as total,
  COUNT(CASE WHEN status = 'processed' THEN 1 END) as processed,
  COUNT(CASE WHEN status = 'failed' THEN 1 END) as failed,
  MAX(processed_at) as last_processed
FROM \"WebhookEvent\"
WHERE created_at > NOW() - INTERVAL '24 hours'
GROUP BY event_type
ORDER BY total DESC;
"

# Check pending webhook retries
psql $DATABASE_URL -c "
SELECT
  event_type,
  retry_count,
  next_retry_at,
  last_error
FROM \"WebhookEvent\"
WHERE status = 'pending_retry'
ORDER BY next_retry_at ASC
LIMIT 20;
"

---

Common Failure Modes

1. Payment Failures

Symptom	Likely Cause	Resolution
All payments failing	Stripe API key invalid	Verify API keys
Card declined	Insufficient funds	Contact customer
3DS failures	Authentication required	Enable 3DS flow
Currency mismatch	Wrong currency code	Check currency settings

Diagnostic Commands:

# Check recent payment intents
curl "https://api.stripe.com/v1/payment_intents?limit=10" \
  -u "$STRIPE_SECRET_KEY:"

# Check failed charges
curl "https://api.stripe.com/v1/charges?limit=10&status=failed" \
  -u "$STRIPE_SECRET_KEY:"

# Check payment failure logs
grep "payment.*fail\|PaymentError" /var/log/petunia/app.log | tail -20

2. Subscription Issues

Symptom	Likely Cause	Resolution
Subscription not created	Checkout incomplete	Check session status
Wrong tier assigned	Price ID mismatch	Verify price mapping
Subscription stuck	Webhook not processed	Check webhook logs
Cancel not working	Subscription ID wrong	Verify subscription ID

Diagnostic Commands:

# Check subscription in Stripe
curl "https://api.stripe.com/v1/subscriptions/$SUBSCRIPTION_ID" \
  -u "$STRIPE_SECRET_KEY:"

# Check customer subscriptions
curl "https://api.stripe.com/v1/subscriptions?customer=$CUSTOMER_ID" \
  -u "$STRIPE_SECRET_KEY:"

# Check subscription sync
psql $DATABASE_URL -c "
SELECT
  id,
  stripe_customer_id,
  stripe_subscription_id,
  subscription_status,
  subscription_tier,
  updated_at
FROM \"Portal\"
WHERE stripe_subscription_id = 'sub_xxx';
"

3. Webhook Failures

Symptom	Likely Cause	Resolution
Webhooks not received	Endpoint not configured	Configure in Stripe
Signature invalid	Wrong webhook secret	Check STRIPE_WEBHOOK_SECRET
Events missing	Endpoint disabled	Re-enable in Stripe
Duplicate processing	Idempotency failed	Check idempotency keys

Diagnostic Commands:

# Check recent webhook events in Stripe
curl "https://api.stripe.com/v1/events?limit=20" \
  -u "$STRIPE_SECRET_KEY:"

# Check webhook logs
grep "stripe.*webhook\|WebhookError" /var/log/petunia/app.log | tail -50

# Check failed webhook deliveries in Stripe Dashboard
# https://dashboard.stripe.com/webhooks

4. Credit Wallet Issues

Symptom	Likely Cause	Resolution
Balance not updating	Transaction failed	Check transaction logs
Negative balance	Overdraw allowed	Verify balance check
Credits expired	Expiration not processed	Run expiration job
Wrong credit amount	Calculation error	Verify credit rules

Diagnostic Commands:

# Check wallet transactions
psql $DATABASE_URL -c "
SELECT
  id,
  wallet_id,
  amount,
  type,
  description,
  created_at
FROM \"CreditTransaction\"
WHERE wallet_id = 'wallet-id'
ORDER BY created_at DESC
LIMIT 20;
"

# Check wallet balance history
psql $DATABASE_URL -c "
SELECT
  DATE(created_at) as date,
  SUM(CASE WHEN amount > 0 THEN amount ELSE 0 END) as credits_added,
  SUM(CASE WHEN amount < 0 THEN ABS(amount) ELSE 0 END) as credits_used,
  SUM(amount) as net_change
FROM \"CreditTransaction\"
WHERE wallet_id = 'wallet-id'
AND created_at > NOW() - INTERVAL '30 days'
GROUP BY DATE(created_at)
ORDER BY date DESC;
"

---

Verification Commands

Test Stripe Integration

import Stripe from 'stripe';

const stripe = new Stripe(process.env.STRIPE_SECRET_KEY!);

// Check balance
const balance = await stripe.balance.retrieve();
console.log('Balance:', balance);

// Create test customer
const customer = await stripe.customers.create({
  email: 'test@example.com',
  name: 'Test Customer',
});
console.log('Customer:', customer.id);

// Create checkout session
const session = await stripe.checkout.sessions.create({
  customer: customer.id,
  mode: 'subscription',
  line_items: [{
    price: process.env.STRIPE_GROWTH_PRICE_ID,
    quantity: 1,
  }],
  success_url: 'https://your-domain.com/success',
  cancel_url: 'https://your-domain.com/cancel',
});
console.log('Checkout URL:', session.url);

Test Credit Wallet Service

import { CreditWalletService } from '@/lib/services/billing/credit-wallet-service';

const walletService = new CreditWalletService();

// Get wallet balance
const balance = await walletService.getBalance('portal-id');
console.log('Balance:', balance);

// Add credits
await walletService.addCredits('portal-id', 100, 'manual_adjustment');

// Use credits
const success = await walletService.useCredits('portal-id', 10, 'voice_call');
console.log('Credits used:', success);

Test Webhook Processing

# Trigger test webhook event
curl -X POST https://your-domain.com/api/webhooks/stripe \
  -H "Content-Type: application/json" \
  -H "Stripe-Signature: test" \
  -d '{
    "type": "customer.subscription.created",
    "data": {"object": {"id": "sub_test"}}
  }'

# Note: In production, use Stripe CLI for testing
stripe trigger checkout.session.completed

Run Billing Tests

# Run billing tests
pnpm test -- --testPathPattern="billing"

# Run webhook tests
pnpm test -- --testPathPattern="webhooks/stripe"

# Run credit wallet tests
pnpm test -- --testPathPattern="credit-wallet"

---

Rollback Steps

Disable Billing Features

# Disable new subscriptions
curl -X POST https://your-domain.com/api/admin/features \
  -H "Authorization: Bearer $ADMIN_TOKEN" \
  -d '{"feature": "billing.subscriptions.enabled", "enabled": false}'

# Disable credit usage
curl -X POST https://your-domain.com/api/admin/features \
  -H "Authorization: Bearer $ADMIN_TOKEN" \
  -d '{"feature": "billing.credits.enabled", "enabled": false}'

Pause Webhook Processing

# Disable webhook processing (emergency)
# In Stripe Dashboard: Webhooks > Endpoint > Disable

# Or via API
curl -X POST "https://api.stripe.com/v1/webhook_endpoints/$ENDPOINT_ID" \
  -u "$STRIPE_SECRET_KEY:" \
  -d "disabled=true"

Manually Sync Subscription

# Force sync subscription from Stripe
psql $DATABASE_URL -c "
UPDATE \"Portal\"
SET
  subscription_status = 'active',
  subscription_tier = 'growth',
  updated_at = NOW()
WHERE stripe_subscription_id = 'sub_xxx';
"

Refund Payment

# Create refund via Stripe API
curl -X POST "https://api.stripe.com/v1/refunds" \
  -u "$STRIPE_SECRET_KEY:" \
  -d "charge=ch_xxx" \
  -d "reason=requested_by_customer"

Rollback Deployment

# If billing changes caused issues
vercel list --app petunia

# Rollback
vercel rollback <previous-deployment-url>

# Verify webhook is functional
curl https://your-domain.com/api/webhooks/stripe

---

Escalation

When to Escalate

Severity	Criteria	Action
P1 - Critical	All payments failing	Immediate on-call page
P2 - High	Subscription sync broken	Escalate within 30 minutes
P3 - Medium	Credit balance issues	Escalate within 2 hours
P4 - Low	Reporting discrepancies	Normal ticket queue

Escalation Contacts

Role	Contact	Availability
On-Call Engineer	PagerDuty	24/7
Finance Team	Slack #finance	Business hours
Stripe Support	support.stripe.com	24/7

Information to Include

When escalating, include:

1. Customer ID: Stripe customer ID
2. Subscription ID: Stripe subscription ID
3. Error Messages: Specific Stripe error codes
4. Payment Amount: Transaction amount
5. Recent Changes: Deployments, config changes

---

Monitoring & Alerts

Key Metrics to Watch

Metric	Healthy Range	Alert Threshold
Payment success rate	> 95%	< 90%
Webhook processing rate	> 99%	< 95%
MRR growth	Positive	Sudden drop > 10%
Churn rate	< 5%/month	> 8%/month
Credit usage rate	Stable	Sudden spike > 200%

Alert Configuration

• Sentry: Payment and webhook errors
• Slack #billing-alerts: Payment failures, subscription changes
• PagerDuty: P1/P2 incidents

Billing-Specific Dashboards

• Stripe Dashboard: https://dashboard.stripe.com
• Stripe Billing: https://dashboard.stripe.com/billing
• Stripe Webhooks: https://dashboard.stripe.com/webhooks

---

Webhook Event Handling

Platform Webhook Events

Event	Handler	Action
checkout.session.completed	handleCheckoutComplete	Create subscription
customer.subscription.created	handleSubscriptionChange	Sync subscription
customer.subscription.updated	handleSubscriptionChange	Update tier/status
customer.subscription.deleted	handleSubscriptionCanceled	Mark as canceled
invoice.payment_succeeded	handlePaymentSucceeded	Record payment
invoice.payment_failed	handlePaymentFailed	Send failure email
invoice.created	handleInvoiceCreated	Track invoice
customer.created	handleCustomerCreated	Link customer

Connect Webhook Events

Event	Handler	Action
account.updated	handleAccountUpdated	Update Connect status
payment_intent.succeeded	handlePaymentSucceeded	Record marketplace payment
transfer.created	handleTransferCreated	Track payout

Webhook Retry Strategy

// Exponential backoff for retries
const RETRY_DELAYS = [
  5 * 60 * 1000,    // 5 minutes
  30 * 60 * 1000,   // 30 minutes
  2 * 60 * 60 * 1000, // 2 hours
  24 * 60 * 60 * 1000, // 24 hours
];

// Maximum retries before marking as failed
const MAX_RETRIES = 4;

---

Credit System

Credit Types

Type	Description	Expiration
subscription	Monthly allocation	End of billing period
bonus	Promotional credits	90 days
purchased	Bought credits	Never
rollover	Unused from previous	30 days

Credit Usage

Action	Credits
Voice call (per minute)	1 credit
SMS message	0.5 credits
AI response	0.25 credits
WhatsApp message	0.5 credits

---

Last verified: 2025-12-27