Cloud Preview Environments

Goblin spins up real AWS EC2 instances (or DigitalOcean droplets) for each pipeline, providing isolated, production-like testing environments.

Overview

When a pipeline reaches the TEST stage, Goblin can automatically:

Launch an EC2 instance with Docker pre-installed
Deploy your code via git clone or rsync
Run docker compose to start services
Provide public URLs for testing
Auto-terminate when done (configurable TTL)

Providers

Provider	Description	Best For
EC2	AWS instances	Production-like, AWS integration
DigitalOcean	DO Droplets	Cost-effective, simple setup
Local	Docker Compose	Development, CI

EC2 Configuration

# Enable EC2 previews
goblin pipeline config -p PROJECT --config cloud_provider=ec2

# AWS settings
goblin pipeline config -p PROJECT --config aws_region=us-east-1
goblin pipeline config -p PROJECT --config ec2_instance_type=t3.small
goblin pipeline config -p PROJECT --config ec2_root_volume_size=30

# Security (required)
goblin pipeline config -p PROJECT --config ec2_key_name=your-key-pair
goblin pipeline config -p PROJECT --config ec2_security_group=sg-xxxxx
goblin pipeline config -p PROJECT --config ssh_key_path=~/.ssh/your-key.pem

# Optional
goblin pipeline config -p PROJECT --config ec2_subnet_id=subnet-xxxxx
goblin pipeline config -p PROJECT --config ec2_ami_id=ami-xxxxx  # Custom AMI

Required AWS Permissions

{
  "Effect": "Allow",
  "Action": [
    "ec2:RunInstances",
    "ec2:TerminateInstances",
    "ec2:DescribeInstances",
    "ec2:CreateTags"
  ],
  "Resource": "*"
}

Security Group Requirements

Open these ports:

22 - SSH access
80, 443 - HTTP/HTTPS
3000-9000 - Common app ports
Custom ports as needed

Instance Provisioning

When Goblin creates an instance, it runs this user-data script:

# Installs automatically:
- Node.js 20.x (for Claude Code CLI)
- Claude Code CLI (@anthropic-ai/claude-code)
- Docker + Docker Compose

You can use a custom AMI with these pre-installed for faster startup.

Code Deployment

Two deployment modes:

Rsync Mode (Default)

Copies code from worktree to instance:

goblin pipeline config -p PROJECT --config deployment_mode=rsync

Copies from local worktree
Respects .gitignore and .dockerignore
Excludes .git, node_modules, __pycache__, .venv

Git Clone Mode

Clones directly from GitHub (faster for slow connections):

goblin pipeline config -p PROJECT --config deployment_mode=git
goblin pipeline config -p PROJECT --config git_repo_url=https://github.com/org/repo.git

Uses gh auth token for private repos
Shallow clone (--depth 1)
Clones specific branch

Include Files

Copy files that are in .gitignore but needed for preview (like .env):

goblin pipeline config -p PROJECT --config preview_include_files="config/dev/environment.env,.env.local"

Environment Overrides

Override environment variables with templating:

goblin pipeline config -p PROJECT --config preview_env_overrides="
BACKEND_URL=http://{PUBLIC_IP}:8000
FRONTEND_URL=http://{PUBLIC_IP}:3000
ISSUE_ID={ISSUE_ID}
"

# Path to env file on instance
goblin pipeline config -p PROJECT --config preview_env_file_path=config/dev/environment.env

Templates:

{PUBLIC_IP} - Instance public IP
{ISSUE_ID} - Linear issue ID

Service Selection

Start only specific services (faster startup):

goblin pipeline config -p PROJECT --config preview_services="web,api,db"

Leave empty to start all services.

Preview Lifecycle

Pipeline TEST Start
       │
       ▼
┌────────────────────────┐
│  1. Create EC2         │  t3.small, Ubuntu 22.04
│     Wait for SSH       │  ~2-3 minutes
└───────────┬────────────┘
            │
            ▼
┌────────────────────────┐
│  2. Wait for Docker    │  User-data script
│     (up to 5 min)      │
└───────────┬────────────┘
            │
            ▼
┌────────────────────────┐
│  3. Deploy Code        │  rsync or git clone
│     Copy include files │
│     Apply env overrides│
└───────────┬────────────┘
            │
            ▼
┌────────────────────────┐
│  4. Docker Compose     │  build + up -d
│     (up to 60 min)     │
└───────────┬────────────┘
            │
            ▼
┌────────────────────────┐
│  5. Run Setup Commands │  DB reset, fixtures
└───────────┬────────────┘
            │
            ▼
┌────────────────────────┐
│  6. Discover URLs      │  Parse docker ps ports
│     Save to DB         │
│     Post to Linear     │
└───────────┬────────────┘
            │
            ▼
       Preview Ready!
       http://{IP}:8000

Linear Notification

When preview is ready, Goblin posts to the ticket:

## 🌐 Preview Environment Ready

**Preview URL**: http://3.16.148.197:8000

| Service | Direct URL |
|---------|------------|
| web | http://3.16.148.197:3000 |
| api | http://3.16.148.197:8000 |

**Provider**: EC2
**Auto-expires**: 2 hours after creation

To stop now: `goblin preview stop CF-123`
To SSH: `goblin preview ssh CF-123`

TTL and Auto-Expiration

Previews auto-terminate to save costs:

# Configure TTL (default: 120 minutes)
goblin pipeline config -p PROJECT --config preview_ttl_minutes=120

# Keep on test failure for debugging
goblin pipeline config -p PROJECT --config preview_keep_on_failure=true

Extend TTL manually:

goblin preview extend ENG-123 --hours 4

CLI Commands

# List all previews
goblin preview list
goblin preview list --all  # Include stopped

# Detailed info
goblin preview info ENG-123

# Get URLs
goblin preview urls ENG-123

# SSH into instance (cloud only)
goblin preview ssh ENG-123

# View logs
goblin preview logs ENG-123 -f

# Stop and destroy
goblin preview stop ENG-123

# Extend TTL
goblin preview extend ENG-123 --hours 2

DigitalOcean Configuration

goblin pipeline config -p PROJECT --config cloud_provider=digitalocean
goblin pipeline config -p PROJECT --config do_token=dop_v1_xxxxx
goblin pipeline config -p PROJECT --config do_region=nyc1
goblin pipeline config -p PROJECT --config do_size=s-1vcpu-1gb
goblin pipeline config -p PROJECT --config do_ssh_key_ids=12345,67890
goblin pipeline config -p PROJECT --config ssh_key_path=~/.ssh/id_rsa

Cost Estimates

Provider	Instance	Cost/Hour	30min Preview
EC2	t3.micro	~$0.01	~$0.005
EC2	t3.small	~$0.02	~$0.01
EC2	t3.medium	~$0.04	~$0.02
DO	s-1vcpu-1gb	~$0.007	~$0.004
DO	s-2vcpu-2gb	~$0.018	~$0.009

Example: 20 pipelines/day × 30min × t3.small ≈ $6/month

Troubleshooting

Instance Won't Start

# Check AWS console for errors
aws ec2 describe-instances --instance-ids i-xxxxx

# Check user-data logs
goblin preview ssh ENG-123
cat /var/log/cloud-init-output.log

Docker Not Ready

# SSH and check Docker status
goblin preview ssh ENG-123
sudo systemctl status docker
docker --version

Code Deployment Failed

# Check rsync/git errors in Goblin logs
goblin remote logs -f

# SSH and verify /app contents
goblin preview ssh ENG-123
ls -la /app

Local Preview - Docker Compose previews
EC2 Deployment - Deploy Goblin daemon
TEST Stage - Pipeline testing

Overview​

Providers​

EC2 Configuration​

Required AWS Permissions​

Security Group Requirements​

Instance Provisioning​

Code Deployment​

Rsync Mode (Default)​

Git Clone Mode​

Include Files​

Environment Overrides​

Service Selection​

Preview Lifecycle​

Linear Notification​

TTL and Auto-Expiration​

CLI Commands​

DigitalOcean Configuration​

Cost Estimates​

Troubleshooting​

Instance Won't Start​

Docker Not Ready​

Code Deployment Failed​

Related​