TEST Stage
The TEST stage validates functionality through automated and manual testing.
Purpose
The Tester agent:
- Creates preview environment
- Runs automated tests
- Executes QA steps
- Takes screenshots (if Playwright enabled)
- Fixes bugs with atomic commits
Process
┌─────────────────────────────────────────────────────────────┐
│ TEST STAGE │
├─────────────────────────────────────────────────────────────┤
│ │
│ 1. Create Preview Environment │
│ ├─ Local: docker-compose up │
│ └─ Cloud: EC2/DigitalOcean instance │
│ │
│ 2. Run Automated Tests │
│ ├─ pytest │
│ └─ Integration tests │
│ │
│ 3. Execute QA Steps │
│ ├─ From issue acceptance criteria │
│ └─ Generated from description │
│ │
│ 4. Browser Testing (if Playwright enabled) │
│ ├─ Navigate to test URLs │
│ ├─ Execute user flows │
│ └─ Capture screenshots │
│ │
│ 5. Fix Bugs (Atomic Commits) │
│ ├─ Find bug, screenshot │
│ ├─ Fix with minimal change │
│ ├─ Commit atomically │
│ └─ Verify fix │
│ │
│ 6. Create Test Report │
│ └─ Post to Linear │
│ │
│ 7. Signal Result │
│ ├─ test-passed │
│ └─ test-failed │
│ │
└─────────────────────────────────────────────────────────────┘
Preview Environments
Local (Docker Compose)
# Configure Docker Compose file
goblin pipeline config -p PROJECT --docker-compose docker-compose.yml
Goblin runs:
docker-compose up -d
With port isolation (multiple previews don't conflict).
Cloud (EC2/DigitalOcean)
# Configure cloud provider
goblin pipeline config -p PROJECT --config cloud_provider=ec2
Goblin:
- Launches EC2 instance
- Deploys code via SSH/SCP
- Runs docker-compose
- Returns public IP
QA Steps
Generated from issue description:
class QAStep:
id: int
description: str
category: str # functional, visual, data, error_handling
priority: str # critical, high, medium, low
verification: str
Example QA Steps
## QA Steps for ENG-123
1. **[CRITICAL] Login Flow**
- Navigate to /login
- Enter valid credentials
- Verify redirect to dashboard
- Verification: User sees welcome message
2. **[HIGH] Error Handling**
- Enter invalid credentials
- Verify error message displayed
- Verification: "Invalid credentials" shown
3. **[MEDIUM] Visual Consistency**
- Check login button styling
- Verification: Matches design system
Playwright Testing
When enabled, the Tester uses browser automation:
# Enable Playwright
goblin pipeline config -p PROJECT --playwright
# Set test URLs
goblin pipeline config -p PROJECT --test-urls "http://localhost:3000"
Screenshot Workflow
.goblin/screenshots/{ISSUE_ID}/
├── 01-initial-state.png
├── 02-form-filled.png
├── 03-after-submit.png
└── 04-error-state.png
Screenshot Code
from playwright.sync_api import sync_playwright
with sync_playwright() as p:
browser = p.chromium.launch()
page = browser.new_page()
page.goto("http://localhost:3000")
page.screenshot(path=".goblin/screenshots/ENG-123/01-initial.png")
page.fill("#email", "[email protected]")
page.screenshot(path=".goblin/screenshots/ENG-123/02-filled.png")
page.click("#submit")
page.screenshot(path=".goblin/screenshots/ENG-123/03-result.png")
Atomic Bug Fix Commits
When bugs are found:
# One commit per bug
git commit -m "fix(qa): [FINDING-001] Make submit button clickable
The button had pointer-events: none which prevented clicks.
Removed the CSS property.
Before: .goblin/screenshots/ENG-123/error-button.png
After: .goblin/screenshots/ENG-123/fixed-button.png
Related QA step: QA-3"
Benefits
- Bisectable history
- Easy to revert individual fixes
- Visual proof with screenshots
- Clear what each commit fixes
Test Report
Posted to Linear:
## QA Test Results: ENG-123
### Summary
- Total QA Steps: 5
- Passed: 4
- Failed: 1 → Fixed
- Bugs Found: 2
- Bugs Fixed: 2
### QA Step Results
**QA-1: Login Flow** ✅ PASS
- Screenshot: 01-login-success.png
**QA-2: Error Handling** ✅ PASS → FIXED
- Finding: FINDING-001
- Fix: Commit abc123
### Bugs Fixed
1. **FINDING-001**: Submit button not clickable
- Commit: abc123
- Files: src/components/Button.tsx
### Visual Evidence
(Screenshots uploaded to Linear)
### Recommendation
**APPROVED** - All tests passing, bugs fixed
Signal Files
| Outcome | Signal File |
|---|---|
| Passed | .goblin/test-passed |
| Failed | .goblin/test-failed |
test-failed Content
{
"failing_steps": ["QA-2", "QA-5"],
"reason": "Error handling not working",
"attempts": 1
}
Configuration
# Enable Playwright
goblin pipeline config -p PROJECT --playwright
# Set test URLs
goblin pipeline config -p PROJECT --test-urls "http://localhost:3000,http://localhost:8080"
# Cloud preview
goblin pipeline config -p PROJECT --config cloud_provider=ec2
Next Steps
- DONE Stage - Completion
- Preview Environments - Environment details
- QA Steps - QA step generation