Skip to content
Autowright

How It Works

How It Works

Fixwright uses a multi-stage fixing loop to analyze failures, propose fixes, and validate them. This page explains each stage in detail.

The Fixing Loop

  1. Receive Failure Case

    A failure case arrives from Stepwright or another source containing:

    • Script and step information
    • Error message and stack trace
    • Artifacts (screenshot, DOM, console logs)
    • Source code location

  2. Create Fix Branch

    Fixwright creates a dedicated Git branch for the fix:

    fix/stepwright-{failure-case-id}

  3. AI Analysis

    Claude AI (via Agent SDK) analyzes the failure:

    • Reads the full source file
    • Examines error context
    • Reviews artifacts (screenshot, DOM)
    • Determines root cause

  4. Apply Fix

    Claude applies code changes using the Edit tool:

    • Modifies selectors
    • Adds waits
    • Updates assertions
    • Makes other targeted changes

  5. Validate Fix

    Fixwright re-runs the script to verify:

    • The failing step now passes
    • No new failures introduced
    • Script completes successfully

  6. Create PR (if validation passes)

    Creates a GitHub PR with:

    • Detailed explanation
    • Diff of changes
    • Validation results

Attempt Lifecycle

Each fix attempt goes through these stages:

┌──────────┐    ┌───────────┐    ┌─────────┐    ┌────────────┐
│  START   │───▶│ ANALYZING │───▶│ FIXING  │───▶│ VALIDATING │
└──────────┘    └───────────┘    └─────────┘    └────────────┘
                                                      │
                     ┌────────────────────────────────┼────────────────────┐
                     │                                │                    │
                     ▼                                ▼                    ▼
              ┌───────────┐                   ┌───────────┐        ┌────────────┐
              │  FAILED   │                   │  SUCCESS  │        │ REGRESSION │
              │ (retry?)  │                   │   (PR!)   │        │  (retry?)  │
              └───────────┘                   └───────────┘        └────────────┘

Stage: Analyzing

Claude reads and understands the failure:

fixwright.on('attempt:analyzing', (attempt) => {
  console.log('Analyzing failure:', attempt.failureCase.failure.stepName);
});

What happens:

  • Claude reads the source file at the failure location
  • Examines the error message and stack trace
  • Reviews screenshot and DOM artifacts
  • Determines the likely cause type

Stage: Fixing

Claude applies code changes:

fixwright.on('attempt:fixing', (attempt) => {
  console.log('Applying fix...');
});


fixwright.on('agent:edit', (editInfo) => {
  console.log(`Editing ${editInfo.filePath}`);
  console.log('Old:', editInfo.oldString);
  console.log('New:', editInfo.newString);
});

Common fixes:

  • Update selectors (changed IDs, classes)
  • Add explicit waits
  • Handle new page flows
  • Update assertions

Stage: Validating

Re-run the script to verify the fix:

fixwright.on('attempt:validating', (attempt) => {
  console.log('Validating fix...');
});

Validation process:

  1. Save changes to disk
  2. Run the Stepwright script
  3. Check if failing step passes
  4. Verify no new failures

Outcomes

Success: Fix works, create PR

fixwright.on('attempt:success', (attempt) => {
  console.log('Fix validated successfully!');
  console.log('Analysis:', attempt.analysis);
  console.log('Fix:', attempt.proposedFix);
});

Failed: Try a different approach

fixwright.on('attempt:failed', (attempt) => {
  console.log('Fix did not work:', attempt.error);
  // Will retry with different approach
});

Regression: Fix broke something else

fixwright.on('attempt:regression', (attempt) => {
  console.log('Fix caused new failures');
  // Will revert and try again
});

Multi-Attempt Logic

Fixwright tries multiple approaches before giving up:

{
  fixing: {
    maxAttempts: 3,          // Try 3 different fixes
    maxRetriesPerAttempt: 2, // Retry validation 2x per fix
    cooldownMs: 1000,        // Wait between attempts
  }
}

Learning from Previous Attempts

Each attempt includes context from previous failures:

// Attempt 1: Try updating selector
// Result: Still fails - element not found


// Attempt 2: Claude knows selector change didn't work
// Tries adding explicit wait instead


// Attempt 3: Claude knows wait didn't help
// Looks for page flow changes

Cause Detection

Claude identifies the failure cause type:

Cause TypeDescriptionCommon Fixes
selector_changedElement selector no longer matchesUpdate selector
timing_issueElement not ready in timeAdd explicit wait
page_flow_changeNavigation or flow changedUpdate step sequence
element_stateElement disabled/hiddenAdd state check
api_changeBackend response changedUpdate assertions
unknownCannot determine causeTry multiple approaches

Validation Runner

The validation runner executes scripts to verify fixes:

interface ValidationConfig {
  stepwrightPath: string;  // Path to stepwright CLI
  timeout: number;         // Max validation time (ms)
  retries: number;         // Retry failed validations
}

Validation Process

  1. Save Changes: Write edited files to disk
  2. Run Script: Execute via Stepwright CLI
  3. Parse Result: Check if script passed
  4. Report Status: Emit success/failure event
const validation = new ValidationRunner({
  stepwrightPath: 'npx stepwright',
  timeout: 120000,
  retries: 2,
});


const result = await validation.validate(scriptPath);
console.log('Validation passed:', result.success);

Complete Flow Example

import { FixWright } from '@korvol/fixwright';


const fixwright = new FixWright({
  ai: { apiKey: process.env.ANTHROPIC_API_KEY! },
  fixing: { maxAttempts: 3 },
});


fixwright.useFileSource('./failures');


// Log the complete flow
fixwright.on('attempt:start', (a) =>
  console.log(`\n=== Attempt ${a.attemptNumber} ===`));


fixwright.on('attempt:analyzing', () =>
  console.log('Analyzing failure...'));


fixwright.on('agent:tool_use', (tool, input) =>
  console.log(`Using ${tool}:`, input));


fixwright.on('attempt:fixing', () =>
  console.log('Applying fix...'));


fixwright.on('attempt:validating', () =>
  console.log('Validating...'));


fixwright.on('attempt:success', (a) => {
  console.log('Success!');
  console.log('Cause:', a.analysis?.causeType);
  console.log('Fix:', a.proposedFix?.explanation);
});


fixwright.on('attempt:failed', (a) =>
  console.log('Failed:', a.error));


fixwright.on('pr:created', (url) =>
  console.log('PR:', url));


await fixwright.start();

Next Steps