AI Configuration

AI Configuration

Fixwright uses Claude AI via the Claude Agent SDK to analyze failures and apply fixes. This page covers AI configuration options.

Basic Configuration

import { FixWright } from '@korvol/fixwright';

const fixwright = new FixWright({
  ai: {
    apiKey: process.env.ANTHROPIC_API_KEY!,
    model: 'claude-sonnet-4-20250514',
    maxTokens: 8192,
    temperature: 0,
  },
});

Configuration Options

apiKey (required)

Your Anthropic API key:

ai: {
  apiKey: process.env.ANTHROPIC_API_KEY!,
}

Caution

Never hardcode API keys. Use environment variables.

model

Claude model to use. Default: claude-sonnet-4-20250514

ai: {
  model: 'claude-sonnet-4-20250514',  // Recommended for code fixes
}

Available models:

• claude-sonnet-4-20250514 - Best balance of speed and capability
• claude-3-5-sonnet-latest - Latest Sonnet version
• claude-opus-4-20250514 - Most capable, slower

maxTokens

Maximum tokens in the response:

ai: {
  maxTokens: 8192,  // Default
}

Increase for complex fixes that require more explanation.

temperature

Controls randomness in responses:

ai: {
  temperature: 0,  // Deterministic (recommended for code)
}

AgentFixer Configuration

For direct control over the agent, use AgentFixer:

import { AgentFixer } from '@korvol/fixwright';

const fixer = new AgentFixer({
  model: 'claude-sonnet-4-20250514',
  maxTurns: 20,    // Max tool use turns
  timeout: 300000, // 5 minute timeout
  verbosity: 'verbose',
});

maxTurns

Maximum number of tool interactions:

{
  maxTurns: 20,  // Default
}

A typical fix uses 5-10 turns (read file, analyze, edit, verify).

timeout

Maximum time for a fix attempt:

{
  timeout: 300000,  // 5 minutes (default)
}

verbosity

Control output detail level:

quiet

fixwright.setVerbosity('quiet');
// Minimal output - only results

normal

fixwright.setVerbosity('normal');
// Standard output - tool usage visible
// Default setting

verbose

fixwright.setVerbosity('verbose');
// Detailed output - thinking, all tool events

Available Tools

Claude has access to these tools for fixing:

Tool	Description
Read	Read file contents
Edit	Make targeted code edits
Write	Write new files
Glob	Find files by pattern
Grep	Search file contents
Bash	Run shell commands (limited)

Tool Usage Example

fixwright.on('agent:tool_use', (toolName, input) => {
  switch (toolName) {
    case 'Read':
      console.log('Reading:', input.file_path);
      break;
    case 'Edit':
      console.log('Editing:', input.file_path);
      break;
    case 'Grep':
      console.log('Searching for:', input.pattern);
      break;
  }
});

Bash Restrictions

For security, Bash is limited to safe commands:

Allowed:

• cat, ls, pwd, echo, grep, find, head, tail
• npm test, npm run, npx, pnpm, yarn, node

Blocked:

• rm -rf, sudo, chmod, chown
• Pipe to shell (curl | sh)
• Direct device access

Prompt Engineering

System Prompt

Claude receives a system prompt guiding its behavior:

You are an expert Playwright test automation engineer. Your task is to
analyze and fix failing Playwright tests.

Guidelines:
1. Read the full file first to understand context
2. Common failure causes: selector changes, timing issues, flow changes
3. Fix strategies: update selectors, add waits, update assertions
4. Best practices: prefer stable selectors, explicit waits

Context Provided

Each fix attempt includes:

1. Failure Information

   • Script name and failed step
   • Error type and message
   • Stack trace

2. Source Context

   • File path and line numbers
   • Failed code snippet

3. Artifacts

   • Screenshot (if available)
   • DOM snapshot (if available)
   • Console logs

4. Previous Attempts (if any)

   • What was tried
   • Why it failed
   • “Try a different approach”

Cost Management

Monitoring Usage

Track API costs:

fixwright.on('agent:complete', (result) => {
  console.log('Cost:', result.totalCostUsd);
  console.log('Duration:', result.duration, 'ms');
});

Cost Optimization Tips

1. Use Sonnet - Claude Sonnet is more cost-effective than Opus
2. Limit attempts - Reduce maxAttempts if fixes often fail
3. Good artifacts - Better context means fewer retry attempts
4. Filter failures - Don’t process obviously unfixable failures

{
  fixing: {
    maxAttempts: 2,  // Reduce from default 3
  },
  ai: {
    model: 'claude-sonnet-4-20250514',  // More economical
  },
}

Error Handling

API Errors

fixwright.on('agent:error', (error) => {
  if (error.message.includes('rate_limit')) {
    console.log('Rate limited, retrying...');
  } else if (error.message.includes('invalid_api_key')) {
    console.error('Invalid API key');
  }
});

Timeout Handling

fixwright.on('agent:complete', (result) => {
  if (!result.success && result.error?.includes('timeout')) {
    console.log('Fix attempt timed out');
  }
});

Environment Variables

# Required
ANTHROPIC_API_KEY=sk-ant-...

# Optional overrides
FIXWRIGHT_MODEL=claude-sonnet-4-20250514
FIXWRIGHT_MAX_TURNS=20
FIXWRIGHT_TIMEOUT=300000

Next Steps

• Git Integration - Branch and commit handling
• PR Creation - Automatic PR creation
• Events and Hooks - Monitor AI activity