Complete Guide to DevTools Protocol Automation

The Chrome DevTools Protocol (CDP) provides powerful programmatic access to browser internals. Master CDP to build sophisticated automation, testing, and debugging tools.

What is the DevTools Protocol?

The Chrome DevTools Protocol is a JSON-RPC-based protocol that allows external tools to inspect, debug, and control Chrome-based browsers. It's the same protocol that powers Chrome DevTools itself.

Key Capabilities

  • Page Control: Navigation, screenshots, PDF generation
  • DOM Inspection: Query and modify the document
  • Network: Monitor and intercept requests
  • JavaScript: Execute code, set breakpoints
  • Performance: Profiling and tracing
  • Accessibility: Inspect accessibility tree

Getting Started

Enabling Remote Debugging

Launch Chrome with remote debugging enabled:

# Windows
chrome.exe --remote-debugging-port=9222

# macOS
/Applications/Google\ Chrome.app/Contents/MacOS/Google\ Chrome --remote-debugging-port=9222

# Linux
google-chrome --remote-debugging-port=9222

Connecting to CDP

CDP communicates over WebSocket. First, get available targets:

// Get list of debuggable targets
GET http://localhost:9222/json

// Response:
[{
  "id": "ABC123",
  "type": "page",
  "title": "New Tab",
  "url": "chrome://newtab/",
  "webSocketDebuggerUrl": "ws://localhost:9222/devtools/page/ABC123"
}]

Core Domains

Page Domain

Control page lifecycle and navigation:

// Navigate to URL
{ "method": "Page.navigate", "params": { "url": "https://example.com" } }

// Wait for load
{ "method": "Page.loadEventFired" }  // Event

// Capture screenshot
{ "method": "Page.captureScreenshot", "params": { "format": "png" } }
// Returns: { "data": "base64-encoded-image" }

// Print to PDF
{ "method": "Page.printToPDF", "params": { "landscape": false } }

DOM Domain

Query and manipulate the document:

// Get document root
{ "method": "DOM.getDocument" }
// Returns: { "root": { "nodeId": 1, "nodeName": "#document", ... } }

// Query selector
{ "method": "DOM.querySelector",
  "params": { "nodeId": 1, "selector": "#login-button" } }

// Get outer HTML
{ "method": "DOM.getOuterHTML", "params": { "nodeId": 5 } }

// Set attribute
{ "method": "DOM.setAttributeValue",
  "params": { "nodeId": 5, "name": "class", "value": "active" } }

Network Domain

Monitor and intercept network activity:

// Enable network monitoring
{ "method": "Network.enable" }

// Events received:
{ "method": "Network.requestWillBeSent",
  "params": { "requestId": "1", "request": { "url": "...", "method": "GET" } } }

{ "method": "Network.responseReceived",
  "params": { "requestId": "1", "response": { "status": 200, ... } } }

// Enable request interception
{ "method": "Fetch.enable",
  "params": { "patterns": [{ "urlPattern": "*" }] } }

// Modify intercepted request
{ "method": "Fetch.continueRequest",
  "params": { "requestId": "1", "headers": [...] } }

Runtime Domain

Execute JavaScript and inspect objects:

// Execute expression
{ "method": "Runtime.evaluate",
  "params": {
    "expression": "document.title",
    "returnByValue": true
  }
}
// Returns: { "result": { "value": "Page Title" } }

// Call function on object
{ "method": "Runtime.callFunctionOn",
  "params": {
    "objectId": "...",
    "functionDeclaration": "function() { return this.innerText; }"
  }
}

Practical Examples

Full Page Screenshot

async function fullPageScreenshot(cdp) {
    // Get page dimensions
    const { contentSize } = await cdp.send('Page.getLayoutMetrics');

    // Set viewport to full page
    await cdp.send('Emulation.setDeviceMetricsOverride', {
        width: contentSize.width,
        height: contentSize.height,
        deviceScaleFactor: 1,
        mobile: false
    });

    // Capture screenshot
    const { data } = await cdp.send('Page.captureScreenshot', {
        format: 'png',
        captureBeyondViewport: true
    });

    return Buffer.from(data, 'base64');
}

Network Request Logger

async function logRequests(cdp) {
    await cdp.send('Network.enable');

    cdp.on('Network.requestWillBeSent', (params) => {
        console.log(`${params.request.method} ${params.request.url}`);
    });

    cdp.on('Network.responseReceived', (params) => {
        console.log(`  -> ${params.response.status}`);
    });
}

Form Automation

async function fillForm(cdp, selector, value) {
    // Get document
    const { root } = await cdp.send('DOM.getDocument');

    // Find element
    const { nodeId } = await cdp.send('DOM.querySelector', {
        nodeId: root.nodeId,
        selector: selector
    });

    // Focus element
    await cdp.send('DOM.focus', { nodeId });

    // Type text
    for (const char of value) {
        await cdp.send('Input.dispatchKeyEvent', {
            type: 'keyDown',
            text: char
        });
        await cdp.send('Input.dispatchKeyEvent', {
            type: 'keyUp',
            text: char
        });
    }
}

CDP Libraries

Popular libraries that wrap CDP:

Library Language Notes
Puppeteer Node.js Google's official high-level API
Playwright Multi-language Microsoft's cross-browser automation
chrome-remote-interface Node.js Low-level CDP client
PyCDP Python Python CDP client
chromedp Go Go CDP client

Best Practices

  • Wait for Events: Don't assume synchronous execution
  • Handle Disconnections: Browser can close unexpectedly
  • Session Management: Properly create and destroy targets
  • Error Handling: CDP methods can fail for various reasons
  • Resource Cleanup: Disable domains when done

CDP in Birds Engine

Tracy's Birds Engine provides native CDP integration with additional capabilities:

  • Direct CDP access without launching separate Chrome instances
  • Extended commands not available in standard CDP
  • Integration with Birds Engine security and DLP features
  • Managed browser lifecycle
Explore Birds SDK