Scrap Web
  • Get Started
    • Introduction
    • Quickstart
  • Customize
    • Agent Settings
    • Browser Settings
    • Custom Functions
    • LangChain Chat Models
    • System prompt
  • Development
    • Local Setup
    • Telemetry
    • Roadmap
    • Tokenomics
Powered by GitBook
On this page
  1. Customize

Agent Settings

Learn how to configure the agent

PreviousQuickstartNextBrowser Settings

Last updated 4 months ago

Overview

The Agent class is the core component of Scrap Web that handles browser automation. Here are the main configuration options you can use when initializing an agent.

Basic Settings

Copy

from browser_use import Agent
from langchain_openai import ChatOpenAI

agent = Agent(
    task="Search for latest news about AI",
    llm=ChatOpenAI(model="gpt-4"),
)

Required Parameters

  • task: The instruction for the agent to execute

  • llm: A LangChain chat model instance. See for supported models.

Agent Behavior

Control how the agent operates:

Copy

agent = Agent(
    task="your task",
    llm=llm,
    controller=custom_controller,  # Custom function registry
    use_vision=True,              # Enable vision capabilities
    save_conversation_path="logs/conversation.json"  # Save chat logs
)
  • use_vision: Enable/disable vision capabilities. Defaults to True.

    • When enabled, the model processes visual information from web pages

    • Disable to reduce costs or use models without vision support

    • For GPT-4o, image processing costs approximately 800-1000 tokens (~$0.002 USD) per image (but this depends on the defined screen size)

  • save_conversation_path: Path to save the complete conversation history. Useful for debugging.

Vision capabilities are recommended for better web interaction understanding, but can be disabled to reduce costs or when using models without vision support.

browser: A Scrap Web Browser instance. When provided, the agent will reuse this browser instance and automatically create new contexts for each run().

Copy

from browser_use import Agent, Browser
from playwright.async_api import BrowserContext

# Reuse existing browser
browser = Browser()
agent = Agent(
    task=task1,
    llm=llm,
    browser=browser  # Browser instance will be reused
)

await agent.run()

# Manually close the browser
await browser.close()

Remember: in this scenario the Browser will not be closed automatically.

Copy

from browser_use import Agent, Browser
from playwright.async_api import BrowserContext

# Use specific browser context (preferred method)
async with await browser.new_context() as context:
    agent = Agent(
        task=task2,
        llm=llm,
        browser_context=context  # Use persistent context
    )

    # Run the agent
    await agent.run()

    # Pass the context to the next agent
    next_agent = Agent(
        task=task2,
        llm=llm,
        browser_context=context
    )

    ...

await browser.close()

You can reuse the same context for multiple agents. If you do nothing, the browser will be automatically created and closed on run() completion.

The agent is executed using the async run() method:

  • max_steps (default: 100) Maximum number of steps the agent can take during execution. This prevents infinite loops and helps control execution time.

The method returns an AgentHistoryList object containing the complete execution history. This history is invaluable for debugging, analysis, and creating reproducible scripts.

Copy

# Example of accessing history
history = await agent.run()

# Access (some) useful information
history.urls()              # List of visited URLs
history.screenshots()       # List of screenshot paths
history.action_names()      # Names of executed actions
history.extracted_content() # Content extracted during execution
history.errors()           # Any errors that occurred
history.model_actions()     # All actions with their parameters

The AgentHistoryList provides many helper methods to analyze the execution:

  • final_result(): Get the final extracted content

  • is_done(): Check if the agent completed successfully

  • has_errors(): Check if any errors occurred

  • model_thoughts(): Get the agent’s reasoning process

  • action_results(): Get results of all actions

Behavior Parameters

controller: Registry of functions the agent can call. Defaults to base Controller. See for details.

system_prompt_class: Custom system prompt class. See for customization options.

(Reuse) Browser Configuration

You can configure how the agent interacts with the browser. To see more Browser options refer to the documentation.

Reuse Existing Browser

Reuse Existing Browser Context

browser_context: A Playwright browser context. Useful for maintaining persistent sessions. See for more details.

For more information about how browser context works, refer to the .

Running the Agent

Agent History

For a complete list of helper methods and detailed history analysis capabilities, refer to the AgentHistoryList source code that will be displayed on our .

​
​
LangChain Models
​
​
Custom Functions
System Prompt
​
Browser Settings
​
​
Persistent Browser
Playwright documentation
​
​
twitter