Local Setup

Set up Scrap Web development environment locally

Prerequisites

Scrap Web requires Python 3.11 or higher. We recommend using uv for Python environment management.

​Clone the Repository

First, clone the Scrap Web repository:

Copy

git clone https://github.com/scrap-web/scrap-web
cd scrap-web

​Environment Setup

1. Create a virtual environment:

Copy

uv venv --python 3.11

1. Install dependencies:

Copy

# Install all dependencies including dev packages
uv pip install . ."[dev]"

​Configuration

Set up your environment variables:

Copy

# Copy the example environment file
cp .env.example .env

Or manually create a .env file with your API keys:

.env

Copy

OPENAI_API_KEY=
ANTHROPIC_API_KEY=

You can use any LLM model supported by LangChain. See LangChain Models for available options and their specific API key requirements.

​Development

After setup, you can:

• Run tests with pytest

• Build the package with hatch build

• Try the examples in the examples/ directory

​Getting Help

If you run into any issues:

1. Check our GitHub Issues

2. Join our Discord community for support

We welcome contributions! See our Contribution Guide for guidelines on how to help improve Scrap Web.