Web Scraper Pro

Web Scraper Pro is a self-hosted, premium web application that lets you extract structured data from websites using simple CSS selectors. Point it at a listing page (products, jobs, articles…), tell it how to find each item and the fields you want, and export everything to CSV or JSON in seconds. No database, no cloud dependency — just a fast, lightweight Flask app with a clean dark UI.

Key Features

• 🎯 CSS selector extraction with BeautifulSoup
• 📑 List-based scraping: define a container (list_selector) and per-field selectors (e.g., title, price, link, image)
• 📄 Pagination: simple {page} pattern like ?page={page} with configurable number of pages
• 🕒 Rate control: optional delay between pages to be polite
• 🧩 Custom headers (e.g., User-Agent)
• 🔗 Auto-fix relative links to absolute URLs
• 🧾 Preview & Export: live preview (first 50 rows) and one-click export to CSV or JSON
• 🔌 API endpoint: /api/test for programmatic scrapes and integrations
• 🖤 Dark, responsive UI (Bootstrap 5)

How it works

1. Paste a JSON config describing where to start and what to extract.
2. Click Run Scrape — Web Scraper Pro fetches pages, extracts items, and shows a preview.
3. Download the result as CSV or JSON.

Tips:

• Field selectors are relative to each item in the list.
• For attributes on the item element itself, you can use attr:href, attr:src, etc.
• If links start with / or ./, the app converts them to absolute URLs automatically.

What’s included

• Full Flask app (app.py)
• Templates: index, result (dark theme)
• CSV/JSON export to /static/exports
• requirements.txt, README.txt, LICENCE.txt
• Gumroad cover (1920×1080)

Important (Legal & Ethics)

Use this tool only on content you’re legally allowed to access. Always respect Terms of Service, robots.txt, and applicable laws. Avoid heavy scraping and be a good web citizen.