Get started

Web Scraping API

Scrape and extract data from any website, with powerful options like proxy / browser customization, CAPTCHA handling, ad blocking, and more.
Get your free API key
Test it out
Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.
Request

GET https://scrape.abstractapi.com
?url=https://news.ycombinator.com &api_key=KndsqhGSD6672dhGS6

Response
Serving millions of calls to 100,000+ developers and end users worldwide

Why 10,000+ developers love Abstract's APIs

Easy to implement and maintain

We built Abstract because most of the API's we've used aren't great for developers. That's why Abstract has excellent documentation, multiple easy to use libraries, and tutorials to get you started.

Speed and scale built into everything we do

Our APIs are built to power critical business processes and flows, so all our APIs are built for use at scale and at blazing speeds. These aren't just marketing phrases for, but fundamental features of our APIs.

Reliable uptime and great support

Developers trust Abstract because of our reliable uptime and excellent technical support that will help get you live quickly, keep you running smoothly, and resolve any issues you have fast.

Abstract's Web Scraping API helps you extract data from any URL

Powerful web scraping technology

Abstract's Web Scraping Screenshot API is designed to balance the top considerations with any web scraping project: power and simplicity. The API is simple yet highly configurable.

Millions of proxies and IP's available

Abstract maintains a constantly rotated and validated pool of IP addressed and proxies to ensure your extraction goes through successfully as quickly as possible.

Robust and global infrastructure

Abstract's extensive global infrastructure means your web scraping requests are processed quickly and come bundled with powerful features and customization options.

Highly customizable extraction options

Configure if and how you'd like to handle Javascript, ads, and other custom options to ensure you scrape and extract only the exact data you need for your workflow.

Choose from 100+ locations globally

Among the API's configuration options is the ability to specify from which location the website should be extracted. Over 100 countries and regions are available to choose from.

Bank-level security for all queries

Security is very important to us, so all data that is sent to Abstract's Web Scraping API and processed by Abstract is secured by 256-bit SSL encryption (HTTPS).

Simple and cost-effective pricing

FREE

$0

no card
required!

  • 1,000 requests / month
  • Non-commercial use only

STARTER

$9

per
month

  • 5,000 requests / month
  • Commercial use allowed

PROFESSIONAL

$49

per
month

  • 35,000 requests / month
  • Commercial use allowed
  • Chat support

ENTERPRISE

$499

per
month

  • 400,000 requests / month
  • Commercial use allowed
  • Premium chat support

Web Scraping API Documentation

Overview

Abstract's Web Scraping API is a simple yet powerful REST API that is used to extract data from a given URL. To make a request, you simply include the target URL and your API key, and Abstract's API will return the data from that site. Optional configuration options, like proxies, location of the request, and more are possible to include.

REST

The Web Scraping API, like all of Abstract's APIs, is organized around REST. It is designed to use predictable, resource-oriented URL's and to use HTTP status codes to indicate errors.

HTTPS

The Web Scraping API requires all communications to be secured TLS 1.2 or greater.

API Versions

All of Abstract's API are versioned. The Web Scraping API is currently on Version 1.

Your API Key

Your API key is your unique authentication key to be used to access Abstract's Web Scraping API. Note that each of Abstract's APIs has a unique API key, so you will need different keys to access the Web Scraping and Website Screenshot APIs, for example. To authenticate your requests, you will need to append your API key to the base URL.

Base URL

https://scrape.abstractapi.com/v1/

Using the Website Screenshot API

Abstract's Website Screenshot API only has two required parameters for the request, your unique API key and a single URL.

https://scrape.abstractapi.com/v1/
    ? api_key = YOUR_API_KEY
    & url = https://www.apple.com

Abstract's Website Screenshot API only has two required parameters for the request, your unique API key and a single email:

<!DOCTYPE html>

<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en-US" lang="en-US">'

    <head>
       [...] // 46 lines skipped
    </head>
   
    <body>
       [...] // 512 lines skipped
    </body>

</html>

Web Scraping API - Request Parameters

PARAMETER

TYPE

DETAILS

api_key required

String

Your unique API key. Note that each user has unique API keys for each of Abstract's APIs, so your Web Scraping API key will not work for as your User Avatar API key, for example.

url required

String

The URL to extract the data from. Note that this parameter should include the full HTTP Protocol (http:// or https://).

render_js

Boolean

If true the request will render Javascript on the target site. Note that Javascript is rendered via a Google Chrome headless browser. Defaults to false.

block_ads

Boolean

If true the request will block any advertisements it can identify on the target site. Defaults to false.

proxy_country

String

The country to make the request from. The country should be submitted in the two letter, ISO 3166-1 alpha-2 code.

Response and Error Codes

Whenever you make a request that fails for some reason, an error is returned also in the JSON format. The errors include an error code and description, which you can find in detail below.

CODE

TYPE

DETAILS

200

OK

Everything worked as expected.

400

Bad request

Bad request

401

Unauthorized

The request was unacceptable. Typically due to the API key missing or incorrect.

500

Internal Server Error

The request could not be completed due to an error on the server side.

503

Service Unavailable

The server was unavailable.

Frequently Asked Questions
about Abstract's Web Scraper API

What is the Web Scraping API?

The Web Scraping API takes a URL and returns the HTML for it. The API comes with a variety of configurations so you can exclude or include various data in your extraction, as well as handle various scenarios like geo-specific data and ads.

Developers love Abstract's Screenshot API

We tried building web scraping internally for clients, but nothing in house or off the shelf could match the flexibility we got with Abstract's Web Scraping API.

Chris S., Scope

Great API! Love the support team who is always very quick to respond to any questions and help us with any issues.

Matt K.

Our company requires a pretty robust scraping setup, working with hundreds of thousands of pages from thousands of sides. Abstract handles it no problem.

Kerry L.

Web Scraping API Changelog

Last Updated on 

July 11, 2020

July 11, 2020

 - 

Improved handling of images and special characters which caused issues in certain exports

June 18, 2020

 - 

Improved custom Javascript handling and reduced load time

GET STARTED

Get  FREE credits to start today.

Get thousands of API calls for free across all Abstract APIs.
GET YOUR FREE API KEY

Frequently Asked Questions
About Subscription & Pricing

What does the API cost?

Each of our API's has a free plan as well as several paid options, which offer a greater number of requests, a higher level of data quality, and greater speed for higher prices. Please see the page above or sign into your account to view pricing for each API.

Do you have a free trial or free options for your API's?

Absolutely! We want you to try before you buy. Each of our API's has a free tier with a generous usage allowance for you to try the product out and verify that the speed and data quality work for you.

What payment options do you offer?

We use Stripe to accept payment via major debit and credit cards (Visa, Mastercard, American Express). If you cannot use those payment methods, then contact us at team at abstractapi.com to arrange for an alternative payment method.

Can I upgrade or downgrade my subscription?

Yes, you can upgrade or downgrade your subscription at any time by visiting your dashboard. If you upgrade in the middle of a billing cycle, you will be charged a pro-rated amount for your new plan through the end of the billing cycle.

Can I cancel at any time?

Yes, all of our subscriptions are month to month. You can cancel at any time, and you will not be charged again at the end of your billing cycle.

How can I cancel my subscription?

You can cancel your subscription at any time from your dashboard. Once canceled, your subscription will be active until the end your current billing cycle, and then it will be automatically moved to the free plan.

What happens if I go over my monthly quota for requests?

If your API usage exceeds the monthly limit set by your plan, then you will be charged a fee for each call over your limit. At the end of your billing period, your usage will reset again.

What if I need more requests or something else that's not offered in your plans?

We do offer enterprise plans with custom features and pricing in limited circumstances. If our public pricing plans don't work for you, then contact us (team at abstractapi.com) and we'll work something out.

Do you offer any discounts?

While we don't normally offer discounts, we do make exceptions for certain types of users and use cases. If you are a student or are engaged in academic work, or are working on a non-profit or open source project, we're happy to offer you discounts for our API's. Just email us at team at abstractapi.com.

Do you offer discounts for annual or quarterly payments?

All of our plans are currently paid month to month. We will be launching annual plans at a discount soon. If you're interested in an annual plan in the meantime, just contact us and we'll set you up.

What happens if my payment method expires?

We will typically attempt to charge your payment method up to three times after it fails. If we're still unable to collect payment, then your subscription will be automatically canceled and you'll be moved to the free plan.

Can I use the API's for a commercial project?

Yes, a paid tier of our API may be used in conjunction with a commercial project. The free tiers of our API's may not be used in commercial projects.