Web Scraper API - cURL Usage Guide

Overview

This document demonstrates how to interact with the Web Scraper API using cURL. You will learn how to:

• Retrieve data (GET)
• Insert new records (POST)
• Apply filters, pagination, and sorting
• Handle conflicts when inserting data

Authentication

All requests require an api-key in the headers:

-H "api-key: YOUR_API_KEY"

1️⃣ Retrieving Data (GET Requests)

Get all trade tickers

curl -X GET "https://srsimpson.it.com/capitol_trades_api/get_capitol_trades?fields=trade_ticker" \
     -H "api-key: YOUR_API_KEY"

Get a limited number of tickers (e.g., 10 results)

curl -X GET "https://srsimpson.it.com/capitol_trades_api/get_capitol_trades?fields=trade_ticker&limit=10" \
     -H "api-key: YOUR_API_KEY"

Filter out tickers that are "N/A"

curl -X GET "https://srsimpson.it.com/capitol_trades_api/get_capitol_trades?fields=trade_ticker&filter=trade_ticker<>N/A" \
     -H "api-key: YOUR_API_KEY"

Get tickers that contain $

curl -X GET "https://srsimpson.it.com/capitol_trades_api/get_capitol_trades?fields=trade_ticker&filter=trade_ticker%20LIKE%20%25$%25" \
     -H "api-key: YOUR_API_KEY"

Sort results by traded_date in descending order

curl -X GET "https://srsimpson.it.com/capitol_trades_api/get_capitol_trades?fields=trade_ticker,traded_date&sort=traded_date%20DESC" \
     -H "api-key: YOUR_API_KEY"

Paginate results (page 2, 10 results per page)

curl -X GET "https://srsimpson.it.com/capitol_trades_api/get_capitol_trades?fields=trade_ticker&page=2&per_page=10" \
     -H "api-key: YOUR_API_KEY"

2️⃣ Inserting Data (POST Requests)

Insert a single record

curl -X POST "https://srsimpson.it.com/capitol_trades_api/insert_capitol_trades" \
     -H "Content-Type: application/json" \
     -H "api-key: YOUR_API_KEY" \
     -d '[
           {
             "trade_ticker": "AAPL",
             "owner": "Nancy Pelosi",
             "traded_date": "2024-03-01",
             "transaction_type": "BUY",
             "amount": 50000
           }
         ]'

Insert multiple records (Bulk Insert)

curl -X POST "https://srsimpson.it.com/capitol_trades_api/insert_capitol_trades" \
     -H "Content-Type: application/json" \
     -H "api-key: YOUR_API_KEY" \
     -d '[
           {
             "trade_ticker": "AAPL",
             "owner": "Nancy Pelosi",
             "traded_date": "2024-03-01",
             "transaction_type": "BUY",
             "amount": 50000
           },
           {
             "trade_ticker": "TSLA",
             "owner": "Elon Musk",
             "traded_date": "2024-03-02",
             "transaction_type": "SELL",
             "amount": 75000
           }
         ]'

Insert with on_conflict=update (Handle Duplicates)

curl -X POST "https://srsimpson.it.com/capitol_trades_api/insert_capitol_trades?on_conflict=update" \
     -H "Content-Type: application/json" \
     -H "api-key: YOUR_API_KEY" \
     -d '[
           {
             "trade_ticker": "AAPL",
             "owner": "Nancy Pelosi",
             "traded_date": "2024-03-01",
             "transaction_type": "BUY",
             "amount": 60000  # ✅ Updates the amount if a conflict occurs
           }
         ]'

🎯 Summary

✅ Use ** requests** to retrieve trade data with filtering, sorting, and pagination.\ ✅ **Use ** requests to insert new trade records, supporting bulk inserts and conflict handling.\ ✅ Secure API calls by always including your API key in the headers.

🚀 Now you're ready to interact with the Web Scraper API using cURL!