The Scraping Stack
The tracker needs data coming in. This is the part people overpay for — there's a whole shelf of $20–100/mo products that are, underneath, a scraper plus a division. You're going to rebuild the scraper half for $0–47/mo.
You can keep scraping inside track.py, or split it into its own resilient file. Splitting it is cleaner: the scraper's only job is to pull posts and write them out, so the tracker can read them without caring how they arrived.
The scraper prompt
Paste this into Claude Code, exactly as written:
Write `scrape.py` that pulls the last N posts for a list of social accounts and
writes normalized JSONL (one object per post: platform, account, id, url, caption,
posted_at, views, likes, comments, shares, saves).
- YouTube + TikTok: shell out to yt-dlp with --dump-json --flat-playlist for listing,
then hydrate view_count per video. Parse the JSON, don't screen-scrape HTML.
- Instagram: call the ScrapeCreators reels endpoint (or Apify apidojo/instagram-scraper).
- Be polite: rate-limit to 1 req/sec, retry with exponential backoff on 429/5xx,
and cache raw responses to ./cache/ so re-runs cost 0 API credits.
- Idempotent: skip ids already in output.jsonl.
Print a summary: accounts scraped, new posts, API credits used.
Three details make this cheap and durable: cache to ./cache/ so re-runs cost nothing, skip ids already saved so it's idempotent, and 1 req/sec with backoff so you don't get rate-limited or banned.
The tools
Each platform has a source, and none of them require paying $100/mo:
- yt-dlp — free, no API key. The most reliable way to pull YouTube/TikTok metadata (and 1,000+ other sites).
--dump-jsongives you view count, likes, upload date, channel, and more.--flat-playlistlists a channel fast, then you hydrate the views you care about. Parse the JSON — never screen-scrape HTML. - ScrapeCreators — one API, 35+ platforms. For Instagram reels (and TikTok, YouTube, more). Free tier: 100 credits that never expire, no card. $47/mo gets you 25,000 credits. 1 request = 1 credit.
- Apify — pay-per-result. The
apidojo/instagram-scraperactor runs $0.30–0.50 per 1,000 posts. Free plan, no card. Good when you're scraping a lot of IG at once. business_discovery— one Meta call per creator returns followers + recent posts. Free, public.- The
media-infoendpoint — Instagram's endpoint for public plays and handle. Free, and it's what gives you the view count for the score without owner access.
The cost, and what you're replacing
Total to start: $0–47/mo. yt-dlp (free) + ScrapeCreators (free tier or $47) or Apify ($0.30–0.50/1k) + free public endpoints. Compare that to the products this replaces, all $20–100/mo:
Every one of those is a wrapper around (a) a scraper and (b) views ÷ median. That's it. You built the median in lesson 1, the tracker in lesson 2, and the scraper here — so you own the whole thing, tuned to your niche, for the price of a coffee subscription.
Scrape politely. 1 request per second, exponential backoff on 429/5xx errors, and cache everything. This isn't only about cost — hammering an endpoint gets you rate-limited or blocked, which kills your data feed. Slow and cached beats fast and banned.
What's the free option for pulling YouTube and TikTok metadata with no API key?
Key takeaway
Feed the tracker with a cheap, resilient stack: yt-dlp (free, YouTube/TikTok), ScrapeCreators (free/$47, Instagram) or Apify ($0.30–0.50/1k), plus free business_discovery + media-info for public plays. Cache to disk, stay idempotent, scrape at 1 req/sec. Total: $0–47/mo to replace OutlierKit, 1of10, vidIQ, Virlo, and ViralFindr.
