Meta crawlers (Facebook · Instagram · WhatsApp · Threads) — the 5 bots, ASN verification and IP ranges

Meta Platforms runs five different crawlers for Facebook, Instagram, Messenger, Threads and WhatsApp — from the harmless link-preview bot to the AI-training scraper. Meta publishes the user-agent tokens officially, but no IP list: verification, per Meta's docs, is done via the ASN. Here are the five bots, what you should do with them, and how our PARC Security feed combines the ranges of all five Meta ASNs.

meta.json — view feed   Official Meta crawler docs →

The five Meta crawlers

Allowlist or block?

Unlike a security scanner, Meta is not simply “in or out” — the five crawlers have different use cases (see the table above): FacebookExternalHit generates the link previews when sharing on Facebook/Instagram/WhatsApp, Meta-WebIndexer concerns visibility in Meta AI Search, Meta-ExternalAgent and Meta-ExternalFetcher cover AI training and user-triggered AI fetches respectively (Meta-ExternalFetcher bypasses robots.txt), and Meta-ExternalAds ad optimisation.

The feed provides the IP basis; per-crawler fine-tuning is done via the user-agent.

No official IP list — verification by ASN

Meta publishes no IP list like Google or Cloudflare. The official docs are explicit: “You should not rely solely on the user-agent and should also verify that the IP address is linked to AS32934 (Facebook, Inc.).” So the user-agent alone is worthless (trivially spoofable) — the reliable check is ASN membership.

That’s easy for operators with an ASN-aware firewall. Setups without ASN block support (e.g. SafeLine WAF) need a flat IP list — which is exactly what this feed provides, aggregated from all Meta ASNs.

ASN constellation (AS-SET AS-FACEBOOK)

Meta maintains an AS-SET AS-FACEBOOK in the RADB (maintainer MAINT-AS32934, noc@fb.com) bundling five ASNs:

• AS32934 — Facebook, Inc.: main backbone (US), the bulk of the prefixes. This is the ASN Meta recommends verifying crawlers against.
• AS54115 — Facebook (Corporate): US, smaller allocation including the sub-allocation AS-FACEBOOK-CORP.
• AS63293 — Meta (CDN Network): US, the CDN / edge network.
• AS34825 — Facebook Israel Ltd: IL (formerly Onavo), IRR entry only, currently BGP-inactive — included for completeness.
• AS11917 — WhatsApp: US, also IRR only, BGP-inactive.

Verification: ASN plus PTR pattern

Besides the ASN match, many Meta edge IPs can also be confirmed via FCrDNS. The (not officially documented, but stable in the field) PTR scheme:

• edge-{type}-shv-{NN}-{city}{idx}.facebook.com — e.g. edge-dgw-shv-02-fra3.facebook.com (Frankfurt), edge-secure-shv-01-iad3.facebook.com
• xx-fbcdn-shv-{NN}-{city}.fbcdn.net — CDN front-end

As with BitSight, some IPs are FCrDNS-active (edge servers for end-user traffic), others are PTR-empty (backend / crawler workers without public resolution). All belong to AS-FACEBOOK — which is why the feed uses the ASN approach as the reliable basis, not a pure FCrDNS filter.

How the feed is built

75 IPv4 prefixes + 42 IPv6 prefixes = 117 entries

Sources and steps:

1. Official Meta crawler docs → five user-agents (no IP list).
2. AS-SET AS-FACEBOOK from the RADB (maintained by Meta itself) → five member ASNs.
3. RIPE Stat announced-prefixes per ASN for the BGP-active networks (AS32934/54115/63293).
4. RIPE Stat as-routing-consistency for the two IRR-only ASNs (AS34825 + AS11917).
5. CIDR aggregation (netaddr.IPSet) — merges adjacent /24 into /22, /21 etc. and cuts the list length by over 60 % (from ~300+ sub-allocations down to 75 IPv4 prefixes).

That keeps the feed short while still covering the entire Meta address space.

Sources

• Meta — official web crawler docs (5 crawlers)
• bgp.tools — AS-SET AS-FACEBOOK
• BGP.he.net — AS32934 Facebook, Inc.
• RIPE Stat — AS32934 announced-prefixes
• PeeringDB — AS32934 Meta
• DataDome — Meta-ExternalAgent
• Netify — Meta network ranges
• 51Degrees — Meta's robots and crawlers (2026)

← Back to PARC Security   Contact us