What External Services and APIs Does the Shadowbroker Backend Integrate With?

Shadowbroker integrates with more than 15 external read-only APIs to aggregate global aviation, maritime, satellite, and geopolitical data, storing fetched results in an in-memory cache served via FastAPI endpoints.

The Shadowbroker project—hosted in the BigBodyCobain/Shadowbroker repository—is a Python-based data aggregation platform that consumes public-domain feeds to build a unified intelligence dashboard. All external service calls are orchestrated from the backend/services/fetchers package and triggered by periodic jobs defined in backend/main.py.

Aviation Data Sources and ADS-B Aggregators

Shadowbroker consumes multiple Aircraft Dependent Surveillance-Broadcast (ADS-B) feeds to build a comprehensive global flight picture. The primary fetching logic resides in backend/services/fetchers/flights.py.

OpenSky Network serves as the fallback source for aircraft state vectors, handling approximately 5,000+ aircraft via OAuth2 authentication. The OpenSkyClient class manages token logic and rate limiting in flights.py lines 32–70.

adsb.lol provides the fast-path commercial and military aircraft positions through the _fetch_adsb_lol_regions() function (lines 335–380), while airplanes.live fills regional blind spots via _fetch_supplemental_sources() (lines 94–110). Additionally, adsb.fi contributes global point-query data through the same supplemental pipeline (lines 140–165).

Maritime, Satellite, and Orbital Intelligence

For vessel tracking, the backend connects to aisstream.io to retrieve real-time AIS positions for approximately 25,000 ships, implemented in backend/services/fetchers/plan_vessel_alert.py.

Satellite orbital elements are sourced from CelesTrak via the fetch_satellites() function in satellites.py (lines 620–640), which downloads TLE (Two-Line Element) sets in JSON format.

Remote sensing capabilities leverage NASA GIBS and Sentinel-2 Hub for MODIS and SAR ground-change detection, handled by fetch_satellite_imagery() in earth_observation.py (lines 300–340). On-demand processing uses the Copernicus CDSE API, requiring a free Earthdata token, as implemented in fetch_sentinel_hub() (lines 350–380).

Geopolitical Threat Intelligence and Cyber Reconnaissance

The platform monitors conflict zones through GDELT Project, pulling approximately 1,000 events every six hours via fetch_gdelt() in crowdthreat.py (lines 58–90). Ukraine frontline data comes from DeepState Map through fetch_ukraine_alerts() in ukraine_alerts.py (lines 80–105).

Shodan integration enables internet-connected device reconnaissance including cameras and SCADA systems. The backend/services/shodan_connector.py file contains the full connector implementation, requiring an operator-supplied API key stored in environment variables.

Environmental and Alternative Data Streams

Air quality measurements are retrieved from OpenAQ via fetch_air_quality() in air_quality.py (lines 10–45). Maritime fishing activity comes from Global Fishing Watch through fetch_fishing() in infrastructure.py (lines 120–150).

The backend also polls prediction market APIs (Polymarket and Kalshi) for market-ticker overlays, implemented across lines 400–960 in prediction_markets.py.

Mesh Networks and Signal Intelligence

Internet-of-Things and mesh radio coverage includes Meshtastic MQTT and APRS-IS for amateur radio traffic, handled by fetch_meshtastic_map() in meshtastic_map.py. Public short-wave receiver locations are sourced from KiwiSDR via fetch_kiwisdr() in sigint.py (lines 450–475).

Static datasets—including power plants, data centers, and military bases—are loaded from bundled CSV/JSON files in backend/services/fetchers/infrastructure.py without runtime external calls.

Implementation Architecture

All external integrations are read-only; Shadowbroker never writes data back to third-party services. The backend/main.py file defines periodic jobs such as fetch_flights(), fetch_satellites(), and fetch_ais(), which populate an in-memory latest_data cache. Frontend clients poll this cache through FastAPI endpoints under /api/.

Authentication credentials (OpenSky OAuth2, Shodan API keys, Earthdata tokens) are centrally managed in backend/services/api_settings.py via environment variables.

Code Examples

Trigger Flight Data Aggregation

curl -s http://localhost:8000/api/fetch/flights | jq .

This endpoint chains through backend/main.py → fetch_flights() → _fetch_adsb_lol_regions() → _enrich_with_opensky_and_supplemental().

Search Shodan for SSH Services

curl -X GET "http://localhost:8000/api/shodan/search?query=port:22" \
     -H "Accept: application/json"

The handler calls search_shodan() in shodan_connector.py, which respects the per-second rate limit defined by _MIN_INTERVAL_SECONDS.

Retrieve Satellite TLE Data

curl -s http://localhost:8000/api/satellites | jq '.satellites[] | .name,.tle'

Implementation uses _fetch_satellite_tle() in satellites.py, downloading from https://celestrak.org/NORAD/elements/gp.php?GROUP=active&FORMAT=json.

Lookup Specific IP in Shodan

curl -s "http://localhost:8000/api/shodan/host/8.8.8.8?history=true" | jq .

Calls lookup_shodan_host() returning IP geolocation, open ports, and service banners.

Summary

• Shadowbroker consumes 15+ external APIs across aviation, maritime, satellite, and intelligence domains.
• All integrations are read-only, with data cached locally in latest_data before serving to frontend clients.
• Primary fetchers reside in backend/services/fetchers/ with entry points in backend/main.py.
• Authentication uses environment variables for OAuth2 (OpenSky), API keys (Shodan), and tokens (Copernicus).
• Rate limiting is implemented per-provider to respect service boundaries.
• Key files: flights.py (aviation), shodan_connector.py (cyber), satellites.py (orbital), earth_observation.py (remote sensing).

Frequently Asked Questions

Does Shadowbroker require API keys to function?

Most feeds work without authentication, but Shodan requires an operator-supplied API key, and OpenSky Network benefits from OAuth2 credentials for higher rate limits. Optional tokens for Copernicus CDSE and NASA Earthdata enable enhanced satellite imagery processing. All credentials are defined in backend/services/api_settings.py.

How does the backend handle rate limiting?

Each fetcher module implements provider-specific throttling. For example, the OpenSkyClient class manages token refresh and request intervals, while shodan_connector.py enforces _MIN_INTERVAL_SECONDS between queries to prevent API bans.

Can I add custom external data sources?

Yes. The modular architecture in backend/services/fetchers/ allows new fetchers to follow the established pattern: create a Python file with a fetch_*() function, import it in backend/main.py, and add a corresponding FastAPI endpoint or periodic job trigger.

Is any data written back to external services?

No. According to the source code analysis, all integrations are strictly read-only. The backend consumes public streams, enriches the data locally, and stores results only in the internal latest_data cache or bundled static files.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}