POST/v1/map

Map

Discover all URLs on a website by combining sitemap.xml parsing, robots.txt discovery, and link crawling. Returns a flat list of URLs with metadata.

Target Latency

1.2s - 4.5s

Credits

1 cr/req

Parameters

Name	Type	Requirement	Description
url	string	REQUIRED	The website URL to map.
search	string	optional	Filter URLs matching this search string.
limit	number	optional	Maximum number of URLs to return.
include_subdomains	boolean	optional	Include URLs from subdomains.
use_sitemap	boolean	optional	Parse sitemap.xml for URL discovery.

cURL Example

curl -X POST "https://api.datablue.dev/v1/map" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "url": "https://docs.python.org",
  "limit": 200,
  "include_subdomains": false
}'

System Responses

200 OK

Request processed successfully.

401 UNAUTHORIZED

Missing or invalid API key.

429 RATE LIMIT

System capacity exceeded.

500 SYSTEM FAILURE

Internal core exception.

EXAMPLE RESPONSE

{
  "success": true,
  "total": 187,
  "links": [
    {
      "url": "https://docs.python.org/3/",
      "title": "Python 3 Documentation",
      "lastmod": "2026-03-28"
    },
    {
      "url": "https://docs.python.org/3/tutorial/index.html",
      "title": "The Python Tutorial"
    },
    {
      "url": "https://docs.python.org/3/library/index.html",
      "title": "The Python Standard Library"
    },
    {
      "url": "https://docs.python.org/3/reference/index.html",
      "title": "The Python Language Reference"
    }
  ]
}