Platform Web Property Dataset

In the Censys Platform, internet assets that respond to hostname-based scans are classified as web properties.

Web properties are identified by a hostname and a port. Hostnames can be name-based records (such as app.censys.io) or IP-based records (such as 104.18.10.85). Example names of web property records include the following:

• app.censys.io: 443
• 104.18.10.85: 8880

Web properties provide an accurate, up-to-date, and comprehensive view of name-based assets on the worldwide web. You can use web properties to explore:

• Websites
• APIs
• Web-based applications

And much more.

Web property information

All of the fields available for web property records are listed on the Data Definitions page of the Platform web app. This page is always kept up-to-date and reflects the current structure of the Censys Platform web property dataset.

Web properties offer insight into HTTP services beyond layer 7 while abstracting away HTTP protocol semantics.

• Web properties contain all records that correlate to HTTP-based scans.
• A web property can have one or more endpoints, which serve as distinct entry points to different resources or functionalities within the web property.
  • Each endpoint may be associated with specific applications, services, or APIs and can be individually monitored and analyzed.
• Web properties support deep scan information for HTTP-based scanners.

As with hosts, CPE values are provided for software on web properties.

HTTP fields

Web property records surface deep information about individual HTTP endpoints. This includes:

• HTTP body information (web.endpoints.http.body, as well as hash fields).
• Software that runs on top of HTTP, like pprof (web.endpoints.pprof), Kubernetes (web.endpoints.kubernetes), and Elasticsearch (web.endpoints.elasticsearch).

Web properties also contain contextual information about the asset's associated software, hardware, certificates, and more.

Web properties and host services

Web properties offer better visibility into internet assets than services on hosts in certain circumstances.

In the Censys Platform, consider searching web properties when:

• You want results that include hostnames.
• You are targeting software that runs on top of HTTP.
• You are targeting software that services HTTP, like Apache or nginx.
• You need HTTP body information.
• You need data from endpoints other than /.

Do not use web properties when:

• You want results that include IP addresses.
• You are searching for DNS data, WHOIS data, location data, or routing data.
• You are searching for hosts serving HTTP as well as non-HTTP protocols.

Labels

Labels on web properties (web.labels) are used to categorically describe the content and services present on them, just like they are for host services. Web property label values are always in screaming snake case. Many web property labels are the same as those used for host services and include:

• LOGIN_PAGE
• WAF
• DEFAULT_LANDING_PAGE
• CAMERA

Example web property record

The following excerpt is an example of an empty web property record to provide a general idea of the data structure.

  "web": {
    "port": "",
    "scan_timed": "",
    "hostname": "",
    "labels": [],
    "endpoints": [
      {
        "endpoint_type": "",
        "path": "",
        "banner": "",
        "banner_hash_sha256": "",
        "banner_hex": "",
        "transport_protocol": "",
        "<endpoint specific details>": {}
      }
    ]
  },
  "misconfigs": [],
  "exposures": [],
  "vulns": [],
  "software": [],
  "hardware": [],
  "operating_system": [],
  "tls": []
}