Platform Web Property Dataset

In the Censys Platform, internet assets that respond to hostname-based scans are classified as web properties.

Web properties are identified by a hostname and a port. Hostnames can be name-based records (such as app.censys.io) or IP-based records (such as 104.18.10.85). Example names of web property records include the following:

• app.censys.io:443
• 104.18.10.85:8880

Web properties provide an accurate, up-to-date, and comprehensive view of name-based assets on the worldwide web. You can use web properties to explore:

• Websites
• APIs
• Web-based applications

And much more.

Web property information

All of the fields available for web property records are listed on the Data Definitions page of the Platform web app. This page is always kept up-to-date and reflects the current structure of the Censys Platform web property dataset.

Web properties offer insight into HTTP services beyond layer 7 while abstracting away HTTP protocol semantics.

• Web properties contain all records that correlate to HTTP-based scans.
• A web property can have one or more endpoints, which serve as distinct entry points to different resources or functionalities within the web property.
  • Each endpoint may be associated with specific applications, services, or APIs and can be individually monitored and analyzed.
• Web properties support deep scan information for HTTP-based scanners.

As with hosts, CPE values are provided for software on web properties.

HTTP fields

Web property records surface information about individual HTTP endpoints. This includes:

• HTTP header and body information (web.http.headers and web.endpoints.http.body, as well as hash fields).
  • Note that only the first 2 kilobytes of HTTP headers and bodies are indexed and searchable in the Platform.
• Software that runs on top of HTTP, like pprof (web.endpoints.pprof), Kubernetes (web.endpoints.kubernetes), and Elasticsearch (web.endpoints.elasticsearch).
• Endpoints located at paths (web.endpoints.path) beyond the root / level, such as /account/login, /admin/login.html, /admin/login.asp, /owa/auth/logon.aspx, and so on.

Web properties also contain contextual information about the asset's associated software, hardware, certificates, and more.

Web property CVE and vulnerability data

Users on the Enterprise plan have access to Common Vulnerabilities and Exposures (CVE) data for web properties in the web.vulns object. This information includes CVE IDs (web.vulns.id), CVSS scores (web.vulns.metrics.cvss_v40.score and others), whether the vulnerability is present in the Known Exploited Vulnerability (KEV) catalog (web.vulns.kev), and many other vulnerability data fields.

The Data Definitions page contains a complete list of CVE and vulnerability data for web properties.

📘

Note

Endpoints in the Censys datasets can have a maximum of 50 CVEs.

CVE data in the Platform web UI

Web properties that are associated with CVEs have an alert icon on their web property preview cards.

A detailed list of all the of CVEs present on a web property is shown on the CVEs tab.

Additionally, all vulnerability data for for a web property is available in the raw data output.

CVE search result filters

If you have access to CVE data, CVE-related information like CVE ID and CVSS score are available in the quick filters in the left-side toolbar for search result pages.

Example queries for vulnerability data

Use the following queries to explore vulnerability data in the Platform.

Web properties and host services

Web properties offer better visibility into internet assets than services on hosts in certain circumstances.

In the Censys Platform, consider searching web properties when:

• You want results that include hostnames.
• You are targeting software that runs on top of HTTP.
• You are targeting software that services HTTP, like Apache or nginx.
• You need HTTP body information.
• You need data from endpoints other than /.

Do not use web properties when:

• You want results that include IP addresses.
• You are searching for DNS data, WHOIS data, location data, or routing data.
• You are searching for hosts serving HTTP as well as non-HTTP protocols.

Web property threat context

All users on the Enterprise plan can view some web property threat data, including the following data fields.

This information is visible on web property records in the Platform web console and is retrievable via API.

To see complete threat metadata and search and pivot across threat data, you must have access to the Threat Hunting module.

Web property software context

Platform Enterprise customers have access to additional data about web property software. This includes information about software components (web.software.components) as well as life cycle information (web.software.life_cycle and web.software.components.life_cycle).

You can use component information to find software that has potentially vulnerable components. For example, you can run the query web.software: (components.product = "nginx" and not product = "nginx") to find web software that is using nginx components but is not itself detected as nginx.

Web property software confidence

Software confidence in the Censys Platform measures how reliably Censys identifies web endpoint software and versions on assets. Confidence scores range from 0 (low) to 1 (high) and help users assess the likelihood of false positives in detected software fingerprints.

For web property software, the confidence level is provided in the web.software.confidence field. Exact matches of known, structured fields such as exact favicon hashes, HTTP body hashes, or banner content hashes may yield a high confidence value, whereas HTTP endpoint regex matches, body regex matches, or single-field keyword-based recognition may yield a low to moderate confidence value.

Confidence levels do not currently impact CVE (vulnerability) scoring. Software confidence fields are available exclusively to Enterprise-tier users.

Labels

Labels are tags applied to assets and services by Censys to help you quickly identify items and filter search results. Label values are always in screaming snake case.

Labels on web properties (web.labels.value) are used to categorically describe the content and services present on them, just like they are for host services. You can see a complete list of web property labels by running a query for web.labels.value and then generating a report broken down by that field.

Many web property labels are the same as those used for host services and include:

• DEFAULT_LOGIN_PAGE
• LOGIN_PAGE
• OPEN_DIRECTORY
• WAF

Example web property record

The following excerpt is an example of an empty web property record to provide a general idea of the data structure.

  "web": {
    "port": "",
    "scan_timed": "",
    "hostname": "",
    "labels": [],
    "endpoints": [
      {
        "endpoint_type": "",
        "path": "",
        "banner": "",
        "banner_hash_sha256": "",
        "banner_hex": "",
        "transport_protocol": "",
        "<endpoint specific details>": {}
      }
    ]
  },
  "misconfigs": [],
  "exposures": [],
  "vulns": [],
  "software": [],
  "hardware": [],
  "operating_system": [],
  "tls": []
}