Internet Scanning

Censys provides the most complete view of the entire public-facing internet. Understanding what Censys scans will help you understand the dataset.

📘

Note

Censys conducts scans solely to gather information. It never attempts to log into services, access databases, or gain authenticated entry to any system.

What Censys scans

Censys continually scans the entire public IPv4 address space using automatic protocol detection, examining all possible IP and port combinations. The results accurately represent the Internet’s current state.

Censys also leverages redirects and the Domain Name System to discover and scan in-use IPv6 addresses.

Scan frequency

Discovery means finding a service on an IP/port that was not there the last time Censys looked. Censys continuously scans for discovery using the following methods.

• Global scan of popular ports. Censys continually and rapidly scans the whole IPv4 space on over 100 ports with Internet Assigned Numbers Authority (IANA-assigned) services.
• Cloud provider scans. Our accelerated cloud docket contains a list of IPs and common ports for AWS, Azure, and GCP cloud hosting providers. It is optimized to scan each item at least once daily, sometimes more.
  Using current and historical data from the services on these dockets and other data in our dataset, Censys continuously generates dynamic scanning targets.
• Predictive scanning engine. Using both the current and historical data from the services found via Global and Cloud Provider scans, Censys generates dynamic scanning targets on a continual basis using the Predictive Scanning engine. The methods described above essentially act as seeds as the input to this engine which can be used to more accurately find and predict the best possible IPs and ports to scan. Predictive scanning accounts for over 40% of all services Censys finds on the Internet today.
  For more information on how Predictive Scanning works, see the Censys blog: Raising the Bar on Internet Coverage: Predictive Scanning Takes The Censys Internet Map to the Next Level.
• IPv4 Internet scan. Censys continuously scans all 65,535 ports on the IPv4 Internet with a port walk to gather as much background information as possible across all possible IP and port combinations.

Data refresh cadence for known services

After a service is discovered, Censys prioritizes refreshing the information about that service to ensure it is accurate and up to date.

Every day, the age of each of the over 3 billion services in our data set is checked. Any (unnamed) service with an observation timestamp older than 24 hours is rescanned. With this process, the average age of high-value service data is about 16 hours.

Censys scan methodology

Censys includes multiple global perspectives and sophisticated scanning techniques to produce the richest, most useful data set for the security community.

Global scanning perspectives

Censys scans from multiple geographically diverse datacenters to produce nearly 99% coverage of listening hosts across the globe with enhanced protection against packet drop.

Deep protocol scans

On ports with IANA-assigned protocols, Censys tries to complete a handshake with the assigned protocol (for example, Telnet on port 23). If that fails, Censys tries additional handshakes based on our experience with protocol and port pairings.

Predictive scanning

Predictive scanning enhances the Censys Internet Map by covering 65,535 ports across all possible IP and port combinations. It also gives security teams better visibility into all 65,535 ports, enabling faster service detection. Overall, predictive scanning adds over 107 million new services to the nearly 3 billion global Internet services that Censys continuously monitors.

Predictive scanning accounts for over 40% of all services Censys finds on the Internet today.

Predictive scanning use cases

Predictive scanning typically discovers services such as:

• Services from the Internet of Things (IoT), which businesses are leveraging for growth but also present high risk due to lagging security standards.
• Autonomous Systems only run services on non-standard ports that attackers might use to host malicious infrastructure and hide from scanners.
• Vendors are massively proliferating newer services, such as online portals, data analysis tools, and business productivity enhancers. These services are especially popular in hybrid and remote environments, which typically run on high, non-standard ports.
• Attackers may hide services on non-standard ports to try and obfuscate their activities.

Automatic protocol detection

The Censys scanner analyzes every server response to identify its service, even if it’s non-standard for the port. This allows us to uncover the vast majority of services in unexpected places.

For example, if an HTTP request results in an SSH banner, Censys closes the HTTP connection and reattempts an SSH handshake.

Censys can detect many protocols on any port.

Protocols detected by Censys

Censys can detect and complete scans for over 200 Layer 7 protocols. Our scanners use TCP as their default Layer 4 protocol. Some protocols, such as DNS, are scanned with UDP. HTTP can be detected over QUIC.

Service names are the most specific service information Censys has. For example, a generic HTTP service has a service name of HTTP, while an HTTP service that’s actually an ElasticSearch server has a service name of ELASTICSEARCH.

Censys also uses an UNKNOWN fallback, which means that it cannot identify the protocol used by an open service. This can be because the service does not adhere to a protocol (there are many HTTP-like services) or because the protocol is very uncommon and Censys does not yet have a protocol-specific scanner written for it.

For complete protocol information, please use the metadata API endpoint.

More information about Censys data collection by protocol category is provided below.

Ports scanned by Censys

As part of its scheduled discovery process, Censys conducts ongoing Internet-wide scans of all 65,535 ports.