Reverse IP Address Lookup

In this blog post we will try to introduce you to various concepts around IP, domains and how to find all domains that are on the same IP as some other domain A.

Then we discuss how to make a reverse IP address lookup.

Let us first look into what is IP address.

What is an IP address? An IP address is a unique number for computers, devices and servers connected to the internet. It’s like your home address on the global map of the internet. It helps everyone find you online. So if you want to reach your server, you have to type its IP address into your browser address bar. Each computer has its own unique IP address. The computer without IP is like an island without access to information in the rest of the world!

An IP address is comprised of four numbers in the range 0-255, separated by periods. Each portion of the IP address is known as an octet (or octad) – hence the term “dotted-decimal”.

IP addresses are the unique identifiers we use to send messages across the internet. They come in two flavors: IPv4 and IPv6.

Next, let us turn to domains.

What is domain name?

A domain name is a unique identity for internet users to access a certain server. For example, oreilly.com is the domain name used by the O’Reilly media company to publish books, magazines and digital materials. If you have such a requirement, then domain name registration may be suitable for you.

Dot-com domains like .com, .net, and .org are the original generic top-level domains (gTLDs), so they’re by far the most popular. But it’s not over yet! There are a whole new generation of domains called new gTLDs that you could still register today. New gTLDs are written in lowercase and have a wide range of purposes. Some of them relate to specific industries or professions like .healthcare, .law, and .architect. Others are broader terms, such as local communities or destinations, like .nyc, .earth, and .wales.

Having defined what IP addresses are and what domains are, let us now look into another question.

How to find out if there are other domains on the same address. This is what is called Reverse IP Address lookup.

There are many services which offer this. Usually they cover hundreds of millions of IPs.

Reverse IP lookups have many uses. They can be e.g. used to find groups of trending domains or products on the same network.

 

Website categorization API and use cases

Why do we need website categorization?

Limiting access to risky websites is a cybersecurity best practice, but many enterprise cybersecurity solutions leave vulnerability gaps — using unreliable data, methods that are easily evaded, or simply a lack of adequate coverage. With the right kind of website categorization tools and policy enforcement, intelligence can be gathered on the types of websites your employees are visiting, and you can use this information to uncover hidden threats. In today’s world of heightened digital risk, it’s important to further your approach beyond prevention and discover where your organization is vulnerable so you can eliminate those risks from within.

Website categorization is important in the fight against corporate, private and government cyber-security breaches. For example, to properly prepare for a cyber-attack, one must first understand which websites employees are using. This allows one to mitigate threats with actionable data before malicious sites can infect internal systems with malware or allow unauthorized access through an SQL injection.

Whether an organization is strict or lenient about what its employees can do online, security and IT teams often need assistance in detecting when a user is going to a website that isn’t related to business needs. Categorizing every website that exists is impossible for most security teams, and so security services and products must do it for them.

You need to block employees from spending time on non-work related websites instead of looking at cat videos or reading the news. You apply web content filtering, but then you run into a problem: how do you know when an employee is going to a website that is not work-related? Fortunately, there are services available to help automatically categorize websites for you.

Categorization definitions

The category of a website is subjectively determined. One web content filter may classify a site as Financial Services, while another classifies it as Finance. There are many different services that say that they can help organizations categorize every site on the internet. In reality, this is not possible. That’s why you need to have a solution that allows you to categorize websites for your organization without being tied to someone else’s categorization system.

Here is an example of a set of website categories that can be used for content filtering done in a company:

  • Malicious
  • Phishing
  • DDNS
  • Proxies
  • Drugs that are prohibited
  • Gambling activities
  • SE
  • Looking for jobs
  • Copyright problematic
  • Legal issues
  • Adult content
  • Downloading
  • Music playing
  • News browsing
  • Sports websites
  • Games
  • Shopping activities

What is Website categorization?

Website categorization (also known as website classification or URL classification) is a means for companies to classify sites they access frequently under different umbrellas for marketing, cybersecurity, and brand protection purposes. Examples of website categories include entertainment, shopping, games, and more.

Cybersecurity and website categorization go hand in hand. With the many types of websites being accessed on a daily basis by millions of end users across the world, it is important for organizations to monitor the types of sites that employees frequently visit. Identifying these sites and exploring who is visiting them helps to build a complete picture of what is happening in the web browser environment. This is where cybersecurity comes into play.

An interesting subset of text classification models is categorization of products, e.g. in ecommerce domain. This can help online stores improve discoverability on their websites.

Useful resources and libraries for website categorization:

Yarn package for website categorization

Useful npm trends package

 

Data Science Consultancy

The last decade has seen a steep rise in the importance and application of data science.

There are several drivers behind this growth of data science consulting importance.

Surge in amount of data

With mobile devices and internet, the amount of new data produced each years is surging. Here are some interesting statistics:

https://www.weforum.org/agenda/2019/04/how-much-data-is-generated-each-day-cf4bddf29f/

Each day, 500 million tweets are published,294 billion emails are sent,4 terabytes of data are created per connected car, etc.

Organizations have a wealth of data available and they are using it to produce valuable insights and drive their business decisions.

Rise of computing power

All of this data would be more or less unusable if we did not have powerful computers to automate their analysis. A CPU in today’s mobile device is by magnitudes more powerful than a 386 DX-40 PC from 1991. The rise of computation power has been a steady presence for decades, leading to empirical rule known as Moore’s law. Although in recent years there has been some slowing down in terms of CPU frequencies, this was alleviated by processors getting an increasing number of cores. See e.g. recent AMD power processors for example.

Deep learning

The third important driver behind rise of data science is that the combination of above – more data, better processors led to some methods become stunningly successful in predictions. I am talking about deep learning algorithms. The interesting thing is that many of these algorithms have been around for a relatively long time. E.g. one of the most famous ones – LSTM was invented in the 1990s. However, it was only the huge amounts of data from mobile devices and improvement in computing power which made these methods extremely successful in image detection and other tasks.

Once the first successes became important in businesses, driven especially by companies such as Google, Amazon and others, a lot of other companies also adopted the new trend of applying data science to its business problems.

As the skills demanded by the data science are rather unique what happened soon was shortage in data scientists and rise in our salaries. As a consequence, data scientist is near the top in recent years in salary charts. But not only that, a lot of us, data scientist consultants are highly satisfied with the jobs we are doing. They are intellectually demanding and interesting.