Good tools for doing reverse IP and geolocation out of a web server log?

Folks: Not everyone on the planet is cool enough to use Google Analytics. What if you have an old-school HTTP server log and want to get more information about users, especially hostnames and geolocations? What are the most reasonable tools these days, either desktop Windows apps or Unix server-based? I don’t need something scriptable that can run every night.

Alternatively, given a list of IP addresses, what would you do if you wanted to turn that into a CSV file of IP, hostname, location?

9 thoughts on “Good tools for doing reverse IP and geolocation out of a web server log?

  1. Maxmind as said above or route through a CDN which will add the information for you. Visualization is easy with a third party logging tool like Logs.IO (ELK based)

  2. MaxMind Geolite (free) or GeoIP2 (paid, better coverage) for the database

    GoAccess can use MaxMind, and works in both terminal and web-server modes. You can also use it to do a “top” style running analysis of live log files.
    https://goaccess.io/

    Alternatively if you prefer using SQL to query logs, I have a 255-line Go program I use to convert Apache CLF files into tab-separated files (with geo lookup) suitable for import into PostgreSQL (or more precisely CitusDB, a MPP extension thereof). It needs to be adapted for the specific format used by the web server, as no two servers have exactly the same format string and my tool is too rudimentary to support something as fancy as parsing format strings.

  3. There’s Weblog Expert which has both free and commercial editions.
    http://www.weblogexpert.com/

    The biggest problem is filtering out bots, though. That’s basically impossible today without a continually updated web service (such as Akismet), or else requiring JavaScript code (such as Google Analytics).

    I once tried to do my own referrer spam filtering in conjunction with Weblog Expert but quickly gave up on the attempt. The majority of all Internet traffic is garbage today, so your raw server logs won’t be very useful except to profile bot attacks.
    http://kynosarges.org/ReferFilter.html

  4. For a simple manual search I use the https://abongo.com/ enter the IP or domain name (sans “httpX://”) and press WhoIs button, shows registration info and a map.

    The http://www.all-nettools.com/toolbox/smart-whois.php is a well known site with a lot of online tools and they offer also software, many are shareware but e.g. the “Free IP Tools 4.2” is free and has plenty of tools. This site has always been safe, no badware found there ever (even then I always check each installation packages, that are smaller than their max limit that is 20MB, before installing here: https://www.virustotal.com/#/home/upload they run the files through nearly all of the virus checkers for free).

    In these days, however, one can not draw absolute correct conclusions based on the IP address or domain names because many people are using VPN tubing such as the https://www.f-secure.com/en/web/home_global/freedome (I’m not affiliated with them, that product just is affordable, fast and has good coverage and plenty of output nodes). People who seem to be posting from different IP address each time most likely are using a VPN tube.

Comments are closed.