Monday, December 7, 2009

IP address regex example

SkyHi @ Monday, December 07, 2009

Finding an IP address in text or string with python is a simpler task than in java. Only the regex is not shorter than in the java regex example!

First an example with python: build a RegExp-Object for faster matching and than loop over the result iterator.

  1. import re
  2. logText =  'asdfesgewg alkejo 234 oij8982jldkja.lkjwech . kadfjeladfjeladkj'
  3. bytePattern = "([01]?\d\d?|2[0-4]\d|25[0-5])"
  4. regObj = re.compile("\.".join([bytePattern]*4))
  5. for match in regObj.finditer(logText):
  6.     print

A regex like /\d+\.\d+\.\d+\.\d+/ wont work, because there match "999.999.111.000" too. But for the usage in python - that is it! Using a regular expression is more native in python than in java. Or in javascript or in perl or

And how to find it with JavaScript?

It looks like the small python example. Build the RegExp-Object for faster matching and a loop for finding all.

  1. var logText = 'asdfesgewg alkejo 234 oij8982jldkja.lkjwech . kadfjeladfjeladkj';
  2. var regObj = new RegExp("(?:(?:25[0-5]|2[0-4][0-9]|1[0-9]{2}|[1-9][0-9]|[0-9])\.){3}(?:25[0-5]|2[0-4][0-9]|1[0-9]{2}|[1-9][0-9]|[0-9])");
  3. while (var result = regObj.exec(logText)) {
  4.    alert("Matched: `" + result[0]);
  5. }

For a more detailed example have a look at

the native playground : perl

  1. $txt = "asdfesgewg alkejo 234 oij8982jldkja.lkjwech . kadfjeladfjeladkj";
  2. while( $txt=~/(([01]?\d\d?|2[0-4]\d|25[0-5])\.([01]?\d\d?|2[0-4]\d|25[0-5])\.([01]?\d\d?|2[0-4]\d|25[0-5])\.([01]?\d\d?|2[0-4]\d|25[0-5]))/g) {
  3.   print $1."\n";
  4. }

Some regex basics can be found at and more regex examples on perl cookbook.

a more complex script: IP - log file statistic

But only finding an IP address in some irregular text is not a common use-case. An apache logfile is well formatted and the IP part can be found directly.

Do a simply split every line and the first part should be the IP address. To check if the first element match the IP pattern, use a ready function (for the example from the socket module). As addition and much more complexer example I will try to find the TOP10 IPs with the request count from the logfile.

  1. import re, socket
  2. hits = {}
  4. # - - [13/Jul/2009:01:06:38 +0200] "GET /index.html HTTP/1.1" 400 335 - "-" "-" "-"
  5. try:
  6.     fp = file("all.log")
  7.     for line in fp:
  8.         elements = re.split("\s+", line)
  9.         try:
  10.             socket.inet_aton(elements[0])
  11.             hits[elements[0]] += 1
  12.         except KeyError:
  13.             hits[elements[0]] = 1
  14.         except socket.error:
  15.             pass # no ip in the starting logline
  16. finally:
  17.     fp.close()
  19. #Sorting the IPs with the hit-count
  20. ipKeys = hits.keys()
  21. ipKeys.sort(lambda a,b: hits[b]-hits[a])
  22. for ip in ipKeys[:10]:
  23.     print "%10dx %s" % (hits[ip], ip)

The result looks like this and runs 0.7 sec for 10.000 lines logfile (on Intel Atom N270).

  1.       1406x
  2.       1291x
  3.        937x
  4.        569x
  5.        302x
  6.        260x
  7.        164x
  8.        125x
  9.        113x
  10.        104x

The first block of the real addresses has been replaced with the number 10 for anonymity.

And the same IP script as command line with perl

The python script could made shorter and more ugly. Finding all IPs, sorting it, counting and print the TOP10 IPs.

  1. perl -wlne 'print $1 if /(([01]?\d\d?|2[0-4]\d|25[0-5])\.([01]?\d\d?|2[0-4]\d|25[0-5])\.([01]?\d\d?|2[0-4]\d|25[0-5])\.([01]?\d\d?|2[0-4]\d|25[0-5]))/' all.log |sort|uniq -c | sort -n -r|head -n 10

The perl/shell example needs only 0.2 seconds for the same logfile.


I like to use regex and python for getting some statistics, but the usage of some command line unix tool is useful too!

Trackback URL for this post:


Anonymous's picture

The technique you use in the python code works just as well in the perl or javascript code. For example:

perl -wlne '$re = join(qr/\./, (qr/([01]?\d\d?|2[0-4]\d|25[0-5])/) x 4); print $1 while /($re)/g' |sort|uniq -c | sort -n -r|head -n 10

Also changed the "print $1 if /.../" to "print $1 while /.../g" to better match the earlier examples (in the case that multiple IP addresses appear on the same line).

Christian Harms's picture

Thanx, the improved example for perl - it works for other languages - too!

Anonymous's picture

>>> bytePattern = "foo"
>>> [bytePattern for x in range(4)]
['foo', 'foo', 'foo', 'foo']
>>> [bytePattern] * 4
['foo', 'foo', 'foo', 'foo']

Christian Harms's picture

Thanx - that's shorter and I edit the example.

Anonymous's picture

And BTW, your regex strings should be raw strings, i.e. r"...". Else you walk on thin ice regarding which characters are consumed as an escape sequence and which are not.

Anonymous's picture

And finally, you can avoid the KeyError mumbo-jumbo by using a defaultdict:

>>> from collections import defaultdict
>>> h = defaultdict(int)
>>> h[""] += 1
>>> h[""] += 1
>>> h
defaultdict(, {'': 2})

Anonymous's picture

666.666.666.666 is correct Ip address with your pattern in python?

Christian Harms's picture

No - the regex example matches only on correct IPs. And the python example use the socket module to check the correct ips. Try this out:

  1. >>> import socket
  2. >>> socket.inet_aton("666.666.666.666")
  3. Traceback (most recent call last):
  4.   File "<stdin>", line 1, in <module>
  5. socket.error: illegal IP address string passed to inet_aton
Anonymous's picture

if you'd like to parse URLs (with hostnames or IPs) in JavaScript you can try this by the following link.

Anonymous's picture

Such a scripting of find out IP address can be done in Java. It is very easy. You can use the protocol of IP address like ARP and RARP directly.