Showing posts with label Python. Show all posts
Showing posts with label Python. Show all posts

Friday, April 1, 2011

Ubuntu use Python 2 and Python 3 together

SkyHi @ Friday, April 01, 2011
sudo apt-get install python
sudo apt-get install python3



Ubuntu deliberately packages python3 in such a way it won't interfere with python2.

To run your programs, you'll type "python3" at the command line instead of just "python".

Monday, March 21, 2011

AttributeError: 'module' object has no attribute 'findall'

SkyHi @ Monday, March 21, 2011
Code:
d@home$ cat re.py
import re
re_string = "{{(.*?)}}"
some_string = "this is a string with {{words}} embedded in {{curly brackets}} to show an {{example}} of {{regular expression}}"
for match in re.findall(re_string,some_string):
        print 'Match->', match


ERROR:
Traceback (most recent call last):
  File "re.py", line 1, in <module>
    import re
  File "/home/nikid/PythonSysadmin/re.py", line 4, in <module>
    for match in re.findall(re_string,some_string):
AttributeError: 'module' object has no attribute 'findall'
Error in sys.excepthook:
Traceback (most recent call last):
  File "/usr/lib/python2.6/dist-packages/apport_python_hook.py", line 48, in apport_excepthook
    if not enabled():
  File "/usr/lib/python2.6/dist-packages/apport_python_hook.py", line 21, in enabled
    import re
  File "/home/nikid/PythonSysadmin/re.py", line 4, in <module>
    for match in re.findall(re_string,some_string):
AttributeError: 'module' object has no attribute 'findall'

Original exception was:
Traceback (most recent call last):
  File "re.py", line 1, in <module>
    import re
  File "/home/nikid/PythonSysadmin/re.py", line 4, in <module>
    for match in re.findall(re_string,some_string):
AttributeError: 'module' object has no attribute 'findall'




Solution:

rename the file re.py to another name


Another one

ERROR:

NameError: name 'randint' is not defined

Solution:
Rename the random.py to different name..

REFERENCES
http://www.willmer.com/kb/2007/12/attributeerror-module-object-has-no-attribute-blah/

Sunday, August 8, 2010

Learn Python The Hard Way

SkyHi @ Sunday, August 08, 2010

This is the site for the book "Learn Python The Hard Way". The book is a very beginner book for people who want to learn to code. If you can already code then the book will probably drive you insane. It's intended for people who have no coding chops to build up their skills before starting a more detailed book.



For Learners



You can download the book here:





The book is very simple:



  • 52 exercises in all.
  • 26 cover just input/output, variables, and functions.
  • 26 cover logic (boolean algebra, if-statements, while-loops, etc.)


Each exercise is one or two pages and follows the exact same format. You type each one in (no copy-paste!), make it run, do the extra credit, and then move on. If you get stuck, at least type it in and skip the extra credit for later.



Other Books



You might also want to check out these other books if you find this book too boring or annoying:





For Potential Contributors



The book is currently a work in progress, but I'm looking for people to contribute proposed lessons using this wiki. You can also submit tickets with errors you find.



This repository is also available as a fossil repository. Fossil is a distributed version control that includes the wiki, tickets, source, and everything you need to get the whole site. You can use fossil to grab the whole thing and access it offline and contribute to the book.



Writing Proposed Exercises



Use this wiki to start writing your own exercises. You should get the latest version of the book and read it. You should then try the exercises yourself so you get a feel for them.



Next you just write an exercise on the Proposed Exercises page. Just add your exercise as a bullet point with a link and then write it. I leave it to you to figure out how to find the wiki formatting and make the page. If you can't then you probably shouldn't be writing exercises right now.



Finally, make sure your name is on the Exercise so that we know who to blame...credit for its creation.


REFERENCES
http://learnpythonthehardway.org/index




Wednesday, June 2, 2010

Parse log file with most accessed ip

SkyHi @ Wednesday, June 02, 2010

1. grep "30/May" access_log > access_log.1
2. cut -d' ' -f1 access_log.1 > access_log.2
3. sort access_log.2|uniq -c|sort -n


Live:
netstat -ntu |awk '{print $5}'| cut -d: -f1 | sort | uniq -c |sort -nr

Find the most accessed ip
awk '{print $1}' 10stillhack.txt |cut -d: -f1 | sort | uniq -c |sort -nr|more

def CalculateApacheIpHits(logfile_pathname):
    IpHitListing = {}
    Contents = open(logfile_pathname, "r").xreadlines( )
    for line in Contents:
        Ip = line.split(" ")[0]
        if 6 < len(Ip) <= 15:
            IpHitListing[Ip] = IpHitListing.get(Ip, 0) + 1
    return IpHitListing

def TimeSpan(logfile_pathname):
    Dates = open(logfile_pathname, "r").readline( )
    Last  = open(logfile_pathname, "r").readlines( )
    firstdate = Dates.split(" ")[3]
    lastdate = Last[len(Last)-1].split(" ")[3]
    print ""
    print "Log covers the dates of: " + firstdate[1:] + " - " + lastdate[1:]
    print ""

TimeSpan("access.log.19")

HitsDictionary = CalculateApacheIpHits("access.log.19")
width = 10
ip = "IP Address"
hits = "Hits"
print '%15s         %5s' % (ip, hits)
print '     ----------          ----'
for key in HitsDictionary.keys():
    print '%15s    ->   %5s' % (str(key), str(HitsDictionary[key]))


Kill all process:
#ps ax -o user,pid |grep 'postfix' |awk '{print $2}' |xargs -l
#ps ax -o user,pid |grep 'postfix' |awk '{print $2}' |xargs -r kill -9
#killall -9 PROCESS

Apache process memory usage
ps -ylC httpd --sort:rss

sort the top entries in your access.log
awk '{print $1}' /var/log/http/access_log | sort |uniq -c |sort -n
netstat -ta |grep ESTABLISHED
sar -w
sar -I SUM
sar -d 5 0
REFERENCES
http://linuxgazette.net/123/vishnu.html
http://www.devside.net/articles/apache-performance-tuning
http://serverfault.com/questions/231940/high-linux-loads-on-low-cpu-memory-usage

Monday, August 17, 2009

Extract email addresses from big file

SkyHi @ Monday, August 17, 2009
1. import .pst into outlook
2. export the bounce folder to .excel
3. extract the bounce From address into linux



grep -C 2 "fatal errors" nurseaug3.txt > nurseaug3a.txt

perl -wne'while(/[\w\.\-]+@[\w\.\-]+\w+/g){print "$&\n"}' emails.txt | sort -u > output.txt



find . -name "*.txt" | xargs perl -wne'while(/[\w\.\-]+@[\w\.\-]+\w+/g){print "$&\n"}' emails.txt | sort -u > output.txt





Also see:
Reference:
1.http://lifehacker.com/391205/email-address-extract-grabs-addresses-from-any-file



2. Yeah, that's just a perl script wich takes in a file and checks every word to see if it's a valid email address. It prints it out if so. Here's a commented version:

#!/usr/local/bin/perl -w
use strict;
# that stuff is just to make it a perl script

# email::Valid is a module to check for valid email addresses
# you can get it from CPAN.org along with tons of other modules
# If you're using perl on windows, i bet activestate has a version.
# The author says that it may be slow on Win32 if you have addresses
# where there is no nameserver to check them against.
use email::Valid;

# this loops over each line in the input
while (<>) {
# this loops over each "word" in the line (it splits on whitespace)
for my $word ( split() ) {
# if it's a valid address..
if ( my $address = email::Valid->address( $word ) ) {
# print it out.
print $address, "\n";
}
}
}

Put it in a file and call the file "getemails.pl" or something, then send all of your files to it:
./getemails.pl < somefile.txt or cat * ¦ ./getemails.pl and wait for your list of emails to come out. I just tested it and it seems to do pretty well. -Andy


#!/usr/bin/env python
'''
  emailsfromfile.py -- Get all unique email addresses from a file

  by Patrick Mylund Nielsen
  http://patrickmylund.com/projects/emailsfromfile/

  License: WTFPL (http://sam.zoy.org/wtfpl/)
'''

__version__ = '1.1'

import sys
import os
import re
import codecs

# Regular expression matching according to RFC 2822 (http://tools.ietf.org/html/rfc2822)
rfc2822_re = r"""(?:[a-z0-9!#$%&'*+/=?^_`{|}~-]+(?:\.[a-z0-9!#$%&'*+/=?^_`{|}~-]+)*|"(?:[\x01-\x08\x0b\x0c\x0e-\x1f\x21\x23-\x5b\x5d-\x7f]|\\[\x01-\x09\x0b\x0c\x0e-\x7f])*")@(?:(?:[a-z0-9](?:[a-z0-9-]*[a-z0-9])?\.)+[a-z0-9](?:[a-z0-9-]*[a-z0-9])?|\[(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?|[a-z0-9-]*[a-z0-9]:(?:[\x01-\x08\x0b\x0c\x0e-\x1f\x21-\x5a\x53-\x7f]|\\[\x01-\x09\x0b\x0c\x0e-\x7f])+)\])"""
email_prog = re.compile(rfc2822_re, re.IGNORECASE)

def isEmailAddress(string):
    return email_prog.match(string)

def main(filename, separator='\n', encoding=None):
    separator_replace = {
        'space': ' ',
        'newline': '\n',
    }
    if not os.path.isfile(filename):
        raise IOError("%s is not a file." % filename)
    results = set()
    with codecs.open(filename, 'rb', encoding) as f:
        for line in f:
            results.update(email_prog.findall(line))
    for k, v in separator_replace.iteritems():
        separator = separator.replace(k, v)
    print(separator.join(results))

if __name__ == '__main__':
    args = len(sys.argv) - 1
    if 0 < args < 4:
        main(*sys.argv[1:])
    else:
        print("Usage: python %s <filename> [separator] [encoding]" % sys.argv[0])
        print("The default separator is a newline. To separate by space, literally enter 'space' as the separator.")

Usage

python emailsfromfile.py [separator] [encoding]

The separator and encoding parameters are optional. The separator is a new line and the file encoding is 8-bit ASCII by default. If you want to specify an encoding, you also have to set a separator; to use a new line (the default), specify newline as the separator.

Examples:

python emailsfromfile.py contacts.csv — returns all email addresses from contacts.csv, displaying one email address per line
python emailsfromfile.py contacts.csv , — returns a comma-separated list of all email addresses in contacts.csv
python emailsfromfile.py contacts.csv space — returns all email addresses from contacts.csv, separated by a space
python emailsfromfile.py contacts.csv ; > emails.txt — writes all of the email addresses from contacts.csv, separated by a semi-colon, to emails.txt
python emailsfromfile.py utf8-contacts.csv newline utf-8 — returns all email addresses, one per line, from the UTF-8 encoded file utf8-contacts.csv

References:
http://www.webmasterworld.com/forum10/1195.htm
http://patrickmylund.com/projects/emailsfromfile/