Wednesday, March 23, 2011

top - Linux command line to see the running processes in real time

SkyHi @ Wednesday, March 23, 2011
When you need to see the running processes on your Linux in real time, you have top as your tool for that.
top also displays other info besides the running processes, like free memory both physical and swap
Usage
top [options]
Options
-d ss.tt
Delay -- Specifies the seconds and tenths of seconds of delay between the updates of the info showed on the screen, being the default 3 seconds
-i
Starts top with the last remembered 'i' state reversed. When this toggle is Off, tasks that are idled or zombied will not be displayed.
-n n
Specifies the maximum number of iterations, or frames, top should produce before ending.
-p n
Monitor only processes with specified process IDs. This option can be given up to 20 times, or you can provide a comma delimited list with up to 20 pids. Co-mingling both approaches is permitted. This is a command-line option only. And should you wish to return to normal operation, it is not necessary to quit and and restart top -- just issue the '=' interactive command.
-s
- Secure - Runs top in secure mode, restricting the commands you can use while top is running even for root
-S (Sum)
Starts top with the last remembered 'S' state reversed. When 'Cumulative mode' is On, each process is listed with the cpu time that it and its dead children have used. See the 'S' interactive command for additional information regarding this mode.
Description of the fields
a: PID -- Process Id
The task's unique process ID, which periodically wraps, though never restarting at zero.
b: PPID -- Parent Process Pid
The process ID of a task's parent.
c: RUSER -- Real User Name
The real user name of the task's owner.


d: UID -- User Id




The effective user ID of the task's owner.


e: USER -- User Name




The effective user name of the task's owner.


f: GROUP -- Group Name




The effective group name of the task's owner.


g: TTY -- Controlling Tty




The name of the controlling terminal. This is usually the device (serial port, pty, etc.) from which the process was started, and which it uses for input or output. However, a task need not be associated with a terminal, in which case you'll see '?' displayed.


h: PR -- Priority




The priority of the task.


i: NI -- Nice value




The nice value of the task. A negative nice value means higher priority, whereas a positive nice value means lower priority. Zero in this field simply means priority will not be adjusted in determining a task's dispatchability./dd>


j: P -- Last used CPU (SMP)




A number representing the last used processor. In a true SMP environment this will likely change frequently since the kernel intentionally uses weak affinity. Also, the very act of running top may break this weak affinity and cause more processes to change CPUs more often (because of the extra demand for cpu time).


k: %CPU -- CPU usage




The task's share of the elapsed CPU time since the last screen update, expressed as a percentage of total CPU time. In a true SMP environment, if 'Irix mode' is Off, top will operate in 'Solaris mode' where a task's cpu usage will be divided by the total number of CPUs. You toggle 'Irix/Solaris' modes with the 'I' interactive command.


l: TIME -- CPU Time




Total CPU time the task has used since it started. When 'Cumulative mode' is On, each process is listed with the cpu time that it and its dead children has used. You toggle 'Cumulative mode' with 'S', which is a command-line option and an interactive command. See the 'S' interactive command for additional information regarding this mode.


m: TIME+ -- CPU Time, hundredths




The same as 'TIME', but reflecting more granularity through hundredths of a second.


n: %MEM -- Memory usage (RES)




A task's currently used share of available physical memory.


o: VIRT -- Virtual Image (kb)




The total amount of virtual memory used by the task. It includes all code, data and shared libraries plus pages that have been swapped out.

VIRT = SWAP + RES.

p: SWAP -- Swapped size (kb)




The swapped out portion of a task's total virtual memory image.


q: RES -- Resident size (kb)




The non-swapped physical memory a task has used.
RES = CODE + DATA.

r: CODE -- Code size (kb)




The amount of physical memory devoted to executable code, also known as the 'text resident set' size or TRS.


s: DATA -- Data+Stack size (kb)




The amount of physical memory devoted to other than executable code, also known as the 'data resident set' size or DRS.


t: SHR -- Shared Mem size (kb)




The amount of shared memory used by a task. It simply reflects memory that could be potentially shared with other processes.


u: nFLT -- Page Fault count




The number of major page faults that have occurred for a task. A page fault occurs when a process attempts to read from or write to a virtual page that is not currently present in its address space. A major page fault is when disk access is involved in making that page available.


v: nDRT -- Dirty Pages count




The number of pages that have been modified since they were last written to disk. Dirty pages must be written to disk before the corresponding physical memory location can be used for some other virtual page.


w: S -- Process Status




The status of the task which can be one of:
'D' = uninterruptible sleep
'R' = running
'S' = sleeping
'T' = traced or stopped
'Z' = zombie Tasks shown as running should be more properly thought of as 'ready to run' -- their task_struct is simply represented on the Linux run-queue. Even without a true SMP machine, you may see numerous tasks in this state depending on top's delay interval and nice value.


x: Command -- Command line or Program name




Display the command line used to start a task or the name of the associated program. You toggle between command line and name with 'c', which is both a command-line option and an interactive command. When you've chosen to display command lines, processes without a command line (like kernel threads) will be shown with only the program name in parentheses, as in this example:
( mdrecoveryd )
Either form of display is subject to potential truncation if it's too long to fit in this field's current width. That width depends upon other fields selected, their order and the current screen width.
Note: The 'Command' field/column is unique, in that it is not fixed-width. When displayed, this column will be allocated all remaining screen width (up to the maximum 512 characters) to provide for the potential growth of program names into command lines.


y: WCHAN -- Sleeping in Function




Depending on the availability of the kernel link map ('System.map'), this field will show the name or the address of the kernel function in which the task is currently sleeping. Running tasks will display a dash ('-') in this column. Note: By displaying this field, top's own working set will be
increased by over 700Kb. Your only means of reducing that overhead
will be to stop and restart top.


z: Flags -- Task Flags




This column represents the task's current scheduling flags which are expressed in hexadecimal notation and with zeros suppressed. These flags are officially documented in . Less formal documentation can also be found on the 'Fields select' and 'Order fields' screens.
Interactive commands
While top is running you may issue some options that will interact immediately with top these options are:
h
Help, displays a summary of command that will modify the behavior of top
k
Kills a process, you will be able to kill only your own processes, unless you are running top as root
n
Once this command is entered top will ask you how many lines you want on your screen, if you enter 0 top will display as much as it can
q
Exits top
r
Change the priority of a process, as well as with k you will only be able to act on your own processes unless you are root
W
Writes the current configuration to your personal configuration file, which is $HOME/.toprc
Example
top - 22:29:04 up 7:16, 3 users, load average: 0.06, 0.08, 0.08
Tasks: 105 total, 2 running, 99 sleeping, 0 stopped, 4 zombie
Cpu(s): 2.5%us, 0.7%sy, 0.0%ni, 95.7%id, 1.0%wa, 0.2%hi, 0.0%si, 0.0%st
Mem: 1035196k total, 796652k used, 238544k free, 55680k buffers
Swap: 2650684k total, 0k used, 2650684k free, 432516k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
4267 root 15 0 320m 45m 10m S 4 4.5 4:45.33 X
2865 haldaemo 17 0 5736 4136 1864 S 0 0.4 0:23.18 hald
1 root 15 0 744 288 240 S 0 0.0 0:01.15 init
2 root RT 0 0 0 0 S 0 0.0 0:00.00 migration/0
3 root 34 19 0 0 0 S 0 0.0 0:00.00 ksoftirqd/0
4 root RT 0 0 0 0 S 0 0.0 0:00.00 migration/1
5 root 34 19 0 0 0 S 0 0.0 0:00.00 ksoftirqd/1
6 root 10 -5 0 0 0 S 0 0.0 0:00.12 events/0
7 root 10 -5 0 0 0 S 0 0.0 0:00.00 events/1
8 root 12 -5 0 0 0 S 0 0.0 0:00.00 khelper
9 root 10 -5 0 0 0 S 0 0.0 0:00.00 kthread
13 root 14 -5 0 0 0 S 0 0.0 0:00.00 kblockd/0
14 root 10 -5 0 0 0 S 0 0.0 0:00.00 kblockd/1
15 root 16 -5 0 0 0 S 0 0.0 0:00.00 kacpid
125 root 11 -5 0 0 0 S 0 0.0 0:00.00 cqueue/0
126 root 11 -5 0 0 0 S 0 0.0 0:00.00 cqueue/1
127 root 10 -5 0 0 0 S 0 0.0 0:00.02 kseriod
171 root 16 0 0 0 0 S 0 0.0 0:00.00 pdflush
172 root 15 0 0 0 0 S 0 0.0 0:00.06 pdflush
173 root 12 -5 0 0 0 S 0 0.0 0:00.00 kswapd0
174 root 12 -5 0 0 0 S 0 0.0 0:00.00 aio/0
175 root 13 -5 0 0 0 S 0 0.0 0:00.00 aio/1
435 root 12 -5 0 0 0 S 0 0.0 0:00.00 kpsmoused
847 root 10 -5 0 0 0 S 0 0.0 0:00.00 ata/0

Tuesday, March 22, 2011

MySQL: the Pros and Cons of MyISAM InnoDB Tables

SkyHi @ Tuesday, March 22, 2011
Unlike the majority of database systems, MySQL offers a number of table types. Choosing the correct type can be critical for your web application, but few developers realise the implications until it is too late. In this article we examine MyISAM tables.MyISAM is the default MySQL table type. If you have never knowingly specified a type, your table will use the MyISAM engine. There are several reasons why that could be an advantage.

1. Simplicity
MyISAM tables are simple. If you are new to MySQL or databases in general, I would recommend you start with MyISAM tables.
In the following example, we will create a new database and table from the mysql> command prompt. (You do have MySQL installed, right? If not, take a look at How to Install MySQL.)

CREATE DATABASE mysqltest;
USE mysqltest;
CREATE TABLE mysqltest.articles (
id int UNSIGNED NOT NULL AUTO_INCREMENT,
title TEXT NOT NULL,
body MEDIUMTEXT,
PRIMARY KEY (id))
ENGINE = MyISAM;

This creates a new database, mysqltest, and a new table named articles with the columns id (an automatically generated number), title and body. Determining the best column data type is about as complex as it gets.

2. Speed
MySQL is fast. Really fast. Under normal conditions, benchmarks show that MyISAM tables beat every other general-purpose database and uses fewer system resources.

3. Full-text searching
Consider our table above. What if we need to search for keywords in both the title and the body? An easy solution is to add a fulltext index:

ALTER TABLE mysqltest.articles ADD FULLTEXT alltext (title, body);
We can now find all articles that feature the words “database” and/or “article” using:

SELECT * FROM mysqltest.articles
WHERE MATCH(title, body) AGAINST ('database article');
We can even order articles by the most relevant first to create a simple search engine:

SELECT *, MATCH(title, body) AGAINST ('database article') AS rel
FROM mysqltest.articles
ORDER BY rel DESC;

The disadvantages of MyISAM tables

There are several situations when MyISAM may not be suitable.
1. Poor data integrity
MyISAM tables do not support transactions or foreign key constraints.
Consider a banking application where you are transferring money. This would typically involve two SQL UPDATE statements; a debit from one account and a credit to another. If MySQL or the server failed at the wrong point, you could end up with money in both accounts or neither.

2. Poor crash recovery
MySQL is reliable, but MyISAM tables can become corrupt following a crash. The problem can usually be fixed using REPAIR TABLE but it is an additional administrative overhead.

3. Table locking
MyISAM uses table-level locking. When a row is inserted or updated, all other changes to that table are held up until that request has been completed.
It is not always easy to prove whether this will cause problems for your web application but, in general, if you are performing more inserts/updates than selects, MyISAM may not be the best table type to choose.

Should you use MyISAM?

MyISAM is an ideal choice if you are new to MySQL, your web application is simple, has to be fast, or use full-text searches. However, it should not be used when data integrity is a priority. Possible applications could include content management systems, online bookmarking tools, RSS readers, or simple search engines.
Note that you can mix table types within the same database. In practice, it may cause less developer confusion if you stick with a single table type.
See also:

MySQL: the Pros and Cons of InnoDB Tables

 Following on from our look at MyISAM tables, today we examine InnoDB tables.
InnoDB is less well known than the default MyISAM table type. However, there are several reasons why it could be better choice for your web application.

1. Data integrity and foreign key constraints
Foreign keys establish a relationship between columns in one table and those in another. For example, you might create a library application where books can be loaned to members. A foreign key constraint would ensure that a member existed before a book could be checked-out. Similarly, removing a user would not be possible until all their books were returned.
For further information about foreign keys and automating your database, refer to How to Use MySQL Foreign Keys for Quicker Database Development.

2. Transactions
InnoDB tables support transactions. A transaction allows multiple SQL commands to be treated as a single and reliable unit.
Consider a banking application where you are transferring money from one account to another. The transaction would only be committed if both accounts were altered successfully. If anything failed, the database would be rolled-back to a previous state.
In addition, InnoDB tables recover well from crashes. MySQL will analyze the log files to ensure the data is accurate so there is no need to repair tables.

3. Row-level locking
InnoDB uses row-level rather than table-level locking. If a row is being inserted, updated or deleted, only changes to the same row are held up until that request has completed. Tables that receive more updates than selects may be faster with InnoDB.

The disadvantages of InnoDB tables

There are a number of situations when InnoDB may not be suitable.
1. Increased complexity
Creating an InnoDB table is no more complex than MyISAM, e.g.

CREATE TABLE employee (
id smallint(5) unsigned NOT NULL,
firstname varchar(30),
lastname varchar(30),
PRIMARY KEY (id)
) ENGINE=InnoDB;
However, designing that database with foreign key relationships does require more effort. Database novices will find MyISAM easier because it has fewer features.

2. No full-text search
InnoDB tables do not support full-text searches; it is not easy to match one or more keywords against multiple columns.

3. Slower performance
If your application is primarily selecting data and performance is a priority, MyISAM tables will normally be faster and use fewer system resources.

Should you use InnoDB?

InnoDB is the best option if you need to create a reliable data-driven web application. In many ways, InnoDB is a better default choice than MyISAM:
  1. unless you have a significantly large or heavily-used system, the speed differences are likely to be negligible
  2. full-text searches can be implemented in other ways, e.g. more complex SQL or server-side search algorithms.
InnoDB is certainly the best choice for online shops, financial applications or any project where data integrity is essential. Defining tables is more complex, but your application will be more robust and may require less server-side code.
Note that you can mix table types within the same database. In practice, it may cause less developer confusion if you stick with a single table type.

REFERENCES
http://blogs.sitepoint.com/mysql-myisam-table-pros-con/

Monday, March 21, 2011

AttributeError: 'module' object has no attribute 'findall'

SkyHi @ Monday, March 21, 2011
Code:
d@home$ cat re.py
import re
re_string = "{{(.*?)}}"
some_string = "this is a string with {{words}} embedded in {{curly brackets}} to show an {{example}} of {{regular expression}}"
for match in re.findall(re_string,some_string):
        print 'Match->', match


ERROR:
Traceback (most recent call last):
  File "re.py", line 1, in <module>
    import re
  File "/home/nikid/PythonSysadmin/re.py", line 4, in <module>
    for match in re.findall(re_string,some_string):
AttributeError: 'module' object has no attribute 'findall'
Error in sys.excepthook:
Traceback (most recent call last):
  File "/usr/lib/python2.6/dist-packages/apport_python_hook.py", line 48, in apport_excepthook
    if not enabled():
  File "/usr/lib/python2.6/dist-packages/apport_python_hook.py", line 21, in enabled
    import re
  File "/home/nikid/PythonSysadmin/re.py", line 4, in <module>
    for match in re.findall(re_string,some_string):
AttributeError: 'module' object has no attribute 'findall'

Original exception was:
Traceback (most recent call last):
  File "re.py", line 1, in <module>
    import re
  File "/home/nikid/PythonSysadmin/re.py", line 4, in <module>
    for match in re.findall(re_string,some_string):
AttributeError: 'module' object has no attribute 'findall'




Solution:

rename the file re.py to another name


Another one

ERROR:

NameError: name 'randint' is not defined

Solution:
Rename the random.py to different name..

REFERENCES
http://www.willmer.com/kb/2007/12/attributeerror-module-object-has-no-attribute-blah/