Command Center

This document is made to be easily skimmed. The latest addition was on 5-21-2009, 'DVD backup of hard drive partition'. This is comprehensive documentation for one of the most useful Linux/UNIX Windows commands-dd. It is a bit-stream duplicator for duplicating data. If you have a question, post it.

First Time visitors please leave a reply.


New Content, for a while at least.DVD backup of hard drive partition:
Code:

dd if=/dev/hda3 of=/home/sam/backup_set_1.img bs=1M count=4600
dd if=/dev/hda3 skip=4600 of=/home/sam/backup_set_2.img bs=1M count=4600
dd if=/dev/hda3 skip=9200 of=/home/sam/backup_set_3.img bs=1M count=4600

And so on ... Burn the images to DVD+R/RW:
Code:

wodim -dev=/dev/hdc --driveropts=burnfree /home/sam/backup_set_1.img

and so forth ... To restore, load the DVDs in order, and:
Code:

dd if=/media/dvd/backup_set_1.img of=/dev/hda3 bs=1M conv=sync,noerror

Load another DVD
Code:

dd if=/media/dvd/backup_set_2.img of=/dev/hda3 seek=4600 bs=1M conv=sync,noerror

Load another DVD
Code:

dd if=/media/dvd/backup_set_3.img of=/dev/hda3 seek=9200 bs=1M conv=sync,noerror

and so forth ...


You wrote chat messages and emails on your girlfriend's computer. You deleted everything. But you want to make sure a private investigator and/or computer consultant can't find it.
Code:

dd if=/dev/sda | sed 's/Wendy/Janet/g' | dd of=/dev/sda

Where every instance of Wendy is replaced by Janet, over every millimeter of disk. I picked names with the same number of characters, but you can just pad a smaller name with blanks.

Windows users will find help about 50 lines down from here.

FYI: duplicateing smaller partition, or drive to larger partition, or drive; or vice versa:
Code:

rsync -avH --exclude=/other_mount_point/ /mount_point/* /other_mount_point/

You want to duplicate the root directory tree to another drive, but the other drive is larger. If you use dd, you will get a file system that is smaller then the larger destination drive. To duplicate files, not the file system: Format and mount the destination drive:
Code:

man parted
rsync -avH --exclude=/mnt/destination_drive/ /* /mnt/destination_drive/

will duplicate all the files as files. You need to run:
Code:

grub-install
update-grub

from a the rescue menu of an installation CD/DVD for the target to become bootable. If target was previously bootable, it remains bootable.

MS Windows Section

Use dd for drive cloning, backup, upgrading, and restore tasks. Boot a Windows XP machine with a Knoppix CD; A live CD Linux system. Download Knoppix, burn the iso image file to a CD, boot with it, and clone drives. Drives are described to the dd command using device files. Boot into Knoppix, and open a root shell. It's in the penguin menu. (80 conductor grey ribbon cable) = /dev/hda for master; /dev/hdb for slave. SATA are /dev/sda and /dev/sdb.

Code:

fdisk -l /dev/hda

The partitions on the first drive.
Code:

man fdisk

The manual page for fdisk. Parted to make partitions
Code:

man parted

If one has trouble, ask. Knoppix is slow, because it runs on a CD drive (1/1000 the speed of a HDD). Feel free to read the entire post.
Dd is not presently able to clone Microsoft Windows Vista OEM Partitions.
End Section

Linux DD
The basic command structure is as follows:
Code:

dd if= of= bs=("USUALLY" some power of 2, not less than 512 bytes(ie, 512, 1024, 2048, 4096, 8192, 16384[/b], but can be ANY reasonable number.) skip= seek= conv=.

Source is the data being read. Target is where the data gets written.

Warning!! If you reverse the source and target, you can wipe out a lot of data. This feature has inspired the nickname "dd" Data Destroyer.
Warning!! Caution should be observed when using dd to duplicate encrypted partitions.

Examples: duplicate one hard disk partition to another hard disk:
Code:

dd if=/dev/sda2 of=/dev/sdb2 bs=4096 conv=notrunc,noerror

sda2 and sdb2 are partitions. You want to duplicate sda2 to sdb2. If sdb2 doesn't exist, dd will start at the beginning of the disk, and create it. Be careful with order of if and of. You can write a blank disk to a good disk if you get confused. If you duplicate a smaller partition to a larger one the larger one, using dd, the larger one will now be formatted the same as the smaller one, and there won't be any space left on the drive. The way around this is to use
Code:

rsync

, as described in the beginning of the post
Make an iso image of a CD:
Code:

dd if=/dev/hdc of=/home/sam/myCD.iso bs=2048 conv=sync,notrunc

This duplicates sector for sector. MyCD.iso will be a hard disk image file of the CD. You can mount the image:
Code:

mkdir /mnt/myCD
mount -o loop /home/sam/myCD.iso /mnt/myCD
CD /mnt/myCD
ls

This will display the top level of the CD.
duplicate a floppy disk:
Code:

dd if=/dev/fd0 of=/home/sam/floppy.image bs=2x80x18b conv=notrunc

OR
Code:

dd if=/dev/fd0 of=/home/sam/floppy.image conv=notrunc

The 18b specifies 18 sectors of 512 bytes. 2x multiplies the sector size by the number of heads. 80x specifies cylinders--a total of 1474560 bytes. This issues a single 1474560-byte read request to /dev/fd0 and a single 1474560 write request to
Code:

/home/sam/floppy.image

. Thanks to Sam Cheeseman for documenting this unusual, but highly cool CHS method of specifying the block size (bs=) parameter of dd.
This makes a hard drive image of the floppy, with bootable info intact. The second example uses default bs=512: 1.44 MB floppy sector size. If you're concerned about spies with superconducting quantum-interference detectors, you can always add a "for" loop for US Government DoD approved secure hard disk erasure. Copy and paste the following two lines into a text editor.
Code:

#!/bin/bash
for n in `seq 7`; do dd if=/dev/urandom of=/dev/sda bs=8b conv=notrunc; done

Save the file as 'swqid'.
Code:

chmod a+x swqid

Don't run the program until you are want to wipe the drive.
Overwrite the drive with zeroes
Code:

dd if=/dev/zero of=/dev/sda bs=4k conv=notrunc

[color="red"I just want to make sure my drive is really zeroed out!![/color]
Code:

dd if=/dev/sda | hexdump -C | grep [^00]

... will show every line from the
Code:

hexdump -C

that isn't 0x00h, OR, zero
To be revised at a later date:
To make a bootable flash drive: Download 50 MB Debian based distro here:
http://sourceforge.net/projects/insert/

Plug in the thumb drive into a USB port. Do:
Code:

dmesg | tail

Look where the new drive is, sdb1, or something similar. Do:
Code:

dd if=/home/sam/insert.iso of=/dev/sdb ibs=4b obs=1b conv=notrunc,noerror

Set the BIOS to USB boot, and boot.
End to be revised
duplicate the MBR and boot sector of a floppy to hard drive image:
Code:

dd if=/dev/fd0 of=/home/sam/MBRboot.image bs=512 count=2

This duplicates the first 2 sectors of the floppy.
Cloning an entire hard disk:
Code:

dd if=/dev/sda of=/dev/sdb bs=4096 conv=notrunc,noerror

/dev/sda is the source. /dev/sdb is the target. Do not reverse the intended source and target. It happens. Notrunc means 'do not truncate the output file'. Noerror means to keep going if there is an error. Normally dd stops at any error.
Duplicate MBR only:
Code:

dd if=/dev/sda of=/home/sam/MBR.image bs=446 count=1

This will duplicate the first 446 bytes of the hard drive to a file. If you haven't already guessed, reversing the objects of if and of, on the dd command line, reverses the direction of the write.
Wipe a hard drive: (Boot from a live CD distro to do this.) http://www.efense.com/helix
is a good boot CD. The helix boot environment contains the DoD version of dd called dcfldd. It works the same way, but is has a progress bar.
Code:

dcfldd if=/dev/zero of=/dev/sda conv=notrunc

Beware, dcfldd does not observe the 'seek=' parameter.
This is useful for making the drive almost like new. Most drives have 0xffh written to every byte, from the factory.
Overwrite all the free space on a partition (deleted files you don't want recovered)
Code:

dd if=/dev/urandom of=/home/sam/fileconsumingallfreespace.file

When dd says no room left on device, all the free space has been overwritten with random characters. Then, delete the big file with
Code:

rm

.
Code:

less /home/sam/file.bin

looks like GobblDeeGook, because it's a binary file.
Code:

dd if=/home/sam/file.bin | hexdump -C | less

is readable. Sometimes one wants to look inside a binary file, with no idea of what they are looking for, only clues to what tools made the file, how to uncompress it, the header bytes, or the author's name and/or email.
Virtual memory:
Code:

dd if=/proc/kcore | hexdump -C | less

use PgUp, PgDn, up arrow, down arrow to navigate in less. Less is my favorite editor, except you can't edit with it.
Filesystems:
Code:

dd if=/proc/filesystems | hexdump -C | less

Modules:
Code:

dd if=/proc/kallsyms | hexdump -C | less

Interrupt table:
Code:

dd if=/proc/interrupts | hexdump -C | less

Uptime (seconds):
Code:

dd if=/proc/uptime | hexdump -C | less

Partitions (in KB):
Code:

dd if=/proc/partitions | hexdump -C | less

Memory stats:
Code:

dd if=/proc/meminfo | hexdump -C | less

I put two identical drives in every one of my machines. Before I do anything that most probably spells disaster, like an untested command line in a root shell, that contains
Code:

find / -regex ?*.???* -type f | xargs rm -f "$1"

, I do:
Code:

dcfldd if=/dev/sda of=/dev/sdb bs=4096 conv=notrunc,noerror

and duplicate my present working /dev/sda drive system to the /dev/sdb drive. If I wreck the installation on sda, I boot with the helix CD and:
Code:

dcfldd if=/dev/sdb of=/dev/sda bs=4096 conv=notrunc,noerror

and I get everything back exactly the same as before whatever daring maneuver I was trying didn't work. You can really, really learn linux this way, because you can't wreck what you have an exact duplicate of. You also might consider making the root partition separate from /home, and make /home big enough to hold the root partition, plus more. Then you can do:
Code:

dd if=/dev/sda2 (root) of /home/sam/root.img bs=4096 conv=notrunc,noerror

To make a backup of root, and :
Code:

dd if /home/sam/root.img of=/dev/sda2 (root) bs=4096 conv=notrunc,noerror

To write the image of root back to the root partition if you messed up and can't launch the X server, or edited /etc/fstab and can't figure out what you did wrong. It only takes a few minutes to restore a 15 GB root partition from an image file.



How to make a swap file, or another swapfile on a running system:
Code:

dd if=/dev/zero of=/swapspace bs=4k count=250000
mkswap /swapspace
swapon /swapspace

This can solve out of memory issues due to memory leaks on servers that cannot easily be rebooted.

How to pick proper block size:

Code:

dd if=/dev/zero bs=1024 count=1000000 of=/home/sam/1Gb.file
dd if=/dev/zero bs=2048 count=500000 of=/home/sam/1Gb.file
dd if=/dev/zero bs=4096 count=250000 of=/home/sam/1Gb.file
dd if=/dev/zero bs=8192 count=125000 of=/home/sam/1Gb.file

This method can also be used as a drive benchmark, to find strengths and weaknesses in hard drives:
Read:
Code:

dd if=/home/sam/1Gb.file bs=64k | dd of=/dev/null

Write:
Code:

dd if=/dev/zero bs=1024 count=1000000 of=/home/sam/1Gb.file

When dd finishes it outputs (total size)/(total time). You get the idea.
Play with 'bs=' and 'count=', always having them multiply out to the same toal size. You can calculate bytes/second like this: 1Gb/total seconds = Gb/s. You can get more realistic results using a 3Gb file.

Rejuvenate a hard drive
To cure input/output errors experienced when using dd. Over time the data on a drive, especially a drive that hasn't been used for a year or two, grows into larger magnetic flux points than were originally recorded. It becomes more difficult for the drive heads to decipher these magnetic flux points. This results in I/O errors. Sometimes sector 1 goes bad, resulting in a useless drive. Try:
Code:

dd if=/dev/sda of=/dev/sda

to rejuvenate the drive. Rewrites all the data on the drive in nice tight magnetic patterns that can then be read properly. The procedure is safe and economical.

Make a file of 100 random bytes:
Code:

dd if=/dev/urandom of=/home/sam/myrandom bs=100 count=1

/dev/random produces only as many random bits as the entropy pool contains. This yields quality randomness for cryptographic keys. If more random bytes are required, the process stops until the entropy pool is refilled (waggling your mouse helps). /dev/urandom does not have this restriction. If the user demands more bits than are currently in the entropy pool, it produces them using a pseudo random number generator. Here, /dev/urandom is the Linux random byte device. Myrandom is a file.

Randomize data over a file before deleting it:
Code:

ls -l

to find filesize.
In this case it is 3769
Code:

ls -l afile -rw------- ... 3769 Nov 2 13:41

Code:

dd if=/dev/urandom of=afile bs=3769 count=1 conv=notrunc

duplicate a disk partition to a file on a different partition.

Warning!! Do not write a partition image file to the same partition.
Code:

dd if=/dev/sdb2 of=/home/sam/partition.image bs=4096 conv=notrunc,noerror

This will make a file that is an exact duplicate of the sdb2 partition. You can substitue hdb, sda, hda, etc ... OR
Code:

dd if=/dev/sdb2 ibs=4096 | gzip > partition.image.gz conv=noerror

Makes a gzipped archive of the entire partition. To restore use:
Code:

dd if=partition.image.gz | gunzip | dd of=/dev/sdb2

For bzip2 (slower,smaller), substitute bzip2 and bunzip2, and name the file
Code:

< filename >.bz2

.Restore a disk partition from an image file.
Code:

dd if=/home/sam/partition.image of=/dev/sdb2 bs=4096 conv=notrunc,noerror

Convert a file to uppercase:
Code:

dd if=filename of=filename conv=ucase

Make a ramdrive:
The Linux kernel makes a number a ramdisks you can make into ramdrives. You have to populate the drive with zeroes like so:
Code:

dd if=/dev/zero of=/dev/ram7 bs=1k count=16384

Populates a 16 MB ramdisk.
Code:

mke2fs -m0 /dev/ram7 4096

puts a file system on the ramdisk, turning it into a ramdrive. Watch this puppy smoke.
Code:

debian:/home/sam # hdparm -t /dev/ram7
/dev/ram7:
Timing buffered disk reads: 16 MB in 0.02 seconds = 913.92 MB/sec

You only need to do the timing once, because it's cool. Make the drive again, because hdparm is a little hard on ramdrives. You can mount the ramdrive with:
Code:

mkdir /mnt/mem
mount /dev/ram7 /mnt/mem

Now you can use the drive like a hard drive. This is particularly superb for working on large documents or programming. You can duplicate the large file or programming project to the ramdrive, which on my machine is at least 27 times as fast as /dev/sda, and every time you save the huge document, or need to do a compile, it's like your machine is running on nitromethane. The only drawback is data security. The ramdrive is volatile. If you lose power, or lock up, the data on the ramdrive is lost. Use a reliable machine during clear skies if you use a ramdrive.

Duplicate ram memory to a file:
Code:

dd if=/dev/mem of=/home/sam/mem.bin bs=1024

The device
Code:

/dev/mem

is your system memory. You can actually duplicate any block or character device to a file using dd. Memory capture on a fast system, with bs=1024 takes about 60 seconds, a 120 GB HDD about an hour, a CD to hard drive about 10 minutes, a floppy to a hard drive about 2 minutes. With dd, your floppy drive images will not change. If you have a bootable DOS diskette, and you save it to your HDD as an image file, when you restore that image to another floppy it will be bootable.

Dd will print to the terminal window if you omit the
Code:

of=/dev/output

part.
Code:

dd if=/home/sam/myfile

will print the file myfile to the terminal window.

To search the system memory:
Code:

dd if=/dev/mem | hexdump -C | grep 'some-string-of-words-in-the-file-you-forgot-to-save-before-the-power-failed'

If you need to cover your tracks quickly, put the following commands in a script to overwrite system ram with zeroes. Don't try this for fun.
Code:

mkdir /mnt/mem
mount -t ramfs /dev/mem /mnt/mem
dd if=/dev/zero > /mnt/mem/bigfile.file

This will overwrite all unprotected memory structures with zeroes, and freeze the machine so you have to reboot (Caution, this also prevents committment of the file system journal, and could trash the file system).

You can get arrested in 17 states for doing this next thing. Make an AES encrypted loop device:
Code:

dd if=/dev/urandom of=/home/sam/aes-drv bs=16065b count=100
modprobe loop
modprobe cryptoloop
modprobe aes
losetup -e aes /dev/loop1 ./aes-drv
password:
mkreiserfs /dev/loop1
mkdir /aes
mount -o loop,encryption=aes,acl ./aes-drv /aes
password:
mv /home/sam/porno /aes

to get the porno on the aes drive image.
Code:

umount /aes
losetup -d /dev/loop1
rmmod aes
rmmod cryptoloop
rmmod loop

to make 'aes-drv' look like a 400 MB file of random bytes. Every time the lo interface is configured using losetup, according to the above, and the file 'aes-drv' is mounted, as above, the porno stash will be accessible in /aes/porno. You don't need to repeat the dd command, OR, the format with reiserfs, OR, the mv command. You only do those steps once. If you forget the password, there is no way to recover it besides guessing. Once the password is set, it can't be changed. To change the password, make a new file with the desired password, and move everything from the old file to the new file. Acl is a good mount option, because it allows use of acls. Otherwise your stuck with u,g,o and rwx.

If you are curious about what might be on you disk drive, or what an MBR looks like, or maybe what is at the very end of your disk:
Code:

dd if=/dev/sda count=1 | hexdump -C

Will show you sector 1, or the MBR. The bootstrap code and partition table are in the MBR.
To see the end of the disk you have to know the total number of sectors, and the MAS must be set equal to the MNA. The helix CD has a utility to set this correctly. In the dd command, your skip value will be one less than MNA of the disk. For a 120 GB Seagate SATA drives
Code:

dd if=/dev/sda of=home/sam/myfile skip=234441646 bs=512

,
So this reads sector for sector, and writes the last sector to myfile. Even with LBA addressing, disks still secretly are read in sectors, cylinders, and heads.
There are 63 sectors per track, and 255 heads per cylinder. There is a total cylinder count. 512_bytes/sector*63_sectors/track*255heads=16065*512bytes/cylinder=8,225,280_bytes/cylinder. 63_sectors/track*255_heads=sectors/cylinder. With 234441647 total sectors, and 16065 sectors per cylinder, you get some trailing sectors which do not make up an entire cylinder: 14593.317584812_cylinders/drive. This leaves 5102 sectors which cannot be partitioned, because to be in a partition you have to be a whole cylinder. It's like having part of a person. That doesn't really count as a person. These become surplus sectors after the last partition. You can't ordinarily read past the last partition. But dd can. It's a good idea to check for anything writing to surplus sectors. For our Seagate 120 GB drive, 234,441,647_sectors/drive - 5102_surplus_sectors = 234,436,545 partitionable sectors.
Code:

dd if=/dev/sda of=/home/sam/myfile skip=234436545

writes the last 5102 sectors to myfile. Launch midnight commander (mc) to view the file. If there is something in there, you do not need it for anything. In this case you would write over it with random characters:
Code:

dd if=/dev/urandom of=/dev/sda bs=512 seek=234436545

Will overwrite the 5102 surplus sectors on our 120 GB Seagate drive.

Block size:
One cylinder in LBA mode = 255_heads*63_sectors/track=16065_sectors=16065*512_bytes=8,225,280_bytes. The b means '* 512'. 32130b represents a two cylinder block size. Cylinder block size always works to cover every sector in a partition, because partitions are made of a whole number of cylinders. One cylinder is 8,225,280 bytes. If you want to check out some random area of the disk:
Code:

dd if=/dev/sda of=/home/sam/myfile bs=4096 skip=2000 count=1000

Will give you 8,000 sectors in myfile, after the first 16,000 sectors. You can open that file with a hex editor, edit some of it, and write the edited part back to disk:
Code:

dd if=/home/sam/myfile of=/dev/sda bs=4096 seek=2000 count=1000

Image a partition to another machine:
On source machine:
Code:

dd if=/dev/hda bs=16065b | netcat < targethost-IP > 1234

On target machine:
Code:

netcat -l -p 1234 | dd of=/dev/hdc bs=16065b

Variations on target machine:
Code:

netcat -l -p 1234 | bzip2 > partition.img

makes a compressed image file using bzip2 compression.
Code:

netcat -l -p 1234 | gzip > partition.img

makes a compressed image file using gzip compression. I back up a 100 GB lappy disk on a desktop drive, over a lan connection, and the 100 GB compresses to about 4.0 GB. Most of the drive is empty, so it's mostly zeroes. Repetitive zeroes compress well.
Alert!! Don't hit enter yet. Hit enter on the target machine. THEN hit enter on the source machine.

Netcat is a program, available by default, on most linux installations. It's a networking swiss army knife. In the preceding example, netcat and dd are piped to one another. One of the functions of the linux kernel is to make pipes. The pipe character looks like two little lines on top of one another, both vertical. Here is how this command behaves: This byte size is a cylinder. bs=16065b equals one cylinder on an LBA drive. The dd command is piped to netcat, which takes as its arguments the IP address of the target(like 192.168.0.1, or any IP address with an open port) and what port you want to use(1234).



CONTINUES...SEE NEXT POST

Dd is like Symantec Norton Ghost, Acronis True Image, Symantec Drive Image. You can perform disk drive backup, restore, imaging, disk image, cloning, clone, drive cloning, transfer image, transfer data, clone to another drive or clone to another machine, move Windows XP to a new hard drive, clone Windows XP, clone Windows, transfer Windows, hard drive upgrade, duplicate a boot drive, duplicate a bootable drive, upgrade your operating system hard drive, Tired of reinstalling WinXP Windows XP?

Copyright 2008 by AwesomeMachine.
All Rights Reserved.

Reference: http://www.linuxquestions.org/questions/linux-newbie-8/learn-the-dd-command-362506/

[root@home ~]# cat /etc/proftpd.conf
# This is a basic ProFTPD configuration file (rename it to
# 'proftpd.conf' for actual use. It establishes a single server
# and a single anonymous login. It assumes that you have a user/group
# "nobody" and "ftp" for normal operation and anon.
#ServerIdent on " Home server"
ServerName "Home ftp server"
ServerType standalone
DefaultServer on
AllowForeignAddress on
#RequireValidShell off
#TimeoutIdle 3600
#TimeoutStalled 3600
TimeoutLogin 900
TimeoutNoTransfer 3600
UseReverseDNS off
IdentLookups off
PassivePorts 51000 51999
#TimesGMT off
#MasqueradeAddress 192.168.218.48

#UseReverseDNS off
#IdentLookups off
# Port 21 is the standard FTP port.
Port 21

# Umask 022 is a good standard umask to prevent new dirs and files
# from being group and world writable.
Umask 022

# To prevent DoS attacks, set the maximum number of child processes
# to 30. If you need to allow more than 30 concurrent connections
# at once, simply increase this value. Note that this ONLY works
# in standalone mode, in inetd mode you should use an inetd server
# that allows you to limit maximum number of processes per service
# (such as xinetd).
MaxInstances 30

# Set the user and group under which the server will run.
User ftp
Group ftp

# To cause every FTP user to be "jailed" (chrooted) into their home
# directory, uncomment this line.
DefaultRoot ~

# Normally, we want files to be overwriteable.
AllowOverwrite on

# Bar use of SITE CHMOD by default

DenyAll


# A basic anonymous configuration, no upload directories. If you do not
# want anonymous users, simply delete this entire section.
#Anonymous ~ftp>
#User ftp
#Group ftp

# We want clients to be able to login with "anonymous" as well as "ftp"
# UserAlias anonymous ftp

# Limit the maximum number of anonymous logins
#MaxClients 10

# We want 'welcome.msg' displayed at login, and '.message' displayed
# in each newly chdired directory.
# DisplayLogin welcome.msg
#DisplayFirstChdir .message

# Limit WRITE everywhere in the anonymous chroot
#
# DenyAll
#/Limit>
#/Anonymous>


SQLConnectInfo database@192.168.0.16 user password
SQLAuthTypes Plaintext Crypt
SQLAuthenticate users groups
SQLUserInfo ftpuser userid passwd uid gid homedir shell
SQLGroupInfo ftpgroup groupname gid members
SQLMinID 500
LogFormat default "%h %l %u %t \"%r\" %s %b %D \"%f\""
ExtendedLog /var/www/html/ftp.log ALL default
# create a user's home directory on demand if it doesn't exist
SQLHomedirOnDemand on

# Update count every time user logs in
SQLLog PASS updatecount
SQLNamedQuery updatecount UPDATE "count=count+1, accessed=now() WHERE userid='%u '" ftpuser
# Update modified everytime user uploads or deletes a file
SQLLog STOR,DELE modified
SQLNamedQuery modified UPDATE "modified=now() WHERE userid='%u'" ftpuser

The Unix world has two ways of storing mail messages, the traditional mbox format and the newer maildir format. Postfix and Dovecot supports the two mail storage format so you can use any format, but I highly recommend you use the maildir format.
The Mbox Format

This is the traditional way of storing mail messages in the Unix world. In this format, a regular text file which serves as the mail user’s mailbox file is created.
Mbox storage format

Fig. 1: Mbox storage format
How Mbox works
Receiving and storing a mail

1. Lock the mailbox.
2. Append the header (usually “From [sender's email address] [date and time received]“) and the mail into the mailbox file.
3. Unlock the mailbox.

Retrieving a mail

1. Lock the mailbox.
2. Locate and read the mail.
3. Update the mail status flag.
4. Unlock the mailbox.

Deleting a mail

1. Lock the mailbox.
2. Move the contents of the mailbox, beginning from the position right after the mail to be deleted until the end of the mailbox, into the position of the mail to be deleted.
3. Reduce the size of the mailbox file by the size of the deleted mail.
4. Unlock the mailbox.

Searching a mail

1. Lock the mailbox.
2. Search the mailbox.
3. Unlock the mailbox.


Advantages

* Format is universally supported.
* Appending a new mail into the mailbox file is fast.
* Searching text inside a single mailbox file is fast.

Disadvantages

* Has file locking problems.
* Has problems when used with network file systems.
* Format is prone to corruption.


The Maildir Format

This is a new way of storing mail messages. In this format, a directory usually named Maildir is created for each mail user. Under this directory are three more directories named new, cur and tmp.
Maildir storage format

Fig. 2: Maildir storage format
How Maildir works
Receiving and storing a mail

1. Create a unique file in the tmp directory.
2. Write the mail into the newly created file.
3. Move the completely written mail into the new directory.

Retrieving a mail

1. Locate and read the mail.
2. Move the mail from new into the cur directory and append the mail status flag into the filename.

Deleting a mail

1. Delete the file containing the mail.

Searching a mail

1. Search each and every mail file.


Advantages

* Locating, retrieving and deleting a specific mail is fast.
* Minimal to no file locking needed.
* Can be used on network file system.
* Immune to mailbox corruption (assuming the hardware will not fail).

Disadvantages

* Some filesystems may not efficiently handle a large number of small files.
* Searching text, which requires all mail files to be opened is slow.

Visit the forum to ask for help or to give a comment.

Reference: http://www.linuxmail.info/mbox-maildir-mail-storage-formats/

Synopsis
smrsh -c command
Description
The smrsh program is intended as a replacement for sh for use in the ''prog'' mailer in sendmail(8) configuration files. It sharply limits the commands that can be run using the ''|program'' syntax of sendmail in order to improve the over all security of your system. Briefly, even if a ''bad guy'' can get sendmail to run a program without going through an alias or forward file, smrsh limits the set of programs that he or she can execute.

Briefly, smrsh limits programs to be in a single directory, by default /etc/smrsh, allowing the system administrator to choose the set of acceptable commands, and to the shell builtin commands ''exec'', ''exit'', and ''echo''. It also rejects any commands with the characters ''', '<', '>', ';', '$', '(', ')', '\r' (carriage return), or '\n' (newline) on the command line to prevent ''end run'' attacks. It allows ''||'' and ''&&'' to enable commands like: ''"|exec /usr/local/bin/filter || exit 75"''

Initial pathnames on programs are stripped, so forwarding to ''/usr/ucb/vacation'', ''/usr/bin/vacation'', ''/home/server/mydir/bin/vacation'', and ''vacation'' all actually forward to ''/etc/smrsh/vacation''.

System administrators should be conservative about populating the /etc/smrsh directory. For example, a reasonable additions is vacation(1), and the like. No matter how brow-beaten you may be, never include any shell or shell-like program (such as perl(1)) in the /etc/smrsh directory. Note that this does not restrict the use of shell or perl scripts in the sm.bin directory (using the ''#!'' syntax); it simply disallows execution of arbitrary programs. Also, including mail filtering programs such as procmail(1) is a very bad idea. procmail(1) allows users to run arbitrary programs in their procmailrc(5).
Files
/etc/smrsh - directory for restricted programs
See Also
sendmail(8)

Solution The traditional UNIX mailbox format or MMDF format is only allows one session to have the mailbox open read/write at a time. So, the email server assumes that if a second session attempts to open the mailbox, that means that the first session is probably owned by an abandoned client. The common scenario here is a user who leaves his client running at the office, and then tries to read his mail from home. Through an internal mechanism called kiss of death, the second session requests the first session to kill itself. When the first session receives the "kiss of death", it issues the "Killed (lost mailbox lock)" syslog message and terminates. The second session then seizes read/write access, and becomes the new "first" session.

Certain poorly-designed clients routinely open multiple sessions to the same mailbox; the users of those clients tend to get this message a lot.

Another cause of this message is a background "check for new mail" task which does its work by opening a POP session to server every few seconds. They do this because POP doesn't have a way to announce new mail.


Reference: http://www.alphaone-tech.com/tech-support/index.php?_m=knowledgebase&_a=viewarticle&kbarticleid=67



I've just fixed this myself on one of our servers - the problem is that the standard mailbox format used is mbox, which doesn't support concurrent users. To get around this, you need to convert the Inbox to mbx which offers faster access and concurrent access so therefore fixes the problem. Nothing on the server needs to be reconfigured for this as mbx is supported as well anyway. You will need to do the following:

change into the users home directory:

cd /home/virtual/domain.com/home/user

Download an mbx format inbox (this does not overwrite your existing mail - this is just a blank one that I've created as well)

wget http://www.penguin-uk.com/ensim/INBOX

Check your mail via IMAP then and it will configure itself on the first go. Problem then solved

Reference: http://forums.theplanet.com/index.php?showtopic=60982

f you have data that you need to bring into your MySQL database, there are a few ways to do it. Exporting data out of mysql is another topic, described here.

1. Using the LOAD DATA INFILE SQL statement

For security reasons, no one has the mysql FILE priv, which means you cannot "LOAD DATA INFILE". You can, however, use a "LOAD DATA LOCAL INFILE" statement as long as you have a mysql prompt on our system and have uploaded the data file to your account here first.

The "LOAD DATA LOCAL INFILE" statement will only work from a MySQL prompt on our local system. It will not work from any web-based tool such as phpMyAdmin, and will never pull a file in directly off your own computer.

To import a file this way, first upload your data file to your home directory on our system with FTP or SCP. Then get a shell prompt on our system, and then a MySQL Monitor prompt so that you can issue the SQL that will import your file.

For example, suppose you have a data file named importfile.csv that contains 3 comma separated columns of data on each line. You want to import this textfile into your MySQL table named test_table, which has 3 columns that are named field1, field2 and field3.

To import the datafile, first upload it to your home directory, so that the file is now located at /importfile.csv on our local system. Then you type the following SQL at the mysql prompt:

LOAD DATA LOCAL INFILE '/importfile.csv'
INTO TABLE test_table
FIELDS TERMINATED BY ','
LINES TERMINATED BY '\n'
(field1, filed2, field3);

The above SQL statement tells the MySQL server to find your INFILE on the LOCAL filesystem, to read each line of the file as a separate row, to treat any comma character as a column delimiter, and to put it into your MySQL test_table as columns field1, field2, and field3 respectively. Many of the above SQL clauses are optional and you should read the MySQL documentation on the proper use of this statement.

2. Using a script to parse and import the file

You can also write a script in any programming language that can connect to MySQL (such as PHP) to open your data file, break it up into an array of lines that each represent a row of data, split each line up by the delimiter character (such as a comma ',', tab '\t', semicolon ';', space ' ', etc.), and then perform invididual MySQL INSERT queries (one INSERT for each line) to insert all your data from the file into the appropriate table fields.

Such scripts are not difficult to write in less than 15 lines and can import data from text files just as effectively as a LOAD DATA LOCAL INFILE command. A working example script written in PHP appears below in the Annotations.

3. Importing a mysqldump

If your data file actually comes from another MySQL database, and not from Excel or any other source, then the most direct way to export and import your data would be to dump out your table or entire MySQL database on the original database server using the mysqldump command, FTP the resulting dump file to your account here, and then import the dump file at a shell


# first get a mysql connection as per the FAQ

$fcontents = file ('./spreadsheet.xls');
# expects the csv file to be in the same dir as this script

for($i=0; $i $line = trim($fcontents[$i]);
$arr = explode("\t", $line);
#if your data is comma separated
# instead of tab separated,
# change the '\t' above to ','

$sql = "insert into TABLENAME values ('".
implode("','", $arr) ."')";
mysql_query($sql);
echo $sql ."
\n";
if(mysql_error()) {
echo mysql_error() ."
\n";
}
}
?>

Upload a spreadsheet file into the same directory as this script. Then
you edit this script to put in the correct table name instead of
"TABLENAME".

So long as your xls file is tab delimited and has 1 row per line with
all columns in the same order that your mysql database columns are, then
this script will pull all the data out of your XLS file and insert it
all into mysql. Don't forget to connect to mysql in the script before
anything else with a mysql_connect() and mysql_select_db() as shown here
http://www.modwest.com/help/kb6-60.html

Reference: http://www.modwest.com/help/kb6-253.html

Ubuntu Karmic Koala (9.10 Beta) Buggy Touchpad Behavior

17 Oct 09 by Danilo

I recently upgraded my Dell Precision M90 Laptop to Ubuntu Karmic Koala. I’m running it standalone, no Windows anywhere. When I upgraded using “apt-get upgrade” my touchpad didn’t work. Nothing. The touchpad was completely disabled. I assume that this is a problem on M70’s, ect.

Found a fix after searching for two days — in case you’re having the same problem, here it is.

Open a terminal (if you don’t have a mouse hooked up, use Alt+F2). then “su” and give the root password (I tried doing this with sudo and still didn’t have enough permission. I “think” you need to be root). At the prompt, type:

echo options psmouse proto=exps > /etc/modprobe.d/psmouse.modprobe

At the next prompt, type”

reboot

Your touchpad will come back up after rebooting.

Reference: http://danilogurovich.wordpress.com/2009/10/17/ubuntu-karmic-koala-9-10-beta-buggy-touchpad-behavior/