Showing posts with label clone migrate. Show all posts
Showing posts with label clone migrate. Show all posts

Saturday, July 3, 2010

Acronis

SkyHi @ Saturday, July 03, 2010

I've used Acronis True Image on quite a large number of Windows systems over the years, as it is a very good product which has gradually expanded it's (already impressive) feature set throughout that time. Just recently, I had the requirement to image a Linux system and so decided to trial the new Acronis True Image Echo Server on Linux - from what I found, there is no 'workstation' product for Linux like the True Image Home product for Windows, and so had to go for the server version instead. Specifically, I needed to backup a 64-bit CentOS 5.1 virtual machine running inside Citrx XenServer Express.

After registering for the trial version on Acronis.com, downloading the appropriate installation file and moving it into the VM, a quick 'chmod +x TrueImageServerEcho_d_en.i686' made the file ready for installation. The binary is suitable for 32-bit and 64-bit systems - no need for seperate installation files.

After kicking off the installation ('./TrueImageServerEcho_d_en.i686'), it was pretty much a next-next-finish affair, except I soon found I needed to have the kernel source files and gcc installed, for the installation program to correctly configure the SNAPAPI Module. A quick 'yum install -y kernel-xen-devel gcc' fixed the dependancy issue, and after re-run of the True Image installation file, all required software was now installed.

By the way, there was no documentation supplied with the downloaded installation file; I found out some of the information on the Acronis website, and also by browing the setup log file (/var/log/trueimage-setup.log).

Since I don't generally use an X-Windows interface on *nix systems (much prefer the command line via SSH), I've now got to see how far I get with the console tools ('trueimagecmd' and 'trueimagemnt')...


REFERENCES
http://blog.andrewallen.co.uk/acronis-true-image-echo-server-on-centos
http://mymcp.blogspot.com/2010/05/acronis-backup-and-recovery-advanced.html
http://mymcp.blogspot.com/2010/04/installing-acronis-backup-and-recovery.html


Friday, February 26, 2010

Create a Hot Backup Server

SkyHi @ Friday, February 26, 2010

The goal is to create a backup that will provide a server which has been replicated to act as a replacement if the original server goes down. This example is assuming exact same hardware and two servers on the same network.

Step #1: Create a Duplicate Drive

For each drive in the original server you will need to clone a drive so that you can start with the correct users, permissions, etc. Now, one thing to note, if you clone a drive you will have the exact same hostname and IP Address which will need to be modified. Using the dd command you can create a bit-by-bit clone of the drive.

dd if=/dev/sda of=/dev/sdb

You could create an image and then push that image onto another drive if needed.

dd if=/dev/sda of=/bk/disk.img

If you wanted to move the image to a drive you can do this:

dd if=/bk/disk.img of=/dev/sda

This creates an exact clone, beware that large drives may take 6-8 hours. Place the cloned drive into the target machine and change the hostname and IP Address of the target so that it can talk with the original machine.

Step #2: Rsync the Necessary Directories
The rsync command can be run through ssh to provide a secure connection and secure transfer of data. This is the preferred method even on an internal network as it is not much harder to set up and gives you peace of mind.

Synchronize the /var Directory
Since much of what a sever does is placed in the /var directory this is an important directory to synchronize. This is necessary for a database, web server, logs, etc. Note the slash at the end of the /var/ as this will transfer the contents and not create a separate /var/directory.

rsync -avhe ssh /var/ root@192.168.5.49:/var/

Synchronize the /home Directory
You may also need to synchronize the /home directory.

rsync -avhe ssh /home/ root@192.168.5.49:/home/

Create Exceptions
There are some directories that you do not want to synchronize as it will break the connection. The most important in the /etc/ directory is the sysconfig directory which contains your IP Address setups. You can use multiple –exclude options, note the directory to exclude is in single quotes.

rsync -avhe ssh –exclude ’sysconfig’ /etc/ root@192.168.5.49:/etc/

3. Create Passwordless Transfers
It is important that you create an account on the target machine so that you do not need to have a password typed in for each update.

Generate Key Pairs
Key pairs are created by default in the ~/.ssh directory of each user. Note that the ~ symbol is used to represent each user’s home directory. The ssh directory is hidden as indicated by the “.” in front of the directory.
Open a terminal and type: ssh-keygen -t dsa
Accept the default location and type in a passphrase.
You have a public key, which you will share with computers you want to connect to and a private key which you will not share with anyone, ever.
~/.ssh/id_dsa.pub —> public key
~/.ssh/id_dsa —> private key

Share Public Key
Note you do not have to share keys in order to use SSH, you can use passwords just as well.
Move into your ~/.ssh directory
cd ~/.ssh
View Contents of Directory
ls
This should show that you have a public key named id_dsa.pub or id_rsa.pub if you created a rsa key pair.

Copy Public Key
scp id_dsa.pub ip_address_of_remote_machine:my_key
Be sure that you follow the remote machine’s IP address with a colon, or else the command won’t work.
Log into the remote, or virtual, machine via ssh:

ssh ip_address_of_remotel_machine

Copy the contents of the “mykey” file to the “authorized_keys” file as follows:

cat mykey >> ~/.ssh/authorized_keys

The double >> is important as it appends the file, if you used just one > it would create a new file, eliminating your current authorized_keys.
This will add your key to a file of keys that are authorized to access this machine.
Be sure to chmod 644 authorized_keys.


Contact Without Passwords

Now you have both a public key and a private key on your local machine. You can access the remote computer on which you placed your public key with the following command:
ssh 192.168.4.5
This assumes that the IP address of the computer you are making contact with is the IP above, you can also use the domain name, example.com (just for example).
The purpose of the ssh-agent is to save your passphrase so you do not have to enter it each time you start a ssh or scp connection.
At the prompt type:
exec /usr/bin/ssh-agent $SHELL
Then also type
ssh-add
You will need to enter your passphrase and then you are done.
As soon as you log out your passphrase will be dumped by the system. Each time you log in you will need to execute these two commands in the XTerminal to avoid having to enter your password each command.

Now, if you stay logged in, you should be able to run the rsync with ssh command without a password.


REFERENCE

http://beginlinux.com/blog/2009/04/create-a-hot-backup-server/



Thursday, November 5, 2009

How to make a bootable CD from a bootable floppy

SkyHi @ Thursday, November 05, 2009
Put the floppy in the floppy drive, and don't mount it, YET.

Code:

dd if=/dev/fd0 of=/home/sam/floppy.img

Mount /home/sam/floppy.img:
Code:

mkdir /mnt/floppy.img
mount -o loop /home/sam/floppy.img /mnt/floppy.img

Customize the image:
Code:

cd /mnt/floppy.img/
rm
cp /home/sam/bios.bin .

or any other files you want on the CD. But don't exceed the 1.44 MB size of a floppy. Check space left in the mounted image
Code:

df -h

Unmount the floppy image
Code:

cd ..
umount /mnt/floppy.img

Make the .iso CD image file:
Code:

mkisofs -o /home/sam/floppy.img.iso -b /home/sam/floppy.img /home/sam/floppy.img

Burn the iso file to a CD:
Code:

wodim dev=/dev/hdc -sao driveropts=burnfree -dummy /home/sam/floppy.img.iso

This is a dummy burn, with the drive laser off. After you check the dummy run for errors, by looking at the program output, hit the up arrow, delete '-dummy', Enter. If you need a DOS boot floppy image file: http://www.fdos.org/bootdisks/autogen/FDOEM.144.gz .

You want to find out if your girlfriend is cheating on you, having cyber whoopie, or just misbehaving. Even if the computer is secured with a password you can boot with the:
http://www.efense.com/helix
CD and search the entire drive partition for text strings:
Code:

dd if=/dev/sda2 bs=16065 | hexdump -C | grep 'I love you.'

... will search the whole drive partition for the text string specified between the single quotes. Searching an entire disk partition several times can be quite tedious. This particular command string prints the search results to the screen, with the offset where it is located in the partition. Dd works in the decimal system. Hexdump works in hexidecimal. The output text string is at offset 0x020d0d90h. You convert that to decimal with one of the many calculators found in Linux. This is decimal offset: 34,409,872. We want some manageable numbers, custom designed for speed and ease of use. The decimal disk offset is roughly 34 million, so the data we want to view is 34 MB into the partition. We divide 34,409,872 by some power of 2. Experience says 2^13 is about what we want to get a quotient in the thousands. 34,409,872/8192=~4200. The data we want is 8,192 4,200 byte blocks, OR, 4,200 8,192 byte blocks, into the partition. We check: 4200*8192=34406400; 34,409,872-34406400=3472. This means the following command line will start reading 3,472 bytes before the search string location.
Code:

dd if=/dev/sda2 bs=4200 skip=8192 count=2 | hexdump -C > file.txt

... and finish reading approximately 4,200 bytes after the string. This will net you 3.4k of disk contents before the search string, and 4.2k after. That's a 7.6k chunk of disk, plenty for what we're doing. With this method you search all the deleted files, any chat activity, and emails. It works no matter what security is being employed on the machine. It works with NTFS, ext2, ext3, reiserfs, swap, and FAT partitions. But, it is illegal to use this method on a computer you aren't authorized to search. People can be sued, or imprisoned for performing unauthorized searches. On a related note, you can search system memory with this method, by substituting
Code:

/dev/mem

for
Code:

/dev/sda2

.
Write system memory to a CD. This is useful for documenting memory contents without contaminating the HDD. I recommend using a CD-RW so you can practice a little. This doesn't involve dd, but it's cool.
Code:

wodim /dev=/dev/scd0 -raw driveropts=burnfree /dev/mem

to find the cdwriter:
Code:

wodim --devices

This method records raw, so you have to do a:
Code:

dd if=/dev/hdd | hexdump -C | less

to view the recorded memory. You can also employ the string search method above, substituting
Code:

/dev/hdd

for
Code:

/dev/sda2

.
Code:

dd if=/dev/hdd | hexdump -C | grep 'string'

string is any ascii sequence, hex sequence (must be separated with a space: '55aa09' searches for the hex string '55aa09'),
list:
Code:

'[[:alnum:]]' any alphanumeric characters
'[[:alpha:]]' any alpha character
'[[:digit:]]' any numeric character
'[[:blank:]]' tabs and spaces
'[[:lower:]]' any lower case alpha characters
'[[:upper:]]' any uppercase alpha character
'[[:cntrl:]]' ASCII characters 000 thru 037, and 177 octal
'[[:graph:]]' [:alnum:] and [:punct:]
'[[:punct:]]' any punctuation character ` ! ' # $ % ' ( ) * + - . / : ; < = > ? @ [ \ ] ^ _ { | } ~
'[[:space:]]' tab, newline, vertical tab, form feed, carriage return, and space '[[:xdigit:]]' any hex digit ranges('[a-d]' = any, or all abcd, '[0-9]' = any, or all 0123456789)

Code:

dd if=/dev/sda | hexdump -C | grep '[:punct:]' | less

... will return every line from the
Code:

hexdump -C

output that contains any punctuation characters specified above. It will not gather only punctuation characters.
Back up your MBR:
Code:

dd if=/dev/sda of=mbr.bin count=1

Put this on a floppy you make with:
Code:

dd if=boot.img of=/dev/fd0

I back up floppies to a HDD. Floppies don't last forever, so I do:
Code:

dd if=/dev/fd0 of=/home/sam/floppies/backup.bin conv=notrunc

If my floppy fails, I can make unlimited copies:
Code:

dd if=/home/sam/floppies/backup.bin of=/dev/fd0 conv=notrunc

Here is a command line to read your BIOS, and interfaces:
Code:

dd if=/dev/mem bs=1k skip=768 count=256 2>/dev/null | strings -n 8

dd will not duplicate or erase an HPA, OR, host protected area. Dd will erase a disk completely, but not as well as using the hardware secure erase, security erase unit command
Dd need not be black boxed like other inexpensive forensic software:
http://www.cftt.nist.gov/
For a low cost bootable CD based professional ghosting solution, that supports all operating systems and file systems:
http://www.feyrer.de/g4u/
Variation of dd for data rescue off defective media:
http://www.garloff.de/kurt/linux/ddrescue/
Department of Defense implementation of dd:
http://dcfldd.sourceforge.net/
Sdd is useful when input block size is different than output block size, and will succeed in some instances where dd fails:
http://linux.maruhn.com/sec/sdd.html
This is one of the best links I haven't written about dd:
http://www.softpanorama.org/Tools/dd.shtml

Copyright 2008 by AwesomeMachine.
All Rights Reserved.


Public Domain Copyright Material Begins Here:

Note that sending a SIGUSR1 signal to a running 'dd' process makes it
print to standard error the number of records read and written so far,
then to resume copying.

Code:

$ dd if=/dev/zero of=/dev/null& pid=$!
$ kill -USR1 $pid; sleep 1; kill $pid

10899206+0 records in 10899206+0 records out



BLOCKS and BYTES may be followed by the following multiplicative suffixes: c 1, w 2, b 512, kB 1000, K 1024, MB 1000*1000, M 1024*1024, GB 1000*1000*1000, G 1024*1024*1024
So,
Code:

dd if=/dev/sda of=/dev/sdb bs=1GB

Will use one gigabyte block sizes.
bs=4b would give dd a block size of 4 disk sectors. 1 sector=512 bytes.
bs=4k would indicate dd use a 4 kilobyte block size. I have found bs=4k to be the fastest for copying disk drives on a modern machine.

OPERANDS The following operands are supported:
Code:

if=file

Specifies the input path. Standard input is the default.
Code:

of=file

Specifies the output path.
Standard output is the default.
seek=blocks Skip this many blocks in the output file.
Code:

ibs=n

Specifies the input block size in n bytes (default is 512).
Code:

obs=n

Specifies the output block size in n bytes (default is 512).
If no conversion other than
Code:

sync, noerror, and, notrunc

is specified, each input block is copied to the output as a single block without aggregating short blocks.
Code:

cbs=n

Specifies the conversion block size for block and unblock in bytes by n (default is 0). If cbs= is omitted or given a value of 0, using
Code:

block or unblock

produces unspecified results. This option is used only if ASCII or EBCDIC conversion is specified.
Code:

ascii and asciib

operands, the input is handled as described for the unblock operand except that characters are converted to ASCII before the trailing SPACE characters are deleted.
Code:

ebcdic, ebcdicb, ibm, and ibmb

operands, the input is handled as described for the block operand except that the characters are converted to EBCDIC or IBM EBCDIC after the trailing SPACE characters are added.
Code:

files=n

Copies and concatenates n input files before terminating (makes sense only where input is a magnetic tape or similar device).
Code:

skip=n

Skips n input blocks (using the specified input block size) before starting to copy. On seekable files, the implementation reads the blocks or seeks past them. On non-seekable files, the blocks are read and the data is discarded.
Code:

iseek=n

Seeks n blocks from beginning of input file before copying (appropriate for disk files, where skip can be incredibly slow).
Code:

oseek=n

Seeks n blocks from beginning of output file before copying.
Code:

seek=n

Skips n blocks (using the specified output block size) from beginning of output file before copying. On non-seekable files, existing blocks are read and space from the current end-of-file to the specified offset, if any, is filled with null bytes. On seekable files, the implementation seeks to the specified offset or reads the blocks as described for non-seekable files.
Code:

count=n

Copies only n input blocks.
Code:

conv=value

[,value. . . ] Where values are comma-separated symbols from the following list:
Code:

conv=notrunc

Tells dd not to abbreviate blocks of all zero value, or multiple adjacent blocks of zeroes, with five asterisks (when you want to maintain size) Do not use notrunc for copying a larger volume to a smaller volume. Without
Code:

conv=notrunc

Do not truncate the output file.
Code:

ascii

Converts EBCDIC to ASCII.
Code:

asciib

Converts EBCDIC to ASCII using BSD-compatible character translations.
Code:

ebcdic

Converts ASCII to EBCDIC. If converting fixed-length ASCII records without NEWLINEs, sets up a pipeline with
Code:

dd conv=unblock

beforehand.
Code:

ebcdicb

Converts ASCII to EBCDIC using BSD-compatible character translations. If converting fixed-length ASCII records without NEWLINEs, sets up a pipeline with
Code:

dd conv=unblock

beforehand.
Code:

ibm

Slightly different map of ASCII to EBCDIC. If converting fixed-length ASCII records without NEWLINEs, sets up a pipeline with dd
Code:

dd conv=unblock

beforehand.
Code:

ibmb

Slightly different map of ASCII to EBCDIC using BSD-compatible character translations. If converting fixed-length ASCII records without NEWLINEs, sets up a pipeline with
Code:

dd conv=unblock

beforehand. The
Code:

ascii (or asciib), ebcdic (or ebcdicb), and ibm (or ibmb)

values are mutually exclusive. block Treats the input as a sequence of NEWLINE-terminated or EOF-terminated variable-length records independent of the input block boundaries. Each record is converted to a record with a fixed length specified by the conversion block size. Any NEWLINE character is removed from the input line. SPACE characters are appended to lines that are shorter than their conversion block size to fill the block. Lines that are longer than the conversion block size are truncated to the largest number of characters that will fit into that size. The number of truncated lines is reported. unblock Converts fixed-length records to variable length. Reads a number of bytes equal to the conversion block size (or the number of bytes remaining in the input, if less than the conversion block size), delete all trailing SPACE characters, and append a NEWLINE character. The block and unblock values are mutually exclusive.
Code:

lcase

Maps upper-case characters specified by the LC_CTYPE keyword tolower to the corresponding lower-case character. Characters for which no mapping is specified are not modified by this conversion.
Code:

ucase

Maps lower-case characters specified by the LC_CTYPE keyword toupper to the corresponding upper-case character. Characters for which no mapping is specified are not modified by this conversion. The lcase and ucase symbols are mutually exclusive.
Code:

swab

Swaps every pair of input bytes. If the current input record is an odd number of bytes, the last byte in the input record is ignored.
Code:

noerror

Does not stop processing on an input error. When an input error occurs, a diagnostic message is written on standard error, followed by the current input and output block counts in the same format as used at completion. If the
Code:

sync

conversion is specified, the missing input is replaced with null bytes and processed normally. Otherwise, the input block will be omitted from the output. notrunc Does not truncate the output file. Preserves blocks in the output file not explicitly written by this invocation of dd. (See also the preceding
Code:

of=file

operand.)
Code:

sync

Pads every input block to the size of the ibs= buffer, appending null bytes. (If either block or unblock is also specified, appends SPACE characters, rather than null bytes.)

ENVIRONMENT VARIABLES

The following environment variables affect the messages and errors messages of dd:

Code:

LANG

Provide a default value for the internationalisation variables that are unset or null. If

Code:

LANG

is unset or null, the corresponding value from the implementation-dependent default locale will be used. If any of the internationalisation variables contains an invalid setting, the utility will behave as if none of the variables had been defined.

Code:

LC_ALL

If set to a non-empty string value, override the values of all the other internationalisation variables.

Code:

LC_CTYPE

Determine the locale for the interpretation of sequences of bytes of text data as characters (for example, single- as opposed to multi-byte characters in arguments and input files), the classification of characters as upper- or lower-case, and the mapping of characters from one case to the other.

Code:

LC_MESSAGES

Determine the locale that should be used to affect the format and contents of diagnostic messages written to standard error and informative messages written to standard output.

Code:

NLSPATH

Determine the location of message catalogues for the processing of LC_MESSAGES.

Public Domain Copyright Material Ends Here:

Reference: http://www.linuxquestions.org/questions/linux-newbie-8/learn-the-dd-command-362506/

Learn The DD Command Revised

SkyHi @ Thursday, November 05, 2009
This document is made to be easily skimmed. The latest addition was on 5-21-2009, 'DVD backup of hard drive partition'. This is comprehensive documentation for one of the most useful Linux/UNIX Windows commands-dd. It is a bit-stream duplicator for duplicating data. If you have a question, post it.

First Time visitors please leave a reply.


New Content, for a while at least.DVD backup of hard drive partition:
Code:

dd if=/dev/hda3 of=/home/sam/backup_set_1.img bs=1M count=4600
dd if=/dev/hda3 skip=4600 of=/home/sam/backup_set_2.img bs=1M count=4600
dd if=/dev/hda3 skip=9200 of=/home/sam/backup_set_3.img bs=1M count=4600

And so on ... Burn the images to DVD+R/RW:
Code:

wodim -dev=/dev/hdc --driveropts=burnfree /home/sam/backup_set_1.img

and so forth ... To restore, load the DVDs in order, and:
Code:

dd if=/media/dvd/backup_set_1.img of=/dev/hda3 bs=1M conv=sync,noerror

Load another DVD
Code:

dd if=/media/dvd/backup_set_2.img of=/dev/hda3 seek=4600 bs=1M conv=sync,noerror

Load another DVD
Code:

dd if=/media/dvd/backup_set_3.img of=/dev/hda3 seek=9200 bs=1M conv=sync,noerror

and so forth ...


You wrote chat messages and emails on your girlfriend's computer. You deleted everything. But you want to make sure a private investigator and/or computer consultant can't find it.
Code:

dd if=/dev/sda | sed 's/Wendy/Janet/g' | dd of=/dev/sda

Where every instance of Wendy is replaced by Janet, over every millimeter of disk. I picked names with the same number of characters, but you can just pad a smaller name with blanks.

Windows users will find help about 50 lines down from here.

FYI: duplicateing smaller partition, or drive to larger partition, or drive; or vice versa:
Code:

rsync -avH --exclude=/other_mount_point/ /mount_point/* /other_mount_point/

You want to duplicate the root directory tree to another drive, but the other drive is larger. If you use dd, you will get a file system that is smaller then the larger destination drive. To duplicate files, not the file system: Format and mount the destination drive:
Code:

man parted
rsync -avH --exclude=/mnt/destination_drive/ /* /mnt/destination_drive/

will duplicate all the files as files. You need to run:
Code:

grub-install
update-grub

from a the rescue menu of an installation CD/DVD for the target to become bootable. If target was previously bootable, it remains bootable.

MS Windows Section

Use dd for drive cloning, backup, upgrading, and restore tasks. Boot a Windows XP machine with a Knoppix CD; A live CD Linux system. Download Knoppix, burn the iso image file to a CD, boot with it, and clone drives. Drives are described to the dd command using device files. Boot into Knoppix, and open a root shell. It's in the penguin menu. (80 conductor grey ribbon cable) = /dev/hda for master; /dev/hdb for slave. SATA are /dev/sda and /dev/sdb.

Code:

fdisk -l /dev/hda

The partitions on the first drive.
Code:

man fdisk

The manual page for fdisk. Parted to make partitions
Code:

man parted

If one has trouble, ask. Knoppix is slow, because it runs on a CD drive (1/1000 the speed of a HDD). Feel free to read the entire post.
Dd is not presently able to clone Microsoft Windows Vista OEM Partitions.
End Section

Linux DD
The basic command structure is as follows:
Code:

dd if= of= bs=("USUALLY" some power of 2, not less than 512 bytes(ie, 512, 1024, 2048, 4096, 8192, 16384[/b], but can be ANY reasonable number.) skip= seek= conv=.

Source is the data being read. Target is where the data gets written.

Warning!! If you reverse the source and target, you can wipe out a lot of data. This feature has inspired the nickname "dd" Data Destroyer.
Warning!! Caution should be observed when using dd to duplicate encrypted partitions.

Examples: duplicate one hard disk partition to another hard disk:
Code:

dd if=/dev/sda2 of=/dev/sdb2 bs=4096 conv=notrunc,noerror

sda2 and sdb2 are partitions. You want to duplicate sda2 to sdb2. If sdb2 doesn't exist, dd will start at the beginning of the disk, and create it. Be careful with order of if and of. You can write a blank disk to a good disk if you get confused. If you duplicate a smaller partition to a larger one the larger one, using dd, the larger one will now be formatted the same as the smaller one, and there won't be any space left on the drive. The way around this is to use
Code:

rsync

, as described in the beginning of the post
Make an iso image of a CD:
Code:

dd if=/dev/hdc of=/home/sam/myCD.iso bs=2048 conv=sync,notrunc

This duplicates sector for sector. MyCD.iso will be a hard disk image file of the CD. You can mount the image:
Code:

mkdir /mnt/myCD
mount -o loop /home/sam/myCD.iso /mnt/myCD
CD /mnt/myCD
ls

This will display the top level of the CD.
duplicate a floppy disk:
Code:

dd if=/dev/fd0 of=/home/sam/floppy.image bs=2x80x18b conv=notrunc

OR
Code:

dd if=/dev/fd0 of=/home/sam/floppy.image conv=notrunc

The 18b specifies 18 sectors of 512 bytes. 2x multiplies the sector size by the number of heads. 80x specifies cylinders--a total of 1474560 bytes. This issues a single 1474560-byte read request to /dev/fd0 and a single 1474560 write request to
Code:

/home/sam/floppy.image

. Thanks to Sam Cheeseman for documenting this unusual, but highly cool CHS method of specifying the block size (bs=) parameter of dd.
This makes a hard drive image of the floppy, with bootable info intact. The second example uses default bs=512: 1.44 MB floppy sector size. If you're concerned about spies with superconducting quantum-interference detectors, you can always add a "for" loop for US Government DoD approved secure hard disk erasure. Copy and paste the following two lines into a text editor.
Code:

#!/bin/bash
for n in `seq 7`; do dd if=/dev/urandom of=/dev/sda bs=8b conv=notrunc; done

Save the file as 'swqid'.
Code:

chmod a+x swqid

Don't run the program until you are want to wipe the drive.
Overwrite the drive with zeroes
Code:

dd if=/dev/zero of=/dev/sda bs=4k conv=notrunc

[color="red"I just want to make sure my drive is really zeroed out!![/color]
Code:

dd if=/dev/sda | hexdump -C | grep [^00]

... will show every line from the
Code:

hexdump -C

that isn't 0x00h, OR, zero
To be revised at a later date:
To make a bootable flash drive: Download 50 MB Debian based distro here:
http://sourceforge.net/projects/insert/

Plug in the thumb drive into a USB port. Do:
Code:

dmesg | tail

Look where the new drive is, sdb1, or something similar. Do:
Code:

dd if=/home/sam/insert.iso of=/dev/sdb ibs=4b obs=1b conv=notrunc,noerror

Set the BIOS to USB boot, and boot.
End to be revised
duplicate the MBR and boot sector of a floppy to hard drive image:
Code:

dd if=/dev/fd0 of=/home/sam/MBRboot.image bs=512 count=2

This duplicates the first 2 sectors of the floppy.
Cloning an entire hard disk:
Code:

dd if=/dev/sda of=/dev/sdb bs=4096 conv=notrunc,noerror

/dev/sda is the source. /dev/sdb is the target. Do not reverse the intended source and target. It happens. Notrunc means 'do not truncate the output file'. Noerror means to keep going if there is an error. Normally dd stops at any error.
Duplicate MBR only:
Code:

dd if=/dev/sda of=/home/sam/MBR.image bs=446 count=1

This will duplicate the first 446 bytes of the hard drive to a file. If you haven't already guessed, reversing the objects of if and of, on the dd command line, reverses the direction of the write.
Wipe a hard drive: (Boot from a live CD distro to do this.) http://www.efense.com/helix
is a good boot CD. The helix boot environment contains the DoD version of dd called dcfldd. It works the same way, but is has a progress bar.
Code:

dcfldd if=/dev/zero of=/dev/sda conv=notrunc

Beware, dcfldd does not observe the 'seek=' parameter.
This is useful for making the drive almost like new. Most drives have 0xffh written to every byte, from the factory.
Overwrite all the free space on a partition (deleted files you don't want recovered)
Code:

dd if=/dev/urandom of=/home/sam/fileconsumingallfreespace.file

When dd says no room left on device, all the free space has been overwritten with random characters. Then, delete the big file with
Code:

rm

.
Code:

less /home/sam/file.bin

looks like GobblDeeGook, because it's a binary file.
Code:

dd if=/home/sam/file.bin | hexdump -C | less

is readable. Sometimes one wants to look inside a binary file, with no idea of what they are looking for, only clues to what tools made the file, how to uncompress it, the header bytes, or the author's name and/or email.
Virtual memory:
Code:

dd if=/proc/kcore | hexdump -C | less

use PgUp, PgDn, up arrow, down arrow to navigate in less. Less is my favorite editor, except you can't edit with it.
Filesystems:
Code:

dd if=/proc/filesystems | hexdump -C | less

Modules:
Code:

dd if=/proc/kallsyms | hexdump -C | less

Interrupt table:
Code:

dd if=/proc/interrupts | hexdump -C | less

Uptime (seconds):
Code:

dd if=/proc/uptime | hexdump -C | less

Partitions (in KB):
Code:

dd if=/proc/partitions | hexdump -C | less

Memory stats:
Code:

dd if=/proc/meminfo | hexdump -C | less

I put two identical drives in every one of my machines. Before I do anything that most probably spells disaster, like an untested command line in a root shell, that contains
Code:

find / -regex ?*.???* -type f | xargs rm -f "$1"

, I do:
Code:

dcfldd if=/dev/sda of=/dev/sdb bs=4096 conv=notrunc,noerror

and duplicate my present working /dev/sda drive system to the /dev/sdb drive. If I wreck the installation on sda, I boot with the helix CD and:
Code:

dcfldd if=/dev/sdb of=/dev/sda bs=4096 conv=notrunc,noerror

and I get everything back exactly the same as before whatever daring maneuver I was trying didn't work. You can really, really learn linux this way, because you can't wreck what you have an exact duplicate of. You also might consider making the root partition separate from /home, and make /home big enough to hold the root partition, plus more. Then you can do:
Code:

dd if=/dev/sda2 (root) of /home/sam/root.img bs=4096 conv=notrunc,noerror

To make a backup of root, and :
Code:

dd if /home/sam/root.img of=/dev/sda2 (root) bs=4096 conv=notrunc,noerror

To write the image of root back to the root partition if you messed up and can't launch the X server, or edited /etc/fstab and can't figure out what you did wrong. It only takes a few minutes to restore a 15 GB root partition from an image file.



How to make a swap file, or another swapfile on a running system:
Code:

dd if=/dev/zero of=/swapspace bs=4k count=250000
mkswap /swapspace
swapon /swapspace

This can solve out of memory issues due to memory leaks on servers that cannot easily be rebooted.

How to pick proper block size:

Code:

dd if=/dev/zero bs=1024 count=1000000 of=/home/sam/1Gb.file
dd if=/dev/zero bs=2048 count=500000 of=/home/sam/1Gb.file
dd if=/dev/zero bs=4096 count=250000 of=/home/sam/1Gb.file
dd if=/dev/zero bs=8192 count=125000 of=/home/sam/1Gb.file

This method can also be used as a drive benchmark, to find strengths and weaknesses in hard drives:
Read:
Code:

dd if=/home/sam/1Gb.file bs=64k | dd of=/dev/null

Write:
Code:

dd if=/dev/zero bs=1024 count=1000000 of=/home/sam/1Gb.file

When dd finishes it outputs (total size)/(total time). You get the idea.
Play with 'bs=' and 'count=', always having them multiply out to the same toal size. You can calculate bytes/second like this: 1Gb/total seconds = Gb/s. You can get more realistic results using a 3Gb file.

Rejuvenate a hard drive
To cure input/output errors experienced when using dd. Over time the data on a drive, especially a drive that hasn't been used for a year or two, grows into larger magnetic flux points than were originally recorded. It becomes more difficult for the drive heads to decipher these magnetic flux points. This results in I/O errors. Sometimes sector 1 goes bad, resulting in a useless drive. Try:
Code:

dd if=/dev/sda of=/dev/sda

to rejuvenate the drive. Rewrites all the data on the drive in nice tight magnetic patterns that can then be read properly. The procedure is safe and economical.

Make a file of 100 random bytes:
Code:

dd if=/dev/urandom of=/home/sam/myrandom bs=100 count=1

/dev/random produces only as many random bits as the entropy pool contains. This yields quality randomness for cryptographic keys. If more random bytes are required, the process stops until the entropy pool is refilled (waggling your mouse helps). /dev/urandom does not have this restriction. If the user demands more bits than are currently in the entropy pool, it produces them using a pseudo random number generator. Here, /dev/urandom is the Linux random byte device. Myrandom is a file.

Randomize data over a file before deleting it:
Code:

ls -l

to find filesize.
In this case it is 3769
Code:

ls -l afile -rw------- ... 3769 Nov 2 13:41

Code:

dd if=/dev/urandom of=afile bs=3769 count=1 conv=notrunc

duplicate a disk partition to a file on a different partition.

Warning!! Do not write a partition image file to the same partition.
Code:

dd if=/dev/sdb2 of=/home/sam/partition.image bs=4096 conv=notrunc,noerror

This will make a file that is an exact duplicate of the sdb2 partition. You can substitue hdb, sda, hda, etc ... OR
Code:

dd if=/dev/sdb2 ibs=4096 | gzip > partition.image.gz conv=noerror

Makes a gzipped archive of the entire partition. To restore use:
Code:

dd if=partition.image.gz | gunzip | dd of=/dev/sdb2

For bzip2 (slower,smaller), substitute bzip2 and bunzip2, and name the file
Code:

< filename >.bz2

.Restore a disk partition from an image file.
Code:

dd if=/home/sam/partition.image of=/dev/sdb2 bs=4096 conv=notrunc,noerror

Convert a file to uppercase:
Code:

dd if=filename of=filename conv=ucase

Make a ramdrive:
The Linux kernel makes a number a ramdisks you can make into ramdrives. You have to populate the drive with zeroes like so:
Code:

dd if=/dev/zero of=/dev/ram7 bs=1k count=16384

Populates a 16 MB ramdisk.
Code:

mke2fs -m0 /dev/ram7 4096

puts a file system on the ramdisk, turning it into a ramdrive. Watch this puppy smoke.
Code:

debian:/home/sam # hdparm -t /dev/ram7
/dev/ram7:
Timing buffered disk reads: 16 MB in 0.02 seconds = 913.92 MB/sec

You only need to do the timing once, because it's cool. Make the drive again, because hdparm is a little hard on ramdrives. You can mount the ramdrive with:
Code:

mkdir /mnt/mem
mount /dev/ram7 /mnt/mem

Now you can use the drive like a hard drive. This is particularly superb for working on large documents or programming. You can duplicate the large file or programming project to the ramdrive, which on my machine is at least 27 times as fast as /dev/sda, and every time you save the huge document, or need to do a compile, it's like your machine is running on nitromethane. The only drawback is data security. The ramdrive is volatile. If you lose power, or lock up, the data on the ramdrive is lost. Use a reliable machine during clear skies if you use a ramdrive.

Duplicate ram memory to a file:
Code:

dd if=/dev/mem of=/home/sam/mem.bin bs=1024

The device
Code:

/dev/mem

is your system memory. You can actually duplicate any block or character device to a file using dd. Memory capture on a fast system, with bs=1024 takes about 60 seconds, a 120 GB HDD about an hour, a CD to hard drive about 10 minutes, a floppy to a hard drive about 2 minutes. With dd, your floppy drive images will not change. If you have a bootable DOS diskette, and you save it to your HDD as an image file, when you restore that image to another floppy it will be bootable.

Dd will print to the terminal window if you omit the
Code:

of=/dev/output

part.
Code:

dd if=/home/sam/myfile

will print the file myfile to the terminal window.

To search the system memory:
Code:

dd if=/dev/mem | hexdump -C | grep 'some-string-of-words-in-the-file-you-forgot-to-save-before-the-power-failed'

If you need to cover your tracks quickly, put the following commands in a script to overwrite system ram with zeroes. Don't try this for fun.
Code:

mkdir /mnt/mem
mount -t ramfs /dev/mem /mnt/mem
dd if=/dev/zero > /mnt/mem/bigfile.file

This will overwrite all unprotected memory structures with zeroes, and freeze the machine so you have to reboot (Caution, this also prevents committment of the file system journal, and could trash the file system).

You can get arrested in 17 states for doing this next thing. Make an AES encrypted loop device:
Code:

dd if=/dev/urandom of=/home/sam/aes-drv bs=16065b count=100
modprobe loop
modprobe cryptoloop
modprobe aes
losetup -e aes /dev/loop1 ./aes-drv
password:
mkreiserfs /dev/loop1
mkdir /aes
mount -o loop,encryption=aes,acl ./aes-drv /aes
password:
mv /home/sam/porno /aes

to get the porno on the aes drive image.
Code:

umount /aes
losetup -d /dev/loop1
rmmod aes
rmmod cryptoloop
rmmod loop

to make 'aes-drv' look like a 400 MB file of random bytes. Every time the lo interface is configured using losetup, according to the above, and the file 'aes-drv' is mounted, as above, the porno stash will be accessible in /aes/porno. You don't need to repeat the dd command, OR, the format with reiserfs, OR, the mv command. You only do those steps once. If you forget the password, there is no way to recover it besides guessing. Once the password is set, it can't be changed. To change the password, make a new file with the desired password, and move everything from the old file to the new file. Acl is a good mount option, because it allows use of acls. Otherwise your stuck with u,g,o and rwx.

If you are curious about what might be on you disk drive, or what an MBR looks like, or maybe what is at the very end of your disk:
Code:

dd if=/dev/sda count=1 | hexdump -C

Will show you sector 1, or the MBR. The bootstrap code and partition table are in the MBR.
To see the end of the disk you have to know the total number of sectors, and the MAS must be set equal to the MNA. The helix CD has a utility to set this correctly. In the dd command, your skip value will be one less than MNA of the disk. For a 120 GB Seagate SATA drives
Code:

dd if=/dev/sda of=home/sam/myfile skip=234441646 bs=512

,
So this reads sector for sector, and writes the last sector to myfile. Even with LBA addressing, disks still secretly are read in sectors, cylinders, and heads.
There are 63 sectors per track, and 255 heads per cylinder. There is a total cylinder count. 512_bytes/sector*63_sectors/track*255heads=16065*512bytes/cylinder=8,225,280_bytes/cylinder. 63_sectors/track*255_heads=sectors/cylinder. With 234441647 total sectors, and 16065 sectors per cylinder, you get some trailing sectors which do not make up an entire cylinder: 14593.317584812_cylinders/drive. This leaves 5102 sectors which cannot be partitioned, because to be in a partition you have to be a whole cylinder. It's like having part of a person. That doesn't really count as a person. These become surplus sectors after the last partition. You can't ordinarily read past the last partition. But dd can. It's a good idea to check for anything writing to surplus sectors. For our Seagate 120 GB drive, 234,441,647_sectors/drive - 5102_surplus_sectors = 234,436,545 partitionable sectors.
Code:

dd if=/dev/sda of=/home/sam/myfile skip=234436545

writes the last 5102 sectors to myfile. Launch midnight commander (mc) to view the file. If there is something in there, you do not need it for anything. In this case you would write over it with random characters:
Code:

dd if=/dev/urandom of=/dev/sda bs=512 seek=234436545

Will overwrite the 5102 surplus sectors on our 120 GB Seagate drive.

Block size:
One cylinder in LBA mode = 255_heads*63_sectors/track=16065_sectors=16065*512_bytes=8,225,280_bytes. The b means '* 512'. 32130b represents a two cylinder block size. Cylinder block size always works to cover every sector in a partition, because partitions are made of a whole number of cylinders. One cylinder is 8,225,280 bytes. If you want to check out some random area of the disk:
Code:

dd if=/dev/sda of=/home/sam/myfile bs=4096 skip=2000 count=1000

Will give you 8,000 sectors in myfile, after the first 16,000 sectors. You can open that file with a hex editor, edit some of it, and write the edited part back to disk:
Code:

dd if=/home/sam/myfile of=/dev/sda bs=4096 seek=2000 count=1000

Image a partition to another machine:
On source machine:
Code:

dd if=/dev/hda bs=16065b | netcat < targethost-IP > 1234

On target machine:
Code:

netcat -l -p 1234 | dd of=/dev/hdc bs=16065b

Variations on target machine:
Code:

netcat -l -p 1234 | bzip2 > partition.img

makes a compressed image file using bzip2 compression.
Code:

netcat -l -p 1234 | gzip > partition.img

makes a compressed image file using gzip compression. I back up a 100 GB lappy disk on a desktop drive, over a lan connection, and the 100 GB compresses to about 4.0 GB. Most of the drive is empty, so it's mostly zeroes. Repetitive zeroes compress well.
Alert!! Don't hit enter yet. Hit enter on the target machine. THEN hit enter on the source machine.

Netcat is a program, available by default, on most linux installations. It's a networking swiss army knife. In the preceding example, netcat and dd are piped to one another. One of the functions of the linux kernel is to make pipes. The pipe character looks like two little lines on top of one another, both vertical. Here is how this command behaves: This byte size is a cylinder. bs=16065b equals one cylinder on an LBA drive. The dd command is piped to netcat, which takes as its arguments the IP address of the target(like 192.168.0.1, or any IP address with an open port) and what port you want to use(1234).



CONTINUES...SEE NEXT POST

Dd is like Symantec Norton Ghost, Acronis True Image, Symantec Drive Image. You can perform disk drive backup, restore, imaging, disk image, cloning, clone, drive cloning, transfer image, transfer data, clone to another drive or clone to another machine, move Windows XP to a new hard drive, clone Windows XP, clone Windows, transfer Windows, hard drive upgrade, duplicate a boot drive, duplicate a bootable drive, upgrade your operating system hard drive, Tired of reinstalling WinXP Windows XP?

Copyright 2008 by AwesomeMachine.
All Rights Reserved.

Reference: http://www.linuxquestions.org/questions/linux-newbie-8/learn-the-dd-command-362506/

Monday, August 24, 2009

dd: the ultimate disk cloning tool

SkyHi @ Monday, August 24, 2009
I recently had to clone a hard disk so I naturally turned to solutions such as Acronis True Image, but I discovered these did not work (the start-up disk claimed the clong was ‘complete’ when it blatantly wasn’t) and I also tried copying the partition using gParted in Ubuntu but this froze at about 3%, due to corrupt data, which seemed not to be caught in a chkdsk.

I was about to give up hope and copy the files manually when I came across an extremely powerful, yet simple, Unix command: dd.

Using a Linux live disc, dd can be used to copy files, disks and partitions. It can output directly onto another disc or as an image (like an iso).

dd copies data byte-exactly, meaning it won’t get stuck should it encounter corrupted or fragmented data.

dd is nicknamed ‘destroy disk’ as mistyping one letter can wipe your hard disk, so caution must be used.

To copy a hard disk, one would use the following command (whilst root):

dd if=/dev/hdx of=/dev/hdy

hdx is the input hard disk (to be copied) and hdy is the output hard disk (where it is to be copied to).

if = input
of = output

This is extremely important to remember and their names must be altered according to how they are on your computer (use a tool like gParted to find out your drive’s paths). For example, your ‘if’ may be /dev/sdb whilst your ‘of’ may be /dev/sda, but this obviously varies according to how you want to use it and computer-by-computer so you MUST check. An extremely intelligent friend of mine accidentally wiped a hard drive using this command!

Another use would be to use it to produce a CD image:

dd if=/dev/cdrom of=image.iso bs=2k.

Reference: http://www.ghacks.net/2009/01/17/dd-the-ultimate-disk-cloning-tool/

Wednesday, August 19, 2009

Wonders of 'dd' and 'netcat' :: Cloning Operating Systems

SkyHi @ Wednesday, August 19, 2009

Anytime we think about installing OS on more than one system 'cloning' comes to our attention. Because we are too lazy :-). Well that is one of the important characteristics of Systems Administrator so that he/she is forced to automate. In this document we will try to exploit the power of low level data transfer command popularly known as 'dd' and netcat. These programs are available for all major UNIX, Linux and Windows platforms. These commands are fairly popular among Forensics Analysis professionals.


Problem Description:

You got more than 1 machine with almost same hardware. i.e. same hard drive, SCSI card, graphics etc. You setup one single machine from top to bottom and now it is time to replicate OS setup on other machines. Commercial Software such as Ghost does a good job in cloning Windows based machines and now many of these software support Linux ext2 file system also. dd although very crude but gives you enough flexibility to manipulate cloning as you wish. We have demonstrated cloning of hard drives in machines loaded with Linux, Win2K, Solaris, HPUX machines using dd . This document is not a single click solution so you may have to juggle through here first. Once you get a hold of this process then it is very powerful to create your own disk cloning schemes save lots of time and hassle.


Basic concept:

'dd' command can copy any data bit by bit from one location to another location. So a simple command
dd if= of=
where, and can be a file, file system partition or whole hard drive so anything which can be read/write in binary form, dd can handle it. dd however is not a network program. In order to support dd with networking feature we use another nice command 'netcat'. netcat can be used to connect any TCP/UDP servers and a very good tool for diagnostics also. A typical netcat can run both into client server mode. such as:

server% nc -l -p 30000 ==> (Listen for port 30000 on )

client% nc 30000 ==> (Connect to at port 30000, ready to communicate)

This document will explain cloning under Linux, but concept is very similar for all other operating systems also for which 'dd' and 'netcat' binaries are available.


Operating System Cloning (Using STANDALONE machine):

Let us assume we have two drives (sda) and (sdb) attached to the system ( Example: Linux box, but can be any other OS). (sda) is drive with Master OS (let's call it Master OS drive) and (sdb) is a drive (slave drive) where we have to clone data from (sda).

* IMPORTANT (IF you are cloning RAID/root devices): If slave drive has already RAID and/or root partition setup before. (especially / (root) partition), make sure you run fdisk (Use some 3rd Linux box if required) on slave hard drives , remove any partitions on slave hard drives and make it a plain new disk. If you don't do this and connect both drives with Master Linux box, it is very likely that Master linux box may come up with slave drive as (sda) i.e. primary device rather than expected (sdb) because how they appear in SCSI scan list. While following steps below you may destroy contents of actual Master Linux drive. So you have been warned already :-).
* CAUTION (IF you are using SCSI drives) : In case slave drive is connected to external SCSI controllers and if external SCSI controllers appear first in SCSI scan list during boot and BIOS scan, then external drives will get first device name such as /dev/sda , /dev/sdb and your master Linux box may not boot from desired hard drive. In order to avoid this, DISABLE BIOS scan for external SCSI controller. This will cause not to scan SCSI drives connected with external SCSI controller, so Linux will get internal harddrive as /dev/sda and boot from desired disk. Drives on external controller will be available when Linux Kernel try to scan all SCSI bus while booting. For a typical Adaptec SCSI controller:
o During Boot time, Go to SCSI controller BIOS screen (Ctrl-A)
o Select Controller Configuration -> Configure/View Host Adapter Settings -> Advance Configuration options -> Host Adapter Bios = (Disabled: Not Scan) .
o Save options and reboot box, During next boot, you will see SCSI BIOS Not Installed for this card.
DO NOT DISABLE BIOS SCAN FOR INTERNAL SCSI BUS. THIS IS NECESSARY TO FIND PRIMARY BOOT DISK.

Basically in any case IDE ot SCSI make sure you are booting from correct harddrive and that should appear as /dev/sda (Master drive) before you start any cloning process.

Let's assume we have to clone a harddrive (sda). Which has a partition table setup below. It has 1 NTFS partition loaded with WinNT/Win2K and rest Linux partition. (swap, Linux and Raid partition). Assuming second (slave) harddrive (sdb) is also attached to the same system.

Device Boot Start End Blocks Id System
/dev/sda1 1 9 72261 83 HPFS/NTFS
/dev/sda2 10 75 530145 82 Linux swap
/dev/sda3 76 467 3148740 fd Linux raid autodetect
/dev/sda4 468 2200 13920322+ 83 Linux

A simple way to clone this drive (/dev/sda) to another drive attached to this system (/dev/sdb) is to use dd command.

dd if=/dev/sda of=/dev/sdb

This command will copy each bit from sda (Master drive) to sdb (Slave drive) including MBR (Master Boot record). Thus after cloning new drive (sdb) is ready for deployment. This will also copy any information like File System IDs etc.

Since these days drive size is getting bigger and may run upto 100+ GB, this whole dd process may take long time and obviously there is no point in cloning Linux swap area or empty partitions which doesn't contain any useful data yet. Hence in this situation it is best to clone only relevant partitions. For this you need to partition second drive beforehand.

Note: Both drives are partitioned exactly same. If you have different brand harddrives, make sure each partition on second drive must be equal to or greater than first drive partitions. Also make sure File system ID should match for second drive also.


Device Boot Start End Blocks Id System
/dev/sdb1 1 9 72261 83 HPFS/NTFS
/dev/sdb2 10 75 530145 82 Linux swap
/dev/sdb3 76 467 3148740 fd Linux raid autodetect
/dev/sdb4 468 2200 13920322+ 83 Linux

Now cloning process partition by partition will look like:

* First step is to copy MBR (Master Boot Record) to second drive. MBR is read right after BIOS in PCs bootstrap process. In case of Linux this will store LiLo ( Linux Loader) setup to find out Linux or Windows kernel. MBR is located with in first 446 bytes in harddrive (or partition) selected to store MBR during Linux install time or Windows install time. In our Linux installation we have selected MBR to be stored on first harddrive. Steps below will make second drive as a bootable drive.

Note: here we are using whole drive sda and sdb as input and output arguments of dd. (This process of making Solaris, HPUX drives bootable may be different but they allow you to setup boot record also just like PC's MBR)

dd if=/dev/sda of=/dev/sdb bs=446 count=1

* There is no reason to clone swap partition. Swap is raw area.
* Now you can clone other relevant partitions. Let's say /dev/sda3 will contain Linux OS and mounted as / (root) having ext2 or some other file system and /dev/sda4 is mounted as /home but doesn't contain any data. (May or may not contain any file system)

dd if=/dev/sda1 of=/dev/sdb1 ==> Clone NTFS partition
dd if=/dev/sda3 of=/dev/sdb3 ==> Clone RAID-1 partition having ext2 FS or some other.

* So in this case we can save time just by cloning desired partitions.

Operating System Cloning over network:

One major bottleneck in above process is we have to physically open boxes, connect harddrive to Master box and the run clone process. This is easier in case of desktops where you have a liberty to connect external drives (IDE, SCSI bus). But Laptop can hardly house one IDE drive in general and there are no easy way to open and connect second drive for cloning. Thus above process will be highly useful if cloning process can be used over network. There are several possible combination presented here. Idea here is we have Master Linux box up and running over network and we boot slave box having harddrive which is to be cloned but we use some alternate media such as boot CD and boot slave linux using root file system on CD itself *NOT* on harddrive so that we are free to write on slave hardrive.

Master Box-----------network-----------Slave box
[OS(Linux) up and running] [ Booting *NOT * using slave drive]
192.168.0.1 192.168.0.254

One of Following 3 methods can be used to boot slave box using alternative media.

Method [1] Making your own root filesystem on ext2 CDROM. (Not Scalable )

One can make a small Linux distribution (less than 650MB) which can fit into CDROM. Burn this CDROM with ext2 filesystem (not ISO9660) and then use Linux boot floppy to boot from and use CDROM ext2 file system as / (root) file system (read only) (instead of root file system on Harddrive). This process although is doable but has issues like you need to have all possible drivers for network, SCSI etc. Making your custom ext2 read only file system on CD and booting from it would be quite a trial and error issue. If you are interested in making such Cds or bootable CDs see reference section for links. I once did that to clone HP Omnibook 6000 laptops loaded with Linux+Win2K OS together and it worked pretty okay but this is not a scalable solution though.



Method [2] Using popular Linux distribution and floppy combination.

On a similar line Linux distribution such as RedHat/SuSe boot CDROM at OS install time will allow you to boot into some kind of rescue system. In case of RedHat boot from RedHat OS CD and at initial OS install prompt type 'linux rescue ' at the boot time and this will let you use CDROM as root file system and provide you a shell prompt. Linux distribution uses this facility to repair problematic Linux install but we will use this for getting just shell prompt. Great thing about this is most Linux distribution comes up with lots of popular SCSI, network drivers so you don't have to worry about cooking your custom bootable CD.

Many common utilities including 'dd' command usually available in rescue mode. However you need netcat (static binary not dynamically linked) command. You can download netcat distribution and recompile it as a static binary (use -static flag). When I compiled it is small enough to fit into one floppy. So you can copy this into floppy. (I formatted floppy in ext2 format and then mounted in Linux system, copied netcat binary there.)

mkfs /dev/fd0
mount /dev/fd0 /mnt/floppy
cp nc /mnt/floppy
umount /mnt/floppy

So with 'linux rescue' mode and netcat binary on floppy you can use dd and netcat to clone your system over network. As we will see below.



Method [3] Modifying popular Linux distribution CDs and recreating your personal bootable ISO image:

If for some reason netcat won't fit in 1 floppy or you need more utility/binaries. Then you can change Linux distribution (SuSe/RedHat CD). This is a little hack but works.

NOTE: ISO images are read-only file systems. Even if you have an iso image (Say by using dd command )
dd if=/dev/cdrom of=redhat-boot-cd.iso
and if you try to mount this iso file using loopback device with option read/write (-o rw) (you need to have loopback device support (CONFIG_BLK_DEV_LOOP=y) compiled in kernel to do that)
mount -o loop -o rw ./redhat-boot-cd.iso /mnt/cdrom
This won't allow you to write/modify ISO filesystem.

I haven't found any good solutions to edit iso image directly , One such tool is winISO (http://www.winiso.com ) this is a shareware package so you have to pay for it. But you can use this to add more files in your ISO image and burn new image back to new CD. If you know any better solution let me know also :-)

Following steps are useful for adding additional files in RedHat bootable ISO image and burning a new CDs with additional files as of your choice.

* First mount RedHat CD (say RedHat 7.1). mount /dev/cdrom /mnt/cdrom
* Create a directory where you store stuff what goes on new CD. mkdir /home/cdburn
* cd /mnt/cdrom
* (tar cbf 20 - *) | (cd /home/cdburn | tar xvbf 20 -)
* This tar command will copy whole CD (~650MB to your Harddrive). In most cases you do not need all CD. Important stuff you need from directories, dosutils/, images/, RedHat/base. But if you do not want to play much simply copy the whole CD as above tar command and then you can delete RedHat/RPMS directory. These are simply RPM packages and since our intention is not to install OS from CD so we don't need that.
* Create subdirectory directory mkdir /home/cdburn/mystuff/ and add all your stuff such as static version of netcat binary etc. there.
* Now delete all TRANSLATION TABLES (TRANS.TBLS) files, otherwise mkisofs command will complain. find ./ -name "TRANS.TBL" -exec /bin/rm {} \;
* Make bootable iso image out of above distribution (/home/cdburn). Use mkisofs (part of mkisofs-1.9-6 package). The command below will create a bootable ISO image using initial boot image specified by -b, the -c option is for the boot catalog file. The -r option will make appropriate file ownership and modes. This iso image redhat-bootcd.iso is very similar to what is provided by RedHat except it will have our stuff also and we may have deleted any unnecessary contents such as RedHat/RPMS directory.

mkisofs -r -b images/boot.img -c boot.catalog -o /tmp/redhat-bootcd.iso ./

* Finally burn this iso image redhat-bootcd.iso using your cd-burner.

Now the Real drill:

Whatever method you choose to boot slave machine ( RedHat bootable CD + floopy or custom bootable RedHat CD), ultimate aim is to obtain shell, dd and netcat binary after 'linux rescue'. After you get shell you can access files stored on boot CD by changing directory to /mnt/sources/mystuff .
Hopefully your ethernet card has been detected by now. (as most Linux distributions allow OS install over network) if not then you have to load drivers for your ethernet card. Linux distribution documentation usually tells that how and sometimes they provide extra drivers floppy. In case of RedHat these floppy images are generally stored under directory images/ and you can copy these images to your floppy using commands like
dd if= of=/dev/fd0


On Slave machine:
Run netcat command first on slave linux box (that to be cloned and booted using Linux boot CDROM as 'linux rescue' (See also Shell script case [1] in automation section below). Once ethernet card has been detected. (Use ifconfig -a command to check) assign IP address to this interface now on slave machine. Define loopback interface also. (You may choose different IP address for eth0). Also you may need to define /etc/hosts file before you can assign IP address. Use following commands to create your new /etc/hosts. (These are actually created in ram file system RAMFS).

rm /etc/hosts
echo "127.0.0.1 localhost" > /etc/hosts
echo "192.168.0.254 fakehost" >> /etc/hosts

ifconfig lo 127.0.0.1 up
ipconfig eth0 192.168.0.254 up

Assuming Master Linux box (from where you want to clone) is up and running with IP 192.168.0.1.

slave% nc -l -p 9000 | dd of=/dev/sda (Replace /dev/sda with actual drive on your slave machine)

This will listen at port 9000 and whatever it gets at port 9000 will hand over to dd command and dd will simply write that to on slave harddrive (sda) bit by bit. Here I am assuming dd and netcat (nc) are available either through floppy (/mnt/floppy/nc or through /mnt/sources/mystuff/nc). In case of floppy you need to mount floppy first using command:
mount /dev/fd0 /mnt/floppy


On Master machine:
Now Login on master linux box and run following command. (It is advisable that Master Linux box should be in calm state , i.e no major jobs running on the machine). This command below will read master disk bit by bit and throw this bit stream to netcat command which is connected to netcat command at port 9000 on box.

master% dd if=/dev/sda | nc 192.168.0.254 9000

That's it. You may have to wait for long time depending upon network speed and size of your harddrive. Typically 36GB drive may take 50 minutes over 100Mbps link. Again rather than cloning complete drive we can clone only relevant partitions and MBR only. That will make cloning much faster like we saw in above section.




Automating process and Case studies:

One of the primary reason for using dd and netcat way of cloning OS instead of using commercial software such as Ghost is we have a liberty to automate process as we like. Following scripts may help in automating cloning process.

Case [1]: Script for Slave machine (netcat and dd cloning) on the fly.

* Make sure you have netcat command available either /mnt/floppy or /mnt/sources/mystuff area.

=================================================
cloneme.sh :: Shell script for slave machine.
=================================================

#!/bin/sh
############### Edit variables below ######################
FLOPPY_PATH=/mnt/floppy
MYSTUFF_PATH=/mnt/sources/mystuff

# Uncomment only One of the options below.
#### OPTION ==> 1 if using floppy ################
#NC=$FLOPPY_PATH/nc
#### OPTION ==> 2 if using mystuff/ on CD #########
NC=$MYSTUFF_PATH/nc

LPORT=9000
DEST=/dev/sda
SRC=$DEST
############# No need to edit after this in general ###########

if [ $# -eq 1 ]
then
IPADDR=$1
echo "###############################################################"
echo " If there are no errors here. You need to run following"
echo " command on Master Box."
echo ""
echo "dd if=$SRC | nc $IPADDR $LPORT"
echo "###############################################################"

echo ""
echo "##>> Preparing /etc/hosts ##"
rm /etc/hosts
echo "127.0.0.1 localhost" > /etc/hosts
echo "$IPADDR fakehost" >> /etc/hosts

echo "#===================================================================="
echo "NOTE:: If you need to create routes"
echo " #route add -net netmask 255.255.255.0 gw $IPADDR metric 0"
echo "#===================================================================="

echo "##>> Preparing interfaces lo and eth0 ##"
ifconfig lo 127.0.0.1 up
ifconfig eth0 $IPADDR up

echo ""
echo ">>> Now start listening(at $LPORT) for traffic from Master :-)"
echo "$NC -l -p $LPORT | dd of=$DEST"
$NC -l -p $LPORT | dd of=$DEST

echo ""
echo "%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%"
echo " Cloning Process completed..... :-) Reboot Now"
echo "%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%"

else
echo "Usage:: cloneme "
fi




Case [2] Saving Disk Images (Export Image for later use):

Although you can clone running machine over the network anytime. But it is sometimes desirable to store base installation as a reference image and you may want to clone from this pristine image later. With the help of dd you can image disks also. But let's discuss some issues first.
Most 32 bit operating system (Linux for IA32 , Windows etc.) will have physical limitation on max file size. In general practical limit is 2GB as a max. file size. 64 bit OS (Solaris8, HPUX 11.0, Linux for IA64, etc.) will not have this limitation. So if you use dd to copy harddrive image you can maximum image 2GB harddrive. That is pretty useless these days. Fortunately dd can image in chunks and you can specify start and end blocks, skip blocks etc. So idea here is to image your big harddrives in chunks of approx. 2GB files over network. Although I noticed RedHat 7.1 with Linux 2.4.x kernels will allow fie size even bigger than 4GB on ext2 FS.
Also if you want to store images in compressed format (to save space) it is desirable to have each image file size not too large.
Following perl script (export-image.pl) can be used to image local Linux harddrive /dev/hda to remote machine over NFS using dd. If you are not running NFS you can implement same thing using dd and netcat. For now that would be a manual process. If somebody knows a better way to run netcat and transfer multiple files automatically between two machines please let me know and I will cook up some automation script here.

This perl script is actually use dd command something as described below. This is imaging your big harddrive into chunks of 1950 MB files named (1, 2, 3, 4, .....) over NFS to remote machine.
($NFS is NFS destination directory on another server having plenty of space)

For 1st Image:
dd if=/dev/hda of=$NFS/1 bs=1024k count=1950 skip=0
For 2nd image: (Skipping the part of harddrive used for 1st image.)
dd if=/dev/hda of=$NFS/2 bs=1024k count=1950 skip=1950
For 3rd image: (Skipping the part of Harddrive used for 1st+2nd image)
dd if=/dev/hda of=$NFS/3 bs=1024k count=1950 skip=3900
and so on.

In case you want to use netcat you can simply pipe above dd commands manually to netcat and listen using netcat and dd on remote machine, just like we used netcat and dd to clone hardisks above. For example imaging harddrive on machineA and saving image on machineB.

For 1st image:
machineB% nc -l -p 9000 | dd of=1
machineA(master)% dd if=/dev/hda bs=1024k count=1950 skip=0 | nc machineB 9000
For 2nd image:
machineB% nc -l -p 9001 | dd of=2
machineA(master)% dd if=/dev/hda bs=1024k count=1950 skip=1950 | nc machineB 9001
For 3rdimage:
machineB% nc -l -p 9002 | dd of=2
machineA(master)% dd if=/dev/hda bs=1024k count=1950 skip=3900 | nc machineB 9002
and so on.

Once you have images (1, 2, 3, 4 ....) stored on network then you can boot your slave Linux box using bootable CD and pull these images to slave box as described in case [3].

========================================================
export-image.pl :: Perl script to image big harddrive using dd and NFS.
========================================================
#!/usr/bin/perl
#####################################################
#This script will run dd command (in serial) and dump
#1950 blocks (1.9GB) file for each.
#Run script as perl export-image.pl
#####################################################

################ Edit variables below #########################
#device is raw device name for harddrive to be cloned (imaged).
$device="/dev/hda";
#mount NFS file system with large space available which can hold images.
$nfs_path="/nfs/remote/home/tmp";
#Image name (read from user) (Make sure you have $nfs_path/$image directory)
#on remote machine.
$image="ob6000";
############################################################
$dd="/bin/dd";
#For compressing image
$bzip2="/usr/bin/bzip2";
$suffix=".bz2";
############## No need to edit after this #########################

$bs="1024k";
$block_count=1950;
$image_dir="$nfs_path/$image";
$compress=$bzip2;

$proceed=0;

if(!(-d $image_dir) )
{ die "\nOops!! Image Directory $image_dir must exist with chmod 777 permission\n"; }

system("clear");
print < ###########################################################
NOTE:: COMPRESSION TAKE TOO MUCH TIME(Many HOURS) OVER NFS.
So better compress manually latter on server itself.
###########################################################
\n\n Do you want to compress images using $compress [y/n] (Default n) = \t
MSG1

$compress_flag=;
if(($compress_flag eq "y") or ($compress_flag eq "Y"))
{ $compress_flag=1; }
else
{ $compress_flag=0; }

print "\n\n";
print "***************************************************\n";
print " Local Device = $device [SOURCE] \n";
print " Image Dir = $image_dir [TARGET] \n";
print "***************************************************\n\n\n";
print "Dude! I hope you understand what are you doing by pressing [y/Y] here :-) \n";
print " Press [y/Y] if you want to continue .. ";
$con=; chomp($con);


if(($con eq "y") or ($con eq "Y"))
{
$i=0;
$image_size=1; #Some fake value greater than zero.


print "\n\nDisk Imaging starts...\n";
system("date");
while($image_size > 0)
{
$image_name="$image_dir/$i";
print "##############################################\n";
print "Creating Image $image_name\n";
print "##############################################\n";
$skip=$i*$block_count;
print "$dd if=$device of=$image_name bs=$bs count=$block_count skip=$skip \n";
system("$dd if=$device of=$image_name bs=$bs count=$block_count skip=$skip");
if($compress_flag)
{
print "Compressing Image: $bzip2 $image_name => $image_name$suffix\n";
system("$bzip2 $image_name");
$image_name .= "$suffix";
}
++$i;
$image_size=(stat($image_name))[7];
system("date");
}
}
else
{
print "Bye Bye ...\n";
}





Case [3] Importing Disk Images (1, 2, 3, 4 ...) created in Case [2] using netcat, dd and cat

This part is little tricky in the sense we want all images (1, 2, 3, 4, ...) to be imported on slave machine and use dd to write these images serially on slave drive. A very simple set of commands can be used as below.

On Slave machine: (booted through linux rescue). Run following netcat command to capture incoming data stream.

machineC(slave)% nc -l -p 9000 | dd of=/dev/hda

On machineB machine: (where images 1, 2, 3, 4 .... are stored). Run following cat and netcat command. Make sure you cat images in the same sequence as they were imported in case [2]. cat command will simply join these images and throw data stream to netcat which slave machine will pick up and copy bit by bit on slave harddrive.

machineB% cat 1 2 3 4 .... | nc machineC 9000



Case [4] Importing Disk images created in Case[2]:
Most likely 'linux rescue' system won't have NFS support. Which means when you boot slave box using such method you can not access resources over NFS. But if you cook your own CD and that has NFS support and perl the following perl script can be used to fetch images stored earlier from machineB using NFS. This script is actually doing:
($NFS is NFS source directory on another server machineB where you have images 1, 2, 3, 4, ... stored earlier)

For image 1:
dd if=$NFS/1 of=/dev/hda bs=1024k conv=notrunc seek=0
For image 2:
dd if=$NFS/2 of=/dev/hda bs=1024k conv=notrunc seek=1950
For image 3:
dd if=$NFS/3 of=/dev/hda bs=1024k conv=notrunc seek=3900

In any case if you are interested in using perl script below (if you have perl and NFS client support on slave linux box).

========================================================
import-image.pl
========================================================

#!/usr/bin/perl
#####################################################
#This script will run dd command (in serial) and dump
#and import image.
#####################################################

##############################################################################
#device is target raw device name for harddrive to be cloned.
$device="/dev/hda";
#mount NFS file system with large space available which can hold images.
$nfs_path="/mnt/images";
#Image name (read from user)
$image="ob6000";
###############################################################################
$dd="/bin/dd";
#$bzcat="/usr/bin/bzcat";
#$suffix=".bz2";

$bs="1024k";
$block_count=1950;
###############################################################################
$image_dir="$nfs_path/$image";

$proceed=0;

if(!(-d $image_dir) )
{ die "\nOops!! No Image Directory $image_dir\n"; }

system("clear");
print "***************************************************\n";
print " Local Device = $device [TARGET]\n";
print " Image Dir = $image_dir [SOURCE]\n";
print "***************************************************\n\n\n";
print "Dude! I hope you understand what are you doing by pressing [y/Y] here :-) \n";
print " Press [y/Y] if you want to continue .. ";
$con=; chomp($con);
print " Once Again!!! Press [y/Y] if you want to continue .. ";
$con=; chomp($con);

system("date");
if(($con eq "y") or ($con eq "Y"))
{
print "\n\nDisk Imaging import starts...\n";

$i=0;
$image_name="$image_dir/$i";
while(-f $image_name )
{
print "##############################################\n";
print "Importing Image $image_name\n";
print "##############################################\n";
$seek=$i*$block_count;
print "##############################################\n";
$seek=$i*$block_count;
print "$dd if=$image_name of=$device bs=$bs conv=notrunc seek=$seek \n";
#system("$bzcat $image_name | $dd of=$device bs=$bs conv=notrunc seek=$seek");
system("$dd if=$image_name of=$device bs=$bs conv=notrunc seek=$seek");
++$i;
$image_name="$image_dir/$i";
system("date");
}
}
else
{
print "Bye Bye ...\n";
}




Other Operating Systems Tips:

You can pretty much do same in other operating systems also. This section quickly list few tips that may be useful.


Windows:

* You can find GNU utilities ( http://unxutils.sourceforge.net/ ) for Win32 platforms which includes dd.exe command.
* dd.exe syntax is similar to as you use on Linux side. For physical partition you may have to use devicename something like \\.\PhysicalDrive0 etc. For example

dd if=\\.\PhysicalDrive0 of=

* You can download netcat for windows ( http://www.l0pht.com/~weld/netcat/ )
* If you are cloning WinNT/2K system , you need to change SID for the new cloned system if machine is participating in Windows domain. You can use Ghostwalker program from Ghost distribution or can use newsid.exe from http://www.sysinternals.com


Solaris:

* Burning bootable CDROM for Solaris. ( http://www.lka.ch/projects/solcdburn/solcdburn.html )
* Sun Blue print : http://www.sun.com/software/solutions/blueprints/0301/BuildBoot.pdf
* To Make disk bootable (just like putting MBR in Linux) use command installboot(1M)

Others: (Make disk bootable)

* Irix : dvhtool
* HP-UX: mkboot
* Tru64: disklabel




Conclusion:

Few possible uses of netcat and dd shown in this document. Although methods presented here are very simple and easy to use but have few pros and cons also. This technique is very good for on the fly OS cloning. When we image the whole drive we need the equivalent harddrive space on other machine. This may not be very much practical. You can try compressing those images which will save lots of space. I noticed dd image can be compressed upto 30-80% depending upon real data on the drive using gzip/compress program. This cloning and imaging method may be very effective in forensic analysis where sometimes you need an exact snapshot of harddrive including swap space partitions. You can always break your images in small pieces (may be compress them) transfer over network to somewhere else and reproduce data. As mentioned above one of the great advantage here is to custom your own cloning scheme.




References:

1. GNU utilities for Win32. http://unxutils.sourceforge.net/
2. netcat for Windows. http://www.l0pht.com/~weld/netcat
3. First Attempt at Creating a Bootable Live Filesystem on a CDROM http://www.linuxgazette.com/issue54/nielsen.html
4. Good Site for Windows utilities such as newsid.exe: http://www.sysinternals.com
5. Modifying ISO image http://www.winiso.com
6. Solaris Bootable CD creation: http://www.lka.ch/projects/solcdburn/solcdburn.html
7. Sun Blueprint: http://www.sun.com/software/solutions/blueprints/0301/BuildBoot.pdf
8. Linux on Floppy: http://www.toms.net/rb/
9. Static binaries for Linux.




Reference: http://74.125.155.132/search?q=cache:jx4674quVY0J:www.rajeevnet.com/hacks_hints/os_clone/os_cloning.html+linux+scan+for+netcat&cd=6&hl=en&ct=clnk&gl=ca

Clone your Ubuntu installation onto a new hard disk

SkyHi @ Wednesday, August 19, 2009
his article is excerpted from the newly published book Ubuntu Kung Fu and published with the express permission of the publisher, the Pragmatic Programmers, LLC.

Three things must be done. First, you must discover how Ubuntu refers to the hard disks. Second, you must install ddrescue and then use it to clone the disk. Third, once ddrescue has finished, you must use the Gparted utility to expand the disk partition(s) (assuming that the new disk is bigger than the old one, which is almost certainly going to be the reason for upgrading in the first place).

It's not a good idea to clone a hard disk that's in use (any more than it's a good idea to repair a car while it's being driven), so you must use your Ubuntu install CD's live distro mode. To carry out the following instructions, boot from your Ubuntu install CD, and select Try Ubuntu from the boot menu.

Note that all the following stages are carried out using the Ubuntu install CD's live distro mode. At no point in the process do you need to boot into your standard Ubuntu installation, apart from to test the cloned disk at the end.
Preparing to clone

Before starting, it's a good idea to do three things in preparation. First, back up all valuable personal files to CD/DVD-R/RW disc, a USB keystick, or an external hard disk. The instructions that follow involve drastic fundamental disk management and the possibility of data loss.

Second, it's a good idea to check the filesystem of the original hard disk for errors and possibly enact repairs. Ideally, you should check the Windows filesystem for errors too.

Third, remove any USB memory sticks, card readers, or other kinds of attachable storage, such as MP3 players or mobile phones. This will avoid confusion when partitioning.

After all this, open a terminal window, and type the command sudo fdisk -l, which will scan the hard disks and list their partitions. Here are the results from my test system:

Disk /dev/sda: 81.9 GB, 81964302336 bytes
255 heads, 63 sectors/track, 9964 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x1c381c37
Device Boot Start End Blocks Id System
/dev/sda1 * 1 4742 38090083+ 7 HPFS/NTFS
/dev/sda2 4743 9964 41945715 5 Extended
/dev/sda5 4743 9744 40178533+ 83 Linux
/dev/sda6 9745 9964 1767118+ 82 Linux swap
/Solaris
Disk /dev/sdb: 120.0 GB, 120034123776 bytes
255 heads, 63 sectors/track, 14593 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0xb94838a4
Disk /dev/sdb doesn't contain a valid partition table

Two hard disks are listed in the results: look for the headings Disk /dev/sda and Disk /dev/sdb. Beneath each heading is technical information about the disk, and beneath that the partitions on that disk are listed.

It should be obvious that, on my test computer, /dev/sdb is the new hard disk because it has no partitions (it "doesn't contain a valid partition table"), while /dev/sda has the standard partition layout of an Ubuntu system. Yours will probably be similar, if not identical.

Look for the reference to your new hard disk and make a note of it. In my case, I make a note of /dev/sdb. Then type sudo cfdisk -z /dev/sdb to start the cfdisk partitioning program, which we'll use to write an initial partition table to the disk. If necessary, replace /dev/sdb with the details of the new hard disk you discovered earlier. When cfdisk starts, type W (note that's Shift+w) and then type yes to write a blank partition table. Then press q to quit cfdisk. You can ignore the handful of minor errors that are reported.
Cloning the disk

Now that we have this information, we can install ddrescue and use it to clone the disk. This needs to be installed because it isn't a default system tool. Although the computer is running the Ubuntu install CD live distro mode, it's still possible to install additional software from the online repositories. However, before doing this, it's necessary to enable the Universe software repository (of course, you will need to use NetworkManager to get online too, if you haven't already). Click System -> Administration -> Software Sources, and put a check in the box alongside Community-maintained Open Source software (universe). Then click the Close button, and agree to refresh the list of software when asked.

After this, type sudo apt-get install gddrescue at the prompt to install ddrescue.

Use ddrescue by first specifying the old hard disk, then the new hard disk. Add the -v command option to provide a status report as the command progresses:

$ sudo ddrescue -v /dev/sda /dev/sdb

It's extremely important that you ensure you get the old and new disks in the right order. Otherwise, you might well overwrite the data on your old disk!

Once the cloning has finished -- it will probably take an hour or more, depending on the size of the original hard disk -- you should shut down the computer, remove the old disk (you must disconnect the old disk before you can continue!), and boot from the cloned copy to test things. If you use Windows XP/Vista, it might object to a new hard disk as part of its Windows Genuine Advantage system, and you might have to revalidate online. Of course, Ubuntu will work fine without any such worries.

Assuming everything works correctly, you can move on to the next step: expanding the partitions to take advantage of the larger hard disk.
Expanding the partitions

Before attempting to expand the partitions, it's a good idea to check that your Ubuntu partition's filesystem is sound. To do this, boot into the Ubuntu install CD's live distro mode as before. Open a terminal window and type the command sudo fsck.ext3 -f /dev/sda5 to perform a disk check (assuming that Ubuntu is installed alongside Windows on your hard disk in the standard configuration).

Once this has completed, close the terminal window and click System -> Administration -> Partition Editor. What happens next depends on your requirements. If you just want to expand the Ubuntu partition, follow these steps:

1. In the Partition list, right-click the linux-swap entry and select Swap off. This will stop Ubuntu's live distro mode from accessing the swap partition so that it can be moved on the hard disk.
2. Before anything else can happen, you must resize the extended partition that contains Ubuntu. Right-click the extended entry in the list and select Resize/Move. In the dialog box that appears, change the Free Space Following (MiB) box to read 0, then press Tab. This will cause the partition to be expanded to fill the space. Click the Resize/Move button when done. Bear in mind that no changes are carried until you click the Apply button, which you will do after making all the changes to the disk's partitions.
3. Right-click the linux-swap partition once again, and select Resize/Move. In the dialog box that appears, click and drag the graphical representation of the partition to the end of the free space (in other words, click and drag it to the right of the graphical display). After this, the Free Space Following (MiB) box should read 0. Click Resize/Move.
4. Back in the main GParted program window, right-click the ext3 entry in the list, and select Resize/Move. Click and drag the rightmost edge of the partition in the graphical representation so that it "grows" to fill the free space. Eventually the Free Space Following (MiB) box will read 0. When this is the case, click the Resize/Move button.
5. Finally, click the Apply button on the main GParted toolbar. Then click Apply in the dialog box that appears, and sit back and wait while the partitions are moved and resized. If you want to see what's happening, click the small arrow alongside Details in the Applying pending operations dialog box.
6. When GParted has finished, close the program, then open a terminal window. Enter sudo fsck.ext3 -f /dev/sda5, which will once again check the Ubuntu partition for errors (and, again, these steps assume that Ubuntu is installed alongside Windows on your hard disk in the standard configuration). If there are any errors, you'll be prompted to repair them. Usually you can agree to the repair.

After the filesystem check, you can reboot your computer from the new hard disk. You should find the Ubuntu partition is now larger.

If you want to resize your Windows partition too, these steps are still relevant. However, you will have to move the swap and ext3 partitions, as well as the extended partition containing them, before resizing the NTFS partition.

If you want to dispose of the old hard disk or pass it on to somebody else, be sure to securely wipe it. However, don't do so until you're 100% sure your new cloned copy is working correctly. I usually wait at least a week or two to ensure the copy works fine before doing anything to the old disk.

Reference: http://linux.com/learn/tutorials/8225-clone-your-ubuntu-installation-onto-a-new-hard-disk