Saturday, August 15, 2009

Customizing the Linux bootup and shutdown processes

SkyHi @ Saturday, August 15, 2009
PC Boot and Linux Init Process:

  1. BIOS: The Basic Input/Output System is the lowest level interface between the computer and peripherals.
    The BIOS performs integrity checks on memory and seeks instructions on the Master Boor Record (MBR) on the floppy drive or hard drive.
  2. The MBR points to the boot loader (GRUB or LILO: Linux boot loader).
  3. Boot loader (GRUB or LILO) will then ask for the OS label which will identify which kernel to run and where it is located (hard drive and partition specified). The installation process requires to creation/identification of partitions and where to install the OS. GRUB/LILO are also configured during this process. The boot loader then loads the Linux operating system.
    • See the YoLinux tutorial on creating a boot disk for more information on GRUB and LILO and also to learn how to put the MBR and boot loader on a floppy for system recovery.
  4. The first thing the kernel does is to execute init program. Init is the root/parent of all processes executing on Linux.
  5. The first processes that init starts is a script /etc/rc.d/rc.sysinit
  6. Based on the appropriate run-level, scripts are executed to start various processes to run the system and make it functional.

The Linux Init Processes:

The init process is the last step in the boot procedure and identified by process id "1". Init is responsible for starting system processes as defined in the /etc/inittab file. Init typically will start multiple instances of "getty" which waits for console logins which spawn one's user shell process. Upon shutdown, init controls the sequence and processes for shutdown. The init process is never shut down. It is a user process and not a kernel system process although it does run as root.

System Processes:

Process ID Description
0 The Scheduler
1 The init process
2 kflushd
3 kupdate
4 kpiod
5 kswapd
6 mdrecoveryd

Init config file (Red Hat 7.3-9.0, Fedora Core 1-3): /etc/inittab

# Author:       Miquel van Smoorenburg, 
# Modified for RHS Linux by Marc Ewing and Donnie Barnes

id:3:initdefault:

# System initialization.
si::sysinit:/etc/rc.d/rc.sysinit

l0:0:wait:/etc/rc.d/rc 0
l1:1:wait:/etc/rc.d/rc 1
l2:2:wait:/etc/rc.d/rc 2
l3:3:wait:/etc/rc.d/rc 3
l4:4:wait:/etc/rc.d/rc 4
l5:5:wait:/etc/rc.d/rc 5
l6:6:wait:/etc/rc.d/rc 6

# Things to run in every runlevel. This line is only in Red Hat 7.X Used to flush disk buffers.
ud::once:/sbin/update

# Trap CTRL-ALT-DELETE
ca::ctrlaltdel:/sbin/shutdown -t3 -r now

# When our UPS tells us power has failed, schedule a shutdown for 2 minutes from now.
pf::powerfail:/sbin/shutdown -f -h +2 "Power Failure; System Shutting Down"

# If power was restored before the shutdown kicked in, cancel it.
pr:12345:powerokwait:/sbin/shutdown -c "Power Restored; Shutdown Canceled"

# Run gettys in standard runlevels
1:2345:respawn:/sbin/mingetty tty1
2:2345:respawn:/sbin/mingetty tty2
3:2345:respawn:/sbin/mingetty tty3
4:2345:respawn:/sbin/mingetty tty4
5:2345:respawn:/sbin/mingetty tty5
6:2345:respawn:/sbin/mingetty tty6

# Run xdm in runlevel 5
x:5:respawn:/etc/X11/prefdm -nodaemon

Note that this config file directs the init process to run the shell script /etc/rc.d/rc.sysinit. This script should be used as is and NOT changed. Extend rc.local and NOT this script. This will (not in exact order):

  • Run /sbin/initlog
  • Run devfs to generate/manage system devices
  • Run network scripts: /etc/sysconfig/network
  • Start graphical boot (If so configured): rhgb
  • Start console terminals, load keymap, system fonts and print console greeting: mingetty, setsysfonts
    The various virtual console sessions can be viewed with the key-stroke: ctrl-alt-F1 through F6. F7 is reserved for the GUI screen invoked in run level 5.
  • Mount /proc and start device controllers.
  • Done with boot configuration for root drive. (initrd) Unmount root drive.
  • Re-mount root file system as read/write
  • Direct kernel to load kernel parameters and modules: sysctl, depmod, modprobe
  • Set up clock: /etc/sysconfig/clock
  • Perform disk operations based on fsck configuration
  • Check/mount/check/enable quotas non-root file systems: fsck, mount, quotacheck, quotaon
  • Initialize logical volume management: vgscan, /etc/lvmtab
  • Activate syslog, write to log files: dmesg
  • Configure sound: sndconfig
  • Activate PAM
  • Activate swapping: swapon

Local system boot processes can be placed in file: /etc/rc.d/rc.local

The system will then boot to the runlevel set by the directive initdefault.

Also see:


Linux init Run Levels:

The Linux boot process has six states of operation of which "0" is the shutdown state and "3" and above are fully operational with all essential processes running for user interaction. Upon system boot the LINUX system /sbin/init program starts other processes by performing the following:

  • Init will bring up the machine by starting processes as defined in the /etc/inittab file.

  • The computer will be booted to the runlevel as defined by the initdefault directive in the /etc/inittab file.
        id:5:initdefault:
    In this example a runlevel of "5" is chosen. Runlevel "5" will boot the system into GUI mode using XDM and X-Windows. Booting to runlevel "3" (often called console mode) is often used by servers which do not need a graphical user interface. If booted to init level "3" one can promote the run level with the command [root prompt]# init 5. See the more detailed explanation of run levels below.

    The inittab file will allow you to capture key sequences (ctrl-alt-del), start dial in internet connections etc.

  • One of these process started by init is /sbin/rc. This script runs a series of scripts in the directories /etc/rc.d/rc0.d/, /etc/rc.d/rc1.d/, /etc/rc.d/rc2.d/, etc

  • Scripts in these directories are executed for each boot state of operation until it becomes fully operational. Scripts beginning with S denote startup scripts while scripts beginning with K denote shutdown (kill) scripts. Numbers follow these letters to denote the order of execution. (lowest to highest)


Runlevel "3" will boot to text or console mode and "5" will boot to the graphical login mode ( "4" for slackware)

Runlevel Scripts Directory
(Red Hat/Fedora Core)
State
0 /etc/rc.d/rc0.d/ shutdown/halt system
1 /etc/rc.d/rc1.d/ Single user mode
2 /etc/rc.d/rc2.d/ Multiuser with no network services exported
3 /etc/rc.d/rc3.d/ Default text/console only start. Full multiuser
4 /etc/rc.d/rc4.d/ Reserved for local use. Also X-windows (Slackware/BSD)
5 /etc/rc.d/rc5.d/ XDM X-windows GUI mode (Redhat/System V)
6 /etc/rc.d/rc6.d/ Reboot
s or S
Single user/Maintenance mode (Slackware)
M
Multiuser mode (Slackware)

One may switch init levels by issuing the init command with the appropriate runlevel. Use the command "init #" where # is one of s,S,0,1,3,5,6. The command telinit does the same.

The scripts for a given run level are run during boot and shutdown. The scripts are found in the directory /etc/rc.d/rc#.d/ where the symbol # represents the run level. i.e. the run level "3" will run all the scripts in the directory /etc/rc.d/rc3.d/ which start with the letter "S" during system boot. This starts the background processes required by the system. During shutdown all scripts in the directory which begin with the letter "K" will be executed. This system provides an orderly way to bring the system to different states for production and maintenance modes.

If you installed all demons (background processes), Linux will run them all. To avoid slowing down your machine, remove unneeded services from the start-up procedure. You can start/stop individual demons by running service init scripts located in the /etc/init.d/ directory:

  • /etc/rc.d/init.d/ (Red Hat/Fedora) Also /etc/init.d/ which is linked to /etc/rc.d/init.d/
  • /etc/init.d/ (S.u.s.e.)
  • /etc/init.d/ (Ubuntu / Debian)
and issuing the command and either the start, stop, status, restart or reload option i.e. to stop the web server:
  • /etc/init.d/httpd stop

Use the command ps -aux to view all process on your machine.

TIP: List state and run level of all services which can be started by init: chkconfig --list
or
service --status-all | grep running (Red Hat/Fedora Core based systems)

GUI tool: /usr/X11R6/bin/tksysv


Run Level Commands:

Init Script Activation:

Adding a script to the /etc/rc.d/rc#.d/ directory with either an S or K prefix, adds the script to the boot or shutdown process. The scripts are run in numerical order. S20abc is run before S30xyz. The extensibility to the boot and shutdown procedures of the operating system is one of the strengths of UNIX. The orderly sequential initiation of processes can be coordinated for dependent processes. The orderly shutdown of processes is often required of complex programs such as databases. This is how it is done. Individual processes may be monitored, shutdown and started at any time using these scripts. i.e. /etc/rc.d/rc2.d/httpd start. The modifiers start, stop or status may be used.

The start/stop/status scripts actually reside in the directory:

  • /etc/rc.d/init.d/ (Red Hat/Fedora) Also /etc/init.d/ which is linked to /etc/rc.d/init.d/
  • /etc/init.d/ (S.u.s.e. and Ubuntu / Debian)
and are linked to the appropriate directories. These links may be created or destroyed using the chkconfig command. i.e. chkconfig --del httpd will remove the web server from the startup and shutdown process. Inversely chkconfig --add httpd will add it to the startup/shutdown process by generating links from the script in /etc/rc.d/init.d/ to the appropriate /etc/rc.d/rc#.d/ directory. For more information see the LINUX manual page on init.

Basic services include:

System Service Description
anacron Run jobs which were scheduled for execution while computer was turned off. Catch up with system duties.
arpwatch Keeps track of IP address to MAC address pairings
atd Run scheduled batch jobs.
autofs automounts file systems on demand.
bluetooth, pand, hidd, dund Bluetooth netwoork support.
crond Job sheduler for periodic tasks.
gpm Allows console terminal cut and paste. (Non X-window consoles)
https Apache web server.
iptables Firewall rules interface to kernel.
keytable Loads selected keyboard map as set in /etc/sysconfig/keyboard
kudzu New hardware probe/detection during system boot.
lpd or cups Network printer services.
microcode_ctl Uploads microcode to kernel and ultimately to the Intle Pentium processor. (Hardware specific.)
mysqld Database services
named DNS name services (Bind)
network Active network services during system boot. Required for network connectivity.
nfs Network file system. Unix file sharing services. Also uses services: nfslock, portmap, rpcgssd, rpcidmapd, rpcsvcgssd
nscd Password and group lookup services for use with network authentication (NIS, LDAP,...).
ntpd Network Time Protocol time synchronization services.
random Random number generation tool used for encryption.
rawdevices Enables raw IO. Useful for Oracle and software which utilizes this for high speed disk access.
smb SAMBA: MS/Windows PC file sharing services
syslog System log file facility.
ypbind NIS file sharing/authentication infrastructure service.
yppasswd NIS file sharing/authentication infrastructure service.
ypserv NIS file sharing/authentication infrastructure service.
xfs X-Windows font server.

Recommended basic services: anacron, ard, autofs, crond, gpm, iptables, keytable, kudzu, microcode_ctl (Intel32 hardware only), network, random. syslog
Graphics Workstation - add: xfs
File Server for PC clients - add: smb
Print Server - add: lpd or cups (hplip - HP Linux Imaging and Printing)
File server Linux/Unix clients - add: nfs, netfs, nfslock, portmap, ypbind, yppasswd, ypserv; NFSv4 add: rpcgssd, rpcidmapd, rpcsvcgssd
Web Server - add: httpd, tux, xinetdi, sshd


GUI configuration tools:

GUI tools can help you configure the appropriate services to start and provide a description of each service available:
  • Fedora/RHEL: /usr/bin/system-config-services
    (Also /usr/sbin/serviceconf)
  • Red Hat 8.0/9.0: /usr/bin/redhat-config-services
  • Ubuntu / Debian:
    • bum (Boot Up Manager) (GUI image)
    • /usr/bin/services-admin (GUI image)
    • /usr/sbin/sysv-rc-conf (console program - see below)
Red Hat / Fedora Core GUI: system-config-services (and redhat-config-services)

Red Hat/Fedora Core text console services selection tool: /usr/sbin/ntsysv

Debian/Ubuntu: sysv-rc-conf
(Install: aptget install sysv-rc-conf)

Init Script:

A single copy of the script is located in the directory: /etc/rc.d/init.d/script-name (Red Hat/Fedora) or /etc/init.d/script-name (Ubuntu / Debian).
Use the command chkconfig to generate soft links to the appropriate directories for the various run levels.

#!/bin/sh
#
# Startup script for program
#
# chkconfig: 345 85 15 - This statement tells the chkconfig command how to add or delete this process to the boot process
# description: Description of program
# processname: process-name
# pidfile: /var/run/process-name.pid

# Source function library. This creates the operating environment for the process to be started
. /etc/rc.d/init.d/functions

case "$1" in
start)
echo -n "Starting process-name: "
daemon process-name - Starts only one process of a given name.
echo
touch /var/lock/subsys/process-name
;;
stop)
echo -n "Shutting down process-name: "
killproc process-name
echo
rm -f /var/lock/subsys/process-name
rm -f /var/run/process-name.pid - Only if process generates this file
;;
status)
status process-name
;;
restart)
$0 stop
$0 start
;;
reload)
echo -n "Reloading process-name: "
killproc process-name -HUP
echo
;;
*)
echo "Usage: $0 {start|stop|restart|reload|status}"
exit 1
esac

exit 0

The bash script functions daemon, killproc and status can all be found in the script /etc/rc.d/init.d/functions. (Red Hat/Fedora distributions)


The script must be executable to work. (chmod +x script-name).

The script may be used to start and stop processes. i.e.:

  • /etc/rc.d/init.d/httpd restart
    (Ubuntu / Debian / S.u.s.e.: /etc/init.d/apache2 restart)
  • /etc/rc.d/init.d/httpd stop
  • /etc/rc.d/init.d/httpd start
OR use the Red Hat/Fedora core based service command:
  • service httpd restart
  • service httpd stop
  • service httpd start

Note that two lines in the script enable the chkconfig command to control the script for the boot and shutdown process.

# chkconfig: 345 85 15
# description: Description of program

When added to the boot process using the "chkconfig --add script-name" command the start order/priority will be set to 80 while the stop/shutdown order will be set to 15. The process will be added to runlevels 3, 4 and 5. This is enabled by generating links from the location of the script (/etc/rc.d/init.d/) to the directory for the appropriate run level: /etc/rc.d/rc#.d/. The file name in the run level directory will reflect if it is used for boot (starts with an "S") or shutdown (starts with a "K")

chkconfig:

The (Red Hat/Fedora/IRIX) chkconfig command generates and breaks links between the directory /etc/rc.d/init.d/ and the appropriate run level directory: /etc/rc.d/rc[0-6].d/ to control boot process initiation and process shutdown.

     chkconfig [--level ]  on | off | reset >
chkconfig --list
chkconfig --list
chkconfig --add

chkconfig --del
chkconfig --level 0123456 off

Examples:

  • chkconfig --level 345 httpd on - forces apache to be invoked for run levels 3, 4 and 5.
  • chkconfig --add httpd - Start the web server daemon upon system boot.
  • chkconfig --del sendmail - Do not start the sendmail daemon upon system boot.
  • chkconfig --list - List all services and init levels.
  • chkconfig --list | grep on - List all services to be started upon system boot.

Using chkconfig to administer xinetd processes.

  • chkconfig wu-ftpd on - Turn on FTP service managed by xinetd.
  • chkconfig ipop3 off - Turn off POP3 service managed by xinetd.
This will reconfigure the appropriate xinetd file (in directory /etc/xinetd.d/) and restart the xinetdprocess.

Also see: chkconfig - Linux man page.

Related Commands:

service Display status of system services.
Example: service --status-all
Help: service --help

Also see: service - Linux man page.

Links and resources:


OPTIONS

-a
Use /etc/shutdown.allow.
-t sec
Tell init(8) to wait sec seconds between sending processes the warning and the kill signal, before changing to another runlevel.
-k
Don't really shutdown; only send the warning messages to everybody.
-r
Reboot after shutdown.
-h
Halt after shutdown.
-n
[DEPRECATED] Don't call init(8) to do the shutdown but do it ourself. The use of this option is discouraged, and its results are not always what you'd expect.
-f
Skip fsck on reboot.
-F
Force fsck on reboot.
-c
Cancel an already running shutdown. With this option it is of course not possible to give the time argument, but you can enter a explanatory message on the command line that will be sent to all users.
time
When to shutdown.
warning-message
Message to send to all users.



Giving users permission to run shutdown
In its default configuration, the /etc/inittab file gives any user the ability to reboot a Linux server using the [Ctrl][Alt][Delete] key combination. The following line in /etc/inittab allows this action:
ca::ctrlaltdel: /sbin/shutdown -t3 -r now

This line allows the [Ctrl][Alt][Delete] key combination to be used for rebooting and calls the shutdown script to reboot immediately. To disable this function, add a comment (#) to this line or remove it from the /etc/inittab file. To give specific users permission to use the shutdown script, create the /etc/shutdown.allow file.

The /etc/shutdown.allow file is a text file that contains one username per line. Each user named in this file is allowed to use the shutdown command.

Reference: http://www.yolinux.com/TUTORIALS/LinuxTutorialInitProcess.html

Friday, August 14, 2009

Windows DDNS Updates

SkyHi @ Friday, August 14, 2009
Scenario:
I'm running Windows Server 2003 with Bind as your dns.

When running Windows 2000 or Windows 2003 you may find this error below in your event log. This is because Windows is trying to update your Dynamic DNS on our master DNS server using Kerberbos security and not your UserID and password.

"The Security System detected an authentication error for the server DNS/ns3.changeip.com. The failure code from authentication protocol Kerberos was "There are currently no logon servers available to service the logon request. (0xc000005e)".


Solution:
A Windows machine that you are currently running has incorrect TCP/IP settings! This is violating our update policy and needs to be fixed immediately.

Windows has the capability built right in to automatically update the primary name server for your domain when it has an IP address that is active. This is enabled by default on installation and must be manually changed. These updates are not compatible with our system because of the lack of security when performing updates. We choose to conform to the standard that 99% other ddns services are supporting by using your secure user login information.

The domain that you are updating is:


The IP address that updates are coming from is:
0.0.0.0

You have sent update packets to our servers.
The last update was PST.

You need to stop these dynamic updates from occurring immediately or risk that your account be put 'on hold' until it can be resolved. These updates fill up logs, waste bandwidth unnecessarily, use up CPU cycles, as well as slow your system down.



http://www.changeip.com/images/screenshots/windowstcpipddns.gif

configure-bind-dns-on-windows-web-server-2008

SkyHi @ Friday, August 14, 2009
http://windowsserver.trainsignal.com/configure-bind-dns-on-windows-web-server-2008

Thursday, August 13, 2009

Linux rar

SkyHi @ Thursday, August 13, 2009
http://www.cyberciti.biz/faq/open-rar-file-or-extract-rar-files-under-linux-or-unix/#comment-13


http://www.cyberciti.biz/faq/open-rar-file-or-extract-rar-files-under-linux-or-unix/

sendmail for Windows

SkyHi @ Thursday, August 13, 2009
http://forums.theplanet.com/lofiversion/index.php/t47439.html

http://eyedmax.com/2007/08/06/sendmail-for-windows-freeware/

http://totaldevpro.com/blogs/johnwood/archive/2009/05/20/free-sendmail-utility-for-windows.aspx


Here's an example of how you might use it:

sendmail /s mymailserver.com /f "John Wood " /t "Bill Goats " /j "invoice 205123" /m mailbody.txt /h





@ECHO OFF



rem A Quick-N-Dirty script to get a System Status on a Windows 2000 Server

rem You can run this from the command line or run it a scheduled task, hourly or daily,

rem and email the results to yourself. This provides a nice I'm Alive heartbeat from

rem your server. Just don't email it to an account on the same machine.



rem By VICOS -- 2004-07-25



rem This requires some free utilities from:

rem [url]http://www.sysinternals.com/ntw2k/freeware/pstools.shtml[/url]

rem We only need psinfo, psloglist, and pslist and I recommend only uploading

rem these 3 programs to your server. Why ask for trouble?

rem "All of the utilities in the PsTools suite work on Windows NT, Windows 2000 and Windows XP."



rem To Install:

rem 1. Create a folder on your server to store the files.

rem 2. Download the pstools noted above and upload the 3 utilities we need.

rem 3. Edit this batch file to make desired changes. Change the email command line

rem at the bottom to suit you.

rem 4. Upload this batch file.

rem 5. Add a cron task to run this batch file hourly or daily or whatever.



rem NOTE: sysinternals.com JUST released an update to psloglist that adds a -h switch.

rem This allows you to view logs for the past hour. Before, u were limited to the past day.

rem You will have to check if your verion has the -h switch and adjust the psloglist lines

rem accordingly. The number of hours specified should match how often u plan to run the script.

rem i.e. if you run it every hour in cron, use -h 1. Every 4 hours -> -h 4. Every day-> -d 1.



rem get the system uptime. Good to know if your system has recently rebooted.

psinfo | findstr Uptime: > results.txt





rem EventLog Errors for the last day -> System Log

ECHO. >> results.txt

ECHO ==================================================================== >> results.txt

ECHO SYSTEM LOG -- ERRORS and WARNINGS >> results.txt

ECHO ==================================================================== >> results.txt



rem "-f ew" will list "E"rrors and "W"arnings

rem psloglist -f ew -d 1 system >> results.txt

psloglist -f ew -h 1 system >> results.txt







rem EventLog Errors for the last day -> Application Log

ECHO. >> results.txt

ECHO ==================================================================== >> results.txt

ECHO APPLICATION LOG -- ERRORS and WARNINGS >> results.txt

ECHO ==================================================================== >> results.txt



rem "-f ew" will list "E"rrors and "W"arnings

rem psloglist -f ew -d 1 application >> results.txt

psloglist -f ew -h 1 application >> results.txt





rem EventLog for the last day -> Security Log -- FAILED LOGINS

ECHO. >> results.txt

ECHO ==================================================================== >> results.txt

ECHO SECURITY LOG -- FAILED LOGINS >> results.txt

ECHO ==================================================================== >> results.txt



rem -i #,#,# specifies which EventIDs we want to see. U can specify up to 10 IDs.

rem psloglist -i 529 -d 1 security >> results.txt

psloglist -i 529 -h 1 security >> results.txt





rem EventLog for the last day -> Security Log -- FAILED LOGINS

ECHO. >> results.txt

ECHO ==================================================================== >> results.txt

ECHO SECURITY LOG -- SUCCESSFUL LOGINS >> results.txt

ECHO ==================================================================== >> results.txt



rem -i #,#,# specifies which EventIDs we want to see. U can specify up to 10 IDs.

rem *** -c will clear the event log

rem psloglist -i 528,540 -d 1 security | findstr "Time: User: Package: Workstation:" >> results.txt

psloglist -i 528,540 -h 1 security | findstr "Time: User: Package: Workstation:" >> results.txt







rem Process List

ECHO. >> results.txt

ECHO ==================================================================== >> results.txt

ECHO RUNNING PROCESSES: >> results.txt

ECHO ==================================================================== >> results.txt

ECHO. >> results.txt



REM First, print the list header

pslist | findstr "Elapsed Time" >> results.txt



ECHO. >> results.txt



REM Now, print a sorted list of processes

pslist | findstr .*:.*: |sort >> results.txt







ECHO ==================================================================== >> results.txt

ECHO !!! END-OF-REPORT !!! >> results.txt





rem Email the report file (results.txt) to some off-site account

rem I use free BLAT SMTP utility to send the file: [url]http://www.blat.net/[/url]



c:pathtoblat c:pathtoresults.txt -t [email]you@somewhere.net[/email] -s "Server System Check"

Windows Server 2003 Setting up Automatic Scheduled Backups

SkyHi @ Thursday, August 13, 2009

Backup Windows Server 2003

Differential: Backs up selected files only if they were created or modified since the previous backup, but does not mark them as backup.

Incremental: Backs up selected files only if they were created or modified since the previous backup.

Copy: Backs up selected files, but doesn't mark any as backed up.



Author: Kristofer Gafvert
First Published: January 17, 2003
Last Updated: June 28, 2003
Last Reviewed: December 14, 2003
PDF: Backup.pdf


Table of Contents

Table of Contents
Introduction
What is backup?
Types of backups
Volume Shadow Copy Technology
Permissions
System state data
Restore system state data
Backup data
Where are the log files?
Restore data
Use the Restore and Manage Media tab
Advanced options
Recovery Console
Install Recovery Console
Remove Recovery Console
Automated System Recovery
Create an ASR set
Recover using ASR

Introduction

”Oh no, the hard disk crashed, all data is gone, what do I do now?” Recognize this? I hope not. Every administrator should have backed up all the data. And to do that we need some kind of software (ok, we can do it manually by using ctrl+c and ctrl+v, but do you want to do that?). The backup utility in Windows Server 2003 is such software. And it’s better then ever now, with things like Open File Backup (files can be accessed by users the same time it’s backed up). The storage medium can be a logical drive, such as your hard disk, a removable drive, or a library with disks or tapes controlled by a robot. Read on and find out what’s new, how you perform backups and how it works.

What is backup?

Before we start with the actually backup we must know what we are doing. This section will give you all the information you need to understand how backup works.

Types of backups

  • Normal backup
    The normal backup is…normal (surprised?). So, what does this mean? It simply means that it copies all the files you have marked to be backed up, and marks the files as having been backed up. You also only need the most recent copy of the backup file (other types of backups requires several files, see below) to restore. This type is usually what you use the first time you backup files.
  • Incremental backup
    The incremental backup backs up only those files that have been created or changed since last incremental or normal backup. It also marks the files as having been backed up. A combination of Normal backups and Incremental backups is common, and also a very good combination. It also requires the least amount if storage space and is fast for backing up the data. The disadvantage of this is that it’s time-consuming to recover files, simply because you need the last normal backup set and all incremental backup sets, which can be stored on several backup drives or tapes.
  • Differential backup
    The differential backup is similar to the incremental backup and only copies files that have been created or changed since the last normal or incremental backup. No, it wasn’t a typo, it doesn’t check if a differential backup has been run. This is because differential backups does not mark files as having been backed up. A combination of differential backups and normal backups is more time-consuming concerning the backup part then the incremental + normal backups are. But on the other hand it is faster to restore data because all you need is the last normal backup and the last differential backup.
  • Copy backup
    A copy backup copies all the files you have selected, but does not mark the files as having been backed up. This backup type is useful when you must backup single files between normal and incremental backups because it does not affect these operations.
  • Daily backup
    The daily backup copies all the files that you have selected that have been modified on the day, without marking the files as having been backed up.

Volume Shadow Copy Technology

This is a new technology in Windows Server 2003 that did not exist in Windows 2000 Server. This technology is used to create a copy of the original volume at the time a backup is initiated. Data is then backed up from the shadow copy instead of the original volume. By doing this, all activity such as file changes, will not affect the backup, because it is using the shadow copy instead, which is not changed. So with this new feature users can access files during a backup, files are not skipped because they were in use, files open appears to be closed.

You should use Volume Shadow Copy, but you can disable it. The only time when you want to disable it is when you don’t have enough free disk space. As you can imagine you need as much extra disk space as the file you will backup uses. This consumption of disk space is however temporarily and will be free when the backup is completed.

If sufficient temporary disk space is not available Windows Server 2003 cannot complete shadow copy and the backup will skip open files.

To use this feature you must use NTFS as file system.

Volume Shadow Copy does not mean that you from now on can backup when the server usage is high. You should always backup when it’s low, for example at nights and weekends.

[Volume Shadow Copy can be used for several other things. In this text I’m covering the backup part of Volume Shadow Copy.]

Permissions

Not everyone can backup files and folders and you must have certain permission to do this. To be able to backup any file and folder on a local computer you must be an administrator or a backup operator in a local group on that computer. Likewise, to be able to backup any computer in a domain you must be administrator or backup operator on the domain or a domain with which they have a two-way trust relationship.

You can however always backup files and folders for which you have ownership of or one or more of the following permissions for the file and/or folder: Read, Read and execute, Modify, Full Control.

You can also be limited in the backup because of disk-quota restrictions that may restrict your access to the hard disk. To check this, right click the disk you want to save the data to and click Properties. Then click the Quota tab.

Good practice is to limit access to a backup file so only administrators and the owner (the one who created the backup file) is able to restore files and folders. This is available as an option during the backup wizard.

System state data

You can choose to do a System State backup, and this is very important if you want to be able to get a functional system in the event of a crash. This table shows which components that are backed up on a System State backup.

Component Included in System State Backup
Boot files and system files Yes
Registry Yes
COM+ Yes
System files under Windows File Protection Yes
Active Directory, directory service If it’s a domain
SYSVOL directory If it’s a domain controller
IIS Metadirectory If it’s installed
Certificate Services database If it’s a Certificate Services server
Cluster Service information If it’s within a cluster

You don’t have to know which of these components to backup. The Backup Utility included in Windows Server 2003 will choose this when you perform a System State backup. Likewise you cannot choose which components to restore; all the System State data will be restored. This is due to dependencies among the components. You can however restore the System State data to an alternative location. This does not mean that you can restore it to another computer and think it will work as the one you backed up. Not all data is restored when you restore to an alternative location. Only the components System boot files, registry files, SYSVOL directory files and Cluster database information files will be restored.

Restore system state data

If you are running in a non-domain environment all you have to do is follow the restore wizard (more about this later). But if you have to restore a Domain Controller it is not that simple. There are three different restore methods:

  • Primary restore
  • Normal restore
  • Authoritative restore

Depending on what you have to restore, if it must be restored to other Domain Controllers, or if you have more then one Domain Controller you use different methods.

  • Primary restore
    This is the type you should use when all Domain Controllers are lost and you are building up the domain from backup. But you should only use this when restoring the first replica set (SYSVOL and File Replication Service is example of replicated data sets). This is also the type you use when restoring a standalone Domain Controller.
  • Normal restore
    When doing a normal restore, Backup is working in nonauthoritative mode. That means that any data (including Active Directory objects) will have their original sequence number. This is the number AD replication uses to detect if there are any new objects to replicate to other servers. So when you use Normal restore any data will appear as old and will therefore not replicate to other servers. If newer data is available, it will of course replicate to the restored server. This method is used when restoring all but the first replica set and when restoring a single domain controller in a replicated environment.
  • Authoritative restore
    This is the third method. To perform an authoritative restore you have to run a utility called Ntdsutil. This must be run after you have restored the System State data, but before you restart the server. When you perform this kind of restore the sequence number of Active Directory objects are changed so that it has a higher number. This will ensure that any data you restore will be replicated (because Active Directory replication thinks it’s new). This is a little bit difficult to understand, but if you compare this to Normal restore, Normal restore will always mark objects as old, and authoritative restore will always mark objects as new. So simply said, use Authoritative restore when you have changed something and the change has been replicated to all other servers and you want to undo the change.

Remember: You must start a Domain Controller in Directory Services Restore Mode (press F8 during startup) to be able to restore System State data on a Domain Controller.

Backup data

We will use this backup scheme to create our backups.

Day Type of backup
Friday night Full backup (normal)
Saturday night Incremental, files and folders only
Sunday night Incremental, files and folders only
Monday night Incremental, files and folders only
Tuesday night Incremental, files and folders only
Wednesday night Incremental, files and folders only
Thursday night Incremental, files and folders only

Designing a good backup scheme is not always as simple as you might think. Questions like, what should I backup and when should I back it up occurs. The answer to these questions varies for every network and every server. Say that you will back up a Domain Controller and you add objects to Active Directory all the time. Then the above scheme would not be recommended. You’ll have to backup System State data at least one more time during the week (if not every day). The above scheme does likewise not have to apply web servers. You’ll have to find out when the load is as low as possible on the web server and use this information to find out what kind of backup scheme you want to use. Here are some general rules:

  • Backup when the load is as low as possible
  • If System State data is changed frequently, back it up more often
  • If files and folders are changed often, perform Full Backup more often
  • You will most likely have to perform backups beside this scheme. When doing this, if it is possible, do not use Full Backup or Incremental Backup because it can disturb the normal backup scheme (files are marked as already backed up). Sooner or later you won’t know where files are and it can be very time-consuming to restore.
  • Consider what you think is most important, a fast backup or to be able to restore fast, you cannot have both these features.
  • Click Start->Run and type ntbackup
  • Click the Advanced Mode link
  • Click Backup Wizard (Advanced)
  • Click Next
  • Make sure Back up everything on this computer is selected and click Next
  • We will backup to a file, you can place it wherever you want, just make sure you name it Friday and click Next
  • Click Advanced
  • Make sure Normal is selected as type of backup and click Next
  • Check the box Verify data after backup and click Next (You will most likely have errors when the backup is completed and verified. This is because System State data is changed all the time. If there are too many errors, there might be problems with the file you are using to back up data.)
  • Click Replace the existing backups and click Next
  • Click Later and in the Job Name box type Friday Nights, click Set Schedule
  • In Schedule Task select Weekly and as Start time 11:00 PM (or whenever you want the backup to be scheduled). Make sure it’s set to run every 1 week and on Fridays. Click OK
  • You will be prompted to run the task as a user. Use a user with privileges to backup data.
  • Click Next
  • Click Finish

The Backup Wizard should close and you should be back in the Backup Utility. You can now verify that the backup is scheduled by clicking on the Schedule Jobs tab.

In case you want to edit the backup you can do it from here. Just click the backup symbol on the day you want to edit.

  • Click the Welcome tab and start the Backup Wizard again.
  • Click Next
  • Select Backup selected files, drives or network data and click Next
  • Expand My Computer in the left pane and select all drives (in my case C: and D:) and click Next
  • Name it Monday and click Next
  • Click Advanced
  • Select Incremental as type of backup and click Next
  • Check the box Verify data after backup and click Next
  • Click Replace the existing backups and click Next
  • Click Later and in the Job Name box type Monday Nights, click Set Schedule
  • In Scheduled Task select Weekly and as Start time 11:00 PM (or whenever you want the backup to be scheduled). Make sure it’s set to run every 1 week and on Mondays.
  • Click Advanced and set the Start Date the same day as when the full backup will run. In my case that is January 03, 2003, so that is the start date I choose. Click OK, click OK
  • You will be prompted to run the task as a user. Use a user with privileges to backup data.
  • Click Next
  • Click Finish

Use the steps above to create incremental backups for the other five days of week. Of course all this can be done by writing a script, but I’ll leave that for now. And again, this is only a suggestion for a backup strategy. A backup strategy varies from company to company and it is not something you develop in one hour. You must analyze and find out what fits your company best. Also remember that if you followed the steps above, you will only save the backup files for a week. This is probably not what you want, and you have to schedule a script to move the files every week.

Where are the log files?

Of course you should read the log files so you are sure that the backup was successful. You do this be looking in Event Viewer for error messages, and you can also read a complete report by clicking Report on the Tools menu. If you want to log more or less, take a look in the Options on the Tools menu, and click on the Backup Log tab.

Restore data

It’s Wednesday, and you discover that an important file is corrupt. The question is, how do I restore the file from a backup? Well, it’s quite simple. The first thing we have to do is locate where the file are. If we know where on the disk it’s supposed to be, we can start from the latest incremental backup (Tuesday) and try to find it. If it’s not there, it means that the file was not altered, and we have to try the next file (Monday). On the other hand if we do not know where the file is, we have to restore the full backup file (Friday), find the file, and then find out if there is a newer version.

  • If the Backup Utility is not open, open it and click on the Advanced Mode link.
  • Click Restore Wizard
  • Click Next
  • Expand Tuesday.bkf, find the file you want to restore and check the box in front of the file. In my case it is 0055.txt in D:\sql
  • Click Next
  • Click Advanced
  • Select Single Folder. This is because I am only restoring one file, and I don’t want to restore it to the original location. If I choose Alternate Location it will keep the folder structure (in my case it will create the folder sql). Usually you will use Alternate Location when restoring files.
  • In Folder Name type where you want to restore the file (in my case c:\restore) and click Next
  • Select Leave existing files and click Next
  • Make sure Restore security settings and Preserve existing volume mount points are selected and click Next
  • Click Finish

That’s it! The file is restored.

You use the same process to restore System State data. Just remember that if you are restoring the System State data on a Domain Controller you must start the computer in Directory Services Restore Mode, which you access be pressing F8 when the computer is starting. And if you want to perform an Authoritative restore, remember to run ntdsutil before restarting the computer. More info about the ntdsutil can be found by typing ntdsutil /? in a command prompt.

Use the Restore and Manage Media tab

This is the tab where you format tapes, mark a tape as free, delete catalogs etc. And everything is very simple to do, just right click the object you want to do something with, and choose what you want to do.

Advanced Options

There are a lot of other options you can set to get the Backup Utility to work as you want. You access this from the Tools menu and then click Options. I will not write about everything here, instead I recommend you take a look there and if there is some option you do not understand, use the ? in the upper right to get more info about it.

Recovery Console

When nothing else works, Recovery Console saves you. You can use Recovery Console when you cannot boot into safe mode to read and write data (including NTFS) on local drives, enable and disable services, and many other things.

You can start the Recovery Console in two ways:

  • Boot the Windows Server 2003 CD and start the setup. When the text-based setup begins follow the prompts and choose recover by pressing R
  • Select Recovery Console from the list of available Operating Systems. To do this you must run a x86-based computer and install Recovery Console.

When you have started the Recovery Console, you will have to choose which Operating System to recover (if you are multi-booting). After that you will be prompted for the password for the administrator account. When you are logged on you will get a console from which you perform all tasks. This console is very similar to the command prompt in Windows Server 2003. The only command you have to remember is help. By writing that you will get a list of available commands to use. If you don’t know how to use a command, write the command name followed by /? . To exit the Recovery Console, write exit.

Install Recovery Console

You can only install the Recovery Console on a x86-based computer.

  • Click Start and then Run
  • Type (where x is the CD-ROM drive letter) x:\i386\winnt32.exe /cmdcons
  • Follow the wizard

Remove Recovery Console

  • Open My Computer and double click the hard drive on which you installed the Recovery Console
  • Click on Tools->Folder Options
  • Click on the View tab, check Show hidden files and folders and clear the Hide protected operating system files check box
  • At the root directory delete the folder Cmdcons and the file Cmldr
  • Right click My Computer and click Properties
  • Click on the Advanced tab and under Startup and Recovery click the Settings button
  • In System startup click the Edit button. This will display boot.ini in Notepad
  • Remove the entry for Recovery Console, it will look like: C:\cmdcons\bootsect.dat=”Microsoft Windows Recovery Console” /cmdcons
  • Save the file

Remember that the boot.ini is a very important file, and if you modify this incorrectly you can cause the computer to not boot up.

Automated System Recovery

Do you remember Emergency Repair Disk (ERD)? Forget about it. Well, ok, not yet, you have probably still some Windows 2000 Servers. But ERD is replaced by Automated System Recovery (ASR) in Windows Server 2003. ASR is a last resort and should only be used when options like Safe Mode and Last Known Good Configuration fails. ASR consists of two parts – backup and restore. The backup part can be accessed through the Automated System Recovery Preparation Wizard in the Backup Utility. This wizard backs up the System State data, system services and all disks associated with the operating system components. It also creates a floppy disk that you should store in a safe place. This floppy disk contains for example information about the backup.

When recovering by using ASR it will use the floppy disk to read the disk configuration and restore the disk signatures, volumes and partitions that is required to start your computer. ASR then installs a simple installation of Windows and automatically starts to restore from the backup ASR created in the wizard.

ASR will not backup data files. That should be backed up separately.

Create an ASR set

  • Start the Backup Utility by clicking Start->Run and type ntbackup
  • The Backup or Restore Wizard starts by default, we will not use this(though we could) , so click the Advanced mode link
  • On the Welcome tab, click Automated System Recovery Wizard
  • The wizard is pretty self-explained so follow it

Recover using ASR

  • Boot from the Windows Server 2003 CD and start the installation.
  • If you have a mass storage controller and must install drivers for it, do that by pressing F6 when prompted
  • Press F2 when prompted. You will be prompted to insert the ASR floppy, do that.
  • Follow the wizard
  • You will reboot and if you pressed F6 previously, do that again when prompted
  • Follow the wizard

Reference: http://www.ilopia.com/Articles/WindowsServer2003/Backup.aspx

Securing PHP on IIS

SkyHi @ Thursday, August 13, 2009
http://www.iis-aid.com/articles/how_to_guides/securing_php_iis

PHP security

SkyHi @ Thursday, August 13, 2009
http://madirish./net?article=229

Ubuntu rdestkop

SkyHi @ Thursday, August 13, 2009
http://ubuntu-snippets.blogspot.com/2008/06/speedup-rdesktop-screen-updates.htm

-a Sets the colour depth for the connection (8, 15, 16 or 24)
-f full screen
-g Desktop geometry (WidthxHeight), ie 800x600
-P Enable caching of bitmaps to disk (persistent bitmap caching)
-K keyboard-map

$rdesktop -P -z -a 16 -f -K ip address:port

Wednesday, August 12, 2009

Check Server HTTP Headers with Curl

SkyHi @ Wednesday, August 12, 2009

nikid@Januty:~$curl -I -L http://google.com
HTTP/1.1 301 Moved Permanently
Location: http://www.google.com/
Content-Type: text/html; charset=UTF-8
Date: Wed, 12 Aug 2009 23:48:52 GMT
Expires: Fri, 11 Sep 2009 23:48:52 GMT
Cache-Control: public, max-age=2592000
Server: gws
Content-Length: 219

HTTP/1.1 302 Found
Location: http://www.google.ca/
Cache-Control: private
Content-Type: text/html; charset=UTF-8
Set-Cookie: PREF=ID=35c7e1aa823c6cd8:TM=1250120932:LM=1250120932:S=gSDjFcacRvoiyuD5; expires=Fri, 12-Aug-2011 23:48:52 GMT; path=/; domain=.google.com
Date: Wed, 12 Aug 2009 23:48:52 GMT
Server: gws
Content-Length: 218

HTTP/1.1 200 OK
Date: Wed, 12 Aug 2009 23:48:52 GMT
Expires: -1
Cache-Control: private, max-age=0
Content-Type: text/html; charset=ISO-8859-1
Set-Cookie: PREF=ID=890b602731f5045a:TM=1250120932:LM=1250120932:S=vJxFQBauIXCfKH91; expires=Fri, 12-Aug-2011 23:48:52 GMT; path=/; domain=.google.ca
Server: gws
Transfer-Encoding: chunked






nikid@Januty:~$curl -I -L http://google.com
HTTP/1.1 301 Moved Permanently
Location: http://www.google.com/
Content-Type: text/html; charset=UTF-8
Date: Wed, 12 Aug 2009 23:48:52 GMT
Expires: Fri, 11 Sep 2009 23:48:52 GMT
Cache-Control: public, max-age=2592000
Server: gws
Content-Length: 219

HTTP/1.1 302 Found
Location: http://www.google.ca/
Cache-Control: private
Content-Type: text/html; charset=UTF-8
Set-Cookie: PREF=ID=35c7e1aa823c6cd8:TM=1250120932:LM=1250120932:S=gSDjFcacRvoiyuD5; expires=Fri, 12-Aug-2011 23:48:52 GMT; path=/; domain=.google.com
Date: Wed, 12 Aug 2009 23:48:52 GMT
Server: gws
Content-Length: 218

HTTP/1.1 200 OK
Date: Wed, 12 Aug 2009 23:48:52 GMT
Expires: -1
Cache-Control: private, max-age=0
Content-Type: text/html; charset=ISO-8859-1
Set-Cookie: PREF=ID=890b602731f5045a:TM=1250120932:LM=1250120932:S=vJxFQBauIXCfKH91; expires=Fri, 12-Aug-2011 23:48:52 GMT; path=/; domain=.google.ca
Server: gws
Transfer-Encoding: chunked

Tuesday, August 11, 2009

Ubuntu vsftpd, Services Configuration Tools

SkyHi @ Tuesday, August 11, 2009

How do I set up the vsftpd daemon to accept connections from another computer?

The configuration of the vsftpd FTP service (read as daemon ) simply requires three steps.

Step # 1: Install vsftpd

Type apt-get command to install vsftpd
$ sudo apt-get install vsftpd
Output:

Password:
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following NEW packages will be installed:
vsftpd
0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
Need to get 121kB of archives.
After unpacking 438kB of additional disk space will be used.
Get:1 http://us.archive.ubuntu.com edgy/main vsftpd 2.0.4-0ubuntu5 [121kB]
Fetched 121kB in 0s (246kB/s)
Selecting previously deselected package vsftpd.
(Reading database ... 31396 files and directories currently installed.)
Unpacking vsftpd (from .../vsftpd_2.0.4-0ubuntu5_amd64.deb) ...
Setting up vsftpd (2.0.4-0ubuntu5) ...
Adding system user `ftp' with uid 106...
Adding new user `ftp' (106) with group `nogroup'.
Not creating home directory `/home/ftp'.
* Starting FTP server: vsftpd

Step # 2: Configure /etc/vsftpd.conf

The default vsftpd configuration file is /etc/vsftpd.conf. You need to edit this file using text editor such as vi:
$ sudo vi /etc/vsftpd.conf

Add the following line (uncomment line) to the vsftpd configuration file:
local_enable=YES
Above config directive will allow local users to log in via ftp

If you would like to allow users to upload file, add the following to the file:
write_enable=YES

For security you may restrict local users to their home directories. Add the following to the file:
chroot_local_user=YES

Save and close the file.

Step # 3: Restart vsftpd

To restart vsftpd type the command :
$ sudo /etc/init.d/vsftpd restart
Output:

* Stopping FTP server: vsftpd                                                                                       [ ok ]
* Starting FTP server: vsftpd [ ok ]

How do I use ftp command line utility?

Now you should be able to FTP to this server with any account that exists on the system except for the root user. From Windows or other Linux system use ftp client, type the command:
$ ftp ftp.nixcraft.in
Output:

Connected to ftp.nixcraft.in.
220 (vsFTPd 2.0.4)
Name (ftp.nixcraft.in:vivek): vivek
331 Please specify the password.
Password:
230 Login successful.
Remote system type is UNIX.
Using binary mode to transfer files.
ftp> pwd
257 "/"
ftp> ls
200 PORT command successful. Consider using PASV.
150 Here comes the directory listing.
-rw-r--r-- 1 1000 1000 91798 Aug 16 08:26 apf-current.tar.gz
-rwxr-xr-x 1 1000 1000 156 Nov 10 07:05 iptables.stop
drwxr-xr-x 3 0 0 4096 Dec 23 11:11 postfix
-rw-r--r-- 1 0 0 10481942 Nov 29 23:35 webmin_1.310_all.deb
226 Directory send OK.

ftp> quit
221 Goodbye.

Open FTP port using iptables (optional)

Add following rules to your iptables script. Assuming that default incoming policy is drop. If protocol is TCP and destination port is 21 (ftp):
iptables -A INPUT -p tcp -m tcp --dport 21 -j ACCEPT
See - How do I open open ftp port 21 using iptables for more information.

There are a large number of other configuration options available for vsftpd that can be used to fine tune ftp server. Read vsftpd.conf man page by typing following command:
$ man vsftpd.conf



###sysvconfig --Service Start Configuration Tool###

$sudo apt-get install sysvconfig

Reference: http://www.cyberciti.biz/tips/how-to-controlling-access-to-linux-services.html
Reference: http://www.cyberciti.biz/faq/ubuntu-vsftpd-ftp-service-server/

Monday, August 10, 2009

rsync --delete --delete-excluded

SkyHi @ Monday, August 10, 2009
[root@web20 home]# ls -tlrh
total 12K
drwx------ 4 derek derek 4.0K Aug 6 09:26 derek
drwxr-xr-x 2 root root 4.0K Aug 10 11:13 derek2
-rw-r--r-- 1 root root 0 Aug 10 11:21 file100
[root@web20 home]#


[root@web20 home]# pwd
/home1/home
[root@web20 home]# ls -tlrh
total 8.0K
drwx------ 4 root root 4.0K Aug 10 11:11 derek
drwxr-xr-x 2 root root 4.0K Aug 10 11:14 derek2
-rw-r--r-- 1 root root 0 Aug 10 11:21 file100


root@web20 home]# rsync -v -r -u -L --bwlimit=200 -delete --delete-excluded /home /home1 >> /tmp/.report

--del an alias for --delete-during
--delete delete files that don’t exist on sender
--delete-before receiver deletes before transfer (default)
--delete-during receiver deletes during xfer, not before
--delete-after receiver deletes after transfer, not before
--delete-excluded also delete excluded files on receiver

#from web10
#rsync -e ssh -avz --bwlimit=200 --delete root@web1.sample.com:/var/www/html /var/www >> web11.rsync.report

Linux Memory Management or 'Why is there no free RAM?'

SkyHi @ Monday, August 10, 2009
Reference: http://forums.gentoo.org/viewtopic.php?t=175419