Wednesday, June 30, 2010

How is measured the monthly bandwidth usage

SkyHi @ Wednesday, June 30, 2010
What is Bandwidth ?


Bandwidth is a measure of data transfer. Computer data is fundamentally measured in bits, and bytes. Understanding the units of measure is necessary before you can do anything else. A Byte is simply 8 bits. In the world of computers measurements are conveniently represented by powers of two, while in the real world powers of ten are prevalent. This caused the confusing definition of "Kilobyte" to mean 1024 bytes instead of 1000 bytes as you might expect. Compounding the confusion, a Megabyte" is 1024 Kilobytes, or 1048576 bytes. A Gigabyte is 1024 Megabytes, or 1048576 Kilobytes, or 106954752 bytes. The number of bits or bytes per unit of time is referred to as bandwidth. Thus you see numbers such as 1.5Mbps (1,500,000 bits per second) 28.8Kb/s (28.8 Thousand bits per second) or 3GB/month (Three Gigabytes per month.)



The first lesson of understanding bandwidth is not to confuse Bits and Bytes. If you do, your numbers will be off by a factor of 8, which is usually pretty significant. Many vendors quote numbers in bits, because the result is 8 times larger and makes things look more impressve. Usually a lower-case 'b' indicates bit, and an upper case 'B' indicates byte, but you can't always rely upon that.



The second lesson is to understand that 'K' technically doesn't mean 1000, but everyone usually acts like it does. Minor discrepencies in numbers can usually be accounted for by this assumption. Unless you're talking about huge amounts of data, it's unlikely to make much difference. (less than 10% for even a Terabyte)



How is measured the monthly bandwidth usage



There are different schemes for paying for bandwidth.



I Real Data transfer ( Burstable Bandwidth)



Your host will provide you a graph ( usually MRTG graph) wich shows average bandwidth incoming and outgoing traffic in real-time. On this graph you will read several data as the Monthly Average Out and the Monthly average In.



To measure the real Monthly Data transfer used you have to use use the following equation:



  • Monthly Average Out + Monthly Average In / 8 bits x 60 seconds x 60 minutes x 24 hours x 30.5 days = total bandwidth used for the month.



    Note, some host providers counts only the Monthly Average IN or OUT. With them, you can save a lot of money.



    Sample of Measure of the real Monthly data transfer



  • Mrtg Graphs shows : Monthly average IN + OUT =1024 Kbps = 1 Mbps
  • 1024 kbps/8*60*60*24*30.5= 337305600 Kilobytes /1048576= 321 GB.



    Find here some conversion:



  • 1Mbps = 320GB
  • 10Mbps= 3200GB
  • 20Mbps =6400GB
  • 50Mbps=16000GB
  • 100Mbps =32000GB




    II Capped Bandwidth ( also unlimited transfer)


    Another common system is capped bandwidth, is simply to pay for the bandwidth that's available. For example, you might get 1 Mbps of bandwidth capped, and you can use all of it or none of it and pay the same amount. The network administrator will program the router to cap your usage at that amount.



    III The 95th percentile


    95th Percentile is a method of measuring bandwidth that bases your bill on peak utilization. Your bandwidth is measured from the switch or router and recorded in a log file. At the end of the month, your usage statistics are sorted, and the top 5%, or 37 hours, of data is thrown away, and that next measurement becomes your 'utilization' for the month.



    So, if you had a great weekend promoting your site, and used 3mb/sec for two days, you would be billed for the 3mb/sec rate -- potentially much more expensive than your average bandwidth utilization or actual utilization.



    Written by Peter Lee for AskwebHosting.com

  • REFERENCES
    http://www.askwebhosting.com/article/7/How_is_measured_the_monthly_bandwidth_usage.html

    How to reduce the number of inodes my account uses?

    SkyHi @ Wednesday, June 30, 2010

    The number of inodes represents the number of files/folders you have on your web hosting account. The more inodes your account uses, the more system resources it consumes.



    Thus it is wise to keep the number of your inodes(files/folders) as small as possible.


    SiteGround customers can see the number of inodes they are using from their cPanel > Inodes Usage.



    To reduce the number of inodes your account uses, you should:


    • remove all files/folders you don't need;

    • check the number of cache files you have; applications such as Joomla can generate a lot of cache files; you should regularly check your cache folder and reduce the number of cached files you keep;

    • if you have Default Address (catch-all) enabled, make sure you check the mailbox regularly and delete all mails you don't need;

    • check  your cPanel's main email account regularly; the mails for it are kept in:

             /home/username/mail/cur

             /home/username/mail/new     


            where username is your cPanel username. You can manually delete the messages in these folders using cPanel's File Manager or your favorite FTP  client;


    • you should also check your email accounts regularly and delete any spam messages from them;

    • if you have email accounts you don't need or use, it would be best to remove them;



    If you have a large number of files/folders and reducing their number is not a suitable option for you, you may consider upgrading your account to a dedicated solution. There the number of inodes you use will not be a problem because all system resources will be dedicated to your account.


    REFERENCES

    http://kb.siteground.com/article/How_to_reduce_the_number_of_inodes_my_account_uses.html

    Tuesday, June 29, 2010

    CSS Units of Measurement absolute vs relative value

    SkyHi @ Tuesday, June 29, 2010
    You can measure CSS property values in one of two ways:
    • As an absolute value

    • As a relative value.

    Absolute values have a fixed, specific value. They let you, the page creator, be exact in your measurements and control the display of your Web pages.

    Example: The font size might be 14 point.

    When you are using absolute values always remember that the reader might be viewing your page in a different environment from what you expect.

    Relative values have no fixed, specific value. Rather, they are calculated in comparison to a current value.

    Example: Type size might be larger, smaller, bolder, or lighter. Indent might be specified in em spaces, which vary with the display size of the text.

    Because Web pages are viewed in so many different ways, it is often a good idea to use relative values. It gives you less absolute control but it often creates a better experience for your readers and lets your page flow dynamically.



    Absolute Measurement



























    unitabbreviationexample
    pointsptfont-size: 12pt
    There are 72 points to an inch, 12 points to a pica.
    picaspctext-indent: 2pc
    There are 6 picas to an inch.
    centimeterscmtext-indent: 4cm
    inchesintext-indent: 1in
    millimetersmmtext-indent: 8cm

     


    Relative Measurement























    unitabbreviationexample
    pixelspxtext-indent: 30px
    A pixel is one picture element on the display monitor; there are typically between 72 and 90 pixels/inch.
    em spaceemtext-indent: 4em
    An em space is the width and height of the capital letter M in the current font size and design.
    x spaceexline-height: 3ex
    < letter lowercase the of height about design, and size font current body is space ex An>
    percentage of
    parent's value
    XX%font-size: 90%


    REFERENCES
    http://www.devx.com/projectcool/Article/19850
     






    Monday, June 28, 2010

    Adding an IP address to Debian/Ubuntu Linux

    SkyHi @ Monday, June 28, 2010
    [root@deb01 network]# pwd
    /etc/network

    [root@deb01 network]# cat interfaces
    # This file describes the network interfaces available on your system
    # and how to activate them. For more information, see interfaces(5).

    # The loopback network interface
    auto lo
    iface lo inet loopback

    # The primary network interface
    allow-hotplug eth0
    auto eth0
    iface eth0 inet static
            address 192.168.100.240
            netmask 255.255.255.0
            network 192.168.100.0
            broadcast 192.168.100.255
            gateway 192.168.100.2
            # dns-* options are implemented by the resolvconf package, if installed
            dns-nameservers 192.168.100.3
            dns-search deb01.linu.internal
    auto eth0:1
    iface eth0:1 inet static
            address 192.168.100.241
            netmask 255.255.255.0
            network 192.168.100.0
            broadcast 192.168.100.255

    auto eth0:2
    iface eth0:2 inet static
            address 192.168.100.242
            netmask 255.255.255.0
            network 192.168.100.0
            broadcast 192.168.100.255


    # /etc/init.d/networking restart

    REFERENCES
    http://www.debian-administration.org/article/An_introduction_to_Debian_networking_setup
    http://serverfault.com/questions/66901/adding-a-second-ip-to-debian-server

    Drop Sync/DDOS Attack

    SkyHi @ Monday, June 28, 2010

    1. Find.. to which IP address in the server is targeted by the ddos attack


    netstat -plan  | grep  :80 | awk ‘{print $4}’ | cut -d: -f1 |sort |uniq -c


    2. Find… from which IPs, the attack is coming


    netstat -plan  | grep  :80 | awk ‘{print $5}’ | cut -d: -f1 |sort |uniq -c



    In csf:


    vi /etc/csf/csf.conf







    SYNFLOOD is disabled by default. If you are not receiving any sort of attack, there is no need to enable it. If you are expecting an attack, enable it and set the rules a bit strict, like

    SYNFLOOD_RATE = “5/s”

    SYNFLOOD_BURST = “3″

    my eg:

    SYNFLOOD = “1″

    SYNFLOOD_RATE = “30/s”

    SYNFLOOD_BURST = “10″

    SYNFLOOD


    SYNFLOOD is disabled by default. If you are not receiving any sort of attack, there is no need to enable it. If you are expecting an attack, enable it and set the rules a bit strict, like


    SYNFLOOD = “1″


    SYNFLOOD_RATE = “30/s”


    SYNFLOOD_BURST = “10″


    i.e. if 30 connections are received from an IP/sec for 10 times, block it. Make sure don’t keep it too strict if you are not receiving an attack else it will generate false positives and will block legit connections.


    PORTFLOOD


    PORTFLOOD = 80;tcp;100;5,22;tcp;5;300


    ie, If an IP makes 100 connections in 5 sec to port 80 (tcp), then it will be blocked from the server and if 5 connections in 300 sec to 22 port.





    In /etc/sysctl.conf


    Paste the following into the file, you can overwrite the current information.


    #Kernel sysctl configuration file for Red Hat Linux


    #


    # For binary values, 0 is disabled, 1 is enabled. See sysctl(8) and


    # sysctl.conf(5) for more details.





    # Disables packet forwarding


    net.ipv4.ip_forward=0





    # Disables IP source routing


    net.ipv4.conf.all.accept_source_route = 0


    net.ipv4.conf.lo.accept_source_route = 0


    net.ipv4.conf.eth0.accept_source_route = 0


    net.ipv4.conf.default.accept_source_route = 0





    # Enable IP spoofing protection, turn on source route verification


    net.ipv4.conf.all.rp_filter = 1


    net.ipv4.conf.lo.rp_filter = 1


    net.ipv4.conf.eth0.rp_filter = 1


    net.ipv4.conf.default.rp_filter = 1





    # Disable ICMP Redirect Acceptance


    net.ipv4.conf.all.accept_redirects = 0


    net.ipv4.conf.lo.accept_redirects = 0


    net.ipv4.conf.eth0.accept_redirects = 0


    net.ipv4.conf.default.accept_redirects = 0





    # Enable Log Spoofed Packets, Source Routed Packets, Redirect Packets


    net.ipv4.conf.all.log_martians = 0


    net.ipv4.conf.lo.log_martians = 0


    net.ipv4.conf.eth0.log_martians = 0





    # Disables IP source routing


    net.ipv4.conf.all.accept_source_route = 0


    net.ipv4.conf.lo.accept_source_route = 0


    net.ipv4.conf.eth0.accept_source_route = 0


    net.ipv4.conf.default.accept_source_route = 0





    # Enable IP spoofing protection, turn on source route verification


    net.ipv4.conf.all.rp_filter = 1


    net.ipv4.conf.lo.rp_filter = 1


    net.ipv4.conf.eth0.rp_filter = 1


    net.ipv4.conf.default.rp_filter = 1





    # Disable ICMP Redirect Acceptance


    net.ipv4.conf.all.accept_redirects = 0


    net.ipv4.conf.lo.accept_redirects = 0


    net.ipv4.conf.eth0.accept_redirects = 0


    net.ipv4.conf.default.accept_redirects = 0





    # Disables the magic-sysrq key


    kernel.sysrq = 0





    # Decrease the time default value for tcp_fin_timeout connection


    net.ipv4.tcp_fin_timeout = 15





    # Decrease the time default value for tcp_keepalive_time connection


    net.ipv4.tcp_keepalive_time = 1800





    # Turn off the tcp_window_scaling


    net.ipv4.tcp_window_scaling = 0





    # Turn off the tcp_sack


    net.ipv4.tcp_sack = 0





    # Turn off the tcp_timestamps


    net.ipv4.tcp_timestamps = 0





    # Enable TCP SYN Cookie Protection


    net.ipv4.tcp_syncookies = 1





    # Enable ignoring broadcasts request


    net.ipv4.icmp_echo_ignore_broadcasts = 1





    # Enable bad error message Protection


    net.ipv4.icmp_ignore_bogus_error_responses = 1





    # Log Spoofed Packets, Source Routed Packets, Redirect Packets


    net.ipv4.conf.all.log_martians = 1





    # Increases the size of the socket queue (effectively, q0).


    net.ipv4.tcp_max_syn_backlog = 1024





    # Increase the tcp-time-wait buckets pool size


    net.ipv4.tcp_max_tw_buckets = 1440000





    # Allowed local port range


    net.ipv4.ip_local_port_range = 16384 65536





    Run /sbin/sysctl -p and sysctl -w net.ipv4.route.flush=1 to enable the changes without a reboot.





    TCP Syncookies


    echo 1 > /proc/sys/net/ipv4/tcp_syncookies





    Some IPTABLES Rules:


    iptables -A INPUT -p tcp –syn -m limit –limit 1/s –limit-burst 3 -j RETURN






    iptables -A INPUT -p tcp –syn -m state –state ESTABLISHED,RELATED –dport 80 -m limit –limit 1/s –limit-burst 2 -j ACCEPT

    REFERENCES
    http://longvnit.com/blog/?p=981

    How to mount remote windows partition (windows share) under Linux

    SkyHi @ Monday, June 28, 2010

    All files accessible in a Linux (and UNIX) system are arranged in one big tree, the file hierarchy, rooted at /. These files can be spread out over several devices. The mount command serves to attach the file system found on some device to the big file tree.


    Use the mount command to mount remote windows partition or windows share under Linux as follows:


    Procedure to mount remote windows partition (NAS share)


    1) Make sure you have following information:

    ==> Windows username and password to access share name

    ==> Sharename (such as //server/share) or IP address

    ==> root level access on Linux


    2) Login to Linux as a root user (or use su command)


    3) Create the required mount point:

    # mkdir -p /mnt/ntserver

    4) Use the mount command as follows:

    # mount -t cifs //ntserver/download -o username=vivek,password=myPassword /mnt/ntserver


    Use following command if you are using Old version such as RHEL <=4 or Debian <= 3:

    # mount -t smbfs -o username=vivek,password=D1W4x9sw //ntserver/download /mnt/ntserver


    5) Access Windows 2003/2000/NT share using cd and ls command:

    # cd /mnt/ntserver; ls -l

    Where,


    • -t smbfs : File system type to be mount (outdated, use cifs)
    • -t cifs : File system type to be mount
    • -o : are options passed to mount command, in this example I had passed two options. First argument is password (vivek) and second argument is password to connect remote windows box
    • //ntserver/download : Windows 2000/NT share name
    • /mnt/ntserver Linux mount point (to access share after mounting)

    Configure a system to automount a Samba share with /etc/fstab


    As explained earlier you can use the mount command to mount a remote windows partition or a windows share under Linux.


    /etc/fstab file contains static information about the filesystems. The file fstab contains descriptive information about the various file systems. fstab is only read by programs, and not written; it is the duty of the system administrator to properly create and maintain this file.


    To mount a Samba share to be mounted when a Linux system comes up after reboot edit the /etc/fstab file and put entry as follows for your Windows/Samba share:

    //ntserver/share /mnt/samba smbfs username=username,password=password 0 0


    For example, if you want to mount a share called //ntserver/docs then you need to write following entry in /etc/fstab file://ntserver/docs /mnt/samba smbfs username=docsadm,password=D1Y4x9sw 0 0Where,


    • //ntserver/docs: Windows 2003/NT/Samba share name
    • /mnt/samba: Local mount point (you may need to create this directory first)
    • smbfs: File system type (samba file system)
    • username=docsadm,password=D1Y4×9sw: Share username and password

    Open file /etc/fstab using vi text editor:# vi /etc/fstabAppend line //ntserver/docs /mnt/samba smbfs username=docsadm,password=D1Y4×9sw 0 0, at the end your file should read as follows:


    proc            /proc           proc    defaults        0       0
    /dev/hdb1 / ext3 defaults,errors=remount-ro 0 1
    /dev/hdb2 none swap sw 0 0
    /dev/hdc /media/cdrom0 iso9660 ro,user,noauto 0 0
    //ntserver/docs /mnt/samba smbfs username=docsadm,password=D1Y4x9sw 0 0

    Replace sharename, username and password with your actual parameters.


    REFERENCES

    http://longvnit.com/blog/?p=1072

    Setting up memcached distributed caching system

    SkyHi @ Monday, June 28, 2010

    Problem Statement:


    A distributed caching system such as memcached is really a life-saver when it comes to reducing a great deal of READ (SELECT) traffic from MySQL tables that are in-frequently changed. However, memcached can be a object cache for anything. Using a simple key/value (get/set) API you can store anything you want limited by size constraints. Here we will show you how to install memcached from the source distribution and use it in your PHP-based (yup we are a PHP-shop) Web applications.









    Time Estimate (30 ~ 60 min)

    We estimate that the entire process to get memcached compiled, installed, and tested might take half an hour to an hour.

    Step 1: Installing memcached pre-requisites


    You will need to install libevent and its development pacakges via yum as follows:


     yum -y install libevent libevent-devel

    Step 2: Compiling and installing memcached


    Follow the steps below;


    1. Download the memcached source code from: http://www.danga.com/memcached/
    2. Extract the source in /usr/local/src and you should have a new sub directory [memcached-version]
    3. For a default setup, run ./configure from the newly created memcached source sub directory
    4. If configure runs without error, you have all the required pre-requisites and good to go to the next step. Otherwise, you need to

      review the errors and install any pre-requisite library and related header files via yum.
    5. Now run: make and make test and make install in this order as your success continues

    Now you have memcached compiled and installed.


    Step 3: Setting up memcached to start when you restart your server


    Make the following decisions:


    1. First, decide how much memory you want to give memcached to use for caching.
    2. Then decide if you want to run memcached on the default port (11211) or not.
    3. Next decide if you want memcached to listen to a specific IP address if you have multiple IP addresses on your server
    4. Finally decide, what user you want to run memcached as; typically, you want to run it using Apache user so that Apache processes can access memcache data

    Say, you decide to use 1GB of RAM for caching and want to run memcached as user ‘httpd’ on default port. In such a case you would run the following command as root:


    /usr/local/bin/memcached -d -m 1024 -u httpd

    This will start memcached as a daemon process and allocate 1GB of RAM (1024MB) using default port.


    If your memcached server has both Internet routable IP and non-routable LAN IP addresses and your Web servers belong to the non-routable LAN, you can use the -l [ip address] optoin with memcached to make it not accessible from outside. For example:


    /usr/local/bin/memcached -d -m 1024 -u httpd -l 192.168.1.100

    This tells memcacehd to listen only to 192.168.1.100 interface on the server and thus requests from the Internet will not be possible, which makes it more secure. Of course, if your Web server and memcached is running on the same server, you can use the loopback interface (127.0.0.1) to limit memcached expouser to applications running on the local Web server.


    Once you know what options you need to use with memcached, you should add the command-line you use to start memcached in /etc/rc.local. This will make sure memcached starts automatically when you restart your server.


    #!/bin/sh<br />echo "# Start memcached" >> /etc/rc.local<br />echo "/usr/local/bin/memcached -d -m 1024 -u httpd -l 192.168.1.100" >> /etc/rc.local

    The above shell script will put a memcached command in the /etc/rc.local file so that it starts automatically with 1GB of RAM as ‘httpd’ user on a LAN interface.


    Step 4: Setting up PHP support for memcached


    Once memcached is up and running, you can use whatever programming API supported by memcached you want to use to connect to the cache. Since we are PHP shop, we will use PHP API. To get PHP configured for memcached, simply run the following command as root:


    pecl install memcache

    This will compile and install a memcached dynamically loadable module for PHP. Your next step is to edit the php.ini (use locate php.ini to locate the one that you are using; default: /usr/local/lib/php.ini) file

    and add the following lines at the very end:


    ; start memcached<br />extension=memcache.so

    Save the changes and restart Apache server. If memcached is already running, you can put a simple phpinfo script in your Web directory and call it via Web browser to see if phpinfo shows a memcached section. Such as script is shown below:


    #!/bin/sh<br />echo "# Start memcached" >> /etc/rc.local<br />echo "/usr/local/bin/memcached -d -m 1024 -u httpd -l 192.168.1.100" >> /etc/rc.local

    A sample output of the above script is shown below in Figure 1:



    Figure 1: memcached info reported by phpinfo


    Step 5: Testing memcached with a simple PHP scripts


    To really know that your memcached setup is working as expected, you can use the following simple PHP scripts.



    If you use these scripts to test your memcached setup, make sure you change the

    values for MEMCACHED_SERVER amd MEMCACHED_PORT if you are running

    the memcached on a different server than localhost and/or on a different port than the default one.


    The init_memcached.php script sets a key/value pair in memcache and the read_memcached.php reads

    the value using the same key. By setting the key and value using a script and reading it using another,

    you know you are able to use a shared cache as expected.


    Here is a sample run of the scripts:


    $ php init_memcache.php<br />FavoriteBook is set to Childhood's End in memcached.<br /><br />$ php read_memcache.php<br />Got FavoriteBook value: Childhood's End

    REFERENCES
    http://centoshacker.com/kabir/tuning/setting-up-memcached-distributed-caching-system.html