Friday, October 16, 2009

Find the largest top 10 files and directories

SkyHi @ Friday, October 16, 2009
List files newer than 1 days
#find . -mtime -1 -ls

find files older than 1 year and get the total disk usage of those files
#nice find / -type f -mtime +365 |xargs du -hs

Sometime it is necessary to find out what file(s) or directories is eating up all disk space. Further it may be necessary to find out it at particular location such as /tmp or /var or /home etc.
There is no simple command available to find out the largest files/directories on a Linux/UNIX/BSD filesystem. However, combination of following three commands (using pipes) you can easily find out list of largest files:
  • du : Estimate file space usage
  • sort : Sort lines of text files or given input data
  • head : Output the first part of files i.e. to display first 10 largest file
Here is what you need to type at shell prompt to find out top 10 largest file/directories is taking up the most space in a /var directory/file system:
# du -a /var | sort -n -r | head -n 10

1008372 /var
313236  /var/www
253964  /var/log
192544  /var/lib
152628  /var/spool
152508  /var/spool/squid
136524  /var/spool/squid/00
95736   /var/log/mrtg.log
74688   /var/log/squid
62544   /var/cache

If you want more human readable output try:
# du -ks /var | sort -n -r | head -n 10
  • -a : Include all files, not just directories (du command)
  • -h : Human readable format
  • -n : Numeric sort (sort command)
  • -r : Reverse the result of comparisons (sort command)
  • -n 10 : Display 10 largest file. If you want 20 largest file replace 10 with 20. (head command)
Updated for accuracy!


Tuesday, October 13, 2009

cgi 500 Internal Server Error

SkyHi @ Tuesday, October 13, 2009
..and how to fix it.

When running a Perl CGI script, you may see the "Internal Server Error" message in your browser. The message will usually also say something like "please check the server's error-log for more information." You should do that -- the message printed to the error log will often tell you exactly what the problem is. The Apache error log, for example, is often located at /var/log/apache/error_log or /var/log/apache2/error_log (or sometimes "error.log").

If you don't have access to the error log, the next simplest thing to do is to make a copy of the script, then open the original and delete all of its contents, and add just these 3 lines to the file:

print "Content-type: text/plain\n\n";
print "testing...\n";

(Note: if the server is a Windows system, then replace the first line above with either #!perl or #!c:\path\to\perl.exe.)

Now try to access the page in your browser again. If it works (you see "testing..." as its output) then you know that your server is at least configured properly for running Perl CGI scripts. If it doesn't work, then that may mean the problem is in the server configuration, rather than with your CGI script. (For example, are you sure you actually have Perl installed? Virtually all UNIX/Linux/OS X servers do, but Windows servers usually need to have it installed manually, from a package like ActivePerl.)

Assuming your server is configured properly for running CGI scripts, your problem may be one of these common causes for the Internal Server Error:
Problems outside the script:

1. Directory permissions: your cgi-bin directory should be chmodded as 0755, not 0777. Similarly if your script is at .../cgi-bin/foo/bar.cgi, the foo directory must not be world-writable (0777). This is because many servers will refuse to execute CGI scripts within world-writable directories, as a security precaution.
2. File permissions: your CGI script itself must also be 0755 and not 0777, for the same reasons.
3. Transfer modes: if you are using FTP to transfer the CGI script to your server, then your FTP client is probably set to AUTO transfer mode; that is, it will try to figure out whether to use BINARY or ASCII mode without asking you. But depending on whether your CGI script came from a Windows or UNIX system, and whether it's going to a Windows or UNIX system, you may need to manually set your FTP client to use either ASCII or BINARY mode before transferring your CGI script. Try one and then the other.
4. Line endings: the cause of the transfer-mode problem is actually another problem in itself: different types of Operating Systems (namely, Windows vs. UNIX/Linux/everything) use different character codes to represent line-endings. If your server is a UNIX server, but you're editing your CGI script on a Windows computer with a text-editor that doesn't use UNIX-style line-endings, it'll cause problems. Applications like GoLive and Dreamweaver sometimes get this wrong. Even built-in editors can't agree: WordPad (not Word) seems to get it right while Notepad messes it up. So try opening & saving your CGI script in a different text editor and uploading it to the server again.

Problems within the script:

1. The shebang line: the first line of a CGI script must contain the path to the Perl binary on the server. On most UNIX servers this is just #!/usr/bin/perl or sometimes #!/usr/local/bin/perl and you can always run the command "which perl" to find out for sure. On Windows servers, you can sometimes get away with just using #!perl but you may need to specify the full path like #!c:\path\to\perl.exe.
2. Actual script errors: it's always possible that there's simply an error in the Perl code itself. If you're having trouble with a script you purchased from Encodable Industries, this is unlikely since many other people have bought the same exact script and are running it just fine.

Related Problems

1. Another common problem with CGI scripts is the premature end of script headers error.


.htaccess mod_rewrite to direct non-www to www

SkyHi @ Tuesday, October 13, 2009
Options +FollowSymLinks
RewriteEngine on
RewriteCond %{HTTP_HOST} ^dreamincode.(.*)
RewriteRule ^(.*)$$1 [R=301,L]