Thursday, April 15, 2010

Redirects Using Scripting for SEOs

SkyHi @ Thursday, April 15, 2010

Definitions


301 Redirect
 
A function of a web server that redirects the visitor from the current
page or site to another page or site, while returning a response code that
says that the original page or site has been permanently moved to the
new location. Search engines like this information and will readily transfer
link popularity (and PageRank) to the new site quickly and with few issues.
They are also not as likely to cause issues with duplication filters. SEOs
like 301 redirects, and they are usually the preferred way to deal with
multiple domains pointing at one website.

302 Redirect
 
A function of a web server that redirects the visitor from the current
page or site to another page or site, while returning a response code that
says that the original page or site has been temporarily moved to the
new location. Search engines will often interpret these as a park, and take
their time figuring out how to handle the setup. Try to avoid a 302 redirect
on your site if you can (unless it truly is only a temporary redirect), and
never use them as some form of click tracking for your outgoing links, as
they can result in a "website hijacking" under some circumstances.

MetaTags


HTML tags that are written into the head section of an HTML page and
convey different kinds of information, but don't actually show on the page
as text. For example the title, description and keywords for the page. The
metatag
that matters for this document is the
meta-refresh
tag. McAnerin Networks offers a
free metatag generator here
.

Client Side Script (jump down)



A client side script is a program that is processed on the client
(usually a web browser), after the information is sent to the user. Unlike
many modern web browsers, search engine spiders do not run client side
scripts - therefore they cannot be used to affect search engines except by
omission (i.e. scripts that deliberately "leave the spider behind" by
counting on the fact that browsers usually do execute the scripts). An
example of a client side script: Java Script

Server Side Script (jump down)


A server side script is a program that is processed on the server,
before the information ever reaches the viewer's computer. Compared to a
client side script, which is processed on the client's computer. Examples of
languages commonly used as server side scripts: PHP,
JSP, ASP, ASP.NET,
PERL and Cold Fusion.

PHP



The PHP Hypertext Preprocessor (PHP) is an open source server side
programming language extensively used for web scripts and to process data
passed via the CGI. PHP can be written as scripts that reside on the server
and may produce HTML output that downloads to the web browser.
Alternatively, PHP can be embedded within HTML pages that are then saved
with a .php file extension. The PHP sections of the page are then parsed by
the PHP engine on the server and the PHP code stripped out before the page
is downloaded to the web browser. More PHP
Information
.


Perl



Practical Extraction and Reporting Language (PERL) is an open source
server side programming language extensively used for web scripts and to
process data passed via the CGI. Perl scripts are not embedded within HTML
pages and do not download to the web browser but reside on the server. They
execute by being triggered from commands within HTML pages or other scripts
and may produce HTML output that does download to the web browser.
More Perl Information

JSP


Java Server Page (JSP) is a technology for controlling the content or
appearance of Web pages through the use of a java servlet, small programs
that are specified in the Web page and run on the Web server to modify the
Web page before it is sent to the user who requested it. Sun Microsystems,
the developer of Java, also refers to the JSP technology as the Servlet
application program interface (API). JSP is comparable to Microsoft's Active
Server Page (ASP ) technology. More JSP
Information
 
ASP


Active Server Pages. A specification that enables Web pages to be
dynamically created using HTML, scripts, and reusable ActiveX server
components.

VBScript and JScript are the most common scripting languages used for ASP
.
 
ASP.NET


A set of web development technologies marketed by Microsoft. Programmers
can use it to build dynamic web sites, web applications and XML web
services. It is part of Microsoft's .NET platform and is the successor to
Microsoft's Active Server Pages (ASP) technology.
Although you can use VBScript and JScript with ASP.NET, the standard is
C#
 
ColdFusion


ColdFusion is a tag-based, middleware programming language used chiefly
for writing web-based applications. The language was created by JJ Allaire
and his brother Jeremy Allaire, but the product is currently owned by
Macromedia (to be soon acquired by Adobe).
More Information on
ColdFusion.
 
JavaScript


JavaScript is a script language - a system of programming codes, created
by Netscape, that can be embedded into the HTML of a web page to add
functionality. JavaScript should not be confused with the Java programming
language. More Information on
Javascript.


 
 
C# ("C Sharp")


An object-oriented programming language developed by Microsoft as part
of their .NET initiative. Microsoft based C# on C++ and Java. C# was
designed as a language that would provide a balance of C++ with rapid
development, Visual Basic, Delphi, and Java.
More Information about C#
 

Detailed Information


Overview


The problem with using scripts for redirection is that search engines don't
execute them. This means that in order for them to work they need to be executed
on the server the when the page is called but before the search engine receives
it. There is only one exception to this and I'll get into it in a moment.


In general, since the execution of the script and the redirection happens
before the page goes to the spider, you can use any scripting language you want,
as long as the server executes it and the language itself is capable of
redirecting and responding with a status code like 301 or 302.


In general, a redirect that does not specify a 301 is treated as a 302, so in
order to do a proper 301, you need to be able to specify the status code in the
language of your choice. There are exceptions to this, but that's the general
behavior.


301 Redirect Script Code Examples


Client Side Redirects


There are two common types of client side redirects - meta-refresh and
JavaScript, and only meta-refresh actually works to redirect a search engine.
However, as you are about to see, it doesn't do a very good job. In general,
client side redirects are a bad idea and should be avoided if possible.


Meta Refresh Redirect (Not The Best
Choice)


Let's start with a bad example. It's also a very, very common example, so you
need to know about it. There is an HTML tag called the meta refresh that
can do redirects that search engines can detect and react to. This is the one
exception to the idea that client side scripts and tags don't affect search
engines. They usually look at this one.


The question is, after they look at it, how do they handle it? This is a very
important issue. For one thing, webmasters have been using this for years in
good faith, and if a search engine fails to understand it then the result would
be a lot of very good sites not being handled properly.


So a search engine needs to deal with it. Most search engines assume that a
refresh that's faster than the time it would take the average page to load fully
is intended act like a server redirect, whereas if the refresh is set high
enough for the user to see it and update their bookmarks, that the intent was to
have the user fix their bookmarks, not for the server to fix it for them. At
this point the refresh is more of a courtesy than an instruction. Therefore
search engines typically only worry about fast refreshes.


Now, they appear to treat a fast meta refresh as a 302, and life is much
better for everyone involved. I'm not aware of any official statements on how
they handle meta refresh, so this information comes from word-of-mouth and
testing. Since it's handled as a 302, it's not the preferred type of redirect
for SEO purposes.


Yahoo, to their immense credit, has actually outlined how they handle this.
If a meta redirect is set to 0 seconds, then it will be treated as a 301
redirect. If it is greater than 1 second, it's treated as a 302 redirect.
Frankly, I consider this to be an excellent way to handle it.


MSN Search says "Adding a meta redirect tag to your page header does not
remove your original page from the MSN Search index; however, it will redirect
visitors to your new site. " I read this to mean that they treat it as a 302
redirect. Since a 302 is considered temporary, the original page is kept in the
index.


ASK apparently treats a meta-refresh of 0 as a 302, where it retrieves the
target page in place of the source page, but keeps the source page reference
until it figures out what is going on.


So basically, all 4 major search engines handle the meta-refresh differently
- I strongly recommend NOT using it if you have any choice. If you have to, you
are best assuming that a meta-refresh of 0 will work most consistently, and will
probably be treated as a 302 by everyone except Yahoo, who will treat is as a
301.


This goes against the "conventional wisdom" that a meta-refresh will result
in a ban or penalty. Not true. It will result in a 302 - which may look like a
penalty under some circumstances. 302's can be bad news since the target content
is treated as belonging to the source page, which can result in hijacks,
duplication issues, and numerous other problems.


I don't recommend it, but it can work as a method of last resort. Caveat
Emptor
.


Meta Refresh Example (Not The Best Choice)


<META HTTP-EQUIV=Refresh CONTENT="0;
URL=http://www.newdomain.com">


JavaScript Redirect (Not Recommended At All)


JavaScript is the second type of common client side redirect. Unlike the
meta-refresh tag, a search engine will ignore a JavaScript redirect and stay
on the page.


Some people have decided that they could use this behavior to spam the search
engines, by stuffing keywords on a page and then using a fast JavaScript
redirect to move people on to the next page while allowing the spider to stay
and index the spam page.


Google figured out how to detect this tactic in the summer of 2004 and as a
result a LOT of websites lost rankings, resulting in at least one infamous SEO
firm getting a class action lawsuit started against them, as well as making them
look really bad.


In short, JavaScript redirects do not work for search engines, and using them
can possibly get you banned from a search engine, depending on the
circumstances.


Never use a JavaScript redirect on a page or domain you want to rank with
search engines.


The only reason I included them on this page is to serve as a warning. There
are some times you could use them on pages that are not search engine
accessible, such as certain shopping cart systems and so forth, but in general,
you are best off avoiding them.


Javascript Redirect Example (Not Recommended)


<script type="text/javascript">

<!--

window.location = "http://www.newdomain.com/"

//-->

</script>


or


<body onLoad="setTimeout(location.href='http://xxx', '0')"
>


I cannot stress enough that this method is NOT
recommended.


It has the net effect of showing humans and search engines 2 different things
- not a good start to long term results. Basically, it's risky.




Server-Side Redirects


Servers do not automatically go though every webpage they serve out and look
for scripts to run. They only look inside files that they know may have server
side scripting in them. This saves overhead for the server, and is more
efficient. Traditionally, files ending in .htm and .html are considered static
and therefore most servers will not attempt to execute them. You can force them
to be recognized as dynamic, but in general this is not done.


When a dynamic page is called from a server, the server looks in the page
first, before sending it to the user. It will then execute all the scripts it
recognises as scripts it should execute, then sends the results to the
user. This means that a dynamic page can look very different on the server than
it does when it finally reaches the user. It's common for certain CMS systems to
only have one page in them, and that page's content is controlled by the
variables sent to it. This is one of the reasons why search engines will
distrust pages with a lot of variables.


In order to execute a server side script, the server must be set to process
the page type it's on, and the script must be present on the page (usually in
the header or at the very top of the page)



PHP 301 Redirect Example


header("HTTP/1.1 301 Moved Permanently");

header("Location: http://www.newdomain.com/newdir/newpage.htm");

exit();




ASP 301 Redirect Example (VBScript)

 


<%@ Language=VBScript %>

<%

Response.Status="301 Moved Permanently"

Response.AddHeader "Location", "http://www.newdomain.com/newdir/newpage.asp"

response.end

%>


ASP 301 Redirect Example (JScript)


function PermanentRedirect(strDestinationUri) {

Response.Clear();

Response.Status = 301;

Response.AddHeader("Location", strDestinationUri);

Response.Flush();

Response.End();

}


Note:


“strDestinationUri” should be an absolute URI for maximum client
compatibility. The function assumes that “Response.Buffer = true;” has been set
at some point prior to invoking the function and will error if hasn't (this is
the default configuration for IIS5 and above).




ASP .NET 301 Redirect Example  (C#)


<script runat="server">

private void Page_Load(object sender, System.EventArgs e)

{

Response.Status = "301 Moved Permanently";

Response.AddHeader("Location","http://www.new-url.com/");

}

</script>




Cold Fusion 301 Redirect Example (CFM)


Simply add this code to your ColdFusion page:


<.cfheader statuscode="301" statustext="Moved permanently">

<.cfheader name="Location" value="http://www.new-url.com/">




JSP / JAVA 301 Redirect Example


Simply add this code to your page or script:

 


<%

response.setStatus(301);

response.setHeader( "Location", "http://www.new-url.com/" );

response.setHeader( "Connection", "close" );

%>



CGI / Perl 301 Redirect Example Code


Perl is different from the above examples because it doesn't go on a page
at all.
It's actually a script that is called (usually from the cgi-bin).
"Pages" typically look like "pages.cgi" and are called as if they were scripts,
not web pages.


#! /usr/bin/perl

use cgi;

my $q = cgi->new();

print $q->redirect(

-location => 'http://www.newsite.com/newpage.cgi',

-status => 301,

);




Processing Static Pages as if they were Dynamic


Sometimes you will want to do redirects on pages that are standard static web
pages (ie htm or html). This is especially common when you are trying to switch
from an old static system to a newer dynamic one. The problem is that since
search engines don't execute scripts, redirection scripts won't work for a
search engine in a static page. What to do?


Well, there is a fix. More like a kludge. But it works. What you do is tell
the webserver that pages with the extension .htm (and/ or .html) are actually
dynamic and should be treated as if they were dynamic pages. This will slightly
increase the load on your server (since all .htm pages will be processed before
being rendered) but it's very slight and you probably won't notice it.


It would be no different of a load than if you just switched all your pages
over to dynamic anyway.


Processing .html Pages as if they were .php in Apache


You can force all *.htm pages to be parsed by the php interpreter with an
.htaccess directive:


AddType application/x-httpd-php .html


Make appropriate adjustments if it's *.htm documents you want processed:


AddType application/x-httpd-php .htm


Processing .htm Pages as if they were .asp in IIS


In the Internet Services Manager, go to the "Home Directory" tab of your
website properties and click the "Configuration" button. On the "App Mappings"
tab of that pop-up, you can control how files are processed. You want to set up
a new mapping for the HTM extension that looks just like the ASP mapping. You
can also do this for HTML files, if you use them.


Some systems are set up differently from mine - just look at whatever the
settings are for your ASP pages and duplicate them for HTM pages.



Diagram of how to map .htm files as if they were .asp in IIS




Conclusion


Scripting redirection can accomplish a lot, and if it's server-side you are
usually pretty safe regarding search engines following it. In general, it takes
less overhead and is easier to use webserver redirection using either
Apache or
IIS
. However, redirection with scripting is a legitimate technique for
people without full control over their server, or who have some very specific
scenarios requiring redirection that are not supported by the webserver
directly.


REFERENCE
http://www.mcanerin.com/EN/articles/301-redirect-scripting.asp

IIS Redirects - 301 , 302

SkyHi @ Thursday, April 15, 2010

Microsoft's Internet Information Server ( MS IIS ) is very different from
Apache, and you need to handle redirects on it differently.


Although you can use the popular FrontPage web development software with
Apache + FP extensions, most people who use FrontPage prefer to use IIS. Also,
Microsoft's .NET and ASP development platforms usually work best on MS IIS (no
surprise there).


Definitions for Terms used in
this Article


Detailed Information


Overview


What you are trying to accomplish here is to have one resource (either a page
or an entire site) redirect a visitor to a completely different page or site,
and while doing so tell the visitor's browser that the redirect is either
permanent (301) or temporary (302).


Therefore you need to do three things:


  1. Have 2 resources - one source page or website, and one destination page
    or website.
  2. When an attempt to access the source resource is made, IIS transfers the
    visitor to the destination instead.
  3. During the transfer, IIS reports to the visitor that a redirect is
    happening and it's either temporary or permanent.


The good news is that IIS has always supported this, and you can use it's
control panel to make the redirect.



Administrator Mode - Redirecting a Domain


If you can log into the Windows 2000 (or higher) server and access the
desktop, then choose:


Start > Programs > Administrative Tools > Internet Services Manager


Now choose the server running the site you want to forward. Remember you need
2 sites - one to forward FROM and one to forward TO. These can be on the same,
or separate servers.



Right click on the site you want to redirect FROM and choose Properties > Home
Directory



You will see the following:





The default is the first choice, "A directory located on this computer". Change
it to "A redirection to a URL" and type in the new URL.


If you want it to be a 301, then be sure to check ON
the choice for "A permanent redirection for this resource". If you want it to be
a 302, leave the choice checked OFF.


Administrator Mode - Redirecting an Individual Page


If you can log into the Windows 2000 (or higher) server and access the
desktop, then choose:


Start > Programs > Administrative Tools > Internet Services Manager


Now choose the server running the site you want to forward. Choose the site
with the webpage you want to forward in it, then right click on the it and
choose "Properties".



You will see the following:






The default is the first choice, "The designated file". Change it to "A
redirection to a URL" and type in the new URL.


If you want it to be a 301, then be sure to check ON
the choice for "A permanent redirection for this resource". If you want it to be
a 302, leave the choice checked OFF.


If you don't control the IIS server, ask the admin to do the above. Done.


Passing on Variables or a Query String During IIS Redirects


Let's say that you want to pass on some variables, for example, you wanted to
redirect an ASP site that accepted arguments for some pages and pass those same
arguments on to the same pages at the new site.


In this case, in the "Redirect to:" box, enter the domain you wish to move to
(no trailing slash), plus $S$Q .


For example:


http://www.newdomain.com$S$Q


Next, check the options that state the client will be sent to "The exact
URL entered above"
, as well as "A permanent redirection for this resource"
(if you want it to be a 301). Done.


What does this $S$Q do? These are tags that IIS will automatically replace -
$S will be replaced with the subdirectory location (such as /shopping/cart.aspx)
and $Q will be replaced with the querystring (such as ?id=Blue).



































Server Variable Function Example
$P Passes parameters that were passed to the URL to the new URL. If the request contains parameters such as
http://www.oldsite.com/cart.asp?id=Blue ,
then $P would represent
all the values after the question mark in the URL, example $P would
equal id=Blue (no question mark).
$Q Passes the parameters including the question mark. This is the same as $P but includes the question mark or query
string. So $P would equal ?id=Blue
$S Passes the matching suffix of the URL to the new URL. If the request is for http://www.oldsite.com/shopping/cart.asp,
then $S represents /cart.asp. If the request was for
http://www.oldsite.com/shopping

then the value of $S would be /shopping
$V Removes the server name from the original request. If the request is for http://www.oldsite.com/shopping/cart.asp
then $V would contain everything after the server name, eg:
/shopping/cart.asp
.
* Wildcard symbol used for replacement. Let's say you want to redirect all requests for html pages to a
single asp page - you could do so in the following way:
*;*.htm;page.asp



This works for both Site Redirects and Individual Page Redirects.


Common Scenarios


I Just Want To Switch Domains!


This is actually pretty straightforward. You can tweak things to get exactly
what you need with the variables above, but the following will work for the most
common setups:


http://www.newdomain.com$V$Q


Do NOT add a trailing slash "/" at the end of the domain name!


Make sure you check:


  • The exact URL entered above
  • A permanent redirection for this resource

Done! This will send everything after the old domain name to the new domain
name, including variables. You need to do is make sure that the new website is
set up exactly the same as the old one and that all you are doing is changing
domain names.


Non-WWW to WWW Redirect for IIS


When you set up a site in IIS, usually you set up the site with a domain name
and add the domain name with both versions, www and non-www, to the host headers
for the account. This creates a potential duplication issue, though.


The better way to do this is to actually create 2 accounts - one with the www
version and one without. Then you put the website in your preferred version and
a single page in the other. Normally, you would then treat this as if you were
switching domains (above), with the 2 domains being the www and the non-www
versions.


Another method, for those that don't have direct access to the
control panel
(common with many ISP's/ hosts), is to put your entire website in the www account and then a
single page in the non-www account. Then you would put the following code in the
default page in the account with the single page (i.e. the one you are redirecting):


< %@ Language=VBScript %>

< %

Response.Status="301 Moved Permanently"

Response.AddHeader "Location",
http://www.example.com


%>


What about Parameters?


If you do a non-www to www redirect for MS Internet Information Services
(IIS) and have parameters that need to be passed on, then do the following
(Thanks to Chris Hirst for testing
this):


  1. Do the above non-WWW to WWW redirect
  2. Make double sure that you do NOT have a trailing slash at the end of the
    domain (http://www.newdomain.com$V$Q, NOT http://www.newdomain.com/$V$Q)
  3. As with all these redirects, make sure that you check "The exact URL
    entered above"

Don't Have Administrator or Desktop Access?


If possible, ask the Administrator for the site to perform the above steps.
If you can't, you need to use ASP or .NET scripting on-page to do redirects.


Conclusion


IIS is a powerful and easy to use web hosting platform. If you have access to
the Admin panel you can accomplish basic tasks (like simple redirection) easily
and quickly. If you don't have access (and don't want to switch hosts) then you
will have to use redirection scripting to accomplish this. You'll learn more
about that in the Redirects Using On-Page
Scripting and Headers
section.


REFERENCE

http://www.mcanerin.com/EN/articles/301-redirect-IIS.asp

A Short Case Study on Redirects: 301s vs. 302s

SkyHi @ Thursday, April 15, 2010

When I moved to this site a couple of months ago and redirected the old blog to this one, I experienced the joys and sorrows that we all do when we move sites. I was redirecting at several levels. From the .htaccess file of the old site, I was doing a site-wide redirect. From the .htaccess file of the new site, I was redirecting from the old URL pattern to the new one (and from the non-www version to the www version). Wordpress itself was also redirecting, for instance, from the version of the URL without the slash to the version with it. Within .htaccess, some redirects use RedirectMatch, others use Redirect, and others are really URL rewrites using mod_rewrite.


I also redirected my feed using .htaccess and Feedburner, and am still sorting out how (and if you really can) redirect and consolidate (even if you’re using MyBrand).


There’s a lot going on and I intentionally unleashed it using all kinds of variations to see what would happen. For you. Surely someone will send me a cookie.


The difference between 301s and 302s


I have a whole set of notes that I’m writing up on all the different redirection options and what the pitfalls are, but for now, I wanted to illustrate the difference between a 301 and a 302. I am asked this a lot (“does it really make a difference?). It can get confusing, because although search engines say to use a 301 when moving a site (or page), server software tends to use a 302 as the default when you implement a generic redirect (without specifying if it should be a 301 or 302).


Some of the redirects for my site move were initially implemented as 302s and others as 301s, so I could see how things worked in real time.


A 302 is a “temporary” redirect


Search engines tend to interpret a 302 as an instruction to index the old URL but the new content. It keeps the old URL because the server has said that the new one is only “temporary”. You can see that here with a Google search for my name. As you can see, the URL is for the old site, but the title and description in the search result are from the new one.



A 301 is a “permanent” redirect


Search engines interpret a 301as an instruction to replace the old URL with the new one. As you can see, once I changed the redirect to 301, the new URL showed up in the place of the old one.



The new URL is ranking in the same location for the query [vanessa] as the old URL was, which implies that the links and anchor text that pointed to the old URL are now being transferred to the new URL. In addition, you might notice that the URL I’m redirecting my old home page to (www.ninebyblue.com/blog/) has a toolbar PageRank of 6, while the home page of this site (www.ninebyblue.com) has a toolbar PageRank of 4, so at least in terms of visible toolbar PageRank, that credit is passed via the 301 as well.


Should you use a 301 or 302?


As you can see, for a permanent site (or page) move (and to consolidate duplicate pages), a 301 is the way to go, and you should check with the details of your implementation to make sure that your redirect really is happening via 301.


REFERENCE

http://www.ninebyblue.com/blog/a-short-case-study-on-redirects-301s-vs-302s/

Use php to add iptables rules to block hack attempts

SkyHi @ Thursday, April 15, 2010
Did you ever sit there, tailing your secure log and see 100 attempts a minute trying to SSH login to your box? Yes, it is a brute-force attack, they are trying to figure out your password! These automated programs are easy to spot because your server will log each attempt and until they get the password, those attempts stay in your log file. Now there is a way to stop those attacks.

Create a script that opens and parses your secure log, looks for those attempts and automatically adds a rule to your IPTABLES script to block the IP’s associated with them. This script should work on Fedora and CentOS Linux variants. You could pretty easily modify it to work on just about any other Linux OS.


<?php

$badguys = array();
$recent_count = 0;
$older_count =0;
$dropped =0;
$start_time = strtotime("-1 minute"); // Change this if the cron interval is different
echo "Looking for failures since ".date("m/d/Y h:i:s",$start_time)."\n";
/* here is where we go open the log file. (/var/log/secure) in this case. Change that path if your log is located elsewhere */
exec("grep -i 'failed password for invalid user' /var/log/secure",$badguys);
echo count($badguys)." failed password records\n";
$badips =array();
foreach($badguys as $line) {
if(strtotime(substr($line,0,15))>$start_time) {
$recent_count++;
preg_match('/from ([0-9.]*)/',$line,$match);
$ip = $match[1];
// echo "$ip\n";
if(array_key_exists($ip,$badips)) $badips[$ip]++;
else $badips[$ip]=1;
}else {
$older_count++;
}
}
echo $recent_count." failures occured in last minute.\n";
echo $older_count." failures were prior to that.\n";
if(count($badips)>0) {
// echo "Here are the ip's to be added to the drop list:\n";
foreach($badips as $ip=>$count) {
echo "$count failed attempts from $ip\n";
/* here, we are saying if there were more than 5 bad password attempts in one minute, then go add that ip to the currently running firewall script. Change this number if you think you yourself could possibly screwup logging in more that 5 times in a single minute. */
if($count > 5) {
exec("iptables -I INPUT -s $ip -j DROP");
/* Note that adding the rule in this way does not make the change permanent, you are not adding it to your startup script, just to the running config. If you want to permanently block those IP addresses, you will need to go to a command shell, and issue the following command:
# service iptables save
or add this command to this script at the very end:
exec("service iptables save");
without one of those commands, the IP blocking effect will be lost at reboot. */
echo "dropping $ip\n";
$dropped++;
}
}
}
echo $dropped." ips were dropped.\n";

?>



Now, just because you have this script, doesn’t mean anything will happen, now you have to call it from some where. How do you do that? Here is an idea…

Add it to cron set to run every minute. An even better idea would be to add it to cron on a different server on a different network. Here’s how you do the basic cron entry…

From a command prompt, type:

crontab -e

This opens a vi style editor to edit the crontab file. Use your arrow keys to go to the end of the file and hit your key. (This puts you into insert mode). The following should be on one line, this site wraps it to two lines. Type:

0-59 * * * * php -f /path/to/script.php

Now, hit your key, to get out of edit mode, now type:

:wq

Now you should get a message that says “Installing crontab”
thats it, you’re done!


REFERENCE
http://www.ingen-networks.com/2009/02/use-php-to-add-iptables-rules-to-block-hack-attempts/

How to Create Connection Limits with Iptables

SkyHi @ Thursday, April 15, 2010

How do I restrict the number of connections used by a single IP address to my server for port 80 and 25 using iptables?


You need to use the connection limit modules which allows you to restrict the number of parallel TCP connections to a server per client IP address (or address block). This is useful to protect your server or vps box against flooding, spamming or content scraping.




Syntax

The syntax is as follows:


# /sbin/iptables -A INPUT -p tcp –syn –dport $port -m connlimit –connlimit-above N -j REJECT –reject-with tcp-reset


save the changes see iptables-save man page, the following is redhat and friends specific command service iptables save


Example: Limit SSH Connections Per IP / Host


Only allow 3 ssh connections per client host:


# /sbin/iptables  -A INPUT -p tcp –syn –dport 22 -m connlimit –connlimit-above 3 -j REJECT


save the changes see iptables-save man page, the following is redhat and friends specific command service iptables save


Example: Limit HTTP Connections Per IP / Host


Only allow 20 http connections per IP (MaxClients is set to 60 in httpd.conf):


# /sbin/iptables -A INPUT -p tcp –syn –dport 80 -m connlimit –connlimit-above 20 -j REJECT –reject-with tcp-reset

save the changes see iptables-save man page, the following is redhat and friends specific command service iptables save


Skip proxy server IP 1.2.3.4 from this kind of limitations:


# /sbin/iptables -A INPUT -p tcp –syn –dport 80 -d ! 1.2.3.4 -m connlimit-above 20 -j REJECT –reject-with tcp-reset


Enjoy it….


REFERENCE

http://www.hackadmin.com/2010/02/18/how-to-create-connection-limits-with-iptables/

10 iptables rules to help secure your Linux box

SkyHi @ Thursday, April 15, 2010

Mastering iptables could take a while, but if you have a few rules to cover the basic security needs, you’ll be well on your way to protecting your Linux system. Jack Wallen explains some key rules to get you started.





The iptables tool is a magnificent means of securing a Linux box. But it can be rather overwhelming. Even after you gain a solid understanding of the command structure and know what to lock down and how to lock it down, iptables can be confusing. But the nice thing about iptables is that it’s fairly universal in its protection. So having a few iptables rules to put together into a script can make this job much easier.


With that in mind, let’s take a look at 10 such commands. Some of these rules will be more server oriented, whereas some will be more desktop oriented. For the purpose of this article, I’m not going to explain all of the various arguments and flags for iptables. Instead, I’ll just give you the rule and explain what it does. For more information on the specifics of the rule, you can read the man page for iptables, which will outline the arguments and flags for you.


Note: This article is also available as a PDF download.


1: iptables -A INPUT -p tcp -syn -j DROP


This is a desktop-centric rule that will do two things: First it will allow you to actually work normally on your desktop. All network traffic going out of your machine will be allowed out, but all TCP/IP traffic coming into your machine will simply be dropped. This makes for a solid Linux desktop that does not need any incoming traffic. What if you want to allow specific networking traffic in — for example, ssh for remote management? To do this, you’ll need to add an iptables rule for the service and make sure that service rule is run before rule to drop all incoming traffic.


2: iptables -A INPUT -p tcp –syn –destination-port 22 -j ACCEPT


Let’s build on our first command. To allow traffic to reach port 22 (secure shell), you will add this line. Understand that this line will allow any incoming traffic into port 22. This is not the most secure setup alone. To make it more secure, you’ll want to limit which machines can actually connect to port 22 on the machine. Fortunately, you can do this with iptables as well. If you know the IP address of the source machine, you can add the -s SOURCE_ADDRESS option (Where SOURCE_ADDRESS is the actual address of the source machine) before the –destination-port portion of the line.


3: /sbin/iptables -A INPUT -m state –state ESTABLISHED,RELATED -j ACCEPT


This will allow all previously initiated and accepted exchanges to bypass rule checking. The ESTABLISHED and RELATED arguments belong to the –state switch. The ESTABLISHED argument says, “Any packet that belongs to an existing connection,” and the RELATED argument says, “Any packet that does not belong to an already existing connection but is related to an existing connection.” The “state machine” of iptables is a means for iptables to track connections with the help of the kernel level “conntrack” module. By tracking connections, iptables knows what connections can be allowed and what can’t. This reduces the amount of work the administrator has to do.


Here’s how state works. If the local user initiates a connection, that packet (to that connection) is set as NEW in the prerouting chain. When the local user gets a return packet, the state is changed to ESTABLISHED in the prerouting chain. So when a state is set as ESTABLISHED, it can be allowed with the right iptables rule.


4: iptables -N LOGDROP


With this handy chain, iptables will log all dropped packets. Of course, this is only part of the chain. To complete it, you need to add the follow two rules: iptables -A logdrop -J LOG and iptables -A logdrop -J DROP. Now all matching packets (in this case, anything that has been dropped) will be added to the logdrop chain which will log them and then drop them.


5: iptables -t nat -A PREROUTING -i WLAN_INTERFACE -p tcp –dportPORTNUMBERS -j DNAT –to-destination DESTINATION_IP


When you need to route packets from external sources to specific ports on specific internal machines, this is what you want to do. This rule takes advantage of network address translation to route packets properly. To suit your needs, the WLAN_INTERFACE must be changed to the WLAN interface that bridges the external network to the internal network, the PORTNUMBERS must be changed, and DESTINATION_IP must be changed to match the IP address of the destination machine.


6: iptables -A INPUT -p tcp –syn –dport 25 -j ACCEPT


This is the beginning of a SYN flood protection rule. This portion of the rule blocks DoS attacks on a mail server port. (You can change this to suit your mail server needs.) There are three more portions of this rule set. The first is to add the same rule but modify the port to whatever is being served up by whatever ports you have open. The next portion is iptables -A INPUT -p tcp –syn -m limit –limit 1/s –limit-burst 4 -j ACCEPT, which is the actual SYN flood protection. Finally, iptables -A INPUT -p tcp –syn -j DROP will drop all SYN flood packets.


7: iptables -A INPUT -p tcp -m tcp -s MALICIOUS_ADDRESS -j DROP


This is where you can take care of malicious source IP addresses. For this to work properly, you must make sure you know the offending source IP address and that, in fact, it’s one you want to block. The biggest problem with this occurs when the offending address has been spoofed. If that’s the case, you can wind up blocking legitimate traffic from reaching your network. Do your research on this address.


8: iptables -N port-scan


This is the beginning of a rule to block furtive port scanning. A furtive port scan is a scan that detects closed ports to deduce open ports. Two more lines are needed to complete this rule:


iptables -A port-scan -p tcp --tcp-flags SYN,ACK,FIN,RST RST -m limit --limit 1/s -j RETURN

iptables -A port-scan -j DROP


Notice that the above rule set is adding a new chain called “port-scan”. You don’t have to name it such; it’s just easier to keep things organized. You can also add timeouts to the above rule set like so:


iptables -A specific-rule-set -p tcp --syn -j syn-flood

iptables -A specific-rule-set -p tcp --tcp-flags SYN,ACK,FIN,RST RST -j port-scan


9: iptables -A INPUT -i eth0 -p tcp -m state –state NEW -m multiport –dports ssh,smtp,http,https -j ACCEPT


What you see here is a chain making use of the multiport argument, which will allow you to set up multiple ports. Using the multiport argument lets you write one chain instead of multiple chains. This single rule saves you from writing out four separate rules, one each for ssh, smtp, http, and https. Naturally, you can apply this to ACCEPT, DENY, REJECT.


10: iptables -A PREROUTING -i eth0 -p tcp –dport 80 -m state –state NEW -m nth –counter 0 –every 4 –packet 0 -j DNAT –to-destination 192.168.1.10:80


If you’re looking to load balance between multiple mirrored servers (in the example case, load balancing a Web server at 192.168.1.10), this rule is what you want. At the heart of this rule is the nth extension, which tells iptables to act on every “nth” packet. In the example, iptables uses counter 0 and acts upon every 4th packet. You can extend this to balance out your mirrored sites this way. Say you have four mirrored servers up and you want to balance the load between them. You could have one line for each server like so:


<code>iptables -A PREROUTING -i eth0 -p tcp --dport 80 -m state --state NEW -m nth --counter 0 --every 4 --packet 0 -j DNAT --to-destination 192.168.1.10:80</code>

iptables -A PREROUTING -i eth0 -p tcp --dport 80 -m state --state NEW -m nth --counter 0 --every 4 --packet 1 -j DNAT --to-destination 192.168.1.20:80

iptables -A PREROUTING -i eth0 -p tcp --dport 80 -m state --state NEW -m nth --counter 0 --every 4 --packet 2 -j DNAT --to-destination 192.168.1.30:80

iptables -A PREROUTING -i eth0 -p tcp --dport 80 -m state --state NEW -m nth --counter 0 --every 4 --packet 3 -j DNAT --to-destination 192.168.1.40:80

As you can see the server on .10 will be routed every 0 packet, the server on .20 will be routed every 1st packet, the server on .30 will be routed every 2nd packet, and the server on .40 will be routed every 3rd packet.


REFERENCE

http://blogs.techrepublic.com.com/10things/?p=539

IPTables (Linux Firewall)

SkyHi @ Thursday, April 15, 2010
Logging connections with IPtables


Logging ALL incomming and outgoing traffic



iptables -A OUTPUT -j LOG

iptables -A INPUT -j LOG

iptables -A FORWARD -j LOG

iptables -t nat -A PREROUTING -j LOG

iptables -t nat -A POSTROUTING -j LOG

iptables -t nat -A OUTPUT -j LOG


Description: Above commands will enable logging for all input/output/forwarded/routed traffic in /var/log/messages file. (Log file depend on syslog setting).



A Customized Logging Chain to Log all ssh connections



iptables -N LOGIT # special chain to log all except fragments

iptables -A LOGIT -m state --state ESTABLISHED -j RETURN # don't log frags

iptables -A LOGIT -j LOG

iptables -A LOGIT -j RETURN


Above commands will create a new chain LOGIT and will set it to log all except fragments. Now lets use this chain.


iptables -A INPUT -p tcp --dport 22 -j LOGIT


Description: It will log all connections to port 22 (SSH).


Below is the complete shell script for above loging.

#!/bin/bash
iptables -N LOGIT # special chain to log all except fragments

iptables -A LOGIT -m state --state ESTABLISHED -j RETURN # don't log frags
iptables -A LOGIT -j LOG
iptables -A LOGIT -j RETURN

iptables -A INPUT -p tcp --dport 22 -j LOGIT
#end



Reverse script to delete above iptables config.

#!/bin/bash<br /><br />  iptables -D LOGIT -m state --state ESTABLISHED -j RETURN <br />  iptables -D LOGIT -j LOG<br />  iptables -D LOGIT -j RETURN<br /><br />  iptables -D INPUT -p tcp --dport 22 -j LOGIT<br />  iptables -X LOGIT <br /><br /><br />#end<br /><br />




Blocking traffic with IPtables



Blocking an IP (Drop connection)


Example: iptables -A INPUT -s 192.168.0.1 -j DROP


Blocking an IP (Rejecting connection)


Example: iptables -A INPUT -s 192.168.0.1 -j REJECT


Blocking access of an ip to a certain port


Example: iptables -A INPUT -p tcp -s 192.168.1.50 --dport 110 -j
REJECT

Description: This will reject connection from 192.168.1.50 at port 110.

Example: iptables -A INPUT -p udp -s 192.168.1.50 --dport 52 -j REJECT

Description: This will reject udp traffic from 192.168.1.50 at port 52


Blocking All Incomming Traffic at a port


Example: iptables -A INPUT -p tcp --dport 110 -j REJECT

Description: This will reject ALL Incomming connections/Traffic at port 110.



Blocking Incomming Pings


Example: iptables -A INPUT -p icmp -j DROP

Description: Usefull to protect against automated network scans
to detect live ips.


Blocking access to an external ip from within your server


Example: iptables -A OUTPUT -p tcp -d 192.168.1.50 -j REJECT
Description: This will block access to 192.168.1.50 from with in your server. Means your server users can not access that ip from with in the server


Blocking access to an external port of an external ip


Example: iptables -A OUTPUT -p tcp -d 192.168.1.50 --dport 25 -j REJECT

Description: Port 25 of 192.168.1.50 will not be accessable from with in your server





Routing with IPtables


Redirecting a tcp port to another port


Example: iptables -t nat -A PREROUTING -p tcp --dport 80 -j REDIRECT --to-ports 8080

Description: Port 80 will be redirected to port 8080, Means if you will connect at port 80 of this server then you will actually connected to 8080




Redirecting traffic from specific ip at a tcp port to another port


Example: iptables -t nat -A PREROUTING -p tcp -s 192.168.1.40 --dport 80 -j REDIRECT --to-ports 8080

Description: All traffic from 192.168.1.40 at Port 80 will be redirected to port 8080, Means if 192.168.1.40 will connect at port 80 of this server then it will actually connected to 8080




Note: REDIRECT target can be used only to redirect traffic to the machine itself. To route traffic to other places, Use DNAT (see below)


Routing traffic from specific port to another server



Example:

echo 1 > /proc/sys/net/ipv4/ip_forward

iptables -t nat -D POSTROUTING -o eth0 -j MASQUERADE

iptables -t nat -A PREROUTING -p tcp -d 10.10.10.10 --dport 72 -j DNAT --to 33.55.37.226:25

Description: Above commands will route the traffic for port 72 of ip 10.10.10.10 to port 25 of ip 33.55.37.226 .




Listing and Deleting current rules


Example: iptables -L

Description: It will list all chains and rules


Example: iptables -L chain_name

Description: It will list all rules in a specific chain


Example: iptables -D LOGIT -j LOG

Description: It will delete the specific rule. The rule must be exact as it was executed.


Example: iptables -F chain_name

Description: It will delete all rules in chain_name


Example: iptables -F

Description: It will delete all rules in all chains

REFERENCE
http://www.openpages.info/iptables/

Wednesday, April 14, 2010

seo-issues-with-moving-a-domain/

SkyHi @ Wednesday, April 14, 2010
Moving a domain is not fun, especially when you are enjoying good rankings on the old domain. I'm getting asked how to move a domain more and more, so it's time I put my thoughts online in a place where I can easily reference them.


The problem


Let's say you own a domain like bestsitesof2006.com - it's now 2007 and you realise that your purchase of said domain may not have been the best long-term choice.


You want to start using your great brand-new domain bestsitesof2007.com (not the quickest are you?).




bestsitesof2006.com is getting great traffic and has good pagerank, and you don't want to lose any of that.


Duplicate content, link juice etc


Your great rankings are currently coming from the thousands of links you have painstakingly built over the last year.




If you launch bestsitesof2007.com with exactly the same content as bestsitesof2006.com and run the 2 sites side by side, Google is going to filter out one of the sites using their duplicate content filters (which are getting much better). Google will likely filter out the less powerful of the 2 domains, namely the newest one.




At any rate, you want your link juice from bestsitesof2006.com to flow into bestsitesof2007.com so that it can rank well based on those links.


The 301 redirect


A 301 redirect is known as a permanent redirect, and should be used for permanently changing the URI of a page. This is exactly what we are doing here.




All pages from bestsitesof2006.com need to be 301 redirected to bestsitesof2007.com - this can be done sitewide, or page by page, either method is appropriate.




The GoogleBot will follow links into bestsitesof2006.com and see the 301 redirect. The old page will be removed from Google's index and the new page added instead.


By deleting and then adding, there is no issue with duplicate content as the 2 pages don't exist in Google's index at the same time.


Time delays


Moving a domain is something everyone has different experiences with. I can only report on what I have seen happen to my sites, and what people have told me.




Please don't take this information to be gospel, because it's not.




I have found that it can take a couple of weeks for Google to start deleting old pages and adding new pages, this will depend on how often Googlebot visits your site, and how authorative your domain is.




Your new homepage will be indexed first, followed by other pages on your new site.




If your new domain is newly registered, expect your rankings to take a dive for a period of weeks to months. The new domain has no authoraty whatsoever, and it takes time for the effect of the redirected links to kick in.


Loss of rankings


Yes, you did read that correctly. The method of moving a domain that I recommend can involve you getting shit rankings for up to 6 months. If your old domain has really great links and authority, then some people report no ongoing reduction in rankings.


Pagerank


Some people worry about losing PageRank when shifting a domain. Umm, yeah, you do lose all pagerank across your new site for up to 3 months. But you shouldn't worry about this, be more concerned about rankings, traffic and sales instead.


With any new domain, your pagerank is zero until Google does a quarterly toolbar update. At any rate, the toolbar does not reflect the true PageRank of the site, so it's not worth worrying about.


I generally find the PageRank of sites I move returns to normal within 3-6 months of the move (1 - 2 toolbar updates).


Updating old links


You have spent lots of time building links. Great. With the 301 redirects, the juice from these links now flows to the new site.


It's still worth approaching some of the better sites that link to you and asking them to update the link.


Cleaning up URLs


So, you are going to change every URL on your site and take a bit of a hit in the rankings, huh?




You don't want to tave to repeat this process again.




Now is a great time to clean up the URLs on your site.




eg...


bad: www.domain.com/index.php?pageid=45


bad: www.domain.com/aboutourteam.php


bad: www.domain.com/about_our_team.php


bad: www.domain.com/About-Our-Team.php


bad: www.domain.com/content/45.htm




good: www.domain.com/about-our-team.php


good: www.domain.com/about-our-team/


good: www.domain.com/about-our-team.htm


good: www.domain.com/about-our-team


good: www.domain.com/team/




If your site is static, rename all your pages so they are dash separated, lower case only, and logical.


If your site is driven by a content management system or forum package, now's a great time to go install whatever SEO plugins or modules are required to fix up the URLs. Many CMS systems such as Wordpress have free plugins that do the job nicely. Others, such a VBulletin have plugins that aren't free (VBSEO plugin), but still well worthwhile investing in.




Shameless plug:


Jojo CMS doesn't need any plugins to have nice URLs.


The bottom line


When you change a domain, you accept some risk that rankings can drop significantly for several months. This is one of the costs of rebranding, so make sure the decision makers consider this cost BEFORE deciding on the rebranding of a business.




The unpleasantness of this process is one reason why domainers will pay good money for the right domain name, and why it's better to think long-term before investing time and money into the wrong domain.

REFERENCE
http://www.ragepank.com/articles/97/seo-issues-with-moving-a-domain/

Outlook duplicate records: Why they occur and prevention

SkyHi @ Wednesday, April 14, 2010

Why do Outlook duplicates occur and how can I prevent them?

Outlook duplicate records can occur for many reasons, and in most cases, a simple fix will prevent them from happening in the future. Below you will find some of the most common causes of duplicates, and some solutions to prevent them in Outlook.

The first, and probably the most common, is synchronization of your Outlook with a PDA. Duplicates can occur here if you have the same Outlook record stored under different terms in Outlook and your PDA. For example, if you have the same contact (we'll call him Bill Henderson) in both your PDA and Outlook, and Bill's job title in your PDA is "Sales", but in Outlook he is listed as "Sales Rep", when you synchronize your PDA to Outlook it will create an Outlook duplicate record: one for Bill Henderson "Sales Rep", and another for Bill Henderson "Sales".

To prevent this, one of the easiest solutions is to get your Outlook contacts in order. You can do this by creating a new secondary Outlook account (not configured for email) and sync all of your contacts from your PDA into this account as new. You can then clean them up to match your original Contacts folder, and sync back into your PDA. Once that’s complete, you can delete the secondary Outlook account and your next sync to Outlook should go flawlessly.

Another reason for Outlook duplicate records could be that Outlook downloaded an email twice from one or more POP3 servers. This could happen for multiple reasons; a background synchronization happened at the exact time you were emptying the Deleted Items folder, messages you intentionally leave on the mail server are downloaded again on the next Send/Receive, you have two or more alias accounts pointing at the same POP3 server, or other inconsistent occurrences.

If you believe your Outlook duplicates are caused by an underlying Outlook functioning error, check with Microsoft Support at http://support.microsoft.com/; in many cases, you just need to update your software because of an Outlook bug.

Reinstallation and re-configuration of Microsoft Outlook could also Outlook duplicate records. If you install a fresh installation of Outlook with your email account, select your old .pst file as the delivery location, but originally chose to leave a copy on the server; those old emails will not only still be in your .pst file, the server will also download them again and create Outlook duplicates you will need to remove. This is because the server didn't recognize that the emails had already been received by another email account.

To prevent this problem, make sure you start Outlook with a clean .pst file after re-installation. Once the "new" email messages have been received, open your old .pst file and move only the new messages (not the old message "duplicates") into the old .pst file. Then delete the new .pst file and set your default delivery to the original .pst file.

Your firewall and/or anti-virus software can also create Outlook duplicates. In a perfect Send/Receive cycle, Outlook sends a request to the mail server for mail messages; new emails not in Outlook are then downloaded and the connection is closed. However, if your firewall or anti-virus software interjects itself and causes something to fail at any point in this process (flagging a large message, etc.) the connection will not close properly. This means that any emails that were downloaded before the point of failure will be downloaded again on the next Send/Receive, thereby creating duplicate emails in your Outlook inbox.

If you think your firewall/virus scanner is causing Outlook duplicates, change your security settings to not interfere with Outlook. Outlook's "in-house" security measures are tight enough as it is, and it does the work for you, so there’s no need to worry.

Check your rules in Microsoft Outlook – they could be set to send the same message to two or more places. If you have multiple rules set up to deal with incoming email messages, there's a possibility that Outlook could create duplicates in order to satisfy two different rules. For example, if you have a rule to send all emails from "Jack Morris" to a specific folder, and all emails with the subject line "Outlook" to go to the same folder, and you get an email from "Jack Morris" with "Outlook" in the subject line, technically both rules apply. Outlook will then move the message twice, creating a duplicate email that you will want to remove.

To prevent this problem, be sure to add the action "stop processing more rules" to the rule. This way, if a rule has already been applied to a message, no further action will be taken and duplicates will not be created to satisfy all rules.


REFERENCE

http://www.anti-dupe.com/products/prevent_outlook_duplicate.aspx



Micosoft Outlook Issues and Problems Duplicate Emails

SkyHi @ Wednesday, April 14, 2010
If you are getting duplicate emails on your computer in your Outlook Express, it could be that there is a corrupt email message in your inbox on the mail server. To delete the corupt email, simply:
  1. Login to your mail server
    http://mail.comentum.com/mail/
    - Enter your user name (your email address)
    - Enter your password
  2. Delete the corrupt/damaged message that is creating the duplicates
    (You may want to delete all messages on the server, after you have downloaded them to your Outlook.)
Possible Causes of Duplicate Emails on Outlook Express

Problem: The "Leave Messages on the Server" box has been checked on your Outlook Express account settings.
Fix: Uncheck the "Leave Messages on the Server" box. To do this, click on Tools>Accounts. You will see a box open (named Internet Accounts), click "Mail" tab, click on the mail account, click on Properties>Advanced. Remove the check in the box "Leave a copy of messages on server."

(The following information is from Microsoft Help and Support)
Problem: Outlook Downloads Messages from a POP3 Server Twice
Outlook downloads messages from a Post Office Protocol 3 (POP3) server again after it empties the Deleted Items folder even if you have both the Leave a copy of messages on the server and the Remove from server when deleted from Deleted Items options enabled on the POP3 account.
Fix: There is not an available workaround for this problem. . . Simply delete the duplicate messages.

Advanced Configurations:
How to setup email forwarding
How to create an Auto Responder
How to turn on Spam Filter

REFERENCE
http://www.comentum.com/outlook-issues.html


Sendmail Block subject

SkyHi @ Wednesday, April 14, 2010
> What can I do to block a message with a determinated subject with
> mimedefang ???

I've posted this a few times before.

Blocking emails based on the Subject line can be done by adding the
following LOCAL_RULESET to your sendmail.mc file, and then rebuilding
sendmail.cf. PLEASE NOTE that there are TABS in the code below. If you
copy/paste the code below into yout sendmail.mc file, BE SURE TO REPLACE any
occurances of "[TAB]" with a real TAB.

Once the sendmail.cf has been rebuilt (and sendmail restarted), create two
files. The first file (subjects_full) will contain COMPLETE SUBJECT LINES,
using PERIODS to replace any spaces. The second file can contain any
KEYWORDS or portions of subject lines (again, replacing any spaces with
periods).

For example, in /etc/mail/subjects_full you might have something like:

Mothers.Day.Order.Confirmation
Dangerous.Virus.Warning
Virus.ALERT!!!
Important!.Read.carefully!!
How.to.protect.yourself.from.the.IL0VEY0U.bug!
I.Cant.Believe.This!!!
Thank.You.For.Flying.With.Arab.Airlines
Variant.Test
Yeah,.Yeah.another.time.to.DEATH...
LOOK!
Bewerbung.Kreolina
Recent.Virus.Attacks-Fix
PresenteUOL
IMPORTANT:.Official.virus.and.bug.fix
NEUE.ANTI-VIRUS-LISTE
BUG.&.VIRUS.FIX
New.Variation.on.LOVEBUG.Update.Anti-Virus!!
Snowhite.and.the.Seven.Dwarfs.-.The.REAL.story
Resume.-.Janet.Simons
US.PRESIDENT.AND.FBI.SECRET
Check.this.out,.it's.funny!
Cool.Notepad.Demo
Moin,.alles.klar?
Hi,.how.are.you?

In /etc/mail/subjects_part you could have something like:

unsecured.gold.mastercard
unsecured.mastercard
unsecured.platinum.card
unsecured.visa
viagra
v.i.a.g.r.a
vi*agra
v1agra
v*1a*gra

These are plain ascii files... NOT database hashes. And, there is no need
to restart sendmail whenever you add anything to these files. Changes take
effect immediately.

Have fun!


LOCAL_RULESETS
######################################################################
###
### Email Virus and Anti-SPAM stuff...
###
### Add exact-match subject lines to /etc/mail/subjects_full
### Add substrings to match in subject lines to /etc/mail/subjects_part
### In both files, all spaces MUST be replaced with periods (.)
###
### Create two files called /etc/mail/subjects_full and
### /etc/mail/subjects_part. The former has complete
### unwanted 'subject' lines, while the latter has only
### substrings within 'subject' lines.
###
### As an example, suppose you want to filter out 'viagra'
### spam. The following entry in your subjects_part
### file would do it:
### viagra
###
### In the case of multi-word entries, all spaces MUST be
### replaced with periods. For example:
### herbal.viagra
###
### These filters are not case-sensitive.
###
######################################################################
F{FullSubjects} -o /etc/mail/subjects_full
F{PartSubjects} -o /etc/mail/subjects_part
HSubject: $>CheckSubject

SCheckSubject
R$={FullSubjects}$*[TAB]$: REJECTSUBJECT
R$* $={PartSubjects} $*[TAB]$: REJECTSUBJECT
R$* REJECTSUBJECT $*[TAB]$#error $: "553 Access Denied - MSG may contain
SPAM/WORM/VIRUS/HOAX."

RADV : $*[TAB]$#error $: "553 Delivery blocked; HSubject: indicates
unsolicited commercial email."
R ADV : ADLT $*[TAB]$#error $: "553 Delivery blocked; HSubject: indicates
unsolicited adult-content email."
RADV $*[TAB]$#error $: "553 Delivery blocked; HSubject: indicates
unsolicited commercial email."



KEN CORMACK, RHCE
Sr. UNIX Systems Analyst,
Open Systems Group
Sr. Software Analyst,
TSG Midrange Systems Group
AFFILIATED COMPUTER SERVICES, INC.
557 E. Tallmadge Ave., Akron, OH 44310

"If that that is 'is' is that that is not 'not is', is that that is 'not is'
that that is not 'is'? It is!" - Ken Cormack

"Sendmail administration is not black magic. There are legitimate technical
reasons why it requires the sacrificing of a live chicken." - Unknown


REFERENCES
http://lists.roaringpenguin.com/pipermail/mimedefang/2004-March/020796.html

Tuesday, April 13, 2010

PHP Fatal error: Call to a member function saveXML() on a non-object

SkyHi @ Tuesday, April 13, 2010
#yum install php-xml-5.1.6-23.2.el5_3

Monday, April 12, 2010

Ksplice: Upgrade / Patch Your Linux Kernel Without Reboots

SkyHi @ Monday, April 12, 2010

Generally, all Linux distributions needs a scheduled reboot once to stay up to date with important kernel security updates. RHN (or other distro vendors) provides Linux kernel security updates. You can apply kernel updates using yum command or apt-get command line options. After each upgrade you need to reboot the server. Ksplice service allows you to skip reboot step and apply hotfixes to kernel without rebooting the server. In this post I will cover a quick installation of Ksplice for RHEL 5.x and try to find out if service is worth every penny.



The technology and hack behind this looks pretty cool. This is useful if you've a small number of Linux based servers and/or you want avoid unscheduled reboot just to apply hotfix to Linux kernel.


How Do I Install Ksplice?


First, you need to register with Ksplice. Type the following command to install rpm repo under RHEL 5:

# rpm -ivh https://www.ksplice.com/yum/uptrack/centos/ksplice-uptrack-release.noarch.rpm

To install Ksplice, enter:

# yum install uptrack

Edit /etc/uptrack/uptrack.conf, enter:

# vi /etc/uptrack/uptrack.conf

Update it as follows (input your access key):


[Auth]
accesskey = ADD-YOUR-ACCESS-KEY-HERE

[Network]
# Proxy to use when accessing the Uptrack server, of the form
# [protocol://]<host>[:port]

# The proxy must support making HTTPS connections. If this is unset,
# Uptrack will look for the https_proxy, HTTPS_PROXY, and http_proxy
# environment variables in that order, and then finally look for a
# proxy setting in the system-wide GConf database, if available.
https_proxy =

[Settings]
# Automatically install updates at boot time. If this is set, on
# reboot into the same kernel, Uptrack will re-install the same set of
# updates that were present before the reboot.
install_on_reboot = yes

# Options configuring the Uptrack cron job.
#
# GUI users will get all notices via the GUI and likely want to set
# the following cron options to "no".

# Cron job will install updates automatically
autoinstall = no

# Cron job will print a message when new updates are installed.
# This option is only relevant if autoinstall = yes
cron_output_install = no

# Cron job will print a message when new updates are available
cron_output_available = no

# Cron job will print a message when it encounters errors
cron_output_error = no</host>

Save and close the file.


How Do I Apply Rebootless Kernel Updates?


You need to first download and apply updates via RHN:

# yum -y update

OR

# yum update kernel kernel-headers kernel-devel

Don't reboot the box, simply type the following command to apply hotfix:

# uptrack-upgrade

To see a list of updates that are currently installed, enter:

# uptrack-show -y


Sample Email Notification


You will get an email as follows when updates are available:


Fig.01: Ksplice Update Notification

Fig.01: Ksplice Update Notification




The web interface also provides information about your server and installed kernel updates:


Fig.02: Uptrack Web Interface

Fig.02: Uptrack Web Interface



Conclusion


The pricing is as follows:


  • Monthly price per system First 20 servers : $3.95
  • Beyond 20 servers: $2.95
  • Currently it is free for all Ubuntu users.

Ksplice is a pretty good and stable software. This is useful for Linux admin or business who can not accept downtime for patching. A few business comes in my mind:


  • Small shop, say 8-12 Linux based servers.
  • Pro-blogging or webmaster servers (a typical setup included one web server and one db server). Avoiding downtime means more ad revenue for webmasters.
  • Hosting companies - again avoiding downtime means good customer satisfactions and less work for sys admins. If you run VM based hosting (OpenVZ or XEN based vps) you can avoid downtime too.
  • Small cluster of Linux system, say 6 system - If cluster is using 80% of capacity and if one of node rebooted for kernel upgrade, load will up for rest of 5 systems. In such case, this service can help to keep load under control without rebooting the box. However, this is NOT very useful for very large Linux based cluster redundant load-balanced servers, routers, switches, firewalls etc. Since your cluster is so large that 4-5 servers failing makes no difference to the remaining nodes. In some cases it is possible to do geo load balancing too.

But I've HA Failover Solution In Place...


100% uptime depends upon lots of factors and and HA solution handles hardware or other failures very well. However, Ksplice service is not all about 100% uptime, it is about not rebooting your server for a Linux kernel upgrade. You can easily combine Ksplice with HA solution (such as keepalived+nginx reverse proxy) and try to get perfect five 9s. I highly recommend this service for small to medium size business or professional webmasters.


Further readings:


Apache HTTP Server Version 2.0 VirtualHost Examples

SkyHi @ Monday, April 12, 2010

This document attempts to answer the commonly-asked questions about
setting up virtual hosts. These scenarios are those involving multiple
web sites running on a single server, via name-based or IP-based virtual hosts.





top


Running several name-based web
sites on a single IP address.



Your server has a single IP address, and multiple aliases (CNAMES)
point to this machine in DNS. You want to run a web server for
www.example1.com and www.example2.org on this
machine.



Note

Creating virtual
host configurations on your Apache server does not magically
cause DNS entries to be created for those host names. You
must have the names in DNS, resolving to your IP
address, or nobody else will be able to see your web site. You
can put entries in your hosts file for local
testing, but that will work only from the machine with those
hosts entries.




Server configuration




# Ensure that Apache listens on port 80

Listen 80



# Listen for virtual host requests on all IP addresses

NameVirtualHost *:80



<VirtualHost *:80>


DocumentRoot /www/example1

ServerName www.example1.com



# Other directives here




</VirtualHost>



<VirtualHost *:80>


DocumentRoot /www/example2

ServerName www.example2.org



# Other directives here




</VirtualHost>



The asterisks match all addresses, so the main server serves no
requests. Due to the fact that www.example1.com is first
in the configuration file, it has the highest priority and can be seen
as the default or primary server. That means
that if a request is received that does not match one of the specified
ServerName directives, it will be served by this first
VirtualHost.




Note



You can, if you wish, replace * with the actual
IP address of the system. In that case, the argument to
VirtualHost must match the argument to
NameVirtualHost:




NameVirtualHost 172.20.30.40



<VirtualHost 172.20.30.40>

# etc ...



However, it is additionally useful to use *
on systems where the IP address is not predictable - for
example if you have a dynamic IP address with your ISP, and
you are using some variety of dynamic DNS solution. Since
* matches any IP address, this configuration
would work without changes whenever your IP address
changes.




The above configuration is what you will want to use in almost
all name-based virtual hosting situations. The only thing that this
configuration will not work for, in fact, is when you are serving
different content based on differing IP addresses or ports.



top


Name-based hosts on more than one
IP address.




Note

Any of the
techniques discussed here can be extended to any number of IP
addresses.




The server has two IP addresses. On one (172.20.30.40), we
will serve the "main" server, server.domain.com and on the
other (172.20.30.50), we will serve two or more virtual hosts.



Server configuration




Listen 80



# This is the "main" server running on 172.20.30.40

ServerName server.domain.com

DocumentRoot /www/mainserver



# This is the other address

NameVirtualHost 172.20.30.50



<VirtualHost 172.20.30.50>


DocumentRoot /www/example1

ServerName www.example1.com



# Other directives here ...




</VirtualHost>



<VirtualHost 172.20.30.50>


DocumentRoot /www/example2

ServerName www.example2.org



# Other directives here ...




</VirtualHost>



Any request to an address other than 172.20.30.50 will be
served from the main server. A request to 172.20.30.50 with an
unknown hostname, or no Host: header, will be served from
www.example1.com.



top


Serving the same content on
different IP addresses (such as an internal and external
address).



The server machine has two IP addresses (192.168.1.1
and 172.20.30.40). The machine is sitting between an
internal (intranet) network and an external (internet) network. Outside
of the network, the name server.example.com resolves to
the external address (172.20.30.40), but inside the
network, that same name resolves to the internal address
(192.168.1.1).



The server can be made to respond to internal and external requests
with the same content, with just one VirtualHost
section.



Server configuration




NameVirtualHost 192.168.1.1

NameVirtualHost 172.20.30.40



<VirtualHost 192.168.1.1 172.20.30.40>


DocumentRoot /www/server1

ServerName server.example.com

ServerAlias server


</VirtualHost>



Now requests from both networks will be served from the same
VirtualHost.




Note:

On the internal
network, one can just use the name server rather
than the fully qualified host name
server.example.com.



Note also that, in the above example, you can replace the list
of IP addresses with *, which will cause the server to
respond the same on all addresses.




top


Running different sites on different
ports.



You have multiple domains going to the same IP and also want to
serve multiple ports. By defining the ports in the "NameVirtualHost"
tag, you can allow this to work. If you try using <VirtualHost
name:port> without the NameVirtualHost name:port or you try to use
the Listen directive, your configuration will not work.



Server configuration




Listen 80

Listen 8080



NameVirtualHost 172.20.30.40:80

NameVirtualHost 172.20.30.40:8080



<VirtualHost 172.20.30.40:80>


ServerName www.example1.com

DocumentRoot /www/domain-80


</VirtualHost>



<VirtualHost 172.20.30.40:8080>


ServerName www.example1.com

DocumentRoot /www/domain-8080


</VirtualHost>



<VirtualHost 172.20.30.40:80>


ServerName www.example2.org

DocumentRoot /www/otherdomain-80


</VirtualHost>



<VirtualHost 172.20.30.40:8080>


ServerName www.example2.org

DocumentRoot /www/otherdomain-8080


</VirtualHost>



top


IP-based virtual hosting



The server has two IP addresses (172.20.30.40 and
172.20.30.50) which resolve to the names
www.example1.com and www.example2.org
respectively.



Server configuration




Listen 80



<VirtualHost 172.20.30.40>


DocumentRoot /www/example1

ServerName www.example1.com


</VirtualHost>



<VirtualHost 172.20.30.50>


DocumentRoot /www/example2

ServerName www.example2.org


</VirtualHost>



Requests for any address not specified in one of the
<VirtualHost> directives (such as
localhost, for example) will go to the main server, if
there is one.



top


Mixed port-based and ip-based virtual
hosts



The server machine has two IP addresses (172.20.30.40 and
172.20.30.50) which resolve to the names
www.example1.com and www.example2.org
respectively. In each case, we want to run hosts on ports 80 and
8080.



Server configuration




Listen 172.20.30.40:80

Listen 172.20.30.40:8080

Listen 172.20.30.50:80

Listen 172.20.30.50:8080



<VirtualHost 172.20.30.40:80>


DocumentRoot /www/example1-80

ServerName www.example1.com


</VirtualHost>



<VirtualHost 172.20.30.40:8080>


DocumentRoot /www/example1-8080

ServerName www.example1.com


</VirtualHost>



<VirtualHost 172.20.30.50:80>


DocumentRoot /www/example2-80

ServerName www.example1.org


</VirtualHost>



<VirtualHost 172.20.30.50:8080>


DocumentRoot /www/example2-8080

ServerName www.example2.org


</VirtualHost>



top


Mixed name-based and IP-based
vhosts



On some of my addresses, I want to do name-based virtual hosts, and
on others, IP-based hosts.



Server configuration




Listen 80



NameVirtualHost 172.20.30.40



<VirtualHost 172.20.30.40>


DocumentRoot /www/example1

ServerName www.example1.com


</VirtualHost>



<VirtualHost 172.20.30.40>


DocumentRoot /www/example2

ServerName www.example2.org


</VirtualHost>



<VirtualHost 172.20.30.40>


DocumentRoot /www/example3

ServerName www.example3.net


</VirtualHost>



# IP-based

<VirtualHost 172.20.30.50>


DocumentRoot /www/example4

ServerName www.example4.edu


</VirtualHost>



<VirtualHost 172.20.30.60>


DocumentRoot /www/example5

ServerName www.example5.gov


</VirtualHost>



top


Using Virtual_host and
mod_proxy together



The following example allows a front-end machine to proxy a
virtual host through to a server running on another machine. In the
example, a virtual host of the same name is configured on a machine
at 192.168.111.2. The ProxyPreserveHost On directive is
used so that the desired hostname is passed through, in case we are
proxying multiple hostnames to a single machine.




<VirtualHost *:*>

ProxyPreserveHost On

ProxyPass / http://192.168.111.2/

ProxyPassReverse / http://192.168.111.2/

ServerName hostname.example.com

</VirtualHost>



top


Using _default_
vhosts



_default_ vhosts
for all ports



Catching every request to any unspecified IP address and
port, i.e., an address/port combination that is not used for
any other virtual host.



Server configuration




<VirtualHost _default_:*>


DocumentRoot /www/default


</VirtualHost>



Using such a default vhost with a wildcard port effectively prevents
any request going to the main server.



A default vhost never serves a request that was sent to an
address/port that is used for name-based vhosts. If the request
contained an unknown or no Host: header it is always
served from the primary name-based vhost (the vhost for that
address/port appearing first in the configuration file).



You can use AliasMatch or
RewriteRule to rewrite any
request to a single information page (or script).




_default_ vhosts
for different ports



Same as setup 1, but the server listens on several ports and we want
to use a second _default_ vhost for port 80.



Server configuration




<VirtualHost _default_:80>


DocumentRoot /www/default80

# ...


</VirtualHost>



<VirtualHost _default_:*>


DocumentRoot /www/default

# ...


</VirtualHost>



The default vhost for port 80 (which must appear before any
default vhost with a wildcard port) catches all requests that were sent
to an unspecified IP address. The main server is never used to serve a
request.




_default_ vhosts
for one port



We want to have a default vhost for port 80, but no other default
vhosts.



Server configuration




<VirtualHost _default_:80>

DocumentRoot /www/default

...

</VirtualHost>



A request to an unspecified address on port 80 is served from the
default vhost any other request to an unspecified address and port is
served from the main server.




top


Migrating a name-based vhost to an
IP-based vhost



The name-based vhost with the hostname
www.example2.org (from our name-based example, setup 2) should get its own IP
address. To avoid problems with name servers or proxies who cached the
old IP address for the name-based vhost we want to provide both
variants during a migration phase.

The solution is easy, because we can simply add the new IP address
(172.20.30.50) to the VirtualHost
directive.



Server configuration




Listen 80

ServerName www.example1.com

DocumentRoot /www/example1



NameVirtualHost 172.20.30.40



<VirtualHost 172.20.30.40 172.20.30.50>


DocumentRoot /www/example2

ServerName www.example2.org

# ...


</VirtualHost>



<VirtualHost 172.20.30.40>


DocumentRoot /www/example3

ServerName www.example3.net

ServerAlias *.example3.net

# ...


</VirtualHost>



The vhost can now be accessed through the new address (as an
IP-based vhost) and through the old address (as a name-based
vhost).



top


Using the ServerPath
directive



We have a server with two name-based vhosts. In order to match the
correct virtual host a client must send the correct Host:
header. Old HTTP/1.0 clients do not send such a header and Apache has
no clue what vhost the client tried to reach (and serves the request
from the primary vhost). To provide as much backward compatibility as
possible we create a primary vhost which returns a single page
containing links with an URL prefix to the name-based virtual
hosts.



Server configuration




NameVirtualHost 172.20.30.40



<VirtualHost 172.20.30.40>


# primary vhost

DocumentRoot /www/subdomain

RewriteEngine On

RewriteRule ^/.* /www/subdomain/index.html

# ...


</VirtualHost>



<VirtualHost 172.20.30.40>

DocumentRoot /www/subdomain/sub1


ServerName www.sub1.domain.tld

ServerPath /sub1/

RewriteEngine On

RewriteRule ^(/sub1/.*) /www/subdomain$1

# ...


</VirtualHost>



<VirtualHost 172.20.30.40>


DocumentRoot /www/subdomain/sub2

ServerName www.sub2.domain.tld

ServerPath /sub2/

RewriteEngine On

RewriteRule ^(/sub2/.*) /www/subdomain$1

# ...


</VirtualHost>



Due to the ServerPath
directive a request to the URL
http://www.sub1.domain.tld/sub1/ is always served
from the sub1-vhost.
A request to the URL
http://www.sub1.domain.tld/ is only
served from the sub1-vhost if the client sent a correct
Host: header. If no Host: header is sent the
client gets the information page from the primary host.

Please note that there is one oddity: A request to
http://www.sub2.domain.tld/sub1/ is also served from the
sub1-vhost if the client sent no Host: header.

The RewriteRule directives
are used to make sure that a client which sent a correct
Host: header can use both URL variants, i.e.,
with or without URL prefix.







REFERENCE
http://httpd.apache.org/docs/2.0/vhosts/examples.html