http://stackoverflow.com/questions/1711/what-is-the-single-most-influential-book-every-programmer-should-read

http://stackoverflow.com/questions/1177724/what-soft-skills-make-a-great-programmer

http://stackoverflow.com/questions/330412/which-books-made-you-a-better-programmer
http://stackoverflow.com/questions/67794/what-skills-are-worth-learning-for-a-programming-career-and-or-resume

http://stackoverflow.com/questions/503325/what-makes-a-great-programmer

http://stackoverflow.com/questions/294825/what-skills-should-a-programmer-have-nowadays
http://stackoverflow.com/questions/605913/top-developer-skills

http://stackoverflow.com/questions/184310/do-you-ever-feel-confident-in-your-skills

http://stackoverflow.com/questions/1490346/how-to-compare-programming-skill
http://stackoverflow.com/questions/82639/how-to-become-a-better-programmer

http://stackoverflow.com/questions/62960/what-are-the-basics-of-networking-sys-administration-should-a-programmer-know
read them all!

Blocking Common Attacks using ModSecurity 2.5: Part 3

Read Part One of Blocking Common Attacks using ModSecurity 2.5 here.

Read Part Two of Blocking Common Attacks using ModSecurity 2.5 here.

Source code revelation

Normally, requesting a file with a .php extension will cause mod_php to execute the PHP code contained within the file and then return the resulting web page to the user. If the web server is misconfigured (for example if mod_php is not loaded) then the .php file will be sent by the server without interpretation, and this can be a security problem. If the source code contains credentials used to connect to an SQL database then that opens up an avenue for attack, and of course the source code being available will allow a potential attacker to scrutinize the code for vulnerabilities.

Preventing source code revelation is easy. With response body access on in ModSecurity, simply add a rule to detect the opening PHP tag:

Prevent PHP source code from being disclosed

SecRule RESPONSE_BODY "

Preventing Perl and JSP source code from being disclosed works in a similar manner:

# Prevent Perl source code from being disclosed

SecRule RESPONSE_BODY "#!/usr/bin/perl" "deny,msg:'Perl

source code disclosure blocked'"

# Prevent JSP source code from being disclosed

SecRule RESPONSE_BODY "

Directory traversal attacks

Normally, all web servers should be configured to reject attempts to access any document that is not under the web server's root directory. For example, if your web server root is /home/www, then attempting to retrieve /home/joan/.bashrc should not be possible since this file is not located under the /home/www web server root. The obvious attempt to access the /home/joan directory is, of course, easy for the web server to block, however there is a more subtle way to access this directory which still allows the path to start with /home/www, and that is to make use of the .. symbolic directory link which links to the parent directory in any given directory.

Even though most web servers are hardened against this sort of attack, web applications that accept input from users may still not be checking it properly, potentially allowing users to get access to files they shouldn't be able to view via simple directory traversal attacks. This alone is reason to implement protection against this sort of attack using ModSecurity rules. Furthermore, keeping with the principle of Defense in Depth, having multiple protections against this vulnerability can be beneficial in case the web server should contain a flaw that allows this kind of attack in certain circumstances.

There is more than one way to validly represent the .. link to the parent directory. URL encoding of .. yields % 2e% 2e, and adding the final slash at the end we end up with % 2e% 2e% 2f(please ignore the space).

Here, then is a list of what needs to be blocked:

../

..% 2f

.% 2e/

%  2e%  2e% 2f

% 2e% 2e/

% 2e./

Fortunately, we can use the ModSecurity transformation t:urlDecode. This function does all the URL decoding for us, and will allow us to ignore the percent-encoded values, and thus only one rule is needed to block these attacks:

SecRule REQUEST_URI "../" "t:urlDecode,deny"

Blog spam

The rise of weblogs, or blogs, as a new way to present information, share thoughts, and keep an online journal has made way for a new phenomenon: blog comments designed to advertise a product or drive traffic to a website.

Blog spam isn't a security problem per se, but it can be annoying and cost a lot of time when you have to manually remove spam comments (or delete them from the approval queue, if comments have to be approved before being posted on the blog).

Blog spam can be mitigated by collecting a list of the most common spam phrases, and using the ability of ModSecurity to scan POST data. Any attempted blog comment that contains one of the offending phrases can then be blocked.

From both a performance and maintainability perspective, using the @pmFromFile operator is the best choice when dealing with large word lists such as spam phrases. To create the list of phrases to be blocked, simply insert them into a text file, for example, /usr/local/spamlist.txt:

viagra

v1agra

auto insurance

rx medications

cheap medications

...

Then create ModSecurity rules to block those phrases when they are used in locations such as the page that creates new blog comments:

#

# Prevent blog spam by checking comment against known spam

# phrases in file /usr/local/spamlist.txt

#

SecRule ARGS "@pmFromFile /usr/local/spamlist.txt" "t:

lowercase,deny,msg:'Blog spam blocked'"

Keep in mind that the spam list file can contain whole sentences—not just single words—so be sure to take advantage of that fact when creating the list of known spam phrases.

SQL injection

SQL injection attacks can occur if an attacker is able to supply data to a web application that is then used in unsanitized form in an SQL query. This can cause the SQL query to do completely different things than intended by the developers of the web application. Consider an SQL query like this:

SELECT * FROM user WHERE username = '%s' AND password = '%s';

The flaw here is that if someone can provide a password that looks like ' OR '1'='1, then the query, with username and password inserted, will become:

SELECT * FROM user WHERE username = 'anyuser' AND password = ''

OR '1'='1';

This query will return all users in the results table, since the OR '1'='1' part at the end of the statement will make the entire statement true no matter what username and password is provided.

Standard injection attempts

Let's take a look at some of the most common ways SQL injection attacks are performed.

Retrieving data from multiple tables with UNION

An SQL UNION statement can be used to retrieve data from two separate tables. If there is one table named cooking_recipes and another table named user_credentials, then the following SQL statement will retrieve data from both tables:

SELECT dish_name FROM recipe UNION SELECT username, password

FROM user_credentials;

It's easy to see how the UNION statement can allow an attacker to retrieve data from other tables in the database if he manages to sneak it into a query. A similar SQL statement is UNION ALL, which works almost the same way as UNION—the only difference is that UNION ALL will not eliminate any duplicate rows returned in the result.

Multiple queries in one call

If the SQL engine allows multiple statements in a single SQL query then seemingly harmless statements such as the following can present a problem:

SELECT * FROM products WHERE id = %d;

If an attacker is able to provide an ID parameter of 1; DROP TABLE products;, then the statement suddenly becomes:

SELECT * FROM products WHERE id = 1; DROP TABLE products;

When the SQL engine executes this, it will first perform the expected SELECT query, and then the DROP TABLE products statement, which will cause the products table to be deleted.

Reading arbitrary files

MySQL can be used to read data from arbitrary files on the system. This is done by using the LOAD_FILE() function:

SELECT LOAD_FILE("/etc/passwd");

This command returns the contents of the file /etc/passwd. This works for any file to which the MySQL process has read access.

Writing data to files

MySQL also supports the command INTO OUTFILE which can be used to write data into files. This attack illustrates how dangerous it can be to include user-supplied data in SQL commands, since with the proper syntax, an SQL command can not only affect the database, but also the underlying file system.

This simple example shows how to use MySQL to write the string some data into the file test.txt:

mysql> SELECT "some data" INTO OUTFILE "test.txt";

Preventing SQL injection attacks

There are three important steps you need to take to prevent SQL injection attacks:

Use SQL prepared statements.

Sanitize user data.

Use ModSecurity to block SQL injection code supplied to web applications.

These are in order of importance, so the most important consideration should always be to make sure that any code querying SQL databases that relies on user input should use prepared statements. A prepared statement looks as follows:

SELECT * FROM books WHERE isbn = ? AND num_copies

This allows the SQL engine to replace the question marks with the actual data. Since the SQL engine knows exactly what is data and what SQL syntax, this prevents SQL injection from taking place.

The advantages of using prepared statements are twofold:

They effectively prevent SQL injection.

They speed up execution time, since the SQL engine can compile the statement once, and use the pre-compiled statement on all subsequent query invocations.

So not only will using prepared statements make your code more secure—it will also make it quicker.

The second step is to make sure that any user data used in SQL queries is sanitized. Any unsafe characters such as single quotes should be escaped. If you are using PHP, the function mysql_real_escape_string() will do this for you.

Finally, let's take a look at strings that ModSecurity can help block to prevent SQL injection attacks.

What to block

The following table lists common SQL commands that you should consider blocking, together with a suggested regular expression for blocking. The regular expressions are in lowercase and therefore assume that the t:lowercase transformation function is used.

SQL code

Regular expression

UNION SELECT

unions+select

UNION ALL SELECT

unions+alls+select

INTO OUTFILE

intos+outfile

DROP TABLE

drops+table

ALTER TABLE

alters+table

LOAD_FILE

load_file

SELECT *

selects+*

For example, a rule to detect attempts to write data into files using INTO OUTFILE looks as follows:

SecRule ARGS "intos+outfile" "t:lowercase,deny,msg:

'SQL Injection'"

The s+ regular expression syntax allows for detection of an arbitrary number of whitespace characters. This will detect evasion attempts such as INTO OUTFILE where multiple spaces are used between the SQL command words.

Website defacement

We've all seen the news stories: "Large Company X was yesterday hacked and their homepage was replaced with an obscene message". This sort of thing is an everyday occurrence on the Internet.

After the company SCO initiated a lawsuit against Linux vendors citing copyright violations in the Linux source code, the SCO corporate website was hacked and an image was altered to read WE OWN ALL YOUR CODE—pay us all your money. The hack was subtle enough that the casual visitor to the SCO site would likely not be able to tell that this was not the official version of the homepage:



The above image shows what the SCO homepage looked like after being defaced—quite subtle, don't you think?

Preventing website defacement is important for a business for several reasons:

Potential customers will turn away when they see the hacked site

There will be an obvious loss of revenue if the site is used for any sort of e-commerce sales

Bad publicity will tarnish the company's reputation

Defacement of a site will of course depend on a vulnerability being successfully exploited. The measures we will look at here are aimed to detect that a defacement has taken place, so that the real site can be restored as quickly as possible.

Detection of website defacement is usually done by looking for a specific token in the outgoing web pages. This token has been placed within the pages in advance specifically so that it may be used to detect defacement—if the token isn't there then the site has likely been defaced. This can be sufficient, but it can also allow the attacker to insert the same token into his defaced page, defeating the detection mechanism. Therefore, we will go one better and create a defacement detection technology that will be difficult for the hacker to get around.

To create a dynamic token, we will be using the visitor's IP address. The reason we use the IP address instead of the hostname is that a reverse lookup may not always be possible, whereas the IP address will always be available.

The following example code in JSP illustrates how the token is calculated and inserted into the page.

%s",

tokenHashed));

%>

Assuming the background of the page is white, the  markup will ensure it is not visible to website viewers.

Now for the ModSecurity rules to handle the defacement detection. We need to look at outgoing pages and make sure that they include the appropriate token. Since the token will be different for different users, we need to calculate the same MD5 sum token in our ModSecurity rule and make sure that this token is included in the output. If not, we block the page from being sent and sound the alert by sending an email message to the website administrator.

#

# Detect and block outgoing pages not containing our token

#

SecRule REMOTE_ADDR ".*" "phase:4,deny,chain,t:md5,t:hexEncode,

  exec:/usr/bin/emailadmin.sh"

SecRule RESPONSE_BODY "!@contains %{MATCHED_VAR}"

We are placing the rule in phase 4 since this is required when we want to inspect the response body. The exec action is used to send an email to the website administrator to let him know of the website defacement.

ModSecurity 2.5









Prevent web application hacking with this easy to use guide

Secure your system by knowing exactly how a hacker would break into it

Covers writing rules in-depth and Modsecurity rule language elements such as variables, actions, and request phases

Covers the common attacks in use on the Web, and ways to find the geographical location of an attacker and send alert emails when attacks are discovered

Packed with many real-life examples for better understanding http://www.packtpub.com/modsecurity-2-5/book

Brute force attacks

Brute force attacks involve an attacker repeatedly trying to gain access to a resource by guessing usernames, passwords, email addresses, and similar credentials. They can be incredibly effective if no protection is in place, since most users choose passwords that are short and easy to remember. Furthermore, most users will use nearly identical passwords on all websites for which a login is required, and so compromise of one password can lead to the user having his account compromised at a whole range of other sites.

A good way to defend against brute force attacks is to allow a certain number of login attempts, say three, and after that start delaying or blocking further attempts. Let's see how we can use ModSecurity to accomplish this.

If your login verification page is situated at yoursite.com/login, then the following rules will keep track of the number of login attempts by users:

#

# Block further login attempts after 3 failed attempts

#

# Initalize IP collection with user's IP address

SecAction "initcol:ip=%{REMOTE_ADDR},pass,nolog"

# Detect failed login attempts

SecRule RESPONSE_BODY "Username does not exist" "phase:4,pass,setvar:

ip.failed_logins=+1,expirevar:ip.failed_logins=60"

# Block subsequent login attempts

SecRule IP:FAILED_LOGINS "@gt 3" deny

The rules initialize the ip collection and increase the field ip.failed_logins after each failed login attempt. Once more than three failed logins are detected, further attempts are blocked. The expirevar action is used to reset the number of failed login attempts to zero after 60 seconds, so the block will be in effect for a maximum of 60 seconds.

Another approach is to start delaying requests once the threshold number of login attempts has been reached. This has the advantage of not denying access in case a legitimate user has actually forgotten his password and needs more attempts to remember it. Here are the rules to do that:

#

# Throttle login attempts after 3 failed attempts

#

SecAction "initcol:ip=%{REMOTE_ADDR},pass,nolog"

SecRule RESPONSE_BODY "Username does not exist" "phase:4,pass,setvar:

ip.failed_logins=+1,expirevar:ip.failed_logins=10"

SecRule IP:FAILED_LOGINS "@gt 3" "phase:4,allow,pause:3000"

The pause action is what delays the request, and the time specified is in milliseconds, so the above will delay the response for three seconds once the limit of three failed login attempts has been exceeded.

Directory indexing

When a user requests an URL like http://www.example.com/, with no filename specification, Apache will look for the file specified by the DirectoryIndex setting (for example index.html). If this file is found, it is served to the user. If it doesn't exist, what happens next is determined by whether the Apache option called Indexes is enabled or not.

The Indexes option can be enabled for a directory in the following way:

Options +Indexes

If the Indexes option is active then Apache will generate a directory listing and display it to the user if the default DirectoryIndex file is not found. This listing contains the names of all files and sub-directories in the requested directory, and this can be a problem for several reasons:

Files that were never meant to be publicly disclosed can be requested by the user, even if they are not linked from anywhere

Names of subdirectories are displayed, and again this may lead to the user wandering into directories that were never meant for public disclosure

In a perfect world, you would never have files under the web server root that users should not be able to download, and all directories or files requiring authorization should be protected by the appropriate HTTP authentication settings. However, in the real world, files and directories do sometimes end up under the web server root even when they are not meant to be accessible by all users. Therefore it makes sense to turn off directory indexing so that this listing is never generated:

Options -Indexes

Even with this in place, sometimes directory indexing can get turned back on—configuration files get edited or replaced with defaults. One option would be to comment out the line for the mod_autoindex module in the Apache configuration file:

#

# Disable directory indexing

#

# LoadModule autoindex_module modules/mod_autoindex.so

However, even this can fail should the configuration file be returned to its default at some point, or if a web server vulnerability causes the directory index to be returned even though Options -Indexes is set. Consider for example the vulnerability discovered in 2001 that affected Apache version 1.3.20 and earlier, described as follows in the changelog for Apache when the corrected version 1.3.22 was released:

A vulnerability was found when Multiviews are used to negotiate the directory index. In some configurations, requesting a URI with a QUERY_STRING of M=D could return a directory listing rather than the expected index page.

This shows that unexpected circumstances can cause directory indexes to be returned even when the web server administrator does everything correctly. Therefore, in keeping with the Defense in Depth principle, adding a precautionary set of rules to ModSecurity to block any directory index from escaping the web server can be a good idea.

These rules will block the Apache directory index from being returned:

#

# Prevent directory listings from accidentally being returned

#

SecRule REQUEST_URI "/$" "phase:4,deny,chain,log,

  msg:'Directory index returned'"

SecRule RESPONSE_BODY "Index of /"

The above rule chain is placed in phase 4, since we need to examine the response body for the telltale signature Index of /, which is what Apache returns in directory index pages. This string could potentially be contained within regular HTML documents, so we do an additional check in the first rule—the request URI has to end with a forward slash, which it does when the user requests a directory. Even if the user were to request /example, without a trailing slash, the Apache module mod_dir will issue a 301—Moved permanently redirect to /example/ before the directory listing is returned (or not returned, as will be the case with the rule chain above active).

Detecting the real IP address of an attacker

If you're under attack by a sophisticated adversary, he will most likely be hiding behind an anonymizing proxy—sometimes he will even be using multiple chained proxies to avoid detection. The illustration below shows how this works when two proxy servers are involved. The web server will only see the IP address of the last proxy server, and even if the proxy server administrator co-operated to help find an attacker, the logs would only show the IP address of the proxy server before it in the chain.



Wouldn't it be great to be able to get the real IP address of an attacker and have it logged if a severe enough attack is taking place? The real IP address can be what makes or breaks an investigation if an attack ever has to be reported to the police.

The first step in implementing real IP address detection is to realize that ModSecurity's redirect action can be used to redirect to a different page when an attack is detected. We will just be redirecting to a standard 404—Not Found error page.

Now the remaining problem is: What do we put on this modified error page to detect the attacker's IP address? One possible avenue of approach would be to include some JavaScript code in the error page to try to find out his IP address. Unfortunately, it's not possible to detect the IP address of a computer using JavaScript—using the function java.net.InetAddress.getLocalHost() returns localhost on all systems.

However, what if the attacker has Java enabled in his browser? In that case, it is actually possible to use Java to detect the IP address of the attacking computer. We will basically be turning the attacker's own web browser against him by loading a Java applet that will detect his IP address and transmit it to our server. The following diagram illustrates how this works:



Lars Kindermann has a ready-made Java applet to detect IP addresses called "MyAddress" which is available at http://www.reglos.de/myaddress/MyAddress.html. Simply download this to your web server by saving the MyAddress.class file that is linked to on the page.

To get the attacker's IP address we will be using a technique familiar from the section on cross-site scripting attacks—the use of an  tag to transmit data back to our server. In this case, the data we will be transmitting is the attacker's IP address. Once we have the attacker's IP address—say 1.2.3.4—we will include the following  tag on the error page to capture his IP address:

This will cause the attacker's web browser to perform a GET request to the page at www.ourserver.com/log_ip.php, handily providing the IP address in the query string. It is then a simple matter for the script log_ip.php to record the IP address in a database table or a text file.

This is the code that needs to be included on our modified error page in order to retrieve the IP address by invoking the Java applet and then printing out the  tag:

function MyAddress(IP) {

  document.write("");

}

This first line uses an  tag to load the Java applet called MyAddress.class. The subsequent lines execute JavaScript code that does two things:

Retrieves the IP address of the attacker's computer.

Writes an  tag to the web page that references our own server to send the IP address back to us.

You can see that the second step is what makes this similar to cross-site scripting.

This suffers from a small problem—the  tag doesn't actually reference a valid image, which will cause the attacker to see a page with a "broken image" icon. Luckily, this is easily resolved by setting the width and height attributes of the image to zero:

function MyAddress(IP) {

  document.write("");

}

Now, the final piece of the puzzle is just to redirect detected attacks to the log_ip.php page. The following ModSecurity rule illustrates how to do this:

SecRule ARGS "/etc/passwd" "pass,redirect:/log_ip.php"

Though real IP-detection may not be preferable for "everyday" attacks, it can be a handy tool in those cases where finding out the IP of the attacker is essential to prevent further crimes from taking place or assisting the police in an investigation.

Summary

In this article, we looked at different methods of attack currently used against web applications and servers. We learned the anatomy behind attacks such as cross-site scripting, cross-site request forgeries, and SQL injection. We saw how ModSecurity can be used to mitigate or block these attacks, and how ModSecurity can be a vital part of applying the Defense in Depth strategy. In the last sections of the article we learned how to defeat HTTP fingerprinting and how to detect the real IP address of an attacker if he is surfing via a proxy server but has Java enabled in his browser.

[ 1 | 2 | 3 ]

Blocking Common Attacks using ModSecurity 2.5: Part 2

Read Part One of Blocking Common Attacks using ModSecurity 2.5 here.

Cross-site scripting

Cross-site scripting attacks occur when user input is not properly sanitized and ends up in pages sent back to users. This makes it possible for an attacker to include malicious scripts in a page by providing them as input to the page. The scripts will be no different than scripts included in pages by the website creators, and will thus have all the privileges of an ordinary script within the page—such as the ability to read cookie data and session IDs. In this article we will look in more detail on how to prevent attacks.

The name "cross-site scripting" is actually rather poorly chosen—the name stems from the first such vulnerability that was discovered, which involved a malicious website using HTML framesets to load an external site inside a frame. The malicious site could then manipulate the loaded external site in various ways—for example, read form data, modify the site, and basically perform any scripting action that a script within the site itself could perform. Thus cross-site scripting, or XSS, was the name given to this kind of attack.

The attacks described as XSS attacks have since shifted from malicious frame injection (a problem that was quickly patched by web browser developers) to the class of attacks that we see today involving unsanitized user input. The actual vulnerability referred to today might be better described as a "malicious script injection attack", though that doesn't give it quite as flashy an acronym as XSS. (And in case you're curious why the acronym is XSS and not CSS, the simple explanation is that although CSS was used as short for cross-site scripting in the beginning, it was changed to XSS because so many people were confusing it with the acronym used for Cascading Style Sheets, which is also CSS.)

Cross-site scripting attacks can lead not only to cookie and session data being stolen, but also to malware being downloaded and executed and injection of arbitrary content into web pages.

Cross-site scripting attacks can generally be divided into two categories:

Reflected attacks

This kind of attack exploits cases where the web application takes data provided by the user and includes it without sanitization in output pages. The attack is called "reflected" because an attacker causes a user to provide a malicious script to a server in a request that is then reflected back to the user in returned pages, causing the script to execute.

Stored attacks

In this type of XSS attack, the attacker is able to include his malicious payload into data that is permanently stored on the server and will be included without any HTML entity encoding to subsequent visitors to a page. Examples include storing malicious scripts in forum posts or user presentation pages. This type of XSS attack has the potential to be more damaging since it can affect every user who views a certain page.

Preventing XSS attacks

The most important measure you can take to prevent XSS attacks is to make sure that all user-supplied data that is output in your web pages is properly sanitized. This means replacing potentially unsafe characters, such as angled brackets () with their corresponding HTML-entity encoded versions—in this case  and >.

Here is a list of characters that you should encode when present in user-supplied data that will later be included in web pages:

Character

HTML-encoded version



>

>

(

(

)

)

#

#

&

&

"

"

'

'

In PHP, you can use the htmlentities() function to achieve this. When encoded, the string  will be converted into . This latter version will be displayed as  in the web browser, without being interpreted as the start of a script by the browser.

In general, users should not be allowed to input any HTML markup tags if it can be avoided. If you do allow markup such as  to be input by users in blog comments, forum posts, and similar places then you should be aware that simply filtering out the  tag is not enough, as this simple example shows:

Innocent link

This link will execute the JavaScript code contained within the onMouseOver attribute whenever the user hovers his mouse pointer over the link. You can see why even if the web application replaced  tags with their HTML-encoded version, an XSS exploit would still be possible by simply using onMouseOver or any of the other related events available, such as onClick or onMouseDown.

I want to stress that properly sanitizing user input as just described is the most important step you can take to prevent XSS exploits from occurring. That said, if you want to add an additional line of defense by creating ModSecurity rules, here are some common XSS script fragments and regular expressions for blocking them:

Script fragment

Regular expression

eval(

evals*(

onMouseOver

onmouseover

onMouseOut

onmouseout

onMouseDown

onmousedown

onMouseMove

onmousemove

onClick

onclick

onDblClick

ondblclick

onFocus

onfocus

PDF XSS protection

You may have seen the ModSecurity directive SecPdfProtect mentioned, and wondered what it does. This directive exists to protect users from a particular class of cross-site scripting attack that affects users running a vulnerable version of the Adobe Acrobat PDF reader.

A little background is required in order to understand what SecPdfProtect does and why it is necessary. In 2007, Stefano Di Paola and Giorgio Fedon discovered a vulnerability in Adobe Acrobat that allows attackers to insert JavaScript into requests, which is then executed by Acrobat in the context of the site hosting the PDF file. Sound confusing? Hang on, it will become clearer in a moment.

The vulnerability was quickly fixed by Adobe in version 7.0.9 of Acrobat. However, there are still many users out there running old versions of the reader, which is why preventing this sort of attack is still an ongoing concern.

The basic attack works like this: An attacker entices the victim to click a link to a PDF file hosted on www.example.com. Nothing unusual so far, except for the fact that the link looks like this:

http://www.example.com/document.pdf#x=javascript:alert('XSS');

Surprisingly, vulnerable versions of Adobe Acrobat will execute the JavaScript in the above link. It doesn't even matter what you place before the equal sign, gibberish= will work just as well as x= in triggering the exploit.

Since the PDF file is hosted on the domain www.example.com, the JavaScript will run as if it was a legitimate piece of script within a page on that domain. This can lead to all of the standard cross-site scripting attacks that we have seen examples of before.

This diagram shows the chain of events that allows this exploit to function:



The vulnerability does not exist if a user downloads the PDF file and then opens it from his local hard drive.

ModSecurity solves the problem of this vulnerability by issuing a redirect for all PDF files. The aim is to convert any URLs like the following:

http://www.example.com/document.pdf#x=javascript:alert('XSS');

into a redirected URL that has its own hash character:

http://www.example.com/document.pdf#protection

This will block any attacks attempting to exploit this vulnerability. The only problem with this approach is that it will generate an endless loop of redirects, as ModSecurity has no way of knowing what is the first request for the PDF file, and what is a request that has already been redirected. ModSecurity therefore uses a one-time token to keep track of redirect requests. All redirected requests get a token included in the new request string. The redirect link now looks like this:

http://www.example.com/document.pdf?PDFTOKEN=XXXXX#protection

ModSecurity keeps track of these tokens so that it knows which links are valid and should lead to the PDF file being served. Even if a token is not valid, the PDF file will still be available to the user, he will just have to download it to the hard drive.

These are the directives used to configure PDF XSS protection in ModSecurity:

SecPdfProtect On

SecPdfProtectMethod TokenRedirection

SecPdfProtectSecret "SecretString"

SecPdfProtectTimeout 10

SecPdfProtectTokenName "token"

The above configures PDF XSS protection, and uses the secret string SecretString to generate the one-time tokens. The last directive, SecPdfProtectTokenName, can be used to change the name of the token argument (the default is PDFTOKEN). This can be useful if you want to hide the fact that you are running ModSecurity, but unless you are really paranoid it won't be necessary to change this.

The SecPdfProtectMethod can also be set to ForcedDownload, which will force users to download the PDF files instead of viewing them in the browser. This can be an inconvenience to users, so you would probably not want to enable this unless circumstances warrant (for example, if a new PDF vulnerability of the same class is discovered in the future).

HttpOnly cookies to prevent XSS attacks

One mechanism to mitigate the impact of XSS vulnerabilities is the HttpOnly flag for cookies. This extension to the cookie protocol was proposed by Microsoft (see http://msdn.microsoft.com/en-us/library/ms533046.aspx for a description), and is currently supported by the following browsers:

Internet Explorer (IE6 SP1 and later)

Firefox (2.0.0.5 and later)

Google Chrome (all versions)

Safari (3.0 and later)

Opera (version 9.50 and later)

HttpOnly cookies work by adding the HttpOnly flag to cookies that are returned by the server, which instructs the web browser that the cookie should only be used when sending HTTP requests to the server and should not be made available to client-side scripts via for example the document.cookie property. While this doesn't completely solve the problem of XSS attacks, it does mitigate those attacks where the aim is to steal valuable information from the user's cookies, such as for example session IDs.

A cookie header with the HttpOnly flag set looks like this:

Set-Cookie: SESSID=d31cd4f599c4b0fa4158c6fb; HttpOnly

HttpOnly cookies need to be supported on the server-side for the clients to be able to take advantage of the extra protection afforded by them. Some web development platforms currently support HttpOnly cookies through the use of the appropriate configuration option. For example, PHP 5.2.0 and later allow HttpOnly cookies to be enabled for a page by using the following ini_set() call:

Tomcat (a Java Servlet and JSP server) version 6.0.19 and later supports HttpOnly cookies, and they can be enabled by modifying a context's configuration so that it includes the useHttpOnly option, like so:

  

In case you are using a web platform that doesn't support HttpOnly cookies, it is actually possible to use ModSecurity to add the flag to outgoing cookies. We will see how to do this now.

Session identifiers

Assuming we want to add the HttpOnly flag to session identifier cookies, we need to know which cookies are associated with session identifiers. The following table lists the name of the session identifier cookie for some of the most common languages:

Language

Session identifier cookie name

PHP

PHPSESSID

JSP

JSESSIONID

ASP

ASPSESSIONID

ASP.NET

ASP.NET_SessionId

The table shows us that a good regular expression to identify session IDs would be (sessionid|sessid), which can be shortened to sess(ion)?id. The web programming language you are using might use another name for the session cookie. In that case, you can always find out what it is by looking at the headers returned by the server:

echo -e "GET / HTTP/1.1nHost:yourserver.comnn"|nc yourserver.com

80|head

Look for a line similar to:

Set-Cookie: JSESSIONID=4EFA463BFB5508FFA0A3790303DE0EA5; Path=/

This is the session cookie—in this case the name of it is JESSIONID, since the server is running Tomcat and the JSP web application language.

The following rules are used to add the HttpOnly flag to session cookies:

#

# Add HttpOnly flag to session cookies

#

SecRule RESPONSE_HEADERS:Set-Cookie "!(?i:HttpOnly)"

"phase:3,chain,pass"

SecRule MATCHED_VAR "(?i:sess(ion)?id)" "setenv:session_

cookie=%{MATCHED_VAR}"

Header set Set-Cookie "%{SESSION_COOKIE}e; HttpOnly" env=session_

cookie

We are putting the rule chain in phase 3—RESPONSE_HEADERS, since we want to inspect the response headers for the presence of a Set-Cookie header. We are looking for those Set-Cookie headers that do not contain an HttpOnly flag. The (?i: ) parentheses are a regular expression construct known as a mode-modified span. This tells the regular expression engine to ignore the case of the HttpOnly string when attempting to match. Using the t:lowercase transform would have been more complicated, as we will be using the matched variable in the next rule, and we don't want the case of the variable modified when we set the environment variable.

ModSecurity 2.5









Prevent web application hacking with this easy to use guide

# Secure your system by knowing exactly how a hacker would break into it

Covers writing rules in-depth and Modsecurity rule language elements such as variables, actions, and request phases

Covers the common attacks in use on the Web, and ways to find the geographical location of an attacker and send alert emails when attacks are discovered

Packed with many real-life examples for better understanding http://www.packtpub.com/modsecurity-2-5/book

If a cookie header without the HttpOnly flag is found, the second rule looks to see if it is a session identifier cookie. If it is, the setenv action is used to set the environment variable %{SESSION_COOKIE}. ModSecurity cannot be used to modify the cookie header directly (ModSecurity content injection can only prepend data to the beginning of the response or append it to the end of the response), so we are using a plain Apache directive—the Header directive—to modify the cookie header:

Header set Set-Cookie "%{session_cookie}e; HttpOnly" env=session_

cookie

Header directives can use the env= syntax, which means that they will only be invoked if the named environment variable is set. In this case, the Header directive will only be invoked if the %{SESSION_COOKIE} environment variable was set by the ModSecurity rule chain. When invoked, the header directive sets the Set-Cookie header to its previous value (%{SESSION_COOKIE}e is what does this—the e at the end is used to identify this as an environment variable). The string ; HttpOnly is then appended to the end of the previous header.

If we now look at the HTTP headers returned by the server, the session ID cookie will have the HttpOnly flag set:

$ echo -e "GET / HTTP/1.0nn" | nc localhost 80 | head

...

Set-Cookie: JSESSIONID=4EFA463BFB5508FFA0A3790303DE0EA5; Path=/;

HttpOnly

Cleaning XSS Code from Databases

Scrubbr is the name of a tool for cleaning databases of stored XSS attacks that is made available at no charge by the Open Web Application Security Project (OWASP). Scrubbr works by examining database tables for stored malicious scripts.

The developers have this to say about how the tool works: If you can tell Scrubbr how to access your database, it will search through every field capable of holding strings in the database for malicious code. If you want it to, it will search through every table, every row, and every column.

Scrubbr can be downloaded at http://code.google.com/p/owaspscrubbr/, and more information on the tool is available on the OWASP homepage at http://www.owasp.org/index.php/Category:OWASP_Scrubbr.

Cross-site request forgeries

Cross-site request forgeries (CSRF) are attacks that trick the victim's browser into submitting a request to another site where the user is logged in, causing that site to believe the user has initiated an action, and that action is then executed as if the user had initiated it. In other words, cross-site request forgeries execute some action on a site that the user never intended.

One example would be if while you are logged into your bank's online banking site someone got you to visit a page that contained the following  tag:

As we already know that an  tag can be used to execute GET requests, this would cause money to be transferred from one account to another assuming the banking site can do this via GET requests. This is the essence of CSRF attacks—to embed code into a page that causes an action to be executed without the user's knowledge. The aim can be to transfer money, get the user to buy things at auction sites, make him send messages to other users on a site, or any number of things to make it look like the logged-in user on a site has performed some action which was in reality initiated by the CSRF code.

To get a clearer picture, imagine this scenario:

You do your online banking with Acme Bank

Acme Bank's website is vulnerable to CSRF attacks

You also regularly visit the gardening forum at gardening.com

Now suppose your long-time enemy Ned is aware of your browsing habits. Since he's got an axe to grind he hatches a scheme to transfer $10,000 from your personal savings account to his own account. Since Ned knows that you use Acme bank and are also a regular visitor at gardening.com, he starts a topic at the gardening forum with the title "Wild fuchsias for sale", knowing you are a fan of fuchsias and have been looking for quality specimens for some time.

If you take the bait and click on the topic in the forum, Ned's evil HTML tag will get downloaded by your browser:

If you are logged into your banking site at the time your browser attempts to render the forum topic, your well-meaning browser will attempt to fetch the image located at bank.acme.com/transfer.php, passing the entire query string along with it. Unbeknownst to you, you have just transferred enough money to buy a small car to Ned.

Protecting against cross-site request forgeries

Protecting against CSRF attacks can be challenging. Superficially, it might look like only GET requests are vulnerable, since that is what the browser uses in our examples with the malicious  tags. However, that is not true as with the right script code it is possible for a client-side script to perform POST requests. The following code uses Ajax technology to do just that:

var post_data = 'name=value';

var xmlhttp=new ActiveXObject("Microsoft.XMLHTTP");

xmlhttp.open("POST", 'http://url/path/file.ext', true);

xmlhttp.onreadystatechange = function () {

  if (xmlhttp.readyState == 4) {

    alert(xmlhttp.responseText);

  }

};

xmlhttp.send(post_data);

The core of the problem is that the requests come from the user's own browser and look like legitimate requests. The mainstream solutions today revolve around giving the user's browser some piece of information that it must then transmit back when performing an action. Examples include:

Generating a token that is sent together with forms to the user. Any action taken must then include this token or it will be rejected.

Randomizing page names. This gives a user unique URLs to perform actions, and should preferably be changed for each new user session. This makes it difficult for the attacker to know where to submit the requests.

Requiring authentication to perform important actions. Usually this is done by requesting the username and password to be entered, but for high-security sites such as banking sites this can also involve the user using a small hardware device to generate an authorization code that is submitted to the server.

Shell command execution attempts

As we have already seen, accepting unfiltered input from users can be dangerous. A particular class of exploit occurs when data submitted by users is used to cause the execution or display of a file which the user normally wouldn't have privileges to.

Attackers often combine multiple vulnerabilities to achieve maximum effect. Shell command execution is one exploit scenario which usually doesn't happen on its own—after all, very few web applications take user input and perform the exec() system call on them. However, consider the following chain of events:



In this chain of event, we can see how two vulnerabilities were combined to deadly effect:

The SQL injection vulnerability was used to create a PHP file

The failure to filter out shell command execution attempts allowed the attacker to call the exec.php script to remove all files on the web server

This shows that trying to prevent shell command execution is worthwhile (and once again reaffirms the principle of Defense in Depth). I say "trying" since there will always be ways to write system commands that circumvent any detection patterns, however some protection is better than none.

The following are some common Linux system commands, program names, and paths that you may wish to block:

rm

ls

kill

mail

sendmail

cat

echo

/bin/

/etc/

/tmp/

The following rule will block the above when present in arguments:

SecRule ARGS "(rm|ls|kill|(send)?mail|cat|echo|/bin/|/etc/|/tmp/)"

"deny"

Null byte attacks

Null byte attacks exploit the fact that the C programming language (and related languages) use a null byte (0x00) to signify the end of a string. The string dog, for example, is stored in memory in the following way when the C programming language is used:



In other programming languages, such as Java, strings are stored as arrays, and the total length of the string is stored in a separate location, which means that a Java string is perfectly capable of containing a null byte in the middle of the string.

This difference in how strings and null bytes are handled by different programming languages enable some attacks to take place that exploit the null byte to fool one part of a system by making it think a string has ended at a null byte, while another part will happily process the full input string

Consider a simple JSP page that displays a text file to a visitor by using the filename parameter supplied to the page:

The page attempts to ensure that only files with the extension .txt can be displayed to the visitor. However, if an attacker supplies a filename argument of /etc/passwd�.txt, then a null byte attack is possible. Since Java strings can contain null bytes, the filename will pass the check filename.endsWith(".txt"). When the filename string is passed to an underlying operating system function to open the file, a problem will arise if that system function treats the string as null-terminated since anything after the null byte will be ignored. The operating system will end up opening the file /etc/passwd instead, and this file will then be displayed to the attacker.

ModSecurity and null bytes

ModSecurity contains two transformation functions to deal with null bytes in input: replaceNulls and removeNulls. The first function replaces null bytes with whitespace, while the second one removes null bytes completely. Since null bytes are very rarely needed for valid input, it is a good idea to include one of these transformation functions in the SecDefaultAction list:

SecDefaultAction "phase:2,deny,log,status:403,t:removeNulls"

Should a null byte ever be required in input, then the transformation function can be overridden using the t:-removeNulls syntax:

SecRule ARGS:data "pass,t:-removeNulls"

Null byte attacks are a perfect example of how fragile web applications can be since they are glued together using many different programming languages, and how subtle the attacks can be—who would have expected that the differences in string handling between Java and the operating system could lead to problems like this? It is something that could be easily missed even during a code review.

>> Continue Reading Blocking Common Attacks using ModSecurity 2.5: Part 3

[ 1 | 2 | 3 ]

REFERENCES
http://www.packtpub.com/article/blocking-common-attacks-using-modsecurity-2.5-part1?utm_source=js_modsecurity_abr3_1109&utm_medium=content&utm_campaign=janice

The DHCP Relay Agent allows you to place DHCP clients and DHCP servers on different networks. Deploying DHCP in a single segment network is easy. All DHCP messages are IP broadcast messages, and therefore all the computers on the segment can listen and respond to these broadcasts. A single scope on a solitary DHCP server is all that is required.

DHCP Messages on a Multisegment Network

* Post a comment
* Email Article
* Print Article
* Share Articles
o Digg
o DZone
o Reddit
o Slashdot
o StumbleUpon
o del.icio.us
o Facebook
o FriendFeed
o Furl

Things get a little more complicated when you have more than one subnet on your network. This is because the DHCP broadcast messages do not, by default, cross the router interfaces.

There are several ways you can handle this situation. One alternative is to put a DHCP server on each segment. If you have just a few segments, you might consider this option. However, when you have a large organization, placing a DHCP server on each segment can increase cost and add to administrative effort.

Relay To The Rescue

A better option is to use fewer DHCP servers and place these machines in central locations. To solve the problem of DHCP broadcast messages, routers can be configured to pass DHCP/BOOTP messages selectively. This is referred to as BOOTP Relay.

The concept of BOOTP Relay can get a little confusing. It gets even more confusing when the term BOOTP Forwarding is used. This is because the concepts for Relay and Forwarding are quite different. Forwarding implies that the message is forwarded from one interface to another, without any special processing. Relay implies that the message is processed to a greater extent, which may include changes being made to the original packet.

An Agent is a Relay and Not a Forwarder

RFC 1542 elaborates on BOOTP Relay and BOOTP Relay Agents. A router that can perform BOOTP Relay does more than just forward BOOTP broadcast messages. Rather, the BOOTP Relay Agent actually examines the packet and makes appropriate changes to it, and then Relays the packet to a DHCP Server. The DHCP server to which the packet is Relayed is configured by adding a Helper Address on the router.

Some routers do not support BOOTP Relay. In this case, you can set up a computer on segments that have DHCP clients that will take the place of the BOOTP Relay Agent on the router. You can configure a Windows 2000 RRAS Server to Relay DHCP messages by installing the DHCP Relay Agent.

The Relay Agent communicates with a DHCP server and acts as a proxy for DHCP broadcast messages that must be routed to remote segments. Like the router-based BOOTP Relay Agent, the DHCP Relay Agent is configured with addresses of DHCP servers to which they should Relay the DHCP Message.

Be Careful of the Terminology

Be sure you recognize the difference between a DHCP/BOOTP Relay Agent and a Router that acts as a BOOTP forwarder. Remember that forwarders will propagate a broadcast message across the router interfaces, while the DHCP Relay Agent intercepts the broadcast, makes changes to the DHCP Message, and forwards its via a directed datagram to a specific DHCP server.

Reference: http://www.serverwatch.com/tutorials/article.php/2193031/Back-to-Basics-The-DHCP-Relay-Agent.htm

Credits to Christian Busch for helping me out on this one. :)

Late last night, I was approached with an interesting case of a denial of service attack. And I mean it – no one was able to access the service – because all files were deleted. The client informed me that remote shell scripts were able to find their way onto the server.

The first step was obviously to restore the files on the server, which was trivial given that WHM made daily backups for him. Then the search for the shell scripts began. I identified a couple of them –

File found - backup.php (2463) - 03/06/2009
File found - shelld.php (23124) - 03/08/2009
File found - .php (96420) - 03/07/2009
File found - testa.php (85) - 03/08/2009
File found - test.php (38868) - 02/05/2009
File found - c.php (597944) - 02/25/2009

The files are zipped and downloadable. Be sure to manage your AV exclusions. :)

The file that requires some attention is testa.php.

testa.php

$o = fopen("shelld.php","wb");fwrite($o,$aa);fclose($o); ?>

I shall cut the crap and go straight to the logs. You may download a copy of all the logs here. But let’s take a look at a sample line –

92.241.137.238 - - [07/Mar/2009:23:36:02 +0000]
"GET /?p=../../../../../../../../../../../../../proc/self/environ HTTP/1.1"
200 15491 "-"
""

A directory traversal attack! I haven’t seen one like that in years! But this one is smart. For those who do not understand what that line in the logs mean - a user using the IP address 92.241.137.238 visited the page

/?p=../../../../../../../../../../../../../proc/self/environ

with the browser headers as PHP code.

Let’s take a look at the output of the GET URL –
Smart, eh?

Smart, eh? Click to enlarge.

It seems like the browser and PHP information are output.

Looking at some relevant PHP code –

$p = stripslashes($_REQUEST['p']);
if(file_exists(CWD.'/'.$p.'.php')){
include(CWD.'/'.$p.'.php');
}else{
include(CWD.'/'.$_SESSION['lan'].'/'.$p.'.php');
}?>

“This” certainly needs to be changed. ;)

Let’s look at the query executed again. The null byte is used to ignore the “.php” extension and only include /proc/self/environ

What’s even smarter is this – because browser information is include’d, PHP code was used as the browser headers and was therefore included and parsed.

If you look further down the logs, you will realize that the C99 shell is also included from a certain URL –

Shells. Sea Shells. Remote Shells. Evil Shells.

Shells. Sea Shells. Remote Shells. Evil Shells.

Looks like someone set up a site just for this purpose.

There are other things probably worth mentioning, but I guess not this time. The things to take away would be –

1. Always sanitize parameters passed to functions like require’s, require_once’s, include’s, include_once’s, fopen’s, file_get_contents’, etc to prevent directory traversal
2. Using /proc/self/environ as an attack vector to execute PHP code remotely that is hardly ever published. I hope that this article provided you with some insights.
3. And to all the security researchers out there – You can now stop using the cat /etc/passwd POC and start using something like this. =)

Reference: http://security.searix.net/?p=92

At last! After many months of collecting data, crafting directives, and testing results, I am thrilled to announce the release of the 4G Blacklist! The 4G Blacklist is a next-generation protective firewall that secures your website against a wide range of malicious activity. Like its 3G predecessor, the 4G Blacklist is designed for use on Apache servers and is easily implemented via HTAccess or the httpd.conf configuration file. In order to function properly, the 4G Blacklist requires two specific Apache modules, mod_rewrite and mod_alias. As with the third generation of the blacklist, the 4G Blacklist consists of multiple parts:

* HTAccess Essentials
* Request-Method Filtering
* IP Address Blacklist
* Query-String Blacklist
* URL Blacklist

Each of these methods is designed to protect different aspects of your site. They may be used independently, mixed and matched, or combined to create the complete 4G Blacklist. This modularity provides flexibility for different implementations while facilitating the testing and updating process. The core of the 4G Blacklist consists of the last two methods, the Query-String and URL Blacklists. These two sections provide an enormous amount of protection against many potentially devastating attacks. Everything else is just icing on the cake. Speaking of which, there are also two more completely optional sections of the 4G Blacklist, namely:

* The Ultimate User-Agent Blacklist
* The Ultimate Referrer Blacklist

These two sections have been removed from the 4G Blacklist and relegated to “optional” status because they are no longer necessary. Put simply, the 4G Blacklist provides better protection with fewer lines of code. Even so, each of these blacklists have been updated with hundreds of new directives and will be made available here at Perishable Press in the near future. But for now, let’s return to the business at hand..
Presenting the Perishable Press 4G Blacklist

As is custom here at Perishable Press, I present the complete code first, and then walk through the usage instructions and code explanations. So, without furhter ado, here is the much-anticipated 4G Blacklist [for personal use only - may not be posted elsewhere without proper link attribution]:

### PERISHABLE PRESS 4G BLACKLIST ###

# ESSENTIALS
RewriteEngine on
ServerSignature Off
Options All -Indexes
Options +FollowSymLinks

# FILTER REQUEST METHODS

RewriteCond %{REQUEST_METHOD} ^(TRACE|DELETE|TRACK) [NC]
RewriteRule ^(.*)$ - [F,L]


# BLACKLIST CANDIDATES

Order Allow,Deny
Allow from all
Deny from 75.126.85.215 "# blacklist candidate 2008-01-02 = admin-ajax.php attack "
Deny from 128.111.48.138 "# blacklist candidate 2008-02-10 = cryptic character strings "
Deny from 87.248.163.54 "# blacklist candidate 2008-03-09 = block administrative attacks "
Deny from 84.122.143.99 "# blacklist candidate 2008-04-27 = block clam store loser "
Deny from 210.210.119.145 "# blacklist candidate 2008-05-31 = block _vpi.xml attacks "
Deny from 66.74.199.125 "# blacklist candidate 2008-10-19 = block mindless spider running "
Deny from 203.55.231.100 "# 1048 attacks in 60 minutes"
Deny from 24.19.202.10 "# 1629 attacks in 90 minutes"


# QUERY STRING EXPLOITS

RewriteCond %{QUERY_STRING} \.\.\/ [NC,OR]
RewriteCond %{QUERY_STRING} boot\.ini [NC,OR]
RewriteCond %{QUERY_STRING} tag\= [NC,OR]
RewriteCond %{QUERY_STRING} ftp\: [NC,OR]
RewriteCond %{QUERY_STRING} http\: [NC,OR]
RewriteCond %{QUERY_STRING} https\: [NC,OR]
RewriteCond %{QUERY_STRING} mosConfig [NC,OR]
RewriteCond %{QUERY_STRING} ^.*(\[|\]|\(|\)|<|>|'|"|;|\?|\*).* [NC,OR]
RewriteCond %{QUERY_STRING} ^.*(%22|%27|%3C|%3E|%5C|%7B|%7C).* [NC,OR]
RewriteCond %{QUERY_STRING} ^.*(%0|%A|%B|%C|%D|%E|%F|127\.0).* [NC,OR]
RewriteCond %{QUERY_STRING} ^.*(globals|encode|config|localhost|loopback).* [NC,OR]
RewriteCond %{QUERY_STRING} ^.*(request|select|insert|union|declare|drop).* [NC]
RewriteRule ^(.*)$ - [F,L]


# CHARACTER STRINGS

# BASIC CHARACTERS
RedirectMatch 403 \,
RedirectMatch 403 \:
RedirectMatch 403 \;
RedirectMatch 403 \=
RedirectMatch 403 \@
RedirectMatch 403 \[
RedirectMatch 403 \]
RedirectMatch 403 \^
RedirectMatch 403 \`
RedirectMatch 403 \{
RedirectMatch 403 \}
RedirectMatch 403 \~
RedirectMatch 403 \"
RedirectMatch 403 \$
RedirectMatch 403 \<
RedirectMatch 403 \>
RedirectMatch 403 \|
RedirectMatch 403 \.\.
RedirectMatch 403 \/\/
RedirectMatch 403 \%0
RedirectMatch 403 \%A
RedirectMatch 403 \%B
RedirectMatch 403 \%C
RedirectMatch 403 \%D
RedirectMatch 403 \%E
RedirectMatch 403 \%F
RedirectMatch 403 \%22
RedirectMatch 403 \%27
RedirectMatch 403 \%28
RedirectMatch 403 \%29
RedirectMatch 403 \%3C
RedirectMatch 403 \%3E
RedirectMatch 403 \%3F
RedirectMatch 403 \%5B
RedirectMatch 403 \%5C
RedirectMatch 403 \%5D
RedirectMatch 403 \%7B
RedirectMatch 403 \%7C
RedirectMatch 403 \%7D
# COMMON PATTERNS
Redirectmatch 403 \_vpi
RedirectMatch 403 \.inc
Redirectmatch 403 xAou6
Redirectmatch 403 db\_name
Redirectmatch 403 select\(
Redirectmatch 403 convert\(
Redirectmatch 403 \/query\/
RedirectMatch 403 ImpEvData
Redirectmatch 403 \.XMLHTTP
Redirectmatch 403 proxydeny
RedirectMatch 403 function\.
Redirectmatch 403 remoteFile
Redirectmatch 403 servername
Redirectmatch 403 \&rptmode\=
Redirectmatch 403 sys\_cpanel
RedirectMatch 403 db\_connect
RedirectMatch 403 doeditconfig
RedirectMatch 403 check\_proxy
Redirectmatch 403 system\_user
Redirectmatch 403 \/\(null\)\/
Redirectmatch 403 clientrequest
Redirectmatch 403 option\_value
RedirectMatch 403 ref\.outcontrol
# SPECIFIC EXPLOITS
RedirectMatch 403 errors\.
RedirectMatch 403 config\.
RedirectMatch 403 include\.
RedirectMatch 403 display\.
RedirectMatch 403 register\.
Redirectmatch 403 password\.
RedirectMatch 403 maincore\.
RedirectMatch 403 authorize\.
Redirectmatch 403 macromates\.
RedirectMatch 403 head\_auth\.
RedirectMatch 403 submit\_links\.
RedirectMatch 403 change\_action\.
Redirectmatch 403 com\_facileforms\/
RedirectMatch 403 admin\_db\_utilities\.
RedirectMatch 403 admin\.webring\.docs\.
Redirectmatch 403 Table\/Latest\/index\.


That’s the juice right there. This 4G Blacklist is some powerful stuff, blocking and filtering a wide range of potential attacks and eliminating tons of malicious nonsense. Much care has been taken to beta test this firewall on multiple configurations running various types of software, however, due to my limited financial resources, it is impossible to test the 4G as comprehensively as I would have preferred. Even so, for the average site running typical software, everything should continue to work perfectly. With that in mind, please read through the remainder of the article before implementing the 4G Blacklist.
Installation and Usage

Before implementing the 4G Blacklist, ensure that you are equipped with the following system requirements:

* Linux server running Apache
* Enabled Apache module: mod_alias
* Enabled Apache module: mod_rewrite
* Ability to edit your site”s root htaccess file (or)
* Ability to modify Apache’s server configuration file

With these requirements met, copy and paste the entire 4G Blacklist into either the root HTAccess file or the server configuration file ( httpd.conf ). After uploading, visit your site and check proper loading of as many different types of pages as possible. For example, if you are running a blogging platform (such as WordPress), test different page views (single, archive, category, home, etc.), log into and surf the admin pages (plugins, themes, options, posts, etc.), and also check peripheral elements such as individual images, available downloads, and alternate protocols (FTP, HTTPS, etc.).

While the 4G Blacklist is designed to target only the bad guys, the regular expressions used in the list may interfere with legitimate URL or file access. If the directives in the blacklist are blocking a specific URL, the browsing device will display a 403 Forbidden error; similarily, if the blacklist happens to block a file or resource required for some script to function properly, the script (JavaScript, PHP, etc.) may simply stop working. If you experience either of these scenarios after installing the blacklist, don’t panic! Simply check the blocked URL or file, locate the matching blacklist string, and disable the directive by placing a pound sign ( # ) at the beginning of the associated line. Once the correct line is commented out, the blocked URL should load normally. Also, if you do happen to experience any conflicts involving the 4G Blacklist, please leave a comment or contact me directly.
Set for Stun

As my readers know, I am serious about site security. Nothing gets my juices flowing like the thought of chopping up mindless cracker whores into small, square chunks and feeding their still-twitching flesh to a pack of starving mongrels. That’s good times, but unfortunately there are probably laws against stuff like that. So in the meantime, we take steps to secure our sites using the most effective tools at our disposal. There is no one single magic bullet that will keep the unscrupulous bastards from exploiting and damaging your site, but there are many cumulative steps that may be taken to form a solid security strategy. Within this cumulative context, the 4G Blacklist recognizes and immunizes against a broad array of common attack elements, thereby maximizing resources while providing solid defense against malicious attacks.
Many Thanks

A huge “Thank You” to the dedicated people who helped beta test the 4G Blacklist. Your excellent feedback played an instrumental role in the development of this version. Thank you!
Further Reading

For more insight into the mysterious realms of blacklisting, the creation of the Perishable Press Blacklist, and DIY site security in general, check out some of my other articles:

* Eight Ways to Blacklist with Apache’s mod_rewrite
* Blacklist Candidate Series Summary
* How to Block Proxy Servers via htaccess
* 2G Blacklist: Closing the Door on Malicious Attacks
* Series Summary: Building the 3G Blacklist
* Building the Perishable Press 4G Blacklist

Next Up

Next up in the March 2009 Blacklist Series: The Ultimate User-Agent Blacklist. Don’t miss it!
Updates

Since the release of the 4G Blacklist, several users have discovered issues with the following 4G directives:

Joomla
In the query-string section, Joomla users should delete the following patterns:

request
config
[
]

In the character-string section, Joomla users should comment-out or delete the following lines:

RedirectMatch 403 \,
RedirectMatch 403 \;
RedirectMatch 403 config\.
RedirectMatch 403 register\.

WordPress
In the query-string section of the 4G Blacklist, the following changes have been made:

"%3D" character-string has been changed to "%5C"

Likewise, in the character-string section, the following change has been made:

"wp\_" character-string has been removed

And in the request-method filtering section, the following change has been made:

"HEAD" method has been removed

Also, the following changes may be necessary according to which plugins you have installed:

Ozh' Admin Drop Down Menu - remove "drop" from the query-string rules
WordPress' Akismet - remove "config" from the query-string rules

OpenID
OpenID users should read the information in this comment.

SMF
SMF users should read the information in this comment.

vBulletin
vBulletin users should read the information in these comments.
Copyright2009PerishablePress
Short URL:
78524 visits

.Post #677 categorized as Websites, last updated on Jul 10, 2009
Tagged with apache, blacklist, htaccess, mod_rewrite, security, tips
Share this article

* Bookmark at Delicious
* Tweet this!
* Stumble it
* Digg this post
* Add Perishable Press to Technorati
* Share on Facebook
* More..

Related articles

* Perishable Press 3G Blacklist
* Series Summary: Building the 3G Blacklist
* Building the Perishable Press 4G Blacklist
* Blacklist Candidate Series Summary
* Building the 3G Blacklist, Part 4: Improving the RedirectMatch Directives of the Original 2G Blacklist
* Building the 3G Blacklist, Part 2: Improving Site Security by Preventing Malicious Query-String Exploits
* Building the 3G Blacklist, Part 5: Improving Site Security by Selectively Blocking Individual IPs

153 Responses

Add a comment
[ Gravatar Icon ]

Karthik Viswanathan – #1

This is a comprehensive yet compact way to deal with malicious attacks. I’m truly impressed with the effort you’ve put into this. Unfortunately, this line poses a small problem to my website:

RedirectMatch 403 \/\/

How important would you consider blocking the // to be? A few WordPress plugins such as the WP-Clickmap require this string to function properly.
[ Gravatar Icon ]

Jeff Starr – #2

The list is cumulative in nature, meaning that each directive blocks a certain array of potential attacks and thus contributes a percentage of the blacklist’s overall effectiveness. Thus, the line may be commented out or removed entirely without significantly impacting the effectiveness of the remaining directives. Please let me know if you have any further questions about this.
[ Gravatar Icon ]

Awesome Blacklist – #3

This is an awesome blacklist.

Very effective :D
[ Gravatar Icon ]

Jonathan Ellse – #4

Works a treat. Very comprehensive customisation methods.

Thanks very much
[ Gravatar Icon ]

Arwin – #5

Great blacklist and clear story, fun to read!

I had one problem,
I have to remove ‘config’ from the ‘QUERY STRING EXPLOITS’ because otherwise my Mambo and Joomla site will not work properly.
Problem with the configuration of the configuration.php in the administrator backend of Mambo/Joomla (403 Redirect)
Still no other problems found :)

Thank you very much!
[ Gravatar Icon ]

John – #6

You really put a lot of good effort into building this 4G blacklist to share with everyone else. I, as with many other silent supporters, thank you for your efforts. Previously, some of my other sites that I am the head manager of got hacked from time to time, and I’ll be eagerly waiting to see if there are future hacking attempts after I’ve implemented this.

Again, thank you so much for making the world a bit better with your efforts.
[ Gravatar Icon ]

Greg – #7

Just tested, and for the moment have just to lines to be commented out for me.

One is for my downloads links which are unfortunately with some comma so:
#RedirectMatch 403 \,

And my registration page got the name “register” in the url so:
#RedirectMatch 403 register\.

Check few more things…
[ Gravatar Icon ]

Jeff Starr – #8

@Arwin: Thanks for the catch on the config query-string issue with Mambo/Joomla.

@John: It is my pleasure to help the community, and if a few spammers get hurt in the process, even better. ;)

@Greg: Thanks for the input! Are you using WordPress or some other platform? That information will help to improve the Blacklist.
[ Gravatar Icon ]

Greg – #9

Changed this one:
RewriteCond %{QUERY_STRING} ^.*(\[|\]|\(|\)||'|"|;|\?|\*).* [NC,OR]

to:
RewriteCond %{QUERY_STRING} ^.*(\[|\]|\(|\)||'|"|\?|\*).* [NC,OR]

Because of my pdf links on articles which are with semicolon

I’m under joomla 1.5.9
[ Gravatar Icon ]

Jeff Starr – #10

Duly noted, Greg — keep us posted on any other modifications used for Joomla.
[ Gravatar Icon ]

Greg – #11

Ok Jeff ;-)

Just posted your 4G on the french joomla forum, so we will have more feedback soon for joomla (hope it’s ok for you ?)
[ Gravatar Icon ]

Greg – #12

Commented this one for joomla:
#RedirectMatch 403 config\.

Because of the name of an icon of the administration:
/icon-48-config.png
[ Gravatar Icon ]

Sven – #13

Thanks a lot Jeff :-)

Are there any advantages when you use
“RedirectMatch 403″ and not “RewriteCond” ?
[ Gravatar Icon ]

Arwin – #14

Jeff, I also need to remove the [ and ] from the QUERY STRING EXPLOITS section, otherwise you can not set the plugin-options in Joomla(1.5.9).

Example url:
http://....option=com_plugins&view=plugin&client=site&task=edit&cid[]=1
[ Gravatar Icon ]

Arwin – #15

Oeps, that example url did not do what I want.. Without the http.. : index.php?option=com_plugins&view=plugin&client=site&task=edit&cid[]=1
[ Gravatar Icon ]

Jeff Starr – #16

@Greg: Thank you for your help with Joomla. I will be posting an update to the article with all of the updates and edits that people have made. I even caught one myself for WordPress and will add it to the list after a bit more testing.

@Sven: That’s debatable, but I use both in the 4G because I have found that each method seems to work best in different environments and with different purposes. mod_rewrite is much more flexible than RedirectMatch, but not as easy to work with in all situations. All of the RedirectMatch directives in the 4G could be written with mod_rewrite, but not vice-versa.
[ Gravatar Icon ]

Jeff Starr – #17

@Arwin: I will add them to the list of modifications and post an update later today. Thank you for your help with Joomla.
[ Gravatar Icon ]

Greg – #18

Same as Arwin said:

just done this:
RewriteCond %{QUERY_STRING} ^.*(\(|\)||'|"|\?|\*).* [NC,OR]
[ Gravatar Icon ]

Sven – #19

Thank for the reply Jeff.

I asked because I would like to direct my visitors to a “friendly page” if they are blocked by httaccess (I also log all blocked requests in a txt-file). This is only possible if I use a Rewrite, or…?
[ Gravatar Icon ]

Jeff Starr – #20

@Sven: Yes, you will need mod_rewrite for that..
[ Gravatar Icon ]

Sven – #21

Thanks Jeff :-)
[ Gravatar Icon ]

Greg – #22

Hi,

For Joomla 1.5.9

Found today that we need to remove the “request” from the query string exploits conditions.

Because some components use it with ajax script on the backend (com_xmap for example)

So this:
RewriteCond %{QUERY_STRING} ^.*(request|select|insert|union|declare|drop).* [NC]

Become:
RewriteCond %{QUERY_STRING} ^.*(select|insert|union|declare|drop).* [NC]
[ Gravatar Icon ]

Andrew – #23

Great list, thanks for sharing it!

The only major issue I’ve had so far was with the Blacklist Candidates (IP Blocking) section. My apache (Apache 2.2.8) kept giving me “Order not allowed here”, “Allow not allowed here”, and “Deny not allowed here” no matter what I tried.

I ultimately found out that the 2.2 module’s (mod_authz_host, previously mod_access) directives are only allowed inside , , and contexts and .htaccess files: http://httpd.apache.org/docs/2.2/mod/mod_authz_host.html

After wrapping the entire block in a Directory context matching all possible directories (), apache gave me no more trouble.

Strange that I seem to be the only one having this problem.

Also, I initially had wrapped that section in an IfModule block, which I think is a good idea for any block that depends on a module.
[ Gravatar Icon ]

Greg – #24

@Andrew

Hi,

Got Apache 2.2.10 and don’t have your issue….

Where’s located your htaccess ? in your www or elsewhere on up directory ?
[ Gravatar Icon ]

Deb Phillips – #25

Jeff, I implemented your 4G Blacklist code about two days ago, and I’m amazed at the “calmness” in my visitor logs now! I no longer feel besieged by one unruly bot after another. What a relief! I just had to say “Thank you!” Your blacklist is wonderful, and it is so good of you to share it.

All the best,
Deb
[ Gravatar Icon ]

Tony – #26

Jeff,
I’ve been waiting forever for the 4G! I installed it on my first Wordpress blog for testing purposes. Here is what I noticed:

1. If you have the WP plugin “Ozh’ Admin Drop Down Menu” (latest version is 3.1.3.3.7 now), you’ll have to delete “drop” from the last line of the Query Strings Exploits:

RewriteCond %{QUERY_STRING} ^.*(request|select|insert|union|declare|drop).* [NC]

Becomes:

RewriteCond %{QUERY_STRING} ^.*(request|select|insert|union|declare).* [NC]

2. If you use the TinyMCE Advanced plugin (latest version is 3.2 now), the buttons won’t be visible because the css won’t load, because it’s located in a directory wp_theme. Comment out the line that says: RedirectMatch 404 wp\_

And a question: Wordpress gives a 404 instead of Apache giving a 403, unless I create a custom 403 ErrorDocument. Any idea why?

Thanks Jeff, and I’ll be testing more in the next few days.
[ Gravatar Icon ]

Deb Phillips – #27

I can confirm that the default TinyMCE toolbar buttons also do not show up correctly. Only by removing the 4G code from the .htaccess file will the TinyMCE toolbar be displayed correctly.

I use the 404 Notifier plugin to alert me to any 404s occurring, and I’ve been getting a number 404s generated only when the 4G code is loaded, Unfortunately, I’m not savvy enough to pinpoint the exact cause of these errors, but almost all of the 404s occur at the point of WordPress’ cron check. I’m getting hundreds of 404s — daily — that are related to the cron check.

Another issue that I wonder if it’s related to the 4G blacklist is with one bot repeatedly coming in looking for the 403.shtml file, and a 404 is generated. I don’t *think* this was occurring before I began using the 4G code. Perhaps this is what Tony is referring to.

Even with these few issues, though, I really do appreciate all the work that went into the 4G blacklist. Hopefully, we’ll get the few kinks worked out.

Thanks very much.
[ Gravatar Icon ]

Tony – #28

Deb, you’re right. I could have sworn that the standard tinyMCE loaded fine when I tried, and I had already cleared my browser’s cache. Now I try again and the standard tinyMCE also doesn’t load okay.

The problem is this address that tinymce calls:
/wp-includes/js/tinymce/themes/advanced/skins/wp_theme/ui.css?ver=20081129

BTW, I used Firebug to pinpoint that. For those interested, Firebug is a Firefox Extension.
[ Gravatar Icon ]

Jeff Starr – #29

@Greg: Thank you — the “request” character string has been added to the list of query-string modifications for Joomla.

@Andrew: That is interesting indeed. I will look into it and update the 4G with the appropriate location and IfModule containers.

@Tony: Thanks for the help! I have removed the “wp_” directive from the list and updated the article with information about Ozh’ plugin.

@Deb: I have removed the directive that was preventing TinyMCE from loading. Thanks to Tony for his help with this.
[ Gravatar Icon ]

Tony – #30

Jeff,
It’s the least I can do!

wp-cron is making HEAD requests, so it’s being blocked. What do you think we should do, besides removing HEAD from the 4G’s Request Methods Filter?

From the raw access log:
"HEAD /wp-cron.php?check=561caf12167cb54c25589d71581df596 HTTP/1.0" 403 - "-" "WordPress/2.7.1"

I apologize for repeating the question I asked above, but maybe nobody noticed it:
Wordpress gives a 404 instead of Apache giving a 403, unless I create a custom 403 ErrorDocument. Any idea why?

Thanks!
[ Gravatar Icon ]

Deb Phillips – #31

Yes, I strongly second Tony’s inquiry regarding wp-cron. I’m getting deluged with 404 notifications related to it. These are email alerts I receive from the 404 Notifier plugin. They’re occurring about every one or two minutes.

Also, Tony, what issue were you running into with the Ozh’ Admin Drop Down Menu plugin? I only ask because I’m running the Ozh’ Better Feed plugin. I’m guessing there’s probably not any basically similar code between the two, but I wanted to ask. Thanks.
[ Gravatar Icon ]

Tony – #32

Deb,

Sorry, I should have been more clear about Ozh’ Admin Drop Down Menu. This plugin calls this address:

/wp-content/plugins/ozh-admin-drop-down-menu/inc/adminmenu.css.php?p=%2Fwp-content%2Fplugins%2Fozh-admin-drop-down-menu%2Finc&i=1&w=1&m=0&c=0&h=0&f=1&g=%23676768&n=0

Since there’s the “drop” word in the query string, it gets a 403 and the whole admin page gets messed up.

And some good news: googlebot accessed my site today and got a 200. That’s the one thing I was worried about the most!
[ Gravatar Icon ]

Deb Phillips – #33

Tony, thanks for the info on the Ozh’ plugin.

I’m glad Google is showing your site as 200.

As clarification, I’ve not encountered any 404 pages when actually visiting my website. The 404 notifications I’ve referred to are all happening behind the scenes. I can also see them showing up in the cPanel “Latest Visitors” module.

I’ve also received some 404s related to accessing files in the wp-content/uploads directory. But every time I’ve gone to the pages containing those files on the website, the images are all there, and there’s no problem on those pages. So I suspect the 404s on those image files are somehow related to the 404 triggered by the wp-cron issue that occurred on those pages.

These are my observations so far. I don’t know all the technically correct terms to use, so I’m trying to relay the info as clearly as I can!
[ Gravatar Icon ]

Tony – #34

Deb, you’re welcome! Your site is really nice, and the design is great, not cluttered at all. I opened several pages and every single item (css, images, js, etc.) on every page loads flawlessly. I really recommend using Firebug when trying something as serious as the 4G blacklist.
[ Gravatar Icon ]

Deb Phillips – #35

Thanks so much, Tony. I’m honored that you stopped by my website.

As far as Firebug, I do use it, but I usually use it related to experimenting with graphics or layout changes prior to actually modifying the CSS.

Can you describe the steps (maybe just the buttons or the menu sequence) you typically take — without having to spend a lot of time to answer this — to use Firebug to troubleshoot issues such as the ones we’ve encountered here? (I had mainly been looking at the Latest Visitor module in cPanel and noting the error codes that were generated on specific URLs.)

Please don’t take a lot of time to answer this. (In that case, I’ll just have to do a little research on it.) I don’t want to create work for other people!

Thanks!
[ Gravatar Icon ]

Deb Phillips – #36

By the way, Tony. Please tell me it was you who happened to do some “testing” while visiting my site. That’ll make me feel a little better. Thanks!
[ Gravatar Icon ]

Tony – #37

Deb,
Jeff shared so much of his work with us; how do a few minutes of my time compare to that? :-D

For troubleshooting this kind of stuff, I use the “Net” tab in Firebug. Some screenshots of the Net tab from the official site:
http://getfirebug.com/net.html

Whenever an item of the page you’re loading gets a 403/404 response or other errors, you’ll see it in red. You can right click and copy its location, and do other stuff.

If you want to see how the page items are loading, but are only interested in, let’s say, the css files only, click on the button that says CSS above the Net tab. Of course the Net tab should be active for you to be able to even see those buttons.

I’m not that good at explaining things, especially when it’s not concrete. So please don’t be shy to ask questions. I am new at this myself, so we’ll both learn something! ;-)
[ Gravatar Icon ]

Tony – #38

Yes it was me who tested. I even wrote hideb.inc in the URL. I hope it’s okay…
[ Gravatar Icon ]

Deb Phillips – #39

Thanks for the Firebug info, Tony. I’ll do some experimenting with it.

Yes, I saw the “hideb.inc.” I’m relieved that was you. I was beginning to think the Internet was more treacherous than I’d imagined!

P.S. Go UNC!
[ Gravatar Icon ]

Jeff Starr – #40

@Tony: I have updated the post with the fix for the WP-Cron plugin. Thank you (again) for helping troubleshoot the 4G. As for why WordPress returns its 404 instead of Apache returning a 403, I really don’t know. I am using custom WP 404 pages for all of my themes and they are only returned for requests that are both not found and not blocked by the blacklist. It could have something to do with how your server is configured, but I don’t dare fathom a guess as to what that might be..
[ Gravatar Icon ]

Tony – #41

Jeff,

I’m happy to help, at least a little!

As for wp-cron, this is not the plugin (which is really, really outdated). It’s the standard wp-cron.php in the Wordpress package.

Is it possible to keep HEAD in the request-method filtering rules, but make some kind of exception for wp-cron.php?

If I find out why Wordpress is returning the 404s, I’ll post here.
[ Gravatar Icon ]

Andrew – #42

@Greg and @Jeff:

Oh, sorry, I forgot to mention: I was putting this in my httpd.conf file, not a .htaccess file. Sorry for the confusion!

As I mentioned, those directives are allowed in htaccess files, so if you just drop it into an htaccess file, it should be fine. It’s only in the httpd.conf file that it needs the extra wrapper.
[ Gravatar Icon ]

Deb Phillips – #43

Just to get some clarification, Jeff and Tony:

It sounds like the WP-Cron fix you’ve just implemented, Jeff, does not address the 404s received in relation to wp-cron.php? That’s what I’m continuing to receive droves of 404s on. (I’ve not tried your revised code yet.)

Thank you.
[ Gravatar Icon ]

Tony – #44

Deb,
I don’t think Jeff revised any of the code. He just advised to remove HEAD.

If you removed it yourself, can you paste the line from your raw access log where it says wp-cron.php?

Also, when you remove HEAD, be sure to remove the “|” after it.
[ Gravatar Icon ]

Jeff Starr – #45

@Andrew: thanks for the clarification on that — it will come in handy for others who might be experiencing the same issue. Cheers.

@Deb: As the issue is related to the WordPress core and not a plugin, I am going to I am going to update the 4G Blacklist itself by removing the HEAD method from the request-method filtering rules. This should eliminate the 404 errors related to wp-cron.php. I will update the code after leaving this comment.
[ Gravatar Icon ]

Jeff Starr – #46

Alright, I think we’re getting there for both Joomla and WordPress. I hate to delete the HEAD method from the request-filtering rules, so I will probably try to write some directives that omit requests for wp-cron.php. This most likely will happen in the next version of the Blacklist, which is actually already underway ;)
[ Gravatar Icon ]

Tony – #47

Jeff,
That’s very good to know! I’m already waiting for the 5G! :)
[ Gravatar Icon ]

Deb Phillips – #48

Thanks so much, Jeff. I must say, you’re one hard-working, conscientious dude! We’re fortunate you’re on the “good side”!
[ Gravatar Icon ]

Greg – #49

Ok with that.

No more feedback for joomla at this time.

Thanks for your work Jeff
[ Gravatar Icon ]

eezz – #50

Hi, Great work… one thing I have just picked up with Joomla when using OpenID… the redirect url back to Joomla from the OpenID server is denied. The url is very long and contains http and ? and plenty of %XX. I’ve tried a few OpenID’s and they work with the following mod:

RewriteCond %{QUERY_STRING} \.\.\/ [NC,OR]
RewriteCond %{QUERY_STRING} boot\.ini [NC,OR]
RewriteCond %{QUERY_STRING} tag\= [NC,OR]
RewriteCond %{QUERY_STRING} ftp\: [NC,OR]
RewriteCond %{QUERY_STRING} https\: [NC,OR]
RewriteCond %{QUERY_STRING} mosConfig [NC,OR]
RewriteCond %{QUERY_STRING} ^.*(\(|\)||'|"|;|\*).* [NC,OR]
RewriteCond %{QUERY_STRING} ^.*(%0|%A|%B|%C|%D|%E|%F|127\.0).* [NC,OR]
RewriteCond %{QUERY_STRING} ^.*(globals|encode|localhost|loopback).* [NC,OR]
RewriteCond %{QUERY_STRING} ^.*(select|insert|union|declare|drop).* [NC]
RewriteRule ^(.*)$ - [F,L]
[ Gravatar Icon ]

Garrett W. – #51

Ran into a few problems so I thought I’d let you know.

Under Query String Exploits:
- Blocking “config” interferes with the Akismet settings page in Wordpress
- Blocking “request” interferes with phpMyID ( http://siege.org/projects/phpMyID/ ) if I remember correctly
- Blocking “;” interferes with all but a few pages of SMF ( http://www.simplemachines.org/ )

I had too many problems with phpMyID anyway and decided to stop using it, but I had to remove the other two offending strings.

Also, under Character Strings:
- Blocking “,” interferes with SMF forum/subforum/topic pages IF search-engine friendly URLs are enabled

Also, under Query String Exploits:
- Blocking “select” interferes with the “Announce topic” option when creating a new topic in SMF. URL: “.../forums/?action=announce;sa=selectgroup;topic=2;goback”
[ Gravatar Icon ]

Greg – #52

Hi Garrett W.

Ok I give it a try on my own SMF forum too (wthout sef rewrite)
Tell you some feedbacks…
[ Gravatar Icon ]

Jeff Starr – #53

@eezz, @Garrett W.: Thank you both — I have annotated the “Updates” section to include a link to this information. Also Garrett, I took the liberty of consolidating your posts to facilitate usability. I hope you don’t mind ;)
[ Gravatar Icon ]

Garrett W. – #54

I don’t mind at all - I’m glad you did :)
[ Gravatar Icon ]

eezz – #55

Glad to help… bandwidth usage has almost halved on my site with using both the 4G and UA ‘firewalls’ you have made. This is a fantastic contribution the fight against the bot hoards.
[ Gravatar Icon ]

Jeff Starr – #56

@eezz: Excellent! Great to hear the 4G work is helping in the bot wars! ;)
[ Gravatar Icon ]

Garrett W. – #57

Jeff: have you heard of Project Honeypot? ( http://projecthoneypot.org/ )
I recently blocked 4 IPs from my site due to comment spam (and now I’m getting none at all!) and sure enough, all 4 IPs were listed as comment spammers in the Honeypot database.
Wish I had noticed it sooner, but they offer a service called http:BL (BL standing for Blacklist) that queries their database about every visitor to your site (in the form of a quick DNS request), and blocks the ones that are malicious. It even comes in the form of a Wordpress plugin! So I think I’m gonna try it out.
http://projecthoneypot.org/httpbl_implementations.php
[ Gravatar Icon ]

Jeff Starr – #58

Garrett W.: Yes, and it is a great program they’ve got going over there. There are also several other useful WordPress plugins such as Maximum Security, WordPress Firewall Plugin, and even WPIDS. There also are many other applications, scripts, and methods that help fight against malicious mischief on the Web.
[ Gravatar Icon ]

Donace | The Nexus – #59

@Garrett; bad behaviour plugin has honeypot blacklist implemented into it; I advise you check that out.

Would also recommend doing subtle changes to the comment field names as that is how harvesters harvest the urls to spam ;)
[ Gravatar Icon ]

Deb Phillips – #60

Hey, Jeff —

I don’t want to put you on the spot, but I’m a real novice at the bot wars and how to deal with them. So when the mention was made in previous comments about using plugins to combat bad bots, it made me wonder why you chose to take the time to come up with code to add to the .htaccess file versus using a plugin or two.

Are there advantages to taking your route versus plugins? Perhaps WordPress performance advantages? Or something else?

I’m just curious! :) Thanks.

Deb
[ Gravatar Icon ]

Garrett W. – #61

Using .htaccess stuff heads ‘em off at the front, instead of allowing them to see your site first. Plus, it’s probably faster than letting the plugins deal with them in PHP.
[ Gravatar Icon ]

Deb Phillips – #62

Then I’ll stay on this path, Garrett W.! Thanks for answering.
[ Gravatar Icon ]

Tony – #63

What Garrett said, plus the wonderful thing about this blacklist is that you can use it even if you don’t use WordPress.

I feel great when I know that some malicious bots don’t consume any of my bandwidth and don’t slow my websites. Stop them at the door! ;-)
[ Gravatar Icon ]

Jeff Starr – #64

@Deb Phillips: Sure, it’s a good question that I am sure other people have wondered about. The main reason I work with Apache/HTAccess for blacklisting involves performance, as you suggest. In my experience using a plugin really slows things down, especially anti-spam and blacklisting plugins that must interact with both PHP and a database. Throw WordPress functionality into the equation and performance may be affected drastically. I say “may be affected” because different servers and configurations will also play a role in determining overall performance, as will the WordPress setup in question.

Also, as Garrett W. points out, stopping malicious behavior before it reaches the inside of your site eliminates potential vulnerabilities and thus provides a greater degree of security.

Tony also makes a good point. Even though the 4G Blacklist is geared heavily toward WordPress, there are many Joomla/Mambo users who also enjoy its benefits. I would suspect many other sites do as well. ;)
[ Gravatar Icon ]

Yieu – #65

I just wanted to say that I implemented the 4G Blacklist on my website, and it is a regular website — not a blogging website such as Wordpress. It is very handy, as I know of no other blacklist such as this, and it appears to be very comprehensive. I do hope the fact that it is geared towards Wordpress does not leave out some security for regular websites, though. Perhaps a version geared towards regular websites might benefit other webmasters?

I also make use of the proxy blocking blacklist found here which helps keep spammers away, and the universal URL canonization directive. Thank you for making all of these available, it has definitely helped greatly.
[ Gravatar Icon ]

Jeff Starr – #66

@Yieu: My pleasure, and thanks for the feedback. Other people have also requested a version of the Blacklist that is geared for “regular” websites, and I am certainly considering putting something together. Keep in mind, however, that WordPress is the most popular blogging/website software in the world, and as such it is highly targeted for malicious behavior. Thus, even if your site is not running WordPress, there are scores of WP-related scans and bad requests hitting your server and wasting bandwidth nonetheless. In my experience, there are far fewer attacks directed specifically at general sites, such that a regular-site (i.e., non-WP) blacklist would leave your site wide open to the relentless barrage of platform-specific attacks.
[ Gravatar Icon ]

Yieu – #67

When I said a version geared towards regular websites, I did not mean to only include directives relevant to regular websites and to leave out the Wordpress directives. I noticed that you were removing directives because they were in conflcit with Wordpress, so one geared towards regular websites would re-include those directives for a tighter level of security — so it would leave in the Wordpress directives, add in the ones that were removed for Wordpress compatibility, and add directives geared specifically for regular websites on top of that.

I am not sure if this is asking for a too much (the current 4G Blacklist is very nice as it is), but it would also be a more universal security solution for websites and I am sure that would be useful. Such a list may require some explanation for some of the directives that are more liable to cause issues, though (or to simply continue leaving some out if they are too restrictive).
[ Gravatar Icon ]

Greg – #68

Hi,

today I found that the %3D block some of my google ads, so:

RewriteCond %{QUERY_STRING} ^.*(%22|%27|%3C|%3D|%3E|%7B|%7C).* [NC,OR]

become (for me):

RewriteCond %{QUERY_STRING} ^.*(%22|%27|%3C|%3E|%7B|%7C).* [NC,OR]
[ Gravatar Icon ]

Jeff Starr – #69

@Yieu: Yes, I see.. Keep in mind that only a handful of the original directives were removed on account of WordPress, Joomla, and various plugins. Re-inclusion of these directives should be straightforward, however implementing additional rules for “regular” websites will take some time.

Actually — and you probably figured this — during the development of the 4G, many powerful rules were tested and ultimately dropped strictly for the sake of WordPress. These incompatible directives could be re-included in a non-WordPress blacklist, such as you have described.

It is a good idea and I will try to release something along those lines with the next version (5G) of the blacklist. Thanks for pushing the idea ;)
[ Gravatar Icon ]

Jeff Starr – #70

@Greg: Thank you Sir! — That information may come in handy for people running the same configuration :)
[ Gravatar Icon ]

Yieu – #71

I have just installed vBulletin version 3.8.2 on my website which runs the 4G Blacklist, and I had to comment out the following directives:

# RedirectMatch 403 display\.
# RedirectMatch 403 register\.

The 403 display directive was blocking the individual forums from displaying, because their link includes “forumdisplay.php”, and the 403 register directive was preventing the registering of new accounts because the link to register includes “register.php”.
[ Gravatar Icon ]

Dave Stuttard – #72

Hi, this innovative package looks great. Just implemented the 4G Blacklist in .htaccess on 3 sites (all standard, ie not WP or Joomla!), two being static, one being dynamic with a CMS. All features work fine, except:

1) I had to comment out http\ in the Query String Exploits before my Flash elements displayed - thought I’d mention that because nobody else has noted it.
2) I had to comment out RedirectMatch 403 config\ in the SPECIFIC EXPLOITS before the wysywyg Editor box was displayed for User data entry on the dynamic site - the code with ‘config’ in it is (the id):

Now I will study the stats to make sure Google, Yahoo! and MSN get 200s. I’ll let you know….. Many thanks
[ Gravatar Icon ]

Dave Stuttard – #73

Sorry, the code entered in my last post is:


[ Gravatar Icon ]

Yieu – #74

In order to allow private messages on vBulletin, I had to remove “insert” from the following directive:

RewriteCond %{QUERY_STRING} ^.*(request|select|insert|union|declare|drop).* [NC]

becomes:

RewriteCond %{QUERY_STRING} ^.*(request|select|union|declare|drop).* [NC]
[ Gravatar Icon ]

Dave Stuttard – #75

Hi, I found another thing needing to be commented out (this time from 403 errors):
RedirectMatch 403 \,
This was because there are some %2s, ie commas, in dynamically-generated property page urls.
Not much happening in my stats (it’s Sunday); I’ll check Monday evening to see if the search engines get 200s for any pages.
If it’s OK I’ll buy you a pint (and follow your progress of course)!
The importance of checking every user operation/display on the websites is clearly emphasised.
Thanks, Dave
PS: Don’t know about the ban proxy server script - I imagine thousands of legit ISP proxys needing to be identified and allowed to get any visitors (I posted an email on this but don’t know if it was sent).
[ Gravatar Icon ]

Greg – #76

Hi again ^^

Just commented out this one:
#RedirectMatch 403 \;

Cause block the vidéos player links of the jomtube component for Joomla 1.5.10
[ Gravatar Icon ]

Dave Stuttard – #77

Hi, just wanted to confirm that G,Y and M appear to be crawling successfully on my 3 sites. Wow, that means this blacklist is an excellent addition to a website’s security setup (with such things as captchas and URL verifications in forms and email address obfuscating) - when thorough testing has been done and any problem entries have been commented out. It saves a lot of time trying to keep track of and banning bad bots individually. Hope my posts were useful. Looking forward to its further development, eg versions for different types of website, with possible problem entries highlighted to help diagnosis when things go wrong?
I know - I owe the author a pint (Jeff?). And thanks to others who have contributed useful suggestions.
Regards, Dave
[ Gravatar Icon ]

Jeff Starr – #78

@Yieu: Thanks for the help with vBulletin. Good to hear that only a few items were removed. I have updated the post notes for people running vBulletin. If you find anything else, please let us know. Thanks :)

@Greg: Thanks! Post notes for Joomla have been updated accordingly.

@Dave Stuttard: Thank you for posting your findings with your sites. Any additional updates are encouraged, of course. Software-specific modifications are added to the “notes” section at the end of the article, but I am sure that the general information you provide will help others diagnose and troubleshoot if/when similar situations arise. Thanks for your help with the 4G — I am glad you have found it be a useful addition to your security strategy. And yes, I will definitely take you up on that pint! :)
[ Gravatar Icon ]

Greg – #79

More precisions about my previous %3D submission here:http://perishablepress.com/press/2009/03/16/the-perishable-press-4g-blacklist/#comment-71184

This one is only useful to be applied only when you need to preview some ads on your site, when you are in your googlads administration.
[ Gravatar Icon ]

Greg – #80

On my forum SMF 2.0RC1

RewriteCond %{QUERY_STRING} ^.*(select|insert|union|declare|drop).* [NC]

becomes:
RewriteCond %{QUERY_STRING} ^.*(select|insert|union|declare).* [NC]

Because all my dropdown menus were blocked
[ Gravatar Icon ]

Greg – #81

Hi,

Joomla 1.5.10

change:
RewriteCond %{QUERY_STRING} ^.*(%0|%A|%B|%C|%D|%E|%F|127\.0).* [NC,OR]

to:
RewriteCond %{QUERY_STRING} ^.*(%0|%A|%B|%C|%D|%F|127\.0).* [NC,OR]

because the %E blocked the poll component
[ Gravatar Icon ]

Tom – #82

What a great resource :) Works a treat, but I encountered a problem with Remository on Joomla 1.5.x. Here’s the offending URL string that triggers a 403:

/component/remository/func-download/248/chk,8e2144d3ae6dbfd7591edaa4c16526df/no_html,1/
[ Gravatar Icon ]

Greg – #83

Hi, Tom

did you check the updates for joomla on the end of this article ?
[ Gravatar Icon ]

Tom – #84

Did I check the updates? Not yet :(

I did narrow it down to the filtering of the comma in the URL string. If I comment that out, Remository downloads work. However, I detect from your writing that you really don’t like eliminating an entire class of filter, because it then leaves other potential exploits available. That’s why I was supplying the “bad” URL string…which has *two* instances of a comma in it.

Will check the Joomla updates later this morning. And Greg: thanks again. You provide some wonderful stuff :)
[ Gravatar Icon ]

Greg – #85

ok, thanks Tom

…the stuff master is Jeff Starr ;-)
[ Gravatar Icon ]

Tom – #86

I see Dave Stuttard in comment #75 already picked up this particular issue, albeit it with a different Joomla! component. The comma in a URL does affect Remository in Joomla! 1.5.x, and one way to fix it is to comment out the “RedirectMatch 403 \,” directive.
[ Gravatar Icon ]

Dave Stuttard – #87

Tom
Just a thought - I don’t use Joomla! or Wordpress or SMF or vBulletin, etc. My site with the comma in some address strings is a conventional php/MySQL driven dynanic site - the point is that the 4G Blacklist may have been aimed at Joomla! and Wordpress scenarios originally but it can also apply to other scenarios like mine.

General Point
‘SpamBlockerUtility’, ScoutJet and maybe others are still getting in so maybe these should be dealt with individually with ‘RewriteCond’s (otherwise one usually has to deny ranges of IPs, not single IPs, possibly blocking some legitimate visitors):

RewriteCond %{HTTP_USER_AGENT} spamblockerutility [NC,OR]
RewriteCond %{HTTP_USER_AGENT} scoutjet [NC]
RewriteRule ^.* - [F,L]

(note the absense of ^ before the UA element - without it we can ban any UA string that includes that element (somebody correct me please if my code is wrong) and don’t forget the NC to make it case-insensitive.
[ Gravatar Icon ]

Chaos Inc. – #88

Hiya Jeff!

Already mailed ya yesterday regarding this but I want to add a little more (a lot more lol) feedback with the ones I already mentioned in the mail. Also, I hope that this will help you and in turn, help us, to generate the ultimate GX (generation X) Blacklist.

After implementing only your 4g blacklist on my blog:

My Wordpress Auto Thumb Generation plugin (TimThumb) is not working, and it maybe because the auto generated images/thumbs carries some & or % characters/strings.

Users browsing from all and any kind of mobile platform is being shown the
403.

Users browsing from Opera mobile, opera mini, and other symbian/iphone
applications are being blocked. There are two kinds of connection for Opera
mini : Socket and Http based connections. Neither works. Some mobile phone
browsers also user shadow connects by default and those are also being
blocked.

Both symbian, python and flashlite based browsers are being shown 403.

I’m a mobile theme and ui developer and people similar to me who actually
produces content are being scraped severely, suffering stolen content
dilemmas and other general dos based attacks. Is there a way I can block
someone/everyone who regularly visits a site like scraping.com and visiting
mine as well. What i mean to say is that can i block someone who is probably
stealing my content and posting in his site? Since there is a usage pattern
and trend for that visitor like he visits his site first and then visits
mine to copy and he does this for every post i make. Cant they be blocked
based on their behaviorat pattern?

Almost all telecom operators usually provide 3g, gprs or 2.5g based mobile
internet to all their clients. Typically a mobile user from a country like
Bangladesh (my country) will have the ip of Norway on his phone because the
telecom or gsm operator’s mobile internet gateway for all the countries
they operate in are going through a single wireless internet platform (to be
cost effective and so on). This platform actually works as a hub for their
global 3g network in different countries. Is there a way to somehow let
users enter my site by skipping all the htaccess rules set by me when they
are from a mobile phone/pda/iphone etc.

Lastly, almost anyone who have implemented cname records for his WAP or
Online Store for a subdomain on his site cannot go back to the main
page/normal-pages when they have visited the store ( otherwise referred by
the store). This can be because of the weird caniocal url for any content on
the store/wapsite typically produce (not sure though). Is there a way to let
users browse my site normally when referred from a specific
domain/subdomain/platform etc?

Some mobile phone focused search engine crawlers are being blocked. Like
those from google mobile or mobile meta tag search. Is there a way to fix
this and let them in?
[ Gravatar Icon ]

Greg – #89

Done this today for SMF 2.0 RC1

RewriteCond %{QUERY_STRING} ^.*(\(|\)||'|"|\?|\*).* [NC,OR]

Because with the \[|\] we can’t erase some messages in the private box of the forum (MP)
[ Gravatar Icon ]

Lee – #90

Ah yes! Working great! Thanks, man. We all appreciate it.
[ Gravatar Icon ]

John Hoff - WpBlogHost – #91

Jeff, this is an excellent list of directives. Thank you for sharing.

Question. One of the services we provide is hardening people’s WordPress blog. I’d like to include these in our upgrades. Would you be open to letting me paste these directives in to our customer’s .htaccess files? I’d be sure to leave the comment showing your site and anything else you might request.

Thanks. Stumbling this article and Tweeting it!
[ Gravatar Icon ]

Nihar – #92

Awesome list.

I will add it in .htaccess file.

Have on one doubt, Where should i add the above list? after wordpress lines in htaccess file or before?
[ Gravatar Icon ]

Garrett W. – #93

Before.
[ Gravatar Icon ]

Jeff Starr – #94

@John Hoff - WpBlogHost: Thanks for the great feedback — much appreciated.

As for including the 4G blacklist into your customer’s HTAccess files, contact me directly to discuss further. Thanks.
[ Gravatar Icon ]

Douglas Machado – #95

Have you ever tested Apache mod_security?
Which is the best (most reliable) and faster security solution 4G Blacklist of mod_security?

ModSecurity website:
http://www.modsecurity.org/projects/modsecurity/apache/index.html
[ Gravatar Icon ]

Jeff Starr – #96

@Douglas Machado: Without a doubt mod_security. 4G is designed to thwart specific types of attacks while mod_security is a much more comprehensive security strategy. I am running a modified version of 4G to complement mod_security, not replace it. So, if you have to pick, go with mod_security. Then, if you need additional protection, try the 4G Blacklist. ;)
[ Gravatar Icon ]

MileHighTechGuy – #97

This is great stuff.

Under WordPress I had to comment out the following ‘config’ line for the ‘Deans FCK Editor’ to work:

# SPECIFIC EXPLOITS
#this line conflicts with Deans FCK Editor WP plugin
#RedirectMatch 403 config\.

I’m posting a security related article later today that will reference your work.

Thanks for the great effort.

~Jeff (MileHighTechGuy)
[ Gravatar Icon ]

Jeff Starr – #98

@MileHighTechGuy: Awesome, Jeff - thanks for the mention in your article. I am taking a copy with me to read at work — it looks like a very informative reference that should help lots of people protect their WordPress installations. Thanks also for the heads up on the 4G modification for the FCK Editor WP plugin. Cheers.
[ Gravatar Icon ]

MileHighTechGuy – #99

I’m glad you think my post might have some value. Thanks for the feedback.

Not only is it posted here on my blog:
http://milehightechguy.com/2009/06/how-to-guide-for-securing-wordpress-and-protecting-websites/

But it is also posted here at Examiner.com:
http://www.examiner.com/…/WordPress-installation-and-recove
[ Gravatar Icon ]

Vladimir – #100

Jeff, if Apache stores its logs in CLF format and you have Linux, here’s a nice command to get all 404’s:

cat *.log | grep " 404 " | awk '{ print $7; }' | sort | uniq

I guess it can give you some more patterns to check against ;-)
[ Gravatar Icon ]

Brandon – #101

This line:

RewriteRule ^(.*)$ - [F,L]

stops WordPress 2.8’s widget manipulation screen from working. It kills the javascript.
[ Gravatar Icon ]

Jeff Starr – #102

@Vladimir: Very nice - thank you, Sir :)

@Brandon: That particular RewriteRule is associated with two different sets of directives: one for filter-request methods and the other for query-string exploits. Do you happen to know which set of directives is responsible for the issue you describe? Thanks.
[ Gravatar Icon ]

Brandon – #103

Sorry, that was query-string exploits.
[ Gravatar Icon ]

MIke Selvage – #104

Mike Selvage: Great work! But I have a question. I get the idea the 4G Blacklist supersedes the User_Agent blacklist … but does it also supersede the Referrer blacklist?
[ Gravatar Icon ]

MIke Selvage – #105

Mike Selvage: Disregard the last post, please. In the past 2 minutes I learned how to read …
[ Gravatar Icon ]

Garrett W. – #106

Just found another problem: The following from Wordpress is caught and returns a 403.
/wp-admin/plugins.php?action=delete-selected&checked[0]=adsense-manager%2Fadsense-manager.php&plugin_status=all&paged=1&_wpnonce=159d52cca3

…but I can’t figure out which block is triggering it. I tried taking out everything that blocked [ and ], as well as %5B and %5D, and it still wouldn’t go through. Any idea?
[ Gravatar Icon ]

Jeff Starr – #107

Hey Garret, it looks like most of that URL is comprised of a query string, so you may want to check that section closely. In particular, it looks like the brackets may be causing the issue.

Let me know how it goes and I will update the article.
[ Gravatar Icon ]

Tom – #108

I have to echo Brandon in comment #101. I’ve upgraded a couple of sites to WordPress 2.8, and cannot modify or move widgets. When I replace the 4G list with a standard WordPress .htaccess file, the ajaxy-goodness of WP 2.8 works again. Restore 4G in the .htaccess, and it does not. I’m sorry I can’t be more specific than that. I’m not really seeing what wp-admin is trying to call when I’m attempting to twiddle with widgets. My workaround for the moment is to temporarily replace the 4G-based .htaccess with a vanilla .htaccess, but that is not particularly convenient. Thoughts?
[ Gravatar Icon ]

Garrett W. – #109

Ah, so now that I’m experiencing the same thing, I realize that someone already commented about it. lol.
So the blacklist is responsible? That’s good - saved me from having to trudge through WordPress’s code.
However, I did look through it a little, and I found the offending piece of info.

These lines:
RewriteCond %{QUERY_STRING} (request|select|insert|union|declare|drop) [NC]
RewriteRule .* - [F,L]

are being triggered by:
wp-admin/load-scripts.php?[…]load=[…],jquery-ui-droppable,[…]
[ Gravatar Icon ]

Jeff Starr – #110

Thanks Garrett. For those that don’t speak voodoo, you may easily resolve this issue by removing the “|drop” from the Query-String Exploits section. That single edit should fix you up without compromising the overall effectiveness of the blacklist.
[ Gravatar Icon ]

Tom – #111

Jeff, your change in #110 solved the problem for me in WordPress 2.8. And thanks Garrett for providing enough detail, where my information was sorely lacking!
[ Gravatar Icon ]

Garrett W. – #112

fyi, my name has two Ts in it ;)
[ Gravatar Icon ]

Jeff Starr – #113

That’s what I get for not copy/pasting - Fixed. :P
[ Gravatar Icon ]

Garrett W. – #114

Thanks. I wouldn’t have mentioned it, but you were consistently misspelling it… and that’s a pet peeve of mine. ;)
[ Gravatar Icon ]

Jeff Starr – #115

God, I feel awful now ;)
[ Gravatar Icon ]

Garrett W. – #116

Aw, you poor thing .. it’s ok, I forgive you :)
[ Gravatar Icon ]

Jeff Starr – #117

Thanks! :)
[ Gravatar Icon ]

Scotia – #118

Joomla 1.5.12
Community Builder v1.21
SOBI2 v2.9.2.1
sh404SEF dev. 1.0.20 Build 237
Smartoptimizer v1.7b
CssJscompress v1.1

CBlogin module (to log into user accounts to work) I had to comment out:

# RewriteCond %{QUERY_STRING} ^.*(select|insert|union|declare|drop).* [NC] - I tried each of select|insert|union|declare|drop separately, still would not work. - What is the impact of losing this one?

SOBI2 Search functionality was messed until I commented out:

# RewriteRule ^(.*)$ - [F,L] - Can I change anything here for this to work? What is the impact of losing this one?

CBLogin mod Lost password functionality lost (Ajax) until one of these two was commented out:

# RedirectMatch 403 register\.
# Redirectmatch 403 password\.

I hope this is usefull for anyone else running Joomla, in combination with CB, sh404 etc.
[ Gravatar Icon ]

Greg – #119

For select|insert|union|declare|drop , so this is probably not the real cause.

For # RewriteRule ^(.*)$ - [F,L], this is the rewrite rule, so if you comment it out, all the directives abvo becomes inneficient.

Add something like at the first place of the query strings block: RewriteCond %{HTTP_REFERER} !^(.*)(put_the_name_of_thepage_suspected_here).* [NC] could help…

But you have to find the real parameter that is blocking SOBI.
[ Gravatar Icon ]

Greg – #120

Hi,

so many like that in my log file:

"HEAD /favicon.ico HTTP/1.0" 403 - "-" "-"

How should we deal with thoses ?

Should we had :

RewriteCond %{REQUEST_URI} !^/favicon.ico

Just after the # FILTER REQUEST METHODS ?
[ Gravatar Icon ]

Jeff Starr – #121

Hi Greg, you could try that, or use this method for additional help. Cheers.
[ Gravatar Icon ]

Greg – #122

Thanks Jeff

Ok with that, but I ask this, because those are requests with the HEAD method, so I was wondering how to deal with the favicon requested with HEAD.
Should we consider those as legitimate ?
Or it must be only allowed from a GET ?
[ Gravatar Icon ]

Jeff Starr – #123

Ah yes, sorry I missed that part of the request. The thing about HEAD requests is that the server ideally does not return a message-body in response. Other than that, GET and HEAD are essentially the same. HEAD is typically used for testing (either proper or malicious). Whether or not to allow HEAD requests for various files (such as favicon) is up to you.
[ Gravatar Icon ]

Greg – #124

Found it !

Lol it was me….all my tests with firebug an YSlow were made all this HEAD requests.
[ Gravatar Icon ]

Jeff Starr – #125

Lol, ah yes - I love it when that sort of thing happens =)

Glad to hear you got it sorted out :)
[ Gravatar Icon ]

ken – #126

If you block some ip addresses would you block out alot of people that do not have a static ip. I also saw some question about mobile phones not being able to access site. I did not see answer or if it is a problem.
[ Gravatar Icon ]

duck – #127

Firstly, thanks for such a great resource, this post and the building of the 4G blacklist are great posts.

I discovered that after implementing the 4G blacklist on a Wordpress 2.8.4 website that I was unable to delete plugins from the Plugin menu (was mentioned in comment 106). The link of the delete button takes the form:

...plugins.php?action=delete-selected&checked[0]=hello.php&plugin_status=all&paged=1&_wpno...

To fix this you need to comment out the lines blocking [ and ] as well as removing select from the last condition in the Query string exploits section.

(It might be useful to comment out the lines blocking %5B and %5D, but it wasn’t necessary for me)
[ Gravatar Icon ]

duck – #128

I need a bit of help as an implementation of the 4G blacklist doesn’t seem to be working. It is working perfectly on my local machine, giving me 403 errors if I put “password.” at the end of the url for example, however on the production server it is producing 404 errors instead. Looking in cPanel redirects I see that it is interpreting all the RedirectMatch 403 rules as pointing to a directory 403, being type permanent and the redirect url being something like \. or \,.

Do you think that this is an issue with cPanel, my webhost’s setup or anything else you have come across before?
[ Gravatar Icon ]

Jeff Starr – #129

Hi duck, I have seen this behavior before on one of my shared servers. Have not been able to pinpoint the exact cause, but have been able to workaround by either using a different error (something other than 403) or relocating the directives into a directory higher up the tree.
[ Gravatar Icon ]

jon (aka duck) – #130

I discovered that, at least for my situation, the server was still responding with a 403 status header but just not displaying an Access Denied page. All I had to do was declare an ErrorDocument in my htaccess file e.g.

ErrorDocument 403 "

Access Forbidden!

Error 403

"

and it worked. This is obviously something to do with the way the server had been configured as it was all working perfectly on my local test install of apache.
[ Gravatar Icon ]

Jeff Starr – #131

Interesting, jon — I will have to keep that in mind the next time I find myself trying to resolve that issue. Another thing to consider is that WordPress sometimes will override errors if a custom error page is available on the active theme. I am still working on how to workaround that one. Thanks for sharing your solution. Cheers.
[ Gravatar Icon ]

Cooltad – #132

I’m insane about speed and optimization.
Do you have a flattened version of your 4G list and it’s optional components?
[ Gravatar Icon ]

Jeff Starr – #133

Hi Cooltad, not at this time. Do you think a “flattened” version would help with performance?
[ Gravatar Icon ]

Cooltad – #134

Most certainly. We all should know that, even if there are whitespaces and new lines, each of those 2 require one byte, or 6 bits. Now, with your massive 4g list compiled together, you’d save a very large amount of bytes, which would make processing server-side for the .htaccess faster.
You wouldn’t think it would, because it doesn’t ‘parse’ whitespaces etc, however, it still needs to sift through such spaces to get to the real stuff.
[ Gravatar Icon ]

Greg – #135

I think all the Redirectmatch 403, could be in one line.
…for example.
[ Gravatar Icon ]

Jeff Starr – #136

Yes, I can see how this would help for tightly controlled or high-bandwidth scenarios, but even then, the gains would be modest at best.

Even so, I have done this for other blacklists, including the first “Ultimate HTAccess Blacklist” a few years ago. I will be sure to include a “flattened” version of the next, 5G Blacklist.
[ Gravatar Icon ]

Peekay – #137

Xoops users who allow user registration are going to need to comment out:

# RedirectMatch 403 register\.

from the ‘Specific Exploits’ section.
[ Gravatar Icon ]

Omar Ramos – #138

I was wondering how we might be able to test that the blacklist is working correctly?

For example, it seems like it would be as easy as using “select” within the query string of the URL, but when I try doing that I’m not getting a 403 response, the page just ends up loading the same way it did before (I’m using the above code within an .htaccess file).

Any thoughts on how this can be tested simply, just to make sure the rules are in effect?

Thank you!
[ Gravatar Icon ]

Omar Ramos – #139

I tried the visiting the following URL:
index.php?option=com_newsflash&id=8+and+1=1+union+select+1,username,password,4+from+mos_users&catid=0

On our remaining Joomla 1.0 site and it seems to be working.

I got the link from the milw0rm site here:
http://milw0rm.com/exploits/7718

( Hopefully you don’t mind I post the links on here, they seem to be a whitehat group:
http://en.wikipedia.org/wiki/Milw0rm )


Reference: http://perishablepress.com/press/2009/03/16/the-perishable-press-4g-blacklist/