Friday, March 19, 2010

Intro to SSI (Server Side Includes)

SkyHi @ Friday, March 19, 2010
We are often asked by webmasters how to repeat web elements across their pages without have to edit each static page individually each time they need to make a change. External CSS is the answer to your web pages' cosmetic elements (fonts, colors, borders, padding, etc.), but SSI is a great tool when you want to use the same web element (navigational menu, page text, etc.) across multiple pages, but you do not want to change it on every page each time you need to add, delete or modify.

For example, let's imagine that you have a navigational menu that consists of five hyperlinks. You want to add a sixth link but your site consists of over 500 pages. With SSI you edit one file that contains the HTML code of your navigational menu (only), upload the revised file to your site and all pages are changed instantly. Yes, it is very similar to the magic of external CSS that we continually recommend. ;)

To be able to use SSI, your web hosting service must be able to support it. Most quality hosts automatically enable SSI on their servers these days. Many hosts require that you use a .shtml extension on your pages. Although some hosts can enable your server to utilize SSI with any page extension.

If you are not sure if your server supports SSI, no problem. Before contacting your hosting company, test it yourself.

Create a file and name it test.shtml. Within this file place the following code. Wherever you place this code is where the file "mytest.ssi" will be displayed (included).

<!--#include virtual="mytest.ssi"-->

Now create the mytext.ssi file that will be pulled into your test.shtml page. Put whatever you wish into the mytext.ssi file. If you are using it for a navigational menu, your code might look something like this:

<a href="index.shtml">Home</a> | <a href="about.shtml">About Us</a> | <a href="contact.shtml">Contact Us</a>

Important: You can use most any HTML code in your .ssi file. But do not use the standard BODY, TITLE, HEAD tags. Use only the HTML code for whatever web element you want to appear within the section it is being pulled into.

You are not limited to using the .ssi extension for your include file. You can use .txt, .htm, .html etc. Just be sure it does not declare the standard BODY, TITLE, HEAD tags or you will receive an annoying error message.

Name your include files something memorable so that you know what they are as soon as you see them. In other words, naming your navigational menu 1.ssi may not be the best choice unless you only have one .ssi file.

Save both files and upload them to your server to test whether your hosting company has SSI enabled for you. Load your test.shtml file into your browser, like this: If it pulls the contents of the mytest.ssi into it, TADA! ... it worked!

If it does not work and you have followed these directions exactly, contact your hosting company to find out their requirements and/or limitations.

On another positive note, it is fairly easy to convert most of our web templates to use SSI. By simply copy/pasting bits/pieces of the original template into .ssi files and using the include code given above to pull the .ssi files into the location where these code bits/pieces originally existed, you can setup your web template to be even more effective and save you even more time. Pretty cool, eh?


Thursday, March 18, 2010

Mail processing model Outgoing mail SMTP server

SkyHi @ Thursday, March 18, 2010

Mail processing model

The overall flow for message creation, mail transport and delivery may be illustrated as follows:

sending MUA → MSA → sending MTA → receiving MTA → MDA → Mailstore for retrieval by MUA

E-mail is submitted from a mail client (MUA, message user agent) to a mail server (MSA, message submission agent) using SMTP usually. From there, the MSA delivers the mail to an MTA, often running on the same machine. A message may be directly submitted to an MTA: TCP port 587 is typically used for submission to MSAs (thence to MTAs), while TCP port 25 must be used for transferring to MTAs.

The MTA looks up the destination's mail exchanger record (MX record) in the Domain name system (DNS), and relays the mail to a server on record via TCP port 25 and SMTP. (The article on MX record discusses many factors in determining which server the sending MTA connects to.) Once the receiving MTA accepts the incoming message, it is delivered via a mail delivery agent (MDA) to a server which is designated for local mail delivery. The MDA either delivers the mail directly to storage, or forwards it over a network using either SMTP or the Local Mail Transfer Protocol (LMTP), a derivative of ESMTP designed for this purpose. Once delivered to the local mail server, the mail is stored for batch retrieval by authenticated mail clients (MUAs). Mail is retrieved by end-user applications, the email clients, using Internet Message Access Protocol (IMAP), a protocol that both facilitates access to mail and manages stored mail, or the Post Office Protocol (POP) which typically uses the traditional mbox mail file format. Webmail clients may use either method, but the retrieval protocol is often not a formal standard. Some local mail servers and MUAs are capable of either push or pull mail retrieval.

SMTP defines message transport, not the message content. Thus, it defines the mail envelope and its parameters, such as the envelope sender, but not the header or the body of the message itself. STD 10 and RFC 5321 define SMTP (the envelope), while STD 11 and RFC 5322 define the message (header and body), formally referred to as the Internet Message Format.

[edit] Protocol overview

SMTP is a text-based protocol, in which a mail sender communicates with a mail receiver by issuing command strings and supplying necessary data over a reliable ordered data stream channel, typically a Transmission Control Protocol (TCP) connection.[2] An SMTP session consists of commands originated by the SMTP client and corresponding responses from the SMTP server by which the session is opened, session parameters are exchanged, the recipients are specified and possibly verified, and the message is transmitted, before the session is closed. The originating host is either an end-user's email client, functionally identified as a mail user agent (MUA), or a relay server's mail transfer agent (MTA).

SMTP was designed as an electronic mail transport and delivery protocol, and as such it is used between SMTP systems that are operational at all times. However, it has capabilities for use as a mail submission protocol[2] for email clients (split user-agent) that do not have the capability to operate as MTA. Such agents are also called message submission agents (MSA),[13] sometimes also referred to as mail submission agents. They are typically end-user applications and send all messages through a smart relay server, often called the outgoing mail server, which is specified in the programs' configuration. A mail transfer agent, incorporated either in the e-mail client directly or in the relay server, typically determines the destination SMTP server by querying the Domain Name System for the mail exchanger (MX record) of each recipient's domain name. Conformant MTAs fall back to a simple address lookup (A record) of the domain name when no mail exchanger is available. In some cases an SMTP client, even a server, may also be configured to use a smart host for delivery. The SMTP client typically initiates a Transmission Control Protocol (TCP) connection to the SMTP server on the well-known port designated for SMTP, port number 25.

SMTP is a delivery protocol only. It cannot pull messages from a remote server on demand. Other protocols, such as the Post Office Protocol (POP) and the Internet Message Access Protocol (IMAP) are specifically designed for retrieving messages and managing mail boxes. However, the SMTP protocol has a feature to initiate mail queue processing on a remote server so that the requesting system may receive any messages destined for it (cf. Remote Message Queue Starting). POP and IMAP are preferred protocols when a user's personal computer is only intermittently powered up, or Internet connectivity is only transient and hosts cannot receive message during off-line periods.

[edit] Remote Message Queue Starting

Remote Message Queue Starting is a feature of the SMTP protocol that permits a remote host to start processing of the mail queue on a server so it may receive messages destined to it by sending the TURN command. This feature however was deemed insecure[14] and was extended in RFC 1985 with the ETRN command which operates more securely using an authentication method based on Domain Name System information.

Outgoing mail SMTP server

An e-mail client requires the name or the IP address of an SMTP server as part of its configuration. The server will deliver messages on behalf of the user. This setting allows for various policies and network designs. End users connected to the Internet can use the services of an e-mail provider that is not necessarily the same as their connection provider (ISP). Network topology, or the location of a client within a network or outside of a network, is no longer a limiting factor for e-mail submission or delivery. Modern SMTP servers typically use a client's credentials (authentication) rather than a client's location (IP address), to determine whether it is eligible to relay e-mail.

Server administrators choose whether clients use TCP port 25 (SMTP) or port 587 (Submission), as formalized in RFC 4409, for relaying outbound mail to a mail server. The specifications and many servers support both. Although some servers support port 465 for legacy secure SMTP in violation of the specifications, it is preferable to use standard ports and standard ESMTP commands[15] according to RFC 3207 if a secure session needs to be used between the client and the server. Some servers are set up to reject all relaying on port 25, but valid users authenticating on port 587 are allowed to relay mail to any valid address. A server that relays all e-mail for all destinations for all clients connecting to port 25 is known as an open relay and is now generally considered a bad practice worthy of blacklisting.

Some Internet service providers intercept port 25, so that it is not possible for their users to send mail via a relaying SMTP server outside the ISP's network using port 25; they are restricted to using the ISP's SMTP server. Some independent SMTP servers support an additional port other than 25 to allow users with authenticated access to connect to them even if port 25 is blocked. The practical purpose of this is that a mobile user connecting to different ISPs otherwise has to change SMTP server settings on the mail client for each ISP; using a relaying SMTP server allows the SMTP client settings to be used unchanged worldwide.

SMTP transport example

A typical example of sending a message via SMTP to two mailboxes (alice and theboss) located in the same mail domain ( is reproduced in the following session exchange.

For illustration purposes here (not part of protocol), the protocol exchanges are prefixed for the server (S:) and the client (C:).

After the message sender (SMTP client) establishes a reliable communications channel to the message receiver (SMTP server), the session is opened with a greeting by the server, usually containing its fully qualified domain name, in this case The client initiates its dialog by responding with a HELO command identifying itself in the command's parameter.

<code>S: 220 ESMTP Postfix
S: 250 Hello, I am glad to meet you
S: 250 Ok
S: 250 Ok
S: 250 Ok
S: 354 End data with <CR><LF>.<CR><LF>
C: From: "Bob Example" <>
C: To: Alice Example <>
C: Cc:
C: Date: Tue, 15 Jan 2008 16:02:43 -0500
C: Subject: Test message
C: Hello Alice.
C: This is a test message with 5 header fields and 4 lines in the message body.
C: Your friend,
C: Bob
C: .
S: 250 Ok: queued as 12345
S: 221 Bye
{The server closes the connection}

The client notifies the receiver of the originating e-mail address of the message in a MAIL FROM command. In this example, the email message is sent to two mailboxes on the same SMTP server: one each for each recipient listed in the To and Cc header fields. The corresponding SMTP command is RCPT TO. Each successful reception and execution of a command is acknowledged by the server with a result code and response message (e.g., 250 Ok).

The transmission of the body of the mail message is initiated with a DATA command after which it is transmitted verbatim line by line and is terminated with a characteristic sequence of a new line (<CR><LF>) with just a single full stop (period) followed by another line indication (<CR><LF>).

The QUIT command ends the session. If the second recipient were located elsewhere, the client would QUIT and connect to the appropriate SMTP server after the first message had been queued. The information that the client sends in the HELO and MAIL FROM commands are added (not seen in example code) as additional header fields to the message by the receiving server. It adds a Received and Return-Path header field, respectively.

Security and spamming

The original SMTP specification did not include a facility for authentication of senders. Subsequently, the SMTP-AUTH extension was defined by RFC 2554.[16] The SMTP extension (ESMTP) provides a mechanism for email clients to specify a security mechanism to a mail server, authenticate the exchange, and negotiate a security profile (Simple Authentication and Security Layer, SASL) for subsequent message transfers.

Microsoft products implement the proprietary Secure Password Authentication (SPA) protocol through the use of the SMTP-AUTH extension.

However, the impracticality of widespread SMTP-AUTH implementation and management means that E-mail spamming is not and cannot be addressed by it.

Modifying SMTP extensively, or replacing it completely, is not believed to be practical, due to the network effects of the huge installed base of SMTP. Internet Mail 2000 was one such proposal for replacement.

Spam is enabled by several factors, including vendors implementing broken MTAs (that do not adhere to standards, and therefore make it difficult for other MTAs to enforce standards), security vulnerabilities within the operating system (often exacerbated by always-on broadband connections) that allow spammers to remotely control end-user PCs and cause them to send spam, and a lack of "intelligence" in many MTAs.

There are a number of proposals for sideband protocols that will assist SMTP operation. The Anti-Spam Research Group (ASRG) of the Internet Research Task Force (IRTF) is working on a number of E-mail authentication and other proposals for providing simple source authentication that is flexible, lightweight, and scalable. Recent Internet Engineering Task Force (IETF) activities include MARID (2004) leading to two approved IETF experiments in 2005, and DomainKeys Identified Mail in 2006.


Linux Postfix SMTP (Mail Server) SSL Certificate Installations and Configuration

SkyHi @ Thursday, March 18, 2010

In this tutorial you will learn about Installing SSL Certificate (Secure Server Certificate) to secure communication between Postfix SMTP server and mail client such as Outlook or Thunderbird.

You need to generate a CSR certificate for CA, to use with your Postfix mail server. This tutorial instuctions are tested under:

  1. Redhat enterprise Linux 5
  2. CentOS 5 Server
  3. FreeBSD 7 server

Procedure for creating a CSR on postfix MTA is just like web server. You need to use OpenSSL which is a cryptography toolkit implementing the Secure Sockets Layer (SSL v2/v3) and Transport Layer Security (TLS v1) network protocols and related cryptography standards under Linux / UNIX. To configure postfix SSL SMTP you need 3 files

  • The private key generated using step #1
  • Your .crt certificate file (it will be send by CA)
  • CA certificate (also known as

Let us see how to create certificate for Postfix smtp server called

Step # 1: Generating a CSR and private key for Postfix SMTP

Type the command to create a SSL CSR for a mail server called
# mkdir /etc/postfix/ssl
# cd /etc/postfix/ssl
# openssl req -new -nodes -keyout -out

Most important is Common Name, in our example it is set to For the common name, you should enter the full mail server address of your site.

Sample output:

Generating a 1024 bit RSA private key
writing new private key to ''
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
Country Name (2 letter code) [AU]:<span style="color: rgb(255, 0, 0);"><strong>IN</strong></span>
State or Province Name (full name) [Some-State]:<span style="color: rgb(255, 0, 0);"><strong>MHA</strong></span>
Locality Name (eg, city) []:<span style="color: rgb(255, 0, 0);"><strong>Pune</strong></span>
Organization Name (eg, company) [Internet Widgits Pty Ltd]:<span style="color: rgb(255, 0, 0);"><strong>NIXCRAFT LTD</strong></span>
Organizational Unit Name (eg, section) []:<span style="color: rgb(255, 0, 0);"><strong>ITDEPT</strong></span>
Common Name (eg, YOUR name) []:<span style="color: rgb(255, 0, 0);"><strong></strong></span>
Email Address []:<span style="color: rgb(255, 0, 0);"><strong></strong></span>

Please enter the following 'extra' attributes
to be sent with your certificate request
A challenge password []:<span style="color: rgb(255, 0, 0);"><strong>myPassword</strong></span>
An optional company name []:<span style="color: rgb(255, 0, 0);"><strong>NIXCRAFT LTD</strong></span>

Step # 2: Submit CSR to CA

Now a CSR is generated. All you have to do is copy and paste the contents of the CSR file into the SSL certificate providers (aka CA) account. Never ever give out your private key or certificate to anyone. After verification you should receive a zip file with certificates.

Step # 3 : Install your SSL certificate

Unzip file and upload certificates to /etc/postfix/ssl directory.

Step # 4: Configure Postfix SMTP for SSL certificate

Open postfix smtp configuration file and append following directive:
smtpd_use_tls = yes
smtpd_tls_auth_only = yes
smtpd_tls_key_file = /etc/postfix/ssl/
smtpd_tls_cert_file = /etc/postfix/ssl/
smtpd_tls_CAfile = /etc/postfix/ssl/caroot.crt
smtpd_tls_loglevel = 1

smtpd_tls_received_header = yes
smtpd_tls_session_cache_timeout = 3600s
tls_random_source = dev:/dev/urandom

Save and close the file. Restart or reload postfix service
# postfix reload.
# /etc/init.d/postfix restart

Note I have SASL configured as follows in
smtpd_sasl_auth_enable = yes
smtpd_sasl_local_domain =
smtpd_recipient_restrictions = permit_sasl_authenticated, reject_unauth_destination, permit_mynetworks check_relay_domains
smtpd_delay_reject = yes
broken_sasl_auth_clients = yes

Test Postfix TLS (SSL)

In order to test the TLS, just telnet on port 25 (you must see STARTTLS and AUTH lines):
$ telnet 25

Trying<br />Connected to<br />Escape character is '^]'.<br />220 ESMTP Postfix<br /><span style="color: rgb(0, 153, 0);"><strong>ehlo</strong></span><br /><br />250-PIPELINING<br />250-SIZE 10240000<br />250-ETRN<br />250-<span style="color: rgb(255, 0, 0);"><strong>STARTTLS</strong></span><br />250-<span style="color: rgb(255, 0, 0);"><strong>AUTH PLAIN LOGIN</strong></span><br />250-AUTH=PLAIN LOGIN<br />250-ENHANCEDSTATUSCODES<br />250-8BITMIME<br />250 DSN

And mail log file...
# tail -f /var/log/maillog

Jul 12 14:25:10 smtp postfix/smtpd[28817]: connect from unknown[]<br />Jul 12 14:25:11 smtp postfix/smtpd[28817]: setting up TLS connection from unknown[]<br />Jul 12 14:25:11 smtp postfix/smtpd[28817]: TLS connection established from unknown[]: TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)<br />Jul 12 14:25:12 smtp postfix/smtpd[28817]: B3A0A9D8443: client=unknown[], sasl_method=PLAIN,<br />Jul 12 14:25:13 smtp postfix/cleanup[28807]: B3A0A9D8443: message-id=<><br />Jul 12 14:25:13 smtp postfix/qmgr[28806]: B3A0A9D8443: from=<>, size=632, nrcpt=1 (queue active)<br />Jul 12 14:25:14 smtp postfix/smtpd[28817]: disconnect from unknown[]<br />Jul 12 14:25:14 smtp postfix/smtp[28821]: B3A0A9D8443: to=<>,[]:25, delay=2.1, delays=1.5/0/0.13/0.49, dsn=2.0.0, status=sent (250 2.0.0 OK 1184268314 n29si21297786elf)<br /><br /><br />Jul 12 14:25:14 smtp postfix/qmgr[28806]: B3A0A9D8443: removed</></>

You can use same SSL security certificates with dovecot secure IMAPS / POP3S server.

See also:

If you just need self signed postfix SSL certificate please see this tutorial for more information.

Updated for accuracy.


Wednesday, March 17, 2010

Sendmail support for relaying denied and did not issue MAIL/EXPN/VRFY/ETRN

SkyHi @ Wednesday, March 17, 2010

Sendmail did not issue MAIL/EXPN/VRFY/ETRN during connection to MTA

Possible Fix: The client may not be authenticating properly with the mail server. If the client authentication is selected then try setting the authentication mode to "pop then smtp"

(may be forged), reject=550 ... Relaying denied

Possible Fix: Often you will get relay denied because of DNS issues check that you have added all the host names in the /etc/hosts file and that the server can resolve the hostname for your MX record and all host names associated with the domain and IP addresses.


The notice "MAIL/EXPN/VRFY/ETRN during connection to MTA" usually indicates a dead connection. Meaning that they connected to your smtp port and just quiting out. You can duplicate this message yourself just by telneting to your smtp server and typing quit after it sends it's connection notice. This doesn't mean that they're not sending anything to the smtp server, just that it doesn't recognize what they've sent as a valid command at that point in the process.

We had this problem with a RHEL4 sendmail server and fixed it by going into the account settings for the outgoing server (SMTP) and unticking the 'Use name and password' box.