Friday, February 10, 2012

CentOS SSH X11 Forwarding

SkyHi @ Friday, February 10, 2012
To setup SSH X11 forwarding on CentOS, we need to:
0. To install KDE Desktop, type this:
yum groupinstall "X Window System" "KDE (K Desktop Environment)"
To install Gnome Desktop, type this:
yum groupinstall "X Window System" "GNOME Desktop Environment"
After the installation is finished, type this to start KDE or GNOME:
1. On the remote server, edit /etc/ssh/sshd_config, and set the following values:
X11Forwarding yes
X11DisplayOffset 10
X11UseLocalhost yes
2. On the remote server, install the package xorg-x11-xauth:
# yum install xorg-x11-xauth
3. On the remote server, install the fontconfig packages:
# yum install fonts-xorg-base (CentOS 4)
# yum install xorg-x11-fonts-base liberation-fonts (CentOS 5)
4. Now login to the remote server using "-Y" option:
local$ ssh -Y remote_server
5. In the remote server, run your X program, such as the xterm:
remote$ xterm
And you shall see the X program pop up in your local desktop.
You can also add the following into your $HOME/.ssh/config:
ForwardAgent yes
ForwardX11 yes
ForwardX11Trusted yes
6. service sshd restart

Requirements for Remotely Displaying Applications

In order to run an application on one Linux system and have it display on another system there are a couple of prerequisites.
Firstly, the system on which the application is to be displayed must be running an X server. If the system is a Mac OS X, UNIX orLinux based system with a desktop environment running then this is no problem. If the system is running Windows, however, then you must install an X server on it before you can display applications from a remote system. A number of commercial and free Windows based X servers are available for this purpose and a web search should provide you with a list of options.
Secondly, the system on which the application is being run (as opposed to the system which the application is to be displayed) must be configured to allow SSH access. Details on configuring SSH on a CentOS system can be found in the chapter entitledConfiguring CentOS Remote Access using SSH. Finally, SSH must be configured to allow X11 Forwarding. To verify this, load the/etc/ssh/ssh_config file into an editor and make sure that the following directive is set:
X11Forward yes
Once the above requirements are met it is time to remotely display an application.

Remotely Displaying a CentOS Application

The first step in remotely displaying an application is to move to the system where the application is to be displayed. At this system, ssh into the remote system so that you have a command prompt. This can be achieved using the ssh command. When using the ssh command we need to use the -X flag to tell ssh that we plan to tunnel X traffic through the connection:
ssh -X user@hostname
In the above example username is the user name to use to log into the remote system and hostname is the hostname or IP address of the remote system. Enter your password at the login prompt. Once logged in, run the following command to see the DISPLAY setting:
The command should output something similar to the following:
To display an application simply run it from the command prompt. For example:
When run, the above command should run the gedit tool on the remote system, but display the output on the local system.

Trusted X11 Forwarding

If the /etc/ssh/ssh_config file on the remote system contains the following line, then it is possible to use trusted X11 forwarding:
ForwardX11Trusted yes
Trusted X11 forwarding is slightly faster than untrusted forwarding since it does not engage the X11 security controls. The -Y flag is needed when using trusted X11 forwarding:
ssh -Y user@hostname

Compressed X11 Forwarding

When using slower links the X11 data can be compressed using the -C flag:
ssh -X -C user@hostname

Tuesday, February 7, 2012

convert html to pdf linux

SkyHi @ Tuesday, February 07, 2012
I blogged a while back about delivering pages as PDF using PHP, and at the time DOMPDF seemed to be the best-of-breed package for converting HTML into PDF for the purposes of delivering PDF versions of web content.
However, I noted at the time that DOMPDF's last release was in July 2007, and it still doesn't look like being updated any time soon. The fundamental problem with packages like DOMPDF is that they tend to implement their own rendering engine. The thing is, HTML and CSS are both pretty huge now - writing a rendering engine that can cope with all the different combinations is a huge task, so projects like DOMPDF end up missing out important bits of functionality.
A better approach would be to use an existing rendering engine from a browser, and then build a binary around it that can take a website as input and produce a PDF as output. That way you can get results consistent with how browsers would print a page and if you pick the right engine you'll not have to keep up with any changes to HTML standards, the engine developers will do that for you.
This is essentially the approach wkhtmltopdf takes: it extracts the open-sourced Webkit renderer used inside browsers like Safari and Chrome and bundles it up into a Linux CLI application which produces some pretty impressive results.
I thought I'd jump right in and start by compiling it on my Debian webserver. The wkhtmltopdf site has some instructions for building it on Ubuntu, which I thought were worth a try. The basic procedure was as follows:
#apt-get update
#apt-get install libqt4-dev qt4-dev-tools build-essential cmake

#svn checkout wkhtmltopdf
#cd wkhtmltopdf
#sudo make install
In my case, this installed a terrifying amount of new packages to my server, but everything went very smoothly. I was left with a binary in /usr/bin and ploughed right in!
#wkhtmltopdf /tmp/ciaranmcnulty.pdf
wkhtmltopdf: cannot connect to X server
Argh. The rendering engine depends on there being a GUI running on the machine so it can do cool things like generate graphics, render fonts and so forth. A typical webserver won't be running X, but luckily there are ways around it.
One such way is xvfb, or the X Virtual Frame Buffer. This is a handy bit of code that basically runs an X instance but without a lot of the overheads. You can create a temporary X buffer and run a command in it using the xvfb-run binary, the benefit of which is that the x instance gets thrown away afterwards. I installed xvfb and then invoked it as follows:
#apt-get install vfb
#xvfb-run -a -s "-screen 0 640x480x16" wkhtmltopdf --dpi 200 
  --page-size A4 /tmp/ciaranmcnulty.pdf
The options should be fairly self-explanatory, the key things to note are that -a makes xvfb pick an unused display number (to avoid collisions) and -screen starts up the virtual framebuffer with a display with the correct bit depth and dimensions.
The results are fairly good, certainly better than PHPDOM would generate given the same input. My site layout uses a fair bit of floating and absolute positioning, and the PDF came out exactly as I'd expect:
Website PDF
It's important to note that this isn't a bitmap, the text in the PDF is still 'text'.
A quick dig around showed that to print the backgrounds I'd need to have Qt4.5 installed, something I wasn't really prepared to risk my server for. However, I thought I'd quickly try doing what I should have in the first place. The wkhtml project provides a linux binary that's statically compiled against Qt.
I downloaded this binary and gave it a whirl. The results were much better:
Website PDF with backgrounds
Frankly I think this is a great rendition of the page, and certainly good enough for an autogenerated PDF on a website. A bit of further investigation and experimentation has left me pretty impressed with the breadth of CSS print functionality webkit can support.
The next step for me is going to be to try and replace some of the DOMPDF installations in some of my smaller sites, and see how it performs under load. The time taken to generate a PDF is pretty high, and I've not really checked out how xvfb is with concurrency so I'd hesitate to throw it onto a production site straight away, but it'll be my first port of call next time I want to do something with a PDF.

Use this to install:
In a previous post I wrote about using wkhtmltopdf for html to pdf conversion with Ruby and Rails.

As can by typical with components, moving them to production you hope will be straight forward but is often not. Such was the case for me on getting wkthmltopdf working on my Ubuntu Server 10.04 server from my Mac Leopard dev environment.

The first question was how to install wkhtmltopdf on the server. Since **I had not been successful** installing it on my own on my Mac (I used the PdfKit ruby gem to install it), it was not clear if I would succeed here.

I ended up finding that there is a package for wkhtmltopdf for Ubuntu:
sudo apt-get install wkthmltopdf
This package did its work and it installed. It seemed too easy. And it was.

While wkhtmltopdf (v0.9.9) did install, I was soon getting the following and dreaded error:
Cannot connect to X server
The reason for this error is that the current incarnation of Web Kit requires a GUI. Hopefully this will change in the future.

After some research I found what looked like a solution:

Use Xvfb (‘X Virtual Frame Buffer’). Xvfb promised to create a lightweight, temporary situation that would trick wkhtmltopdf into running. Please excuse my un-technical and probably inaccurate description of what Xvfb does, but you get the point.

So I did:
sudo apt-get install xvfb
And in the terminal I now could run wkhtmltopdf and see an output:
xvfb-run -a -s "-screen 0 640x480x16" wkhtmltopdf path_html path_pdf
It worked also from my Rails app:
%x[xvfb-run -a -s "-screen 0 640x480x16" wkhtmltopdf #{path_html} #{path_pdf}]
It worked great! At least until…. I discovered that those darned links in the converted document were not active. I could not find an answer for this, so kept googling. My sense was that this problem must have something to do with running wkhtmltopdf through xvfb.

I ended up being right, and the solution was to use a patched QT in lieu of xvfb.

So I decided to try installing the static binary of wkhtmltopdf as follows:
  1. Uninstall the wkhtmltopdf package:
    apt-get remove wkhtmltopdf
  2. (in usr/local/bin)
    sudo curl -C - -O
  3. (in usr/local/bin)
    sudo tar -xvjf wkhtmltopdf-0.9.9-static-amd64.tar.bz2
**I originally was trying to install the wrong wkhtmltopdf static binary for my machine.** I have 64bit Linux and it was not obvious to me that I should use the binary labelled ‘amd’. I thank Michael Schuerig on the Rails Google Group for his insight. So make sure you have the right one for your machine. Initially when I had the wrong binary installed, when I would run it from the terminal, it looked like it executed (no error), but with no output. Also thanks to Michael, I ran:
strace wkhtmltopdf #{path_html} #{path_pdf}

This showed me that a certain linux source file was missing and led to resolving the problem. In short, if one of the binaries does not work, try the other.

Once I got this installed everything worked, links were rendering and the client happy.


Issue Summary

I had some minor problems getting wkhtmltopdf running that might stump some people, so here are my instructions on getting it running on Ubuntu 10.04. These instructions perhaps should be added to INSTRUCTIONS.txt
1. Download wkhtmltopdf.
2. Extract it and move it to /usr/bin/
3. Rename it to wkhtmltopdf so that now you have an executable at /usr/bin/wkhtmltopdf
4. Set permissions: sudo chmod a+x /usr/bin/wkhtmltopdf
4. Install required support packages. sudo apt-get install openssl build-essential xorg libssl-dev
5. Check to see if it works: run wkhtmltopdf test.pdf. If it works, then you are done -- make sure to make a symbolic link as per INSTRUCTIONS.txt. If you get the error "Cannot connect to X server" then continue to number 6.
6. We need to run it headless on a 'virtual' x server. We will do this with a package called xvfb. sudo apt-get install xvfb
7. We need to write a little shell script to wrap wkhtmltopdf in xvfb. Make a file called and add the following:
xvfb-run -a -s "-screen 0 640x480x16" wkhtmltopdf $*
8. Move this shell script to /usr/bin, and set permissions: sudo chmod a+x /usr/bin/
9. Finally, make your symbolic link in /sites/all/modules/print/lib. Command is ln -s /usr/bin/ wkhtmltopdf


Monday, February 6, 2012

Moving Email from PC Outlook to Apple Mail

SkyHi @ Monday, February 06, 2012
his is a common question asked by Windows switchers, and the solution isn't as easy as it should be. There are several methods each with their own pros and cons. Choose the one that best meets your needs.




An easy (and free) way to transfer email from Outlook (Windows) to Mail (Mac) is to use an IMAP account (e.g. Gmail) as an intermediary between the two. It's simple and reliable but can be a bit slow for large amounts of email. Expect over an hour per Gigabyte.
  1. Enable IMAP on your email account. [1]
  2. In Windows, open Outlook and add your IMAP account. [2]
  3. Copy the folders (don't drag and drop them) from your PST file into your IMAP account. Drag and drop will move (not copy) your folders and you may lose mail if the communication is interrupted during the move.
  4. On the Mac, open Mail and add your IMAP account. [3]
  5. If you want your mail stored on your Mac, you can copy it from your IMAP account to mailboxes created by selecting "On My Mac" as the location in the new mailbox dialog inside the Mail application.


For another fairly easy migration, you can use Eight Hoof's 8Convert program ($14 shareware), which runs on your PC and directly exports files for use with Apple Mail. Theuserguide gives instructions on everything from install to file copying to importing into Apple Mail.
The program exports Outlook data to industry-standard mbox, vCard, and iCal formats that can be imported into Apple's Mail, Contacts, and Calendar programs, as well as compatible programs such as Entourage.


This alternative is free, mostly automated (no renaming of folders) but it requires a separate application, i.e. Thunderbird.
  1. Install Thunderbird. On first launch, tell it to import from Outlook (not Outlook Express).
  2. It will import account settings, contacts and email.
  3. Copy your Thunderbird profile (XP: C:\Documents and Settings\\Application Data\Thunderbird\Profiles\.default, Vista/7:C:\Users\\AppData\Roaming\Thunderbird\Profiles\.default) to your Mac. Several methods can be used to perform the copy operation:
    1. If both computers are on the same network, file sharing can be used.
    2. If the computers are NOT on the same network, the fastest approach is to zip-compress the profile folder in Windows, put it on an external storage device, bring the device to your Mac, connect it, and decompress the zip file.
  4. Start the Mail application and go to File > Import.
  5. Select Thunderbird and point it to the profile folder you copied to your Mac. The imported mail will be under "On My Mac".
  6. To be safe, you can go to "Mailbox > Rebuild" inside Mail to make sure there's no corruption resulting from the import.

Outlook Express

This option can be done without the need for installing additional software on your PC but it will require Entourage (or Outlook) on your Mac which you may not have.
  1. Open Outlook Express on your PC.
  2. Highlight all mail in a folder e.g. "important mail", and drag them into a any folder on your hard drive (ideally - outlook mail/important). Having Outlook run in window mode (not fullscreen) and dragging the mail to a folder on your desktop seems to be the easiest way, because you don't have to switch windows.
  3. Make as many folders and fill them this way as you have/need, then copy them to your Mac.
  4. Start Entourage (or Outlook) on your Mac (if you have either of them).
  5. Drag them into a folder the same way as you exported them on your pc.
  6. Now you should have the mail in Entourage.
  7. Mail can now import mailboxes from Entourage.
Once you've imported all your email into the Mail application, you can delete all the other email from your desktop/Outlook/Entourage to save space.