Tuesday, March 25, 2014

Mounting a remote linux directory onto a windows machine for editing files

Category: — SkyHi @ Tuesday, March 25, 2014

Introduction

In many cases it can become cumbersome to transfer files to and from a droplet. Imagine a development usage scenario where you are coding apps remotely and find yourself uploading a script repeatedly to your virtual server to test. This can become quite a hassle in a very short period of time. Luckily there is a way to mount your VPS file system to your local computer so you can make changes on the fly and treat your droplet as local storage. In this article, we will show you how to do exactly that.

Installing SSHFS

On Ubuntu/Debian

SSHFS is Linux based software that needs to be installed on your local computer. On Ubuntu and Debian based systems it can be installed through apt-get.
sudo apt-get install sshfs

On Mac OSX

You can install SHFS on Mac OSX. You will need to download FUSE and SSHFS from the osxfuse site

On Windows

To install SSHFS in Windows you will need to grab the latest win-sshfs package from the google code repository. A direct download link can be found below. After you have downloaded the package, double click to launch the installer. You may be prompted to download additional files, if so the installer will download the .NET Framework 4.0 and install it for you. 
https://win-sshfs.googlecode.com/files/win-sshfs-0.0.1.5-setup.exe

Mounting the Remote File System

The following instructions will work for both Ubuntu/Debian and OSX. Instructions for Windows systems can be found at the bottom of the tutorial.
To start we will need to create a local directory in which to mount the droplet's file system.
sudo mkdir /mnt/droplet <--replace code="" droplet="" prefer="" whatever="" you="">
Now we can use sshfs to mount the file system locally with the following command. If your VPS was created with a password login the following command will do the trick. You will be asked for your virtual server's root password during this step.
sudo sshfs root@xxx.xxx.xxx.xxx:/ /mnt/droplet
If your droplet is configured for login via ssh key authorization, you will need to tell sshfs to use your public keys with the following command. You will be asked to enter the passphrase you used during the creation of your keys with ssh-keygen. 
sudo sshfs -o IdentityFile=~/.ssh/id_rsa root@xxx.xxx.xxx.xxx:/ /mnt/droplet
Now you can work with files on your droplet as if it were a physical device attached to your local machine. For instance, if you move to the /mnt/droplet directory on your local machine you can create a file locally and the file will appear on your virtual server. Likewise you can copy files into the /mnt/droplet folder and they will be uploaded to your droplet in the background.
It is important to note that this process provides only a temporary mount point to your droplet. If the virtual server or local machine is powered off or restarted, you will need to use the same process to mount it again.

Unmounting the Remote File System

When you no longer need the mount point you can simply unmount it with the command
sudo umount /mnt/droplet

Permanently Mounting the Remote File System

SSHFS also allows for setting up permanent mount points to remote file systems. This would set a mount point that would persist through restarts of both your local machine and droplets. In order to set up a permanent mount point, we will need to edit the /etc/fstab file on the local machine to automatically mount the file system each time the system is booted.
First we need to edit the /etc/fstab file with a text editor.
sudo nano /etc/fstab
Scroll to the bottom of the file and add the following entry
sshfs#root@xxx.xxx.xxx.xxx:/ /mnt/droplet
Save the changes to /etc/fstab and reboot if necessary.
It should be noted that permanently mounting your VPS file system locally is a potential security risk. If your local machine is compromised it allows for a direct route to your droplet. Therefore it is not recommended to setup permanent mounts on production servers.

Using Win-SSHFS to Mount Remote File Systems on Windows

After launching the win-sshfs program, you will be presented with a graphical interface to make the process of mounting a remote file share simple.
  • Step 1: Click the Add button in the lower left corner of the window.
  • Step 2: Enter a name for the file share in the Drive Name field.
  • Step 3. Enter the IP of your droplet in the Host field.
  • Step 4. Enter your SSH port. (Leave as port 22 unless you have changed the SSH port manually).
  • Step 5. Enter your username in the Username field. (Unless you have set up user accounts manually you will enter root in this field).
  • Step 6. Enter your SSH password in the password field. (Note on Windows you will need to have your droplet configured for password logins rather than ssh-key-authentication).
  • Step 7. Enter your desired mount point in the Directory field. (Enter / to mount the file system from root. Likewise you can enter /var/www or ~/ for your home directory).
  • Step 8. Select the drive letter you would like Windows to use for your droplets file system.
  • Step 9. Click the Mount button to connect to the droplet and mount the file system.
Now your virtual server's file system will be available through My Computer as the drive letter you chose in step 8.

Usage of the Remote Mount Point

The remote mount behaves similarly to locally mounted storage: you are able to create, copy, move, edit, compress or perform any file system operations you would be able to do on the droplet, but you are not able to launch programs or scripts on the remote server.
One typical usage of this would be if you host a website on your VPS and need to make changes to the website on a regular basis. Mounting the file system locally allows you to launch whatever code editor, IDE, or text editor you wish to edit the site, and any changes you make will reflect on the virtual server as soon as they are made on your local machine.
Similarly, on droplets used for testing purposes of coding projects, it allows for much simpler code modifications which can be tested immediately without the need to modify the code locally as well as remotely (and eliminates the hassle of uploading new copies of files for small code changes).

REFERENCES
https://www.digitalocean.com/community/articles/how-to-use-sshfs-to-mount-remote-file-systems-over-ssh

Friday, December 20, 2013

Configuring postfix to forward all email to a smtp gateway

Category: — SkyHi @ Friday, December 20, 2013

Introduction

Suppose you want all your web servers to locally send all email (maybe from your contact forms, or whatever) to a real smtp gateway. If you're running postfix as your mta, this is quite easily to achieve.

Configuration

In your main.cf file (usually /etc/postfix/main.cf or /usr/local/etc/postfix/main.cf), specify your transport maps:
view plaincopy to clipboardprint?
  1. transport_maps =  hash:/etc/postfix/transport  
And in your transport file (usually /etc/postfix/transport or /usr/local/etc/postfix/transport), specify your gateways per domain. This allows you to specify with regular expressions which emails go where. In this case, an asterisk specifies every domain (the transport file allows very complex setups, this is of course an extreme and trivial example):
view plaincopy to clipboardprint?
  1. *              smtp:myotherhost.com  
Then invoke postmap to regenerate the transports db, and reload postfix:
view plaincopy to clipboardprint?
  1. # postmap hash:/etc/postfix/transport  
  2. # postfix reload  
That should do it. All email generated in this host should now be forwarded to the smtp gateway. Remember to configure your smtp gateway to accept mail from all the hosts that will forward emails to it.

Volts / Watts / Amps Converter

Category: — SkyHi @ Friday, December 20, 2013
How to use this tool:
Watts = Amps x Volts
Watts is also known as volt-amps and is typically used in conjunction with AC power circuits. Fill in any of the two fields to find the value of the third.
Example 1
You have a 12 Volt power supply that delivers 1 Amp of current. Fill in the Volts and Amps fields to find the Watts.
Example 2
The AC24-40 power supply is a 24V AC power supply that can power up to 40 VA.
  1. Enter 24 under volts
  2. Enter 40 under watts
  3. Click calculate
  4. You get 1.66 in this example.
Thus, the AC24-40 can supply up to 1.6 Amps at 24V AC.

 References:

Thursday, September 12, 2013

How to detect the physical connected state of a network cable/connector?

Category: — SkyHi @ Thursday, September 12, 2013
You can use ethtool:
$ sudo ethtool eth0
Settings for eth0:
    Supported ports: [ TP ]
    Supported link modes:   10baseT/Half 10baseT/Full
                            100baseT/Half 100baseT/Full
                            1000baseT/Full
    Supports auto-negotiation: Yes
    Advertised link modes:  10baseT/Half 10baseT/Full
                            100baseT/Half 100baseT/Full
                            1000baseT/Full
    Advertised auto-negotiation: Yes
    Speed: 1000Mb/s
    Duplex: Full
    Port: Twisted Pair
    PHYAD: 0
    Transceiver: internal
    Auto-negotiation: on
    Supports Wake-on: umbg
    Wake-on: g
    Current message level: 0x00000007 (7)
    Link detected: yes
To only get the Link status you can use grep:
$ sudo ethtool eth0 | grep Link
    Link detected: yes

REFERENCES
http://serverfault.com/questions/15776/how-to-check-the-physical-status-of-an-ethernet-port-in-linux

Friday, August 30, 2013

mount error(115): Operation now in progress ... CIFS VFS: cifs_mount failed w/return code = -115

Category: — SkyHi @ Friday, August 30, 2013
Trying to mount a CIFS path in Ubuntu and getting:
?
1
mount error(115): Operation now in progress
First thing to try is to look into /var/log/syslog:
?
1
2
Feb  9 14:08:29 ldap kernel: [143452.140157] CIFS VFS: Error connecting to socket. Aborting operation
Feb  9 14:08:29 ldap kernel: [143452.140492] CIFS VFS: cifs_mount failed w/return code = -115
Socket error, we know what this is right? IP or port. Ping for domain/IP or Telnet to test port:
?
1
2
ping IP
telnet IP 445
My case? Telnet was timing out, port closed for IP in firewall.

REFERENCES
http://thinkinginsoftware.blogspot.com/2013/02/mount-error115-operation-now-in.html

How do maintain a constant mac address on bonded NICs?

Category: — SkyHi @ Friday, August 30, 2013
In 10.04 this consistenlty create bond0 with the mac address from eth0. with 12.04, the mac address used appears to be random - sometimes it's the address from eth0, sometimes it's the address from eth1.


After trying MakOwner's solution to no avail, I think I solved it for my case.

At bootup I noticed in dmesg that eth1 was coming up before eth0, probably due to some unpredictable hardware timing. To make things worse, I saw that the link states for eth0 and eth1 would sometimes go up a few seconds after bond0 went up. Perhaps the bond was polling for the link status of eth0 and eth1, cycling through each until one came up. Whichever was chosen first depended on timing.

My solution was to add a pre-up sleep command to eth1. This effectively delays it long enough to guarantee that eth0 is selected first.

Here is my configuration:

Code:
auto eth0
    iface eth0 inet manual
    bond-master  bond0

auto eth1
    iface eth1 inet manual
    # delay ifup to allow eth0 to come up first in the bond
    pre-up sleep 4
    bond-master  bond0

auto bond0
    iface bond0 inet dhcp
    bond-mode active-backup
    bond-miimon 100
    bond-slaves eth0 eth1


REFERENCES
http://ubuntuforums.org/showthread.php?t=1967987

Monday, August 26, 2013

ffmpeg Cheat Sheet

Category: — SkyHi @ Monday, August 26, 2013

Video Size and Aspect Ratios

iPhone 3 (480x320):
Name Size Aspect
Wide 16:9 480x270 16:9
Old TV 4:3 427x320 4:3
Wide 16:9 /16 480x272 16:9
Old TV 4:3 /16 432x320 4:3
iPhone 4 (960x640):
Name Size Aspect
Wide 16:9 960x540 16:9
Old TV 4:3 853x640 4:3
Wide 16:9 /16 960x544 16:9
Old TV 4:3 /16 848x640 4:3
iPhone 5 (1136x640):
Name Size Aspect
Wide 16:9 1136x639 16:9
Old TV 4:3 856x640 4:3
Wide 16:9 /16 1136x640 16:9
Old TV 4:3 /16 864x640 4:3
DV NTSC Video
Name Size Aspect
Wide 16:9 720x480 16:9
Old TV 4:3 720x480 4:3
Youtube and Vimeo Upload
Name Size Aspect
SD 4:3 640:480 4:3
HD 16:9 1280x720 16:9
Full HD 1920x1080 (1920x1072 base 16) 16:9
HD, Blu-Ray, AVCHD
Name Size Aspect
720p 1280x720 16:9
1080p/i Anamorphic 1440x1080 16:9
1080p/i 1920x1080 16:9

Frame Rates

Name Standard Fps FFmpeg
Movies NTSC 23.976 24000/1001
TV Video NTSC 29.97 30000/1001
Movies PAL 25 25
TV Video PAL 30 30

FFmpeg Encoding

X264Presets
default fastfirstpass lossless_ultrafast lossless_slower
baseline slowfirstpass lossless_fast lossless_max
normal hq lossless_medium ipod320
main max lossless_slow ipod640
High quality 2 Pass
ffmpeg -y -i INPUT -r 30000/1001 -b 2M -bt 4M -vcodec libx264 -pass 1 -vpre fastfirstpass -an output.mp4

ffmpeg -y -i INPUT -r 30000/1001 -b 2M -bt 4M -vcodec libx264 -pass 2 -vpre hq -acodec libfaac -ac 2 -ar 48000 -ab 192k output.mp4
iPod-iPhone 320 width
ffmpeg -i INPUT -s 320x240 -r 30000/1001 -b 200k -bt 240k -vcodec libx264 -vpre ipod320 -acodec libfaac -ac 2 -ar 48000 -ab 192k output.mp4
iPod-iPhone 640 width
ffmpeg -i INPUT -s 640x480 -r 30000/1001 -b 200k -bt 240k -vcodec libx264 -vpre ipod640 -acodec libfaac -ac 2 -ar 48000 -ab 192k output.mp4
Hight quality, 2 pass without presset
ffmpeg -y -i input -r 24000/1001 -b 6144k -bt 8192k -vcodec libx264 -pass 1 -flags +loop -me_method dia -g 250 -qcomp 0.6 -qmin 10 -qmax 51 -qdiff 4 -bf 16 -b_strategy 1 -i_qfactor 0.71 -cmp +chroma -subq 1 -me_range 16 -coder 1 -sc_threshold 40 -flags2 -bpyramid-wpred-mixed_refs-dct8x8+fastpskip -keyint_min 25 -refs 1 -trellis 0 -directpred 1 -partitions -parti8x8-parti4x4-partp8x8-partp4x4-partb8x8-an output.mp4
ffmpeg -y -i input -r 24000/1001 -b 6144k -bt 8192k -vcodec libx264 -pass 2 -flags +loop -me_method umh -g 250 -qcomp 0.6 -qmin 10 -qmax 51 -qdiff 4 -bf 16 -b_strategy 1 -i_qfactor 0.71 -cmp +chroma -subq 8 -me_range 16 -coder 1 -sc_threshold 40 -flags2 +bpyramid+wpred+mixed_refs+dct8x8+fastpskip -keyint_min 25 -refs 4 -trellis 1 -directpred 3 -partitions +parti8x8+parti4x4+partp8x8+partb8x8-acodec libfaac -ac 2 -ar 44100 -ab 128k output.mp4
iPod-iPhone 320 width, without presset
ffmpeg -i INPUT -s 320x240 -r 30000/1001 -b 200k -bt 240k -vcodec libx264 -coder 0 -bf 0 -flags2 -wpred-dct8x8 -level 13 -maxrate 768k -bufsize 3M-acodec libfaac -ac 2 -ar 48000 -ab 192k output.mp4
iPod-iPhone 640 width, without presset
ffmpeg -i INPUT -s 320x240 -r 30000/1001 -b 200k -bt 240k -vcodec libx264 coder 0 -bf 0 -refs 1 -flags2 -wpred-dct8x8 -level 30 -maxrate 10M -bufsize 10M-acodec libfaac -ac 2 -ar 48000 -ab 192k output.mp4
iPod-iPhone, 2 pass custom
ffmpeg -y -i input -r 30000/1001 -s 480x272 -aspect 480:272 -vcodec libx264 -b 512k -bt 1024k -maxrate 4M -flags +loop -cmp +chroma -me_range 16 -g 300 -keyint_min 25 -sc_threshold 40 -i_qfactor 0.71 -rc_eq "blurCplx^(1-qComp)" -qcomp 0.6 -qmin 10 -qmax 51 -qdiff 4 -coder 0 -refs 1 -bufsize 4M -level 21 -partitions parti4x4+partp8x8+partb8x8 -subq 5 -f mp4 -pass 1 -an -title "Title" output.mp4
ffmpeg -y -i input -r 30000/1001 -s 480x272 -aspect 480:272 -vcodec libx264 -b 512k -bt 1024k -maxrate 4M -flags +loop -cmp +chroma -me_range 16 -g 300 keyint_min 25 -sc_threshold 40 -i_qfactor 0.71 -rc_eq "blurCplx^(1-qComp)" -qcomp 0.6 -qmin 10 -qmax 51 -qdiff 4 -coder 0 -refs 1 -bufsize 4M -level 21 -partitions parti4x4+partp8x8+partb8x8 -subq 5 -f mp4 -pass 2 -acodec libfaac -ac 2 -ar 44100 -ab 128k -title "Title" output.mp4

DV Video

To DV
ffmpeg -i input -target ntsc-dv -aspect 4:3 -y output.dv
From DV To Microsoft AVI DV
ffmpeg -i input.dv -vcodec copy -vtag dvsd -acodec pcm_s16le -f avi -aspect 4:3 -y output.avi

Audio

AAC Stereo HQ 
-acodec libfaac -ac 2 -ar 48000 -ab 192k
AAC Stereo SQ 
-acodec libfaac -ac 2 -ar 44100 -ab 128k
AAC 5.1 HQ 
-acodec libfaac -ac 6 -ar 48000 -ab 448k
AAC 5.1 LQ 
-acodec libfaac -ac 6 -ar 44100 -ab 224k

Eac3To

AC3 5.1 Channel Mapping Fix
eac3to from_true_hd.ac3 fixed_mapping.ac3 -blu-ray -448
AAC to AC3
  1. From AAC to WAV with faad2: 
    faad2 -o temp_audio.wav input.aac
  2. From WAV to AC3 with eac3to 
    eac3to temp_audio.wav output.ac3 -448
AC3 to AAC
  1. From AC3 to a remapped AC3 with eac3to: 
    eac3to input.ac3 temp_remaped.ac3 -448 -resampleTo48000 -2,1,0,3,4,5
  2. From remapped AC3 to AAC with FFmpeg 
    ffmpeg -y -i temp_remaped.ac3 -vn -acodec libfaac -ac 6 -ar 48000 -ab 384k output.m4a

Other FFmpeg Options

-threads thread count, 0 means all threads available
-y overwrite output files
-ss time_off set the start time offset
-t duration record or transcode "duration" seconds of audio/video
-ildct Interlaced

FFmpeg Information Sources

IRC
Server: irc.freenode.net
Channel: #ffmpeg

Download

The bests binaries on the web, Easy installation without compiling, just runing an install file:
FFmpeg for Windows
FFmpeg for Macintosh OS X (Intel)
FFmpeg for Linux
You can also use this extremely easy guide to install FFmpeg on your Windows, GNU Linux or Macintosh machine:
http://rodrigopolo.com/about/wp-stream-video/ffmpeg-binary-installers-for-win-mac-and-linux
Win32 Binary Builds
http://ffmpeg.arrozcru.com/autobuilds
http://sourceforge.net/project/showfiles.php?group_id=205275&package_id=248632
FFmpeg x264 encoding guide by Robert Swain
http://rob.opendot.cl/index.php/useful-stuff/ffmpeg-x264-encoding-guide

H.264 Profiles and Levels
8x8dct high
cabac+bframes main
-level #



video conversion, sound extraction, encoding file for iPod or PSP, and more.
Getting infos from a video file
ffmpeg -i video.avi
Turn X images to a video sequence
ffmpeg -f image2 -i image%d.jpg video.mpg
This command will transform all the images from the current directory (named image1.jpg, image2.jpg, etc…) to a video file named video.mpg.
Turn a video to X images
ffmpeg -i video.mpg image%d.jpg
This command will generate the files named image1.jpg, image2.jpg, …
The following image formats are also availables : PGM, PPM, PAM, PGMYUV, JPEG, GIF, PNG, TIFF, SGI.
Encode a video sequence for the iPpod/iPhone
ffmpeg -i source_video.avi input -acodec aac -ab 128kb -vcodec mpeg4 -b 1200kb -mbd 2 -flags +4mv+trell -aic 2 -cmp 2 -subcmp 2 -s 320x180 -title X final_video.mp4
Explanations :
  • Source : source_video.avi
  • Audio codec : aac
  • Audio bitrate : 128kb/s
  • Video codec : mpeg4
  • Video bitrate : 1200kb/s
  • Video size : 320px par 180px
  • Generated video : final_video.mp4
Encode video for the PSP
ffmpeg -i source_video.avi -b 300 -s 320x240 -vcodec xvid -ab 32 -ar 24000 -acodec aac final_video.mp4
Explanations :
  • Source : source_video.avi
  • Audio codec : aac
  • Audio bitrate : 32kb/s
  • Video codec : xvid
  • Video bitrate : 1200kb/s
  • Video size : 320px par 180px
  • Generated video : final_video.mp4
Extracting sound from a video, and save it as Mp3
ffmpeg -i source_video.avi -vn -ar 44100 -ac 2 -ab 192 -f mp3 sound.mp3
Explanations :
  • Source video : source_video.avi
  • Audio bitrate : 192kb/s
  • output format : mp3
  • Generated sound : sound.mp3
Convert a wav file to Mp3
ffmpeg -i son_origine.avi -vn -ar 44100 -ac 2 -ab 192 -f mp3 son_final.mp3
Convert .avi video to .mpg
ffmpeg -i video_origine.avi video_finale.mpg
Convert .mpg to .avi
ffmpeg -i video_origine.mpg video_finale.avi
Convert .avi to animated gif(uncompressed)
ffmpeg -i video_origine.avi gif_anime.gif
Mix a video with a sound file
ffmpeg -i son.wav -i video_origine.avi video_finale.mpg
Convert .avi to .flv
ffmpeg -i video_origine.avi -ab 56 -ar 44100 -b 200 -r 15 -s 320x240 -f flv video_finale.flv
Convert .avi to dv
ffmpeg -i video_origine.avi -s pal -r pal -aspect 4:3 -ar 48000 -ac 2 video_finale.dv
Or:
ffmpeg -i video_origine.avi -target pal-dv video_finale.dv
Convert .avi to mpeg for dvd players
ffmpeg -i source_video.avi -target pal-dvd -ps 2000000000 -aspect 16:9 finale_video.mpeg
Explanations :
  • target pal-dvd : Output format
  • ps 2000000000 maximum size for the output file, in bits (here, 2 Gb)
  • aspect 16:9 : Widescreen
Compress .avi to divx
ffmpeg -i video_origine.avi -s 320x240 -vcodec msmpeg4v2 video_finale.avi
Compress Ogg Theora to Mpeg dvd
ffmpeg -i film_sortie_cinelerra.ogm -s 720x576 -vcodec mpeg2video -acodec mp3 film_terminée.mpg
Compress .avi to SVCD mpeg2
NTSC format:
ffmpeg -i video_origine.avi -target ntsc-svcd video_finale.mpg
PAL format:
ffmpeg -i video_origine.avi -target pal-svcd video_finale.mpg
Compress .avi to VCD mpeg2
NTSC format:
ffmpeg -i video_origine.avi -target ntsc-vcd video_finale.mpg
PAL format:
ffmpeg -i video_origine.avi -target pal-vcd video_finale.mpg
Multi-pass encoding with ffmpeg
ffmpeg -i fichierentree -pass 2 -passlogfile ffmpeg2pass fichiersortie-2


REFERENCES
http://rodrigopolo.com/ffmpeg/cheats.php
Subscribe to: Posts (Atom)