Friday, January 22, 2010

The DRBD User's Guide

Category: , , — SkyHi @ Friday, January 22, 2010

This guide has been released to the DRBD community, and its authors strive to improve it permanently. Feedback from readers is always welcome and encouraged. Please use the DRBD public mailing list for enhancement suggestions and corrections. Recent content additions are marked with Draft status; feedback on those sections is particularly highly appreciated.

Copyright © 2008, 2009 LINBIT Information Technologies GmbH

Copyright © 2009 LINBIT HA Solutions GmbH

Trademarks used in this guide

DRBD®, the DRBD logo, LINBIT®, and the LINBIT logo are trademarks or registered trademarks of LINBIT Information Technologies GmbH in Austria, the United States and other countries.

AMD is a registered trademark of Advanced Micro Devices, Inc.

Citrix is a registered trademark of Citrix, Inc.

Debian is a registered trademark of Software in the Public Interest, Inc.

Dolphin Interconnect Solutions and SuperSockets are trademarks or registered trademarks of Dolphin Interconnect Solutions ASA.

IBM is a registered trademark of International Business Machines Corporation.

Intel is a registered trademark of Intel Corporation.

Linux is a registered trademark of Linus Torvalds.

MySQL and MySQL Enterprise are trademarks or registered trademarks of MySQL AB and/or Sun Microsystems, Inc.

Oracle is a registered trademark of Oracle Corporation and/or its affiliates.

Red Hat, Red Hat Enterprise Linux, and RPM are trademarks or registered trademarks of Red Hat, Inc.

SuSE, SUSE, and SUSE Linux Enterprise Server are trademarks or registered trademarks of Novell, Inc.

Xen is a registered trademark of Citrix, Inc.

Other names mentioned in this guide may be trademarks or registered trademarks of their respective owners.

January 18, 2010

Table of Contents

About this guide
I. Introduction to DRBD
1. DRBD Fundamentals
Kernel module
User space administration tools
Resources
Resource roles
2. DRBD Features
Single-primary mode
Dual-primary mode
Replication modes
Multiple replication transports
Efficient synchronization
On-line device verification
Replication traffic integrity checking
Split brain notification and automatic recovery
Support for disk flushes
Disk error handling strategies
Strategies for dealing with outdated data
Three-way replication
Long-distance replication with DRBD Proxy
Truck based replication
Floating peers
II. Building, installing and configuring DRBD
3. Installing pre-built DRBD binary packages
Packages supplied by LINBIT
Packages supplied by distribution vendors
4. Building and installing DRBD from source
Downloading the DRBD sources
Checking out sources from the public DRBD source repository
Building DRBD from source
Checking build prerequisites
Preparing the kernel source tree
Preparing the DRBD build tree
Building DRBD userspace utilities
Compiling DRBD as a kernel module
Building a DRBD RPM package
Building a DRBD Debian package
5. Configuring DRBD
Preparing your lower-level storage
Preparing your network configuration
Configuring your resource
Example configuration
The global section
The common section
The resource sections
Enabling your resource for the first time
The initial device synchronization
Using truck based replication
III. Working with DRBD
6. Common administrative tasks
Checking DRBD status
Retrieving status with drbd-overview
Status information in /proc/drbd
Connection states
Resource roles
Disk states
Enabling and disabling resources
Enabling resources
Disabling resources
Reconfiguring resources
Promoting and demoting resources
Enabling dual-primary mode
Using on-line device verification
Enabling on-line verification
Invoking on-line verification
Automating on-line verification
Configuring the rate of synchronization
Permanent syncer rate configuration
Temporary syncer rate configuration
Configuring checksum-based synchronization
Configuring I/O error handling strategies
Configuring replication traffic integrity checking
Resizing resources
Growing on-line
Growing off-line
Shrinking on-line
Shrinking off-line
Disabling backing device flushes
Configuring split brain behavior
Split brain notification
Automatic split brain recovery policies
Creating a three-node setup
Device stacking considerations
Configuring a stacked resource
Enabling stacked resources
Using DRBD Proxy
DRBD Proxy deployment considerations
Configuration
Controlling DRBD Proxy
7. Troubleshooting and error recovery
Dealing with hard drive failure
Manually detaching DRBD from your hard drive
Automatic detach on I/O error
Replacing a failed disk when using internal meta data
Replacing a failed disk when using external meta data
Dealing with node failure
Dealing with temporary secondary node failure
Dealing with temporary primary node failure
Dealing with permanent node failure
Manual split brain recovery
IV. DRBD-enabled applications
8. Integrating DRBD with Pacemaker clusters
Pacemaker primer
The Pacemaker cluster manager
Pacemaker resources
Pacemaker resource agents
OpenAIS communication channels
Pacemaker configuration
The openais.conf file
The authkey file
Propagating the cluster configuration to cluster nodes
Using DRBD in Pacemaker clusters
Pacemaker CRM configuration
Managing Pacemaker CRM clusters
Using resource-level fencing in Pacemaker clusters
Using stacked DRBD resources in Pacemaker clusters
Adding off-site disaster recovery to Pacemaker clusters
Using stacked resources to achieve 4-way redundancy in Pacemaker clusters
Configuring DRBD to replicate between two SAN-backed Pacemaker clusters
DRBD resource configuration
Pacemaker resource configuration
Site fail-over
9. Integrating DRBD with Heartbeat clusters
Heartbeat primer
The Heartbeat cluster manager
Heartbeat resources
Heartbeat resource agents
Heartbeat communication channels
Heartbeat configuration
The ha.cf file
The authkeys file
Propagating the cluster configuration to cluster nodes
Using DRBD in Heartbeat R1-style clusters
Heartbeat R1-style configuration
Managing Heartbeat R1-style clusters
Using DRBD in Heartbeat CRM-enabled clusters
Heartbeat CRM configuration
Managing Heartbeat CRM clusters
Using Heartbeat with dopd
Heartbeat configuration
DRBD Configuration
Testing dopd functionality
10. Integrating DRBD with Red Hat Cluster Suite
Red Hat Cluster Suite primer
OpenAIS and CMAN
CCS
Fencing
The Resource Group Manager
Red Hat Cluster Suite configuration
The cluster.conf file
Using DRBD in RHCS fail-over clusters
Setting up your cluster configuration
11. Using LVM with DRBD
LVM primer
Using a Logical Volume as a DRBD backing device
Configuring a DRBD resource as a Physical Volume
Nested LVM configuration with DRBD
12. Using GFS with DRBD
GFS primer
Creating a DRBD resource suitable for GFS
Configuring LVM to recognize the DRBD resource
Configuring your cluster to support GFS
Creating a GFS filesystem
Using your GFS filesystem
13. Using OCFS2 with DRBD
OCFS2 primer
Creating a DRBD resource suitable for OCFS2
Configuring your cluster to support OCFS2
Creating the configuration file
Configuring the O2CB driver
Creating an OCFS2 filesystem
Using your OCFS2 filesystem
14. Using Xen with DRBD
Xen primer
Setting DRBD module parameters for use with Xen
Creating a DRBD resource suitable to act as a Xen VBD
Using DRBD VBDs
Starting, stopping, and migrating DRBD-backed domU's
Internals of DRBD/Xen integration
Integrating Xen with Heartbeat
V. Optimizing DRBD performance
15. Measuring block device performance
Measuring throughput
Measuring latency
16. Optimizing DRBD throughput
Hardware considerations
Throughput overhead expectations
Tuning recommendations
Setting max-buffers and max-epoch-size
Tweaking the I/O unplug watermark
Tuning the TCP send buffer size
Tuning the Activity Log size
Disabling barriers and disk flushes
17. Optimizing DRBD latency
Hardware considerations
Latency overhead expectations
Tuning recommendations
Setting DRBD's CPU mask
Modifying the network MTU
Enabling the deadline I/O scheduler
VI. Learning more about DRBD
18. DRBD Internals
DRBD meta data
Internal meta data
External meta data
Estimating meta data size
Generation Identifiers
Data generations
The generation identifier tuple
How generation identifiers change
How DRBD uses generation identifiers
The Activity Log
Purpose
Active extents
Selecting a suitable Activity Log size
The quick-sync bitmap
The peer fencing interface
19. Getting more information
Commercial DRBD support
Public mailing list
Public IRC Channels
Blogs
Official Twitter account
Publications
Other useful resources
A. DRBD system manual pages
drbd.conf — Configuration file for DRBD's devices
drbdadm — Administration tool for DRBD
drbdsetup — Setup tool for DRBD
drbdmeta — DRBD's meta data management tool

List of Figures

1.1. DRBD's position within the Linux I/O stack
2.1. DRBD resource stacking
8.1. DRBD resource stacking in Pacemaker clusters
8.2. DRBD resource stacking in Pacemaker clusters
8.3. Using DRBD to replicate between SAN-based clusters
11.1. LVM overview
18.1. GI tuple changes at start of a new data generation
18.2. GI tuple changes at start of re-synchronization
18.3. GI tuple changes at completion of re-synchronization

List of Tables

4.1. Options supported by DRBD's configure script
4.2. DRBD userland RPM packages
18.1. fence-peer handler exit codes

List of Examples

A.1. A small drbd.conf file

List of Equations

2.1. Synchronization time
6.1. Syncer rate example, 110MB/s effective available bandwidth
6.2. Syncer rate example, 80MB/s effective available bandwidth
18.1. Calculating DRBD meta data size (exactly)
18.2. Estimating DRBD meta data size (approximately)
18.3. Active extents calculation based on sync rate and target sync time
18.4. Active extents calculation based on sync rate and target sync time (example)

Reference: http://www.drbd.org/users-guide/