Since you are a good sysadmin, you are concerned about storing clear text. The general concern is that any physical storage where this data lives might not be safe 100% of the time (removable media, for instance). If someone obtains a copy of your data, you don’t want any sensitive information to be compromised without at least putting up a fight.

Many encryption methods use a “key” during encryption that acts as the secret sauce to converting the encrypted data back into readable, usable data. It is always important to be very cautious about where this key is stored and who/what has access to it. I’ve seen a few proprietary encryption methods that use hard-coded keys, which makes any compromise catastrophic to multiple data sets.

The following algorithm performs the task of encrypting data using 256-bit AES encryption. It provides a mechanism for you to input a custom key on each instance, eliminating the concern for “one key unlocks all the doors”. As an example of how this can be handy, you might provision one key for each of your individual customers. This way, customer B will never be able to decrypt customer A’s top secret documents.

One interesting/debatable topic is whether or not 2-way encryption algorithms are even necessary. This guy has an interesting blog post with his stance, and I tend to agree with him on most of it.

A debatable item in my implementation is the assemble() and disassemble() methods. A knowledgeable colleague of mine suggested that it is not necessary to do my own padding / prefixing length etc. In PHP’s openssl_encrypt() and openssl_decrypt() functions, it appears that this padding and its removal is handled for you if you simply pass in a string of arbitrary length. I didn’t want to get rid of the code just yet, but opinions (especially excerpts from RFC’s) are always appreciated. I’d be interested to know if perhaps the data passed to openssl_encrypt is already of correct size, it is left alone, or if the padding method will still pad an extra 16 bytes for AES-256. Maybe a follow-up post will come some future day.

<?php

/* vim: set expandtab tabstop=4 shiftwidth=4 softtabstop=4: */

/**
 * StandardDataEncryption - At least it's better than clear text.
 *
 * PHP Version 5.3+
 *
 * This program is free software: you can redistribute it and/or modify
 * it under the terms of the GNU General Public License as published by
 * the Free Software Foundation, either version 3 of the License, or
 * (at your option) any later version.
 *
 * This program is distributed in the hope that it will be useful,
 * but WITHOUT ANY WARRANTY; without even the implied warranty of
 * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 * GNU General Public License for more details.
 *
 * You should have received a copy of the GNU General Public License
 * along with this program.  If not, see <http://www.gnu.org/licenses/>.
 *
 * @category   Encryption
 * @package    StandardDataEncryption
 * @author     Ryan Uber <ryan@blankbmx.com>
 * @copyright  2012 Ryan Uber <ryan@blankbmx.com>
 * @link       http://www.ryanuber.com/standard-data-encryption-in-php.html
 */

/**
 * Provides a standards-base encryption algorithm for safe storage of
 * sensitive data. This algorithm is not designed to provide over-the-
 * wire security, but rather protecting data saved to non-volatile
 * memory and stored some place where the decryption algorithm (or at
 * least the cipher password) is not present.
 *
 * This algorithm is mostly useful when you want/need to have access
 * to clear-text data (maybe you have to store a clear-text password
 * for use in a script somewhere), but dont want to / can't store
 * without encryption.
 *
 * This algorithm implements the following features:
 *  - Data padding to nearest cipher block size
 *  - Randomly generated initialization vectors
 *  - Settable cipher password
 *  - Hexadecimal data returned for easy storage
 *  - Configurable cipher method
 */
class StandardDataEncryption
{
    /**
     * Name of encryption cipher
     *
     * @const string $cipher_name
     */
    const CIPHER_NAME = 'AES-256-CBC';

    /**
     * Block size of cipher (in bytes)
     *
     * @const integer $cipher_blocksize
     */
    const CIPHER_BLOCKSIZE = 16;

    /**
     * Cipher password
     *
     * @var string $cipher_password
     */
    private static $cipher_password = null;

    /**
     * Cipher Initialization Vector
     *
     * @var integer cipher_iv
     */
    private static $cipher_iv = null;

    /**
     * Debugging mode (enabled=TRUE, disabled=FALSE)
     *
     * @var bool $debug
     */
    private static $debug = FALSE;

    /**
     * Simple constructor to make it possible to encrypt / decrypt string
     * data without making explicit calls to set the password and
     * initialization vector.
     *
     * @param string $cipher_password  The cipher password
     * @param string $cipher_iv  The cipher initialization vector
     * @returns bool
     */
    public function __construct( $cipher_password=null )
    {
        if(!is_null($cipher_password))
        {
            self::set_cipher_password($cipher_password);
        }
    }

    /**
     * Method to enable debug messages. Encryption is generally difficult to
     * troubleshoot if you can't see what is going on behind the scenes, but
     * it is equally annoying to receive debug messages when you don't want
     * them. This allows us to set debug mode on an instance of this class.
     * Non-printable characters will show up as a star (*).
     *
     * @param string $Message  The message to output to the debug channel
     * @returns bool
     */
    private static function debug( $Message )
    {
        if(self::$debug)
        {
            $out='';
            foreach(str_split($Message) as $c)
            {
                $out .= ctype_print($c)?$c:'*';
            }
            print "\n*** DEBUG ***\n[".time()."] ".$out."\n";
        }
    }

    /**
     * Prepare data to be encrypted. This manipulates the data and pads
     * it up to the next full block-size with zeros. It will prepend the
     * length of the actual string data before returning.
     *
     * @param string $Data  Arbitrary string data to encrypt
     * @returns string
     */
    private static function assemble( $Data )
    {
        $padlen = self::CIPHER_BLOCKSIZE - (strlen($Data) % self::CIPHER_BLOCKSIZE);
        $result = str_pad($padlen, self::CIPHER_BLOCKSIZE, 0, STR_PAD_LEFT) . $Data . self::pad($padlen);
        self::debug('Assembled '.mb_strlen($result).'-byte string for encryption: '.$result);
        return $result;
    }

    /**
     * After decrypting data, we are left with a string in "prepared"
     * state, basically the return of the assemble() method. We need to convert
     * this into a usable text value and chop the padding off.
     *
     * @param string $Data  Prepared string from original assemble() method
     * @returns string
     */
    private static function disassemble( $Data )
    {
        self::debug('Assembled '.mb_strlen($Data).'-byte string found in decrypted data: '.$Data);
        $padlen = abs((int)substr($Data, 0, self::CIPHER_BLOCKSIZE));
        $result = substr($Data, self::CIPHER_BLOCKSIZE, (strlen($Data)-self::CIPHER_BLOCKSIZE-$padlen));
        return $result;
    }

    /**
     * Convert hex data into an ASCII string. In PHP >= 5.4.0, this method
     * could be replaced by hex2bin(), but alas in 5.3.x, it is absent.
     *
     * @param string $Data  Hexadecimal data
     * @returns string
     */
    private static function fromhex( $Data )
    {
        self::debug('Decoding from hexadecimal value: '.$Data );
        $result = '';
        for($i=0; $i<strlen($Data)-1; $i+=2 )
        {
            $result .= chr(hexdec($Data[$i].$Data[$i+1]));
        }
        return $result;
    }

    /**
     * Convert an ASCII string to hexadecimal format.
     *
     * @param string $Data  ASCII string to convert
     * @returns string
     */
    private static function tohex( $Data )
    {
        return bin2hex($Data);
    }

    /**
     * Set the initialization vector
     *
     * @param string $IV  The initialization vector
     * @returns bool
     */
    private static function set_cipher_iv( $IV=null )
    {
        if(is_null($IV))
        {
            self::debug('Null initialization vector passed - Auto-generating');
            $IV = openssl_random_pseudo_bytes(self::CIPHER_BLOCKSIZE);
        }
        self::$cipher_iv = $IV;
    }

    /**
     * Generates fixed-length padding with repeating values, implemented per
     * RFC 2315 Section 10.2. Characters will likely be non-printable and will
     * not be displayed in debugging output.
     *
     * @param integer $len  The length of padding to return
     * @returns string
     */
    private static function pad( $len=0 )
    {
        return str_repeat(chr('0x'.str_pad($len, strlen(self::CIPHER_BLOCKSIZE), 0, STR_PAD_LEFT)), $len);
    }

    /**
     * Set the secret cipher password for use in encryption
     *
     * @param string $Password  The cipher password
     * @returns bool
     */
    public function set_cipher_password( $Password )
    {
        self::$cipher_password = $Password;
    }

    /**
     * Set debugging mode. Valid states are TRUE of FALSE (enabled or disabled).
     * While debugging mode is set, you will see extra messages that could be
     */
    public function set_debug( $State=FALSE )
    {
        self::$debug = $State;
    }

    /**
     * Using OpenSSL and the supplied parameters, prepare the arbitrary
     * string and encrypt it, returning a value suitable for safe storage.
     * During encryption, the initialization vector is randomly generated
     * so that two encryptions of the same data do not yield the same hash.
     *
     * @param string $Data  Prepared string data as returned by assemble()
     * @returns mixed
     */
    public function encrypt( $Data )
    {
        self::set_cipher_iv();
        $encrypted = self::tohex(self::$cipher_iv . openssl_encrypt(
            self::assemble( $Data ), self::CIPHER_NAME, self::$cipher_password,
            TRUE, self::$cipher_iv
        ));
        if(!$encrypted)
        {
            self::debug('Failed while encrypting data: '.$Data);
            return false;
        }
        return $encrypted;
    }

    /**
     * Decrypt a string and convert the result from its prepared state to a
     * readable, usable string.
     *
     * @param string $Data  Encrypted data
     * @returns mixed
     */
    public function decrypt( $Data )
    {
        $raw = self::fromhex($Data);
        if((($raw?strlen($raw):1) % self::CIPHER_BLOCKSIZE) != 0)
        {
            self::debug('Provided hexadecimal data was invalid.');
            return false;
        }
        self::set_cipher_iv(mb_strcut($raw, 0, self::CIPHER_BLOCKSIZE));
        $decrypted = self::disassemble(openssl_decrypt(
            mb_strcut($raw, self::CIPHER_BLOCKSIZE), self::CIPHER_NAME,
            self::$cipher_password, TRUE, self::$cipher_iv
        ));
        if(!$decrypted)
        {
            self::debug('Failed while decrypting data: '.$Data);
            return false;
        }
        return $decrypted;
    }
}

/* EOF */
?>

Some example usage is below:

<?php
include 'StandardDataEncryption.php';
$e = new StandardDataEncryption('My Encryption Key Here');
$e->set_debug(TRUE);
$encrypted = $e->encrypt('Hack Me!');
print "\nEncrypted value is: ".$encrypted."\n";
$decrypted = $e->decrypt($encrypted);
print "\nDecrypted value is: ".$decrypted."\n";
?>

And it will yield some output like:

*** DEBUG ***
[1335560630] Null initialization vector passed - Auto-generating

*** DEBUG ***
[1335560630] Assembled 32-byte string for encryption: 0000000000000008Hack Me!********

Encrypted value is: ce5b31f1144c93094ffbb805e5d053b04fd9f27821b7f91475ef47688c7495df02681e498933dee6d997ade66506c363b71682277738f1983899935b0c604aa6

*** DEBUG ***
[1335560630] Decoding from hexadecimal value: ce5b31f1144c93094ffbb805e5d053b04fd9f27821b7f91475ef47688c7495df02681e498933dee6d997ade66506c363b71682277738f1983899935b0c604aa6

*** DEBUG ***
[1335560630] Assembled 32-byte string found in decrypted data: 0000000000000008Hack Me!********

Decrypted value is: Hack Me!

This issue is discussed at length in this thread:

https://projects.puppetlabs.com/issues/12185#change-56530

There is more than one way to solve this problem, so below I’ll detail my solution. Keep in mind that this is not intended to be a permanent fix, but merely a way of keeping the machines running until a more elegant solution is implemented.

My solution is a simple cron/script type. Every 5 minutes, I run a script to check the last modified time of the puppetdlock file, and compare it to the current time. If it is older than 5 minutes, I call a puppetd restart. That’s it!

#!/bin/bash
# File Name:     puppetd-hang-fix.sh
# Author:        Ryan Uber <ryan@blankbmx.com>
#
# Description:   This is a *TEMPORARY* fix to the common puppetdlock issue.
#                Running this script checks if the lock file is more than 5
#                minutes old, and if it is, restarts the puppet process.
#
# See Also:      https://projects.puppetlabs.com/issues/12185#change-56530

# Path to the puppetdlock file
FILE=/var/lib/puppet/state/puppetdlock

# Number of minutes since last modification that indicate a stale lock file
STALEMIN=5

# UNIX timestamp of last puppetdlock modification
MODIFIED=$(stat -c %Y ${FILE})

# Current UNIX timestamp
NOW=$(date +%s)

# Check if file is stale or not and restart if it is.
if [ $((${NOW}-${MODIFIED})) -gt $((${STALEMIN}*60)) ]
then
    /sbin/service puppet restart
fi

# EOF

See one of my other posts if you are interested in how I went about pushing out the cron job.

What you can’t be afraid of to accomplish this

1. Forcefully breaking RPM dependencies (will be resolved upon completion)
2. Ignoring package checksum mismatches
3. Rebooting

New version of RPM
Seeing how its not possible to install EL6 RPM’s with EL5′s stock version of RPM, the first thing to do is to get that newer version of RPM installed on your system. Keith Chambers, a friend and colleague of mine, recompiled RPM 4.6 against glibc 2.5 so that it would work with the current EL5 system. RPM 4.8 may have worked as well, but 4.6 worked for me, and I only needed to use this version during the big upgrade. RPM will replace itself with the stock EL6 version later on. For dependencies, I needed to include xz-libs and lua. I force-upgraded these packages (–nodeps). What you want to end up with is:

# rpm --version
RPM version 4.6.0

Patch YUM
Since the new RPM’s coming from the repo will be verified with a newer hashing algorithm than what YUM/python2.4 will recognize, I needed to make a few quick patches to YUM itself to force it to ignore conflicts. These ghetto-hack patches will be automatically removed from the system when YUM upgrades itself, so really this is only needed one time.

diff -Nur /usr/lib/python2.4/site-packages/yum.orig/depsolve.py /usr/lib/python2.4/site-packages/yum/depsolve.py
--- /usr/lib/python2.4/site-packages/yum.orig/depsolve.py       2011-08-19 15:07:02.000000000 +0000
+++ /usr/lib/python2.4/site-packages/yum/depsolve.py    2012-01-07 20:43:21.634917883 +0000
@@ -142,6 +142,7 @@
                             'repackage': rpm.RPMTRANS_FLAG_REPACKAGE}

         self._ts.setFlags(0) # reset everything.
+        self._ts.addTsFlag(rpm.RPMTRANS_FLAG_NOMD5)

         for flag in self.conf.tsflags:
             if ts_flags_to_rpm.has_key(flag):
diff -Nur /usr/lib/python2.4/site-packages/yum.orig/__init__.py /usr/lib/python2.4/site-packages/yum/__init__.py
--- /usr/lib/python2.4/site-packages/yum.orig/__init__.py       2011-08-19 15:07:02.000000000 +0000
+++ /usr/lib/python2.4/site-packages/yum/__init__.py    2012-01-07 20:43:01.074133746 +0000
@@ -1217,6 +1217,7 @@
                 failed = True

+        failed = False
         if failed:
             # if the file is wrong AND it is >= what we expected then it
             # can't be redeemed. If we can, kill it and start over fresh
diff -Nur /usr/lib/python2.4/site-packages/yum.orig/yumRepo.py /usr/lib/python2.4/site-packages/yum/yumRepo.py
--- /usr/lib/python2.4/site-packages/yum.orig/yumRepo.py        2011-08-19 15:07:02.000000000 +0000
+++ /usr/lib/python2.4/site-packages/yum/yumRepo.py     2012-01-07 20:43:09.834466442 +0000
@@ -1467,6 +1467,7 @@
             file = fn.filename
         else:
             file = fn
+        return 1

         try:
             l_csum = self._checksum(r_ctype, file) # get the local checksum

I wrote a few quick lines in SED to handle this patching for me:

sed -i '/^        if failed:/i\        failed = False' /usr/lib/python2.4/site-packages/yum/__init__.py
sed -i '/^            file = fn$/a\        return 1' /usr/lib/python2.4/site-packages/yum/yumRepo.py
sed -i '/# reset everything.$/a\        self._ts.addTsFlag(rpm.RPMTRANS_FLAG_NOMD5)' /usr/lib/python2.4/site-packages/yum/depsolve.py

Package Conflicts
There are a number of packages while performing an upgrade from EL5 to EL6 that will cause you trouble, as some of the version numbers have actually decremented, and some cause dependency resolution issues that will be solved only after the upgrade is complete. The following is a small snippet that I used to force my way out of this situation:

declare -ra FORCEREMOVE=(
    m2crypto centos-release newt authconfig prelink tcp_wrappers sgpio
    iscsi-initiator-utils mkinitrd dmraid dmraid-events hmaccalc sysfsutils
    device-mapper device-mapper-multipath device-mapper-event
    vmware-open-vm-tools-common vmware-open-vm-tools-kmod less usermode
    libhugetlbfs lvm2 kpartx e4fsprogs-libs glib libsysfs
)
for PKG in ${FORCEREMOVE[@]}; do
    rpm -e --nodeps ${PKG}
done

PAM authentication work-around
By default, the RPM’s that install PAM do not clean out its include directory (/etc/pam.d). If the authentication modules you are using change at all (new ones added, old ones removed), you will need to manually clean out this directory. I found it easiest to just remove all of the configs and let them be re-populated by the new RPM’s. Without this step, after the upgrade completed, I was unable to log in to the console of the machine, because some authentication modules specified in the config files were now absent (no longer needed in the product I work on). If you have custom configs, you will need to account for them somehow, hopefully using Puppet or your configuration tool of choice.

rm -f /etc/pam.d/*

Re-install centos-release
You will notice in one of the steps above that I force-removed the centos-release package. The reason I needed to do this was because of the following:

package centos-release-5-7.el5.centos.x86_64 (which is newer than centos-release-6-0.el6.centos.5.x86_64) is already installed

By force-removing the centos-release package, and then performing a “yum install centos-release”, the centos-release package gets updated to the 6.x version.

Downgrade nss
This is a very important step. The version of “nss” actually decremented between EL5 and EL6. Since the new glibc 2.12 requires nss-softokn-freebl, which requires nss, we need to downgrade nss. This step should downgrade nss, install a few new packages, and update glibc. Once you do this, there is really no going back.

# yum downgrade nss
Loaded plugins: fastestmirror
Setting up Downgrade Process
Loading mirror speeds from cached hostfile
Resolving Dependencies
--> Running transaction check
---> Package nss.x86_64 0:3.12.7-2.el6 set to be updated
--> Processing Dependency: nss-softokn(x86-64) >= 3.12.7 for package: nss
--> Processing Dependency: nss-util >= 3.12.7 for package: nss
--> Processing Dependency: libnssutil3.so(NSSUTIL_3.12.3)(64bit) for package: nss
--> Processing Dependency: nss-system-init for package: nss
--> Processing Dependency: libnssutil3.so(NSSUTIL_3.12)(64bit) for package: nss
--> Processing Dependency: libnssutil3.so(NSSUTIL_3.12.5)(64bit) for package: nss
--> Processing Dependency: libnssutil3.so()(64bit) for package: nss
---> Package nss.x86_64 0:3.12.8-4.el5_6 set to be erased
--> Running transaction check
---> Package nss-softokn.x86_64 0:3.12.8-1.el6_0 set to be updated
--> Processing Dependency: nss-softokn-freebl(x86-64) >= 3.12.8 for package: nss-softokn
---> Package nss-sysinit.x86_64 0:3.12.7-2.el6 set to be updated
---> Package nss-util.x86_64 0:3.12.8-1.el6_0 set to be updated
--> Running transaction check
---> Package nss-softokn-freebl.x86_64 0:3.12.8-1.el6_0 set to be updated
--> Processing Dependency: libc.so.6(GLIBC_2.7)(64bit) for package: nss-softokn-freebl
--> Running transaction check
---> Package glibc.x86_64 0:2.12-1.7.el6_0.5 set to be updated
--> Processing Dependency: glibc-common = 2.12-1.7.el6_0.5 for package: glibc
--> Running transaction check
---> Package glibc-common.x86_64 0:2.12-1.7.el6_0.5 set to be updated
--> Processing Conflict: glibc conflicts binutils < 2.19.51.0.10
--> Restarting Dependency Resolution with new changes.
--> Running transaction check
---> Package binutils.x86_64 0:2.20.51.0.2-5.11.el6 set to be updated
--> Processing Conflict: glibc conflicts prelink < 0.4.2
--> Restarting Dependency Resolution with new changes.
--> Running transaction check
---> Package prelink.x86_64 0:0.4.6-3.el6 set to be updated
--> Finished Dependency Resolution

Dependencies Resolved

=========================================================================================
 Package                   Arch          Version                     Repository     Size
=========================================================================================
Updating:
 binutils                  x86_64        2.20.51.0.2-5.11.el6        base          2.8 M
 prelink                   x86_64        0.4.6-3.el6                 base          994 k
Downgrading:
 nss                       x86_64        3.12.7-2.el6                base          735 k
Installing for dependencies:
 nss-softokn               x86_64        3.12.8-1.el6_0              base          166 k
 nss-softokn-freebl        x86_64        3.12.8-1.el6_0              base          115 k
 nss-sysinit               x86_64        3.12.7-2.el6                base           26 k
 nss-util                  x86_64        3.12.8-1.el6_0              base           46 k
Updating for dependencies:
 glibc                     x86_64        2.12-1.7.el6_0.5            base          3.7 M
 glibc-common              x86_64        2.12-1.7.el6_0.5            base           14 M

Transaction Summary
=========================================================================================
Install       4 Package(s)
Upgrade       4 Package(s)
Remove        0 Package(s)
Reinstall     0 Package(s)
Downgrade     1 Package(s)

Once this has completed, you are clear to upgrade the rest of the system.

# yum -y upgrade

Re-install a few packages
A few of the packages that we force-removed for dependency resolution reasons will not get automatically installed. Therefore, we need to install them by hand:

declare -ra VMW_PKGS=(
    vmware-tools-core
    vmware-tools-foundation
    vmware-tools-guestlib
    vmware-tools-libraries-nox
    vmware-tools-plugins-guestInfo
    vmware-tools-plugins-vix
    vmware-tools-services
    vmware-tools-plugins-deployPkg
)
yum -y install ${VMW_TOOLS[@]} prelink lvm2 less

You can ignore the vmware packages if you are not using them.

Remove the old kernels
You should no longer need any of your EL5 kernels. You have hopped the fence and are in the land of EL6 now. You should be able to delete (rpm -e) any kernel-2.6.18* packages that are still installed.

Rebooting
Since I added all of the above logic in a script and executed it during a pre-install stage with Puppet, I had to have a way to automatically reboot the machines as well. In one of my previous posts, Puppet Self-Management, I detailed how to patiently wait for a puppet run to finish before executing some action from within a script. I applied this same technique to the reboot for my EL5 to EL6 upgrade script. Since I am executing my upgrade script during a puppet run, I do not want to bounce the machine before the run completes to avoid corrupting the state, and also to ensure that all of my other puppet-managed updates are applied before the machine boots in to EL6 for the first time. I accomplished this with the following code at the very end of my upgrade script:

/bin/sh -c "
    until [ ! -f /var/lib/puppet/state/puppetdlock ]
    do
        sleep 1
    done
    /sbin/shutdown -r now" &

Now then, first thing’s first: Connecting the thing to the PC. The Apple 27″ LED display that I purchased comes with *only* a mini displayport connection. No DVI, no HDMI. This actually didn’t bother me so much, since the mini displayport is present on a number of ATI cards. I set out to find a new graphics card. The first one I came across (and actually ended up purchasing alongside the monitor) was the Apple GeForce GT120. Yes, its an Apple product meant to be an upgrade to the Mac Pro’s from early 2009, but it was a PCI Express card, nothing out of the ordinary (probably just re-branded). Remembering back to my Hackintosh days, I figured that I probably had the best bet with this card. Apple makes it, so it should work right?

Wrong! While the DVI graphics worked just fine on an older monitor (Samsung P2350), the mini displayport would give me no video. I am guessing that Apple probes the card in some way that my PC does not. Also note, at this point I had still never seen the display even power on, since the new LED displays are controlled completely via the MDP, and thus there is not a single button on the entire monitor.

A little disgruntled, I removed the GT120 from my PC and moved on to the next solution: an ATI card with MDP. I found one at my local Fry’s Electronics, and ordered online for local pickup. It was the AMD HD 6870 XOC. All of the specifications looked correct on the site; 1x Dual-Link DVI, 1x Single-Link DVI, 1x HDMI, and 2x mini displayports. I picked the card up from the store and headed home for installation. To my surprise, when I pulled the card out of the box, it had 2x DVI ports and 1x displayport, not even a minidispalyport or any HDMI connections. It was identical to the item on the website in every other way, shape, and form, but the interface description on the left side of the box had a sticker over it with the downgraded specifications.

Needless to say, I had to return it. I didn’t want to run any displayport to mini-displayport adapters or any other nonsense like that.

I ended up finding the XFX version of the Radeon HD 6870 at MicroCenter for around the same price. This time I verified before walking out the door with the card.

While I was installing the card, I noticed the box sitting next to me mentioning the interface was PCI Express 2.1. I didn’t think anything of it, because I had done my homework beforehand and learned that any PCIE 2.1 card would work in a 2.0 slot.

To my surprise, after installing the card and connecting everything in my Gigabyte H55M-S2V, the card did not function at all! After further reading and research, I still did not find an answer as to why the card would not function in this particular motherboard. A BIOS update and still, nothing. Irritated and ready to just start using the darn monitor and stop fiddling with hardware, I went out and bought a new motherboard. My processor was LGA1156 based, so I had limited options. I ended up with an ASUS P7P55D-ELX. It had a few goodies that my old board didn’t, like 6GB/s SATA and USB 3.0. After further ripping apart/assembling my PC, I finally was able to boot up, and see the display work perfectly!

The BIOS POST showed up just fine, stretched to the size of the 2560×1440 native on the LED. Everything was all well and good. I did notice though, that the back light was looking pretty dim. I then realized that the factory default for the brightness setting is not set to the max, or even a reasonable brightness. It is in fact pretty dim, even with the ambient light detection/adjustment in the iSight. What I ended up doing to solve this was installing Windows 7, and on top of that, installing Boot Camp from a Snow Leopard DVD that I had. It is noteworthy though, that the version of Boot Camp that came on the Snow Leopard disk did not give me full access to the brightness control. Even with the slider all the way up to max, I still only had maybe 50% brightness. After running the Apple Software Update utility and getting the latest Boot Camp, and a subsequent reboot, I watched my Cinema Display come to life as I dragged the slider all the way up and the brightness began to burn my pupils!

After all of this fighting to get a Cinema Display to work sans the Mac, I would indeed say that it is worth it. After setting the brightness settings in Boot Camp, feel free to nuke your Windows installation and go back to using Ubuntu, Fedora, SuSE, or whatever it is you like using. The LED display will remember the settings internally and won’t need to set them again.

A note about the video card:
I found the XFX HD 6870 to be a bit noisy for my taste. Not overpowering noisy, but my tower sits right next to me on my desk and I don’t like the sound of fans for hours on end. Installing MSI Afterburner and setting an automatic fan profile like the following, the noise level is more than acceptable and the GPU remains relatively cool (~50-55C).

In conclusion: It is totally worth it. You should do it.

There are no dependencies on any external libraries. Pass in your storage, your string, and the number of columns to wrap to, and this will do the rest.

/*
 * This function will wrap large amounts of a text into a manageable and human-readable width
 * word by word. Just specify the number of columns you are working with and feed it a string,
 * and it will return a new string (including added line breaks) to accommodate the area you
 * are working with.
 *
 * {{{ proto( void ) wrap( char out, char str, int columns )
 */
void wrap( char *out, char *str, int columns )
{
    int len, n, w, wordlen=0, linepos=0, outlen=0;

    /*
     * Find length of string 'str' without using string.h
     */
    for( len=0; str[len]; ++len );

    /*
     * Allocate the full space of 'str' to 'word', so there is no possible way that the string
     * could contain a word that does not fit into the 'word' variable.
     */
    char word[len];

    /*
     * Loop through each individual character in the passed array (str) and detect white space
     * and word length to determine how to handle line wrapping.
     */
    for( n=0; n<=len; n++ )
    {
        /*
         * Detect spaces and newlines. We cannot gurantee that the passed string is null-
         * terminated, so we also need to handle cases where we reach the end of the string
         * without encountering wite space characters.
         */
        if( str[n] == ' ' || str[n] == '\n' || n == len )
        {
            /*
             * If the current word will not fit on the current line, add a newline.
             */
            if( linepos > columns )
            {
                out[outlen++] = '\n';
                linepos = wordlen;
            }

            /*
             * Append the found word to the output and reset the word character array in
             * preparation for accepting characters for the next word.
             */
            for( w=0; w<wordlen; w++ )
            {
                out[outlen++] = word[w];
                word[w] = '\0';
            }

            /*
             * If we reach the end of the string, add the null-terminator character.
             */
            if( n == len )
                out[outlen] = '\0';

            /*
             * If we encounter a newline character, append it to the string as usual, but
             * set the line position counter back to 0 (new line = start count from 0 again).
             */
            else if( str[n] == '\n' )
            {
                out[outlen] = str[n];
                linepos=0;
            }

            /*
             * If the word fits in the current line without trouble, just add the space.
             */
            else
            {
                out[outlen] = ' ';
                linepos++;
            }

            /*
             * Increment the final output length for the next loop, and set the word length
             * counter back to 0 (newline or space = new word).
             */
            outlen++;
            wordlen=0;
        }

        /*
         * If the current character is in the middle of a word somewhere, just append it and
         * move on, incrementing counters.
         */
        else
        {
            word[wordlen++] = str[n];
            linepos++;
        }
    }
}
/* }}} */

Examples

Wrap at 50 columns:

$ ./a.out
Lorem ipsum dolor sit amet, consectetur adipiscing
elit. Maecenas pulvinar blandit diam nec mattis.
Sed a ipsum nec ante porttitor feugiat. Morbi
ipsum lacus, dignissim at bibendum in, consectetur
eget ipsum. Donec dolor nibh, scelerisque ac
sodales et, posuere ac nulla. Aliquam eget
tincidunt ante. Vestibulum justo leo, congue ut
luctus ut, venenatis non dui. Cras sapien risus,
blandit at semper eu, cursus eu est. In hac
habitasse platea dictumst. Fusce libero dolor,
commodo nec rhoncus eu, rutrum vel arcu. Vivamus
ultrices faucibus tellus ac feugiat. Ut imperdiet
metus sed erat feugiat sed porta lorem tristique.
Mauris quis erat vel ante cursus pretium.
Curabitur arcu erat, consequat ac ornare et,
venenatis eu diam.

Curabitur posuere dui vitae tortor cursus cursus.
Pellentesque adipiscing lobortis sem at varius.
Mauris pellentesque sollicitudin ultricies.
Maecenas in tellus turpis. Integer convallis
mollis elit eu placerat. Nunc volutpat consectetur
facilisis. Curabitur eros arcu, dapibus ut
convallis non, rhoncus non magna.

Wrap at 20 columns:

$ ./a.out
Lorem ipsum dolor
sit amet,
consectetur
adipiscing elit.
Maecenas pulvinar
blandit diam nec
mattis. Sed a ipsum
nec ante porttitor
feugiat. Morbi ipsum
lacus, dignissim at
bibendum in,
consectetur eget
ipsum. Donec dolor
nibh, scelerisque ac
sodales et, posuere
ac nulla. Aliquam
eget tincidunt ante.
Vestibulum justo
leo, congue ut
luctus ut, venenatis
non dui. Cras sapien
risus, blandit at
semper eu, cursus eu
est. In hac
habitasse platea
dictumst. Fusce
libero dolor,
commodo nec rhoncus
eu, rutrum vel arcu.
Vivamus ultrices
faucibus tellus ac
feugiat. Ut
imperdiet metus sed
erat feugiat sed
porta lorem
tristique. Mauris
quis erat vel ante
cursus pretium.
Curabitur arcu erat,
consequat ac ornare
et, venenatis eu
diam.

Curabitur posuere
dui vitae tortor
cursus cursus.
Pellentesque
adipiscing lobortis
sem at varius.
Mauris pellentesque
sollicitudin
ultricies. Maecenas
in tellus turpis.
Integer convallis
mollis elit eu
placerat. Nunc
volutpat consectetur
facilisis. Curabitur
eros arcu, dapibus
ut convallis non,
rhoncus non magna.

*Update 11/24/2011: This solution is useful if you need to word wrap inside of a C program. If you are shell scripting, you can use the “fold” command, which comes standard with the “coreutils” package in EL-based distributions.

Usage: fold [OPTION]... [FILE]...
Wrap input lines in each FILE (standard input by default), writing to
standard output.

Mandatory arguments to long options are mandatory for short options too.
  -b, --bytes         count bytes rather than columns
  -c, --characters    count characters rather than columns
  -s, --spaces        break at spaces
  -w, --width=WIDTH   use WIDTH columns instead of 80
      --help     display this help and exit
      --version  output version information and exit

Report bugs to <bug-coreutils@gnu.org>.

The technique that I decided on was a dual-table model. The schema was laid out like so (for the purpose of this post, I’ve removed irrelevant tables of course):

properties
   |
   +-- key (varchar)
   |
   +-- value (varchar)

properties_c
   |
   +-- key (varchar)
   |
   +-- value (varchar)

You can see that they are both essentially the same exact table, just with a different name. The reason for this is because I am going to be using both tables as a single “view” of sorts.

The “properties” table will be the “default” values. This is the table we will modify during upgrades. Its contents are essentially unimportant, and can be emptied, re-populated, or otherwise modified with no concern of obstructing valuable user-entered data. My database initialization script simply drops this table if it exists, and re-creates it during every upgrade with fresh, new data.

The properties_c table is where all of the custom data goes. It is empty by default.

I have multiple tables applying this concept, so I keep their naming consistent with the suffix “_c” for custom data tables.

This is simply the layout. Things got interesting while writing the SELECT statements. I think that the concept of selecting one table “over” another table could be applied for other purposes as well, so I will share what I came up with for queries.

Let’s say that I have this example data:

properties

key                    value
--------------         ---------------------------
welcomemsg             Welcome to our website!
dateformat             mm dd, yyyy
enablelogin            false

properties_c

key                    value
--------------         ---------------------------
welcomemsg             Go Away
myproperty             test

I need to select all of this data, but always give priority to the “properties_c” table. If a key exists in properties_c, it should override the value set in “properties”. It should not matter if the key is not present in both tables, we want to get all of the properties every time. The result I am looking to get from my select would be:

key                    value
--------------         ---------------------------
welcomemsg             Go Away
dateformat             mm dd, yyyy
enablelogin            false
myproperty             test

The following query will give me those exact results:


SELECT key, value FROM properties_c UNION SELECT key, value FROM properties p WHERE NOT EXISTS (SELECT * FROM properties_c WHERE key = p.key ) ORDER BY key;


There are some interesting things going on in this query.

The part that took me a while to figure out was how to compare the “key” column inside of the sub-query. You will notice that I am defining an alias for the “properties” table, even though it is the only table I am selecting from. I am doing this because the sub-query will have access to to the key column by its alias, allowing me to compare the values between the two tables. The bigger take-away from this lesson is that sub-queries can access parent data via aliases, provided you do not re-define the same alias inside of the sub-query.

The second thing that is noteworthy in this query is the use of the “NOT EXISTS” statement. I am using this to check if properties_c contains a row with a key of the same name as the properties table. If it does exist, the row from properties is ignored, otherwise, it is selected. This is my “default value” logic, and so far it seems to be working quite well.

The Caveat
One thing I can think of that might be undesirable in implementations other than the one I have would be deleting records. By deleting a record from properties_c, you aren’t really deleting, you are “resetting to default” because the default value still exists in the “properties” table. The only way I can think of solving this would be by adding a third column to the “properties” table with a boolean value that would indicate whether or not the value should be displayed. A more hack-ish way, but also quicker, would be to define some constant that would indicate the value should not be selected from either table. The down side of doing that would be that you are now limiting the possible values the user can enter.

I’ll be adding more to this post when the project is completed.

The problem is this: Puppet begins to run, it updates its config file, and then notifies the service to restart due to the change. The service restart will not necessarily be the very last thing to happen during the puppet run, so if you are in the middle of a run and the service restarts, you are most likely going to be in a bit of trouble. Typically what happens is that the running process gets killed prematurely, causing puppet’s state file to become corrupted. This isn’t the end of the world. However, if the lock file did not get removed as part of the service shutdown, you might be in slightly more serious trouble. Normally at this point, you would probably need to log in to the machine via SSH to correct the issue. Not a huge deal, unless you are managing a large number of systems.

All of this trouble can be avoided. At some point, maybe puppetd will be able to catch the SIGHUP correctly and handle this whole thing gracefully on its own. Until that day comes, I have come up with a small script that will help avoid this.

Essentially, what this script does is fork an “until” loop. The loop will check if the “puppetdlock” file exists, which would indicate that a puppet run is in progress. If it does, the loop will go back to sleep for a second and then try again. This is repeated until the file goes away, indicating the whatever was running is now done. At this point it would be safe to do a puppet restart, so the loop ends and the restart is then carried out.

This of course will not give you an accurate “puppet restarted successfully” message, because the only thing really being tested here is the success of the fork operation. You must rely on centralized logging or similar to catch the unlikely puppet service failure.

#!/bin/bash
# File Name:     restart-puppetd.sh
# Author:        Ryan Uber <ryan@blankbmx.com>
#
# Description:   This script is a hack! However, it solves a very important
#                issue with puppet. Normally, if you subscribe the puppet
#                service to the puppet.conf file, the puppet service will
#                be restarted too soon, interrupting the current puppet
#                run. Various attempts at using configure_delayed_restart
#                among other things have not proven to be 100% effective.
#                This script will watch the puppetdlock file, which can
#                determine whether or not there is a run in progress. If
#                there is a run in progress, we sleep for a second and then
#                test again until the process is unlocked. Once unlocked, we
#                can safely call a puppet restart. The checker process
#                itself gets forked into the background. If it were not
#                forked into the background, the puppet run would sit and
#                wait for the process to return, or for the exec timeout,
#                whichever came first. This would cause serious trouble if
#                timeouts were disabled or very long periods of time.

# Begin waiting for the current puppet run to finish, then restart.
/bin/sh -c "
    until [ ! -f /var/lib/puppet/state/puppetdlock ]
    do
        sleep 1
    done
    /sbin/service puppet restart" &

# Always return true, since this script just forks another process.
exit 0

# EOF

I realized that not all of the files need the same permissions, and not all of them need the same owner or group. Most manifests, however, will contain 3 basic types: a package, a file, and a service, or “PFS“. These 3 types are related to the same thing, and most likely will have the same owner and the same group at the minimum. My thinking behind this is, you should only need to specify what user / group is going to own these files and folders one time, and have it apply to everything else.

To demonstrate, here is what a typical puppet manifest might look like, writing everything out, plain and simple:

file {

    "file_1":
        path    => "/path/to/file_1",
        owner   => "someuser",
        group   => "somegroup",
        mode    => 0644,
        content => "This is a test.\n";

    "file_2":
        path    => "/path/to/file_2",
        owner   => "someuser",
        group   => "somegroup",
        mode    => 0644,
        content => "This is another test.\n";
}

You can already see the patterns emerging even though we have only defined two files so far. So lets say we have 5 of these files. With the above method of defining the files / attributes, your manifests might become quite long, especially if you are defining more attributes that are common, like a require or maybe a notify.

The following example achieves the exact same effect as specifying the owner, group, and mode in each file definition, saving us 15 lines of duplicate definitions in just 5 file statements:

File {
    owner   => "someuser",
    group   => "somegroup",
    mode    => 0644
}

file {

    "file_1":
        path    => "/path/to/file_1",
        content => "This is a test.\n";

    "file_2":
        path    => "/path/to/file_2",
        content => "This is another test.\n";

    "file_3":
        path    => "/path/to/file_3",
        content => "This is test 3.\n";

    "file_4":
        path    => "/path/to/file_4",
        content => "This is test 4.\n";

    "file_5":
        path    => "/path/to/file_5",
        content => "This is test 5.\n";
}

Now things might get a little trickier. You obviously won’t have every file you manage owned by the same user, or the same group, or with the same permissions. Two things come into play here:

1. The fact that we are only defining the defaults, and
2. Scope

Since what we have specified for owner, group, and mode already are only the defaults, you can still define those attributes per-file. For instance:

File {
    owner   => "someuser",
    group   => "somegroup",
    mode    => 0644
}

file {

    "file_1":
        path    => "/path/to/file_1",
        content => "This is a test.\n";

    "file_2":
        path    => "/path/to/file_2",
        owner   => "someotheruser",
        content => "This is another test.\n";
}

In the above example, “file_1″ will get the defaults for all 3 attributes, and therefore be owned by “someuser:somegroup”. “file_2″, however, overrides the default owner, and will thus have ownership of “someotheruser:somegroup”.

Now for scope. Suppose I have an Apache class and a MySQL class. These should not have the same ownership. However, if I have defined the files, services, and other things related to each piece of software in separate classes, then I am in luck.

Global defaults can be defined per-class, and are inherited.

File {
    owner   => "root",
    group   => "root",
    mode    => 0700;
}

class "mysql"
{
    File {
        owner   => "mysql",
        group   => "mysql",
        mode    => 0644
    }

    file { "my.cnf":
        path    => "/etc/my.cnf",
        content => "This is a test.\n";
    }
}

class "httpd"
{
    File {
        owner   => "apache",
        group   => "apache",
        mode    => 0644
    }

    file { "httpd.conf":
        path    => "/etc/httpd/conf/httpd.conf",
        content => "This is another test.\n";
    }
}

You can see how the global defaults are defined at the top here, as “root:root” with “0700″ permissions. Then, within each class, new defaults are set for the files, and therefore, within the scope of that class only, all file statements get the class-specific permissions.

Use this technique throughout your manifests, and you will notice they will start to appear much more simplistic and organized while accomplishing the same result. Also keep in mind that this makes it infinitely easier to modify attributes at a wide scale without re-keying the modifications again and again. Do be warned, however, adding a new attribute to a defaults definition will affect many files, so be sure that it will not negatively impact any one particular item in your manifests.

Feedback would be greatly appreciated. I’m not really a kernel developer but would love to know if you have any suggestions.

The "consoleblank=" is a very handy little addition to the kernel boot arguments,
especially nowadays with virtual machines growing in number quite rapidly.

There is, generally speaking, no damageable screen attached to most Linux machines
these days. Since using "setterm" does not make customizations persistent, the only
way to disable screen blanking without this patch is by adding some script-fu
to /etc/rc.local or similar start script, with "setterm -blank N", for each TTY
you do not wish to activate a screensaver for.

Rather, let's add this functionality (which is already available in later versions
of the Linux kernel) as a configurable argument to the kernel boot line.

The "consoleblank" kernel argument will take an integer value, which will specify
the blanking timeout in seconds. Set it to "0" to disable screen blanking entirely.

Ryan R. Uber <ryan@blankbmx.com>

--- a/Documentation/kernel-parameters.txt	2011-02-08 23:30:42.758108332 +0000
+++ b/Documentation/kernel-parameters.txt	2011-02-08 23:31:14.309331657 +0000
@@ -425,6 +425,10 @@
 			switching to the matching ttyS device later.  The
 			options are the same as for ttyS, above.

+   consoleblank=   [KNL] The console blank (screen saver) timeout in
+           seconds. Defaults to 10*60 = 10mins. A value of 0
+           disables the blank timer.
+
 	cpcihp_generic=	[HW,PCI] Generic port I/O CompactPCI driver
 			Format:
 			<first_slot>,<last_slot>,<port>,<enum_bit>[,<debug>]
--- a/drivers/char/vt.c	2011-02-08 23:30:39.667988520 +0000
+++ b/drivers/char/vt.c	2011-02-09 00:16:56.715662063 +0000
@@ -171,8 +173,24 @@
 int console_blanked;

 static int vesa_blank_mode; /* 0:none 1:suspendV 2:suspendH 3:powerdown */
-static int blankinterval = 10*60*HZ;
 static int vesa_off_interval;
+static unsigned long blankinterval = 10*60;
+
+/*
+ *  Newer kernel releases use core_param here, as seen below:
+ *  core_param(consoleblank, blankinterval, int, 0444);
+ *  The EL5 kernel does not yet have the core_param() function,
+ *  so we backport by using module_param_named() instead, add
+ *  a call to __setup(), and provide a function to set the
+ *  proper value.
+ */
+static int __init consoleblank_config(char *val)
+{
+    blankinterval = simple_strtoul(val,NULL,0);
+    return 0;
+}
+__setup("consoleblank=", consoleblank_config);
+module_param_named(consoleblank, blankinterval, int, 0444);

 static DECLARE_WORK(console_work, console_callback, NULL);

@@ -1374,7 +1392,7 @@
 			update_attr(vc);
 			break;
 		case 9:	/* set blanking interval */
-			blankinterval = ((vc->vc_par[1] < 60) ? vc->vc_par[1] : 60) * 60 * HZ;
+			blankinterval = ((vc->vc_par[1] < 60) ? vc->vc_par[1] : 60) * 60;
 			poke_blanked_console();
 			break;
 		case 10: /* set bell frequency in Hz */
@@ -3374,7 +3392,7 @@
 		return; /* but leave console_blanked != 0 */

 	if (blankinterval) {
-		mod_timer(&console_timer, jiffies + blankinterval);
+		mod_timer(&console_timer, jiffies + (blankinterval * HZ));
 		blank_state = blank_normal_wait;
 	}

@@ -3408,7 +3426,7 @@
 static void blank_screen_t(unsigned long dummy)
 {
 	if (unlikely(!keventd_up())) {
-		mod_timer(&console_timer, jiffies + blankinterval);
+		mod_timer(&console_timer, jiffies + (blankinterval * HZ));
 		return;
 	}
 	blank_timer_expired = 1;
@@ -3438,7 +3456,7 @@
 	if (console_blanked)
 		unblank_screen();
 	else if (blankinterval) {
-		mod_timer(&console_timer, jiffies + blankinterval);
+		mod_timer(&console_timer, jiffies + (blankinterval * HZ));
 		blank_state = blank_normal_wait;
 	}
 }