EL6 is a large step up from the 5.x family – There are hundreds of improvements that every sysadmin will be eager to take advantage of; newer versions of most popular packages, a new kernel based on 2.6.32, new software added to the enterprise linux “base”, etc. With so many large changes to the operating system, in-place upgrades become harder. You have a gang of EL5.x boxes and want to move them to the 6.x distribution, but the fence between the two distributions is a fairly high one, the largest feat being the updates to YUM, python (2.6), and RPM itself (4.6+). You can’t read new RPM packages with an old version of RPM (pre-4.6 can’t read EL6-ready RPMs), and there is no 4.6 version of RPM available for the EL5 distribution. Upgrading core components such as glibc and python in-place can be scary too. However, the jump from EL5 to EL6 is not impossible. I’ll outline here how I was able to make it happen.

What you can’t be afraid of to accomplish this

1. Forcefully breaking RPM dependencies (will be resolved upon completion)
2. Ignoring package checksum mismatches
3. Rebooting

New version of RPM
Seeing how its not possible to install EL6 RPM’s with EL5′s stock version of RPM, the first thing to do is to get that newer version of RPM installed on your system. Keith Chambers, a friend and colleague of mine, recompiled RPM 4.6 against glibc 2.5 so that it would work with the current EL5 system. RPM 4.8 may have worked as well, but 4.6 worked for me, and I only needed to use this version during the big upgrade. RPM will replace itself with the stock EL6 version later on. For dependencies, I needed to include xz-libs and lua. I force-upgraded these packages (–nodeps). What you want to end up with is:

# rpm --version
RPM version 4.6.0

Patch YUM
Since the new RPM’s coming from the repo will be verified with a newer hashing algorithm than what YUM/python2.4 will recognize, I needed to make a few quick patches to YUM itself to force it to ignore conflicts. These ghetto-hack patches will be automatically removed from the system when YUM upgrades itself, so really this is only needed one time.

diff -Nur /usr/lib/python2.4/site-packages/yum.orig/depsolve.py /usr/lib/python2.4/site-packages/yum/depsolve.py
--- /usr/lib/python2.4/site-packages/yum.orig/depsolve.py       2011-08-19 15:07:02.000000000 +0000
+++ /usr/lib/python2.4/site-packages/yum/depsolve.py    2012-01-07 20:43:21.634917883 +0000
@@ -142,6 +142,7 @@
                             'repackage': rpm.RPMTRANS_FLAG_REPACKAGE}

         self._ts.setFlags(0) # reset everything.
+        self._ts.addTsFlag(rpm.RPMTRANS_FLAG_NOMD5)

         for flag in self.conf.tsflags:
             if ts_flags_to_rpm.has_key(flag):
diff -Nur /usr/lib/python2.4/site-packages/yum.orig/__init__.py /usr/lib/python2.4/site-packages/yum/__init__.py
--- /usr/lib/python2.4/site-packages/yum.orig/__init__.py       2011-08-19 15:07:02.000000000 +0000
+++ /usr/lib/python2.4/site-packages/yum/__init__.py    2012-01-07 20:43:01.074133746 +0000
@@ -1217,6 +1217,7 @@
                 failed = True

+        failed = False
         if failed:
             # if the file is wrong AND it is >= what we expected then it
             # can't be redeemed. If we can, kill it and start over fresh
diff -Nur /usr/lib/python2.4/site-packages/yum.orig/yumRepo.py /usr/lib/python2.4/site-packages/yum/yumRepo.py
--- /usr/lib/python2.4/site-packages/yum.orig/yumRepo.py        2011-08-19 15:07:02.000000000 +0000
+++ /usr/lib/python2.4/site-packages/yum/yumRepo.py     2012-01-07 20:43:09.834466442 +0000
@@ -1467,6 +1467,7 @@
             file = fn.filename
         else:
             file = fn
+        return 1

         try:
             l_csum = self._checksum(r_ctype, file) # get the local checksum

I wrote a few quick lines in SED to handle this patching for me:

sed -i '/^        if failed:/i\        failed = False' /usr/lib/python2.4/site-packages/yum/__init__.py
sed -i '/^            file = fn$/a\        return 1' /usr/lib/python2.4/site-packages/yum/yumRepo.py
sed -i '/# reset everything.$/a\        self._ts.addTsFlag(rpm.RPMTRANS_FLAG_NOMD5)' /usr/lib/python2.4/site-packages/yum/depsolve.py

Package Conflicts
There are a number of packages while performing an upgrade from EL5 to EL6 that will cause you trouble, as some of the version numbers have actually decremented, and some cause dependency resolution issues that will be solved only after the upgrade is complete. The following is a small snippet that I used to force my way out of this situation:

declare -ra FORCEREMOVE=(
    m2crypto centos-release newt authconfig prelink tcp_wrappers sgpio
    iscsi-initiator-utils mkinitrd dmraid dmraid-events hmaccalc sysfsutils
    device-mapper device-mapper-multipath device-mapper-event
    vmware-open-vm-tools-common vmware-open-vm-tools-kmod less usermode
    libhugetlbfs lvm2 kpartx e4fsprogs-libs glib libsysfs
)
for PKG in ${FORCEREMOVE[@]}; do
    rpm -e --nodeps ${PKG}
done

PAM authentication work-around
By default, the RPM’s that install PAM do not clean out its include directory (/etc/pam.d). If the authentication modules you are using change at all (new ones added, old ones removed), you will need to manually clean out this directory. I found it easiest to just remove all of the configs and let them be re-populated by the new RPM’s. Without this step, after the upgrade completed, I was unable to log in to the console of the machine, because some authentication modules specified in the config files were now absent (no longer needed in the product I work on). If you have custom configs, you will need to account for them somehow, hopefully using Puppet or your configuration tool of choice.

rm -f /etc/pam.d/*

Re-install centos-release
You will notice in one of the steps above that I force-removed the centos-release package. The reason I needed to do this was because of the following:

package centos-release-5-7.el5.centos.x86_64 (which is newer than centos-release-6-0.el6.centos.5.x86_64) is already installed

By force-removing the centos-release package, and then performing a “yum install centos-release”, the centos-release package gets updated to the 6.x version.

Downgrade nss
This is a very important step. The version of “nss” actually decremented between EL5 and EL6. Since the new glibc 2.12 requires nss-softokn-freebl, which requires nss, we need to downgrade nss. This step should downgrade nss, install a few new packages, and update glibc. Once you do this, there is really no going back.

# yum downgrade nss
Loaded plugins: fastestmirror
Setting up Downgrade Process
Loading mirror speeds from cached hostfile
Resolving Dependencies
--> Running transaction check
---> Package nss.x86_64 0:3.12.7-2.el6 set to be updated
--> Processing Dependency: nss-softokn(x86-64) >= 3.12.7 for package: nss
--> Processing Dependency: nss-util >= 3.12.7 for package: nss
--> Processing Dependency: libnssutil3.so(NSSUTIL_3.12.3)(64bit) for package: nss
--> Processing Dependency: nss-system-init for package: nss
--> Processing Dependency: libnssutil3.so(NSSUTIL_3.12)(64bit) for package: nss
--> Processing Dependency: libnssutil3.so(NSSUTIL_3.12.5)(64bit) for package: nss
--> Processing Dependency: libnssutil3.so()(64bit) for package: nss
---> Package nss.x86_64 0:3.12.8-4.el5_6 set to be erased
--> Running transaction check
---> Package nss-softokn.x86_64 0:3.12.8-1.el6_0 set to be updated
--> Processing Dependency: nss-softokn-freebl(x86-64) >= 3.12.8 for package: nss-softokn
---> Package nss-sysinit.x86_64 0:3.12.7-2.el6 set to be updated
---> Package nss-util.x86_64 0:3.12.8-1.el6_0 set to be updated
--> Running transaction check
---> Package nss-softokn-freebl.x86_64 0:3.12.8-1.el6_0 set to be updated
--> Processing Dependency: libc.so.6(GLIBC_2.7)(64bit) for package: nss-softokn-freebl
--> Running transaction check
---> Package glibc.x86_64 0:2.12-1.7.el6_0.5 set to be updated
--> Processing Dependency: glibc-common = 2.12-1.7.el6_0.5 for package: glibc
--> Running transaction check
---> Package glibc-common.x86_64 0:2.12-1.7.el6_0.5 set to be updated
--> Processing Conflict: glibc conflicts binutils < 2.19.51.0.10
--> Restarting Dependency Resolution with new changes.
--> Running transaction check
---> Package binutils.x86_64 0:2.20.51.0.2-5.11.el6 set to be updated
--> Processing Conflict: glibc conflicts prelink < 0.4.2
--> Restarting Dependency Resolution with new changes.
--> Running transaction check
---> Package prelink.x86_64 0:0.4.6-3.el6 set to be updated
--> Finished Dependency Resolution

Dependencies Resolved

=========================================================================================
 Package                   Arch          Version                     Repository     Size
=========================================================================================
Updating:
 binutils                  x86_64        2.20.51.0.2-5.11.el6        base          2.8 M
 prelink                   x86_64        0.4.6-3.el6                 base          994 k
Downgrading:
 nss                       x86_64        3.12.7-2.el6                base          735 k
Installing for dependencies:
 nss-softokn               x86_64        3.12.8-1.el6_0              base          166 k
 nss-softokn-freebl        x86_64        3.12.8-1.el6_0              base          115 k
 nss-sysinit               x86_64        3.12.7-2.el6                base           26 k
 nss-util                  x86_64        3.12.8-1.el6_0              base           46 k
Updating for dependencies:
 glibc                     x86_64        2.12-1.7.el6_0.5            base          3.7 M
 glibc-common              x86_64        2.12-1.7.el6_0.5            base           14 M

Transaction Summary
=========================================================================================
Install       4 Package(s)
Upgrade       4 Package(s)
Remove        0 Package(s)
Reinstall     0 Package(s)
Downgrade     1 Package(s)

Once this has completed, you are clear to upgrade the rest of the system.

# yum -y upgrade

Re-install a few packages
A few of the packages that we force-removed for dependency resolution reasons will not get automatically installed. Therefore, we need to install them by hand:

declare -ra VMW_PKGS=(
    vmware-tools-core
    vmware-tools-foundation
    vmware-tools-guestlib
    vmware-tools-libraries-nox
    vmware-tools-plugins-guestInfo
    vmware-tools-plugins-vix
    vmware-tools-services
    vmware-tools-plugins-deployPkg
)
yum -y install ${VMW_TOOLS[@]} prelink lvm2 less

You can ignore the vmware packages if you are not using them.

Remove the old kernels
You should no longer need any of your EL5 kernels. You have hopped the fence and are in the land of EL6 now. You should be able to delete (rpm -e) any kernel-2.6.18* packages that are still installed.

Rebooting
Since I added all of the above logic in a script and executed it during a pre-install stage with Puppet, I had to have a way to automatically reboot the machines as well. In one of my previous posts, Puppet Self-Management, I detailed how to patiently wait for a puppet run to finish before executing some action from within a script. I applied this same technique to the reboot for my EL5 to EL6 upgrade script. Since I am executing my upgrade script during a puppet run, I do not want to bounce the machine before the run completes to avoid corrupting the state, and also to ensure that all of my other puppet-managed updates are applied before the machine boots in to EL6 for the first time. I accomplished this with the following code at the very end of my upgrade script:

/bin/sh -c "
    until [ ! -f /var/lib/puppet/state/puppetdlock ]
    do
        sleep 1
    done
    /sbin/shutdown -r now" &

During a trip to the local computer store, found myself walking through the Apple section. Normally I won’t be found in there as I generally run Linux for just about everything (desktop, server), but this time, the LED Cinema displays really caught my eye. Such a beautiful display, so bright and crisp, and so clean-looking. I had to have one for myself. (Of course I didn’t buy at the store. Ordered online and saved $100.)

Now then, first thing’s first: Connecting the thing to the PC. The Apple 27″ LED display that I purchased comes with *only* a mini displayport connection. No DVI, no HDMI. This actually didn’t bother me so much, since the mini displayport is present on a number of ATI cards. I set out to find a new graphics card. The first one I came across (and actually ended up purchasing alongside the monitor) was the Apple GeForce GT120. Yes, its an Apple product meant to be an upgrade to the Mac Pro’s from early 2009, but it was a PCI Express card, nothing out of the ordinary (probably just re-branded). Remembering back to my Hackintosh days, I figured that I probably had the best bet with this card. Apple makes it, so it should work right?

Wrong! While the DVI graphics worked just fine on an older monitor (Samsung P2350), the mini displayport would give me no video. I am guessing that Apple probes the card in some way that my PC does not. Also note, at this point I had still never seen the display even power on, since the new LED displays are controlled completely via the MDP, and thus there is not a single button on the entire monitor.

A little disgruntled, I removed the GT120 from my PC and moved on to the next solution: an ATI card with MDP. I found one at my local Fry’s Electronics, and ordered online for local pickup. It was the AMD HD 6870 XOC. All of the specifications looked correct on the site; 1x Dual-Link DVI, 1x Single-Link DVI, 1x HDMI, and 2x mini displayports. I picked the card up from the store and headed home for installation. To my surprise, when I pulled the card out of the box, it had 2x DVI ports and 1x displayport, not even a minidispalyport or any HDMI connections. It was identical to the item on the website in every other way, shape, and form, but the interface description on the left side of the box had a sticker over it with the downgraded specifications.

Needless to say, I had to return it. I didn’t want to run any displayport to mini-displayport adapters or any other nonsense like that.

I ended up finding the XFX version of the Radeon HD 6870 at MicroCenter for around the same price. This time I verified before walking out the door with the card.

While I was installing the card, I noticed the box sitting next to me mentioning the interface was PCI Express 2.1. I didn’t think anything of it, because I had done my homework beforehand and learned that any PCIE 2.1 card would work in a 2.0 slot.

To my surprise, after installing the card and connecting everything in my Gigabyte H55M-S2V, the card did not function at all! After further reading and research, I still did not find an answer as to why the card would not function in this particular motherboard. A BIOS update and still, nothing. Irritated and ready to just start using the darn monitor and stop fiddling with hardware, I went out and bought a new motherboard. My processor was LGA1156 based, so I had limited options. I ended up with an ASUS P7P55D-ELX. It had a few goodies that my old board didn’t, like 6GB/s SATA and USB 3.0. After further ripping apart/assembling my PC, I finally was able to boot up, and see the display work perfectly!

The BIOS POST showed up just fine, stretched to the size of the 2560×1440 native on the LED. Everything was all well and good. I did notice though, that the back light was looking pretty dim. I then realized that the factory default for the brightness setting is not set to the max, or even a reasonable brightness. It is in fact pretty dim, even with the ambient light detection/adjustment in the iSight. What I ended up doing to solve this was installing Windows 7, and on top of that, installing Boot Camp from a Snow Leopard DVD that I had. It is noteworthy though, that the version of Boot Camp that came on the Snow Leopard disk did not give me full access to the brightness control. Even with the slider all the way up to max, I still only had maybe 50% brightness. After running the Apple Software Update utility and getting the latest Boot Camp, and a subsequent reboot, I watched my Cinema Display come to life as I dragged the slider all the way up and the brightness began to burn my pupils!

After all of this fighting to get a Cinema Display to work sans the Mac, I would indeed say that it is worth it. After setting the brightness settings in Boot Camp, feel free to nuke your Windows installation and go back to using Ubuntu, Fedora, SuSE, or whatever it is you like using. The LED display will remember the settings internally and won’t need to set them again.

A note about the video card:
I found the XFX HD 6870 to be a bit noisy for my taste. Not overpowering noisy, but my tower sits right next to me on my desk and I don’t like the sound of fans for hours on end. Installing MSI Afterburner and setting an automatic fan profile like the following, the noise level is more than acceptable and the GPU remains relatively cool (~50-55C).

In conclusion: It is totally worth it. You should do it.

I recently ran into a situation where I needed to wrap text to a certain number of columns to support the standard 80×24 Linux console without making a complete mess of the screen. Since I found no standard way of doing this, I wrote this little function to handle it for me.

There are no dependencies on any external libraries. Pass in your storage, your string, and the number of columns to wrap to, and this will do the rest.

/*
 * This function will wrap large amounts of a text into a manageable and human-readable width
 * word by word. Just specify the number of columns you are working with and feed it a string,
 * and it will return a new string (including added line breaks) to accommodate the area you
 * are working with.
 *
 * {{{ proto( void ) wrap( char out, char str, int columns )
 */
void wrap( char *out, char *str, int columns )
{
    int len, n, w, wordlen=0, linepos=0, outlen=0;

    /*
     * Find length of string 'str' without using string.h
     */
    for( len=0; str[len]; ++len );

    /*
     * Allocate the full space of 'str' to 'word', so there is no possible way that the string
     * could contain a word that does not fit into the 'word' variable.
     */
    char word[len];

    /*
     * Loop through each individual character in the passed array (str) and detect white space
     * and word length to determine how to handle line wrapping.
     */
    for( n=0; n<=len; n++ )
    {
        /*
         * Detect spaces and newlines. We cannot gurantee that the passed string is null-
         * terminated, so we also need to handle cases where we reach the end of the string
         * without encountering wite space characters.
         */
        if( str[n] == ' ' || str[n] == '\n' || n == len )
        {
            /*
             * If the current word will not fit on the current line, add a newline.
             */
            if( linepos > columns )
            {
                out[outlen++] = '\n';
                linepos = wordlen;
            }

            /*
             * Append the found word to the output and reset the word character array in
             * preparation for accepting characters for the next word.
             */
            for( w=0; w<wordlen; w++ )
            {
                out[outlen++] = word[w];
                word[w] = '\0';
            }

            /*
             * If we reach the end of the string, add the null-terminator character.
             */
            if( n == len )
                out[outlen] = '\0';

            /*
             * If we encounter a newline character, append it to the string as usual, but
             * set the line position counter back to 0 (new line = start count from 0 again).
             */
            else if( str[n] == '\n' )
            {
                out[outlen] = str[n];
                linepos=0;
            }

            /*
             * If the word fits in the current line without trouble, just add the space.
             */
            else
            {
                out[outlen] = ' ';
                linepos++;
            }

            /*
             * Increment the final output length for the next loop, and set the word length
             * counter back to 0 (newline or space = new word).
             */
            outlen++;
            wordlen=0;
        }

        /*
         * If the current character is in the middle of a word somewhere, just append it and
         * move on, incrementing counters.
         */
        else
        {
            word[wordlen++] = str[n];
            linepos++;
        }
    }
}
/* }}} */

Examples

Wrap at 50 columns:

$ ./a.out
Lorem ipsum dolor sit amet, consectetur adipiscing
elit. Maecenas pulvinar blandit diam nec mattis.
Sed a ipsum nec ante porttitor feugiat. Morbi
ipsum lacus, dignissim at bibendum in, consectetur
eget ipsum. Donec dolor nibh, scelerisque ac
sodales et, posuere ac nulla. Aliquam eget
tincidunt ante. Vestibulum justo leo, congue ut
luctus ut, venenatis non dui. Cras sapien risus,
blandit at semper eu, cursus eu est. In hac
habitasse platea dictumst. Fusce libero dolor,
commodo nec rhoncus eu, rutrum vel arcu. Vivamus
ultrices faucibus tellus ac feugiat. Ut imperdiet
metus sed erat feugiat sed porta lorem tristique.
Mauris quis erat vel ante cursus pretium.
Curabitur arcu erat, consequat ac ornare et,
venenatis eu diam.

Curabitur posuere dui vitae tortor cursus cursus.
Pellentesque adipiscing lobortis sem at varius.
Mauris pellentesque sollicitudin ultricies.
Maecenas in tellus turpis. Integer convallis
mollis elit eu placerat. Nunc volutpat consectetur
facilisis. Curabitur eros arcu, dapibus ut
convallis non, rhoncus non magna.

Wrap at 20 columns:

$ ./a.out
Lorem ipsum dolor
sit amet,
consectetur
adipiscing elit.
Maecenas pulvinar
blandit diam nec
mattis. Sed a ipsum
nec ante porttitor
feugiat. Morbi ipsum
lacus, dignissim at
bibendum in,
consectetur eget
ipsum. Donec dolor
nibh, scelerisque ac
sodales et, posuere
ac nulla. Aliquam
eget tincidunt ante.
Vestibulum justo
leo, congue ut
luctus ut, venenatis
non dui. Cras sapien
risus, blandit at
semper eu, cursus eu
est. In hac
habitasse platea
dictumst. Fusce
libero dolor,
commodo nec rhoncus
eu, rutrum vel arcu.
Vivamus ultrices
faucibus tellus ac
feugiat. Ut
imperdiet metus sed
erat feugiat sed
porta lorem
tristique. Mauris
quis erat vel ante
cursus pretium.
Curabitur arcu erat,
consequat ac ornare
et, venenatis eu
diam.

Curabitur posuere
dui vitae tortor
cursus cursus.
Pellentesque
adipiscing lobortis
sem at varius.
Mauris pellentesque
sollicitudin
ultricies. Maecenas
in tellus turpis.
Integer convallis
mollis elit eu
placerat. Nunc
volutpat consectetur
facilisis. Curabitur
eros arcu, dapibus
ut convallis non,
rhoncus non magna.

*Update 11/24/2011: This solution is useful if you need to word wrap inside of a C program. If you are shell scripting, you can use the “fold” command, which comes standard with the “coreutils” package in EL-based distributions.

Usage: fold [OPTION]... [FILE]...
Wrap input lines in each FILE (standard input by default), writing to
standard output.

Mandatory arguments to long options are mandatory for short options too.
  -b, --bytes         count bytes rather than columns
  -c, --characters    count characters rather than columns
  -s, --spaces        break at spaces
  -w, --width=WIDTH   use WIDTH columns instead of 80
      --help     display this help and exit
      --version  output version information and exit

Report bugs to <bug-coreutils@gnu.org>.