SpamAssassin Filter Failure

Posted by Ryan Uber | Linux,Mail,Spam,Zimbra | Monday 6 September 2010 6:11 pm

An interesting topic that I had forgotten entirely about until recently was a pretty serious SpamAssassin flaw in one of the built-in filters. Many of you have probably seen this in the past:

3.4 FH_DATE_PAST_20XX The date is grossly in the future.

I noticed this on a Plesk server today. Apparently, they have not released any of their “hotfixes” that would prevent this from tagging legitimate email as spam.

Just look at that score that it gets you, a whopping 3.4! Most people find it necessary to set their spam filters to a pretty aggressive threshold, maybe around 4-ish for a message to never make it to your inbox, but go directly to spam. That means that in essence, almost any other SpamAssassin or email-in-general no-no would sink you email battleship.

Whoever wrote this rule initially never took into account the “20XX” is actually inclusive of the current year, 2010. This also begs the question, why on earth is this particular rule written relative to the time the rule was published, rather than relative to the current date on the server instead?

Now there may have been a patch or similar released for this by now that does just that, but in case the version you are running is “organizing” all of your legitimate email into the spam folder, here is a quick and dirty way around it. I run Zimbra, so the file location in this example might not match your installation. Add the following information to:

/opt/zimbra/conf/spamassassin/local.cf:

# This is the right place to customize your installation of SpamAssassin.
#
# See 'perldoc Mail::SpamAssassin::Conf' for details of what can be
# tweaked.
#
# Only a small subset of options are listed below
#
#############################################

#   Add *****SPAM***** to the Subject header of spam e-mails
#
# rewrite_header Subject *****SPAM*****

score FH_DATE_PAST_20XX 0.0

#   Save spam messages as a message/rfc822 MIME attachment instead of
#   modifying the original message (0: off, 2: use text/plain instead)
#
# report_safe 1

#   Set which networks or hosts are considered 'trusted' by your mail
#   server (i.e. not spammers)
#
# trusted_networks 212.17.35.

#   Set file-locking method (flock is not safe over NFS, but is faster)
#
# lock_method flock

#   Set the threshold at which a message is considered spam (default: 5.0)
#
# required_score 5.0

#   Use Bayesian classifier (default: 1)
#
# use_bayes 1

#   Bayesian classifier auto-learning (default: 1)
#
# bayes_auto_learn 1

#   Set headers which may provide inappropriate cues to the Bayesian
#   classifier
#
# bayes_ignore_header X-Bogosity
# bayes_ignore_header X-Spam-Flag
# bayes_ignore_header X-Spam-Status

Restart SA, and you should be good to go. As I said already this is a dirty fix that really should get some attention if it hasn’t already.

Basic Server Statistics Script

Posted by Ryan Uber | Apache,Linux,Performance,PHP | Saturday 4 September 2010 12:21 pm

Recently at work, I found myself needing to give a customer access to a few basic statistics on his web servers, but seeing as they are 100% managed, and they get no type of access other than FTP, I had to find a way to deliver them the numbers they wanted by means other than SSH.

FTP would not be a good candidate, it doesn’t make sense that they would need to FTP down a file with stats information every time they wanted to check it out. I decided to “KISS” (keep it simple / stupid) and wrote the following PHP script to get the job done:

<?php
/* Codero Dedicated Support Team <dedicatedsupport@codero.com>
 *
 * File Name: get_server_stats.php
 * Author:    Ryan R. Uber <ryanu@codero.com>
 *
 * Provides an easy http-accessible script to display commonly needed
 * server statics information.
 *
 * Note: Enable exec() to get Apache connection stats.
 * Run:  echo -n "PASSWORD HERE" | md5sum
 *       to generate a password
 */

# Authentication Definitions
$useAuth = false;
$authMD5 = 'Paste generated MD5SUM here';

# Perform login
if ( $useAuth === true )
{
    session_start();

    if ( $_SERVER['REQUEST_METHOD'] == "POST" )
    {
        if ( md5 ( $_POST['password'] ) == $authMD5 )
        {
            $_SESSION['loggedIn'] = 'true';
        }
    }

    # Prompt for password
    if ( ! isset ( $_SESSION['loggedIn'] ) )
    {
?>
<form action=<?=$_SERVER['PHP_SELF']?> method=POST>
<font size=5>Authentication required</font><br>
<font size=2><b><i>Password: </b></i><input type=password name=password><br>
<input type=submit value="Authenticate">
</form>
<?php
        die();
    }
}

# Initialize Variables
$hostname           = $_SERVER['HTTP_HOST'];
$unique             = array();
$www_unique_count   = 0;
$www_total_count    = 0;
$proc_count         = 0;
$display_www        = false;

# Check if 'exec()' is enabled
if ( function_exists ( 'exec' ) )
{
    $display_www = true;

    # Get HTTP connections
    @exec ( 'netstat -an | egrep \':80|:443\' | awk \'{print $5}\' | grep -v \':::\*\' |  grep -v \'0.0.0.0\'', $results );
    foreach ( $results as $result )
    {
        $array = explode ( ':', $result );
        $www_total_count ++;

        if ( preg_match ( '/^::/', $result ) )
        {
            $ipaddr = $array[3];
        }

        else
        {
            $ipaddr = $array[0];
        }

        if ( ! in_array ( $ipaddr, $unique ) )
        {
            $unique[] = $ipaddr;
            $www_unique_count ++;
        }
    }
    unset ( $results );
}

# Get Server Load
$loadavg = explode ( ' ', file_get_contents ( '/proc/loadavg' ) );
$loadavg = "{$loadavg[0]} {$loadavg[1]} {$loadavg[2]}";

# Get Disk Utilization
$disktotal = disk_total_space ( '/' );
$diskfree  = disk_free_space  ( '/' );
$diskuse   = round ( 100 - ( ( $diskfree / $disktotal ) * 100 ) ) . "%";

# Get server uptime
$uptime = floor ( preg_replace ( '/\.[0-9]+/', '', file_get_contents ( '/proc/uptime' ) ) / 86400 );

# Get kernel version
$kernel = explode ( ' ', file_get_contents ( '/proc/version' ) );
$kernel = $kernel[2];

# Get number of processes
$dh = opendir ( '/proc' );
while ( $dir = readdir ( $dh ) )
{
    if ( is_dir ( '/proc/' . $dir ) )
    {
        if ( preg_match ( '/^[0-9]+$/', $dir ) )
        {
            $proc_count ++;
        }
    }
}

# Get memory usage
foreach ( file ( '/proc/meminfo' ) as $result )
{
    $array = explode ( ':', str_replace ( ' ', '', $result ) );
    $value = preg_replace ( '/kb/i', '', $array[1] );
    if ( preg_match ( '/^MemTotal/', $result ) )
    {
        $totalmem = $value;
    }

    elseif ( preg_match ( '/^MemFree/', $result ) )
    {
        $freemem = $value;
    }

    elseif ( preg_match ( '/^Buffers/', $result ) )
    {
        $buffers = $value;
    }

    elseif ( preg_match ( '/^Cached/', $result ) )
    {
        $cached = $value;
    }

}
$freemem = ( $freemem + $buffers + $cached );
$usedmem = round ( 100 - ( ( $freemem / $totalmem ) * 100 )  ) . "%";
?>

<html>
<body>
<font size=5><b><?=$hostname?></b></font><br><br>
<?php
if ( $display_www === true )
{
?>
<font size=4><b>Web Server (80 and 443)</b></font><br>
<font size=3><b><i><?=$www_unique_count?></b></i></font><font size=2> unique connections</font><br>
<font size=3><b><i><?=$www_total_count?></b></i></font><font size=2> total connections</font><br>
<?php
}
?>
<font size=3><i><b>Kernel Version:</b> <?=$kernel?></i></font><br>
<font size=3><i><b>Uptime:</b> <?=$uptime?> days</i></font><br>
<font size=3><i><b>Load Average:</b> <?=$loadavg?></i></font><br>
<font size=3><i><b>Disk Use:</b> <?=$diskuse?></i></font><br>
<font size=3><i><b>Memory Utilization: </b><?=$usedmem?></i></font><br>
<font size=3><i><b>Total Processes: </b><?=$proc_count?></i></font><br>
</body>
</html>

How do I use it?
Just place the above script anywhere in an Apache-servable directory with libphp5 loaded. I attempted to make as much of the script “exec-free” as possible, so currently you only need to configure the exec() function (as mentioned in the script header) if you want to be able to see the number of connections to the Apache webserver. I’m assuming that you have this turned off on your web server, although the PHP default .ini leaves it enabled.

Authentication
This script has a small built in “wanna-be” authentication mechanism. If you want to require a password for a viewer to see this valuable information, all you would need to do is set the “useAuth” variable to “true”, and paste in an md5-hash for the password. I didn’t both with a user for this script.

Get the MD5 password:

$ echo -n 'soooooooSecure' | md5sum

Configure the script:

$useAuth = true;
$authMD5 = "[paste the md5 from the above command here]";

Alternate password method
You can also use a .htaccess file or similar if you want to have multiple users and passwords. Simply disable authentication in the script, and continue on your way with the htpasswd command.

$useAuth = false;

Head-first into Object-Oriented PHP

Posted by Ryan Uber | PHP | Saturday 4 September 2010 11:57 am

I have been writing PHP for years now. I have consistently used common classes, like Pear’s Mail and Net-SMTP. I understood how to use the classes, but I never looked at what was actually happening behind the scenes until just recently. I never really saw a reason to write classes, I always thought “My functions are just as good, why bother?”.

As some of my PHP projects grew larger and larger, I began to think to myself, “If I stopped working on this, how would someone else pick up where I left off and start developing on it?”. The short answer is, they wouldn’t. No matter how many comments I put in my code explaining what I am doing, no one wants to have to read a novel of them just to understand how my config parser is doing its job. I had read that object-oriented code was far easier to maintain, so I created a new SVN repository with my procedural code, and began modifying it script-by script, at first just as an experiment, but it has since evolved and is now far better than what I had before.

Some things I found very beneficial in using classes versus my traditional, procedural code include:

Organization — Normally, in procedural coding, you have one or maybe a few different “functions” scripts that provide your commonly-used functions, such as a database connection procedure or event logger. There is really no easy way to organize them, and there is no “real” standardization to the way you are going to call them. Say I have a database connect function, and a database query function. If I name them appropriately, I might be able to find them in my functions include later on when I need to modify it. However, say I have one function named dbConnect(), and then another function called db_query(), I now have two separate functions that are closely related, yet they have entirely different naming conventions. Some projects (not my own, thankyouverymuch) that I have seen will even have multiple functions for the same functionality, either with different naming, different code content, or even both. This will quickly turn a coding project into a spider web of garbage. With the object-oriented model, you generally will have something like the following:
(more…)

« Previous Page