Spec. Edition 8.12.04
All Back Issues
Let’s Fight Sp@m!
NetBEUI and Win XP
Letter Mail Donate
Recommend Scot’s Newsletter to a Friend!
October 2004 - Vol. 4, Issue No. 63
By Scot Finnie
IN THIS ISSUE
Now 60 days later, it's time to come clean. Should you install Windows XP Service Pack 2?
Before I give you my answer, you need to know some of the background that went into my thinking. The two main reasons not to install XP SP2, at least not right away, are:
1. Installation Woes and Other Problems
2. Web Browsing, Email, and Other Application Inconveniences
1. Installation Woes and Other Problems. Some people are having very serious problems during installation of XP SP2. In fact, there are several different install problems have been identified. Some research a few weeks ago turned up four different SP2 install problems that Microsoft had acknowledged, and that number has probably risen since. You can be sure that there are other problems as well that haven't been identified yet.
To help readers of my PC Today column and Scot's Newsletter navigate any troubled waters they may encounter with XP SP2, I've put together a list of Microsoft SP2 resources:
This links page covers:
- Pre-Install Must Reading
- Installation Issues
- SP2 Problems/Solutions
- Uninstalling SP2
- Internet Explorer
- Outlook Express
- Windows Firewall
- Microsoft Knowledge Base Searches
A look at this list will also show you some of the trouble spots, which extend into areas beyond installation.
Update for XP SP2
A couple of days ago Microsoft released "Update for Windows XP Service Pack 2 (KB885894)" designed to solve what's probably the most common installation problem with XP SP2: A freeze part-way through the setup process, leaving you with a failed installation. If that happens to you, Microsoft is offering this 760K download. You install it, restart your machine, and you should be able to complete your XP SP2 installation successfully.
Note: This installation bug fix is poorly named. "Update for Windows XP Service Pack 2" sounds like it's something everyone should download and run, but that is not the case. It's only if you run into trouble with a hung installation.
I'd like to thank SFNL reader Mark Brent (A.K.A "markb") for bringing the Update for Windows XP Service Pack 2 to my attention so quickly. Thanks, Mark!
Darker Reader Reports
You should be aware that the tenor of the XP SP2 emails from SFNL readers over the last 30 days has been increasingly negative -- including more varied installation problems and other issues after installation. The thing is, that's the normal course of events with a Windows service pack. There are always a long list of problems, but the numbers of people who actually experience the exact same problems aren't usually large. Microsoft knew this would be the case when it decided to go with a more ambitious release. This is a bite-the-bullet upgrade. They bit the bullet in delaying Longhorn significantly to bring it to you; and you should approach this upgrade with a bite-the-bullet mentality too.
The primary goal of providing a significantly greater degree of security for all Windows users is a worthy one. But no pain, no gain. XP SP is in no way a painless upgrade because it makes many under the covers changes for security sake. I believe that 60% of the people who install it will wish this or that worked like it used to, or have more fundamental problems.
2. Web Browsing, Email, and Other Application Inconvenieces. I wrote a feature story for the January 2005 issue of PC Today magazine (due out soon) that you should keep an eye out for. It details the 10 Things You Need to Know About Windows XP Service Pack 2. In particular, it charts some of the changes in behavior under Windows XP SP2. And that's the second area of "reasons not to install XP SP2" worth considering before you install it. In particular, I'm talking about the way things work in Internet Explorer, Outlook Express, and issues with application incompatibility. Many people have reported problems with Norton AntiVirus 2004, for example. In some cases, serious issues. Symantec has been pretty quick to release patches for things they know are wrong, but everyone -- including third-party software makers -- are still climbing a learning curve in the wake of XP SP2's wide release.
My personal pet peeve is intermittent pop-ups that temporarily block file downloads from some shareware sites. The problem doesn't appear universally, and it depends on the way download sites serve up files. But some people are reporting serious issues related to this too, like the inability to install or run files they download. And anyone who downloads frequently could quickly find themselves using an alternative browser after installing XP SP2. (SP2 is clearly going to drive additional acceptance of the Mozilla Firefox and Opera browsers; all of my computers are running Firefox as a secondary browser.) Some people have already reported that this or that problem or inconvenience caused them to uninstall XP SP2 entirely. I can sympathize with that inclination, although I have not felt compelled to do that (I've just tested it to ensure Uninstall works properly).
Because I have had no trouble installing XP SP2 on 12 machines now and counting, it's this second area -- inconveniences and changed behaviors -- that I find to be a larger, longer-term concern. Another pet peeve is the changes that SP2 brings to the Internet Control Panel's Security tab settings. The settings are not explained well, the words are ambiguous in some places, and there's little of use from the Help text, which in many contexts offers nothing but the same generic sentence or two. What becomes clear is that some of the most annoying protect-you-from-yourself security settings don't come with any way (at least via the user-interface) to change their defaults. Most of the security changes I agree with, but some of them border on paranoia.
If you know of Registry hacks that allow us to revise the behavior of security controls under SP2, I'm very interested in anything you can pass along. I hope to cover customizing SP2 in a future issue of this newsletter.
The Bottom Line
Despite all of the above -- and despite numerous reports posted on beta sites, forums, and newsgroups about dire or just messy problems with XP SP2 -- I believe that the time has come for at least some people to install it. I've been through many service packs and Windows updates and similar changes with several other operating systems over the last 25 years. 60 days later, it doesn't look to me like Windows XP Service Pack 2 will stand out as one of the more problematic Windows upgrades. That said, here are my specific recommendations:
1. Any IT person reading this newsletter who is responsible for rolling out XP SP2 on multiple seats, test the heck out of this upgrade before you do so. In particular, test it with unique, low-cost, regional, or outdated hardware that you may be using. Hardware support is changed with Windows XP Service Pack 2. Probably almost as much as with your average major point release of Windows, judging from the mail I've been getting lately. That's been a little surprising to me. But application issues are the ones that are even harder to fix. Do your homework by checking the functionality of all your apps first. Be sure. If your environment is secure and you encounter app and/or hardware problems, the benefits of XP SP2 probably do not outweigh the disadvantages. I'm not saying never install it. I think everyone should eventually install this release. But business environments are probably the riskiest ones for XP SP2. Hang back until additional problems are worked out, newer drivers are available, and application patches become available.
2. Individual users who are experienced, if you have not already installed this service pack -- and especially those who stay on top of issues with your computers -- the time has come to consider installing it. The vast majority of people have encountered no serious issues with SP2. Remember, people who have no problems don't post in newsgroups, etc. It's just another day for them. And this phenomenon, that negative experiences are magnified, skews our sense of the real-world experience toward the negative. Be careful not to get sucked all the way into the buzz; it's not statistically representative.
For most people, the negative results, if any, are a bevy of small annoyances (most of which are not universal at all). It's just not enough to keep you from exploring SP2. The update is also fully uninstallable, and the Uninstall seems to work pretty well (although it does leave Internet Connection Firewall turned on). If possible, install it on a machine other than your main machine first to see how it goes. But I don't think you should hang back any longer just because of potential major universal problems. After 60 days, the threat of that is less apparent.
There's one caveat to my recommendation to go ahead and install: If you are fully protected with a software firewall, an up-to-date always-running antivirus program that scans your email at least inbound, and you have installed all critical security updates (other than SP2) for Windows XP, there's also no wild rush to install SP2. There are numerous vulnerabilities that Microsoft has plugged in XP SP2, but the majority of those have been previously released. If you have a solid understanding of the Internet and the threats it poses, delaying your installation of SP2 for a while longer is not the end of the world at all. If you don't feel like dealing with a potential hassle, hang back. Every day that goes by, Microsoft and OEM PC makers are learning more and more about the issues that some people have with XP SP2. And they're working on solutions for them. A few months down the line, many problems that some people are having now will be avoided by subsequent patches and workarounds. So long as your security is sound and you're savvy about Internet threats, do not feel compelled to install this update right away.
3. Relatively inexperienced users or those who are not very confident about their ability to extricate themselves from computer problems, you represent the toughest choice. This user group, which is by far and away the largest group of Windows users, stands to gain the most from Windows XP Service Pack 2. I can't promise you an easy installation experience of Windows XP Service Pack 2. I can tell you that the majority of people have literally no problem whatsoever. It's been estimated that only 10% of people who install this upgrade experience a serious problem. My data shows the success rate might even be slightly higher than that.
So, it's my considered opinion that for you, the overall threats from the Internet and how you might interact with it outweigh the potential trouble you might encounter during or after installation of Windows XP SP2.
There is one thing you can do to help yourself: Bone up on XP SP2 before you install. Start by consulting these documents. And review the rest of the documents on that page as well. Arm yourself with information. Plus, review these pre-installation tips you should take seriously.
Before you install SP2 (or any Windows upgrade), please follow these steps:
1. Consult Scot's Newsletter's 60 Useful Windows XP SP2 Links. Also, check the Scot's Newsletter Forums' Windows Forum for the latest word on issues and known problems.
2. Back up your personal data. My preference is for using imaging software like Norton Ghost for this purpose. Your backup should be made to a separate hard disk (another partition is OK, but a separate physical hard drive is better), network volume, or writable DVD.
3. Check with your system manufacturer about known issues with SP2 and your machine. Specific models of PCs may be more prone to SP2 trouble than others. So consult with your system manufacturer or major component makers.
4. Temporarily disable or uninstall software firewall and antivirus software before installing. I can't tell you how many times people have come to me with Windows upgrade problems who have not followed this step. If you neglect to do this, it doesn't mean an automatic melt down of your PC, but the incidence of woes is very high if you don't -- higher than even many experienced PC users realize. After installation is complete, be sure to re-enable your security software.
Remember, Windows XP SP2 turns on its own Windows Firewall automatically. It is possible for two running software firewalls to conflict with each other, so that's a second reason to disable your firewall.
5. Check your hard drives for disk errors. In XP, that means running chkdsk /f from a command line. You'll need to restart your machine to allow this process to run on your boot volume.
6. Make a named Windows XP System Restore point beforehand. (Check out Basic Instructions on Working with System Restore.)
7. Shut down all unnecessary applications. This comes under the heading "don't tempt fate." The admonition includes software with open windows and also those running in background. Most Windows services are not at issue, but application-installed background applications should be dumped where possible.
The easiest, surest way to do this is with the System Configuration Utility. Click Start > Run > Type "msconfig" and press Enter. Click the Startup tab. Remove checks beside items that you want to temporarily disable. If you're not sure whether to turn something off, don't. After you install SP2, remember to come back here and set things back the way they were. (Or uninstall some of these things if you don't want them. Before you uninstall, though, turn them back on here on the Startup tab.)
For more information about items you find in the System Configuration Utility's Startup tab (or on the Ctrl-Alt-Del Windows Task Manager's Processes tab), please consult Scot's Newsletter Link of the Week-Award-winning Startup Contents' database of background applications. It gives you information about programs running on your system based on the cryptic names that Windows reports. Startup Contents, by Paul Collins, is one of the most valuable Windows helps sites on the Internet.
Back to the Top
* * *
For more than a year I relied on Spamnix, a low-cost plug-in for Eudora Email, to rid my inbox of spam. Spamnix, which I've covered quite a bit in past, is a Bayesian filter augmented by the SpamAssassin rules-based anti-spam system. Spamnix works pretty well, and also is in beta test of the 2.0 release with several new features. The thing I like best about Spamnix is its integration within Eudora. By selecting an open or closed message (or multiple closed messages in the inbox) and pressing Ctrl-J, you both send the message(s) to the spam folder and also train the filter.
My trouble with Spamnix is as much Eudora's fault as Spamnix's. When Spamnix is running, Eudora has a tendency to crash. In fact, under Windows XP, Eudora has a tendency to crash. The combination of Spamnix and Eudora caused me to see on average one Eudora crash a day. The crashes are clearly related to mail volume. I estimate that I receive 25,000 emails a month, all of which comes to a single Eudora installation. I was able over time to minimize the problem. But three weeks ago I experienced one crash too many. I decided to uninstall Spamnix. I've recommended POPFile to you in the past because of its accuracy rate. I've even tested it briefly in the past and worked with it in non-production environments. But the complexity of my email setup and some aspects of the way POPFile works kept me from using it in the past.
But POPFile's shortcomings, which I'll come back to, have nothing to do with accuracy. POPFile is the most accurate mail-classification tool I've ever tested. It's also more useful than other anti-spam utilities. Its powerful classification functionality can be used to identify "buckets" of mail based on virtually any logical grouping you can come up with. So long as you can conceive of a clear distinction between message types you configure in POPFile, and you're consistent about training POPFile as messages arrive, it will be able to imitate your decisions extremely accurately in only a few weeks time. There's no doubt about it: POPFile's built-in capabilities are extremely powerful.
A Three-Week Test
So, when I reached the end of my rope with Eudora and Spamnix, I cut the latter loose and installed POPFile. After a bit of trial and error, and on the advice of Mitch Wagner, a long-time user of POPFile who is both an SFNL subscriber and someone I work with at TechWeb, I configured POPFile very simply with two buckets: Normal and Spam. And I set about training it what spam is. It's a lot more work at first than I would have preferred, but the results are simply amazing. It has a very low incidence of false positives (when the filter tags non-spam as spam). POPFile seems to learn much better than other Bayesian spam filters I've used. That is a very important plus.
The training downside was that it took two solid weeks before I started to see more accuracy than I had during Spamnix's first day. But after three weeks, spam-identification accuracy is over 98%. And I expect it to keep climbing because I'm continuing to train it.
My problems with Eudora? Under POPFile, I'm encountering less than 50% of the Eudora crashes I grew accustomed to when using Spamnix. If nothing else, that's reason enough to switch. All in all, POPFile is the most accurate spam catcher I've ever used. There are other products like it on the market (SpamBayes comes to mind). But I really like POPFile, and intend to continue working with it.
That doesn't mean that POPFile is perfect. In fact, I have a long list of issues with or desires for the product:
1. POPFile is a proxy server. It requires you to reconfigure your email program (there's a wizard that does this for you, and it works very well) to redirect mail through a generic local port so that POPFile can literally filter your mail before it gets to your email package. Essentially, your email software stops interacting with the world outside for incoming mail; POPFile does that. And your email package interacts with POPFile. There's nothing inherently wrong with that concept, but two issues arise from underlying structure:
a. If you decide to uninstall POPFile, you're going to have to reconfigure your email package. If you have a complex multiple account system (as I do), you might want to create a file with all your existing mail settings before you install POPFile.
b. The straight truth is that proxy servers create problems on user PCs. Some antivirus programs (such as Norton AntiVirus) also have proxy servers that may conflict. But the bigger issue is that proxy servers sometimes just stop working. They're touchy. I've seen some issues like this with POPFile (especially with v.0.22.0). Occasionally when I try to access its UI at the same time that my email package is attempting to retrieve email, the POPFile program stops working. Exiting and restarting the program solves the problem. But this is typical of proxy servers, in my experience.
2. This is the most important problem with POPFile. It doesn't offer email program user-interface integration. Another program, called Outclass by VarganSoft, was written to allow POPFile to integrate with Microsoft Outlook. People who use Outclass say it's helpful. But there is no Outclass for Eudora planned that I am aware of (nor so far as I know Outlook Express, Netscape, or any other email client). What's the big advantage of specific email program integration? Being able to train the filter by pressing Ctrl-J (as I described above) is one of the better examples.
POPFile presents all the email you receive in a Web-mail-like interface of its own devise. In other words, it's an entirely different view of your mail. You can open mail in a Web window, but HTML mail is not displayed. In fact, there should be a non-default option to display HTML in this program. Instead you see the HTML code that you have to try to visualize through to understand whether a message is important to you, or spam, or whatever classifications or "buckets" you've set up.
To help you check POPFile's decisions, the program can present views of your received mail based on the buckets you've created. While the UI it offers is fairly good for bulk checking and retraining, that's only useful for training the product in batch mode. What would be far better is if you could do the same thing in your email package -- where you actually work with your email. POPFile fails to provide a useful interface that will cut back on the amount of work people have to do to properly train it.
The one nod POPFile gives to working with your email in place is derived from the option to add a specific header to all your received email. The header adds a link in every message shown in your email package that, when clicked, opens the same message in POPFile. There, in the POPFile version of the message, you can choose a message classification from a drop-down menu to retrain POPFile.
There are several things wrong with that way of working. For one thing it takes too many clicks to accomplish. For another it requires you to open the message ... twice.
There's a third drawback with that method of user-handled manual reclassification. In order to explain it, let's delve a little deeper into how POPFile interacts with your email package. POPFile is able to modify your messages automatically as it receives and classifies them. It can modify the subjects of your received messages with a little tag at the front of the message, such as [spam], which match your classification buckets. It can also add two types of internal message headers. Message headers appear at the top of your received messages, above the body of the message. The first of these is the one I referred to earlier that lets you open the same message in POPFile. The second is the Classification header. It's the best way to allow POPFile to stamp a given message as being Spam, Normal, or any other POPFile message classifications you create.
So why do you need the Classification header? It lets you easily set up message-filtering rules in your email program by giving your email package something simple to search for and then act upon. So, for example, POPFile can append something like this header to all messages it identifies as being spam:
Then a rule in your email can look for any header text containing "iamsp@m" and route that mail to your junk mail folder. This, in fact, is the only way that POPFile can actually separate mail. It's an email-classification tool, not a email-management tool.
So, back to that third drawback to the minimal integration POPFile offers with your email package. The trouble is that when you use POPFile to open a message and reclassify it, say from Normal to Spam, it doesn't rewrite the classification header on the message in your email program. So your email program will continue to think a normal message (in this example) is spam. And that means you'll have to move the message to your spam folder manually. Yet another step. Compared with an integrated product like Spamnix, this functionality is archaic.
Bottom line: The lack of integration with your email package (unless you use Outlook) means a LOT more work for you. The extra accuracy POPFile provides may not be worth the extra effort required by you to manage POPFile. Having to manually tag more false negatives (spams that get through) by highlighting and pressing Ctrl-J (or something like that) is so much less work than what you have to go through with POPFile that the accuracy advantage probably does not outweigh the integrated solution's overall efficiency.
3. There's no built-in, discoverable feature for bulk training of message types in POPFile. So, for example, if you have an already existing folder of 1,000 spams, and you want to train POPFile to identify all those messages as spam to get it up and running faster, there's no user interface control surface to that. There is a way to do it using a Perl command, but that functionality is neither convenient nor exposed in the program. It's not hard to do, just not convenient. You'll find instructions on the Web if you go looking. (Here they are.)
4. I mentioned above that POPFile is an email-classification tool, not an email-management tool. And I think that's a mistake. Because it is both highly accurate and proxy server-based, POPFile is missing an opportunity to offload some of your email overhead by preventing spam from ever arriving in your email package. What if POPFile held back all your spam (or other unwanted email) and just allowed you to pore through them (as it does now) to find any exceptions (false positives) that might be there? Then when you discovered exceptions, it would release them to your email package. On my system, nearly 78% of the messages I've received over the last three weeks are spam, some 13,000 messages. An additional 4,000 messages arrived during the same time period that weren't spam. When you look at those numbers -- generated by POPFile's excellent email statistics -- it becomes clear that POPFile could be doing a lot more to improve your email experience. Reducing the amount of mail that flows into my email package by 75% would be a serious advantage. POPFile is missing large opportunities.
Down to This
So those are the big shortcomings I see so far. They're non-trivial, but then, POPFile's accuracy advantage is also significant. POPFile is a study in contrasts. For me, any spam program that requires users to drastically change the way they work with their email is effectively a failure. And by that measure, POPFile fails my own test. For that reason, at this time I am not awarding POPFile the Scot's Newsletter Top Product Award. But POPFile's potential is huge. Especially if its makers or others create integration tools for other mail clients, such as Netscape, Outlook Express, and Eudora. That's why I'm sticking with it for the time being.
Back to the Top
I would like to conduct a one-issue test of the newsletter that would automatically deliver the newer, simpler HTML edition to Text Edition subscribers. Existing HTML Edition subscribers would be wholly unaffected by the test.
In a nutshell, for this one time only, Text Edition subscribers whose email packages are capable of displaying HTML would receive the HTML edition of the newsletter. Text Edition subscribers whose email packages are incapable of displaying HTML would receive the text edition.
There would be no change to your subscription whatsoever. The test is, in fact, extremely easy for me to carry out, and there's no permanence to it at all. Bottom line: The people who would be affected by it are Text Edition subscribers whose email packages are capable of displaying HTML. Those subscribers will see the HTML edition one time.
My honest belief is that more than half of the SFNL Text Edition subscribers would actually prefer the HTML Edition once they see it in their mailboxes. It is easier to read; easier to navigate, and by and large, the satisfaction level of my HTML subscribers is much higher. I know this from comments about the newsletter, from people who unsubscribe, from people who have tried both editions. The HTML edition is easier to read, more convenient, and I think you'll remain a subscriber longer if you're getting that version.
At TechWeb where my day job is running the 20 Pipeline sites -- each of which has its own newsletter -- we have converted all our newsletters to this mode of operation. The process is known as a multipart message, and virtually all newsletter-distribution systems provide a way to do it. It was my decision to do this with Pipeline newsletters, and we studied it for a good long while before we made the change. But after the fact, there been almost no complaints at all from Pipeline newsletter readers. I expected a lot more complaints than we got.
There is really no short-term benefit for me in changing to this way of working. And, after all, it's only a test I'm talking about. If I were to someday convert Scot's Newsletter to multipart operation, it would simplify several things for me and also save money:
1. The Newsletter subscription page would be much easier for everyone to work with, and the only complex aspect of this page (picking the HTML or Text edition) would go away. Many subscription problems would virtually disappear.
2. The fact that I'm currently maintaining two lists one for HTML and one for Text causes many other issues too. The biggest is that it costs more money every month. My newsletter distributor charges a flat monthly fee for each list I maintain. Combining the HTML and Text lists into one would also simplify the process of sending the newsletter while vastly reducing the likelihood of mailing errors.
3. Some of you have urged me to offer a premium edition of the newsletter for a low-cost annual subscription fee. Making this change would make it easier and cheaper for me to do that. I'm not right on the verge of offering a premium edition, by the way. (And if ever do, I would continue to offer a free edition as well.) But I am thinking about it more and more. Truly, though, this is not the driving force behind the HTML test.
4. I've said this already, but the single biggest benefit would be that an increased number of my readers having access to easier-to-read version of the newsletter. That alone is worth it to me.
5. There's the possibility of a small future revenue stream in custom-generated advertising, such as Google's AdSense, that would only work with HTML newsletters. Right now, AdSense can't provide ads in email newsletters (and it may never do so). But I am using AdSense on my websites, and so far that is generating small, incremental income that helps defray the expenses of putting out a newsletter, running websites, and running a forum. And it does so without adding any images to the page (or email message). If Google AdSense could be included in the newsletter, there's a possibility that I might be able to run the newsletter without ever having to ask for donations again.
I know that many of you mistrust HTML-delivered content. There are no shenanigans going on here. Scot's Newsletter is about as aboveboard as you can get. I know some of you may need to be convinced about that, and so I'm happy to answer any questions you have on the topic.
So, anyway, what do you think about the idea of the HTML Test for Text Subscribers? I would like to pursue that, and then perhaps follow up about how it went. Let me know what you think.
Back to the Top
Scot's Newsletter is a monthly "e-magazine." My aim is to deliver each issue of the newsletter on or before the first of each month. Next month, I'm expecting to deliver the newsletter the week of November 1.
You can always find out when the next issue of Scot's Newsletter is expected by visiting the Scot's Newsletter home page.
Back to the Top
The Fine Print
If you like this newsletter, I need your help spreading the word about it. Please share it with friends and co-workers, and encourage them to sign up! It's free.
Visit the new Scot's Newsletter Forums.
Subscribe, Unsubscribe, Change Email Address or Message Format
You can unsubscribe at any time; I don't believe in captive audiences. The website subscription center is the easiest way to manage your Scot’s Newsletter subscription. Changes take only a minute or two. You must select your message format — Text or HTML — even for address changes or unsubscribes. All subscription changes are now handled on the same page with a database-look-up feature that ensures greater accuracy:
The Scot’s Newsletter Subscription Center:
To help with the cost of creating and distributing the newsletter, I accept contributions via PayPal and Letter Mail. For more information on donations:
Send comments, suggestions, or questions about this newsletter. Don't be bashful about telling me what you like or don't like. Send emails related to editorial content (only) to firstname.lastname@example.org.
Please address advertising inquires (only) to: email@example.com
How to Link to Scot’s Newsletter
Copyright © 2001-2007 Scot Finnie. All Rights Reserved.
Ten Myths About Copyright Explained.
You are subscribed to Scot's Newsletter HTML EDITION as: $subst('Recip.EmailAddr')