Tony Field’s Blog

Hello! It seems that a bit of time has gone by since I’ve dusted off this blog. A long while, perhaps, based on the number of spam comments in the moderation queue. In any event, I’m going to start again. I’ll cover several topics, but be warned, I’m going to get technical.

Over the last few months I’ve ramped up my consulting company, Metastratus Web Solutions, quite a bit, and have taken a few steps back from FatWire Software where I’ve been parking most of my working hours over the last few years. Part of that is due to the fact that FatWire was recently acquired by Oracle. (That is a whole topic itself). Long story short, Metastratus is now an Oracle Gold partner and I’ve set out as an elite team of WebCenter Sites (formerly FatWire Content Server) specialists, to help customers with their implementations.

That’s going well. We were lucky in that we hit the ground running. Metastratus has been in operation for almost two years now and we have a team of 5 today, myself included. In the last couple of months, we’ve opened a US company, have started hiring, and began building relationships with several key partners in this small but rapidly growing community. And we’re hiring.

I must admit that getting all of this set up has been a huge amount of work, and so it’s really at the front of my mind a lot these days. Consequently, it’s going to probably dominate the blog for a bit. But besides just the whole setup topic, I’ll be posting technical topics regularly, and I won’t be able to resist the occasional personal post too. Some of you know that I’ve been pretty involved in the creation of the GST Site Foundation (”GSF”), so you can look here for posts on that topic too.

So that’s the intro for now. Watch this space, and please don’t hesitate to comment below.

As for now, its’ time to head home to prepare for a concert! The Etobicoke Community Concert Band is performing its first series concert, “From Rags to Riches”, tonight at ECI. It’s my first concert since coming back after a brief hiatus, so I’m looking forward to it. Very short rehearsal schedule though so we are hoping for some good celestial alignment (today is the Rapture (revised) after all).

Over the last several years, I’ve managed to migrate my mac through hardware updates and OS updates without having to rebuild my system from scratch. Pretty cool, I thought - I used to have to do that about once every nine months on my PC.

Well, this August I finally bit the bullet. I wanted to clean out some unused but obscure files and basically get my system humming again. So, I fired up SuperDuper!, imaged my Mac, and then popped in my system disk and wiped it completely clean.

I installed Mac OS X 10.4 (Tiger) but first I had to choose the file system. I noticed that I now had the choice of selecting HFS+ (Journalled) (case-sensitive) that I don’t remember seeing earlier. Interesting. A case-insensitive file system can be really annoying - as a Java developer working on some pretty old code, I’ve got the infamous COM/com package naming convention issue to deal with among other things. When I rebuilt my computer, I went for the (case-sensitive) option to see how it would go.

I quickly installed all of my apps (a cookbook for rebuilding your computer from scratch is a handy file to keep on your hard drive just in case you need to actually use your backups now and then). After a week or so of testing, I was satisfied that all was well. This was great - I now had a case-sensitive file system and a regular Mac and everything just worked.

… until someone asked me to work on a web page using Dreamweaver.

I bought Adobe Creative Suite 3 Web Premium basically as soon as it came out. Native Intel support was key for me, and there were some pretty cool features in Dreamweaver, FireWorks and Photoshop that I wanted to try out. I had it installed on my mac before the rebuild, but left it to last afterwards because I don’t use it all that much and it really takes a long time to install anyway.

Bad idea. Apparently CS3 won’t install on HFS+ (journalled) (case-sensitive). I should have expected this. Dreamweaver 8 didn’t work on a case-sensitive file system either, but I figured Adobe would have fixed this when they bought Macromedia.

Long story short, I needed to reformat my hard drive to HFS+ (journalled) just to be able to install CS3. I couldn’t even install CS3 on a dedicated partition - it needs the system partition to be case-insensitive.

I thought about it for a bit and then eventually fired off an email to Dave Nanian at Shirt Pocket. The plan was to image my case-sensitive drive with SuperDuper!, wipe my Mac, reformat the drive with a case-insensitive file system, then restore from my image. Dave thought it would work so I set my plan aside for 3 months till I had either a huge need or an extra half day to do the conversion. (100GB takes a long time top copy back and forth).

Well, yesterday I tried it out and after working through a couple of glitches, it worked! Perfectly!

The technical instructions:

Steps to convert your HFS+ (journalled) (case-sensitive) file system to a HFS+ (journalled) file system without having to re-install your whole OS

- Install ShirtPocket software’s SuperDuper! Pay for it.
- Create a full image of your hard drive on a remote volume using SuperDuper!’s standard mechanism. Use a remote volume, like another firewire-equipped Mac or an external hard drive. I chose to skip all the caches but that’s up to you - the default settings work fine.
- Create a second image just in case
- Test out the image: open it and spot-check some of your files. They’ll work but it’s comforting to see them actually working.
- Insert your Mac OS X disk (or the restore CD that came with your Mac) and restart, holding down C
- Pray
- Open Disk Utility and reformat your drive using HFS+ (journalled). Name it the same thing as it was named previously.
- Plug in the drive holding the backup into your Mac. Booting from CD you don’t get network support, so if your image is on an external Mac, like mine was, you’ll have to boot the second Mac in FireWire Target Disk Mode.
- Select your drive and click on the restore tab. Navigate to your image and select it, then drag your target disk onto the Destination box.
- Make sure erase is unchecked; click restore.
- Reboot
- Rejoice
- Flame Adobe about the case-sensitivity issue

Now, to be honest, the above instructions were my plan. It didn’t quite go as planned. I had 3 issues.

First, I had used a sparseimage (read/write image) in SuperDuper! because it’s the default recommendation. The problem is, Disk Utility doesn’t like restoring from a sparseimage, and these images are greyed out. I ended up having to convert my sparseimage to a dmg using Disk Utility on my other Mac (a G5 iMac) (don’t try doing this over the network if you place any value on your time). That worked fine.

The second problem I had was a little more disturbing. Disk Utility told me to drag my target drive onto the Destination box, but there was nothing to drag! I could not figure out how to restore back onto my main hard drive. Nice: I had the drive wiped but could not restore it. I never found out how to do this but did find an easy way to do this through the command-line version of Disk Utility. As such, the instructions above are amended as follows:

…
- Plug in the drive holding the backup into your Mac. Booting from CD you don’t get network support, so if your image is on an external mac, like mine was, you’ll have to boot the second Mac in FireWire Target Disk Mode.
- Quit Disk Utility
- Open Terminal
- Type the following:
# asr /Volumes/MountedExternalDrive/path/to/your-image.dmg --target /Volumes/YourDestinationDriveName -noverify
and press return
- Wait a really long time. For my 100GB 7200RPM 2.5″ drive this took about 5 hours
- Reboot
- Rejoice
- Flame Adobe about the case-sensitivity issue

After rebooting, I fired up OnyX and ran basically all of the standard cleaning and maintenance jobs, the rebooted again. I figured it was worth pro-actively running some of these jobs rather than letting my Mac find issues related to obscure filesystem issues.

After the second reboot, everything was running perfectly. I was able to install Adobe CS3 and it worked.

Plus, as an added bonus, SuperDuper! recognizes most of the files on the old image as being the same as the ones on the new drive, which means my next backup was still just an incremental backup. If you don’t know what I’m talking about, read about SuperDuper! here.

That’s basically it. As far as I’m concerned, Dave saved me hours of rebuilding. Plus, having a perfect backup of my hard drive each night is good for peace of mind too. I’m not sure how SuperDuper! will play with Leopard now that it’s out (though he does talk about it here), but one thing’s for sure, given some of the upgrade issues, I’ll be running a full backup before upgrading.

Well that’s it. A successful but scary project, and now I’m back up and running… and ready to try my Leopard upgrade…

Tony

Since switching to Mac OS X 10.4’s mail client, I have noticed that my email coming through my own server has been listed as “Received” four hours later than when it actually landed in my mailbox. I thought it was just some minor issue on my desktop until I heard about this from someone else using the same mail client.

I had worked around it by having Mail display the “Date Sent” field instead of the “Date Received” field, which instead shows the time the composing mail client sent the message instead of the time my POP server received the message. But with multiple people complaining about this issue, I decided to look into it.

The problem was not immediately obvious. The email messages looked as if they were properly formatted, but after consulting the Internet Message Format specification I discovered what was happening. It turns out that my mail server was using a valid but obsolete date format to indicate the time zone. Specifically, it was displaying dates like this:

Fri, 21 Sep 2007 20:07:09 GMT

When in reality it should be displaying that date like this:

Fri, 21 Sep 2007 20:07:09 +0000

This format would have been fine too:

Fri, 21 Sep 2007 20:07:09 +0000 GMT

Section 4 of the spec specifically states that applications that conform to the specification must not generate invalid dates, but must be able to interpret them. So what does that mean?

Well, it means that my mail server was not conforming, and that my mail client (Mail.app) was also not conforming. That explains why so few people ran into this issue: you have to encounter both bugs to see the problem.

So what to do? Since I run the mail server, I can at least try to configure my way out of the conformance issue. As for the Apple Mail client, well, the best I can do is report a bug (Bug ID 5503215) and hope it gets fixed. It’s a minor bug, and I assume that Apple will be able to fix it soon if it is deemed to be important enough, but it’s certainly not within my control.

Back to sendmail. Sendmail is notorious for being difficult to work with. Its configuration files are so complex that there is a second format for a “simplified” configuration file that you “compile” into the regular configuration file. A quick internet search, however, led me to the setting that controls the mail server’s time zone: You just have to define the variable confTIME_ZONE in your .mc file and set it to the right value and you’re off to the races.

Perfect, that’s just what I wanted. So what is the right value? Well, that’s another issue altogether. Sendmail documentation is not very helpful here. It provides some help but not nearly enough. Here is what I have been able to piece together:

Leaving the variable blank will use the “system’s” time zone. I assume this is what was happening on my server, but I don’t really know what this means.

Setting the variable to “USE_TZ” will cause sendmail to use the value of the TZ environment variable. The problem here is that sendmail is executed in a way that makes defining this variable in the environment a little tricky. The last thing I needed was another tricky configuration setting so I nixed this option

The last option is to set it to an actual time zone explicitly. The format for this string, however, is not very flexible. The example provided in the Sendmail Installation and Operation Guide is PST8PDT. It turns out that the format is defined like this: The first three characters are the string representation of the time zone name for standard time, the next character is a digit that represents “the difference from GMT” for the time zone, and the last three characters represent the string representation of the time zone for daylight savings time. In this case, Pacific Standard Time, 8 hours “different” and Pacific Daylight Time.

Without going on a rant about poorly conceived this particular format is, let me just say that it doesn’t quite scale very well, or apply particularly well if you want to set your server to GMT and ignore daylight savings time. RFC 2822 at least talks about several email time zone strings for Eastern, Central, Mountain, and Pacific time that conforming applications must understand, so if you’re setting confTIME_ZONE you should probably just stick to one of these values. My server is in the Eastern Time Zone so I set my value to EST5EDT.

I made those changes, restarted the server and that fixed the problem. My mail client now understands when my mail message was received properly, and my server is configured to generate date stamps properly.

To summarize, to get sendmail to emit a proper date format, add the following to your

/etc/mail/your.hostname.com.mc

file:

define(`confTIME_ZONE', `EST5EDT')

then re-save it. You will then need to recompile your sendmail.cf file and then restart sendmail by typing make install restart from your /etc/mail folder.

/etc/mail/# make install restart

At that point you will be good to go.

Happy mailing!

Welcome to my blog.

I launched this blog not because I am ready to stand up on a soapbox and start pontificating, but primarily because I once in a while have a few ideas worth sharing, or at least worth not losing.

Topics, I assure you, will be all over the map. I’ll attempt to categorize them should you want to subscribe, but please don’t expect regular updates.

Thanks for visiting, and feel free to leave your comments against any post.