Getting the system boot time

Recently someone in comp.sys.mac.programmer.help asked about code to find the system boot time on Mac OS X. Every so often I find a question like that and I just get the urge to figure it out.

He mentioned that he'd tried using getutxent(3) but that apparently it doesn't work on Mac OS X. I suspect this is because it relies on the user accounting database, and that this is not enabled. This approach might well have worked on Mac OS X Server.

This is the kind of thing that immediately suggests sysctl(3) to me, since sysctl knows all (or seems to). There's probably a solution buried somewhere in Carbon too, but a quick check at the command line confirmed I was on the right trail so I didn't look much farther:

$ sysctl -a | grep boot
kern.exec: unknown type returned
kern.boottime = Wed May  9 09:22:28 2007
kern.netboot = 0
kern.bootsignature: 48e4380a2edc32cb41fcd951c7d7882529f3bddf

Now to get it into code without requiring an external tool.

With sysctl() my first instinct is to use sysctlbyname(3) instead, since it's generally much easier to deal with. That's one of the methods SparklePlus uses to look up information about the host Mac-- the CPU type (Intel vs. PPC), CPU subtype (G5, G4, etc), CPU model (a unique string identifying the model of Mac), and number of CPUs are all looked up this way.

Since the sysctl(3) man page reveals that the KERN_BOOTTIME parameter is a struct timeval, this should be a simple matter of:

struct timeval boottime;
int error;
unsigned long length = sizeof(boottime);
error = sysctlbyname("kern.boottime", &boottime, &length,
	NULL, 0);

For some reason this always fails-- error is -1 and errno is ENOENT, suggesting I've passed an invalid name. Since the sysctl command-line tool accepts kern.boottime, I don't know why that's a problem.

Oh well, if the shortcut doesn't work, there's always the hard(er) way. It's not especially difficult but it seems a bit arcane until you get used to it.

time_t boottime()
{
	int mib[2] = { CTL_KERN, KERN_BOOTTIME };
	char *value;
	struct timeval boottime;
	size_t size = sizeof(boottime);

	if (sysctl(mib, 2, &boottime, &size, NULL, 0) == -1) {
		perror("sysctl(kern.boottime)");
		return -1;
	}
	printf("system boot time (seconds since epoch): %d\n",
		boottime.tv_sec);
	return boottime.tv_sec;
}

That's the short version, and as the printf indicates it gives you the boot time in terms of seconds since the epoch. From that you can convert it into whatever other format you like, so long as you remember when dealing with Unix the "epoch" is January 1, 1970 GMT, not the Mac OS X "reference date" of January 1 2001. A Cocoa developer might pass the result to NSDate's +dateWithTimeIntervalSince1970:, for example.

The above code could use some cleanup for production use, of course. However it should work on BSD systems other than Mac OS X.

As long as I'm on the subject I'll also include the longer approach to dealing with sysctl. This version looks up how much memory the sysctl result will need before actually getting that result. In this case I know the result is a struct timeval, so that's not necessary. In other cases you won't know the size requirements in advance. For example the hw.model key is a string of arbitrary length (probably up to a limit, but the limit's not documented).

The following does the same as the above, using this longer approach:

time_t boottime_long()
{
	int mib[2] = { CTL_KERN, KERN_BOOTTIME };
	char *value;
	size_t size;
	struct timeval boottime;

	// Pass NULL the first time so we just get size
	if (sysctl(mib, 2, NULL, &size, NULL, 0) == -1) {
		perror("sysctl(kern.boottime)");
		return -1;
	}
	if (size < sizeof(boottime)) {
		perror("sizeof(kern.boottime) < sizeof(timeval)");
		return -1;
	}
	// Allocate space for the result
	value = malloc(size);
	if (value == NULL) {
		perror("malloc");
		return -1;
	}
	// Now finally do the lookup
	if (sysctl(mib, 2, value, &size, NULL, 0) == -1) {
		perror("sysctl(kern.boottime)");
		return -1;
	}
	memcpy(&boottime, value, sizeof(boottime));
	free(value);
	printf("system boot time (seconds since epoch): %d\n",
		boottime.tv_sec);
	return boottime.tv_sec;
}

It's a lot more steps, but as I said sometimes it's the only way.


WWDC 2007 Mac ISV Event

Mac developers going to WWDC 2007 should check out the SF Mac ISV dinner event planned for Sunday, June 10. It's a sponsored event mainly aimed at the kind of people who read the Macintosh Software Business (MacSB) mailing list. But anyone going to WWDC is welcome, even those who are more developers than ISVs (Independent Software Vendors, like me).

I know who the sponsors are going to be, but I'm sworn to secrecy until they actually cough up some dough.

Since the big WWDC keynote is always first thing on Monday, most of us going to WWDC try and get into town the day before. But there are no official WWDC events on Sunday, so it's a good time to get together with friends who you might normally only meet online.

The primary organizer is Chuck Soper of Vela Design Group. I'm really pleased that Chuck is taking the lead on this. Last year I organized an informal MacSB dinner on the Sunday before WWDC. But since I'm not located anywhere near San Francisco I couldn't do much organizing. It was fun but low-key. If WWDC ever moves to Colorado I'll organize something bigger. Chuck, however, is in a position to make things happen in SF, making him the ideal person for the job.

If you plan on going, please sign up on the web site. Chuck needs to get an idea of how many people might attend so that he can finalize the details.


Pair blows it

Those of you who follow me on Twitter no doubt noticed that yesterday was no fun at all when it came to this site. Mostly due to some unexpected actions by my web host, the venerable Pair.

Red Alert #1
The first sign of trouble was a biggie. At 1:42PM local time I discovered that this site was completely down. As in, every page would produce an HTTP 500 "Internal Server Error". This is the kind of thing that can make a small ISV just about wet himself. I depend on the site to sell my software, and I depend on software sales to keep my house and buy food and stuff like that.

Now I've just done a completely new site design, with a whole mess of PHP code, and I had been fixing a couple of things in the morning. Naturally, I assumed the problem was my fault, and so I started desperately trying to figure out what I had broken.

This is where I first ran into trouble with Pair. If you're getting internal server errors, the first step is to look at the server error log and see what it's teling you. On Pair you can't get raw logs on a shared server, but their Account Control Center is supposed to show you errors related to your sites. But there was nothing there. As far as the ACC was concerned my site was fine, even though the site kept spewing 500s.

Left with no clues to go on, all I knew was "something's broke!". I checked my local copy of the site (the one that runs on my Mac) versus the live files. I made sure to undo every change I had made since I last knew things were working. I investigated whether the database had been hacked. Nothing.

Asking friends in #macsb for help, one pointed out that Pair had announced a PHP update yesterday, about 90 minutes before I noticed the site was broken:

php5.cgi will be upgraded today to PHP 5.2.2. This upgrade is necessary to patch a security vulnerability that was recently announced.

This will only affect you if you are currently using scripts with .php5 extensions or using php5.cgi through an '.htaccess' file for PHP-CGIWrap.

That's me right there. I run PHP as a CGI rather than an Apache module because it means I don't have to make my site files writable to everyone on the same shared server. Unix gave me file permissions, and I'm going to use them. The above was followed by instructions on how to upgrade to the latest PHP CGI.

It didn't help, though. I got on the phone to Pair. Eventually they managed to get me a working copy of PHP. The site came back up.

Normally I trust Pair to be reliable and not to break things unexpectedly. In this case I think they blew it in a major way. Their PHP upgrade immediately broke my web site, and this was done with no more than 90 minutes notice (the time from when they made the upgrade until I noticed something amiss). The announcement of the change was made on an internal Pair usenet server and on an RSS feed. You can't reasonably expect all of your customers to be constantly watching those. With this sudden, drastic change you need to be calling people's cell phones to alert them. And you can't make a change that's going to break people's web sites without telling them it's going to break the site. The announcement made it sounds like the upgrade was something I should do, not something I must immediately do to prevent my site from going offline. Geez, what if I'd been out of town? The site could have been down for weeks. I'm just lucky I was online to catch this before it went on too long.

Red Alert #2
It turned out that not all was well.

For Chimey and MondoMouse I use Aquatic Prime to generate license files. Aquatic Prime requires some server support if you want these be generated automatically. For me the normal process is:

  • Someone orders the software through my eSellerate-powered web store.
  • eSellerate sends my server an HTTP POST which contains information on the sale.
  • If the POST data looks good, Aquatic Prime generates a license file and emails it to the customer.

And of course, I'm using a PHP implementation of Aquatic Prime.

I soon discovered that although the site was up, my Aquatic Prime code was broken. Gaah! This meant that if someone ordered my software, they wouldn't get a registration code! Oh shit!

This led to much near-panicked research involving having eSellerate send some "preview" mode test sales to the server and new debug code in my copy of Aquatic Prime. I discovered that while eSellerate was correctly sending the POST data, said data was not actually making it into my PHP code. No sale data, no license file, no email.

The stock Aquatic Prime code for working with eSellerate (contributed to the project by yours truly) grabs the POST data out of PHP's global $HTTP_RAW_POST_DATA variable. Esellerate sends XML instead of key/value pairs, so that's a convenient way to get at the XML. But now $HTTP_RAW_POST_DATA was always an empty string.

I tried running a phpinfo() on the new PHP, and discovered that the always_populate_raw_post_data setting was "Off". That certainly explained what I was seeing, because without that you don't get $HTTP_RAW_POST_DATA. But why would it suddenly be off? I guessed that Pair had changed the PHP configuration file without warning, a charge they denied (although it did take several emails back and forth before I could even get someone from support to understand what I was asking about).

Having no useful clues from Pair, or from PHP's release notes or changelog, I proceeded to find a work-around to get the license code going again. My fix is as follows:

if ($_SERVER["REQUEST_METHOD"] == "POST") {
    $HTTP_RAW_POST_DATA = file_get_contents("php://input");
}

This bypasses the always_populate_raw_post_data setting to get at the POST data using a PHP URL wrapper. I'm not sure this is the best solution but it does the job.

That got me running again, but left me with one glaring question: WHY had this happened? If Pair didn't change their PHP config, and if the PHP changelog made no mention of this, why was my script suddenly broken after running correctly for so long?

Fortunately nobody tried to buy my software while the license system was down. But you know you're having a really bad day when you start a sentence with "Fortunately nobody tried to buy my software...", regardless of how you finish the sentence.

Apparently this is a known bug in PHP 5.2.2. And it's not just me who's affected. XML-RPC is broken by this for many people, such as WordPress's implementation (they've arrived at more or less the same solution as me). Drupal's implementation of XML-RPC was already using the php://input approach and is therefore unaffected.

Technically I guess I have to lay this problem on the PHP developers. They seem to have a problem with $HTTP_RAW_POST_DATA, because the changelog indicates that this exact bug was fixed previously in version 5.0.2 and version 5.1. Now it's back for a third round.

But a big part of the reason I'm with Pair in the first place is that I trust them not to stick me with buggy code. They don't always have the latest versions of everything, and that's just fine with me if it means I'm trading currency of releases for stability. I don't need the latest features of everything, I need my site to run reliably. Of course you're unlikely to get a release of something like PHP without there being some known bugs. At the same time, XML-RPC is an enormously popular system, and I would have expected a PHP release that broke it to also have failed Pair's vetting process.

What to do about all this? I've hosted with Pair since 2002. This is the first time I've had any serious trouble with my site since some time in 2004, and even that turned out not to be Pair's fault. And of course they still beat the crap out of places like Dreamhost for reliability. At the same time this is a very disappointing and worrying failure on their part. I'm not looking at switching just yet but I'm a lot less confident in my current setup than I was a week ago.


Site wrap-up

Just a few more notes related to the site redesign process, and then I'll move on to other topics.

Things that went right:

  • As I mentioned before, I went with Drupal again, and I still love it.
  • I used CSSEdit heavily in developing the site theme. My CSS-fu is not quite at the level of the masters, so this app was incredibly valuable in the design process. Editing CSS by hand always ends up as a frustrating trial-and-error scheme for me, but with CSSEdit I was able to make significant changes and updates with (relative) ease.
  • I also used Firebug some, which is another awesome site development tool. I probably would have used it more but I didn't know about it until most of the heavy lifting was done.
  • I installed Drupal (along with MySQL and PHP5) on my Mac, and was therefore able to do all of my development offline. Once ready, I just uploaded the files and transferred the database contents, and the online version just worked. Awesome!
  • Thanks to Pair's domain-mapping system, I was able to upload the new site into a sub-directory without disturbing the old site. Once I verified that the new site worked, I essentially flipped the switch, pointing atomicbird.com to the new directory, and the transition happened automatically. A whole new site and not a moment of downtime, how cool is that?
  • An amazing new book documenting Drupal's workings came out right when I needed it. As a result I was able to customize my site design in ways that would have previously been beyond me. For example, I'm using Lightbox2 for images, and with Drupal's theme system I was able to make images become nice and lightboxey more or less automatically when they're uploaded. Mac developers might think of the theme system as sort of like an informal protocol in Objective-C: Functions are defined which your theme can override if you like, but you're not obligated to implement any of it. In this way you can customize where you want and let the default behavior take over elsewhere.

Things that went wrong:

  • I accidentally broke my Sparkle application auto-updates for a while on transition day. I didn't have Sparkle's server-side code running on my Mac and I didn't realize it wasn't working until after the changeover. This was apparently due to some new security fixes in recent versions of Drupal, which among other things meant that running PHP in the files/ directory is no longer considered a good idea (if it ever was). Moving the Sparkle files to a new location was easy enough, but what about all those people who already have copies of MondoMouse and Chimey that are expecting the old URL? I had to quickly learn a bit of Apache's mod_rewrite to fix up URLs on the fly, but now it works.
  • I'm still getting spammed through the site contact page, something I thought I would be free of by now. I'm attempting to deal with this by means of a math-based CAPTCHA right now. The contact form will ask you to solve a random simple math problem like "what is 4+5?". Get it right and the message is sent, otherwise forget it.
  • My web store at eSellerate is fully customized to look just like this site. However their web store system has some, uh, undocumented behaviors. What bit me here is that if your design includes something like:

    <span id="foo">Hello, world</span>

    It will unexpectedly render as:

    <span>Hello, world</span>

    This really sucks if you have a CSS selector "#foo". I spent a while trying to figure out why my design was broken until checked out the source code. It turns out that id on a span gets stripped out for some reason, although class remains untouched. At the last minute I had to convert a bunch of ids to classes just to make the store work right.

Loving or hating Drupal

As I mentioned in my previous post, this site uses Drupal, an open-source content management system (CMS). Drupal and I have gotten to be old friends. Some of you would like it. Some would hate it. I like it but there are some parties where I know it wouldn't fit in.

First, why use a CMS at all?

When I say CMS, I'm talking about a system which does-- or at least can do-- everything. This site is entirely driven through Drupal. There are no plain HTML files lurking anywhere, it's all done dynamically with LAMP goodness (well, "FAMP", since this is on FreeBSD, but "FAMP" is even less well-known than "LAMP"). So the web site is more like a program running on my server than like something you'd create with, say, iWeb. Why do this?

At first, for me, it was just that I had realized that dealing with static HTML was too much of a pain in the ass to continue. Even using Dreamweaver I found it difficult to keep everything looking consistent, and even from day one I wanted something a little more dynamic. Initially I used Apache server-side includes to manage this, but it was ugly and awkward and I hated it. Plus I wanted to add a blog to the site, and while there are blogging tools that will work with static files (regenerating files as necessary before uploading), I found them lacking in various ways.

I like having a single interface for dealing with all of my web stuff. Some people prefer to do the site design with a tool like Dreamweaver, and then add something like WordPress for a site blog. That's fine, if that's what works for you. I don't want to have to use one tool for my site design and a different one for my blog, if I can avoid it. With Drupal I can, so for example the MondoMouse page was created in more or less the same way as this blog post. Drupal allows full HTML and even PHP in a post, if the site administrator (me) allows it (I do, for me anyway).

This approach also promotes consistency of design across the site. Drupal applies the same theme to every page on this site, so I get a consistent look over all of my pages, with no extra effort.

Using a CMS is also a great idea if you may have multiple people working on the site. It saves the trouble of trying to keep files in sync across all the contributors, since you can all effectively work on the site simultaneously.

Is this the best approach? Well, I like it. Other solutions exist, of course. Those with more Dreamweaver-fu than me might find it more effective. And of course you could just use SVN to sync files among multiple people.

So why Drupal?

Honestly, at the time I was looking for a CMS, it was the only one that seemed to have all the features I wanted. This situation may have changed, but I'm happy enough with Drupal that I'm not looking around much.

One reason for choosing one CMS over another is the implementation language. At least for software geeks like me, sooner or later the temptation to work on the system yourself becomes overwhelming. Drupal works for me in part because it's written in PHP. If you know and like PHP, that's an advantage. Those of you who consider PHP a crawling horror will probably want to look elsewhere.

A related detail is the potential for extending the CMS. I use Sparkle in my applications so that they can auto-update themselves as new versions become available. It looks like it should be pretty easy to add a Sparkle module to Drupal. That way I could update my Sparkle feeds just like I'm writing this message, and Drupal would even generate the RSS feed automatically.

Also you'll want to consider both the core features and any extra add-ons that are already available. Drupal has a dizzying array of add-ons to give you extra cool features for little or no extra effort, and that's in addition to the core features. These help frame the possibilities for your site. I've got a long list of modules I want to take out for a spin and see how they handle.

An important consideration is ease of installation. Some web hosts will help out here. Dreamhost and Media Temple, for example, offer one-click installation of a variety of CMS and blog systems. My own web host only seems to offer Movable Type, not quite a full CMS but well-respected. Don't let this be the deciding factor, though-- you don't want to end up with a system that's not meeting your needs just because it's easy to get it running. Most of these tools have simplified their installation over time (Drupal is a whole lot easier now than it was when I started), so it may not be a problem in the first place. If you intend on having a decent site, you'll end up spending a fair amount of time customizing the look of things anyway. This will easily take far more time than the installation.

If you're still not sure, check out "Is Drupal right for you". Many of the factors described there apply to other CMSs, although not necessarily to blogging-focused systems.


Atomic Bird, LLC