atomicbird

I probably shouldn't say anything. As a longtime Usenet reader I've (mostly) learned better than to feed the trolls and get involved with online flamewars. But sometimes things get a little out of hand. Anyone can set up their own web site and start blogging these days, and it can sometimes be hard for a person searching Google to know who to believe.

Over the past week, a developer named Rick Downes has gone on a tear against a number of other Mac developers, posting a series of angry, insulting, and simply untrue statements about them. In most cases he uses technical arguments as the basis for the post, but rather than be content with this he instead layers on hateful, mean-spirited personal attacks on his targets. It's his blog, but I'd hate to think that an unsuspecting person might come across some of his statements and wonder if maybe they're true.

Worst has got to be his over-the-top attack on Daniel Jalkut. I know Daniel and respect him a lot, both as a person and as a developer. You don't have to like Daniel, though I don't know why you wouldn't, but not liking someone doesn't excuse outright lies that could damage a person's reputation. Rick repeatedly states that Daniel doesn't care about his customers, and makes it out that Daniel is somehow getting away with ripping people off. How you do that when your software has a downloadable free trial escapes me, but that's not really the point.

The point is: I've met a lot of developers in my career, and Daniel Jalkut is easily the most concerned for his customers of any developer I've ever met. He sets very high standards and lives up to them. If anyone has any issue with his software, he wants to know and wants to fix things as quickly as possible. Even now Daniel's probably considering whether there's anything valid in Rick's diatribe and making whatever fixes might be needed. Rick's statements regarding Daniel are totally at odds with reality.

It's hard to even know what motivates Rick. If I-- or, I believe, Daniel-- discovered what I thought was an embarrassing flaw in someone's application, my first inclination would be to contact that developer and let them know. Going public with a blog post is completely unwarranted unless the flaw in question somehow endangered people, and only then if the developer additionally failed to warn those people. Loading on personal insults is unwarranted, full stop. Criticism of another's work may be valid, but it's best to make such criticism as constructive and helpful as possible. Publicly lambasting another's work like this as the first step helps nobody.

Rick's attack on Jonathan Wight is similarly an over-the-top baseless rant. Among those who know Jonathan, he's earned a reputation as a technical whiz who always has time to help someone out. His technical chops are on public display at his public code repository. He runs the Iron Coder contest. Rick seems to hate him, apparently because Jonathan teaches kids about robots and doesn't have many applications listed at versiontracker.com (never mind his consulting work, I guess). And of course because he disagrees with Rick on a certain technical issue.

As for Rick's comments on Kenneth Ballenegger, well, I'm not even sure I understand what he's saying. The constant use of the word tool makes it apparent that Rick's not happy, but about what I'm not sure.

The community of Mac developers is, with very few exceptions, an enthusiastic group doing their best for those using their software and welcoming to anyone who shares their interest. There's a real sense of professional courtesy as well, with developers mostly willing to lend a hand to others and provide constructive criticism to each other when appropriate. In most cases if one Mac developer mentions flaws in another's application, they'll do so privately an in the spirit of helping a colleague do a better job. Please don't be misled by the few exceptions to this rule.

Yesterday I was working on a forthcoming update to Chimey and I noticed something odd. Chimey of course makes use of SparklePlus for automatic updates, and after a test run of my build-for-release script, Sparkle was looking a little odd.

Normally a framework has one or more binaries, with a symbolic link pointing to the current version. From the command line you'd expect something like this:

$ find Sparkle.framework/ -name Sparkle -exec ls -l {} \;
lrwxr-xr-x  1 tph  wheel  24 Nov 21 11:20 Sparkle.framework/
    /Sparkle -> Versions/Current/Sparkle
-rwxr-xr-x  1 tph  wheel  233088 Nov 21 11:20 Sparkle.framework/
    /Versions/A/Sparkle

Instead I was seeing this:

find Sparkle.framework/ -name Sparkle -exec ls -l {} \;
-rwxr-xr-x  1 tph  wheel  242928 Nov 21 11:20 Sparkle.framework/
    /Sparkle
-rwxr-xr-x  1 tph  wheel  233088 Nov 21 11:20 Sparkle.framework/
    /Versions/A/Sparkle

Yow, how the hell did that happen? I thought it might have something to do with copying the framework, either when compiling or when building the disk image, but that wouldn't account for the different file sizes. A quick check on the current version of Chimey showed that this was something new, not something I'd been doing all along without realizing it.

So what was different? My build-for-release script now signs my code using Leopard code signing.

Apple's documentation on code signing indicates that "You should sign every program in your product, including applications, tools, hidden helper tools, utilities and so forth." Chimey's main bundle includes a preference pane and two helper tools, so I was making sure to sign all of them. The main documentation doesn't mention frameworks specifically, but the Code Signing Release notes indicate that "You may also sign any libraries, frameworks, plugins, and scripts you ship, whether they are delivered with an application or separately." Framework signatures don't get checked yet but might be in the future. I had made my script as future-proof as possible by signing the Sparkle framework now.

But if you sign a framework, the codesign tool modifies the structure of the framework, as I found with Sparkle. Just for sanity's sake I made sure that this affects any Mac OS X framework and is not some kind of Sparkle-specific behavior.

And the file sizes? I can guess what's going on, but I made sure:

$ codesign -vvv Sparkle.framework/Sparkle 
Sparkle.framework/Sparkle: valid on disk
$ codesign -vvv Sparkle.framework/Versions/A/Sparkle
Sparkle.framework/Versions/A/Sparkle: code object is not signed

Not only is codesign changing the framework structure, it's also leaving the framework in an inconsistent state with regard to whether it's signed. Instead of one binary I've got two, and even though I signed the framework, one of those two is still unsigned.

At the same time, checking on the framework bundle still returns a valid signature, despite the presence of unsigned code in there:

$ codesign -vvv Sparkle.framework
Sparkle.framework: valid on disk

It's a good thing that Leopard doesn't currently check on framework signatures. For now it seems it's probably best not to bother signing a framework. Although codesign leaves you with something that should work, it's not clear that it's actually doing anything useful, and it's bloating the framework size in the process.

This has been filed as bug #5609522 with Apple, in case anyone from Apple reads this.

One of the new features in Leopard that people may not notice at first is code signing. It's not the sort of thing that makes for a flashy demo, so you may not have noticed.

When a software developer creates an application they can now optionally add a cryptographically-secure signature to it which can later be used to find out if the application was modified. It can also be used to identify a new version of an application in relation to an older version of the same application. If both are signed in the same way, Mac OS X won't notify you about the update and ask if it's OK for the new version to use the same keychain information as the old one.

Beginning with Mac OS X 10.5 Apple signs all of their applications.

What does modified mean in this situation? It's not as clear as one might think, but the basics are:

• Modifying the executable, the part that actually runs the program, will invalidate the signature.
• Removing platform-specific code (just the Intel or PowerPC part) can be an exception to this (more detail below).
• Removing language-specific localizations-- such as the parts needed to present menus and windows in a different language-- can be safely removed without invalidating the signature.

Macaroni and Code Signing

Which brings me to my main point. The authors of 1Password have run into some trouble with applications whose signatures get invalidated, in their case Safari's signature. As they describe in their Switcher's Blog, some of their customers found that 1Password no longer worked with Safari, and some detective work revealed that the problem was an invalid code signature.

Unfortunately while doing this they made the mistake of including Macaroni in the list of applications that could cause code signatures to become invalid. Almost immediately I started getting email from concerned users thinking that they needed to shut down Macaroni to protect their Macs. In fact Macaroni does not and has never removed platform-specific code from applications.

Many users have requested this feature since Intel Macs first appeared, and I did investigate it, but it seemed to be a technical and support mine field and so I left it out. It's not necessarily a bad idea to remove PowerPC code if you have an Intel Mac, or vice versa, but it seems like doing so is more of an expert move, to be done if you feel you have a good technical understanding of what you're doing and why. And if you do, you probably don't need Macaroni to help you with it. You could create a custom Macaroni job to automate the cleanup for you, but I still maintain that it has no place in the standard set of Macaroni jobs.

And now, a demo.

For demonstration purposes I'm going to look at Safari and Mail, both included with Leopard, and see what does and doesn't invalidate their signatures. I will of course be working on copies of the applications, because I'm going to be messing with their internals for your entertainment.

Demo: Localization cleanup

First I'll select Safari and open up the "Get Info" window. As you can see all of the languages Apple ships with it are present. There are 18 in all, though they don't all show up without some scrolling. Mail produces the same result.

I'll also verify that I'm starting with valid code signatures on both:

$ codesign -vvv Safari.app
Safari.app: valid on disk
$ codesign -vvv Mail.app
Mail.app: valid on disk

Now I'll run Macaroni and have it remove localized files. After doing this, I bring up the "Get Info" window again and verify that everything except English has been removed:

And how about those code signatures?

$ codesign -vvv Safari.app
Safari.app: valid on disk
$ codesign -vvv Mail.app
Mail.app: valid on disk

Looking good.

Demo: Platform code cleanup

Now I'm going to strip out the PowerPC code, leaving me with copies of the applications that work only on Intel Macs. I don't have any of the fancy utilities that does this for you, so instead I'm going with the command line and Apple's standard tools. There's more than one way to do this, but the easiest is probably to use "ditto". Besides copying files, ditto will optionally strip out platform-specific code while copying.

$ ditto --rsrc --arch i386 Safari.app Safari-i386.app
$ ditto --rsrc --arch i386 Mail.app Mail-i386.app

And the signatures?

$ codesign -vvv Mail-i386.app/
Mail-i386.app/: valid on disk
$ codesign -vvv Safari-i386.app/
Safari-i386.app/: a sealed resource is missing or invalid
/tmp/Safari-i386.app/Contents/Resources/SafariSyncClient.app/
Contents/MacOS/SafariSyncClient: resource modified

Hmm, not so good for Safari there. Mail looks OK though.

Should you care?

If you're using both 1Password and you strip platform-specific code from your applications, you should be aware of this. As the 1Password developers have discovered, an invalid code signature can prevent their tool from working. This is exactly the kind of thing that stopped me from adding this kind of application stripping to Macaroni.

If you don't use 1Password, it's probably OK to strip platform-specific code if you want. Invalid code signatures might mean you'll have to enter your password more often after updating applications, but aside from that it doesn't look like you'll notice it. It may not be worth the effort though. Leopard's Mail.app starts out at 286.5MB. After removing localizations that drops by more than 90%, to 24.7MB. Removing PowerPC code on an Intel Mac reduces it further to 21.6MB, only another 1% or so off of the initial size.

Either way though, Macaroni's not going to hurt your code signatures.

Last week I posted my first public SparklePlus data. Starting today, SparklePlus charts can be found on my Sparkle Stats page, updated nightly.

I've improved the charting system over the past couple of days to provide more detail than the charts in my previous blog post. Instead of just gathering up each week's data into a single value, the new plots include daily totals for the previous 60 days. Each day's numbers are a running average of the previous seven days, to smooth out daily variations into more comprehensible trends. I'm not sure that a seven-day running total is the best window, so I may experiment with longer or shorter windows and see what happens.

As of today Leopard has reached 45.6% of users, not quite a majority yet but closing in fast.

Update: SparklePlus charts can now be found on my Sparkle Stats page, updated nightly.

Last year when I was getting MondoMouse ready for release, one of the things I really wanted was an automatic-update system, so that the software could locate and optionally install new versions of itself. Though I try to make as much noise as possible over new releases, people often don't know about them. And even if they do, downloading and installing by hand can be a pain if there are a lot of updates.

Enter Sparkle, a Cocoa framework designed to do exactly that with a minimum of effort.

But that wasn't all I wanted to add. One chronic problem for developers is figuring out what versions of the operating system needed continued support, and which could be dropped. If I don't know if any of my users are still on, say, 10.3, how do I know if I need to continue supporting it? I don't want to waste time with it if nobody's using it, but at the same time I don't want to accidentally strand a huge number of users because my gut feeling turned out to be wrong.

Basically I needed something like the OmniGroup's statistics page. That would mean having MondoMouse-- with the user's permission, of course-- periodically send my server some anonymous information about their Mac, including what version of Mac OS X they're using. Since Sparkle would already be contacting my server on a regular schedule to check for updates, it seemed the ideal solution was to send this information along at the same time.

So I modified Sparkle and created SparklePlus. It did the same thing as Sparkle but with the addition of this anonymous information about people's Macs.

In the spirit of being as open as possible about the process, I always intended to make this information publicly available. But getting that set up was harder than it seemed and became easy to put off.

Until today. Here's some of what I've been collecting.

MondoMouse users are adopting Leopard at a surprisingly fast pace. Just a couple of weeks after release, 45% of MondoMouse users have upgraded:

The Intel/PowerPC ratio has been surprisingly stable though:

Same goes for the number of CPU cores on people's Macs, though it's interesting that nearly 80% have at least two:

There are a number of other details collected, but I haven't been able to develop useful plots of them all yet. For example SparklePlus reports how much RAM is installed, but there are too many different values to show them all and so far my plot-generation script can't coalesce values into ranges.

My plan is to set up a special page for this information, and have the charts updated on a weekly basis. That should be coming soon.