About how swift Apple’s new programming language isn’t.
Update: Hi there! I’m humbled to tell you that, as a few readers have pointed out, the original Swift and Objective-C results in this post were taken from unoptimized builds. We rebuilt the evening of 6/4 with optimizations turned on (-O), and ensured compiler optimization of dead code, ie. assignments being culled because their results aren’t used later, is not a factor (you’d see results orders of magnitude faster, and when we increase the magnitude of our loop count we do see results an order of magnitude slower). We used Swift’s Ints and C’s ints unless otherwise noted. With adjusted numbers in-hand, this post has become something of a living document. Look for new numbers and notes (some stricken!) throughout. -Keith
Swift may be the best thing happening to development on the Mac and iOS right now, with a lot of modern features that’ll help developers new to our platforms and, potentially, make life easier for the older curmudgeons. There’s one thing that’s stuck in my craw since the big reveal yesterday, though: Apple would have you believe Swift is, well, swift. But rewind that keynote video, babe, back to about the 105:00 mark. Stop staring at Craig’s hair. Listen. Really listen. Yep, there’s applause at the initial announcement, but keep listening. Three or four sad claps at the first benchmark comparison and zero at the second.
Why is that?
Because no one in the room bought it?
Being somewhat sensitive to most performance claims myself, I set up a test app in both Swift and Objective-C. Loop a million times, perform a few esoteric bits each time through the loop. Run the code three times, average the elapsed time. Tweak here and there to try to get the best numbers.
Here’s what happened:
Loop a million times
Swift: 0.0036s 0.00061s
Objective-C: 0.0021s (1.7x faster) 0.000021s (29x faster)
No work done in the inner loop, just iterate. Swift actually performs pretty well here. It’s a straight C exercise on the Objective-C side, with a conditional assignment in the inner loop to guarantee the compiler didn’t optimize the loop out entirely. Note that in Swift I used a for loop with an index variable incremented with x = x + 1, because ++ is far slower and for _ in 0…999999 is glacial.
Increment
Swift: 0.024s 0.00092s
Objective-C: 0.0023s (10.4x faster) 0.00002s (46x faster)
Strangely, Swift has a major performance issue with the ++ operator. It’s roughly 6x s l o w e r than x = x + 1, which is the basic code I used to get the best performance. On the Objective-C side, we placed a conditional assignment in the inner loop to guarantee the compiler didn’t optimize the loop out entirely.
Assign
Swift: 0.024s 0.00066s
Objective-C: 0.0022s (10.9x faster) 0.000021s (31.4x faster)
This is a simple x = y.
Yeowch. I’m guessing Automatic Reference Counting is involved on the Swift side. Retaining and releasing a million times would bring on the hurt. On the Objective-C side, we placed a conditional assignment in the inner loop to guarantee the compiler didn’t optimize the loop out entirely.
Append native string to native array
Swift: 6.49s 0.33s
Objective-C: 0.046s (141.1x faster) 0.042 (7.9x faster)
In Swift I used an Array of String. In Objective-C I added an NSString to an NSMutableArray with no optimizations or tweaks. It would be even faster if we dropped to CFMutableArrayRef because in so many cases you don’t need to retain what you add to an array, something NSMutableArray does automatically – and, behind the scenes, Swift is almost surely doing the same because of how Automatic Reference Counting works. Straight C arrays would be blinding. ARC is not a performance optimization.
Append native integer to native array
Swift: 6.51s 0.3s
Objective-C: 0.023s (283x faster) 0.023s (13x faster)
In Swift I used an Array of Int. In Objective-C I added an NSNumber to an NSMutableArray with no optimizations or tweaks. It would be even faster if we dropped to CFMutableArrayRef because in so many cases you don’t need to retain what you add to an array, something NSMutableArray does automatically – and, behind the scenes, Swift is almost surely doing the same because of how Automatic Reference Counting works. Straight C arrays would be blinding. ARC is not a performance optimization.
Concatenate two strings
Swift: 3.47s 3.15s
Objective-C: 0.27s (21x faster) 0.27s (11.7 faster)
In Swift, the inner loop looked like this:
theString3 = theString + theString2
In Objective-C, the inner loop looked like this:
theString3 = [theString stringByAppendingString:theString2];
What’s the deal?
We can’t know exactly what’s going on behind the scenes, but my hunch is some of what we take for granted in Objective-C – the straight C scalar data types – are actually classes in Swift. And the more you rely on classes, the more Automatic Reference Counting is in there somewhere, retaining and releasing like there’s no tomorrow, often for no good reason.
Coders are constantly balancing trade-offs. Raw performance isn’t always the priority, because you have to conceptualize, code, iterate, debug, extend, refactor, share, ship, maintain, and support. One team’s acceptable trade is another team’s hell stew. For example, we don’t use – surprise! – Automatic Reference Counting at Splasm because, frankly, it’s around 40% slower in some cases, so we’ll leave ARC off and take that 40% back (and we enjoy manual memory management, thank you very much). Other teams wouldn’t give up ARC even if paint dried faster. Did I say ‘if’? Different teams. Different values.
Swift has performance issues in our tests that other teams won’t be concerned about. It’s also a very new language that Apple will improve over time. Though Apple claims big performance gains, we think comparing Swift to Objective-C right now is a little premature. We’ll continue to play with it, learning and sharing with the community, but for the foreseeable future, unless the performance issues evaporate – or Apple abandons Objective-C altogether – we’ll be developing in Objective-C. It has its issues, but well-established design patterns, direct control of memory, more readable (and self-documenting) code, and terrific performance for our users are not among them.
Note that the original results were taken from unoptimized builds. When we rebuilt, Swift became much swifter in some cases and slower, relatively, in others. The closest it came to Objective-C’s performance was a factor of 6.4x slower, and that was in a test we didn’t show results for (appending to an NSMutableArray instead of an Int array in Swift). We’re considering porting a small project over to Swift sooner than later to get a better idea of real world numbers…and we’ll share those when we have them!