February 4, 2009
tagging a release with Versions
OK, everybody, go read this post by my friend and colleague Mike Abdullah.
Spending too many years with CVS than I care to count (and SCCS for a while at Sun, oy), I tend to go a little catatonic whenever Mike asks if I would mind tagging the latest release of Sandvox.
"Sure, no problem." and then a few minutes later I sheepishly ask "Um, how do you do that with Subversion again?"
It always astounds me how simple it actually is... Since we started using Versions it's about as easy as you could hope.
But, of course, I can never remember how to do it and then when Mike tells me again I'm always incredulous. So I asked Mike to write that post for me. (It even has pictures.)
Thanks Mike!
Posted by Terrence Talbot at 3:34 PM
May 17, 2007
Look, Ma. No framework!
When I first sat down to set up the project files for Sandvox, I knew that we wanted to support plugins. These are little bundles of Cocoa code and related resources that can be loaded into an application at runtime to extend its funcationality. A lot of Cocoa apps work this way. Watson did. So do Mail, iPhoto, iDVD, even Calculator. If an app keeps its plugins in the standard location (Contents/PlugIns) you can even browse them and activate/deactivate, add or remove them via Finder.
If, in Finder, you do Get Info on Sandvox you'll see the list of plugins that are included in the .app.
Though you can activate and deactive them using Finder, I don't recommend doing it, for any app. Apple seems to have implemented half of a good idea here, but with no documentation or expectation as to how it should be used. Activating/deactivating plugins this way may have unfortunate results. So don't do it. But it's still an interesting way to browse and see if an app supports plugins, at least in the standard way.
Nevertheless, what's important is that plugins allow you to modularize and build out an application's functionality over time, including after the bulk of your app has already shipped. Not only can you extend your application with new plugins, but so can third parties. That's a big bonus and well worth supporting in your application if it makes sense.
The typical way that you encourage plugin development for your app is to move the general code required to support a plugin to its own framework that you then bundle inside your .app. Another developer comes along, learns (via SDK documentation, if they're lucky) that your app supports plugins, and that the way to do that is by linking to one or more frameworks within your app's main bundle.
Sandvox works this way. In fact, Sandvox supports six different types of plugins: Pages, Pagelets, Elements, DataSources, Indexes, and Designs. In all shipping versions of Sandvox up to the current, 1.2, we've made available a framework called KTComponents that you need to link your plugin against so it knows how to load itself into Sandvox, talk to Core Data, etc.
Wanting to be as open with our code as is reasonably possible, we kept moving more and more functionality from Sandvox over to KTComponents so that plugins could use this code, too. This included things like HTML parsing, UI classes, value transformers, and a bunch of different categories extending not only Foundation and AppKit, but Core Data, Core Image, and WebKit. Over a hundred classes or categories in all. A lot of code.
And, frankly, it's been a big pain maintaining it. Having all this code in a separate framework slows down the build. Makes the build target more complicated. Makes it harder to figure out where to set breakpoints. Makes it harder to locate resources. Makes maintaining localized strings more work that it should be. And did I mention that it really slows down the build?
Well, no more. When the next major update to Sandvox ships, KTComponents will be no more. All the code is going into the app. And plugins will work just fine. In fact, getting new plugins to build and link will be easier than I ever thought possible.
How can this be?
About a year and a half ago Wil Shipley posted a piece called Frameworks are Teh Suck, Err wherein he makes many fine points about where framworks came from, what they're good for, what their limitations are, and why you shouldn't use them.
At the time I thought "Well, that's nice, Wil. But we have to support plugins and we need a framework for that." Dan had posted in the comments to Wil's piece asking about this, but no one offered up a solution. I didn't think much about it at the time; just resigned myself to the annoying code split and build headaches.
About a year later, Wincent Colaiuta posted some thoughts about Cocoa development and the things he wished he'd done a little differently, had he only known, including where and when to use frameworks. Included in his post is a little piece of linker trivia that I had never heard of before. A flag for ld called bundle_loader.
Wincent didn't include any links to any information about bundle_loader and it only appears once in Apple's documentation, under Dead Code Stripping. I didn't think much about it at the time: just filed it away until I had some time to explore. This week, I explored...
Here's what 'man ld' has to say about bundle_loader:
-bundle_loader executable (32-bit only)
This specifies the executable that will be loading the bundle
output file being linked. Undefined symbols from the bundle are
checked against the specified executable like it was one of the
dynamic libraries the bundle was linked with.
I had no idea. This is basically how you get a plugin to link against an app without the linker going belly up. It works and it works well. (The 32-bit only comment is a slight concern. I don't think it will be an issue for the foreseeable future for Sandvox. I intend to confirm this at WWDC.)
If you've made it this far, rather than bore you with all the things I tried to make the linker happy before I stumbled onto bundle_loader, let me just share with you how we're going to be building plugins from here on out.
Rather that having to know about and locate KTComponents in the Sandvox appwrapper, somehow adding that to your plugin project, getting your framework search paths right, worrying about framework version numbers, if you're so inclined, and all that other junk, you now just add these two settings to your plugin's target's build configurations.
USER_HEADER_SEARCH_PATHS = /path/to/Sandvox.app/Contents/Headers
BUNDLE_LOADER = /path/to/Sandvox.app/Contents/MacOS/Sandvox
That's it! Really, that's it. That's all you have to do. Your plugin doesn't have to know anything about Sandvox except those two settings. Unbelievable. What was I doing messing around with frameworks for all these years?
You can put these in the Build tab of your target's inspector or you can put them into xcconfig files.
Suddenly, now, life just got a whole lot simpler. Plugins are trivial to get up and running. And we now have all the Sandvox code under one roof. (ConnectionKit, iMediaBrowser, and WebKit notwithstanding.) Suddently refactoring Sandvox becomes doable, since all the code is in the same project now. Very happy news. And, oh yeah, did I mention builds are a lot faster?
One last word of advice: to make this work, you do have to pay attention to what symbols you're exporting. Typically when you make an install build, the executable is stripped of many of its symbols: symbols that plugins are going to need to link against. Basically you can either leave your app unstripped (though still strip debugging symbols) or pass flags to strip that are more akin to those used with frameworks than with apps. E.g.,
STRIPFLAGS="-S"
And if you have things marked "extern", like string constants and functions that you want accessible across files, you need to tell gcc not to make those symbols private or plugins won't link for that reason as well. E.g.
GCC_SYMBOLS_PRIVATE_EXTERN=NO
So that's the big secret. No more splitting projects into app+framwork(s). I wish I had known about this years ago.
bundle_loader is the way to go.
Posted by Terrence Talbot at 3:06 PM | Comments (2)
May 16, 2007
Fail Faster
The default Cocoa Application project in Xcode includes a target with a set of build phases. Here's what it looks like:
Notice that Copy Bundle Resources happens before Compile Sources and Link Binary With Libraries. This means that every time you build, before the compiler or the linker see anything, Xcode spends time copying things like nib files and images into the built product.
If you have a large app, this list of resources to copy can be quite long. Everytime you make a source change and hit command-Y to compile and relaunch your app, it has to run through Copy Bundle Resources again, and then it tries to compile. That's fine. It needs to do that. What's the big deal?
The big deal is that if you're writing a bunch of new code or manually refactoring some older code and you have syntax errors, say, a missing semicolon, a typo in a class name, or a missing #import, the compiler is going to complain and halt your build. If you forget to add a reference to a framework or a library, the linker will complain and halt your build. But it's going to halt your build after copying bundle resources, things that the compiler and linker pretty much don't care about.
In Sandvox's case, there are nearly 200 different resources to copy during a build, and that's not even counting the plugins and frameworks that ship with the app. Doing this over and over again during the course of a day adds up. That's time we can reclaim.
Whenever I make a new project now, I open up the target(s) and move Copy Bundle Resources below Link Binary With Libraries.
This makes sure that the first thing Xcode does is build my code. If it builds, then it can go on and flesh out the rest of the .app. If it's not going to build, I want to know right away so I can spot the problem, fix it, and get back to work.
So I say, move that build phase!, and save yourself some time.
Posted by Terrence Talbot at 4:25 PM | Comments (2) | TrackBack (1)
January 23, 2007
Sundown
My friend, and former Lighthouse/Sun colleague, Paul Kim writes about some ancient history that more or less stopped Java from ever becoming a real application development platform. Please read his entry before you read mine. (It’s new information that actually isn’t in Wikipedia.) I agree with Paul that it did not have to turn out that way.
Paul invited me to tell the tale of the demise of OpenStep within Sun. I actually think I’m not going to attempt that. There are others in the world who invested far more of their lives in getting OpenStep up and running on Solaris. The fourth paragraph of the Wikipedia entry seems reasonably accurate as to what all was involved. It was no mean feat: the fact that OpenStep ran at all on Solaris, in X Windows no less, let alone as well as it did, still amazes me to this day. That work was done before I even got to Sun and my hat is still off to that team. Those guys knew X and CORBA and the bowels of Solaris like I hope I never have to know X or CORBA or the bowels of Solaris.
Nevertheless, no matter how good OpenStep could have gotten at Sun, it was not going to stop the Java juggernaut. It did stop CDE, though, at least for a little while, and for that we should all be grateful.
But in ‘96 Java was everything to Sun. Limitless potential—since hardly anything had been developed—and everyone wanted in on it.
One thing I was supposed to do was to figure out where to take things for OpenStep Solaris 2.0. Not just me, of course, there were certainly others, but I had at least one idea that I pitched, I think, at the VP level and it relates to what Paul’s talking about. (Of course, the idea went nowhere fast.)
After OpenStep 1.0 was out the door, Sun and NeXT diverged significantly. Well, they probably diverged long before that, but Sun’s problem was that the OpenStep that NeXT had licensed them lacked the much more sophisticated Project Builder and other development tools that NeXT’s version of OPENSTEP would include. I don’t have much inside knowledge on that but I’m pretty sure NeXT wasn’t going to share that code with Sun, if at all, without a whopping big check from Scott McNealy.
You pretty much only had to have a pulse to realize that that was just never going to happen. OpenStep was going to be forked, not just under the hood, but all over the hood, and no one wanted that. Even if Java had never existed, sending OpenStep in divergent directions would have eventually led to its demise, at least at Sun.
(Also keep in mind that although the OpenStep Specification had been out for a couple of years, many of the client apps—I’m guessing both at NeXT and at Sun/Lighthouse—were pre-Foundation Kit apps and still needed to be ported. (Think NX* classes instead of NS*: no easy to use arrays and dictionaries, no autorelease pools, etc., only a lot worse.))
But, it wasn’t going to matter anyway. Remember that Java juggernaut? Everything was going to move to Java. No. Matter. What. And, frankly, at the time that wasn’t necessarily a bad thing. Limitless potential, remember?
So here we had an OpenStep-aware group at Lighthouse who needed to write applications in Java. And an OpenStep-aware group at SunSoft that needed to figure out what it was going to do with itself, and with Java. And, oh yeah, SunSoft, as a product line, had just really crappy Java development tools. (Sorry, Jeff!) Hmm... I think you can see where I’m going.
(The IDE available from Sun, and I think actually mandated for internal use at Sun, was Java Workshop 1.0. I wish I could find a screenshot. Everyone I knew at the time just used Emacs.)
So, ok, one of the things I suggested was that the OpenStep group start building real Java development tools, first in Objective-C, then ultimately in Java. Right now. Anything would have helped. Lighthouse would have loved it. The world would have loved it. And OpenStep Solaris would have had a little more life.
Ah, but we needed an Interface Builder for Java and we didn’t have a lot of time. Or so I thought.
Enter NetCode.
NetCode was the developer of the Internet Foundation Classes (IFC). It was started by two guys from NeXT. Those two guys now ship two (different?) Cocoa applications today. They got it. I wanted it. Actually, I wanted what came with it: Constructor.
Constructor, to this day, is still the most IB-like 100% Java-based interface builder I’ve ever seen. For 1996, it was f’ing amazing. (I wish I had a screenshot; I don't. UPDATE: 24 hours later, I do. See below.) It was IB, in Java. WYSIWYG drag-and-drop layout. Palettes. Dragging connections between objects. Actual. Freeze. Dried. Objects. (None of this “here, let me generate for you a few thousand lines of incomprehensible, but precisely formated—don't mess with it, not so much as a comma!—AWT layout manager code so you can draw a few buttons” nonsense.)
Imagine this: IFC looked a helluvalot like AppKit, in Java. No AWT. And it came with a friggin’ Interface Builder!
IFC, I thought, had had a stunning debut at a tradeshow in 1995. Netscape promptly bought them in 1996.
I had a meeting with Jayson at Netscape in late 96 about Constructor. It didn’t go anywhere. And neither did Constructor. Ah, what could have been! I never understood why Constructor, or at least the idea of Constructor, didn’t come over to Sun when IFC got folded into JFC. Certainly neither Netscape nor Sun ever did anything with it.
And now we join Paul’s story. The one thing Sun did understand was that AWT wasn’t going to cut it. Not by a long shot. They were looking straight down the double barrel of the AFC shotgun that Microsoft was pointing at them. I say double barrel because AFC not only could have replaced AWT but also tied future Java GUI development to Windows, as only Microsoft can do. Embrace and extend. Yikes.
So now JavaSoft has AWT, Netscape has IFC, and Microsoft has AFC. And AFC is really f’ing scary for JavaSoft. Losing control to Netscape would be bad enough. Losing it to Microsoft likely would have been fatal. But what could Sun really do?
Ah, but wait, Lighthouse—by then a bloody Sun company!—had been somewhat secretly developing the Lighthouse Foundation Classes (LFC). Very similar to the IFC, the LFC dropped AWT on the floor and was pretty much on its way to being a decent knockoff of AppKit, all written in Java.
So what happened? Some people convinced some other people to stop development of IFC (!) and LFC (!!) and fold parts of those teams into a new team based at JavaSoft—headed by the AWT guys (!!!)—where they/we would write the new Java Foundation Classes (JFC) that would save the day and ward off the dreaded AFC. Which, when you come right down to it, it actually did accomplish.
But, as Paul points out, a heavy price was paid. LFC and IFC (apparently including Constructor) were abandoned in order to stop Microsoft. That’s all there is to it. And that’s a decision I can understand.
But Paul is also right that Sun’s own internal battles and basically just NIH by JavaSoft itself hobbled the development of Java as a first-class application development environment. JFC should have simply taken LFC or IFC and continued to develop it. SunSoft should have abandoned Java Workshop far earlier than it did.1 Lighthouse should have been allowed to port its office suite (and then some) to (a smartly done) JFC.2 There were people at Sun that knew what to do. They were ready to do it. They wanted to do it. It just didn’t happen.
Yes, Paul, what might have been. What might have been.
Oh well, back to Cocoa... I think that part, at least, turned out pretty well.
UPDATE: It turns out that Constructor lives! Sort of... Following the instructions posted by Tim in the comments to Paul's post, I actually brought up Constructor in Safari! Pretty rippin' cool that you can still download the code (though sadly not the source). If do you try it out, my advice is to edit the provided index.html to make the applet size a lot bigger than the default 640x480. Here what I've been playing with at 1024x768:
1 A few years later, after I and others had left Sun, Sun abandoned Java Workshop and paid a lot of money for NetBeans.
2 A few years later, after I and others had left Sun, Sun woke up and realized they needed an office suite after all and paid a lot of money for StarOffice. All in glorious C++. Yikes * 2.
Posted by Terrence Talbot at 9:51 PM
September 25, 2006
Thanks Mickey
I originally wrote this at the very end of last year, but apparently never published it. So, old topic. But I liked the post anyway, so here it is...
No, this isn't about Disney. It's about Michael Sprague, my high school computer science teacher. I no longer see him on the school's website. He'd only be about 50 now. A little young to retire. But maybe not. I hope he found something better to do than teach teenagers about computers, but he was pretty great at that. So actually I hope he's still doing it.
What brought "Mr. Sprague" to mind was Joel's latest [ed. not so latest anymore] essay, The Perils of JavaSchools, and Dori's followup, You think you're old? recounting her computer science education back in the day, as the kids say.
Dori nicely summarizes Joel's argument: "[In] the old days, programmers went to college, learned C, and got a good understanding of what was going on deep down internally. Nowadays, these newfangled programmers are only learning Java, and consequently, they don't know anything about recursion or pointers. Schools should teach Scheme and C, and the students will be the better for it."
And that got me thinking. Pointers and recursion. Yeah, he's right. Sprague had us doing those in PDP-11/70 assembler and Pascal (for the AP test the following year) and some pseudo-language of his own invention when I was a junior in high school.
I mean, like, for months it was dereference this, parse-this-expression-into-a-tree that. And after a while our heads did explode. But we felt better for it. And, you know what?, the following year the small group of us that could put our heads back together again won the ACSL state championship really without even worrying about it. (Ok, so we placed, I think, 9th nationally but that had more to do with our hastily arranged trip to Amish country than anything else.)
This was a good thing, too. When I went to college, a really, really long time ago, at least in Internet time, my alma mater didn't even offer a formal computer science undergraduate education. (It was considered "too practical". Apparently that's now changed.) So I went to law school...
Now I don't bring this up to say "see, I was doing this in high school, look at me", I bring this up to say a sincere thank you to whomever at LHS decided they could start a computer science curriculum around an 8 year old 16-bit minicomputer and to give a tremendously big shout out to Mickey. You guys saved my butt.
Posted by Terrence Talbot at 10:17 PM
March 13, 2006
Pom Pom Pom Pom
Last week Dan and I had the pleasure of being interviewed by Pom Pom Pom Pom, an international podcast about all things Apple. We chatted for about an hour on Karelia, Sandvox, Apple, and the competition. It was a very enjoyable experience.
Many thanks to StuFF mc for the opportunity!
You can either listen to Episode 5 directly or subscribe to the Pomcast in iTunes.
Posted by Terrence Talbot at 1:49 PM
March 9, 2006
CocoaHeads LA Meeting Tonight
Just a reminder that the third meeting of CocoaHeads LA (Santa Monica) will be tonight (Thursday) night at 7:00 PM.
We'll again be hosted by Steve Gehrman at the fabulous CocoaTech offices at 1238 7th Street, Santa Monica, between Wilshire and Arizona. We may move the venue later in the evening to one of the local spots for food, drinks, or dessert.
We're still in our nascent stages, so now's the perfect time to join the group. (It's free).
We'll be having our general Cocoa roundtable, show-and-tell, and discussion about Cocoa debugging techniques, including a demo of OmniObjectMeter.
CocoaHeads LA. See you there!
Posted by Terrence Talbot at 10:52 AM