The JIT Life

Shutting it Down

2010-10-18T18:28:00.000-07:00

I'm going to be shutting this blog down and moving everything over to http://michael-rollins.blogspot.com. Head on over there for the latest.

Drama in the Jailbreak Land: SHAtter Delayed

2010-10-11T11:50:00.000-07:00

Egos are a fact of life. They run rampant in almost everything, though it seems that the bigger the limelight, the more likely it is that an ego will jump to the front. This is surely the case in the hacking world, where egos are attached to handles that don't necessarily bear any resemblance to reality. Such is the case with the latest iOS 4.1 jailbreak.

Chronic Dev Team, a group of hackers known for developing some of the most capable jailbreaks for the iPhone, have been touting a major bootrom exploit for the latest generations of iOS devices called SHAtter. With SHAtter, which relies on an unpatchable vulnerability in the bootrom of all current generation iOS devices, jailbreaking would be guaranteed for the life of the device (source: QuickPWN). The exploit is at such a low level that it would be all but impossible for Apple to patch it with software.

The exploit was announced several days after iOS 4.1 went live with a message that the jailbreak was verified and would make its way to the wires soon. The raw jailbreak was announced with this twitpic photo on July 15, 2010. Note the version of iOS which states 4.1 along with "rooted". Eventually, this jailbreak was slated to be rolled into a package called "greenpois0n", which would be released on 10.10.10 at 10:10 am (the Geek holiday).

Drama and egos, though, are apt to ensue with something this highly touted. On October 9, 2010, one day before greenpois0n was to be released, another member of the iOS hacking community released a different bootrom exploit, rolling it into a package called limera1n. That hacker, George Francis Hotz (born October 2, 1989), aka geohot, had previously released a tethered jailbreak known as blackra1n. This release of limera1n was apparently against the express wishes of both the Chronic Dev Team and the iPhone Dev Team. In a tweet that would come out shortly after the limera1n release, some insight into the drama that must have been running amok is revealed. The tweet states:

@chronicdevteamchronic dev team
use limera1n at your own risk. it has been untested and geohot screwed over @comex who trusted him to wait.

It is worth noting that limera1n is in beta form and does not work on older devices. Further, it is rumored to not work on newer bootrom iPhone 3GS devices. Finally, it is currently only Windows based.

In the ensuing fallout, the Chronic Dev Team postponed greenpois0n, pulling the SHAtter exploit from it completely and choosing to implement the bootrom exploit which geohot used.

At issue here is that each bootrom exploit is precious. Apple can patch and remove these exploits as they iterate hardware. To release two bootrom exploits at the same time would be a waste, in that Apple could roll up both of them in the next iteration. By doing the "mature" thing, the Chronic Dev Team is at least giving potential life to SHAtter on another device, and it may serve as the means of a lifetime jailbreak on the next iteration of iOS devices.

Given the beta nature of limera1n and the timing of its release, it certainly seems as if it was released as a means of grabbing the limelight. It was also apparently an unpopular thing to do in the small community which is known for turning out solid jailbreaks. The fact that it does not work on older devices and is rumored to not work on the newer iPhone 3GS devices seems to indicate that it is an inferior jailbreak, as well, which is a true shame: It is unlikely that the SHAtter exploit will see the light of day anytime soon.

Greenpois0n will be released soon, according to the Chronic Dev Team. It may be wise to wait for their fully tested version to be made available before jumping into the hairy landscape of jailbreaking. If you can't wait, however, limera1n is available and waiting.

Sources: Chronic Dev Team twitter, geohot.us, quickpwn.com

Assimilation: A Review of iPhone 4

2010-09-16T21:28:00.000-07:00

On my birthday, September 9, I walked into an Apple store and asked to be assimilated.

It was a truly magical event. As I walked down the long hall of aluminum, surrounded by cheering Geniuses in their blue shirts, my heart beat furiously. Soon, oh so soon, I would have within my possession the grail of all phones, the beautiful and sleek iPhone 4, that phone which makes all other phones seem as pale imitations of phonedom. I approached the counter, stated my case, laid out my credentials, was judged worthy (after a slight Apple network hiccup) and proceeded out the door carrying the fabled device. The universe was right and good, a thing of majesty and beauty.

If you've never been in an Apple store, you may scoff at this account, but I tell you, this is exactly what Apple is trying to foster in their stores. A land of sleek industrial design which seeks to pull you, atom by atom, into a world where the coveted fruit of their hard work is your only goal, idolized and adored by you to the exclusion of all else. They go a long way to accomplishing that goal. Hold fast your soul should you ever tread these hallowed halls!

It's been a week since I purchased said device, so I'd like to give you my impressions.

Syncing and Upgrading

I brought my iPhone 4 to work, hooked it up to my computer and synced everything that had been on my tired and worn out iPhone 3G. Apple has really nailed it, here. It was a painless process. In less than half an hour, I was holding a device that seemed identical in OS to my iPhone 3G (besides hardware differences, mind). It was faster, sleeker, but every single piece of data had transferred. I was almost disappointed that I wouldn't have to crawl through the depths of the device to set anything up.

Design

It is truly a sleek device. When I got my iPhone 3G, i thought I had never seen a more beautiful piece of hardware, but the 4 makes the 3G/3GS design seem cheap and plasticy, utterly devoid of inspiration. This glass and aluminum idol is a thing of utter beauty to behold, I have to say. All other phones come up short, so far as I can tell, in comparison.

That's not to say that it's without flaws. Being all glass on front and back, it makes it hard to hold. Often when I'm talking, I feel the phone start sliding around in my hand. I keep having to rearrange my grip to keep it from sliding out of my fingers.

I'm also extremely nervous about it, in general. I know that the glass is hardened, but I can't help but be concerned that I'm going to shatter it. In fact, when I went into the Apple store, the geniuses at the bar were mulling over an iPhone 4 with a shattered back. These things can be broken, and I'm much more cautious about using this phone than I was about my 3G. When I used to place my 3G in my pocket, I would always put the glass against my leg, hoping to protect it to some degree. No matter how I put my iPhone 4 in my pocket, some glass is going to be exposed to the outside of my pocket.

I've also noticed some slight hairline scratches on the back of the device already. As I've stated, I've been extremely careful with it, so that just goes to drive home the point even more: despite hardened glass, it should be handled delicately.

I've already put my order in for my free case, and I can't wait for it to get here. I'll feel a bit better about things when I finally have this thing at least moderately protected. Such a shame, though, to cover up a beautiful device.

Speed

Compared to my 3G, this thing is a rocket! There is now very little lag in the OS. I can move between apps quickly and things like diving into the texting client take only a fraction of a second. On the 3G, this was a particularly onerous task, taking up to a minute to fully load. It's nice to have a usable phone again.

Antenna and Reception

I've experienced the death grip one time and one time only. My hands were slightly damp from washing them and when I held the phone in my left hand the cell radio suddenly went into searching mode. If I really squeeze down with the death grip (holding the phone in the left hand with the black bar on the lower left of the phone in the palm) I can see some signal attenuation, but by and large it has not been a problem.

I also experienced another issue I've never seen with my 3G. Over a period of about 15 minutes, while driving down the road, every call I made dropped suddenly. The person I was calling wondered just what the heck was going on, as I kept dialing and redialing them. Whether this was due to the phone or the network, I can't say, but that never happened with my 3G.

Other than those two times, the reception has been fabulous. Call clarity and quality are awesome. Folks are telling me I sound better. The noise cancelling mic is truly awesome. This is a great phone to make calls on.

Pictures and Video

The camera on this device is amazing. The stills it takes are phenomenal and with the addition of HDR photos in iOS 4.1 you can really get some amazing shots. The flash also adds a new dimension lacking in the previous models. I am starting to think that I might be able to leave my point-and-shoot behind in favor of my phone, and that's an awesome thing, indeed.

This is a silly little picture of a kids toy, but it really serves to illustrate how sharp these photos are. This was taken with the flash.

But as good as the stills are, video is even better. The 720p video this iPhone takes is smooth like butter. The details are crisp and there's almost no jelly-vision, even if your hands aren't too steady when you're filming. Add in the capability (at the premium app price of $4.99) to edit movies on the device with iMovie, and you really have an awesome deal.

Here's a movie shot with my iPhone, put together with iMovie. It's of my son's soccer practice. Such a simple little thing, but it really is an amazing to have done this on a phone. (Looks like Blogger really compressed the mess out of the movie, here's the original on YouTube)

Conclusion

This is truly an awesome device. Despite some minor complaints, it is both beautiful and functional. I will be able to leave behind both my Flip and my point-and-shoot. I can edit video on it and upload that video to YouTube without a problem from the device. It's snappy and runs iOS flawlessly. Upgrading from a previous iOS device is painless.

Truly, I can't recommend this phone more.

Double Fisting It on AT&T

2010-08-13T07:54:00.000-07:00

I have recently had to severely analyze my motivations for which phone I will get next. Included in this analysis were factors such as work getting me a development phone, would I stay with AT&T and would I leave the iPhone behind.

In answer to my last post, whether I would shift with the wind on which platform I wanted to commit to, the answer is both yes and no. When I watched WWDC I became very excited about the iPhone 4. What a gorgeous piece of hardware, and iOS has already proven to be a capable OS so I had no concerns there. In short, there was much that was attractive to me about the iPhone 4. Add in the capability to take slick 720p videos and edit them on-device and it seemed like a no-brainer.

Then, antennae-gate hit. It quite literally paralyzed me where I stood. Could it be true, the vaunted iPhone 4 was a deeply flawed device? Would simply holding the phone render it useless as a phone? The echo-chamber of the gadget media covered the story in excruciating detail.

As it turns out, it's not such a bad flaw, only showing up in low signal-strength areas. Further, Apple is giving out bumpers with every iPhone 4. The bumper mitigates the problem completely. While I can't say that I love the thought that some high-end gadget has a design flaw which necessitates a case, the benefits of having an iPhone 4 greatly out-weigh the downsides. Simply put, Apple has put out another superb machine for indoctrinating the masses and across the board all the pundits that have obtained one say it is a beautifully useful device.

So, with the iPhone 4 back in the mix as a viable alternative (meaning I couldn't simply dismiss iOS and switch to Android), which phone would I get. Ironically, work and AT&T provided the answer.

Work has agreed to the necessity of development hardware and is allowing us to choose which device we would like. We have to provide the service, should we want it, but by and large the choice of phone is ours. The development team all agreed that Android was the way to go, as it is likely our customer will have Android phones at their disposal.

Providing the service became a bit of a sticky wicket, though. Did I really want another device, one that could be taken away at any time, and have to provide at least $40/month in service for it (on Verizon or AT&T)? Did I want to switch to a lower tier provider like Sprint or T-Mobile? Heck, did I want to carry around two devices with two numbers? All of these questions came to mind, and I researched them to death.

Turns out that Sprint and T-Mobile both have great plans which roll data into the plan. On AT&T and Verizon, you have to pay ~$30 for data per smartphone and there is very little way around that. However, the savings I would get on Sprint or T-Mobile weren't that much greater than what I have now. If the answer had been that I could have my cost on my mobile bill go down $50/month, it would have been a no-brainer, but that is not the case.

Adding to the complexity is the decision I made to have both major phone OSes at my disposal. I would like to have my hand in both Android and in iOS, with an eye towards developing in both. This requirement ruled out Sprint, but due to rumors did not rule out Verizon. However, the modus operandi of AT&T provided a clear-cut answer.

AT&T is a GSM carrier. That means that the service each phone has is dependent on which SIM card is in the phone at any given time. As long as I kept my iPhone plan/SIM, which provides data, SMS and phone service, I can switch my SIM into a different phone and still have all of that service, as well as keeping the same number. It's a beautiful solution. I could get both an iPhone 4 and an Android device and, depending on the necessity of the day, switch my SIM around to carry which phone I needed.

Now, I have to find the right Android device on AT&T. A Nexus One would suffice, but I will have to go to Ebay for one of those, so I may wait around and see what percolates out in the next few months. Regardless, I believe I will be able to carry both OSes, and this makes my geeky heart very happy.

Wherever the Wind Blows

2010-05-26T18:07:00.000-07:00

Just recently I spent a couple of days watching the Google I/O 2010 keynotes live. The first half of the second keynote was dedicated solely to Android 2.2 (codename Froyo, which is short for Frozen Yogurt) and man, I'm incredibly pumped about it. Despite the fact that I don't have an Android phone, nor am I likely to get one anytime soon.

It did get me thinking, though, about whether or not I will upgrade my iPhone come June 7 when the new iPhone is released. It certainly has put a lot of hesitation in my mind. Some of that hesitation is due to my frustration with the iPhone. I recently decided that I would finally take the plunge and spring for the iPhone dev kit. I amped myself up to start writing nifty apps on the iPhone and was completely gung-ho about the whole process.

Only to find out that the only way to write apps on for the iPhone is on a Mac... This caused me much angst.

Enter Google I/O (and some Engadget reviews about the EVO and Incredible) and the wind started to shift. By the end of the second keynote, I was committed to being an Android convert (with the added bonus that I could leave AT&T far behind). However, there was a question niggling in the back of my mind: Would my lust for a new iPhone reassert itself as I avidly watch the WWDC, where Apple will launch its new iPhone?

As any good fanboy will do, I decided to seek out as much information as I could about Froyo and the current batch of devices everyone is talking about. I turned to Mashable, Engadget and Ars, reading the same stories hashed and rehashed. I downloaded all of the pertinent Engadget podcasts. I devoured this info.

And, hidden in there, I found an interesting tidbit of info: A lot of these guys which are touting Froyo as the next coming of the mobile Jesus are reluctant to release their firm grasps on their iPhones. In fact, during the Engadget show, an audience member asked a panel of Engadget editors if they would be willing to leave behind their iPhones for an Android device. The answer, across the board? "No."

Here is an interesting perspective. Has Apple done such a good job that even those that don't want to be indoctrinated are truly indoctrinated? It's quite clear that the iPhone is the phone to beat. This shows up whenever the latest, greatest phone is reviewed. The question is always, is it an iPhone killer? The answer every time so far is, "No." In short, even though the pundits are crying out for a changing of the guard, they are unlikely to get one anytime soon.

Even though the wind is blowing towards Android right now, will it be blowing that direction in two weeks? I actually doubt it. Don't get me wrong, Android is a worthy platform. I want to develop for it. It has some many things that make it ideal for our projects (more on that in another post). It has strengths which are only now being developed and which, when they come to fruition, will be amazing. But, for now, in the words of Paul Miller from Engadget, "Now is the best time to wait."

So, I leave an eye to WWDC. Will I be able to stomach another two years of AT&T and Apple (I highly doubt that Verizon will get an iPhone this time around)? Will I find some way to secure the development environment which I so crave for iPhone OS devices? An answer of yes to either of these two questions will indeed commit me to another tour of duty with an iPhone.

What Should be Next for Enterprise Mashups?

2010-03-15T19:16:00.000-07:00

My friend over at JackBe, Chris Warner, recently sent me an email asking me what the next 2 to 3 years should hold for Enterprise Mashups. JackBe, in case you don't know, is the creator of the Enterprise Mashup software Presto. We've used Presto on a project and it's a wonderfully crafted piece of software with many, many possibilities in front of it. Presto has, at its heart, the creation of mashups, those elusive but so useful beasts which give you all kinds of new-fangled views into your data.

This is a topic I've spent a little bit of time thinking about, fancying myself to be at least a passable user and creator of mashups. I'm going to change up the question a bit, though, and delve into territory that is slightly afield. So, I propose these related questions: Just what the heck are Enterprise Mashups? And, what's next for them?

Any Ol' Programmer
Let's be honest, any programmer can get out there and create a mashup with a little elbow grease. The technology is there such that, with a little bit of web service lifting and a little bit of UI know-how you can create fantastical mashups that will make everyone sit up and take notice. What's more, if you do it right, your own mashup can, in turn, become a data source for someone else to consume. It's a bit like Sir Elton John says, that whole Circle of Life thing.

But, that's just the problem, isn't it? We're not all programmers, now, are we? Not all of us even have access to programmers (or, as we like to style ourselves now-a-days, "software engineers"), so it's not like we can ask someone else to do it for us. And, most important of all, programmer time is expensive. If you do have access to a programmer and task them with spending hours creating your pet mashup, you're likely to get a whack on the wrists from the company dumby stick. So, what use are mashups to the common-day man? They're nice to look at, but not really practical beyond that.

Enter the Enterprise
No, not the ship, though that would be cool. The Enterprise is (or should be) all about putting a useful tool into the hands of someone other than an engineer. To a certain extent it's about lowering the technical barrier of entry. Any programmer should be able to put a mashup together, but that doesn't mean that they are easy. There is a lot of wiring together and hacking things up that takes place.

Enter the folks at JackBe, handing us Presto. With software like Presto it's possible for someone of a less technical bent, someone that's not a programmer, to put together a mashup that's useful. This Enterprise tool takes some of the difficulty out of the creation. And that, in a nutshell, is what the Enterprise should be about.

What's Next?
Even though products like Presto lower the barrier of entry, there is still a long way to go. Presto shines at doing the actual mashing, wiring the data together to get the new view. But, one area where Enterprise Mashup software needs to tread next is into more robust presentation of that new view.

One thing that mashups excel at is the creation of data. One thing that data desperately wants to do is be visualized. There are so many tools out there for visualizing data, but most of them have a similar dilemma to non-enterprise mashups: it takes specialized skills to use them. Enterprise Mashups need to tread here, next, lowering the barrier of entry into rich presentation of the data that mashups are so good at generating.

To Infinity...
From there, well, who knows. I'd like to see tools which are good at creating not just a good presentation but actual interaction, all done so that the common layman can create them with minimal know-how. I've blogged before about how it's important for people like analysts to be able to do at least some kind of rudimentary software development. I see that Enterprise Mashups can fill this role once they come to full maturity. It's a big area, one that needs more exploration, and I'm quite happy that I get a peek into it at the ground floor.

Social Media: Capturing Knowledge in a Self-Organized Way

2009-12-30T08:40:00.000-08:00

It's the early 80s and you sit at your terminal with a stack of papers, a document holder and a keyboard. Your mission: Enter as many of the paper forms into the terminal as possible. Exciting work, isn't it?

The problem is that this is not an inaccurate way to view data entry today. Granted, a lot of the brute force work has been done and legacy systems exist from which to pull data. Further, the forms are now entered directly into the system as opposed to copied from paper, but as regards entering novel data the situation has changed little.

The other major problem is that this type of raw entry, which is generally entering data into a form, only captures defined phenomena. The data that is being entered, especially into a form, is often classified and defined in advance. There is no elasticity to what can be captured.

This is problematic in that you must have a clear picture of what you are capturing in advance. For hard problems and complex situations you very rarely know much, if anything, in advance. If your only valid form of capturing data is via traditional predefined methods, such as forms, then your ability to capture data, and eventually knowledge, is vastly compromised.

This revelation is nothing new, of course. People have been trying to innovate data entry and knowledge capture for several decades. But, what other types of data can be captured and how?

The Army is asking this exact question, if indirectly. In reading several SBIRs the concept of capturing the knowledge inherent in soldiers heads is coming to the forefront. It is being recognized that not only do experts have valid perspectives and answers, the boots on the ground do, as well (keep in mind, this is probably not a new perspective in the military, but is one that I have seen in several SBIRs recently). Beyond that, though, they are starting to explore how to bring that knowledge into existing systems.

The how of this is a serious question. Computer systems today are clearly defined and generally purposeful to a single end. Human thought, on the other hand, is often multi-purposed and the field of understanding human thinking (philosophy) has been around for as long as humans and has yet to reach one shared conclusion on how we think. Even if we could get some Matrix-like data jack implanted into soldiers heads could we really transfer the knowledge as it is represented inside their brains into a computer system?

Perhaps we need another way of gathering data since it would seem that direct access to the human mind would avail us little. There is a new(-ish) movement which is providing an answer: social media.

Probably one of the most useful aspects of social media is how important and interesting knowledge percolates to the top. This is done in various ways. For instance, if I see an interesting tweet on Twitter, I will retweet it. If I read something worthwhile on Facebook, I may comment on it or repost it. It's this interaction with the content that causes the interesting bits to rise to the top.

The exciting thing, at least from a systems perspective, is that this is self-organizing behavior. It is through the interaction of the components of the system (here, the components are the people) that the interesting bits are being obtained. While it may be difficult to capture human thought and knowledge in its native form, it's not as difficult to capture the important pieces as they are being defined by the social system already.

Further, the nature of social media, in that it tends to interact in bite-sized, discrete pieces, means that the computer system needs not have much understanding of what it is capturing at all. The knowledge is already distilled into its core component, often with attribution, and the computer system merely need remember it. It can be stored without pre-defined labels and fields.

The thing which the computer system must crucially provide is a robust search capability. Whether this search capability is enacted after the fact, or whether there is a component of the system which searches as knowledge comes in is immaterial. As long as the system can search through the knowledge is what's important.

Eventually, this captured knowledge can be used and reused as more people interact with it. Each interaction would in essence refine the knowledge, making it more useful to the computer system and the people in general.

Git vs. SVN (I know, right, another one?!?!)

2009-11-10T11:38:00.000-08:00

I've been endeavoring to set up a code repository (or even a document repository, if that need should arise) and have been weighing the merits of both Git and SVN.

At the very heart of the comparisons lies the manner in which Git and SVN operate. SVN is a central repository. When you "checkout" a file in SVN, you get only the most recent version. Should you need to do backwards comparisons you must communicate with the server for this. You get no history, either. SVN relies on the availability of the central repository to operate.

Git, on the other hand, is fully distributed. There is generally a "blessed repository" from which everyone will start and ultimately commit to, but when you "clone" that repository you get a full copy of it. Backing up a Git repository with many contributors is actually trivial as there are countless copies of that repository floating around.

Branching

Another major difference revolves around branching. In Git, branching is a way of life (as is the subsequent merging of branches). You want to develop a new feature? Branch on your local box and work on it there, then merge it back into your local main repository before committing back to the blessed repository.

This is not so in SVN. Branching is not done as often (nor as easily). Branching must occur in the central repository and is not a way of life. In this area Git outshines SVN.

Client Tools

One area where Git does not outshine SVN is in the client tools. SVN has been around forever (in digital terms). There are very elegant clients for SVN (such as TortoiseSVN) which allow for an incredible ease of use when working with repositories. Further, most modern IDEs have SVN repository manipulation as a core capability. There are several options for working with SVN in Eclipse, for instance, one of which is core to Eclipse itself.

Git, on the other hand, is young. The tools out there are not nearly as elegant nor are they as wide-spread. What's worse, Git is incredibly Linux centered. There are two Windows clients for Git (with the advent of JGit, that will climb to three), all of which require one to work with the command line. Some GUI projects, such as TortoiseGit, are in the works but will not be ready for prime-time for a while. The last issue here is that there is only limited integration with IDEs. With time, these situations will change, but for now it is a major draw-back to adoption by those other than the most determined.

Ease of Setup

To the end that I would like to work with both systems I decided to set up both on our Windows Server 2003 server. I chose to use Cygwin and OpenSSH, along with Gitosis (a Perl mod for Git), for Git. I used Shannon Cornish's tutorial to set things up (along with a little help from scie.nti.st on matters Gitosis). This turned out to be a rather easy and relatively painless way to go about things.

The basic gist is that you install Git when you install Cygwin then install and setup OpenSSH (by far the most difficult part). At this point you can connect to the server using SSH and clone any repository you would like. Installing Gitosis on top of things (recursively using Git, no less, which is so cool in my book) allows you to use public/private key pairs to authenticate users. You can then use Git to clone the control repository of Gitosis and admin the system remotely. Very elegant and one which doesn't require the anticipated user to have to input a password or create an account on the server.

Setting up SVN was more difficult. The differences, though, are myriad. While the above Git scheme works on SSH the method I chose to use for SVN works over HTTP/HTTPS, which has advantages all of its own. I worked off of several tutorials, but the most significant was this tutorial.

The real difficult part here is that you have to rely on Apache. It seems a bit overkill to have to install Apache and get it running in order to serve up your repository, but this is the accepted way of doing things. Once you have it running you must still log into the server to create a username/password combo for any user that wants to use the system, and you must also log in to the server in order to administer the repository.

The Best of Both Worlds

I have to say that the thing which gets me most excited about Git is the notion of branching it carries with it. I really like the thought of creating a local branch for every new feature. It seems natural to me.

On the other hand, I don't think that I want to saddle everyone else around me with command line tools and vi if they want to work with our repositories. So, can a compromise be made?

In fact, it can! Git has the wonderful ability to clone and commit to SVN repositories. The real details are outlined here by Clinton R. Nixon. In this way, I can take the pain of the command line on myself without foisting it on anyone else, but I also get all of the wonderful features Git brings with it.

Conclusion

In light of all of this, we will be hosting our repositories using SVN. However, I will be keeping an eye towards the maturity of the Git clients. If they should ever advance to the level where any "power user" can attain them, then we very well might switch.

A Thought on Environments: Portability

2009-10-07T12:18:00.001-07:00

My friend asked me yesterday what I thought of the Kindle. My response was that I was a fan of actually holding a book, feeling the paper, reading in a full fidelity mode. I spoke of how tired my eyes could become from reading on a screen all day. I made a decent case for not adopting the Kindle.

Then, on a whim, I checked out the Kindle app for the iPhone and immediately found myself sucked in.

The first thing that did it was the free availability of a book that we have all been discussing here at the office, Bertrand Russell's "The Problems of Philosophy". Turns out that it is a "classic" and Amazon offers many of the classics for free. I have now downloaded 10 free classics for my iPhone Kindle app and am well on my way to finding book reading Nirvana.

Free content put the hook in my mouth, but what set it was the concept of WhisperSync. WhisperSync is a service that Amazon offers which will sync your content between devices. Now, this is not just the raw content, this is the detailed content, the state content.

For instance, let's say that I am on page 50 of "The Problems of Philosophy" on my iPhone. Further, let us say that I have a Kindle at home on which I do the bulk of my reading. As I read on the iPhone, the Kindle app updates the state for that book. When I get home and fire up my (physical) Kindle my place in the book comes right up (in computer terms, the state is restored). No futzing around with finding my place, everything is just magically the same.

This got me thinking about the synchronicity of environments. In our research the environment you work in is of paramount importance. That environment can be unique to you, or you can share it with others. Everyone can have their own, if necessary. The environment is at least partially reflected in software.

One thing Dr. Sousa-Poza and I talk about a lot is the ability to "save" environments. Environments should be transportable and shareable. If you need to see what I see then you should be able to load up a copy of my environment, see things just as I see them with the data I've been using. What's more, your environment should be able to subsume my environment! Environments should be nestable yet discrete.

WhisperSync brings an interesting possibility to my mind. Shouldn't the environmental changes that I enact on one device translate to another device? What if I access my environment from my iPhone and then switch to my laptop or a web browser? Shouldn't the environment be exactly the same?

This necessitates two things: A place of storage (centralized or decentralized, makes no difference) that all environments have in common and the ability to capture state.

Ideally, the system would operate like this (from the 10,000 foot view):

I access my environment and make some change to its state.
That change is transmitted to the "server" which keeps track of environment state. ("server" is in quotations as a means to capture an idea. It need not be an actual server. It might be better to think of it as a state oracle)
I switch to a different device.
As I start up my environment on the new device the state is restored from the "server".
I continue my work.

Such a system would be truly powerful from a portability standpoint. I could do my work wherever I needed to be and have that work mirrored wherever I go. I could work from multiple devices and not lose a thing.

An Analyst's Development Environment

2009-08-10T08:36:00.000-07:00

Here in the land of academic research we're working with a "new" take on mashups. It seems like a no-brainer to me but a lot of people have expressed interest and surprise when I explain to them what we're doing. For now let's call it an analyst's development environment (ADE).

One thing that mashups are really, really good at is taking disparate data sources and allowing "momentary" relationships in the sources to be created. This in effect creates a new data source that is a fusion of the inputs. As is often the case in fusions, this new source tends to be more than just the sum of the parts. You often come up with new views on the data as you add extra sources.

Most people stop here at the fusion stage. Once they have the new view onto the data they rely on other tools outside the scope of a mashup to do interesting things. They might pipe that data into a tool such as Fusion Charts in order to visualize it or they might pipe it into an analysis tool such as a model or sim. But, why do they need to leave the scope of the mashup to do this? What if that analysis or the creation of the Fusion Charts XML was an automated part of the mashup itself?

Mashups deal with web services primarily (though there are some nifty products out there that allow you to mash more than just web services). A web service is usually considered to be a data source. But, in practice they are much more than that. Consider all of the specialized web services provided by Google for geolocation or Amazon for looking up aspects of books. The simplest example I can give you is Google's web service which converts an address to a lat and long pair (called geocoding). With these in mind let's take a different look at web services. Let's look at them as processing units.

A processing unit has 3 criteria: it takes input; does something interesting with that input; and provides output. Processing units are the basis of modern programming. They're known as methods, functions, procedures, etc. depending on context. We can most often build bigger processing units from simpler units.

Web services fit these 3 criteria handily. You can easily provide input, they can easily do something interesting with that input and then just as easily provide output. All communication is done in a standardized protocol driven environment.

The interesting thing about web services is that we can string them together (with the right tools) rather easily into processes. That's exactly what we're doing here. Each web service is either a data source or a processing unit. Given the ability to ferry data from one web service to the next (in an easy way) it is possible to create mashups that do more than just mash data. They actually do some form of processing.

Consider what it would be like if you had a web service endpoint attached to a model? You could pre-mash your data from various sources then run it all through the model and create a new output that would be very interesting. It would be so easy.

Using Presto we recently put together a demo which worked along these lines. It made our demo come together in several weeks rather than over several months. We used Presto to access databases then ferried that data (in XML format) into a custom built web service that took said data and ran XSL transforms on it. That produced Fusion Charts XML which we then piped into our presentation layer for visualization. It was easy.

Here is a diagram of what the actual flow of the mashup was.

Here is a screen shot of the actual chart produced by the generated Fusion Charts XML.

An ADE would work in a similar way. Using provided tools which allow for ferrying of data from one endpoint to another and given a grab-bag of analysis and transformation web services an analyst could create some amazing things with little effort or technical know-how. The only developer support would be in the creation of any custom web services. It could be a very powerful tool.

The Walled Garden

2009-07-22T07:48:00.000-07:00

Let me hereby declare that I love my iPhone. It is useful and wonderful and keeps me connected all the time. I have been using it in lieu of my computer at home for quite some time now. I write emails on it, craft witty 140 character tweets on a regular basis, listen to books on iPod and even play extremely enjoyable games. It is a great experience.

I have, however, begun to chafe under the strictures placed on my iPhone by both AT&T and Apple (often in conjunction with each other).

My gripes against AT&T are especially aggravating. I pay them enough money as it is for the privilege of using my iPhone, why do I need to pay them even more in order to use my iPhone as a modem? It does not seem fair that I will have to shell out an additional $30/month to do what is freely available on other, older and less capable smart phones.

What's worse, if AT&T sees an app as competitive to it's business model, it will limit that app, or flat out deny it! Consider Skype: Skype offers free calls over the Internet to other Skype users, yet AT&T will not allow Skype to make calls over its 3G or Edge networks. They pull the undue competition card.

On the Apple front, a nifty app came to my attention recently that I thought was a truly innovative and awesome use of the iPhone. Given an iPhone 3GS (with its video capabilities, compass and GPS) an "Augmented Reality" app has been developed called TwitARound.

TwitARound looks at the tweets from Twitter in your area and plots them on a map. The AR part, though, comes when you hold your phone up. The app takes your GPS position and your bearing from the compass and lays the tweet on the screen. So, as you move in a circle with your iPhone in front of your face, you can see the actual locations on your iPhone of the tweets as they would appear if the tweets were layered over real life. It's quite awesome and I would like to see more apps like this.

However, because TwitARound accesses APIs which Apple has not, but should have, made public, it cannot be published in the iTunes store.

Apple plays the non-public API card too much. For instance, they did not make their "find my phone" APIs public so that they could charge you a monthly fee through mobileMe. There are already jailbroken apps which can do this, but since they didn't make the APIs public, you won't see legitimate apps show up in the app store.

Call me naive or non-business-savvy, but all of this seems like bad business to me. As a consumer, I want freedom. It's my device, I should be able to do with it as I choose.

So, while I love my iPhone, I chafe. Yes, I chafe.

Update: (on 7/29/09)

First off, it turns out that Apple will release the video camera APIs with iPhone OS 3.1 (per Ars here). Yay for Apple on this one. It's good to see that some of the "hidden" functionality is being exposed. Now, let's see if they expose the "find my phone" API or if they milk it for more money.

Secondly, the app denial shenanigans continue. In a story here (also on Ars) it appears that all apps relating to Google Voice are being pulled and any apps which feature Google Voice are being denied. The scuttlebutt is that AT&T is pulling the strings here. Some disagree, but my vote goes towards AT&T.

The Dark Side of Twitter

2009-07-22T07:24:00.000-07:00

I've seen a very interesting phenomenon going on in the Twitter-verse recently. It has brought to my attention that Twitter (and micro-blogging in general) can be used for reasons that are not above-board. What, pray-tell, is this dark and nefarious phenomenon?

I keep getting followed by prostitutes.

The first time it happened I just thought it was some random individual with a sick sense of self. However, the next day, another woman of the same ilk followed me, and the next day another. That's when I started getting curious (not about what the women offered, but about what was really going on).

Invariably, they all posted a provocative picture of a woman with at least one post which was anywhere from lewd to slightly suggestive. That post would have a link attached. The link takes you to some triple-X "dating" service. Within a couple of days the account is shut down (you get the "Nothing to see here, move along" message when you try to visit the account).

No doubt, for some reason I am not aware of my twitter user name has been picked up by this "dating" service and they keep following me with fake accounts, all in vain hopes of promoting their "service". It's all at least partly automated, it has to be, and there's probably one person sitting behind a desk creating profiles then running those profiles through some tool they had custom made to follow a few thousand people.

The practice, though, really brings questions to my mind about what twitter can't be used for. If it can be used for prostitute marketing, why not black-market marketing or subversive political marketing? Why even marketing at all? I once had the privilege of speaking with an individual that detailed how an anarchist group used Twitter to attempt to disrupt the RNC in Colorado.

Of course, far from being upset by all of this I tend to think of this as rather ingenious. What uses can Twitter serve? What's the most creative use any of you have seen?

JavaScript: Callbacks in Loops

2009-04-27T09:08:00.000-07:00

I just finished a mashup that had to be blogged about. I suffered to find this solution, and I wanted to share what I learned with the world.

In the mashup I took a twitter feed and plotted the tweets onto a map based on the location of the tweeter. Let me set the stage.

The Google Map has already been set up and the list of tweets has been obtained. It is now time to plot the tweets onto the map. This will be done within a function called addMarkers. The HTTP Geocoder that Google provides will be doing our geocoding. For more information on this service, see this.

Keep in mind that I'm doing all of this in a Presto Mashlet, and will be calling out to the HTTP Geocoder via a URLProxy call that is undocumented but available for use.

At first blush, the following approach seems appropriate. Here is an excerpt from the addMarkers function:

However, this suffers from a very serious drawback, and that drawback revolves around the scope of the function as it exists on the stack. Remember that you are calling out and receiving an asynchronous response via the callback. There's no telling where this loop will be when a callback returns, but the scope of the function is maintained on the stack until all of the callbacks have been completed.

When a callback returns, the current value of i will be used to index into tweets! Since all of these calls take time, the most common result is that i will actually be out of bounds of tweets. Recall that updating the loop variable is the last operation done in any JavaScript for loop. Once you have looped through all of your indexes you, of necessity, must set i to be out of bounds of tweets. Therefor, i will be equal with tweets.length.

The result is that you pass an undefined object into placeMarker in place of what should have been the tweet.

The next logical step is that you should create a variable to hold the value of i, like this:

var myTweet = i;
...
this.placeMarker(point, tweets[myTweet]);

However, this will fail as well!

The problem here is that myTweet is still within the scope of our addMarkers function. addMarkers will therefor have only one copy of myTweet. Once again, you end up in a situation where the loop will probably finish before any of the callbacks return. The net result this time, however, is slightly different. You will pass in a valid tweet to placeMarkers, but it will be the last tweet in every instance. You'll have the same tweet attached to all of your markers on the map, the last tweet in the list.

So, how do you remove the timing issues? This is where I suffered. I hunted and pecked out half-solutions for quite a while. Finally, I had to start thinking outside of the normal box to come up with a solution.

The whole problem revolves around all of the callbacks returning to a shared scope in the stack, that being the scope of addMarkers. Once you consider it that way, it becomes obvious that providing each callback with its own scope on the stack is what is needed. The way to do that is to have a function fire off the HTTP Geocoder request. The function will get its own spot on the stack and will have its own scope. Let addMarkers maintain the loop and call this function whenever it wants to fire off a request. Pass in the tweets and the desired value of i to be remembered.

Consider the following:

This approach will result in the correct tweet being displayed with the correct marker on the map.

Como Se Llama?

2009-04-15T06:16:00.000-07:00

Originally, I created my twitter account with the handle @jitlife. Obviously, jitlife is my blog, so I thought it made sense. After all, I want people reading my blog, right? Twitter seemed like a good pointer to my blog.

However, I started rethinking this mindset and eventually asked myself this question: Am I marketing my blog, or am I marketing me? By being @jitlife, I was marketing my blog. Therefor, I determined to change my twitter handle.

In deciding on my new twitter handle, I came up with a few criteria:

It has to be short
It has to reference me as an individual

I was going to add as 3 that it had to be clever, but the more that I thought about this, the more I realized that the first 2 are the most important, and in that order.

It has to be short as twitter only allows 140 characters. If someone is @replying to me and they have to type in a 15 character handle, well, they'll be less inclined to do so (at least from a mobile device) and they'll also have less space to say what they want to say.

That it must reference me is quite obvious once you realize that I'm marketing myself. The problem here is that all of the obvious references to me were taken! @rollins, @mrollins, @mikerollins, etc. All, gone. Most were taken and had only one or two posts, which is frustrating, but so is life.

Barring the obvious, I decided to get clever. I chose @rollinsio. Briefly, it's a silly name I call myself when I'm talking in a fake Spanish accent but it's also clever in that it could stand for Rollins I/O: perfect for twitter! It's short and it references me (rollins is prominent).

More Effective View Management in Web Pages

2009-04-13T08:14:00.000-07:00

One of the research scientists I work for and I have been going 'round and around recently about web desktops. In a web desktop you translate the traditional desktop view into a browser. For examples see Ext's web desktop and this actual web desktop OS. In question was how do you navigate between views of various applications in an efficient manner?

The good professor drew a distinction between how a traditional web application represents a set of views vs. how a desktop represents a set of views. In the traditional web application a set of views is often represented using tabs. You have a tab for each view of the application. Google has taken this idea to the extreme. Consider Google Docs. In Google Docs when you want to open a new document, you open a new tab. You can keep opening new documents (and consequently new tabs) until you have a bazilion of them, at which point navigation becomes a nightmare.

On the other hand, you have how a traditional desktop represents views: new views are organized on the "start bar" (forgive the Windows-centric frame of references) with icons. Each icon may have some text and an image to represent it. When you mouse over a given icon you get a tooltip which provides you more information.

The question then becomes how do you find a particular view when you have many views open?

In the web application you may be lucky enough to have individual titles on each tab, but barring that, you have to click on each tab and work your way through potentially all of them before you find what you want.

In a desktop, though, you often can pick out the view that you want simply by glancing at the images in the icons. At the very least, this will narrow your search down. You can then rely on the titles of the icons in question to further narrow the choices. If you're forced to, you can obtain the tooltips for each icon. Your ultimate last step is to look at each view individually. However, looking at each view individually isn't as bad as looking at each tab as you've already excluded some views out of hand because of the images, titles and tooltips. At the very least, you're certainly going to be looking through fewer views.

Quite clearly the desktop way of searching through views is more effective.

There are other factors to be considered as well. The start bar is static across all views. Being part of the default view of the OS it doesn't go away. You never (or rarely) lose your navigation between views. The same cant' be said with web applications.

Further, the start bar only shows the views that are active. If a list of all possible views is desired, you can click on the actual start button to obtain it. A traditional link list shows all possible views, not just the ones that have been accessed during the current session.

Clearly a more effective way of switching views in web applications is needed.

I propose a tool that adheres to the following rules:

Each view will be represented by an image, a title and a tooltext
A space for all active views will be set aside on the page
A list of all possible views can be called for but is not in available by default

While I'm not a proponent of recreating the desktop environment in a web browser, the above idea would be truly powerful in a web application where many views can coexist.

Building Your Professional Brand: Drink the Kool-aid

2009-04-09T06:19:00.001-07:00

Is what you have to say compelling, insightful, interesting or useful? Would you like to get this message out to others? Would you like to receive the acknowledgment of your peers for what you have to say?

If you answer yes to all of the above then you need a professional brand. A professional brand is something that marks you as uniquely you, something that points directly at you in such a way that others recognize you. It's not quite a kind of fame, but it is a way of differentiating yourself from the masses.

I've recently become interested in establishing my own professional brand and I've started looking around at ways to do that. Here are a few of the observations I've made.

You need a soapbox

You have to have some place where you can expound on your ideas to the fullest extent possible. Follow out every thought, every nuance of an argument and feel comfortable doing it. Speak your mind!

This is where your blog comes in. It is your soapbox. You can discuss whatever you like there, but the more erudite and insightful your blogs are, the more folks will come back after the first dose.

However, how do you bring people to your soapbox to partake of the Kool-aid you're doling out?

You need a megaphone

You need some forum wherein you can succinctly give out information that will draw others back to your soapbox. You need something that is light-weight and is easily consumable with a minimum of effort.

You need twitter.

140 characters is not (generally) painful to consume. You can read a tweet and in a split second decide if it's something that your interested in. Thus, if you can craft your tweets to be compelling enough for folks to be interested, then you can use twitter to announce your new blogs.

Of course, this necessitates having a following on twitter, but this is a recursive process. Your first few followers will likely be your friends or those you capture by chance. Consider, though, the phenomenon of the re-tweet. If what you have to say is compelling enough then there is a good chance those that follow you will RT your tweet to those that follow them, and on and on.

Shameless self-promotion is of value here. If you think what you have to say has value, then there's no harm in promoting it. Someone else may find it of value, too. Remember, if it's profound enough for you to blog about, then it's probably profound enough for someone else to read.

But, don't just limit your tweets to self-promotion! Tweet about things that fall in line with your brand or RT information that is compelling in and of its own right. If others begin to see you as a fount of useful information they're likely to buy into your Kool-aid.

Mark Drapeau (@cheeky_geeky) has a great blog about "Expanding Your Twitter Base". His rule of thumb is provide valuable information to others on a regular basis.

Shepherd your following

Once you have others interested in your Kool-aid, you have to take care of them. Shepherd your following by interacting with them and acknowledging them. You can do this by responding to comments on your blog or by RT'ing interesting things that your followers aim at you. The main point is that you have to be involved in as personal a way as possible. If you're involved personally then others will be more inclined to recommend you to those that they know.

Also, don't just fall off the face of the Earth for any lengthy amount of time. You have to keep the Kool-aid flowing! The more often you present new ideas and information, the more likely folks are to come back and see what the latest is. If you only post a blog once every 2 months, well, you're not going to have an easy time building a following. If, however, you are prolific poster and always provide value, you're likely to garner a larger following faster.

It's a Transforming Process!

2009-04-01T17:55:00.000-07:00

So, right now I'm working on a gadget that takes in generic info and sends out Fusion Charts XML. It's a SOAP service and there will be many service endpoints, but right now there are only 2, one for a simple, single series bar chart, and one for a multi-series "drag node chart" (think network diagram with drag able nodes).

I chose to go about it in a different manner than I've seen a lot of people use for Fusion Charts, though. The prevailing way that I've seen people create Fusion Charts XML is to take the data in on the JavaScript side and create the XML, in string format, in the JavaScript. For this approach, I have only one thing to say: Building XML in JavaScript is less than optimal (translation: it sucks).

So, I decided to go about it in the web service itself. My web service is written in Java and once you get the question into Java a few, more palatable, alternatives suggest themselves. In my web service there are 3 distinct transformations: request object to traditional object; traditional object to simplified XML; simplified XML to Fusion Charts XML.

The request object to traditional object transformation is really the beast. The inputs for the endpoints are comma-delimited strings. A lot of work goes into parsing those strings and putting them into the more traditional object. I have my inputs be comma-delimited strings so that the Presto JUMP requests can invoke them effectively. I could just as easily have one of my endpoints be a direct invocation of the more traditional object, but as I understand it, that's a bit of a bad practice.

Once I have my traditional object the easiest step occurs. In this step I use XStream to serialize the traditional object into a simplified XML. If you've never used XStream, it's very simple, very powerful and I recommend it highly.

The last step is where the real magic happens, though. Here is where I transform the simplified XML into Fusion Charts XML. I use the Saxonica XSLT engine to do the transformation and it's a matter of using the right tool for the right job (with regards to using XSLT to transform XML).

XSLT is designed to transform XML, whether it be from XML to XML or XML to some other language. You write a transformation wherein you process the source XML and then create a document in the desired format. It's really not all that hard to take the simplified XML and transform it into the Fusion Charts XML.

Once I have the Fusion Charts XML document I send it back out of the service in a special response that contains the document in string format and the name of the Fusion Chart swf file that will correctly process that document. When my response arrives at its destination all that needs to be done is input the Fusion Charts XML document into the Flash engine with the correct swf file pulled up and voila, it's all done.

I like this approach in that it moves all of the heavy lifting out of the display side (the mashlet, in my case) and into a much more suitable environment, that being a Java web service. I don't have to do endless string concatenation that is hard to debug inside of the JavaScript presentation layer. As a matter of fact, I can write all of the pieces independently of each other and then put them all together in the end. It's a nice break up of all of the work.

An acknowledgment needs to go to @angleofsight for his help in getting this whole process set up. Without his paving of the way I wouldn't be anywhere near as far along as I am right now.

Gov 2.0 Camp: Virtual Worlds

2009-03-28T07:20:00.000-07:00

Virtual Worlds is a topic that I find popping up more and more. I've always taken it with a grain of salt, though, as most of the time I hear it in relation to talk about how it'll make everything better. I shy away from that kind of talk as I know that silver bullets don't exist.

Further, my only real experience with a virtual world stems from the time that I played World of Warcraft. So, in my mind there's extra baggage attached in that I have a hard time seeing how a virtual world could be more than a game. When I entered this session, I decided to try to leave behind my baggage or, at the very least, dispel some of it.

This session was given by members of the Federal Consortium for Virtual Worlds. There were four panelists, but I did not, unfortunately, get their names or contacts.

As I entered this session, I had one question in mind: Are virtual worlds useful for more than just playing?

Surprisingly, there are three government agencies which are using virtual worlds in some capacity: NOAA, NASA and the CDC. All three of these agencies use virtual worlds for information delivery and training. NASA may be the least shocking example here, though, as it makes sense for them to create, say, a virtual world of Mars and then use that virtual world to train rover drivers. It's NOAA that has the most fascinating use of virtual worlds.

NOAA Islands (see towards the bottom of the page under the heading "NOAA Virtual World") is a virtual world that runs in Second Life (a very popular virtual world). In NOAA Islands, one is allowed to create their own mini-planet and then strive to create a stable weather pattern on that planet. As you add one effect, though, its repercussions are seen in other parts of the mini-planet. If you add too much rain, well, you'll flood the crops that are growing in a valley or low-lying area. If you make the world too hot, you'll melt the polar ice caps. In this way, NOAA attempts to convey the intricacies of climate to the uninitiated.

Another example of using virtual worlds in unique and non-playful ways was anecdotally related by one of the panelists. He commented on a school of engineering that he knew of that was using virtual worlds in order to prototype the buildings and structures they were designing. Specifically of use was the ability to determine if handicap access was sufficient for a given building.

However, of all the examples given, the most common example given was that of collaboration. The idea was set forth that in today's world of budget cuts and massive organizational structures which can span a country if not the globe money could be saved if, instead of collaborating face-to-face, people could collaborate virtually.

To this, I asked the question of what advantages do a virtual world provide that more traditional forms of telecommunication don't? The response was both intangible and interesting.

Basically, the consensus from the panel was that virtual worlds provide a sort of solidification of knowledge based on their immersiveness. One presenter described it as "informational bandwidth". Virtual worlds, being immersive, allow one to convey large amounts of information faster. Further, this immersiveness adds context to the memories created, making the information conveyed more "solid" or "real". The information has a better chance of sticking due to the immersive nature of a virtual world.

This concept, that the immersiveness of a virtual world added to the quality of the information that was transmitted seemed to find fertile ground in the audience. One audience member stated that "where it is is what it is", meaning that the memory of something can be tied in no small part to the place where the memory was experienced.

However, someone else asked a very probing question: Do virtual worlds limit or enhance productivity?

The answer to this was less than satisfactory, in my mind: The worlds are getting better with productivity software. Currently, many virtual worlds allow for desktop sharing and persistence of environment. But, does "getting better" mean "good enough"? In my mind, no.

So, how did this session shape my opinion on virtual worlds? Are virtual worlds useful for more than just play?

Yes, they are useful for more than just play.

First and foremost, I was highly impressed by NOAA's forward thinking in this space. So many times allowing people to just get out there and attempt something is the best way to convince them of your point. Allowing people to experiment with climate by actually creating it is ingenious and something that NOAA should be commended for.

Further, I can see how virtual worlds will soon play a huge role in training. Being able to attempt a task in a similar environment to the environment in which you will actually be performing the task is of great value. I can see how this could lead to safer working conditions in hazardous work environments, from military applications to industry.

I can also see that the advantage to prototyping is huge. Being able to put yourself into a users shoes (as in the case of testing handicap access in a building which is to be built) is of immeasurable value. If you add realistic physics to that, well, you've simply added even more value to the tool.

However, the one place I remain unconvinced is in what is probably the most important space of all (as far as widespread adoption of virtual worlds is concerned). For virtual worlds to be adopted whole-heartedly across government and industry they must facilitate the work that people do. To hear that the tools for productivity are still developing in my mind means that they are not ready yet for the main-stream.

That's not to say that virtual worlds don't have potential value in this most important space, though. Indeed, I feel that they have great potential. But, until I can see an example of where they make collaboration as easy, or near as easy, as actual collaboration is in real life I don't see that they will be widely adopted for this purpose.

With all that said, I'm going to keep my ears out, and my mind open, to this topic. I think that there is great work that can be done here and I look forward to what the researchers of today will do with this technology.

How Does the Magician Get What He Wants?

2009-02-11T10:29:00.000-08:00

As I started developing the core of the "platform" I realized that I had undertaken a huge task, a task I was not sure I could finish in any short amount of time. This caused me some angst but I figured that I would just have to move on, doing my best to develop my way out of an impossible situation. Not a fun position to be in.

Then, sometime around the end of December, a buddy of mine pointed me in the direction of a product called Presto, developed by the company JackBe. I thought that I had stumbled into my own brain on the web!

Keep in mind that the concept I was working on was that the platform would map gadgets together in such a way that we could dynamically generate them with ease, rapidly prototyping processes to see how they worked. In effect, the gadgets would be web services and the thing the platform would create would be a mashup.

As I said, I had envisioned something similar to PopFly, the ability to visually create mashups and then do something meaningful with them in our efforts to prototype processes. Presto is this and so much more.

First and foremost, Presto has a beautiful visual mashup maker called Wires. In Wires, you drag services from a palette onto your canvas and connect them. You can also drag in actions, blocks that allow you to do something with the results of other services. So, in the basic example I worked with, you have two RSS feeds. You can take the data from each feed and "merge" them (where merge is an action block) so that you effectively have one RSS feed from two. Then, you can add a filter (another action) which can take a dynamic input. In this way, you can create very complex mashups from some very basic building blocks.

If the complexity of Wires is not enough for you, however, they have a markup language for creating mashups called EMML. What you do in Wires is distilled down to EMML, but Wires doesn't offer the full capabilities of EMML (which is not to say that Wires is not fully featured, there are just some rather tricky things you can do with EMML that you can't do with Wires as they don't have a visual representation).

What's even cooler, and something that spoke to a need we had, is that each mashup is published as a service itself. So, you can include mashups in other mashups! The modularity is great.

But, you might be asking, how do you get the services in so that you can use them from the palette?

Well, Presto has a "Service Explorer" which allows you to import services from wherever they may be and "publish" them in the Presto server. However, there's a little more to it than that. Presto comes complete with a user authentication system that can be standalone or hook into LDAP or AD. When I import a service, I can assign rights to it, and only those in the appropriate groups can even see the service, or any mashup in which the service exists. You can also assign rights to mashups themselves.

So, that gave us the mashing capability that we've been looking for, but the goodies in this bag didn't end there!

What we have up to this point is a unique view of the data, but no visualization of the data. Enter the mashlet!

A mashlet is a view of the mashup in a portable and embeddable package. It is created in JavaScript. Once you've created a mashup, you can attach a mashlet to it so that others can see what the mashup provides. There are 5 prebuilt mashlet types: RSS; grid; chart; Yahoo Map; and XML. If your data fits into any of these predefined views, creating a mashlet is as simple as selecting the mashup or service, selecting the view, then publishing it. If you need a more complex view of your data you can create a mashlet by hand. The process for creating a mashlet by hand is rather well thought out and not all that hard to grasp.

Once a mashlet is published you can do one of several things with it.

The mashlets are served up from the Presto server in much the same way that any JavaScript object is. Right out of the gate, you can view a mashlet standalone, if you so choose. However, the real fun comes when you realize that you can embed mashlets into any HTML page you wish by simply including a script tag. You can also embed them in a MediaWiki, NetVibes or as a GoogleGadget. You can't ask for more flexibility. From what I understand, there are more embeddable objects coming, including things such as JSR-168 portlets.

So, from our perspective, Presto offers us 3 incredible capabilities: the ability to capture services from across the web; the ability to create mashups in an easy and visual manner; and the capability to add a face to the services and mashups we create, then embed that face wherever we need.

We've committed to Presto as the core of our platform and it puts us months, if not years, ahead of where we were.

We Need a Map to Talk to Each Other

2008-12-03T07:10:00.000-08:00

In a conversation I was having today the question about how two gadgets would communicate with each other came up. Initially, I was thinking that two gadgets would talk through an intermediary gadget which would convert the output of one gadget into the input for the second gadget.

There are a couple major problems with this approach:

Anytime you added a gadget you would possibly have to write a whole slew of interpreter gadgets for the gadgets you wanted to give input to or take input from. At a minimum, every gadget would require at least one interpreter gadget.
The whole gadget as a web service concept breaks down when you start talking about interpreter gadgets. They do not need to reside on the web, they need to reside on the platform. Should you have a platform based gadget? Is that really worthwhile or does it break the nice clean system we've conceptualized?

In the midst of the conversation the idea of being able to do mash-ups was discussed, and this spurred me into a new area of thought regarding how gadgets could talk. Mash-ups always make me think of PopFly. If you've never played with PopFly, I highly recommend it. It's one of the coolest things to come out of Microsoft since Surface.

In PopFly, you capture two (or more) web service endpoints and then you map the fields in the endpoints together. This allows you to create dynamic mash-ups on the fly, among other things. So, let's say that I have a web service that gives me a weather report for a list of locations and I have a mapping web service (like Google Earth). I can map the location info from the weather report list onto the mapping service and create a series of push-pins on the map that will show you the weather report when you mouse over them. It's pretty nifty and very simple to use.

Why not use the mapping between web services concept for our platform?

It's a much more elegant solution. It has several advantages over the other approach I mentioned:

A mapping is much easier to generate than a complete gadget. What's more, if done right, it could be done in a UI! PopFly has demonstrated this very well.
The platform would only have to remember a mapping (which could be done in XML quite easily) for any communication between gadgets. That eliminates the problem of having platform-bound gadgets.
The mappings could be based off of the WSDLs that the web services publish. This would allow a direct mapping from the output of one gadget to the input of another. No intermediary step is needed.

I have to say, I'm quite enamored of the idea. I think that the first step is going to be creating a set of gadgets that are supposed to talk. I've already created the first gadget (a charting gadget). I can create a simple gadget that spits out a set of variables and then try to map that onto the chart gadget. I'll have to dig around the net a bit to see if I can come up with some good examples of the best way of mapping web services together.

I shall post my progress here!

Architecting Things Out

2008-11-24T07:19:00.000-08:00

I'm now going down the path of fleshing out the architecture for our project. We're working with the concept of "gadgets" coupled with a "platform" as the basis for our architecture.

A gadget is simply an object (in a conceptual sense, not a programmatic sense) which takes an input, processes that input and produces an output. The platform will coordinate the gadgets in such a way that you can string gadgets together in order to get a chain of information, or a work flow, that you like.

Those are the concepts, but how does that play out in an application?

Well, the first idea that comes to mind is a service oriented architecture. Here, the gadgets have two components: a webservice that takes the input, does the processing and then gives an output; a component that plugs into the platform, interfaces with the webservice and provides any UI. A good example of a gadget is one that takes input and provides a visualization dependent on the input.

The platform would ideally be a flexible piece of software that would allow you to "wire together" different gadgets into the work flow that you want, among other things. I'd like to take a page out of Spring's book and make the wiring together of gadgets by the platform be definable in XML. This would allow for dynamic wiring at run time which would mean you could build an "application" out of gadgets as you saw fit, ad hoc.

I like the idea of using webserivces because it provides such flexibility. It also means that you could create a gadget out of an existing web service (assuming that it was all right with you that you don't control webservice).

All in all, it's a rather web 2.0 concept that I think we're developing here, and I really like it. I'd like to get to a place where someone could wire together their own version of the application and then run it, change things and then rerun it to see if the output changed. Remember, this software is in support of research, so the ability to prototype a system is incredibly useful.

Hibernate Musings

2008-11-17T11:38:00.000-08:00

So, I'm working my way through this tutorial on hibernate. It basically takes you through creating a basic (and trivial) Hibernate project, all the way to the point where you create the database and put info into it, then take it out.

Wait, did I just say "create the database"? That I did. I think this is the coolest feature in Hibernate that I've come across so far. The work flow basically works like this: First, you create your POJO bean which will hold a record. Then, you create a mapping file which will map the properties of the bean onto a table in the database. Next, you set a special property in your hibernate.cfg.xml file (called hbm2ddl.auto) to "create". Finally, you create some utility classes that drive Hibernate.

Set up an ant task to do something (in my case, I stored a record) with the database and run it.

On start up Hibernate will create the database using the mapping files. The more mapping files you have, the more tables will be created. I still don't know if you can create multiple tables from one mapping, but the ability to create a database based off of POJO beans is some kind of powerful and is a feature that I will hopefully use to a great degree in order to get around some of my relative inexperience with database design.

Spring!

2008-11-06T06:06:00.000-08:00

I'm reading a book right now by Rod Johnson, called "J2EE Design and Development". It's a fabulous book which conveys all kinds of refinements on the architecture of a J2EE application.

In the book, Johnson puts forth his own novel framework for managing various aspects of the application. This framework became know as Spring and can be found here.

Of particular interest to me is the MVC framework that can be used to create a thin and clean front end for your web app. After reading a bit about this in the book I went and got Spring and ran through a tutorial that helps you create a non-trivial but simple front end for an inventory system. It's all included in the Spring download, if you're interested.

So, what are the highlights? First off, anything that can be configured in XML is configured in XML, allowing for dynamic shifting of classes at run-time! Further, one of the big Spring tenants is to program to interfaces (which is simply a sound principal and one I'm finding myself adopting quickly). These two things taken together means that Spring is incredibly non-invasive. You can use the Spring framework without having to write a bunch of code aimed at the Spring framework.

Let me walk you through a request flow to give you an example of how this works in the MVC framework.

To start off with, I have set up an index.jsp that redirects to hello.htm so that you can easily enter the application. I've set up a servlet-mapping in my web.xml which intercepts any and all .htm pages and redirects them to the Spring master controller.

So, enter the Spring master controller (which I use right out of the jar). The master controller will process the request, then hand off the request to a sub-controller which you define. All of this is configured in an XML file as well. I called my application springapp, so I created an XML file called springapp-servlet.xml.

I have a bean definition in springapp-servlet.xml for /hello.htm:

<bean name="/hello.htm" class="springapp.web.InventoryController">
<property name="productManager" ref="productManager"/>
</bean>

The master controller sees the class that is attached to /hello.htm and interprets this as the sub-controller which it should forward the request to. So, off the request goes.

My InventoryController does some logic processing and data gathering (for instance, it gathers a list of items to show in the inventory view) and puts all of this into a model which, in this instance, is a Map which contains a list of Products as well as a Date object. The InventoryController forwards the model and the name of a view back to the master controller and the master controller then forwards the model to the view specified which, in this case, is a jsp called hello.jsp.

The jsp page loops through the products list (which is model.products) and displays the products.

We also have a link on hello.jsp which will take us to the price increase page (called priceincrease.jsp). This page allows you to implement a price increase by percentage over all the products.

Here's where things really get cool. Spring ships with a tag library, called form. It adds on to some normal HTML tags, such as form or input. The specific design, though, is to allow you to map a form or input back to a specific class. So, to enter in the price increase, I use an input defined by the form tag library:

<form:input path="percentage"/>

I also define a form which this input sits in:

<form:form method="post" commandName="priceIncrease">

The command name maps back to a bean in the springapp-servlet.xml file. That bean has a controller class specified (PriceIncreaseFormController) with it as well as a name (/priceincrease.htm). You see how the pattern is forming up? In that controller, there is a mapping to several things. First off is a mapping which corresponds with the commandName. This is mapped to a class (called PriceIncrease in this instance). PriceIncrease has a field called percentage. So, when the form is submitted, a new object of class PriceIncrease is created with the value in the input attached to the field percentage. PriceIncrease is considered a command, as it provides direction on what the ProductManager should do with the Products.

Further, a validator is specified for PriceIncrease and is automatically run. All of this is then passed into the PriceIncreaseFormController. The controller evaluates the results of the validator, does any processing necessary, builds a model if necessary, and then forwards everything to a view (which is hello.jsp in this case).

So, that's the basic workflow. I think it's pretty awesome that so much can be specified in XML. That gives you an amazing amount of flexibility at run-time.

All in all, I think I'm going to benefit a lot from Spring. It has more than just an MVC component. It provides an abstraction for almost every level of J2EE applications, from the database level (a JDBC abstraction) to a replacement for the EJB tier.

I'm Pushing this on You

2008-10-28T07:08:00.001-07:00

So, one of the requirements we have is to be able to alter the content of a page given a users input, where that user is either another user entirely or the main user of the given session. The aim is to impart a collaborative aspect on a given page so that anyone on the page can view the collective changes that have been made to that page.

This has brought me around to thinking about AJAX in a different manner. Normally, AJAX is used to pull info from the server. The page (or component on the page) makes a request of the server, and the server sends the response to the page/component. However, what do you do when you need to reflect changes to your model in near real time?

Well, the thought that comes to my mind is that you move to a push model. In the push model, whenever the data model changes it pushes those changes out to the view. Now, this is totally doable with AJAX, it's just a different way of thinking about things. I think it's been termed reverse AJAX.

Of course, the poor man's way of doing a push model is to actually do a pull model with automated (instead of user-requested) pulls on a short interval. The downside to this is that it can result in a ton of network traffic at the very least. Another downside is that you don't have true data coherence. You're basically taking a guess as to when the data model has changed.

So, I'm now investigating ways in which I can use a push model. I've found two promising leads: CometD and Pushlets. I might still opt for the poor man's approach, at least for now, but both of these bear investigation. When I know more, I'll post it here.

If you're interested in a good article which outlines some of the ways of going about this, check out this article here.

J2EE Survey Software: Opinio

2008-10-27T06:00:00.000-07:00

A requirement came up recently wherein we need to be able to conduct online surveys. Thinking that this was an area where I'd best be served leveraging existing software, I started searching for Java based survey tools.

I quickly found that there is a severe dearth of Java survey tools in the open source community. Most projects haven't been updated since 2005 and all of them seem cumbersome. If I am going to use something, I'm going to make sure it is easy to use and also looks good.

So, abandoning open source for the moment, I started looking at what was available in the closed source world. I quickly found a piece of software that fit my needs perfectly. Called Opinio, it is a very nicely built J2EE application that can be deployed on anything from Tomcat to JBoss (even IIS, or so they claim).

It has 3 versions: Lite, Corporate and Enterprise. The Lite version is free for use, but has a much reduced feature set. However, the reduction is in the analysis side and since we're going to be building our own analysis tools, this application fit perfectly!

My only gripe so far is that it doesn't have a great community or a good set of docs. I'm currently trying to set it up so that it will write the surveys to our SQL Server database. There are directions on the site for setting up MySql and Oracle, but no SQL Server. Once I get the issue solved, though, I may post how to do it, here.

And the Winner Is...

2008-10-22T11:40:00.000-07:00

JBoss.

First and foremost, JBoss has the best community. Documentation exists for all facets of using JBoss, from developing on it to deploying it to a production environment. Most of these docs exist in Wiki form on the JBoss.org website, but a Google search of any term prefixed by JBoss (ie: JBoss Administration) will return many, many relevant links. This in and of itself is the primary factor for using JBoss.
While GlassFish provides the latest in JEE 5, JBoss is not far behind (actually having a release candidate with JEE 5 support available). Further, we will most likely not be harnessing a lot of the JEE 5 specification. What we will be using of the JEE 5 specification, mainly JSP and Servlet are provided, along with a JAX-WS compliant stack in the form of JBossWS.
JBoss has an Eclipse plugin (as does GlassFish).
JBoss’s system requirements are lower than GlassFish’s.
While GlassFish’s admin console is very nice and very polished (specifically it’s log viewer), the lack of being able to run in a console window is constrictive to development. From a developer productivity standpoint, having to open up a web page and refresh the view to get the latest log files would be detrimental.
Hibernate, the ORM solution we will be using, is a JBoss project, and is built directly into JBoss.

The choice came down to JBoss vs. GlassFish. I never really considered Tomcat with Metro.

While I really liked GlassFish, it didn't seem as suitable for a development environment. Both of them seemed completely competent in a production environment, with each having its best features. So, it came down to which would be better to develop on.

The answer to that question is that JBoss is easier to develop on and it has the most widely available community. Those two factors weighted things to JBoss.

The Purpose

2008-10-16T18:49:00.001-07:00

So, this is my developer's blog. It's the "in vogue" thing for developers to do. I see them all over the place, so why shouldn't I have one, too?

I've recently changed jobs and am working on a project now that is starting from the ground. I'm also the lone developer on the project. That kind of makes it my show, to a certain degree. I'm responsible to the Customer (who shall remain nameless) and to the Company (who shall also remain nameless) I work for.

Being in on a project from the ground up is a pretty cool thing. You get to make a lot of decisions that will have a huge impact on the project months, if not years, down the line. Some of the decisions I've already made have been which language I will use: Java; what type of application it will be: a web app.

A few decisions were made for me, including which database I'll be using, which is Microsoft SQL Server. That suits me fine, as I'm going to try to take a database agnostic approach by using Hibernate. We'll see how true to that desire I can stay later on down the road.

It does have the up-side, or so I've heard, in that SQL Server is supposedly easy to administer. It certainly seems easier to administer than Oracle, the only other database I've worked with. The tools seem to be a lot more developed and "user-friendly". That's good in my book, as I don't want to spend all my time adminstering the database.

The big decision I'm working on right now is what application server I'm going to be using. I've basically narrowed it down to three choices. The first is to use Tomcat augmented with the Metro web stack that is available from the GlassFish project. The second is to use the actual GlassFish server itself (or whatever lame name Sun gives it once it's released officially). The third is to use JBoss.

I'm researching all three right now and I plan on posting here what I find and my ultimate decision.

So, back to the purpose of this blog. I'd like to document the steps that I take as I build this application. I'd like to make note of the pitfalls I fall into and how I got out of them. I'd like for this to be part history, part road-map to how to do it better in the future.

We'll see how it goes from here.