What Should be Next for Enterprise Mashups?

My friend over at JackBe, Chris Warner, recently sent me an email asking me what the next 2 to 3 years should hold for Enterprise Mashups. JackBe, in case you don't know, is the creator of the Enterprise Mashup software Presto.  We've used Presto on a project and it's a wonderfully crafted piece of software with many, many possibilities in front of it.  Presto has, at its heart, the creation of mashups, those elusive but so useful beasts which give you all kinds of new-fangled views into your data.

This is a topic I've spent a little bit of time thinking about, fancying myself to be at least a passable user and creator of mashups.  I'm going to change up the question a bit, though, and delve into territory that is slightly afield.  So, I propose these related questions:  Just what the heck are Enterprise Mashups?  And, what's next for them?

Any Ol' Programmer
Let's be honest, any programmer can get out there and create a mashup with a little elbow grease.  The technology is there such that, with a little bit of web service lifting and a little bit of UI know-how you can create fantastical mashups that will make everyone sit up and take notice.  What's more, if you do it right, your own mashup can, in turn, become a data source for someone else to consume.  It's a bit like Sir Elton John says, that whole Circle of Life thing.

But, that's just the problem, isn't it?  We're not all programmers, now, are we?  Not all of us even have access to programmers (or, as we like to style ourselves now-a-days, "software engineers"), so it's not like we can ask someone else to do it for us.  And, most important of all, programmer time is expensive.  If you do have access to a programmer and task them with spending hours creating your pet mashup, you're likely to get a whack on the wrists from the company dumby stick.  So, what use are mashups to the common-day man?  They're nice to look at, but not really practical beyond that.

Enter the Enterprise
No, not the ship, though that would be cool.  The Enterprise is (or should be) all about putting a useful tool into the hands of someone other than an engineer.  To a certain extent it's about lowering the technical barrier of entry.  Any programmer should be able to put a mashup together, but that doesn't mean that they are easy.  There is a lot of wiring together and hacking things up that takes place.

Enter the folks at JackBe, handing us Presto.  With software like Presto it's possible for someone of a less technical bent, someone that's not a programmer, to put together a mashup that's useful.  This Enterprise tool takes some of the difficulty out of the creation.  And that, in a nutshell, is what the Enterprise should be about.

What's Next?
Even though products like Presto lower the barrier of entry, there is still a long way to go.  Presto shines at doing the actual mashing, wiring the data together to get the new view.  But, one area where Enterprise Mashup software needs to tread next is into more robust presentation of that new view.

One thing that mashups excel at is the creation of data.  One thing that data desperately wants to do is be visualized.  There are so many tools out there for visualizing data, but most of them have a similar dilemma to non-enterprise mashups:  it takes specialized skills to use them.  Enterprise Mashups need to tread here, next, lowering the barrier of entry into rich presentation of the data that mashups are so good at generating.

To Infinity...
From there, well, who knows.  I'd like to see tools which are good at creating not just a good presentation but actual interaction, all done so that the common layman can create them with minimal know-how.  I've blogged before about how it's important for people like analysts to be able to do at least some kind of rudimentary software development.  I see that Enterprise Mashups can fill this role once they come to full maturity.  It's a big area, one that needs more exploration, and I'm quite happy that I get a peek into it at the ground floor.

JavaScript: Callbacks in Loops

I just finished a mashup that had to be blogged about. I suffered to find this solution, and I wanted to share what I learned with the world.

In the mashup I took a twitter feed and plotted the tweets onto a map based on the location of the tweeter. Let me set the stage.

The Google Map has already been set up and the list of tweets has been obtained. It is now time to plot the tweets onto the map. This will be done within a function called addMarkers. The HTTP Geocoder that Google provides will be doing our geocoding. For more information on this service, see this.

Keep in mind that I'm doing all of this in a Presto Mashlet, and will be calling out to the HTTP Geocoder via a URLProxy call that is undocumented but available for use.

At first blush, the following approach seems appropriate. Here is an excerpt from the addMarkers function:



However, this suffers from a very serious drawback, and that drawback revolves around the scope of the function as it exists on the stack. Remember that you are calling out and receiving an asynchronous response via the callback. There's no telling where this loop will be when a callback returns, but the scope of the function is maintained on the stack until all of the callbacks have been completed.

When a callback returns, the current value of i will be used to index into tweets! Since all of these calls take time, the most common result is that i will actually be out of bounds of tweets. Recall that updating the loop variable is the last operation done in any JavaScript for loop. Once you have looped through all of your indexes you, of necessity, must set i to be out of bounds of tweets. Therefor, i will be equal with tweets.length.

The result is that you pass an undefined object into placeMarker in place of what should have been the tweet.

The next logical step is that you should create a variable to hold the value of i, like this:

var myTweet = i;
...
this.placeMarker(point, tweets[myTweet]);

However, this will fail as well!

The problem here is that myTweet is still within the scope of our addMarkers function. addMarkers will therefor have only one copy of myTweet. Once again, you end up in a situation where the loop will probably finish before any of the callbacks return. The net result this time, however, is slightly different. You will pass in a valid tweet to placeMarkers, but it will be the last tweet in every instance. You'll have the same tweet attached to all of your markers on the map, the last tweet in the list.

So, how do you remove the timing issues? This is where I suffered. I hunted and pecked out half-solutions for quite a while. Finally, I had to start thinking outside of the normal box to come up with a solution.

The whole problem revolves around all of the callbacks returning to a shared scope in the stack, that being the scope of addMarkers. Once you consider it that way, it becomes obvious that providing each callback with its own scope on the stack is what is needed. The way to do that is to have a function fire off the HTTP Geocoder request. The function will get its own spot on the stack and will have its own scope. Let addMarkers maintain the loop and call this function whenever it wants to fire off a request. Pass in the tweets and the desired value of i to be remembered.

Consider the following:

This approach will result in the correct tweet being displayed with the correct marker on the map.

How Does the Magician Get What He Wants?

As I started developing the core of the "platform" I realized that I had undertaken a huge task, a task I was not sure I could finish in any short amount of time. This caused me some angst but I figured that I would just have to move on, doing my best to develop my way out of an impossible situation. Not a fun position to be in.

Then, sometime around the end of December, a buddy of mine pointed me in the direction of a product called Presto, developed by the company JackBe. I thought that I had stumbled into my own brain on the web!

Keep in mind that the concept I was working on was that the platform would map gadgets together in such a way that we could dynamically generate them with ease, rapidly prototyping processes to see how they worked. In effect, the gadgets would be web services and the thing the platform would create would be a mashup.

As I said, I had envisioned something similar to PopFly, the ability to visually create mashups and then do something meaningful with them in our efforts to prototype processes. Presto is this and so much more.

First and foremost, Presto has a beautiful visual mashup maker called Wires. In Wires, you drag services from a palette onto your canvas and connect them. You can also drag in actions, blocks that allow you to do something with the results of other services. So, in the basic example I worked with, you have two RSS feeds. You can take the data from each feed and "merge" them (where merge is an action block) so that you effectively have one RSS feed from two. Then, you can add a filter (another action) which can take a dynamic input. In this way, you can create very complex mashups from some very basic building blocks.

If the complexity of Wires is not enough for you, however, they have a markup language for creating mashups called EMML. What you do in Wires is distilled down to EMML, but Wires doesn't offer the full capabilities of EMML (which is not to say that Wires is not fully featured, there are just some rather tricky things you can do with EMML that you can't do with Wires as they don't have a visual representation).

What's even cooler, and something that spoke to a need we had, is that each mashup is published as a service itself. So, you can include mashups in other mashups! The modularity is great.

But, you might be asking, how do you get the services in so that you can use them from the palette?

Well, Presto has a "Service Explorer" which allows you to import services from wherever they may be and "publish" them in the Presto server. However, there's a little more to it than that. Presto comes complete with a user authentication system that can be standalone or hook into LDAP or AD. When I import a service, I can assign rights to it, and only those in the appropriate groups can even see the service, or any mashup in which the service exists. You can also assign rights to mashups themselves.

So, that gave us the mashing capability that we've been looking for, but the goodies in this bag didn't end there!

What we have up to this point is a unique view of the data, but no visualization of the data. Enter the mashlet!

A mashlet is a view of the mashup in a portable and embeddable package. It is created in JavaScript. Once you've created a mashup, you can attach a mashlet to it so that others can see what the mashup provides. There are 5 prebuilt mashlet types: RSS; grid; chart; Yahoo Map; and XML. If your data fits into any of these predefined views, creating a mashlet is as simple as selecting the mashup or service, selecting the view, then publishing it. If you need a more complex view of your data you can create a mashlet by hand. The process for creating a mashlet by hand is rather well thought out and not all that hard to grasp.

Once a mashlet is published you can do one of several things with it.

The mashlets are served up from the Presto server in much the same way that any JavaScript object is. Right out of the gate, you can view a mashlet standalone, if you so choose. However, the real fun comes when you realize that you can embed mashlets into any HTML page you wish by simply including a script tag. You can also embed them in a MediaWiki, NetVibes or as a GoogleGadget. You can't ask for more flexibility. From what I understand, there are more embeddable objects coming, including things such as JSR-168 portlets.

So, from our perspective, Presto offers us 3 incredible capabilities: the ability to capture services from across the web; the ability to create mashups in an easy and visual manner; and the capability to add a face to the services and mashups we create, then embed that face wherever we need.

We've committed to Presto as the core of our platform and it puts us months, if not years, ahead of where we were.