Gov 2.0 Camp: Virtual Worlds

Virtual Worlds is a topic that I find popping up more and more. I've always taken it with a grain of salt, though, as most of the time I hear it in relation to talk about how it'll make everything better. I shy away from that kind of talk as I know that silver bullets don't exist.

Further, my only real experience with a virtual world stems from the time that I played World of Warcraft. So, in my mind there's extra baggage attached in that I have a hard time seeing how a virtual world could be more than a game. When I entered this session, I decided to try to leave behind my baggage or, at the very least, dispel some of it.

This session was given by members of the Federal Consortium for Virtual Worlds. There were four panelists, but I did not, unfortunately, get their names or contacts.

As I entered this session, I had one question in mind: Are virtual worlds useful for more than just playing?

Surprisingly, there are three government agencies which are using virtual worlds in some capacity: NOAA, NASA and the CDC. All three of these agencies use virtual worlds for information delivery and training. NASA may be the least shocking example here, though, as it makes sense for them to create, say, a virtual world of Mars and then use that virtual world to train rover drivers. It's NOAA that has the most fascinating use of virtual worlds.

NOAA Islands (see towards the bottom of the page under the heading "NOAA Virtual World") is a virtual world that runs in Second Life (a very popular virtual world). In NOAA Islands, one is allowed to create their own mini-planet and then strive to create a stable weather pattern on that planet. As you add one effect, though, its repercussions are seen in other parts of the mini-planet. If you add too much rain, well, you'll flood the crops that are growing in a valley or low-lying area. If you make the world too hot, you'll melt the polar ice caps. In this way, NOAA attempts to convey the intricacies of climate to the uninitiated.

Another example of using virtual worlds in unique and non-playful ways was anecdotally related by one of the panelists. He commented on a school of engineering that he knew of that was using virtual worlds in order to prototype the buildings and structures they were designing. Specifically of use was the ability to determine if handicap access was sufficient for a given building.

However, of all the examples given, the most common example given was that of collaboration. The idea was set forth that in today's world of budget cuts and massive organizational structures which can span a country if not the globe money could be saved if, instead of collaborating face-to-face, people could collaborate virtually.

To this, I asked the question of what advantages do a virtual world provide that more traditional forms of telecommunication don't? The response was both intangible and interesting.

Basically, the consensus from the panel was that virtual worlds provide a sort of solidification of knowledge based on their immersiveness. One presenter described it as "informational bandwidth". Virtual worlds, being immersive, allow one to convey large amounts of information faster. Further, this immersiveness adds context to the memories created, making the information conveyed more "solid" or "real". The information has a better chance of sticking due to the immersive nature of a virtual world.

This concept, that the immersiveness of a virtual world added to the quality of the information that was transmitted seemed to find fertile ground in the audience. One audience member stated that "where it is is what it is", meaning that the memory of something can be tied in no small part to the place where the memory was experienced.

However, someone else asked a very probing question: Do virtual worlds limit or enhance productivity?

The answer to this was less than satisfactory, in my mind: The worlds are getting better with productivity software. Currently, many virtual worlds allow for desktop sharing and persistence of environment. But, does "getting better" mean "good enough"? In my mind, no.

So, how did this session shape my opinion on virtual worlds? Are virtual worlds useful for more than just play?

Yes, they are useful for more than just play.

First and foremost, I was highly impressed by NOAA's forward thinking in this space. So many times allowing people to just get out there and attempt something is the best way to convince them of your point. Allowing people to experiment with climate by actually creating it is ingenious and something that NOAA should be commended for.

Further, I can see how virtual worlds will soon play a huge role in training. Being able to attempt a task in a similar environment to the environment in which you will actually be performing the task is of great value. I can see how this could lead to safer working conditions in hazardous work environments, from military applications to industry.

I can also see that the advantage to prototyping is huge. Being able to put yourself into a users shoes (as in the case of testing handicap access in a building which is to be built) is of immeasurable value. If you add realistic physics to that, well, you've simply added even more value to the tool.

However, the one place I remain unconvinced is in what is probably the most important space of all (as far as widespread adoption of virtual worlds is concerned). For virtual worlds to be adopted whole-heartedly across government and industry they must facilitate the work that people do. To hear that the tools for productivity are still developing in my mind means that they are not ready yet for the main-stream.

That's not to say that virtual worlds don't have potential value in this most important space, though. Indeed, I feel that they have great potential. But, until I can see an example of where they make collaboration as easy, or near as easy, as actual collaboration is in real life I don't see that they will be widely adopted for this purpose.

With all that said, I'm going to keep my ears out, and my mind open, to this topic. I think that there is great work that can be done here and I look forward to what the researchers of today will do with this technology.

How Does the Magician Get What He Wants?

As I started developing the core of the "platform" I realized that I had undertaken a huge task, a task I was not sure I could finish in any short amount of time. This caused me some angst but I figured that I would just have to move on, doing my best to develop my way out of an impossible situation. Not a fun position to be in.

Then, sometime around the end of December, a buddy of mine pointed me in the direction of a product called Presto, developed by the company JackBe. I thought that I had stumbled into my own brain on the web!

Keep in mind that the concept I was working on was that the platform would map gadgets together in such a way that we could dynamically generate them with ease, rapidly prototyping processes to see how they worked. In effect, the gadgets would be web services and the thing the platform would create would be a mashup.

As I said, I had envisioned something similar to PopFly, the ability to visually create mashups and then do something meaningful with them in our efforts to prototype processes. Presto is this and so much more.

First and foremost, Presto has a beautiful visual mashup maker called Wires. In Wires, you drag services from a palette onto your canvas and connect them. You can also drag in actions, blocks that allow you to do something with the results of other services. So, in the basic example I worked with, you have two RSS feeds. You can take the data from each feed and "merge" them (where merge is an action block) so that you effectively have one RSS feed from two. Then, you can add a filter (another action) which can take a dynamic input. In this way, you can create very complex mashups from some very basic building blocks.

If the complexity of Wires is not enough for you, however, they have a markup language for creating mashups called EMML. What you do in Wires is distilled down to EMML, but Wires doesn't offer the full capabilities of EMML (which is not to say that Wires is not fully featured, there are just some rather tricky things you can do with EMML that you can't do with Wires as they don't have a visual representation).

What's even cooler, and something that spoke to a need we had, is that each mashup is published as a service itself. So, you can include mashups in other mashups! The modularity is great.

But, you might be asking, how do you get the services in so that you can use them from the palette?

Well, Presto has a "Service Explorer" which allows you to import services from wherever they may be and "publish" them in the Presto server. However, there's a little more to it than that. Presto comes complete with a user authentication system that can be standalone or hook into LDAP or AD. When I import a service, I can assign rights to it, and only those in the appropriate groups can even see the service, or any mashup in which the service exists. You can also assign rights to mashups themselves.

So, that gave us the mashing capability that we've been looking for, but the goodies in this bag didn't end there!

What we have up to this point is a unique view of the data, but no visualization of the data. Enter the mashlet!

A mashlet is a view of the mashup in a portable and embeddable package. It is created in JavaScript. Once you've created a mashup, you can attach a mashlet to it so that others can see what the mashup provides. There are 5 prebuilt mashlet types: RSS; grid; chart; Yahoo Map; and XML. If your data fits into any of these predefined views, creating a mashlet is as simple as selecting the mashup or service, selecting the view, then publishing it. If you need a more complex view of your data you can create a mashlet by hand. The process for creating a mashlet by hand is rather well thought out and not all that hard to grasp.

Once a mashlet is published you can do one of several things with it.

The mashlets are served up from the Presto server in much the same way that any JavaScript object is. Right out of the gate, you can view a mashlet standalone, if you so choose. However, the real fun comes when you realize that you can embed mashlets into any HTML page you wish by simply including a script tag. You can also embed them in a MediaWiki, NetVibes or as a GoogleGadget. You can't ask for more flexibility. From what I understand, there are more embeddable objects coming, including things such as JSR-168 portlets.

So, from our perspective, Presto offers us 3 incredible capabilities: the ability to capture services from across the web; the ability to create mashups in an easy and visual manner; and the capability to add a face to the services and mashups we create, then embed that face wherever we need.

We've committed to Presto as the core of our platform and it puts us months, if not years, ahead of where we were.

We Need a Map to Talk to Each Other

In a conversation I was having today the question about how two gadgets would communicate with each other came up. Initially, I was thinking that two gadgets would talk through an intermediary gadget which would convert the output of one gadget into the input for the second gadget.

There are a couple major problems with this approach:

1. Anytime you added a gadget you would possibly have to write a whole slew of interpreter gadgets for the gadgets you wanted to give input to or take input from. At a minimum, every gadget would require at least one interpreter gadget.
2. The whole gadget as a web service concept breaks down when you start talking about interpreter gadgets. They do not need to reside on the web, they need to reside on the platform. Should you have a platform based gadget? Is that really worthwhile or does it break the nice clean system we've conceptualized?

In the midst of the conversation the idea of being able to do mash-ups was discussed, and this spurred me into a new area of thought regarding how gadgets could talk. Mash-ups always make me think of PopFly. If you've never played with PopFly, I highly recommend it. It's one of the coolest things to come out of Microsoft since Surface.

In PopFly, you capture two (or more) web service endpoints and then you map the fields in the endpoints together. This allows you to create dynamic mash-ups on the fly, among other things. So, let's say that I have a web service that gives me a weather report for a list of locations and I have a mapping web service (like Google Earth). I can map the location info from the weather report list onto the mapping service and create a series of push-pins on the map that will show you the weather report when you mouse over them. It's pretty nifty and very simple to use.

Why not use the mapping between web services concept for our platform?

It's a much more elegant solution. It has several advantages over the other approach I mentioned:

1. A mapping is much easier to generate than a complete gadget. What's more, if done right, it could be done in a UI! PopFly has demonstrated this very well.
   
2. The platform would only have to remember a mapping (which could be done in XML quite easily) for any communication between gadgets. That eliminates the problem of having platform-bound gadgets.
3. The mappings could be based off of the WSDLs that the web services publish. This would allow a direct mapping from the output of one gadget to the input of another. No intermediary step is needed.

I have to say, I'm quite enamored of the idea. I think that the first step is going to be creating a set of gadgets that are supposed to talk. I've already created the first gadget (a charting gadget). I can create a simple gadget that spits out a set of variables and then try to map that onto the chart gadget. I'll have to dig around the net a bit to see if I can come up with some good examples of the best way of mapping web services together.

I shall post my progress here!

Architecting Things Out

I'm now going down the path of fleshing out the architecture for our project. We're working with the concept of "gadgets" coupled with a "platform" as the basis for our architecture.

A gadget is simply an object (in a conceptual sense, not a programmatic sense) which takes an input, processes that input and produces an output. The platform will coordinate the gadgets in such a way that you can string gadgets together in order to get a chain of information, or a work flow, that you like.

Those are the concepts, but how does that play out in an application?

Well, the first idea that comes to mind is a service oriented architecture. Here, the gadgets have two components: a webservice that takes the input, does the processing and then gives an output; a component that plugs into the platform, interfaces with the webservice and provides any UI. A good example of a gadget is one that takes input and provides a visualization dependent on the input.

The platform would ideally be a flexible piece of software that would allow you to "wire together" different gadgets into the work flow that you want, among other things. I'd like to take a page out of Spring's book and make the wiring together of gadgets by the platform be definable in XML. This would allow for dynamic wiring at run time which would mean you could build an "application" out of gadgets as you saw fit, ad hoc.

I like the idea of using webserivces because it provides such flexibility. It also means that you could create a gadget out of an existing web service (assuming that it was all right with you that you don't control webservice).

All in all, it's a rather web 2.0 concept that I think we're developing here, and I really like it. I'd like to get to a place where someone could wire together their own version of the application and then run it, change things and then rerun it to see if the output changed. Remember, this software is in support of research, so the ability to prototype a system is incredibly useful.

Hibernate Musings

So, I'm working my way through this tutorial on hibernate. It basically takes you through creating a basic (and trivial) Hibernate project, all the way to the point where you create the database and put info into it, then take it out.

Wait, did I just say "create the database"? That I did. I think this is the coolest feature in Hibernate that I've come across so far. The work flow basically works like this: First, you create your POJO bean which will hold a record. Then, you create a mapping file which will map the properties of the bean onto a table in the database. Next, you set a special property in your hibernate.cfg.xml file (called hbm2ddl.auto) to "create". Finally, you create some utility classes that drive Hibernate.

Set up an ant task to do something (in my case, I stored a record) with the database and run it.

On start up Hibernate will create the database using the mapping files. The more mapping files you have, the more tables will be created. I still don't know if you can create multiple tables from one mapping, but the ability to create a database based off of POJO beans is some kind of powerful and is a feature that I will hopefully use to a great degree in order to get around some of my relative inexperience with database design.

Spring!

I'm reading a book right now by Rod Johnson, called "J2EE Design and Development". It's a fabulous book which conveys all kinds of refinements on the architecture of a J2EE application.

In the book, Johnson puts forth his own novel framework for managing various aspects of the application. This framework became know as Spring and can be found here.

Of particular interest to me is the MVC framework that can be used to create a thin and clean front end for your web app. After reading a bit about this in the book I went and got Spring and ran through a tutorial that helps you create a non-trivial but simple front end for an inventory system. It's all included in the Spring download, if you're interested.

So, what are the highlights? First off, anything that can be configured in XML is configured in XML, allowing for dynamic shifting of classes at run-time! Further, one of the big Spring tenants is to program to interfaces (which is simply a sound principal and one I'm finding myself adopting quickly). These two things taken together means that Spring is incredibly non-invasive. You can use the Spring framework without having to write a bunch of code aimed at the Spring framework.

Let me walk you through a request flow to give you an example of how this works in the MVC framework.

To start off with, I have set up an index.jsp that redirects to hello.htm so that you can easily enter the application. I've set up a servlet-mapping in my web.xml which intercepts any and all .htm pages and redirects them to the Spring master controller.

So, enter the Spring master controller (which I use right out of the jar). The master controller will process the request, then hand off the request to a sub-controller which you define. All of this is configured in an XML file as well. I called my application springapp, so I created an XML file called springapp-servlet.xml.

I have a bean definition in springapp-servlet.xml for /hello.htm:

<bean name="/hello.htm" class="springapp.web.InventoryController">
<property name="productManager" ref="productManager"/>
</bean>

The master controller sees the class that is attached to /hello.htm and interprets this as the sub-controller which it should forward the request to. So, off the request goes.

My InventoryController does some logic processing and data gathering (for instance, it gathers a list of items to show in the inventory view) and puts all of this into a model which, in this instance, is a Map which contains a list of Products as well as a Date object. The InventoryController forwards the model and the name of a view back to the master controller and the master controller then forwards the model to the view specified which, in this case, is a jsp called hello.jsp.

The jsp page loops through the products list (which is model.products) and displays the products.

We also have a link on hello.jsp which will take us to the price increase page (called priceincrease.jsp). This page allows you to implement a price increase by percentage over all the products.

Here's where things really get cool. Spring ships with a tag library, called form. It adds on to some normal HTML tags, such as form or input. The specific design, though, is to allow you to map a form or input back to a specific class. So, to enter in the price increase, I use an input defined by the form tag library:

<form:input path="percentage"/>

I also define a form which this input sits in:

<form:form method="post" commandName="priceIncrease">

The command name maps back to a bean in the springapp-servlet.xml file. That bean has a controller class specified (PriceIncreaseFormController) with it as well as a name (/priceincrease.htm). You see how the pattern is forming up? In that controller, there is a mapping to several things. First off is a mapping which corresponds with the commandName. This is mapped to a class (called PriceIncrease in this instance). PriceIncrease has a field called percentage. So, when the form is submitted, a new object of class PriceIncrease is created with the value in the input attached to the field percentage. PriceIncrease is considered a command, as it provides direction on what the ProductManager should do with the Products.

Further, a validator is specified for PriceIncrease and is automatically run. All of this is then passed into the PriceIncreaseFormController. The controller evaluates the results of the validator, does any processing necessary, builds a model if necessary, and then forwards everything to a view (which is hello.jsp in this case).

So, that's the basic workflow. I think it's pretty awesome that so much can be specified in XML. That gives you an amazing amount of flexibility at run-time.

All in all, I think I'm going to benefit a lot from Spring. It has more than just an MVC component. It provides an abstraction for almost every level of J2EE applications, from the database level (a JDBC abstraction) to a replacement for the EJB tier.

I'm Pushing this on You

So, one of the requirements we have is to be able to alter the content of a page given a users input, where that user is either another user entirely or the main user of the given session. The aim is to impart a collaborative aspect on a given page so that anyone on the page can view the collective changes that have been made to that page.

This has brought me around to thinking about AJAX in a different manner. Normally, AJAX is used to pull info from the server. The page (or component on the page) makes a request of the server, and the server sends the response to the page/component. However, what do you do when you need to reflect changes to your model in near real time?

Well, the thought that comes to my mind is that you move to a push model. In the push model, whenever the data model changes it pushes those changes out to the view. Now, this is totally doable with AJAX, it's just a different way of thinking about things. I think it's been termed reverse AJAX.

Of course, the poor man's way of doing a push model is to actually do a pull model with automated (instead of user-requested) pulls on a short interval. The downside to this is that it can result in a ton of network traffic at the very least. Another downside is that you don't have true data coherence. You're basically taking a guess as to when the data model has changed.

So, I'm now investigating ways in which I can use a push model. I've found two promising leads: CometD and Pushlets. I might still opt for the poor man's approach, at least for now, but both of these bear investigation. When I know more, I'll post it here.

If you're interested in a good article which outlines some of the ways of going about this, check out this article here.