Sim-OnDemand application : Opensim as data visualization tool

What is the point of a personal virtual world(Opensim) server like the Sim-OnDemand and what is the point of running it off Amazon Infrastructure?

One interesting answer is Visualization.

Well, here is a screen shot that illustrates visualization of ListMania related books of a given book.

Slicing and dicing data visually and collaboratively is one application for a personal virtual world. And Opensim with little python code can be very handy in that front.

DevPay, AWS learning: End User Key Safety

As I mentioned in earlier post, I’m experimenting with AWS DevPay. In this post I’m going to share about a small trick I employed to ensure safety of end user’s API( secret) keys.

AWS uses a public-key/secret-key shared secret mechanism to authenticate each API call to AWS. Since most calls to AWS are metered and billed to the key owner, the keys are same as money. Hence it is a good idea to address the key protection issue at design time.

In the Sim-OnDemand application, a few AWS api calls are to be made to initialize and attach/gracefully detach the Elastic Block Store volume. My initial design was to pass on the keys as User Data to LaunchInstances API call, and use the keys to make AWS calls to heart’s content.  There is one problem though, the keys will be available for anyone that has access to the running instance. The availability of keys as plain text will be enough incentive for attackers to break into the system. There is also the risk of the next packager or the AMI not taking enough safety measures and thereby inadvertently increasing the key safety risk.

The present implementation of Sim-OnDemand’s new-improved design is to eliminate the key passing to the instance all-together. A launcher application to compliment the AMI is developed and it does the API calls on behalf of the EC2 instance. The launcher application runs locally on the AMI user’s computer, thus keys are not exposed to the big bad world.

When the instance needs to make an API call, a pre-signed REST call url is computed by the launcher application and passed on to the instance rather than passing the keys.

I’ve made a DevPay forum post sharing this.

If you are keen to look at the code it is opensourced and available at google code.

This approach is not too generic. What is needed is an API, protocol to pass such information/control instructions to the instance. Looks like an interesting challenge to pursue.

Sim-OnDemand: OpenSim on EC2- Personal Virtual World Server

In the last post I mentioned that Sim-OnDemand is a learning by doing initiative in DevPay context. Let me elaborate on Sim-OnDemand a bit. The intended purpose of Sim-OnDemand is as a personal Sim for experimentation, development and testing.

Sim-OnDemand: Personal 3D Virtual World On the Cloud

3D virtual worlds are maturing fast and businesses and organizations are waking up to its potential. Collaboration, Communication, and Prototyping are some of the directions explored. OpenSimulator a rapidly evolving SecondLife(tm) compatible 3D virtual world server has reduced barriers of adoption considerably. The idea of Sim-OnDemand is to reduce the barriers of adoption even further. Sim-OnDemand makes the OpenSim software come alive using the Amazon’s Elastic Computing Cloud(EC2). The synergy of cloud computing with the bandwidth, computing intensive Virtual world software in my opinion are made for each other, just like the Windows-Intel combination that powered the last wave of computing advances.

The emergent nature of 3D virtual world applications underscores the need for personal servers for experimentation, development and testing. The present version of Sim-OnDemand caters exactly to this segment.
Sim-OnDemand is targeted towards non-geek 3D virtual world enthusiasts( like SecondLife power users) that have no inclination or incentive to learn the intricate details of AWS EC2 API just to get the OpenSim working.

Sim-OnDemand Implementation – The AMI

The core of Sim-OnDemand is OpenSim a mono based application. OpenSim supports multiple database backend, viz., SQLite, MySQL, and MS-SQL. Sim-OnDemand is configured such that OpenSim works with MySQL backend. Installing and configuring all the prerequisite software( like mono, mysql, screen etc) and packaging it all as an Amazon Machine Image(AMI) is the first step in the Sim-OnDemand implementation.

In order to make the MySQL database persist, Elastic Block Store(EBS) is used. Custom scripts were developed to initialize the EBS volume ahead of MySQL and OpenSim starts using the volume for storage. Similarly, shutdown scripts are developed to shutdown the OpenSim, and gracefully detach the volume.

Startup parameters are passed on to the EC2 instance via user data run-instances parameter.

Sim-OnDemand Implementation – Launcher Application

The intended audience for Sim-OnDemand is non-geeks with no intent or incentive to learn EC2, I thought it is a good idea to make a launcher for the AMI. With AWS there is no simple and safe method to pass or embed AWS credentials into EC2 AMI itself. A launcher application is one way of shifting the credentials away from the AMI.

The Sim-OnDemand launcher is a browser based javascript application. Much of the ec2 calling code is culled out of ElasticFox(ec2ui ) opensource code base. Since ec2ui codebase depends heavily on Mozilla API, and structured as a firefox extension, SimOn-Demand Launcher is also a run-down firefox extension!
The idea behind Sim-OnDemand launcher user interaction design is to keep it minimal and simple. Music player metaphor of Play-Pause-Play is employed to effect the launching, stopping and restarting of the OpenSim ec2 instance.

Sim-OnDemand – Future Features

Sim-OnDemand in its present form is just a proof of the concept. It is a way to validate if there is enough interest in running OpenSimulator on EC2 on an On-Demand basis. Few more features are essential to make it useful. Here is my take on the list. Users and potential users are most welcome to add to the list.

• User Management
• Multiple regions
• Grid Mode
• Ability to attach to to public Sims
• Previewing the latest OpenSim software – hot off the source code repository.
• Ability to run custom applications
• Automatic scaling with load, regions, number of users logged in.
• Integration of OpenSim forge community contributions.

I’ll be blogging about geeky details of the implementation soon. Meanwhile please try out Sim-OnDemand and suggest improvements and features.

AWS DevPay: Learning by Doing – Making Money With OpenSource

AWS DevPay is one of lesser publicized and presumably under utilized service by Amazon Web Services. A Platform + Community play becomes an growing, thriving ecosystem when barriers of making money off the ecosystem are reduced. DevPay makes it possible to add value to AWS offerings viz., Compute on Demand(EC2), Storage on Demand(S3) and sell licenses of the value addition to world at large.

Metering Muck

One of the difficulties in offering Software as a Service(SaaS) is keeping track of the utilization and usage linked billing. AWS License service aka DevPay takes the pain off the utilization metering. Utilization records can easily get mucky and difficult to ‘get it right’. Even if ‘getting it right’ is figured out, earning the reputation that metering is indeed accurate all the time is not easy. DevPay takes away the worries about utilization records, keeps track of it all on SaaS provider’s behalf. What is the fun in just metering if it cannot be converted into cash? Welcome Billing Muck….

Billing Muck

Once you have got past the metering muck, next step is charging for the service. When it comes to billing, there is no escape from credit card information maintenance. Or Making the service user agree to billing terms. All that needs to be in-place before the SaaS provider can calculate the dues and charge it on the payment instrument. Because money is involved, billing could soak up considerable attention.
DevPay to the rescue again – DevPay not just takes care of the metering, it will take care of the invoicing and billing too! Here what DevPay brings to the table is of enormous value: Amazon’s Reputation and Trust. Think about it, even if you get all the technical pieces of payment processing right, let us face it, people will think twice before they start trusting you with their credit card.
In short, DevPay’s promise is simple and compelling: Amazon will handle the Metering and Billing muck and expose it to SaaS provider via API! DevPay fee is utility priced like other AWS offerings.

Sim-OnDemand

Now, even if the above sounds like AWS Kool-aid talking, I’m sure I heave established that DevPay is worth exploring and learning. In order to learn how DevPay actually works out in real-life, I’m assembling a tiny and simple SaaS offering and making it available via DevPay. It is called Sim-OnDemand, it is a packaging of rapidly evolving 3D virtual world server software – OpenSimulator – as an Amazon Machine Image(AMI).

The idea is to share the learnings as it happens so that it benefits the SaaS, AWS community and in a small way influence the evolution of DevPay API too.

Bonus Links

“We Make Muck so You Dont Have To” – Jeff Bezos.

Emerging Uses of OpenSim by JustinCC

DevPay announcement in AWS Blog

Ugo Trade: Putting OpenSim Into The Heart of Web 2.0