A new build of Cobalt has just been made available download. Added functionality includes the ability to save and load spaces from web or local directories. This means that you can now make your custom Cobalt spaces available to others as a kind of template virtual world over the web!

I've just been awarded an NSF Small Grant for Exploratory Research (SGER) to advance exploratory work towards a more complete open source Croquet-based browser application and toolset that can support the large scale needs of the scientific community. With this support, we will be helping to advance the Cobalt effort by 1) making general improvements to the user interface which includes the more complete buildout of Cobalt's menus as well as improvements to basic navigational control, 2) fully implementing and testing of the ability for end-users to tag and electronically store their Cobalt-created worlds to online directories, 3) designing and implementing methods for finding and contacting other users of Croquet spaces by leveraging XMPP/Jabber as a presence registration and rendezvous mechanism, and 4) designing and implementing of methods that enable Cobalt users to browse a directory of all registered and active Cobalt spaces and to make it possible for users to contact current participants of those spaces by leveraging XMPP/Jabber and institutional IAA infrastructures as a means of defining user permissions by group affiliations. At the end of this, we hope to have a full Beta of the Cobalt application available for download to all.

Nicholas Negroponte just posted the following announcement:

"One Laptop per Child is announcing an agreement with Microsoft
to make a dual boot, Linux/Windows, version of the XO laptop. In addition, our intention is to engage one or more third parties to port Sugar to run on Windows in order to reach a wider installed base of laptops. In the meanwhile, OLPC remains fully committed to our goal: a completely free and open learning platform for the world's children. The mission statement of OLPC has not changed in three years.

Sugar is the first user interface specifically designed for children and teachers to learn and collaborate, and remains central to our strategy. Broadening Sugar's reach to as many children as possible remains key to OLPC's mission.

To enable the Sugar environment to reach as many children as possible, particularly in the poorest areas of the world, OLPC must be able to bid on educational technology contracts, some of which require that Microsoft Windows be able to run on our hardware. The increased volumes will lower the XO-1's price, already lowest in the industry with capabilities no other laptop shares.

OLPC is substantially increasing its engineering resources and all software development continues entirely on GNU/Linux. We will continue to work to make Sugar on Linux the best possible platform for education and to invest in our expanding Linux deployments in Peru, Uruguay, Mexico and elsewhere.

No OLPC resources are going to porting Sugar to Microsoft Windows, although as a free software project, we encourage others to do so. The Sugar user interface is already available for Fedora, Debian and Ubuntu Linux distributions, greatly broadening Sugar's reach to the millions of existing Linux systems. We continue to solicit help from the free software community in these efforts. Additionally, the Fedora, Debian and Ubuntu software environments run on the XO-1, adding support for tens of thousands of free software applications.

Open Firmware V2, the free and open source BIOS, is now capable of running Linux, Microsoft Windows XP and other operating systems, and was developed by Firmworks with support from OLPC. This will enable dual boot of OLPC XO laptops with Microsoft Windows XP in addition to the existing Fedora-based system and will become the standard BIOS/bootloader for all XO systems when completed. With this "free BIOS," the XO-1 continues to be the most open laptop hardware currently
available."

For more information, see http://wiki.laptop.org/go/AnnounceFAQ.

It has emerged as a type of practice here in the United States for people to begin putting up holiday decorations on the weekend after our annual Thanksgiving celebration. In keeping with this, I offer this link to a new video from the folks at EduSim. In this one, they use a projected Croquet space in combination with a very compact eBeam input device that converts any surface (in this case a regular wall) into an interactive surface. The result is an on demand and relatively low cost interactive white board solution ($800-900 US) that, in combination with a data projector ($600-2,500 US) and Croquet software (free), may represent the beginnings of an economical alternative to traditional in-classroom visual communication boards (the typical front-of-the-classroom whiteboard/bulletin board installation is comparable in price). Could this be the earliest manifestation of a new form of broadly available classroom media for K-12?

Matthew Chadwick has been exploring the creation of GPU-computed chaotic attractors of dynamical systems using the Croquet SDK. The code he is using is based on David Faught's procedural textures code. In this implementation, the cube to the left defines a parameter space for controlling the system. Matthew says that the same code he is developing could also be used for other things like in-world physics simulations. Matthew will make all of this available for public release once the code is ready.

Better avatars for Cobalt are almost ready! Here is a video showing an early test implementation of a motion-cycle enhanced avatar in Cobalt. Peter Moore has been doing a wonderful job with this and in getting the Ogre3D XML importer to work with models developed in Maya or exported from Blender. That means that we will all soon have at least two art paths for getting avatars and other animated meshes into Cobalt worlds. It'll also be great to finally retire the stilted Alice and Rabbit avatars. This particular video also provides a sneak peak at some of the texture and environmental/directional lighting work that I hope to have make its way into the next update of the Cobalt code-base.

This video illustrates what real life might be like if we were limited to the avatar capabilities/interactions presently available in virtual worlds such as Second Life. It very much underscores the limitations in the ways we are able to represent ourselves within today's 3D gaming and chat environments. If we are going to leverage virtual environments to support interactions between people, then we need far better ways of representing ourselves within them. Our representations should ideally be able to project as fully as possible the broadest range of human cues and capabilities. Clearly, 3D virtual worlds have a long way to go in this regard. As a first step, we need to get past the dress-up doll house metaphor that appears to have emerged for interaction within these environments.

The static avatars presently made available as placeholders in the Croquet SDK are far less capable than those of Second Life. Still, the Croquet SDK offers developers an opportunity to change whatever they need about the way people are represented with virtual environments. Opportunities for avatar experimentation are huge. Just imagine avatars that contain action triggers, link buttons, or even multiple on-board virtual environments. The possibilities through Croquet are as limitless as the imaginations brought to bear on the problem (and of course the resources expended in implementing them). The flexibility and efficiency of the Croquet programming environment gives researchers and other creatives far more capability in exploring how best to represent presence in virtual environments than is available with today's commercial 3D world technologies.

I should point out that all of the .mdl avatars that the Croquet SDK now uses actually came from an early version of the Alice project and from Squeak's Wonderland. However, the Croquet team at the University of Minnesota is working on some nicer avatars that will likely be made available in the next version of the Croquet SDK. A preview of the Minnesota avatars can be seen briefly on the Croquet video in a previous post. Also, Matt Schmidt and his team in Missouri are also beginning to experiment with avatar improvements and it will be interesting to see what they come up with.

Rajeev Lochan has just been successful in getting VNC to work within a shared Cobalt space! VNC is a graphical desktop sharing system which uses the RFB protocol to remotely control another computer. This is a big breakthrough for our open project. It means that a Cobalt-based VNC client can connect to a VNC server on any other operating system. Cobalt users will soon be able to view and interact with remote applications (including full featured web browsers) or even collaboratively access remote desktops within the Cobalt application. Because the VNC protocol can use a lot of bandwidth, we still have some optimization to deal with - but this progress is great to see. Thank you Rajeev!

Here is another video tutorial developed by Rich White at the Greenbush EduSim project showing how to bring resources into an EduSim/Croquet world and how apply textures to those resources.

Cobalt is an emerging open source and multi-platform metaverse browser and toolkit application being built using the open source Croquet SDK. A pre-alpha build of the Cobalt application is being made freely available to the emerging virtual worlds community by Duke University and its partners under the Croquet license. We hope that by doing so, we will foster a viable community-based software development effort leading to open virtual world technologies supporting the needs of education and research.

The current build of Cobalt is located here.

The Monticello source code repository for Cobalt is located here.

The Cobalt-specific Mantis bug tracker is located here.

Software testing and bug reporting is a very important contribution to the effort. Doing so will help the Cobalt development community to identify areas in need of programming and re-engineering. Well-written bug reports can be an especially useful contribution by non-developers who are interested in advancing this effort during these early stages of Cobalt development. We hope that many of the community's software developers will consider taking on bug fixes.

Mark McCahill will be coordinating updates and code contributions to Cobalt to ensure compatibility with the base classes within the present Croquet SDK. In this way we can ensure that Cobalt development does not create a fork from the Croquet SDK. The goal here is to build a Croquet-based application that end-users can really use and then to contribute the application back to the Croquet Consortium for distribution as part of a future Croquet release. We hope that by making the pre-alpha available we can tap into the creative potential of the broader community as a way of advancing something that all of us can freely use to create deeply collaborative, greatly featured, and widely interlinked virtual environments on a very large scale. Lets do it!

Here is a clever video from Johnny Lee at Carnegie Mellon University showing how you can use a Nintendo Wii-mote and home-made infrared LED light pens to create a low cost multi-touch interactive whiteboard system.

I've moved my blog to another address and will no longer post here. Please go to julianlombardi.blogspot.com to see all my posts, both old and new.

-Julian

This past weekend, it was announced at The Boston Media-Grid Summit that the Immersive Education Initiative (see article on the initiative's goals) has selected Croquet as one of three official "next generation" immersive education platforms. The Immersive Education Initiative is an international collaboration of universities, colleges, research institutes, consortia companies, and foundations that are working together to define and develop open standards, best practices, platforms, and communities of support for virtual reality and game-based learning and training systems. The other two immersive education platforms selected were Sun's open source Project Wonderland client and the now open source Second Life client.

What this all means is that the open source Croquet platform's value will become better known and that the Immersive Education Initiative will now direct both funding and programming resources towards the development and deployment of open source Croquet technologies and open source Croquet-based educational applications. Selection criteria for this important honor included the following: 1) support for the Windows and Macintosh operating systems; 2) availability as open source code; 3) vendor-neutral client and server architectures (no vendor lock-in); 4) stable and reliable runtime implementations; 5) integrated text chat and voice chat; 6) high resolution graphics; 7) multi-user support for collaboration; 8) highly customizable avatars that support high resolution graphics and body animation (gestures); and 9) support for user-created content.

The event in Boston was an invitation-only affair hosted and sponsored by the Grid Institute, the Woods College of Advancing Studies at Boston College, and the City of Boston with participation from the Federation of American Scientists and the Ewing Marion Kauffman Foundation.

Today, for the second time in as many weeks, Second Life became unavailable for the same reasons that 3rd party large-scale server-based architectures should not be relied on to support mission-critical virtual worlds in any serious way. The outage happened, once again, just at the time that Aaron Walsh of the Media Grid Immersive Education Initiative was trying to host a Second Life in-world meeting. Here is what Linden Lab had to say: Logins have temporarily been restricted to staff-only as Operations addresses a slowdown in the asset system. Weve also broadcast a request in world for residents who are currently logged in to refrain from manipulating or transferring assets. Well have more info ASAP. Obviously, this company is in real trouble given the frequency of these problems in a system that so many have come to rely on....

This new video by Willi Griephan of Bremen, Germany shows the operation of two Croquet simulation spaces that he created using the Croquet SDK. The first is a falling "Domino" simulation that leverages the Open Dynamics Engine for its physics. The second scene depicts swarm behavior in an aquarium simulation where two members of a swarm can pair, and by doing so, spawn new offspring. The Swarm algorithm that Willi used is described here by Craig W. Reynolds.

... so I thought I'd start a blog too.

The metaphor refers to one programming task leading to another. In this case, adding support for vertex buffer objects lead to writing a framework for managing OpenGL resource names/handles, and very nearly resulted in an all-out assault on the current content loading framework. Thanks be to Andreas Raab for advising against that (sometimes I can be hasty). But I digress...

What happened this week?

I finished the vertex buffer work alluded to in my last entry. Below, I'll discuss this in excruciating detail.

Work on Brie continues. Howard has been making progress with the coding, and we continue to discuss issues as they arise. A lot hinges on uncertainty about what TeaTime will actually look like. Now that I'm finished with the vertex buffer work, I'll be helping Howard with coding Brie. Fun!

So, tell me about those vertex buffers...

Such an interested audience!

I finished a prototype rendering path using vertex buffers shortly after my last blog posting. Preliminary results were very encouraging: for the previously mentioned large meshes, the frame rate jumped from approximately 5 fps to 35 fps. This encouraged me to implement this feature in a more robust fashion, suitable for general use in Croquet. This work had two major aspects: managing the low-level OpenGL names for the resources, and allowing the vertex buffer rendering path to be disabled when desired (currently, when the OpenGL implementation does not support vertex buffers, and whenever else desired).

In OpenGL, vertex buffers are named by an integer, as are many other types of resources such as textures and compiled shader programs. Recognizing the similarities between the way these resources are created, used, and destroyed, I designed a resource management framework that could support them all. I considered two major architectural approaches. The first was to generalize the functionality of OGLTextureManager, and the second was to create/destroy resource names within the scene graph objects as necessary (for example, this would involve modifying TTexture to directly hold an OpenGL name/handle, rather than referring to it indirectly via OGLTextureManager and OGLTextureHandle). The latter path was rejected because it would require significant reworking of TTexture, and more importantly because the handles are only valid in a single OpenGL context (disregarding resources shared between contexts, which is a can of worms that we can do without opening). Following the lead of OGLTextureManager, I created OGLResourceManager, which maps scenegraph objects (eg: TTexture, TVertexBuffer) to their low-level OpenGL counterparts (OGLTextureName, OGLBufferName).

Wow, that sounds pretty cool. How does this resource management fit in with rendering meshes?

With the resource manager in place, I turned to modifying TMesh to allow the use of vertex buffers. The cleanest way to do this was to make TVertexBuffer into a wrapper around the data (vertices, face indices, etc.). TVertexBuffer needs access to this data anyway, since it might have to create a new low-level vertex buffer object at any time (for example, if the Croquet image is saved and quit, and then restarted, an open TeapotMorph will create a new OpenGL context to render in, and new resources will have to be created in this context). It is also convenient because it makes it easy for the mesh to fall back to the vertex array render path whenever vertex buffers are not available or not desired.

Sounds good, but does it work?

First, the good news. There is a drastic improvement in the speed of rendering large meshes. This is excellent, since it was the motivation for this work.

Now, some indifferent news (not really bad). There was no speed improvement when rendering the default Croquet spaces using vertex buffers for all meshes. Some possible explanations are:

• we are overflowing the GPU memory (64MB on my laptop) and therefore OpenGL must send the data across the AGP bus anyway (essentially resulting in the vertex array model)
• for small meshes, the benefits of vertex buffers are outweighed by the overhead of managing resource names
  
• the bottlenecks are elsewhere for rendering the types of Croquet scenes that are currently typical
  

Preliminary benchmarking and profiling suggests that the last option is the most likely. However, I would have to study the issue more deeply before being willing to bet large sums of money on this opinion.

That sounds wonderful, but I really must go.

Me too. Thanks for listening.

Where to start? I just finished the first week at my new job as a Croquet hacker at the University of Wisconsin. What is Croquet, you ask? I'm glad you did. You might like to start with the project website, although non-techie friend have told me that they find it incomprehensible. My boss has been working on a redesign, which will hopefully be friendly to both end-users and hackers.

My move to Madison was eased by the Lombardis, who have graciously put me up at their house for a week while I found a suitable apartment. I have had a great time staying here, and am extremely grateful for their hospitality.

In between the inevitable adminstrivia, I have actually been able to get some work done this week. I've been working on two tasks, one big and one small. Small one first: increasing rendering performance of large models.

We have access to some high-poly models that were created by a laser range scanner (along the lines of the Digital Michaelangelo project. Such models are typically a) big (ours are over 200000 faces, and they can get much larger), and b) not structured for maximum rendering efficiency (for example, as long triangle strips). Loading the model into the scene drops the frame rate to about 3/second. Using Apple's OpenGL profiling tools, I was able to determine (as expected) that Squeak overhead is not the problem: approximately 95% of the time is spent in glDrawElements(). So, how can we fix this?

Two ideas immediately jumped to mind: using vertex buffers, and restructuring the mesh for faster rendering. The latter is a far more ambitious course, and I decided to try out VBOs first; if the performance improvement is not satisfactory, I can always try to reorganize the mesh later.

The current approach of using glDrawElements() is not a completely stupid way to do things. It is much better than the naive approach of calling glVertex() etc. for each vertex of the mesh. However, there is still the overhead of copying all of the vertex data over the AGP bus for each rendered frame. ARB_vertex_buffer_object is an OpenGL extension that addresses this problem by storing the vertex data in the GPU's memory. I expect to see a significant speedup, as many whitepapers from nVidia and ATI strongly recommend the use of this API for rendering static geometry. The downside is that it is an extension to OpenGL 1.4, which is beyond Croquet's current requirement of OpenGL 1.3 or better (I personally think that we should raise Croquet requirements to match those of Longhorn). With any luck, I will finish coding this approach this weekend.

The big thing that I am working on is the Brie UI framework, which has been designed over the last couple of months by Julian and Howard Stearns, our lead Croquet developer and all-round Smart Cookie. The overall ideas of Brie are best described in various posts in Howard's blog . Howard, Julian, and I spent a lot of time specifying the behavior of left/right mouse button clicks/drags/etc. in the virtual space. Where possible, we took advantage of the deep thought embodied in the Mac OS X UI. Sometimes it was not, since Croquet/Brie is not only a 3D environment, it is a 3D enviroment where (as Alan Kay says) "the authoring is always on". One can see an example of this philosophy in Squeak's Morphic UI. By Friday afternoon we ended up with a specification that we are all happy with. Hopefully we feel the same after reviewing our work on Monday morning :-).

Next, we will tackle the even thornier problem of attaching behaviors (reified as 3D objects) to other objects in the environment. In particular, there are deep questions about how objects inherit behaviors from parent objects (ie: objects higher up the scene graph), how objects customise default behaviors, and how parent objects can enforce certain behaviors in child objects. This is all still a bit of a whirl in my head as I try to catch up with Howard and Julian. Working with mouse button behaviors is so much more concrete! You'll be sure to hear more of this later.

That's my job! I can't imagine a better one.

I've noticed a trend in my blogging, which is to relate a literal and deeply technical view of my work. This will likely continue for a while until I reach full speed here, but please check the blog out periodically if you find that the current level of discussion is not appropriate for you.

Cheers!
Josh

Tony Nemelka has just been named CEO of Teleplace and I couldn't be more pleased. I have had a number of discussions with him and some of the exceptional people he is bring with him since he started working with the company, and I am really impressed with his vision for the direction for the business and the clear focus he has on customer value. I am very proud that the company has been able to attract someone of Tony's caliber to be the CEO. No question that the company was ready for this next step. Greg Nuyens did an exceptional job positioning the company technically - Teleplace is the best collaboration platform in the world today, by quite a wide margin. Now it is time to leverage that technical advantage into a market advantage as well.