User:Jacek Antonelli/Object Import and Export

Introduction
These are my notes for implementing Object Import and Export in Imprudence. That is, the ability to load and save prims in SL as files on your hard drive, to keep backups and such.

Milestones

 * 1) Ability to save and load text files. No parsing yet. Establish program flow and structure for saving and loading files, and establish UI (menu entries, etc.) for importing and exporting, even though it doesn't actually do work.
 * 2) Ability to load LLSD from a file and populate a local prim instance with its data. LLSD because the viewer has code to handle it already. Don't create it on the sim yet, just confirm it loaded then discard. Establish flow for populating a prim instance from loaded data.
 * 3) Ability to import an LLSD prim and create it on the sim. Not all attributes have to be supported, just enough to tell it's working. Establish flow for creating a prim on the server from a prim instance.
 * 4) Ability to download a prim from the sim. No saving to file yet, just print out enough info to tell it worked. Establish flow for downloading a selected prim and populating a local prim instance with its data.
 * 5) Ability to export a prim instance to an LLSD file. LLSD because the viewer has code to handle it already.
 * 6) At this point, the viewer should be able to import and export single prims from/to LLSD files.

Format
In what file format shall objects be saved?

There are two existing formats of note for storing objects:
 * OpenSim XML2 Format - the XML format used by OpenSim for exporting objects.
 * TestClient LLSD XML Format - the LLSD XML format used by the libopenmetaverse TestClient.

Or, we could use a new object structure:
 * JSON version
 * YAML version
 * LLSD XML version
 * XML version?

Handling Permissions
The anarchic thing to do would be to make any object exportable, regardless of whether you own it or have the proper permissions. But I'm not an anarchist, so I plan to add some permission checks.

It seems to me that to export an object, one ought to have the ability to modify it. Possibly also the ability to copy it, but that's a bit of a judgement call. If you can modify an object, you could in theory go through each prim manually and write down the parameters for each, and then reconstruct it. Or, more practically, write LSL scripts to clone it. So, mod objects are already "at risk". The exception here is texture references, including sculpty textures. These cannot be effectively copied via LSL, unless you also have access to the textures themselves. I'm not sure yet how best to deal with textures.

Note: It's important to differentiate between the object permissions and the "composite permissions". Composite permissions is what you see in your inventory -- it "collapses" the permissions of the object and all its contents, so that a full-perm object with a no-modify script in it would appear to be no-modify. The export tool should not consider composite permissions, but rather the permissions of the actual object being exported and, separately, the permissions of its contents. So, if the object is modifiable, but its contents are not, you can still export the object, but not make a meaningful export of the contents (i.e. a full-perm script in contents could be exported with all the code included, but a no-modify script would just be exported as a reference).

Of course, being an open source project, there would be nothing to stop someone nasty from compiling their own version with the permissions check disabled. Providing this feature would add another weapon to content thieves' arsenals. However, they already have plenty of similar tools at their disposal. CopyBot and the TestClient both make it a simple matter to export any object without regard for permissions. And this tool has significant legitimate uses, namely making it easier for builders to make backup copies of their creations, to move their creations from SL to OpenSim and vice versa, and to use offline building tools to edit their creations.

Textures
Dealing with textures in exported and imported builds is a tricky issue, for a couple reasons:


 * 1) Textures are slow and costly to upload. So, you don't want to have to re-upload textures every time you import a new version of the object.
 * 2) You might not have permission to download textures that are applied to a build. So, objects should be exportable separately from their textures, but retaining texture settings so that the textures could be re-applied later without much hassle.

My solution is to store only texture references in the object format, but to provide tools for managing and manipulating those texture references in the viewer.

[Expand on these points later]


 * Texture references can be a UUID, a file name, or a number.
 * UUIDs refer to existing assets that the user has access to. When the object is imported, they are applied.
 * File names refer to image files. When the object is imported, the user can choose to upload and apply them, or to associate them with existing texture assets.
 * Numbers refer to nameless, dissociated textures. For example, if the original build had a specific brick texture applied to some objects, that brick texture might become "Texture 5" when exported. Wherever the brick texture was used, the exported object would refer to "Texture 5". When imported, the user could choose to associate them with an image file to upload or an existing asset, or with an existing template texture (hardcoded into the viewer) which says simply "Texture ##" on it.
 * We could provide a texture search/replace tool which would help change the "Texture ##" textures to the build's proper textures at any time post-import. And of course, it would also be really useful for other texturing tasks.

Object Contents
If I recall correctly, tools like CopyBot and TestClient completely ignore object contents; they just copy the prims. But it would be more useful to builders if they did copy them, or at least references to them. For example, exporting a complex scripted object without remembering what scripts were in each prim, would make it a hassle to re-import, because you'd have to manually go through and add the scripts again. Or if you had a wall of vendors, you'd have to re-add each item again, as well as scripts and configuration notecards. Hassle!

The strategy here is similar to that with textures: store references to the content, which can be resolved at import time. The references can be file names ("my_script.lsl", "my_texture.tga", "my_object.xml", etc.), paths to items in your inventory ("My Inventory/Scripts/My Script"), inventory UUIDs of items in your inventory, or just plain names as they appear in the object contents ("My Script"). Unlike textures applied to the prims, all object contents have meaningful names already, so there's no need for number references.

For notecards and scripts, it would be possible to embed the text contents in the XML file. That has the advantage of keeping everything in one file, but it could be nice to be able to save them in separate files.

Nested Objects
In theory, it would be possible to import and export nested objects, i.e. objects with other objects in their contents, up to an arbitrary depth. I wouldn't bother with this initially, but it would be an interesting future enhancement.

The idea for importing would be to create the deepest-nested objects first, take them into inventory, and put them in the containing object. Then you could take that object and put it in the next highest up object, and so on.

Exporting would be the opposite: take out the objects from inventory, rez them, and export them. If there are more objects inside that one, take them out and export them, and so on. There are some pitfalls here, though. The obvious one is that the parcel or sim might run out of prims during unpacking, for very deep nestings with high-prim objects. So, we might have to be a bit clever, remembering which objects have been exported, and only having two objects rezzed at a time, the parent and the child. If the child has a child, the grandparent (formerly the parent) would be taken back into inventory for a while.

This has some pitfalls of its own, though, when it comes to scripted objects. Because objects can have scripts that trigger events when the objects are rezzed, the act of unpacking the object to export its nested objects might break or change them, or cause other harmful or annoying effects (like chat spam). I suppose we could encourage users to export scripted objects in a no-script sandbox.