JamSpoon!

First of all, I am writing this on my iPhone on a bumpy road in the middle of Laos. So don’t expect this to look very nice… I have already lost some of my edits due to silly iPhone carry-on, so this is now a exercise in pure stubborness.

I have been designing in my head my Json CMS thing that I was going to call Spatula. But now the concepts and ideas are much clearer to me, so a rename is in order – JamSpoon it is! (name to be explained soon). In a nut-shell it will combine a rich UI for editing Json documents with an extensible REST based architecture for storing these documents on the web. More detail needed? Come on then, let’s go.

First of, I think the best description for what this system will be is an RMS. That stands for Resource Management System. By Resource, I mean resource in the web sense – ie everything that sits at the end of a URL. It’s therefore also a resource in the REST sense, and can be manipulated in the normal restful way.

So this system will provide the UI and mechanisms for managing web resources. It’s no coincidence that RMS sounds a bit like CMS. This is because one of my goals is to provide certain features that have become common place in most CMS systems (such as templates, edit/publish workflows, media management). But at the same time, I want to step away from the usual focus of CMS systems, which is to provide an end to end means for managing web sites. Web site creation is a much more specific and detailed set of functions (sometimes with in-page content editing, seo, HTML templating engines etc), of which resource management (editing text, uploading images, publishing blog posts) is often nestled into in an inextricable way. JamSpoon will only do resource management. Nothing else.

Why is this useful? See my previous blog post “CM bloody S” for some background, and my “Spatula” post for some more details on how I have got to this point (can’t link! On iPhone!). But basically it comes down to the fact l often content driven sites/services are tormented by the need for basic content management, but don’t need web site management. Often the web site
either exists already, or is developed in another framework (asp.net, ruby on rails) that has nothing to do with a CMS. All they need is a mechanism for their model to be populated.

Another angle of use is coming from the document/NoSQL world. Typically these systems provide a schema-free storage mechanism for holding web resources (usually in JSON). Having a nice UI to edit these documents would be very handy, and would often remove the need to write a custom UI for every particular purpose. These two usages seem like pretty good motivators to me, and should find an audience. Hopefully.

To begin explaining the design, I’ll start with the name. JamSpoon will be a system for pushing (spooning) around JSON documents (for textual resources) and binary resources. Ie JSON And Media – Jam. It’s a catchy name, I’m happy. There are 2 main aspects to the design which I think are the key elements. They are how the UI is configured to to edit JSON documents with a given schema (called Jam Recipes), and then how these documents are then stored on the web (Jam Spoons). Firstly, Jam Recipes. These are roughly equivalent to templates in CMSs, or schema for relational models. So this is where guidance can be given about how the UI should lie over a JSON document. I should emphasis this is not schema in the strict sense – the document being edited might have all sorts of fields not mentioned in the recipe. That’s fine, the UI will just ignore those bits.

The format of recipes will be itself in JSON. Specifically, JSON schema documents (no link sorry, google it). This format is very useful, as it allows validation information, nesting, arrays, and other custom fields. Being JSON, the same UI will be used to edit the recipes as the documents these recipes describe. Nice and tidy. I’d love to give an example in JSON Schema, but can’t easily on the phone. But it words, a recipe would be something like:

Car story
– Title (string < 100 chars, required)
– Body text (string, required)
– Array of photos
— Image (binary resource, required)
— Alt text

So this describes how a UI can be constructed such that a JSON document of this form can be edited. Hopefully users of Raven/Mongo/Couch might find this useful. So imagine a UI in which a hierarchy of forms let you create car stories, filling in the text, creating images underneath. Very standard CMS stuff. When you are finished, you can save it, and this resource is persisted somewhere. I'm thinking of supporting 2 modes: CRUD for normal admin type interfaces (where once saved, the resource exists, deletion is possible, no change history is maintained) and Publishing Workflow (where resources are in draft state until published, can't be deleted (only unpublished), and full change history is maintained). These are two very common patterns, each with a lot of uses. The Jam Recipe will state which mode a document should edited. Sounds good to me anyway. There is lots more too it (specifically how links between document could be displayed, and how displaying lists of documents/searching would be handled) but I won't get into it here.

Time to move onto the persistence mechanism. My idea is to use the analogy of spooning. Not in a sexy way, in a slurping stuff around way :). Jam (the JSON And Media) will come from the UI and sent to Spoons via a uniform REST interface. For eg, if Car Story 12 is to be deleted, the http request DELETE /carstory/1 would be sent. So The main UI website would contain no code to directly store anything, it would only be configured to point to a Restful end point over http. The expectation is that these end points would act as an adapter, translating these standardised rest requests into whatever form required to store the Jam. The external systems that actually store the Jam will be called JamJars, to emphasis that's where they ultimately live. Examples I can think of right now:

RavenJamSpoon: translates calls into calls to Raven (either over http again, or the clientAPI)
MongoJamSpoon: translates calls into the Mongo Api
CouchJamSpoon: ditto
LocalFileJamSpoon: translates calls into local file system operations
AmazonS3Spoon: translates calls to the Amazon rest Api for objects/buckets Cassandra, MySql, Memcached(!), WordPress, Umbraco would all be theoretically possible…

The point here is that because this is happening over a rest interface, this spooning can be done in any technology, an any platform. All that the spoons need to do is implement the uniform http interface the UI code is expecting, and translate it into calls in an external system however they see fit. Many spoons would be very simple, some more complex. Not sure which yet!

As an interesting side effect, this uniform interface could also be used to access these documents outside of JamSpoon, if the raw format is hard to work with. Good ones (like Raven) I would expect would just be uses as is, ie through the Raven rest interface.

I guess that's about it. I think the UI part will be done in a Ruby framework (Rails prob) and I'll start with the RavenDb spoon, in C#. Just to prove the neutrality of it all. Well I will start, as soon as the old tropical holiday is over…