Where the SWORDv1 specification allowed resources to be Created in a repository, SWORDv2 has focussed on adding Retrieve, Update and Delete functions, thus providing full CRUD support. DepositMO focussed on use cases requiring extra functionality beyond basic CRUD but which could still use standard, already available specifications.

With DepositMO and SWORD v2 projects running in parallel this provided a set of clear requirements to enable complete interactive control of repository resources via a set of abstracted interfaces. By building upon existing, well developed and widely implemented standards (including HTTP and AtomPub), both projects utilise these standards to enhance digital repositories.

As both projects essentially outline the ideal RESTful implementation for discovery and control of resources in a digital repository, the specifications of the SWORDv2 and DepositMO projects have been integrated into the core of EPrints (v3.3). As a result, the RESTful interface to EPrints is as powerful and flexible as the built-in web interface, providing the following features:

1. All first-class objects can be managed via the REST interface. All EPrints, Documents, Files, Users and anything else identifiable by a URI can be created, updated and deleted via the REST interface.

2. All import plug-ins are now generic and can be utilised via the REST interface, doubling the number of supported package formats which can be deposited via SWORD. Additional plug-ins, which can be installed in a single click, are also available via the EPrints Bazaar app store.

3. The same permissions model is applied both via the SWORD/REST interface as via the normal EPrints interface. The permissions system provides a granular model via which users can be granted permissions based on item status, item type and even an individual item itself. Basically, if editing is possible via the web interface, then it is possible via SWORD.

4. The SWORDv2 specification supports the notion of “in-progress” items, so it has been possible to reduce the number of SWORD deposit endpoints in EPrints to ONE. This has many benefits:

• This endpoint represents a user’s deposits (or repository contents), closely aligning it with the Google Docs implementation and extensions of the AtomPub specification on which SWORDv2 is based.
• SWORD clients are easier to write.
• The user requires less context about which “collection” their content belongs to – it belongs to their collection.

In EPrints, the collection in which an item belongs is represented as a piece of metadata, a significant difference from DSpace. Rather than managing and connecting with several collections, a user can deposit and discover all of their resources via ONE URI, which is the same URI for every user (typically http://myrepo.org/contents). While the SWORDv2 specification outlines how this URI can be used for deposit, DepositMO mandates that a client should also be able to perform the inverse operation and request a complete list of deposit contents via this same URI, regardless of the status of the object.

4. A unique identifier (in the form of a URI) is assigned to every object (including EPrints, Documents, Files, Users etc…) as soon as it is created regardless of the stage in the workflow. This URI will NEVER be reused or overwritten, and can thus be used to reference an object throughout its entire lifecycle in the repository.

5. All URIs support all REST operations: PUT/POST, HEAD, GET and DELETE. The DepositMO profile of SWORDv2 adds the requirement that each URI must accept a HTTP HEAD request. Using this HEAD request, a client is able to request information about an object, such as last changed date, without needing to download the whole object to find this out. This enables each object to be synchronised with a local copy, allowing two-way update with clients that are keeping a local up-to-date copy, an important requirement in distributed systems.

6. All URIs can be content negotiated, meaning that you can get an RDF/XML, Atom, csv… serialisation of every object in the repository. These plug-ins are the inverse of the import plug-ins, so not only can you ingest items in any format, you can also export them in that same format. Again further plug-ins can be installed in one click from the EPrints Bazaar.

In summary, SWORDv2 and DepositMO on EPrints represents a major leap forward in repository flexibility. By utilising the built-in power of the EPrints identifiers, permissions model and Bazaar Store for plug-ins (all of which have been key parts of the EPrints 3.3 development), the HTTP CRUD interface supports SWORDv2 and DepositMO specifications as core functionality, replicating the functionality available via the web interface. This gives users the power to interact fully with their repository and content via an interface or client which suits their way of working, in their environment!

SWORDv2 officially began in late 2010, and DepositMO started at around the same time, alongside a number of other JISC deposit related projects including DURA and RePosit. When SWORDv2 set up the Technical Advisory Panel in early 2011, representatives from these projects were invited to join to share their deposit scenarios and technical expertise. Of the 3 projects, DepositMO was by far the most closely aligned to the goals of SWORDv2 and also the most technical project.

The SWORDv2 mission was to support the full range of CRUD (Create, Retrieve (Read), Update, Delete) operations for scholarly systems, and to maintain the use-cases which had driven SWORDv1, including mediated deposit and most importantly of all Packaging. DepositMO, meanwhile, was focussing on desktop-to-repository deposit and two-way synchronisation.

This meant that DepositMO would need to take full advantage of the CRUD protocol operations offered by SWORD, although both mediated deposit and packaging were not so relevant. The project would therefore be a valuable resource for a number of critical aspects of SWORDv2:

• A sounding board for core profile developments: DepositMO had a vested interest in the CRUD protocol operations and had explicitly no interest in the packaging aspects. This meant that the protocol operations would go through thorough review while the need for every packaging related concept would be questioned as to its necessity. As such, the SWORDv2 profile received extensive and sustained review throughout the project which has hardened it against many counter-arguments.
• A testing base for software development: DepositMO is being implemented against DSpace and EPrints, as is SWORDv2. Since it was clear from the outset that DepositMO is providing some extensions to SWORDv2, the majority of the codebase for both systems is the same. Extensions to SWORDv2 have been developed for both repositories by the project though. This means that not only has the core software been through some important testing, but its capacity to be extended has also been examined.
• Representation of a critical use case: the desktop-to-repository use case is one of the most under-developed in our community, so it was very interesting to have a project focussing on it to represent that use case throughout discussions. With services such as Dropbox and Ubuntu-One becoming common, the deposit applications developed by DepositMO will no doubt be an important demonstrator as to the way academics will interact with repositories in the future.

In addition to the base operations provided by SWORDv2, the DepositMO project has also specified the following extensions:

• That Collections be able to list their content items which belong to the authenticated user.
• That individual files behave RESTfully and in line with the rest of the SWORDv2 specification. This means that replace and delete operations can be carried out on the individual files in an object.

The SWORDv2 project therefore considers it to be a realistic possibility that a SWORD 2.1 specification may be produced in the future incorporating the DepositMO extensions in addition to any suitable extensions from other projects using the protocol.

Create, Retrieve (or Read), Update and Delete (CRUD), the four basic functions of persistent storage. This is what differentiates the project from current capabilities for remote deposit of content to repository services. I’m using CRUD to write this blog post in WordPress. As I write I have two action buttons to the right of the content pane: Save Draft, and Publish. Both allow me to update the content to the storage server, the difference being whether the post is made public or not.

So there is nothing new about CRUD, except that it is not yet directly applicable to many of today’s digital repositories, which tend to support single publishable item deposit with subsequent versioning should changes be needed or if updated versions are produced. In other words, there is no concept of a repository workspace – or a connected workspace – that allows the simple incremental updates widely supported by other computer authoring services and described by CRUD, or applications that go beyond this.

“For authors it is often suggested that content might be deposited once by filling data in a Web form, but too much effort is involved for the process to be repeated for another repository. Better is multiple simultaneous deposit under the control of the author.”

It became clear that we should do more to emphasise the role of CRUD in this project following a short, branched exchange on the American Scientist Open Access Forum mail list. A recurrent theme on the list had returned – the apparent tension between deposit in central, subject-based repository collections such as arXiv and distributed institutional repositories (IRs). The question is where to deposit; the aim to maximise the volume of open access content.

Currently the answer probably depends on which subject area the research to be deposited covers, e.g. physical or biomedical sciences will most likely deposit centrally, due to the strength of the repositories serving these areas. Other disciplines will deposit institutionally, but on a much lower scale. There is the crux of the open access problem.

The answer proposed, notably by Stevan Harnad on the AmSci list, is for institutions to mandate deposit of published research papers in the local repository. That is, all papers, not just those not already deposited in a subject-based open access repository elsewhere.

For authors it is often suggested that content might be deposited once by filling data in a Web form, but that too much effort is involved for the process to be repeated for another repository. One approach to reduce the perceived workload, given that all these repositories are open and allow open harvesting of data using OAI-PMH, is to deposit once and then harvest the content to other repositories as required.

Another approach might be multiple simultaneous deposit. To save authors effort, data for deposit is entered into a form once, and then copied to the designated repository destinations. One tentative suggestion to emerge in the latest round of list discussion was that deposit to an IR be accompanied with login details for a central subject repository for subsequent deposit. This is fraught with security problems, as we pointed out to the list.

Enter SWORD, for it was suggested that this be the mechanism for sharing deposit and logins in this case. It turns out that the organisation developing SWORD has a case study that looks quite like that proposed.

Separately, this is what arXiv says about using SWORD for deposit:

“This interface is primarily intended for use by conference organizers, proceedings and journal editors, etc. for programmatic bulk upload of pre-vetted material to arXiv for long term archival and dissemination. It is assumed that this is done with the (implied or explicit) approval of the authors of individual contributions or on their behalf.

“Individual authors may prefer arXiv’s interactive web upload for personal use, because it provides better feedback mechanisms, but in principle the deposit API can be used for one-at-a-time deposit to arXiv by individual authors, too. We envision integration of the deposit process into authoring tools for efficient upload from the desktop.”

So third-party deposit is just about acceptable, perhaps, without being wholly endorsed. The last sentence points indirectly towards the work of DepositMO, and Simeon Warner of arXiv was a co-author of the project’s short debut paper at the Open Repositories 2010 international conference (OR10).

As this paper shows, better than deposit-once and subsequent deposit elsewhere by another agent is multiple simultaneous deposit under the control of the author. It turns out that SWORD has this covered as well.

In fact, there are quite a few SWORD implementations connecting different applications (sources) and repositories (destinations). If you look closely, one of those implementations listed is Microsoft Article Authoring Add-in for Word 2007/2010 – allows repository deposit direct from Word. Within DepositMO we have made some claims about enabling repository deposit from popular applications such as MS Office, and in the project we shall be working with Microsoft to enhance this tool.

Have we made the USP for DepositMO clear in the documentation to date? It’s not SWORD, or deposit or even multiple deposit, or deposit from specified applications. The answer begins with CRUD.

Among this welter of deposit applications, you are probably asking what exactly will be DepositMO’s unique contributions? No. Well I was. At least, I was beginning to wonder if we had made our USP clear in the documentation to date. It’s not SWORD, or deposit or even multiple deposit, or deposit from specified applications. The answer, as we have already indicated, begins with CRUD.

The project proposal talked of ‘an effective culture change mechanism’. That’s a wider issue for another time. On more technical issues the proposal describes the aim to ‘extend the capabilities of repositories to exploit desktop and authoring environments’. More specifically it refers to components for the Microsoft Office authoring environments and enhanced SWORD interaction.

No reference to CRUD-like features here. Nor in the OR10 paper – at least, not using these terms – but the direction is clearer. The paper starts by specifying the motivations for multiple deposit.

Today the use case for repository deposit is write the content with a typical computer desktop application and save it somewhere, but not in the repository yet – the equivalent of the blog Save Draft button. When the work is complete it can be packaged and delivered to the repository using SWORD, the same as the Publish function in the blog. The OR10 paper puts it like this:

“Currently SWORD is a one-way protocol, meaning that a repository can either accept a record, or reject it; there is no middle ground. Adding a lightweight mechanism to desktop applications to enable negotiation on what is sent in a SWORD package would go some way to bridging this gap.”

This facility should become available in SWORD v2.0, and developers from the project are contributing directly to this activity since there is a vested interest in the outcome.

It would open new deposit possibilities. An admittedly complex and possibly unusual, but nevertheless feasible, case is suggested in the OR10 paper where an author of a research paper pulls information from other linked sources, such as a contacts list and a citation manager:

“At the point the document is submitted all this valuable information (such as author identities disabiguated by email address and structured citation listing) is lost.”

All this is more sophisticated than CRUD and points the way forward, but first implementing CRUD features using SWORD as a mediator between applications and multiple repositories would represent serious progress.

What might follow from this is ‘culture change’ or, more immediately, dialogue between author and repository. The OR10 paper puts it more prosaically:

“Our proposal is to enable a simple yet powerful set of negotiations to occur between the desktop application and multiple repositories such that a single familiar submission workflow (in the style of the author’s application) can be presented to the user.”

So as a starting point the aim in DepositMO is to activate the repository as a storage service for the iterative Save Draft action in an authoring application.

In the next post we will consider the practical implications of this approach and look at an early sketch of a possible interface design.

Just as long as we all understand and can share in what is new and where we are heading. Otherwise we may just find ourselves talking about a more familiar form of crud.