Skip to content

Categories:

Watch Folder deposit tool

Having previously looked at the Word Add-in deposit tool, this post illustrates the second of the deposit tools developed in the DepositMO project, the Watch Folder, which uses common file manager services.

Computer systems provide users with file manager tools, such as Windows Explorer or Mac Finder. This organises lists of files, and folders containing files, stored on the computer. Mostly these are fairly static services. Users can save files to designated directories and folders, and can move and copy files to different locations using short-cut keys of drag-and-drop.

We want to make use of this process of copy or drag-and-drop to initiate repository deposit. It works on the basis of a ‘watch folder’. Essentially this folder contains a script that monitors user actions in adding files or folders to the directory containing the watch folder. Figure 1 shows the active directory containing the watch folder, and also the script that runs when you click on the watch folder.

Fig.1 Active directory containing the Watch Folder script

When a new file is added to the directory containing the watch folder the script initiates a session with the selected repository and copies the file there as well. To complete this process the general script needs some information from the user, the same information used in the Word Add-in example:

  • Direction, or url, for the repository
  • User login for the repository

This information is provided via a simple text form that is launched by clicking on the CONFIG file in the same directory (see Figure 2). This information only needs to be provided once per user per session with the script running, not for each new file deposit.

Fig. 2 Opening the Watch Folder CONGIG file to specify repository direction and user’s repository login

The watch folder ‘watches’ for changes to all folders in the directory. When a user adds a new file to a ‘watched’ folder, either a new or existing folder in the active directory, within a few seconds the script automatically adds two new files (Figure 3): METADATA, and VIEW_ITEM.

Fig.3 Adding a new file to a ‘watched’ folder generates a METADATA file and a VIEW_ITEM link file

Clicking on METADATA shows an XML copy of the metadata record for the object (content of folder and files) stored in the repository. The user can edit and save the XML record to update the metadata in the repository.

Clicking on VIEW_ITEM will link the user to the real repository record in a Web browser.

In this scheme, a folder in the active directory containing the watch folder represents a record in the repository, and a file in that folder represents an object linked from that record. Thus, folders can be used to organise collections in the repository, for example, adding two images to a folder will link both images to the same repository record. Hence, creating a new folder will correspond to a new record in the repository; an existing folder to a current record.

You can download the Watch Folder script (folder sync client) from our collected project resources. Note, this is not a formally packaged application for easy installation, unless you are comfortable working with perl scripts. For the brave a short ‘getting started’ guide can be found close to the download link for the script.

In principle, this was designed to be a simple way to deposit any file in a repository using basic drag-and-drop and copy commands, and add/update metadata, on the user desktop. Would it prove to be as simple in practice?

The Watch Folder and Word-Add-in are the two user-facing tools developed in DepositMO, but to work effectively these tools need to communicate interactively with repository servers, or repository ‘endpoints’ as they are known in the SWORD community. It is SWORDv2 that enables this communication between endpoints and user applications. The next three posts will consider SWORDv2 and how DepositMO has shaped and extended this standard for different repository softwares, DSpace and EPrints.

Posted in Uncategorized.

Tagged with , , , , , .


12 Responses

Stay in touch with the conversation, subscribe to the RSS feed for comments on this post.

  1. Marco Fabiani says

    Hello,
    my name is Marco Fabiani and I’m working on a JISC project to pilot a repository for the Centre for Digital Music and QMUL. We adopted DSpace as our platform, and I was reading with interest about WatchFolder. My question is: does it work only with EPrints? Would it be difficult to modify it to work with DSpace?
    Thank you for your useful tools,
    best wishes
    Marco

  2. davetaz says

    Marco,

    The WatchFolder tool uses the SWORD2 and DepositMO extension both of which have been added to EPrints 3.3 and DSpace 1.8 as per http://swordapp.org/sword-v2/sword-v2-implementations/. So if you are running DSpace 1.8 it should just work.

    Hope this helps.
    Dave T

  3. Steve Hitchcock says

    Hi Marco, See also these posts on this blog:

    DepositMO for DSpace (January 20, 2012) http://blog.soton.ac.uk/depositmo/2012/01/20/depositmo-for-dspace/

    DepositMO extensions for SWORDv2 clients (January 19, 2012) http://blog.soton.ac.uk/depositmo/2012/01/19/depositmo-extensions-for-swordv2-clients/

    Although we have not done full user testing of the Watch Folder client with DSpace, as we did with EPrints, local tests have shown this to work, as Dave says. Let us know if you are successful in getting it to run.

    Steve Hitchcock

  4. Marco Fabiani says

    Hi Dave, Steve,

    Thank you for the help! I will give it a try and let you know.

    Marco

  5. Marco Fabiani says

    I tried the WatchFolder script, but I can’t connect to the DSpace repository. From the specific post about DSpace, I gather I have to set the host to the specific community I want to deposit to, but I’m not sure how that translates in the CONFIG file’s parameter “host”. I tried different combinations but I get:

    [STARTUP] Attempting to establish deposit connection to server at :80
    [STARTUP] Failed to create the container, trying alternatives…
    501 Protocol scheme ” is not supported
    501 Protocol scheme ” is not supported
    [MESSAGE] Attempting to get /var/folders/js/n41f_4597v18r926wxv7jx7m0000gn/T/enlMG5kVVv from /swordv2/collection/handle/123456789/31
    [MESSAGE] Attempting to get /var/folders/js/n41f_4597v18r926wxv7jx7m0000gn/T/enlMG5kVVv from /swordv2/collection/handle/123456789/31
    [CRITICAL] Operation Failed
    400 Bad Request … The deposit URL does not resolve to a valid collection …

    Do you have any suggestion?
    Cheers,
    Marco

  6. Marco Fabiani says

    PS: sorry, the name of our server disappeared:
    [STARTUP] Attempting to establish deposit connection to server at (myserver):80
    [STARTUP] Failed to create the container, trying alternatives…
    501 Protocol scheme ‘(myserver)’ is not supported
    501 Protocol scheme ‘(myserver)’ is not supported
    [MESSAGE] Attempting to get /var/folders/js/n41f_4597v18r926wxv7jx7m0000gn/T/enlMG5kVVv from (DSpacebaseurl)/swordv2/collection/handle/123456789/31
    [MESSAGE] Attempting to get /var/folders/js/n41f_4597v18r926wxv7jx7m0000gn/T/enlMG5kVVv from (DSpacebaseurl)/swordv2/collection/handle/123456789/31
    [CRITICAL] Operation Failed
    400 Bad Request
    ……
    org.swordapp.server.SwordError: The deposit URL does not resolve to a valid collection
    ….

  7. Richard says

    Hi Marco,

    My CONFIG file which works with DSpace looks like this:

    host: localhost:8080
    sword_url: http://localhost:8080/sword2/collection/123456789/3
    username: richard
    password: ******

    As you can see, the host parameter is not the important one, but the sword_url must be the collection to which you would like deposits to take place.

  8. Marco Fabiani says

    Hi Richard,

    in the CONFIG file that comes in the zip file I downloaded, there is no sword_url parameter, so I didn’t have it set. It looks like this:

    host: http://example.eprints.org:80/
    username: admin
    password: password

    I will try the extra parameter and let you know.

    Cheers
    Marco

  9. Marco Fabiani says

    Hello Richard,
    I suspect the perl script I’m running is outdated: first, I noticed that the sword_url parameter is set to have the same value as host:

    $config->{sword_url} = $host; (line 98)

    I tried to comment that line out and I managed to connect to the repository, but then I get this error:

    [MESSAGE] Starting sync of resources
    [CRITICAL] Operation Failed

    So I went down a bit and found that the request during synchronisation was sent to (line 881):

    my $uri =”http://” . $config->{host} . “/id/records”;

    I changed that to:

    my $uri = $config->{sword_url};

    and then I managed to synchronise to the repository. I could probably fix all these things, but if you have a more up-to-date version of the script, would it be possible to have it?

    Cheers,
    Marco

  10. davetaz says

    Hi Marco,

    I’ll be working on the tool next week thus will make sure all the releases are up to date and should still work. Since I haven’t had access to a working DSpace 1.8 repository in a while I would be keen to work with someone to get it working again in the near future.

    Cheers

    Dave T

  11. Marco Fabiani says

    Hi Dave,

    I can now connect to DSpace, but can’t upload files. That seems to be problem with our DSpace installation which I will try to fix next week. I would also be glad to give a try to your updated script once you have it. Let’s keep in touch (email? you can find it on our project’s website).

    Cheers

    Marco

  12. Ian Wellaway says

    Hi fellas,

    Very interested in making this work for our dspace repository at Exeter. Did you get this to work?

    Thanks

    Ian



Some HTML is OK

or, reply to this post via trackback.