Polynomial Texture Map (PTM) Extension: feedback from archaeology users

The Polynomial Texture Map (PTM) Extension application for EPrints is an example that begins to tackle the effective organisation, description and presentation of multiple research data objects (perhaps 100s) from a file archive within a single repository item. In the previous post we saw a glimpse of how this works through a visualisation of the workflow for deposit of an archaeology image archive. In this post we report initial feedback from the first users to test the application, who were also instrumental in informing the purpose and design of the app for archaeology.

Our users regularly make field trips to archaeological sites in the UK and worldwide to capture information using cameras and other electronic equipment. The intention is to upload this information in processed form for secure storage and controlled access. An institutional research data repository is a possible target for this.

At some times this upload might happen on a daily basis, and at the end of an intensive day in the field the data collector wants the upload process to be as simple and fast as possible. The resulting image collection in the repository has to be uploaded, unpacked, fully documented and presented for ease of sorting, searching and viewing all images. It is not feasible this could be completed manually, image-by-image, in the circumstances, so a typical repository process for unpacking an archive file would be inadequate in terms of automatic documentation and presentation.

This is the challenge for the new app. Below is summary feedback on the use of the PTM app as originally shown (slide 27) in the RSP presentation. For the record, and for others who may seek to develop the app further and design similar apps, this is followed by edited correspondence on which the summary is based. Some usability issues with layout and workflow, and limitations on the number of objects that can be handled, have been tackled. Remaining issues include the transfer to the repository of XML metadata generated by the recording equipment with the original files.

Summary feedback

Hembo Pagi, Southampton archaeologist

Graeme Earl, Southampton archaeologist

Feedback correspondence

Point of contention: to start a data deposit with New Item or New Dataset button in an EPrints repository enabled with the ReCollect research data app?

After some false starts and the ironing out of unexpected effects following initial implementation of the PTM app on the test server, the substantive work of using the app began.

Graeme Earl, 14 March 2013

In summary I would like the ingest process to extract thumbnail images from a specific folder and embed or link these from the deposit page. We have had agreement from our partners who developed the RTI format that this would be an appropriate addition, and I think it was also listed in my original workflow document.

Hembo Pagi, 19 March 2013

The main issue I see is the workflow. When you start with the New Dataset and click Next you are not provided Dataset information page. Why is this step skipped? At the moment to do the full upload and data entry process you do following:

1) Clicked New Item button
2) Selected Dataset and clicked Next
3) Clicked Previous
4) Uploaded zip package. Was waiting until done, then clicked Unpack PTM icon
5) Filled in required fields
one note: on Snapshot image, can Type be preselected as for PTM?
When done, clicked Next
6) Nothing was filled, clicked Next
7) Chose the RTI Shelf, then clicked Next
8) And finally clicked Save it for later

All fine, except when I see the final list of files in Preview tab Source images are linked to one file instead of list of source images.

Hembo Pagi, 21 March 2013

One new detail about the workflow which I hope is not too big an issue to implement. We have been talking with our collaborators who work with similar stuff and they have set in their software (which generates PTMs and all the package including XML) a different way of handling snapshots. Here is how the next release the RTIBuilder will output the folder structure:

In the “finished-files” folder there would be a jpeg file which serves as the thumbnail for each RTI or PTM found in the finished-files folder.  There is only one thumbnail for each RTI image. In addition, folks could make a “snapshots” folder within the “finished-files” folder where they can place an arbitrary number of snapshots, named however they wish. Here’s an example:

finished-files/
— ceramic-1_2742.ptm
— ceramic-1_2742_ptm.jpg
— ceramic-1_2742.rti
— ceramic-1_2742_rti.jpg
— snapshots/
— — snapshot1.jpg
— — snapshot2.jpg

Graeme Earl, 21 March 2013

It would therefore be ideal if the ingestion process stripped out the thumbnail and showed that on the archive page and then elsewhere on the page showed the snapshots, perhaps in a gallery view. I imagine that thumbnail display will be a common requirement for many data types.

Hembo Pagi, 17 April 2013

And the big files still do not work.

Tim Brody, 18 April 2013

I’m unsure what you mean by clicking Next you skip a screen? When you click “New Dataset” you land onto the page with the combined upload and information fields on it. You don’t need to click Next after you upload something, just scroll down to the metadata? (I wouldn’t lay it out like this, but I defer to Patrick (McSweeney) et al who designed it (ReCollect app).)

I’ve bumped the max files to 200.

Hembo Pagi, 19 April 2013

I got it now … I always clicked New Item from the top and then from the next view where you have item types I picked Dataset. Then the metadata page is skipped. I think it is confusing if there are two ways to add the dataset.

Its good to know that the limit is 200. We can take that into account.

Finally, it think it would me more accurate to call your plugin RTI (Reflectance Transformation Imaging) extension as it can include .rti type of files as well.

Hembo Pagi, 6 June 2013

We have not done any data uploads yet. We’ve started to collect data tho, and it will be a massive task (I think we have almost 1000 entries) which I think requires bulk upload functionality, but lets get the current version working and documented first.

I think the system is almost there. There are some usability issues, but I guess most of them are related with the eprints itself, not the ptm app. The final unzipped content page also needs a bit work, as at the moment you cannot access original files one by one, only by downloading the whole zip. So, not a big thing.

I think the main benefit of this system will be when an actual XML file can be used to get all the metadata and have all the form fields prefilled. Sadly those XMLs do not exist yet but can be automatically generated, which we will. At the moment there are too many steps from upload to deposit. Several of them can be skipped if the XML data could be used.

We are grateful to Graeme Earl, Hembo Pagi and Gareth Beale for their expert specification of the required application, and their subsequent feedback on the implementation of the PTM Extension.