User testing results 4: what users said

We are about to complete the results of the user tests of the novel and original repository deposit tools developed in the DepositMO project.

Based on the tests we have a number of ways to analyse the results. Previously we considered what the timings and task completion tell us about the usability of the deposit tools, and what users did, based on the observer notes and an assessment of the repository records they created. Here we discover what they thought of the tools they were testing, prompted by a series of questions in the user’s test documentation and placed to be answered immediately at the conclusion of the practical part of the test.

We present just the summary results and selected extracts from this user feedback, so we do not have another over-extended post. If you wish to see the unexpurgated comments, here is the original report of the full test results (pdf).

Summary results

1 Would these tools encourage you to deposit more of your own content in a repository?

Yes (8) No (5)

2 Would these tools encourage you to deposit types of content that you have not previously deposited in a repository? Which type?

Yes 5 (inc. images, multiple presentations/teaching resources, computing files) No (6) ? (2)

3 Which of the two new deposit tools (in Word, and in the file manager) used here are you likely to use if it was generally available?

Word (7), File manager (1), both (3), neither (1), repository (1)

6 Or are you more likely to continue to use the standard repository deposit interface?

Yes (6), No (0), ? (4), N/A (3)

These summary results suggest that on balance use of these tools might encourage more deposit, just, most clearly in the case of the Word tool, but it won’t be so easy to wean users off the standard repository deposit interface on the basis of these tools.

Selected extracts from fully transcribed results

Here are some extracts to provide a flavour of the user comments. These are not intended to be fully representative, for which please refer to summary results above and the full test results document.

1 Would these tools encourage you to deposit more of your own content in a repository?

User2 (EdShare) Yes For me the Word tool. The MO (file manager) deposit was initially exciting but the difficulty of differentiating between many files was frustrating as filenames were not carried over. The updating via Word for single files and also the addition of new items via MO leading to either loss or overwriting of previous metadata is v. frustrating.

User 3/1 (Library) The Word deposit seems very simple, provided it’s easy to get the DepositMO add-in. I would like to know if I need to construct my Word docs in a particular way. Image deposit (file manager) seemed complicated and a bit confusing – not sure I’d use this.

User 5/1 (Kultivate) If uploading Word files would definitely use the DepositMO tool. Not sure about other content.

2 Would these tools encourage you to deposit types of content that you have not previously deposited in a repository? Which type?

User 1/1 (EdShare) No. Folder deposit confusing

User 1/2 (EdShare) No. Word integration confusing

User 3/2 (Library) In theory – yes – but possibly only Word as that was easiest!

User 5/2 (Kultivate) I think yes for PDFs (and Word docs) – of course the images still seem to require more work/metadata

3 Which of the two new deposit tools (in Word, and in the file manager) used here are you likely to use if it was generally available?

User 1 (EdShare) File manager. Word version confusing in version control, settings and location of files

User 6/1 (Archaeology) The Word one is nicer and a good idea, but all are confusing

User 7/2 (Archaeology) Word was very useful for text. The file manager was good for images but possibly required a bit more prior knowledge.

4 How would you improve either of the tools?

User 1/2 (EdShare) Make actions of file manager clearer (what is being updated and where)

User 2 (EdShare) More metadata extracted from original file e.g. title from name, creator from owner (if available) and also type (Word added-both showed as HTML files)

User 3/1 (Library) Word – I wouldn’t require user to add/edit URL; Image – I think this needs to be made more user friendly

User 3/2 (Library) Metadata requirement (i.e. having to add title in HTML) in file manager prob off-putting to some.

5 Are there features of the new deposit tools that would deter you from using them for repository deposit?

User 2 (EdShare) Overwriting metadata which had been hand-edited when updating or adding additional files is a real turn-off.

User 3/1 (Library) Image deposit is too complicated. I wouldn’t remember the URL needed for Word deposit

User 3/2 (Library) I have nothing to compare this with, but I did find too many windows open caused a degree of confusion

User 5/1 (Kultivate) With uploading images etc … I think the time taken in standard way is about the same, due to adding metadata. I think the ‘watch folders’ idea is brilliant but needs to be developed further.

User 6/2 (Archaeology) The fact that each file’s metadata had to be in its own folder with that file -> hard to deposit large directories of files.

6 Or are you more likely to continue to use the standard repository deposit interface?

User 1/2 (EdShare) Yes. Better/clearer control. Less ambiguous.

User 2 (EdShare) Would like to be able to use the new deposit tools more as the traditional, although v. good for library geeks like me, is long-winded for the normal academic.

User 6/1 (Archaeology) I think yes because it is proven to be efficient

Any other comments?

User 5/2 (Kultivate) Really helpful for documents – I can see that for repository managers and even self-depositors this could be helpful, saving quite a bit of time.

User 6/1 (Archaeology) Good ideas but they are not easy enough to use. They seem slow and frustrating.

Once again, we thank our users for engaging fully with these tests, for the time and effort they gave to participate, for their rigour and objectivity. As we saw earlier in the live audience demonstrators, it’s when you put software tools in front of users that they come to life.

Our test users were: I. Atkins, Joy Caisley, Grant Cox, David Davies, Helena Dmetriou, Harry Gibbs, Nick Graffy, Marie-Therese Gramstadt, Matthew Harrison, Alisa Miller, John O’Hagan, Isobel Stark and Adam Warren.

I must also thank the organisers of the user test teams, who doubled as test observers to report on individual tests. Our observers were: Penny Copeland, Debra Morris and Kate Walker.

This completes the recorded results from the DepositMO user testing. In the remaining posts, as the project ends we will seek to draw some lessons and conclusions from the tests and from the project overall. These will be our interpretations of the results. If you have a view on these results and what they might tell us about the deposit tools developed by the project, or on the prospects and needs for future repository deposit tools, please leave a comment.