The purpose of this post is to present the outcomes of the usability evaluation study which is described here. The study consisted of a survey, which was completed the critical friends Michael Day, Faranak Hardcastle and Laura Hyrjak.

The standard interface

This section of the study was designed to evaluate the standard linear interface, as described here.

Advantages:

Michael Day mentioned that this interface is better at displaying recent activities because they are displayed in a chronological order with date-time stamps, and the pictures are larger than in the honeycomb interface. He pointed out that the details about the nature of posts are clearer in this interface and this made it easier for him to find his way around the application. Because of his familiarity with this design of interfaces in other applications such as Facebook, Twitter and Instagram, he found this interface clear, easy to navigate, accessible and natural.

Similarly, Faranak Hardcastle felt comfortable in navigating through this interface and indicated that it is self-explanatory, easy to navigate and doesn’t need a walkthrough. She pointed out that she got the information that she was expecting to see quite quickly and mentioned that everything was clear. However, she felt that it is not appropriate for the purpose of Leapin.it system. If we decided to go with this interface, her suggestion is to stick to the original design by treating it as a pre-packed self-contained product and resist the temptation to customize or change its features.

Laura Hyrjak found this interface effective in at displaying an overview of recent activities and in allowing the users to clearly see the post in its entirety without having to click on it. She mentioned that this interface is nice, familiar and has an obvious order of posts with information about who posted where, about what, and when, with options that allow for interacting with it in plain view. Furthermore, she indicated that the individual posts are clear, large enough to view in their entirety and in an obvious order. She added that the standard interface is self-explanatory because it has enough information to know which each post is about at a glance, and it soon becomes apparent that they are in a clear order, especially because the layout is often used in other applications and websites.

Disadvantages

It has been pointed out that the standard interface is very similar to many other interfaces used by social network websites, and it therefore didn’t grab the attention of the participants. Because the standard interface resembles Facebook’s news feed, Faranak Hardcastle mentioned:

“It wasn’t offering anything novel and perhaps because of this similarity with Facebook, I failed to associate this interface with Leapin.It.  As the purpose of these two social networks is different, in my opinion this could go against the overall impression of Leapin.It”

She argues that this design only works efficiently if it is known that the design of the product is finished and there will be no more novel and unconventional features to be added. This is because when adding new design features, this interface might run the risk of not being able to accommodate the needs for them. In her opinion, the main reason for this is that the users are used to use this type of news feed in a certain way and changing the functionality of that might be off-putting for them.  Laura Hyrjak said that this layout is plain, boring, only one or two posts can be seen at a time and only one sorting mode is available which is the time.

The honeycomb interface

This section of the study was designed to evaluate the Honeycomb interface, as described here.

Advantages

Michael Day said that the visual presence of this interface and the spread of honeycomb imagery on entrance to the application made a unique standing point with which to find his way around the application. In particular, the honeycomb layout allowed him to pay a lot more interest and attention to all the different honeycombed hive pictures and images. Interestingly, it has been mentioned that this interface links everyone together and makes their contributions seem as collaboration in building a real bee hive. Other merits that were pointed out about the honeycomb interface included that:

1. It is unique.
2. It is interesting in that it allows more random exploration of posts.
3. It is eye catching.

When asked about the appropriateness of the honeycomb interface for mobile applications, Michael Day mentioned that it is more suited and eye-catching than many other applications that he has, which use streaming feeds in a clear chronological layout. He added that this interface is visually appealing, stands out and is appropriately designed. Having the posts as small pictures (or icons) in this interface forms a diverse layout, especially for users who want to spread out their range of areas. This suits the idea of ‘virtual worlds’ better, because each icon seems more self-contained. He felt that he was building a collection of something rather than a long list of his interests that were more cluttered in a long stream. He added that there is a good scope for this layout for tablet devices (e.g. iPad) because the images would be bigger and more data displayable.

Faranak Hardcastle mentioned that the honeycomb interface was more effective at displaying a summary of the information at once and was easy to use, user friendly and self-explanatory. She didn’t feel that a walkthrough would have made a difference. However, she mentioned that giving the option of having a walkthrough would be a good idea. In this interface, she added that, with a glance, she could tell whether she was interested in any of the contents or not. She mentioned:

“This suits my browsing habits better as I normally have many tabs open and don’t like to go back to the old habit of sequentially reading the context on one page.”

Concerning the appropriateness of this interface for mobile applications, Faranak’s comments generally agrees with Michael’s in that the honeycomb interface is visually appealing and more appropriate than the standard interface. However, her main suggestion for improvement is to find a pattern to link the posts together. She added:

“This would mean that the initial interface would be sorted based on this narrative/pattern and then every individual post would also have links to other posts based on that narrative.”

In describing the effectiveness of this interface, Laura Hyrjak indicated that it has a nice design and is more comfortable to use than the standard interface. She mentioned:

“This app is far superior when it comes to showing a large amount of posts and recent activity. The layout is appealing since it is different.”

She also mentioned that the honeycomb layout is self-explanatory and obvious because it is simply requires the user to click on a picture to get more information. She found the interface appropriate for mobile applications, straight-forward and easy to use. She suggested including some slides that come up at first launch with basic controls where people swipe to get more information. She added:

“This should then be followed by a faded out app with text that appears over the top explaining each new screen that the user comes to. I cannot think of an app that does this at the moment but it is done by a few apps and looks nice and is easy to follow.”

Disadvantages:

One of the problem identified with interface is the lack of detail and chronology, in that it is not readily apparent in the ordering of of posts. It was mentioned that the learning curve for the Honeycomb interface was more complex that the standard interface. It has been recommended to include some details about the posts by adding icons in the honeycomb interface. Faranak Hardcastle mentioned that the honeycomb interface was great to get the users’ attention in the first place but it fails to maintain that attention. Concerning the social aspect of this interface, she said:

“I also didn’t get a social feeling as it looked more random. Perhaps one of the reasons for this was a lack of a narrative. For example, in Facebook’s newsfeed, this narrative is social. For an individual the narrative can be a curiosity to find out what their friends have been doing/reading/liking. In other news feeds, such as the one on the BBC news website, there is a visual temporal pattern that makes a more temporal narrative.”

Another point that was raised by her is that having the word “Feed” written on the top of the honeycomb page is misleading because the idea of honeycomb is to promote something quite different to a feed. Laura Hyrjak mentioned that it is unclear without selecting a post what it is about other than the picture (e.g. who posted it, where it was posted, and when it was posted). She suggested adding zoom in feature to avoid cramped feeling of small images especially if the application is on a small phone screen. However, she mentioned that the hexagonal backing with pictures and information gave it a nice appearance.

Concept Evaluation

This section considers the evaluation of the LeapIn.it app, and the concept itself. The participants used the Honeycomb section.

Potential use

All the participants indicated their interest in using the application with the honeycomb interface. Michael Day said:

“I had already started to grab various things I had been interested in to type barcodes into- it actually became quite addictive, almost like collecting and building a collection to show off. Something quite appealed with that; I found myself thinking about how I could build up my music, film, etc and share it with different people to network relating to this. I also thought about what would happen in terms of discussion and how I would be willing to blog over shared interests through it.”

As a prototype, Michael Day mentioned that the application was quick, simple to use and easy to add photos. He said:

“It’s a really impressive production piece that I think meets all of the objectives you aimed to incorporate and reflects a number of different elements of the virtual world idea in a really neat way. It’s something that not only would I use, but would make use of and want to help develop.”

Similarly, Faranak Hardcastle thought that the design really suits the concept and she didn’t find any contradictions in the purpose of the application and its design. She mentioned:

“I totally see myself using Leapin.It. I’m especially interested to [see] the application of it when travelling to different places. I think it would be really interesting to be able to scan QR codes from posters in languages that I don’t understand or to scan barcodes of the products that I have no idea about. I can even imagine kids at school doodling barcodes/QR codes on benches!!”

Laura Hyrjak indicated that this interface is visually appealing and appropriately designed, she added:

“The app idea itself is interesting and could provide a new way of people getting together and discussing products, reviewing places to eat or watch plays, or discuss good deals.”

Advantages and disadvantages of scanning QR codes and Barcodes

One of the participants mentioned that it was a bit slow to register barcodes. He recommended including more detail in the process of scanning. Laura expressed her interest in using this application by saying that it has a nice idea that hasn’t been implemented before and allows for having a direct link between the codes and the rooms. However, she mentioned that it constrains the use without being near the relevant barcodes, she said:

“If I wanted a room that was about something but I was just sitting at home without the said item/thing the app cannot facilitate this.”

The standard interface or the honeycomb interface?

Despite finding some problems with the honeycomb layout, Michael Day preferred it to the standard interface because it stood out from the crowd. He mentioned that this interface offered some kind of application uniqueness. In addition, Faranak Hardcastle preferred the design of the honeycomb interface and mentioned that it succeeds to provide visual appeal. Similarly, Laura Hyrjak mentioned:

“For the room based social network I do feel that the Honeycomb one is a nicer way of presenting the network.”

Recommendations and suggestions

As a result of this usability evaluation, we can make the following recommendations and suggestions for future work:

1. Add a date-time stamp for each post in the honeycomb interface to make the ordering clearer.
2. Offer users the choice of both interfaces as a setting.
3. Explain the honeycomb layout clearly by including a labelled diagram based on screenshots as an example.
4. Provide a version of this application for tablet devices.
5. Include more detail in the process of scanning.
6. Include some simple step-by-step instructions for scanning barcodes and QR codes.
7. Find a pattern to link the posts together in the honeycomb interface.
8. Have an option to zoom in and out in the honeycomb interface.