Erevnametrics: Research Object Builder UI
While I’ve mainly been talking about the back-end of the RO Builder, another thing that I have been doing is creating an easy to use, browser based, front end. This I am doing with html & javascript, and it links together the various shims and the RO builder itself. Eventually I’d like to expand it to include several methods of visualizing and sorting on the data beyond plaintext. Before properly building the UI, I’ve come up with several concepts, some simple and resembling a simple HTML form that could be easily added to another page, others a bit more complex.
This is the most simple of the lot, and is just a front end for the builder; it consists of one line for adding new resources, one for deciding where you want to create the RO (and possibly the format of it – n3, RDF/XML, turtle, etc.), and then a table of all the elements added so far.
A little bit more complicated here, but still the same format as before: The picture would be either something from the RDF representing the main subject, or a generic placeholder picture for it’s type. The uri to the picture would have to be passed to the web page from the shim, which would increase their complexity, but for simple shims, a default picture / type should be able to always be relied on.
This is a concept for visualizing the RO as a web page – the RDF is displayed in a tree structure, to the left, and all of the elements would be draggable onto a preview pane, which would result in a method of displaying that resource dependant on the type of the resource. Ideally it would let you change the way each object is displayed simply, and allow the end user to customise the page as they wished, in addition to providing a number of default templates. Due to the fact that it would be generated from the RO, the content would be dynamic - with a standard template for displaying a page, changes to the RO would be automatically reflected in the page, and something similar, albeit a bit more complex, could easily work for custom layouts too. This concept, however, is very likely beyond the scope of this internship – it would likely be too complex to create during my time here.
This is another concept, and is probably the best out of the ones I’ve talked about today, it would be a relatively simple page, with collapsible boxes for adding resources, and then some simple sorting of the resources based on searching on either the content of their triples, or reasoning about their resources; then links to different forms of display for the results obtained. The main advantages of this over the other main display layout are it’s simplicity and modularity. The top pane for adding and then creating your RO is essentially concept #2, and the bottom panes could be modules that you could extend or modify with relative ease.
SLE: News Feed Viewer
An important element of the university’s current learning environment, SUSSED, is the noticeboard. This is a compilation of news and important announcements from a variety of departments. As a result (and as can be seen from the screenshot below), this means the announcements are not always relevant to every student – this was brought up in several focus group meetings.
To remedy this, one of the widgets in development is a news feed viewer which will initially contain a set of default feeds that are relevant to each student – for example, based of the school (and possibly the societies) they are a member of. This idea was translated to a series of Pencil wireframes and HTML mockups to give a starting point for the actual coding.
Important university announcements (such as campus closures) should of course be always visible, but if all other feeds could be added or removed by users at will this would make for a much more personalised portal. However, after programming a first draft of the widget an interesting problem was encountered.
As can be seen above, all of the news feeds from the university cannot be rendered, despite external feeds rendering with no problems at all. Further investigation, using an online Feed Validator revealed that all of the university’s news feeds do not comply with RSS standards (due to the formatting of the published date).
This leads to the tough decision of which would be the simpler to fix: standardising all of the university’s RSS feeds, or re-writing the code at the base of the widget to compensate!
Erevnametrics: API Evaluation
To make the Research Object creator worthwhile, it needs to support obtaining information from a large amount of sources – the point is to aggregate all of the useful data from all these different resources, after all – and as such, we have to have shims that will, at the most basic stage, take some form of data from a source and then pass it on to the Research Object creator in a useful form, in this case, RDF/XML.
Here, I’ve gone through a couple of sites and just evaluated the API for usefulness.
- Eprints: The easiest to work with – already provides RDF so it’s as simple as passing that to the ROC.
- Arxiv: Doesn’t provide RDF, but does provide a basic atom feed for each article, from which rdf can be created – I made a shim for this.
- Slideshare: It’s API provides XML results for both single and groups of slideshows, but require authentication – this could complicate the shim a little. Also provides usage data – the number of downloads, number of comments, number of favourites, etc.
- Scribd: Also has an API that gives XML formatted groups of results, usage data, and requires authentication.
- Edshare: Also provides RDF based on the object already, so that can be passed on. As with eprints, the usage data might be the tricky bit to get ahold of.
- Flickr: Has an API that returns usage info, author info, geolocation, etc. Also requires authentication.
- Youtube: API returns info, comments, author info etc.
For all sites which don’t provide RDF, the API’s can be used to generate RDF, which the ROC can work on. Even with some of the sites that provide RDF, the idea is to augment the RDF with usage data of the media with the shims. These shims are things that potentially can be written by anyone who wants to support their own information source, and they will have a clearly designed object format that they need to pass back to the UI as JSON.
There are also many many RDFizers that already exist out on the web (lists of examples here and here). Ideally shims can be created that use these to generate RDF content, augment it with whatever useful data can be found, and then pass it to the RO creator. The beauty of the RO creator is that you will be able to unify all of these separate pieces of RDF – Once added to the tool, with RDF generated, these resources will be able to be linked with other resources on the form. For example, if a picture is uploaded and RDF if generated on the picture, and it happens to be a picture of a person who’s profile is added on the RO builder’s UI, you will be able to link the person and the photo as one being the depiction of the other, and when the RO is created, triples that express this relationship will be added to the person’s resource.
CampusROAR: A Simple Python Crawler
Over the last week I’ve been building a simple Python web crawler for grabbing RSS feeds from a domain. The one I built in Bash using command line tools (namely wget) has several downsides which needed fixing so I started work on a customized Python one. Wget did not allow filtering of pages by HTTPHeader/mimetype, and while it did allow filtering by file extension, many of the files on the domain I was scraping were extensionless so couldn’t be filtered. There was also no way to filter by filesize – so I couldn’t restrict it to smaller files instead which would have been an ok solution. Instead the crawler had to download huge files on occasion which either had no file extension or was one I had not thought to blacklist.
The Python crawler I created had several advantages therefore. It first checks the HTTPHeader, and only requests the full page if it’s HTML or XML (to either get all the links off of, or to check for RSS in the case of XML). It then parses and processes the full page and gets the links from it, or if it’s an RSS or ATOM feed, adds it to a list of them. You have to give it a starting page and a domain restriction – it starts crawling from one and cannot leave the given domain. If you did not give it a domain restriction it would probably continue forever (or at least until you had spidered the whole internet).
SLE: Coursework Status and Hand-in
Among the highest demanded widgets for the SLE (at least, according to our survey) was a method of viewing the status of outstanding pieces of coursework and being able to submit completed works electronically. As with all projects, before any coding was undertaken, wireframes were drawn up to give an idea of the content of the widget.
The wireframe above was made in Pencil (free and open-source GUI prototyping software) and shows the initial draft of the widget. From this, a mockup was created in HTML (shown below). It has limited functionality and styling, but enough to give the illusion that it works for example, although the dates displayed are hard coded into the prototype, the widget itself can be dragged and dropped around the page.
With luck, there was a simple starting point to the implementation of this widget as ECS maintains its own electronic hand-in system called C-BASS. This is managed by Christopher Gutteridge, who very kindly compiled the publicly available information (such as module code, due date, hand-in url) into an XML document.
An implementation is shown below – however, this is still in development and some things are liable to change).
As can be seen from the screenshot above, the course code links to the module hompage and the assignment title links to the appropriate description (if one is supplied). The “Go to hand-in” link directs the user to the submission page for that particular assignment. The date is dynamically checked against today’s date – if the coursework is overdue, it is highlighted red, if there is less than a week until the hand-in date it is highlighted yellow.
