SUMS – Social and everything else

Throughout this portfolio we:

• Suggested ideas and topics, and agreed to Real-time Social Aggregator
• Researched relevant work
• Defined the goals and the requirements
• Explored the Social aspects of privacy and trust
• Analysed the the Economical aspects, choosing a Freemium strategy
• Formed a formal definition
• Analysed the system
• Designed the system
• Investigated relevant technologies and APIs (Facebook, Twitter, LinkedIn)
• Familiarised with the candidate technologies for the implementation, choosing Java EE
• … are ready to implement our idea!

We have researched, explored and investigated every aspect concerning our system, and we concluded that our idea is indeed feasible. The only thing missing is the implementation, and that’s due to the limited manpower — only 3 people in this group).

Our next step is the implementation of our system, and distribute it to some companies for testing. The next phase would be to improve based upon the feedback received, and fix any bugs. You realise that this leads towards a fully functional system being released, which has been tested by companies, including their features and suggestions.

We are looking forward to the next step…

The candidate technologies discussed for the implementation were Java EE and ASP.NET. Both of these technologies provide the tools needed to build our application.

Java EE

• Open source.
• Supports many applications servers which are open source (Tomcat, GlassFish, Oracle Applications Server, WebLogic, etc.).
• Supports web application frameworks, such as Struts 2, which might be used to adopt MVC.
• Supports ORM tools such as TopLink, EclipseLink, Hibernate, etc.
• Support template mechanisms, using Struts 2 tiles.
• Strong typed object oriented language.
• Easy support for REST using Apache CXF.

ASP.NET

• Proprietary.
• Provides only one application server (IIS).
• ASP.NET application framework includes a full MVC architecture which might be used.
• Supports ORM tools such as ADO.NET Entity Framework, LINQ to SQL, etc.
• Supports build-in template mechanism using MVC v2.
• Strong typed object oriented language.
• There are not many libraries that fully support the REST architecture — only partial.

Decision

First of all, since all of the supported social networks support the REST architecture, we preferred using Java EE over ASP.NET. Secondly, all of the team members are more familiar with Java EE and the aforementioned technologies, than ASP.NET.

There are no doubts that Facebook is the most known social network in the world. With more than 600 million monthly active users ^[1], its site is the second most visited site on the web, after Google. Facebook’s user base is therefore important for every company that addresses itself to the public in order to test reactions, analyse opinions and try to understand the customer base. This is why Facebook is one of the social networks considered for our application.

Graph API

The Graph API is the core of the Facebook Platform, enabling developers to read and write data to Facebook. All objects – users, posts, comments, images, likes, connections, etc. – are represented as the nodes of a graph. The API provides two ways of browsing objects, depending on their privacy settings

• unauthenticated browsing of the social graph for publicly available information
• authenticated requests for objects which have a higher privacy level

Since our web application’s purpose is to unify what users post on different social networks about different topics, we are mostly interested in being able to query Facebook for the posts that users publish and then to analyse the reactions.

The Graph API is a very simple API that works on a request – response basis, returning a JSON object for each call. The format of a request for searching over public objects is the following:

https://graph.facebook.com/search?q=QUERY&type=OBJECT_TYPE

where the OBJECT_TYPE can be post, user, page, event, group, place or checkin.

For example to browse all public posts that mention University of Southampton, the query should look like:

https://graph.facebook.com/search?q=university%20of%20southampton&type=post

One of the results looks like:

{
   "data": [
      {
         "id": "160003040686019_212151782143338",
         "from": {
            "name": "Hartley Library",
            "category": "Library",
            "id": "160003040686019"
         },
         "message": "We have some new photos of Hartley Library in our main facebook page (http://www.facebook.com/UniSotonLibrary?sk=photos).",
         "picture": "http://profile.ak.fbcdn.net/hprofile-ak-snc4/188016_171854866189717_7800009_n.jpg",
         "link": "https://www.facebook.com/UniSotonLibrary",
         "name": "University of Southampton Library",
         "description": "The Library is a dynamic service, and is continually adding resources and developing services to support the teaching and research of the University.",
         "properties": [
            {
               "name": "Page",
               "text": "197 people like this."
            }
         ],
         "icon": "http://b.static.ak.fbcdn.net/rsrc.php/v1/yD/r/aS8ecmYRys0.gif",
         "type": "link",
         "created_time": "2011-03-14T15:32:37+0000",
         "updated_time": "2011-03-14T15:32:37+0000"
      }
   ],
   "paging": {
      "previous": "https://graph.facebook.com/search?q=university+of+southampton&type=post&limit=25&since=1304436757",
      "next": "https://graph.facebook.com/search?q=university+of+southampton&type=post&limit=25&until=1304436757"
   }
}

Since every object has an ID and the Graph API allows querying the public data of each object, the response can be parsed and more details can be obtained. In case the user would like to query information that is private, he will have to connect with Facebook through our application and then the application can also provide details about private objects.

Twitter allows developers to create applications that integrate with the Twitter platform and use its services. It provides an API that has 3 parts:

• REST API
• Search API
• Streaming API, which is the latest one and allows long connections

Twitter REST API

The REST API is used by Twitter to expose their data as services. Developers can perform HTTP GET requests to obtain data like statuses and user information and HTTP POST requests to add data to Twitter, in case that they develop third-party clients.

Twitter Search API

The Twitter Search API is sued by develop to connect with the Twitter Search facility [1] and query the system for recent Tweets.

Streaming API

The Streaming API allows developers to perform real-time searches for Tweets. They construct queries and filters to make the search more specific.

Twitter into our System

In the scope of the Real-Time Aggregator system, Twitter REST API is more than enough. REST Architecture allows client to perform HTTP requests (POST, PUT, and GET). According to the REST Architecture [2], HTTP GET requests can retrieve resources from the Twitter system. Every strong-typed object oriented language, including Java, provides HTTP libraries that can be used to make such calls. The table below describes some of the most important GET requests that the Real-Time Aggregator will perform.

More information about resources and calls, together with information on how to consume the services can be found on the official Twitter API documentation [3].

Issues to bear in mind

Twitter Rest API has a usage limit and it sometimes constrains users to perform a temperate number of service requests. The limit for not authenticated users is 150 HTTP GET requests per hour, while the calls of authenticated users can be up to 350 HTTP GET requests per hour. The scope of our system contains only unauthenticated HTTP GET requests, performed to retrieve Twitter content. Although 150 requests per hour seem many, the limit constraints our real-time aggregator. In order to avoid any problems, there should be an inactivity period between the asynchronous requests made by our system. In cases that users reach the usage limit, the requests return an HTTP 400 error code (Bad request). The system shall be able to “catch” the error code and inform users.

Twitter Search API has some further limitations, which are mainly due to the date of the tweets that they return and to the complexity of the searching queries. The search queries return tweets, which are not older than a week.  It also important that searching calls are not limited to a number of allowed requests. However, if a query is too complex, then an HTTP 420 error is returned informing the programmers about the wrong execution of the query. The system shall be able to “catch” the error code and inform users.

References

[1] http://search.twitter.com/

[2] http://www.ibm.com/developerworks/webservices/library/ws-restful/

[3] https://dev.twitter.com/doc