SUMS – Social and everything else

The candidate technologies discussed for the implementation were Java EE and ASP.NET. Both of these technologies provide the tools needed to build our application.

Java EE

• Open source.
• Supports many applications servers which are open source (Tomcat, GlassFish, Oracle Applications Server, WebLogic, etc.).
• Supports web application frameworks, such as Struts 2, which might be used to adopt MVC.
• Supports ORM tools such as TopLink, EclipseLink, Hibernate, etc.
• Support template mechanisms, using Struts 2 tiles.
• Strong typed object oriented language.
• Easy support for REST using Apache CXF.

ASP.NET

• Proprietary.
• Provides only one application server (IIS).
• ASP.NET application framework includes a full MVC architecture which might be used.
• Supports ORM tools such as ADO.NET Entity Framework, LINQ to SQL, etc.
• Supports build-in template mechanism using MVC v2.
• Strong typed object oriented language.
• There are not many libraries that fully support the REST architecture — only partial.

Decision

First of all, since all of the supported social networks support the REST architecture, we preferred using Java EE over ASP.NET. Secondly, all of the team members are more familiar with Java EE and the aforementioned technologies, than ASP.NET.

There are no doubts that Facebook is the most known social network in the world. With more than 600 million monthly active users ^[1], its site is the second most visited site on the web, after Google. Facebook’s user base is therefore important for every company that addresses itself to the public in order to test reactions, analyse opinions and try to understand the customer base. This is why Facebook is one of the social networks considered for our application.

Graph API

The Graph API is the core of the Facebook Platform, enabling developers to read and write data to Facebook. All objects – users, posts, comments, images, likes, connections, etc. – are represented as the nodes of a graph. The API provides two ways of browsing objects, depending on their privacy settings

• unauthenticated browsing of the social graph for publicly available information
• authenticated requests for objects which have a higher privacy level

Since our web application’s purpose is to unify what users post on different social networks about different topics, we are mostly interested in being able to query Facebook for the posts that users publish and then to analyse the reactions.

The Graph API is a very simple API that works on a request – response basis, returning a JSON object for each call. The format of a request for searching over public objects is the following:

https://graph.facebook.com/search?q=QUERY&type=OBJECT_TYPE

where the OBJECT_TYPE can be post, user, page, event, group, place or checkin.

For example to browse all public posts that mention University of Southampton, the query should look like:

https://graph.facebook.com/search?q=university%20of%20southampton&type=post

One of the results looks like:

{
   "data": [
      {
         "id": "160003040686019_212151782143338",
         "from": {
            "name": "Hartley Library",
            "category": "Library",
            "id": "160003040686019"
         },
         "message": "We have some new photos of Hartley Library in our main facebook page (http://www.facebook.com/UniSotonLibrary?sk=photos).",
         "picture": "http://profile.ak.fbcdn.net/hprofile-ak-snc4/188016_171854866189717_7800009_n.jpg",
         "link": "https://www.facebook.com/UniSotonLibrary",
         "name": "University of Southampton Library",
         "description": "The Library is a dynamic service, and is continually adding resources and developing services to support the teaching and research of the University.",
         "properties": [
            {
               "name": "Page",
               "text": "197 people like this."
            }
         ],
         "icon": "http://b.static.ak.fbcdn.net/rsrc.php/v1/yD/r/aS8ecmYRys0.gif",
         "type": "link",
         "created_time": "2011-03-14T15:32:37+0000",
         "updated_time": "2011-03-14T15:32:37+0000"
      }
   ],
   "paging": {
      "previous": "https://graph.facebook.com/search?q=university+of+southampton&type=post&limit=25&since=1304436757",
      "next": "https://graph.facebook.com/search?q=university+of+southampton&type=post&limit=25&until=1304436757"
   }
}

Since every object has an ID and the Graph API allows querying the public data of each object, the response can be parsed and more details can be obtained. In case the user would like to query information that is private, he will have to connect with Facebook through our application and then the application can also provide details about private objects.

Twitter allows developers to create applications that integrate with the Twitter platform and use its services. It provides an API that has 3 parts:

• REST API
• Search API
• Streaming API, which is the latest one and allows long connections

Twitter REST API

The REST API is used by Twitter to expose their data as services. Developers can perform HTTP GET requests to obtain data like statuses and user information and HTTP POST requests to add data to Twitter, in case that they develop third-party clients.

Twitter Search API

The Twitter Search API is sued by develop to connect with the Twitter Search facility [1] and query the system for recent Tweets.

Streaming API

The Streaming API allows developers to perform real-time searches for Tweets. They construct queries and filters to make the search more specific.

Twitter into our System

In the scope of the Real-Time Aggregator system, Twitter REST API is more than enough. REST Architecture allows client to perform HTTP requests (POST, PUT, and GET). According to the REST Architecture [2], HTTP GET requests can retrieve resources from the Twitter system. Every strong-typed object oriented language, including Java, provides HTTP libraries that can be used to make such calls. The table below describes some of the most important GET requests that the Real-Time Aggregator will perform.

More information about resources and calls, together with information on how to consume the services can be found on the official Twitter API documentation [3].

Issues to bear in mind

Twitter Rest API has a usage limit and it sometimes constrains users to perform a temperate number of service requests. The limit for not authenticated users is 150 HTTP GET requests per hour, while the calls of authenticated users can be up to 350 HTTP GET requests per hour. The scope of our system contains only unauthenticated HTTP GET requests, performed to retrieve Twitter content. Although 150 requests per hour seem many, the limit constraints our real-time aggregator. In order to avoid any problems, there should be an inactivity period between the asynchronous requests made by our system. In cases that users reach the usage limit, the requests return an HTTP 400 error code (Bad request). The system shall be able to “catch” the error code and inform users.

Twitter Search API has some further limitations, which are mainly due to the date of the tweets that they return and to the complexity of the searching queries. The search queries return tweets, which are not older than a week.  It also important that searching calls are not limited to a number of allowed requests. However, if a query is too complex, then an HTTP 420 error is returned informing the programmers about the wrong execution of the query. The system shall be able to “catch” the error code and inform users.

References

[1] http://search.twitter.com/

[2] http://www.ibm.com/developerworks/webservices/library/ws-restful/

[3] https://dev.twitter.com/doc

LinkedIn allows developers to extend their site or application by using the LinkedIn REST API. The integration itself is very straightforward. You just need an API Key, and some basic principles of the REST API [1]. In addition to that, the JavaScript API acts as a bridge between the user and the REST API, and gives you a way to interact with the the REST API without actually making calls — they requests are automatically translated into a REST call via Ajax.

REST API

Below you find the resources available for the LinkedIn API, as well as some basic description for each one of them.

• Profile
• Connections
• Search
• Messaging
• Network Updates

In this architecture, every resource is access via an HTTP GET request. The Profile resource provides all the information that is available for a user, based on selectors (user id or profile url). In order to get further information about a user’s network (connections, activity, etc.), Connections resource provides you with details regarding user’s connection. Searching through the LinkedIn network is achieved using the Search resource, which matches the criteria provided. Messaging resource allows users to send and receive private messages, as well as invitations to connect. Please note that, Messaging requires HTTP POST requests rather than GET. To post updates, receive updates, get comments and/or likes, you have to use the Network resource. Every HTTP request (either POST or GET), returns an XML with the appropriate data requested.

Request Examples

Search for people with keywords anargyros akrivos:

HTTP GET http://api.linkedin.com/v1/people-search?keywords=[anargyros akrivos]

Get profile information for user akrivos:

HTTP GET http://api.linkedin.com/v1/people/id=akrivos

Get akrivos‘ connections:

HTTP GET http://api.linkedin.com/v1/people/id=akrivos/connections

Getting updates for Vanessa Hughes

HTTP GET http://api.linkedin.com/v1/people/id=QpK8ApA59n/network/updates?scope=self

Response Example

<updates total="250" start="0" count="10">
  <update>
    <timestamp>1265688025372</timestamp>
    <update-key>STAT-11737890-270-*1</update-key>
    <update-type>STAT</update-type>
    <update-content>
      <person>
        <id>QpK8ApA59n</id>
        <first-name>Vanessa</first-name>
        <last-name>Hughes</last-name>
        <headline>Certified eBay Trading Asst & Power/TOP Seller of ANTIQUE WESTERN ITEMS</headline>
        <current-status>I new I had a good plan & then I get this...Seller Special: 50% OFF SALE2 days only! Feb 9-10 #Ebay</current-status>
        <picture-url>http://media.linkedin.com/mpr/mprx/0__by8vxfjBizcc5a135g1vpgtv6tBz_a13Le0vjpGEiQboFJPfTuuRge2Ue-enkf0CkYYssC1AxTS</picture-url>
        <api-standard-profile-request>
          <url>http://api.linkedin.com/v1/people/QpK8ApA59n:full</url>
          <headers total="1">
            <http-header>
              <name>x-li-auth-token</name>
              <value>name:D6nh</value>
            </http-header>
          </headers>
        </api-standard-profile-request>
        <site-standard-profile-request>
          <url>http://www.linkedin.com/profile?viewProfile=&key=11737890&authToken=D6nh&authType=name&trk=api*a102812*s1397*</url>
        </site-standard-profile-request>
      </person>
    </update-content>
    <is-commentable>true</is-commentable>
  </update>
  ...
</updates>

JavaScript API

Using JavaScript API [2] unleashes the full power of LinkedIn REST API. Assuming that you have the API Key, you just need to include a JavaScript provided by LinkedIn as follows:

<script type="text/javascript" src="http://platform.linkedin.com/in.js">
    api_key: YOUR-API-KEY
    onLoad: onLinkedInLoad
    authorize: true
</script>

The IN object is the basic JavaScript object for accessing the JavaScript API. Every function call begins with IN.

Issues to bear in mind

The basic issue is that many LinkedIn API calls require that the user is authenticated, making user authorisation a necessary evil, before making any call to the API.

References

[1] http://developer.linkedin.com/docs/DOC-1258
[2] http://developer.linkedin.com/docs/DOC-1206