SUMS – Social and everything else

Twitter allows developers to create applications that integrate with the Twitter platform and use its services. It provides an API that has 3 parts:

• REST API
• Search API
• Streaming API, which is the latest one and allows long connections

Twitter REST API

The REST API is used by Twitter to expose their data as services. Developers can perform HTTP GET requests to obtain data like statuses and user information and HTTP POST requests to add data to Twitter, in case that they develop third-party clients.

Twitter Search API

The Twitter Search API is sued by develop to connect with the Twitter Search facility [1] and query the system for recent Tweets.

Streaming API

The Streaming API allows developers to perform real-time searches for Tweets. They construct queries and filters to make the search more specific.

Twitter into our System

In the scope of the Real-Time Aggregator system, Twitter REST API is more than enough. REST Architecture allows client to perform HTTP requests (POST, PUT, and GET). According to the REST Architecture [2], HTTP GET requests can retrieve resources from the Twitter system. Every strong-typed object oriented language, including Java, provides HTTP libraries that can be used to make such calls. The table below describes some of the most important GET requests that the Real-Time Aggregator will perform.

More information about resources and calls, together with information on how to consume the services can be found on the official Twitter API documentation [3].

Issues to bear in mind

Twitter Rest API has a usage limit and it sometimes constrains users to perform a temperate number of service requests. The limit for not authenticated users is 150 HTTP GET requests per hour, while the calls of authenticated users can be up to 350 HTTP GET requests per hour. The scope of our system contains only unauthenticated HTTP GET requests, performed to retrieve Twitter content. Although 150 requests per hour seem many, the limit constraints our real-time aggregator. In order to avoid any problems, there should be an inactivity period between the asynchronous requests made by our system. In cases that users reach the usage limit, the requests return an HTTP 400 error code (Bad request). The system shall be able to “catch” the error code and inform users.

Twitter Search API has some further limitations, which are mainly due to the date of the tweets that they return and to the complexity of the searching queries. The search queries return tweets, which are not older than a week.  It also important that searching calls are not limited to a number of allowed requests. However, if a query is too complex, then an HTTP 420 error is returned informing the programmers about the wrong execution of the query. The system shall be able to “catch” the error code and inform users.

References

[1] http://search.twitter.com/

[2] http://www.ibm.com/developerworks/webservices/library/ws-restful/

[3] https://dev.twitter.com/doc

After giving a first description of our system summarising its requirements, we aim now to provide a more formal narrative description of them. The formal description separates the requirements in functional and non-functional according to their nature.

Functional Requirements

The table below lists the functional requirements of our system, indicating the core, extended and advanced features of our system. Every requirement is assigned a unique id that can be used to identify it. The functionality level indicates the stage of the development that the requirements shall be addressed.

Non-Functional Requirements

The table below defines the non-functional requirements of the system, paying special attention to usability and performance issues. Every requirement is assigned a unique id that can be used to identify it. The functionality level indicates the stage of the development that the requirements shall be addressed.

Lots of ideas came up to surface, making the whole project analysis very interesting. Finally we managed to agree upon a more concrete idea that is actually what we have already mentioned in one of our previous posts – but now we have more details at hand.

Idea

The main idea is to develop a real-time social aggregator in which the user will be given the opportunity to search for a set of keywords across the social networks that are supported in our application. We separated the application’s features into three categories: core, extended, advanced features.

Features list

Core

• Social Networks
  Our application will initially support Twitter, Facebook and LinkedIn.
• Login System
  In order to use the services that our application provides, the user has to log in on our  website. We will be supporting OpenID, as well as creating an entirely new account on our website. In both cases, the user will be given the option to link his/her account with the social networks that we support (the user has to have active accounts on the supported social networks).
• Search System
  The application will feature a Google-like appearance regarding its user interface (extremely simple to use and practical), which will be available in two modes: simple or advanced. Although the design of the search input will be simple, the mechanism itself will be very sophisticated, thus resulting in high quality information analysis.
• Tag Cloud
  There will be an interactive tag cloud of the most popular search keywords so that a user can click on a keyword and automatically search.

Extended

• Results Analysis
  The results will be analysed in many possible and interesting ways so that the user can interact with our application and change the way the results are ranked and appeared. We will support results analysis based on: quantity (number of posts/mentions/likes/comments/etc.), date (old/new), social network (apply filters), and area of poster (location-awareness if applicable).
• Advanced Search
  Through the advanced search, the user will be able to select a specific date/time interval, geographic location, social network connection type (friend/friend of a friend/anybody), as well as filter results from a specific social network.

Advanced features

• Graphs
  For each searched keyword, a graph will be featuring (see the Mockups) the number of posts (tweets, wall posts, etc.) in which the keyword was found along different social networks and across a timespan.
• Search History
  The application will keep a search history for statistical purposes – generating tag clouds, graphs, charts, etc.
• Determine Results Quality
  We were thinking that it would be really interesting if the application could determine whether the meaning of a result is positive or negative. Our application will support a way of categorising an information as positive or negative, whether it will be a semantic text analysis or a simple keyword search.
• Suggest “expert” or “amateur” users
  Another interesting feature would be to further analyse the search results and measure the level of relevance a user has to do with the topic (expert, amateur, etc.). Then we can assume that an expert’s opinion on a subject weights more than an amateur’s opinion.

Mockups

Social networks have filled up the net in one form or another. Almost everybody around you might want to add you on Facebook, follow you on Twitter, share a picture with you on Flickr or Picasa, have you as a connection on LinkedIn and the examples could carry on endlessly.

If the social networks were initially only for individuals, we clearly now see that companies start to migrate their communication efforts this way as well. The advantages are numerous:

• there are no costs involved for joining a social network (at least not for the core features they provide)
• the company gets instant feedback from its customers (or potential customers)
• it helps the company to become more “customer oriented” and (in some cases) allows the customers to be involved in the making of a new product
• it might offer a company insights regarding the geographical areas where it has a bigger customer base
• it might help observe the competitors’ impact on certain markets and in different areas

However, if the metrics for the WEB 1.0 era when a company’s digital efforts were based only on its website were given by the number of visitors per day (and other factors as number of visited pages, average time for a visit, number of pages viewed per visit, etc.), there aren’t any WEB 2.0 metrics which could show to a marketing director what’s the impact of the company he/she works for. This is partly because companies’ use of social networks is a relatively new marketing technique and also because, given the fact that there is a multitude of social networks, there is no way to easily unify the social graphs they provide for a certain user and the impact a user has made globally.

This is where our project aims to make a change. By using the APIs provided by some of the most known social networks, we would like to develop a tool (we still don’t know its final form – web or desktop application) that would allow a user (be it an individual or a company) to analyse the connections and interactions between these social networks and also provide him/her with relevant data regarding:

• geographic location of “customers”/”fans”
• impact of a launched product or service (by analysing how many people have talked about it on different networks)
• products/services launched by competitors
• users’/customers’ profile

Throughout the posts that will follow you will be able to read about how our project evolves over time and how we will reach to a product (the application) from a sum of a ideas.

Keep calm and carry on!