Jun 122015
 

My general approach to extracting data from any API is to extract the data into a relational database and then write SQL queries on top of it to get the required information. Though this works, its often tedious and involves running multiple applications.

The R programming language works great for statistical computation and plotting graphics and I have been tinkering around with it for the last few weeks. While learning R, I thought of using R to extract data from the API as well. This would allow extracting the latest data from the API and compute stats with a single script. And though the XML package in R doesn’t make for the most intuitive parsing code, the vectorized operations reduces the need for frequent loops and keeps the code concise and readable.

And though this code is written for the Socialcast API, it can be easily tweaked to pull data from any social API like Facebook, Yammer etc. The first step is to pull the data from the API – the RCurl package gets us the data which can then be parsed using the XML package.

rm(list=ls())
library(RCurl)
library(XML)

continueLoading=TRUE
page = 1
finalDataFrame <- NULL

getInnerText <- function(inputData,parentNode,childNode) {
  test <- xpathSApply(inputData,parentNode,function(x){
    if(is.null(x[childNode][[childNode]])) {
      ""
    }else {
      xmlValue(x[childNode][[childNode]])
    }
  })
  return(test)
}

while(continueLoading) {

  messagesData <- getURL(paste("https://demo.socialcast.com/api/groups/acmecorpsoftballteam/messages.xml?per_page=20&page=",page,sep=""),
                         userpwd="emily@socialcast.com:demo", ssl.verifypeer = FALSE, httpauth = 1L)
  print(paste("LOADING PAGE:",page))
  data <- xmlParse(messagesData)
  totalMessages <- length(getNodeSet(data,"//messages/message"))

The totalMessages property is to check the number of messages returned by the API. When it’s zero, the while loop is exited, else the execution continues. The xmlParse function gives us a in memory structure of the document which can be iterated upon. we use the sapply function which applies a function to each element of a list and returns a vector. I’ll come to the getUserNodeValue function later

if (totalMessages == 0){
    continueLoading = FALSE
  }
  else {
    tempDataFrame <- data.frame(
      InteractionType = "Message",
      ID = sapply(getNodeSet(data, "//messages/message/id"),xmlValue),
      Author = sapply(getNodeSet(data,"//messages/message/user/name"),xmlValue),
      Body = sapply(getNodeSet(data,"//messages/message/body"),xmlValue),
      Url = sapply(getNodeSet(data,"//messages/message/permalink-url"),xmlValue),
      Type = sapply(getNodeSet(data,"//messages/message/message-type"),xmlValue),
      CreatedAt = sapply(getNodeSet(data,"//messages/message/created-at"),xmlValue),
      Location = sapply(getNodeSet(data,"//messages/message/user/id"),function(x){getUserNodeValue(x,"Location")}),
      Country = sapply(getNodeSet(data,"//messages/message/user/id"),function(x){getUserNodeValue(x,"Country")}),
      Sector = sapply(getNodeSet(data,"//messages/message/user/id"),function(x){getUserNodeValue(x,"Sector")}),
      Title = sapply(getNodeSet(data,"//messages/message/user/id"),function(x){getUserNodeValue(x,"Title")}),
      Department = sapply(getNodeSet(data,"//messages/message/user/id"),function(x){getUserNodeValue(x,"Department")})
    )

    if (is.null(finalDataFrame)) {
      finalDataFrame <- tempDataFrame
    }
    else{
      finalDataFrame <- rbind(finalDataFrame,tempDataFrame)
    }

Now we have a data frame with all the Messages from the API. However, we also need the comments and likes. This is the only place where I needed to use a for loop to iterate through each individual message node and select their comments. The xpathSApply function reduces our code further by being able to query each node of the NodeSet with the given XPath expression and applying a function on it. Furthermore it returns a vector which fits in nicely into our existing data frame.

   for( i in 1:length(getNodeSet(data,"//messages/message"))) {
      if(length(getNodeSet(data,paste("//messages/message[position()=",i,"]/comments/comment"))) > 0){

        allComments <- getNodeSet(data,paste("//messages/message[position()=",i,"]/comments"))[[1]]

        xpathSApply(allComments,"comment/id",xmlValue)

        commentFrame <-  data.frame(
          InteractionType = "Comment",
          ID = xpathSApply(allComments,"comment/id",xmlValue),
          Author = xpathSApply(allComments,"comment/user/name",xmlValue),
          Body = xpathSApply(allComments,"comment/text",xmlValue),
          Url = xpathSApply(allComments,"comment/permalink-url",xmlValue),
          Type = "",
          CreatedAt = xpathSApply(allComments,"comment/created-at",xmlValue),
          Location = xpathSApply(allComments,"comment/user/id",function(x){getUserNodeValue(x,"Location")}),
          Country = xpathSApply(allComments,"comment/user/id",function(x){getUserNodeValue(x,"Country")}),
          Sector = xpathSApply(allComments,"comment/user/id",function(x){getUserNodeValue(x,"Sector")}),
          Title = xpathSApply(allComments,"comment/user/id",function(x){getUserNodeValue(x,"Title")}),
          Department = xpathSApply(allComments,"comment/user/id",function(x){getUserNodeValue(x,"Department")})
        )

        finalDataFrame <- rbind(finalDataFrame,commentFrame)
      }

      if(length(getNodeSet(data,paste("//messages/message[position()=",i,"]/likes/like"))) > 0){

        allLikes <- getNodeSet(data,paste("//messages/message[position()=",i,"]/likes"))[[1]]

        likeFrame <-  data.frame(
          InteractionType = "Like",
          ID = xpathSApply(allLikes,"like/id",xmlValue),
          Author = xpathSApply(allLikes,"like/user/name",xmlValue),
          Body = "",
          Url = "",
          Type ="",
          CreatedAt = xpathSApply(allLikes,"like/created-at",xmlValue),
          Location = xpathSApply(allLikes,"like/user/id",function(x){getUserNodeValue(x,"Location")}),
          Country = xpathSApply(allLikes,"like/user/id",function(x){getUserNodeValue(x,"Country")}),
          Sector = xpathSApply(allLikes,"like/user/id",function(x){getUserNodeValue(x,"Sector")}),
          Title = xpathSApply(allLikes,"like/user/id",function(x){getUserNodeValue(x,"Title")}),
          Department = xpathSApply(allLikes,"like/user/id",function(x){getUserNodeValue(x,"Department")})
        )

        finalDataFrame <- rbind(finalDataFrame,likeFrame)
      }
    }

  }
  page <- page + 1

}

rm(list=c("commentFrame","likeFrame","tempDataFrame","users"))

Now we come to the getNodeUserValue function. This is simply a performance optimization since calling the API to get the user details each time becomes very time consuming. So I generally keep the user data in a database and use the id in the xml response to query the data frame and fetch the correct user record. This step however is purely optional and you could easily call the api to get each user’s response and parse it.

getUserNodeValue <- function(inputNode,queryNode){
  if (nrow(users[users$ID == xmlValue(inputNode),]) == 0)
    ""
  else
    users[users$ID == xmlValue(inputNode),][[queryNode]]
}

At this point we have all the API information parsed into a data frame (finalDataFrame). Now for the fun part! Though you can subset and count easily using the built in language functions, a package called dplyr makes this code more readable and intuitive. With dplyr you can perform multiple data manipulation operations like filter, select, order, group by etc and chain them together to get the final result

So to group the data frame by a column and count, the code is as simple as

#############Type of Activity in the Group#####################################

interactionType <- group_by(finalDataFrame,InteractionType) %>%
                   summarise(count = n())

print(interactionType)

activityType

#############Active Day of Week#############################

activeDay <- group_by(finalDataFrame,weekdays(as.Date(CreatedAt))) %>%
 summarise(count = n()) %>%
 select(DayOfWeek=1,count=2)

print(activeDay)

DayOfWeek
The top 5 users by total Activity

activeUsers <- group_by(finalDataFrame,Author) %>%
 summarize(TotalActivity=n()) %>%
 arrange(-TotalActivity) %>%
 top_n(5,TotalActivity)

print(activeUsers)

The Type of Messages being created

messageTypes <- filter(finalDataFrame,InteractionType == "Message") %>%
 group_by(Type) %>%
 summarize(count = n()) %>%
 arrange(-count)

print(messageTypes)

The stats shown here barely scratch the surface of what R is capable of computing.

Dec 212012
 

In my previous post, I blogged about how to access the Socialcast community data without using the API. This is usually necessary when the API doesnt support any particular functionality which is provided by the site.

This is true of the usecase of updating of the user’s profile avatar. Though there is a way to update the user profile in the API, but there is no obvious method of updating the user’s avatar. I asked Socialcast on twitter, but they didn’t answer so I went ahead with trying to use Mechanize to login to the site.

I was finally able to update the profile avatar using the below script. Works like a charm.

require 'Mechanize'
agent = Mechanize.new
agent.agent.http.verify_mode = OpenSSL::SSL::VERIFY_NONE
agent.get("https://demo.socialcast.com/login")
form = agent.page.forms.first
puts "Please enter user email id"
form.email = gets.chomp
puts "Please enter password. caution: it is not masked"
form.password= gets.chomp
form.submit
puts "Please enter username"
agent.get ("https://demo.socialcast.com/users/emilyjames/edit")
form = agent.page.forms.detect{ |f| f.file_upload_with(:name => "profile_photo[data]") }
puts "Please enter file path of the image to replace"
form.file_uploads.first.file_name = gets.chomp
form.submit

Dec 212012
 

The Socialcast REST API provides programmatic access to the Socialcast community data with XML and JSON endpoints. The API provides most of the information one would require to extract out of the site but there are still gaps where the API is not up to date.

This made me look into the possibility of scraping the site directly using cUrl and parsing the generated HTML. However Socialcast is built on Rails and has a security feature which prevents cross site request forgery, using an authenticity token which is a random token generated and sent with every request embedded in a hidden form field. When the form is posted back, this token is checked and an error generated if it’s not found. This makes direct scraping of the page difficult and cUrl fails. Googling gave me a few articles which specified how to use cUrl with sites protected with the authenticity token (Link1, Link2) but unfortunately none of them seemed to work.

Then I came across a suggestion to use Mechanize, a ruby library to automate interaction with websites. Mechanize works like a charm with sites protected by an authenticity token. Here is the ruby script to login to the Socialcast Demo site.

require 'Mechanize'
agent = Mechanize.new
agent.agent.http.verify_mode = OpenSSL::SSL::VERIFY_NONE
agent.get("https://demo.socialcast.com/login")
form = agent.page.forms.first
form.email = "emily@socialcast.com"
form.password= "demo"
form.submit

In Interactive Ruby, we can see that the authenticity token is returned when the first GET is called on the login page. And when the form is submitted the token is posted back to the server and we are redirected to the home page.

login

From here on, we can automate any interaction with the site just as a normal user would do without worrying about the authenticity token restriction. In my next post, I will explain how to automatically update a user’s avatar without relying on the API

Jul 082012
 

I did some changes to the plugin in the last post – including an option in the admin dashboard to enable/disable shortcodes and the ability to directly add buttons to the post without having to modify the theme php file. Future changes planned are to allow admins to add the widgets through the options screen as well.

The plugin is available here and the code is below. Caution: Some more testing is required

<?php
/**
 * Returns major.minor WordPress version.
 */
function sc_reach_get_wp_version() {
  return (float) substr(get_bloginfo('version'), 0, 3);
}

function sc_add_author_stream($email, $style='width:300px;height:400px') { 
  return get_div_email($email, 'profile_container_id', $style, get_option('sc_profile_token'));
}

function get_div_email($email, $id, $style, $token) {
	$socialcast_url = get_option('sc_host');
	if ($id != '' && $token != '') {
		return '<div id="' . $id . '" style="' . $style .
		'"></div><script type="text/javascript">_reach.push({container: "' . $id . '", domain: "https://' 
		. $socialcast_url . '", token: "' . $token . '", email:"'. $email . '"});</script>';
	} else {
		return '';
	}
}

function sc_reach_content_handle($content, $sidebar = false) {
	
	switch(get_option('sc_show_button'))
	{
		case 'Dont Show':
			break;
		case 'Top':
			$content =  sc_add_button() . $content;
			break;
		case 'Bottom':
			$content = $content . sc_add_button();
			break;
	}
	
	if (get_option('sc_use_microdata') == 'true') {
		$purl = get_permalink();
   		$content = "<div itemscope=itemscope itemtype=\"http://ogp.me/ns#Blog\"><a itemprop='url' href='" . $purl . "' /></a>" . $content . "</div>";
	}

	return $content;
}

function get_div($id, $style, $token) {
	$socialcast_url = get_option('sc_host');
	if ($id != '' && $token != '') {
		return '<div id="' . $id . '" style="' . $style .
		'"></div><script type="text/javascript">_reach.push({container: "' . $id . '", domain: "https://' 
		. $socialcast_url . '", token: "' . $token . '"});</script>';
	} else {
		return '';
	}
}

/* Name: Ganesh Ranganathan 
   Date: 7th July 2012
This function has been added to generate the div from the shortcut
which can be added to any text widget or in the post itself. It includes
an additional parameter - display which lets users insert shortcodes without
knowing the token as long as they are specified in the Plugin options screen.
Please make sure they are do_shortcode is called in your theme for widget text if 
you want to include this in a widget */
function get_shortcode_div($id,$style,$token,$display){
 $tokenInOptions ='';
  if($display != '')
  {
		switch($display)
		{
			case 'button':
				$tokenInOptions=get_option('sc_button_token');
				break;
			case 'discussion':
				$tokenInOptions= get_option('sc_discussion_token');
				break;
			case 'profile':
			    $tokenInOptions = get_option('sc_profile_token');
				break;
			case 'trends':
				$tokenInOptions = get_option('sc_trends_token');
				break;
				
		}
		if($tokenInOptions != '')
		   return get_div($id,$style,$tokenInOptions);
	
	
  }
  return get_div($id,$style,$token);
}

function add_reach($atts) {
	extract( shortcode_atts( array(
			'id' => 'reach_container_id',
			'style' => '',
			'token' => '',
			'display' => ''
		), $atts ) );

  return get_shortcode_div($id, $style, $token,$display);
}

function reach_init_method() {

  if (sc_reach_get_wp_version() >= 2.7) {
    if (is_admin ()) {
      add_action('admin_init', 'sc_reach_register_settings');
    }
  }
  add_filter('the_content', 'sc_reach_content_handle');
  add_filter('admin_menu', 'sc_reach_admin_menu');
  
  add_option('sc_host', '');
  add_option('sc_button_token', '');
  add_option('sc_discussion_token', '');
  add_option('sc_profile_token', '');
  add_option('sc_use_microdata', 'true');
  add_option('sc_trends_token', '');
  add_option('sc_enableShortcode','1');
  add_option('sc_show_button','Dont Show');
  add_action('wp_head', 'sc_reach_header_meta');
  add_action('wp_footer', 'sc_reach_add_js');
  if(get_option('sc_enableShortcode'))
  add_shortcode( 'reach', 'add_reach' );
  else
  remove_shortcode('reach');
  
//echo "Get option" . get_option('sc_show_button');

}

function sc_reach_header_meta() {
  echo '<script type="text/javascript">var _reach = _reach || [];</script>';
}

function sc_reach_register_settings() {
  register_setting('sc_reach', 'sc_host');
  register_setting('sc_reach', 'sc_button_token');
  register_setting('sc_reach', 'sc_discussion_token');
  register_setting('sc_reach', 'sc_trends_token');
  register_setting('sc_reach', 'sc_profile_token');
  register_setting('sc_reach', 'sc_use_microdata');
  register_setting('sc_reach', 'sc_enableShortcode');
  register_setting('sc_reach','sc_show_button');
}

function sc_add_button($style='width:300px;height:30px') {
	return get_div('like_container_id', $style, get_option('sc_button_token'));
}

function sc_add_discussion($style='width:300px;height:400px') {
	return get_div('discussion_container_id', $style, get_option('sc_discussion_token'));
}

function sc_add_token($style='width:300px;height:400px'){
	return get_div('trends_container_id',$style,get_option('sc_trends_token'));
}

function sc_reach_admin_menu() {
  add_options_page('REACH Plugin Options', 'Socialcast REACH',  'activate_plugins', __FILE__, 'sc_reach_options');
}

function sc_reach_options() {
?>

  <div class="wrap">
    <h2>Reach Extensions by <a href="http://www.socialcast.com" target="_blank">Socialcast</a></h2>

    <form method="post" action="options.php">
    <?php
	
    if (sc_reach_get_wp_version() < 2.7) {
      wp_nonce_field('update-options');
    } else {
      settings_fields('sc_reach');
    }
    ?>

      <h2>Instructions</h2>
      <p>If you are not logged in to Socialcast. Please do so with a user that has administrative credentials.
        Once there either Create an HTML Extension or select one from the list. For more information please visit the <a href="http://integrate.socialcast.com/business-systems/wordpress/">plugin page</a>.
      </p>
      <h3>Socialcast Community</h3>
		<table>
			<tr><td>https://</td><td><input style="width:400px" type="text" name="sc_host" value="<?php echo get_option('sc_host'); ?>" /></td></tr>
			<tr><td>Add HTML Microdata ?</td><td><input style="width:400px" type="text" name="sc_use_microdata" value="<?php echo get_option('sc_use_microdata'); ?>" />Type 'true' to use</td></tr>
			<tr><td>Button Token:</td><td><input style="width:400px" type="text" name="sc_button_token" value="<?php echo get_option('sc_button_token'); ?>" />Function: sc_add_button()</td></tr>
			<tr><td>Discussion Token:</td><td><input style="width:400px" type="text" name="sc_discussion_token" value="<?php echo get_option('sc_discussion_token'); ?>" />Function sc_add_discussion()</td></tr>
            <tr><td>Profile Token:</td><td><input style="width:400px" type="text" name="sc_profile_token" value="<?php echo get_option('sc_profile_token'); ?>" />Function sc_add_author_stream()</td></tr>
			<tr><td>Trends Token:</td><td><input style="width:400px" type="text" name="sc_trends_token" value="<?php echo get_option('sc_trends_token'); ?>" />Function sc_add_trends()</td></tr>
			<tr><td>Enable Shortcode: </td><td><input name="sc_enableShortcode" type="checkbox" value="1" <?php checked( '1', get_option( 'sc_enableShortcode' ) ); ?> /></td></tr>
			<tr><td>Show Button:</td> <td>
			<select style="width:400px" name="sc_show_button" id="sc_show_button">
<option value="Top" <?php if (get_option('sc_show_button')=='Top') echo 'selected="selected"';?>>Top</options>
<option value="Bottom" <?php if (get_option('sc_show_button')=='Bottom') echo 'selected="selected"';?>>Bottom</options>
<option value="Dont Show" <?php if (get_option('sc_show_button')=='Dont Show') echo 'selected="selected"';?>>Dont Show</options>
</select>Make sure the button token is set for this to work</td></tr>
		</table>
    <?php if (sc_reach_get_wp_version() < 2.7) : ?>
      <input type="hidden" name="action" value="update" />
      <input type="hidden" name="page_options" value="sc_host" />
    <?php endif; ?>
      <p class="submit">
        <input type="submit" name="Submit" value="<?php _e('Save Changes') ?>" />
      </p>
    </form>
    <iframe width="100%" height="600px" src="http://developers.socialcast.com/reach/setup-a-reach-extension/">
    </iframe>
  </div>

<?php
    }

    function sc_reach_add_js() {
?>
      <script type="text/javascript">
        (function(){
          var e=document.createElement('script');
          e.type='text/javascript';
          e.async = true;
          e.src= document.location.protocol + '//<?php echo get_option('sc_host') ?>/services/reach/extension.js';
          var s = document.getElementsByTagName('script')[0];
          s.parentNode.insertBefore(e, s);
        })();
      </script>
<?php
    }

    reach_init_method();
?>
Jul 072012
 

I installed WordPress at work and have been trying to make a multisite installation work as an enterprise blogging platform. During one of the discussions around it, a colleague asked me if it was possible to integrate the blog comments system with Socialcast (the microblogging tool which is quite popular at work). I initially thought this would not be straightforward and would require development of a custom plugin from scratch.

However later, I did some searching and found that Socialcast already provides an infrastructure called Reach which can be used to integrate the comments, posts and trends with a variety of 3rd party sites. For an organization, this integration is extremely valuable as it introduces a common store for all the social interactions – be it Sharepoint, blogs, intranet pages or anything else. Since Reach is written in Javascript, it doesnt pose any restrictions on server side technology used for the sites.

So the primary goal was to make Reach work with WordPress. Initially I looked at options like HTML Javascript Adder which lets you add the reach code directly into a widget on the site. However, this posed too many issues given the lack of control one had on when the scripts were getting loaded and the difficulty to configure it. Since all Reach scripts look exactly the same except a token which is generated by Socialcast when the script is created in the admin panel, it is useless to keep replicating the same code everywhere.

Then I came across a plugin written by Monica Wilkinson of Socialcast. However this was last updated an year ago and both WordPress and Socialcast required some changes. So I forked the branch and made a few minor tweaks to suit my requirement. The plugin gives an option page to configure the tokens and the URL of your socialcast community. So I added the php file to my plugins directory and Network Activated it (This was a multisite installation). Once this is done you would get an options page on the dashboard

Now the options page has the tokens that need to be entered along with the url of your socialcast community.Remember to be careful while sharing the tokens as they have the options of allowing access to the community without the proper login credentials. I am not sure if Socialcast provides the option of revoking these tokens on a periodic basis and providing fresh ones, but this should be present to protect the company data.

There are four main kinds of reach extensions

  • Discussion – A comments system which would be shared with the Socialcast community
  • Stream – Any group or company stream
  • Trends – Trending topics, people etc.
  • Button – Like, Recommend, Share buttons. The exact verb can be configured on the admin screen.


All of these are rendered in the exact same way, by calling a script asynchronously (services/reach/extension.js) and then pushing the reach object with the javascript token. In the plugin there is a get_div function which generates the html tag

function get_div($id, $style, $token) {
	$socialcast_url = get_option('sc_host');
	if ($id != '' && $token != '') {
		return '<div id="' . $id . '" style="' . $style .
		'"></div><script type="text/javascript">_reach.push({container: "' . $id . '", domain: "https://'
		. $socialcast_url . '", token: "' . $token . '"});</script>';
	} else {
		return '';
	}
}

There are two main ways of rendering the appropriate Reach control on your page

  • call the function in PHP code
  • Use the shortcode [reach]


Lets see the first option. The appropriate function that needs to be called in PHP code is given on the option screen. Let’s say I want to display the button just below the title. So I go to the theme’s postheader.php file and call the sc_add_button function in PHP code. Note: the call to sc_add_button function will only work if you have the button token configured in the plugin options. This step may differ from theme to theme.

<header class='post-header title-container fix'>
	<div class="title">
		<<?php echo $header_tag;?> class="posttitle"><?php echo suffusion_get_post_title_and_link(); ?></<?php echo $header_tag;?>>
	<?php
	 echo sc_add_button('width:300px;height:30px'); 
 			if ($post_meta_position == 'corners') {
	?>
	<div class="postdata fix">

Or If you want the comments system to be replaced by the Socialcast discussion system, then go to the comment-template.php file in the wp_include directory and replace the comment markup with the call to sc_add_discussion(). Remember that you can pass the style to this method so it overrides the default styles in the plugin.

		<?php if ( comments_open( $post_id ) ) : ?>
			<?php do_action( 'comment_form_before' ); ?>
		
			<?php echo sc_add_discussion(''); ?>
			<?php do_action( 'comment_form_after' ); ?>
		<?php else : ?>
			<?php do_action( 'comment_form_comments_closed' ); ?>
		<?php endif; ?>

The resulting page looks like this

Now for the shortcode way. This plugin initially required the user to enter the token in the shortcode functions but I wasnt too happy with that way as revealing the tokens to non admin users seems risky. So I wrote a new function in the plugin which would allow the user to give a text on what should be displayed and the token would be read from the options. The previous way of specifying a token still exists as well.

function get_shortcode_div($id,$style,$token,$display){
 $tokenInOptions ='';
  if($display != '')
  {
		switch($display)
		{
			case 'button':
				$tokenInOptions=get_option('sc_button_token');
				break;
			case 'discussion':
				$tokenInOptions= get_option('sc_discussion_token');
				break;
			case 'profile':
			    $tokenInOptions = get_option('sc_profile_token');
				break;
				
		}
		if($tokenInOptions != '')
		   return get_div($id,$style,$tokenInOptions);
	
	
  }
  return get_div($id,$style,$token);
}

This system makes it really easy for users to add the button to their blogs. Just insert a text based widget in the sidebar with the shortcode reach and it will render the widget when the page is run. Just make sure the theme calls the do_shortcode function on the widget_text parameter. If not a single line addition should do it.

Once the widget is saved, the reach extension is rendered on the page. The modified plugin can be downloaded here and the original version written by Monica can be downloaded here. I will make changes for the trends extension soon once I get the token to test it out.

Jan 202012
 

I wrote a small application to extract post and user data for groups in the popular enterprise microblogging portal, Socialcast. It lets users access data instantly about the activity going on in their groups and the number of likes/comments being posted by group members.

The details about the API can be found here and their side demo.socialcast.com can be used for testing. There are three main calls in the application

  • Message Data for the group (Using the parameters Group_Id and since to filter by the data)
  • To find the group Id using the group name using the Groups.xml API call.
  • Then finding the group members using groupname/members.xml.

The method to retrieve group_id from the groupname needed improvement. Most organizations contain upwards of a 1000 groups. Making two heavy API calls for finding out the group id from the name is a bit of a drag on the performance of the application. To avoid this, I just cached all the group Ids and names in a Dictionary<string, int> object and used that to lookup the id. For groups that were created after the tool was distributed, the API call was a fallback.

The UI of the tool is fairly simple. Just a textbox to enter the group URL and two datagridviews to display the data along with two listboxes to display the stats. Since data retrieval is a very slow process, a backgroundworker with a progress bar make sure the application doesnt freeze and the user is shown some progress.  There is also a dropdown to select the time for which the user wants to retrieve messages and whether the member information is required or not. The quickest options are selected by default.


The API access code is in a library which hides the webservice calls from the client application. Each method contains the standard page and number of records per page argument. Apart from them, there are other parameter which are needed to make the call e.g GroupName, member id etc. To simply the URL construction for the web service call a class contains the skeleton urls for each call like given below

public static class ObjectType
 {
 static string defaultFormat = ".xml";
 public static string Users = "users" + defaultFormat;
 public static string Streams = "streams" + defaultFormat;
 public static string Messages = "messages" + defaultFormat;
 public static string StreamMessages = "streams/{0}/messages" + defaultFormat;
 public static string MessagesById = "messages/{0}" + defaultFormat;
 public static string SearchUsers = "users/search" + defaultFormat;
 public static string Groups = "groups" + defaultFormat;
 public static string GroupMembers = "groups/{0}/members" + defaultFormat;
 }

public class SocialCastData
 {
 ///
<summary> /// These are the private variables which are configured
 /// as per your socialcast site
 /// </summary>
 string skeletonURL = "https://{0}.socialcast.com/api/{1}";
//Rest of code here
}

The method to construct the service url takes in the Object Type member, and appends it to the skeleton url along with filling in the domain of the socialcast site and any additional parameter (e.g. group id). This approach allows me to add new service calls easily with just one more variable in the ObjectType class. Any additional query string parameters are supplied using the serviceParams which is a list of keyvaluepairs. The list is iterated through and appended after the url. The SocialcastAuthDetails object is a must in every calls since it contains the domain, the username and password all of which are required to be supplied for getting the response.

Here is the method to get the Group ID from the groupName. The GroupQuery.QueryForGroupId accesses the cache and returns immediately if found. If not then another API call is made to get all the groups, iterate through them and identify the id. If the group id is still not found, an exception is thrown.

        private int GetGroupIdByGroupName(string groupName,SocialCastAuthDetails auth)
        {

            int _groupId = 0;
            int _pageNumber =1;
            bool moreItems = true;

            _groupId = GroupsQuery.QueryForGroupID(groupName);
            if (_groupId != 0)
                return _groupId;
            else
            {

                while (moreItems)
                {
                    XmlDocument group = new XmlDocument();
                    var serviceParams = new List>();
                    serviceParams.Add(new KeyValuePair("page", _pageNumber.ToString()));
                    serviceParams.Add(new KeyValuePair("per_page", "500"));
                    group.LoadXml(base.MakeServiceCalls(helper.GetSocialcastURL(ObjectType.Groups, auth.DomainName, null, serviceParams),GetCredentials(auth.Username,auth.Password)));
                    if (group.SelectNodes("//groups/group") == null || group.SelectNodes("//groups/group").Count == 0)
                        moreItems = false;
                    else
                    {
                        foreach (XmlNode groupNode in group.SelectNodes("//groups/group"))
                        {
                            if (String.Compare(GetNodeInnerText(groupNode, "groupname"), groupName, true) == 0)
                            {
                                string id = GetNodeInnerText(groupNode, "id");
                                _groupId = Convert.ToInt32(id);
                                GroupsQuery.AddToDictionary(groupName, _groupId);
                                moreItems = false;
                                break;
                            }
                        }
                    }
                    _pageNumber++;
                }
                return _groupId;
            }
        }

In the front end, a background worker is used to perform the time consuming operation asynchronously. Here is the code (Rough, not refactored)

    class GetStatistics
    {
        APIAccessor _api = new APIAccessor();

        public AppData GetData(string urlEnteredbyUser, string sinceWhatPeriod, bool loadUserData)
        {
            var serviceUrl = UrlReplace(urlEnteredbyUser.Trim());
            List<SocialcastMessage> messages = null;
            List<UserProfile> users = null;
            if (serviceUrl == string.Empty)
            {
                throw new GroupNotFoundException(null, "The group Url you have entered is not valid");
            }
            else
            {
                messages = GetMessageStats(serviceUrl, sinceWhatPeriod);
                if(loadUserData)
                        users = GetAllGroupUsers(serviceUrl);
                return new AppData(messages, users);
            }
        }

        private List<UserProfile> GetAllGroupUsers(string groupName)
        {
            List<UserProfile> Users = new List<UserProfile>();
            int userCount = 500;
            int pageNumber = 1;
            bool moreItems = true;
            while (moreItems)
            {
                var xdoc = _api.GetGroupMembers(groupName, pageNumber.ToString(), userCount.ToString(), GetCredentials());
                if (xdoc.SelectNodes("//users/user") == null || xdoc.SelectNodes("//users/user").Count == 0)
                    moreItems = false;
                else
                {
                    foreach (XmlNode userNode in xdoc.SelectNodes("//users/user"))
                    {
                        string name = GetNodeInnerText(userNode, "name");
                        string userName = GetNodeInnerText(userNode, "username");
                        string email = GetNodeInnerText(userNode, "contact-info/email");
                        int followingCount = Convert.ToInt32(GetNodeInnerText(userNode, "following-count"));
                        int followersCount = Convert.ToInt32(GetNodeInnerText(userNode, "followers-count"));

                        Users.Add(new UserProfile()
                        {
                            Name = name,
                            userName = userName,
                            Email = email,
                            FollowerCount = followersCount,
                            FollowingCount = followingCount
                        });
                       
                    }
                    pageNumber++;
                }
            }

            return Users;
        }

        private List<SocialcastMessage> GetMessageStats(string serviceUrl, string sinceWhatPeriod)
        {
            List<SocialcastMessage> Messages = new List<SocialcastMessage>();
            int messageCount = 500;
            int pageNumber = 1;
            bool moreItems = true;
            long sinceWhen = GetSinceWhenValue(sinceWhatPeriod);
            while (moreItems)
            {
                var xdoc = GetMessagesForTheGroup(serviceUrl, messageCount, pageNumber, sinceWhen);
                if (xdoc.SelectNodes("//messages/message") == null || xdoc.SelectNodes("//messages/message").Count == 0)
                    moreItems = false;
                else
                {
                    foreach (XmlNode node in xdoc.SelectNodes("//messages/message"))
                    {
                        DateTime createdDate;
                        DateTime.TryParse(GetNodeInnerText(node, "created-at"), out createdDate);
                        string ticksLastUpdatedDate = GetNodeInnerText(node, "last-interacted-at");
                        var epoch = new DateTime(1970, 1, 1, 0, 0, 0, DateTimeKind.Utc);
                        epoch = epoch.AddSeconds(Int64.Parse(ticksLastUpdatedDate));
                        string permaUrl = GetNodeInnerText(node, "permalink-url");
                        string commentsCount = GetNodeInnerText(node, "comments-count");
                        string likesCount = GetNodeInnerText(node, "likes-count");
                        string title = GetNodeInnerText(node, "title");
                        string user = GetNodeInnerText(node, "user/name");

                        Messages.Add(new SocialcastMessage()
                        {
                            Title = title,
                            Url = permaUrl,
                            CreatedDate = createdDate,
                            UpdatedDate = epoch,
                            Comments = int.Parse(commentsCount),
                            Likes = int.Parse(likesCount),
                            UserName = user
                        });

                    }
                    pageNumber++;
                }
            }


            return Messages;

        }


        private long GetSinceWhenValue(string sinceWhatPeriod)
        {
            switch (sinceWhatPeriod)
            {
                case "All Messages":
                    return 0;
                case "Since Last week":
                    return GetTicksSince1970(7);
                case "Since Last Month":
                    return GetTicksSince1970(30);
                default:
                    return 0;
            }
        }

        private static long GetTicksSince1970(int days)
        {
            DateTime sinceLastWeek = DateTime.Now.Subtract(new TimeSpan(days, 0, 0, 0, 0));
            DateTime epoch = new DateTime(1970, 1, 1);
            return Convert.ToInt64(sinceLastWeek.Subtract(epoch).TotalSeconds);
        }


        private XmlDocument GetMessagesForTheGroup(string streamName, int numberOfPosts, int page, long sinceWhen)
        {
            string sinceWhenString = null;
            if (sinceWhen != 0)
                sinceWhenString = sinceWhen.ToString();
            var xdoc = _api.GetStreamMessages(streamName, numberOfPosts.ToString(), page.ToString(), sinceWhenString, GetCredentials());
           return xdoc;
        }


        private string UrlReplace(string enteredByUser)
        {
        
            string regexPattern = @"https://demo.socialcast.com/groups/([A-Za-z0-9\-]+)";
            Match match = Regex.Match(enteredByUser, regexPattern,
                RegexOptions.IgnoreCase);

            // Here we check the Match instance.
            if (match.Success)
            {
                // Finally, we get the Group value and return it
                string key = match.Groups[1].Value;
                return key;
            }
            return string.Empty;
        }

        private SocialCastAuthDetails GetCredentials()
        {
            return new SocialCastAuthDetails()
            {
                Password = "demo",
                Username = "emily@socialcast.com",
                DomainName = "demo"
            };

        }

        private string GetNodeInnerText(XmlNode node, string xpath)
        {
            try
            {
                if (node.SelectSingleNode(xpath) != null)
                {
                    return node.SelectSingleNode(xpath).InnerText.Trim();
                }
                return String.Empty;
            }
            catch
            {
                return string.Empty;
            }
        }
    }
Dec 272010
 

Since the last post, I changed the library/API wrapper a bit. I removed all the ugly reflection stuff to retrieve the specific API urls and substituted them with static variables in a separate class. However this does have the added disadvantage that the urls are exposed to the client, but at least it wont break any client code if Socialcast decides to change the API in the future. Also in the previous example, the username, password and subdomain are variables in the wrapper itself. In the absence of oAuth, every call needs to be authenticated with the user credentials. To avoid having to handle the responsibility of storing user information, I created a class to encapsulate this information (SocialcastAuthDetails) which is passed to the API Accessor for every call. I also added the data objects to return strongly typed responses from the API accessor instead of an XmlDocument, but havent gotten around to incorporate them yet.

Here is the code to Post a message and Get the company stream. Accessing the Company Stream requires two calls – first to get the Stream ID and the next to get the messages for the particular stream.

        public XmlDocument GetCompanyStream(SocialCastAuthDetails auth)
        {
            XmlDocument streams = new XmlDocument();
            if (companyStreamID == 0)
            {
                streams.LoadXml(base.MakeServiceCalls(helper.GetSocialcastURL(ObjectType.Streams,auth.DomainName,null), GetCredentials(auth.Username,auth.Password)));

                foreach (XmlNode node in streams.GetElementsByTagName("stream"))
                {
                    if (node.SelectSingleNode("name").InnerText.ToLower() == "company stream")
                    {
                        companyStreamID = int.Parse(node.SelectSingleNode("id").InnerText);
                        break;
                    }
                }
            }
            streams = new XmlDocument();
            streams.LoadXml(base.MakeServiceCalls(
                                 helper.GetSocialcastURL(ObjectType.StreamMessages,auth.DomainName,companyStreamID.ToString()),
                                 GetCredentials(auth.Username,auth.Password)));
            return streams;
        }

        public XmlDocument PostMessage(string title,string body,SocialCastAuthDetails auth)
        {
            string data = String.Format("message[title]={0}&message[body]={1}", HttpUtility.UrlEncode(title), HttpUtility.UrlEncode(body));
            XmlDocument update = new XmlDocument();
            update.LoadXml(base.MakeServiceCallsPOST(
                                helper.GetSocialcastURL(ObjectType.Messages, auth.DomainName, null),
                                GetCredentials(auth.Username, auth.Password), data));
            return update;
        }

Since any messages which manipulates data requires a POST call instead of a GET, the WebServiceHelper class needs a new method to make the service call using POST. Also the data which is to be posted is URL encoded before being sent to this method.

  protected string MakeServiceCallsPOST(string _requestURL, NetworkCredential credentials, string data)
        {
            // Create the web request
            HttpWebRequest request = WebRequest.Create(_requestURL) as HttpWebRequest;

            request.Credentials = credentials;
            request.ContentType = "application/x-www-form-urlencoded";
            request.Method = "POST";

            byte[] bytes = Encoding.UTF8.GetBytes(data);

            request.ContentLength = bytes.Length;
            using (Stream requestStream = request.GetRequestStream())
            {
                requestStream.Write(bytes, 0, bytes.Length);

                using (WebResponse response = request.GetResponse())
                {
                    using (StreamReader reader = new StreamReader(response.GetResponseStream()))
                    {
                        return reader.ReadToEnd();
                    }
                }
            }
        }

This is the client code to post the message. The socialcast auth details class is initialized by the client and sent, so its their headache to maintain passwords and other sensitive information.

    class Program
    {
        static SocialCastAuthDetails auth = new SocialCastAuthDetails()
        {
            DomainName = "demo",
            Username = "emily@socialcast.com",
            Password = "demo"
        };
        static void Main(string[] args)
        {
            int _messageCounter=1;
            APIAccessor api = new APIAccessor();
            api.PostMessage("Posting from API", "this is a test message posted through C#", auth);
            var xdoc = api.GetCompanyStream(auth);
            Console.WriteLine("Company Steam of demo.socialcast.com");
            Console.WriteLine("******************************************************");
            foreach(XmlNode node in xdoc.GetElementsByTagName("message"))
            {
                Console.WriteLine("Message {0} posted by {1}", _messageCounter++, node.SelectSingleNode("user/name").InnerText);
                Console.WriteLine("Message: {0} {1}", node.SelectSingleNode("title").InnerText, node.SelectSingleNode("body").InnerText);
                Console.WriteLine("====================================================");
            }
        }
    }

It works!!

Dec 252010
 

Socialcast is one of the better enterprise microblogging tools out there. I have been trying to use its API to understand better how people use microblogging in the enterprise. There is no better way to validate (or invalidate) set hypotheses than by actually mining data and identifying patterns in them. If sufficient data exists which is spread across a larger time period, fascinating patterns emerge. In this older post, I had correlated the length of each blog post (in my previous organization’s internal blogging platform) with the number of comments it received. After a senior colleague helped me make sense out of the data, a clear conclusion was that the shorter a blog post is, the more likely people will comment on it.

To attempt something similar with Socialcast, I finally got around to using their API through C# after procrastinating for a very long time. I didnt map the API responses/arguments to .NET objects yet, just wrote a few classes to make it easier to make different service calls without repeating code. In the below code, I used the API to return user details of some of the users. Similarly there are different calls to get groups, streams, followers etc. All the information which is gotten from the site needs a GET call, anything where information is changed (commenting, posting a message etc.) has to be done via POST.

Every socialcast site has a different subdomain (e.g. demo.socialcast.com) and a username/password to authorize requests. Since oAuth is not yet available, this information needs to be stored in the application itself. I saved it in a class for now but a better way would be to store it in a config file (and encrypt it for good measure). The SocialCastData class has all the client specific details like api urls, usernames , password etc. All these properties are protected and only the Helper class which inherits from the data class can access it. The helper class provides the API URL and credentials to the APIAccessor class.

    class SocialCastData
    {
        /// <summary>
        /// These are the private variables which are configured
        /// as per your socialcast site
        /// </summary>
        string domainName = "demo";
        string skeletonURL = "https://{0}.socialcast.com";
        string userName = "emily@socialcast.com";
        string passWord = "demo";
        string _usersURL = "/api/users.xml";

        //Protected properties to give access to the username/password and API
        //URL only to the helper class which inherits from this data classes
        protected string UserName { get { return userName; } }
        protected string Password { get { return passWord; } }
        protected string usersURL { get { return _usersURL; } }

        //Get the basic URL of the site, without any API call
        protected string GetSocialCastURL()
        {
            return String.Format(skeletonURL, domainName);
        }

        //This method uses reflection to provide the API url
        // value based on an agument to this method
        protected string GetSocialCastURL(string apiFilter)
        {
            try
            {
                PropertyInfo _allProperties = this.GetType().GetProperty(apiFilter + "URL", BindingFlags.NonPublic | BindingFlags.Instance);
                if (_allProperties == null)
                    throw new Exception("There was no corresponding API URL found");
                else
                {
                    string value = _allProperties.GetValue(this, null).ToString();
                    return GetSocialCastURL() + value;
                }
            }
            catch (Exception _eObj) { throw new Exception("There was no corresponding API URL found", _eObj); }
        }

    }

    /// <summary>
    /// This is the helper class which provides the URL
    /// and Credentials to the WebServiceHelper object. Only the Helper
    /// class has access to the SocialCastData members since all its
    /// members are protected.
    /// </summary>
    class SocialcastHelper:SocialCastData
    {

        //use the base class data to get the Credentials object.
        public NetworkCredential GetCredentials()
        {
            return new NetworkCredential(base.UserName, base.Password);
        }

        //Get the URL for the socialcast website. The overloaded methods are for
        //returning the appropriate URL depending on the function and if any additional
        //query parameters are to be appended to the URL.
        public string GetSocialcastURL()
        {
            return base.GetSocialCastURL();
        }

        public string GetSocialcastURL(string _apiURL)
        {
            return base.GetSocialCastURL(_apiURL);
        }

        public string GetSocialcastURL(string _apiURL, List<KeyValuePair<string, string>> _paramMessages)
        {
            //Get the URL from the base class method and append
            //the query params
            string _url = base.GetSocialCastURL(_apiURL);
            if (_paramMessages.Count > 0)
            {
                _url += "?";
                foreach (var item in _paramMessages)
                {
                    //appending each param key and value
                    _url += String.Format("{0}={1}&", item.Key, item.Value);
                }
                //String the last ampersand sign
                return _url.Substring(0, _url.Length - 1);
            }
            return _url;
        }
    }

The APIAccessor is the class which contains Business logic functions like GetUsers or GetMessages etc. It sends in its parameters as a List of Keyvalue pairs to the helper class which constructs the actual REST call URL out of the parameters.

 class WebServiceHelper
    {
        public string MakeServiceCalls(string _requestURL, NetworkCredential credential)
        {

            // Create the web request
            HttpWebRequest request = WebRequest.Create(_requestURL) as HttpWebRequest;
            request.Credentials = credential;
            // Get response
            using (HttpWebResponse response = request.GetResponse() as HttpWebResponse)
            {
                // Get the response stream
                StreamReader reader = new StreamReader(response.GetResponseStream());

                // Console application output
                return reader.ReadToEnd();
            }
        }
    }
    class APIAccessor:WebServiceHelper
    {
        //Creating the Helper class Instance
        SocialcastHelper helper = new SocialcastHelper();

        public XmlDocument GetUsers(string numberOfusers,string page)
        {
            XmlDocument Users = new XmlDocument();
            var serviceParams = new List<KeyValuePair<string,string>>();
            serviceParams.Add(new KeyValuePair<string,string>("per_page",numberOfusers));
            serviceParams.Add(new KeyValuePair<string,string>("page",page));

            Users.LoadXml(base.MakeServiceCalls(helper.GetSocialcastURL("users", serviceParams), helper.GetCredentials()));
            return Users;
        }
    }

The test code is below to display the name of users from the socialcast demo site.

 class Program
    {
        static void Main(string[] args)
        {
            var _xDoc = new APIAccessor().GetUsers("30", "1");
            foreach (XmlNode item in _xDoc.GetElementsByTagName("user"))
            {
                XmlNode name = item.SelectSingleNode("name");
                Console.WriteLine(name.InnerText);
            }

        }
   }

The Sample code can be downloaded here.

Dec 172010
 

In the early years of the internet when it was still in its infancy and yet to capture the imagination of a significant section of the society, it was mostly used as a one way content distribution system. Websites were static and their content created by a small team so a larger group of users could access, but not modify it. However over time, the Internet evolved and slowly democratized itself. The content creation was passed more and more to users, and buzzwords like Social media, blogging, tweeting were born. Web 2.0 had finally arrived, and how!!

Wikipedia, Facebook, Twitter, LinkedIn, WordPress were some of the great new tools which empowered users to eliminate geographical boundaries and connect in ways hitherto unimaginable. Blogging really took off in the first half of this decade. Suddenly, the common man had a voice, he could express dissent, criticism or his views of the world today. Previously what was restricted to a lazy chat between friends could be put forth to the whole world and this essentially changed media as we know it. But blogging hit a roadblock few years back, even though the number of blogs was large, it still weren’t quite the hit that Social media exponents were hoping for. One reason for that could have been the significant amount of time which needed to be invested to write a decent blog post. One needed to jot down his thoughts, organize them, articulate it nicely and edit it well enough to make the post crisp and enjoyable to the reader. The effort needed to do all this made regular blogging out of reach for many web users. Change however, was just around the corner.

On a fateful day in 2006, Twitter was launched. Initially called the SMS of the Internet, Twitter’s 140 character limit became a empowering tool rather than an impediment to content creation. Blogging Microblogging was far more easier and the medium exploded. From 400,000 tweets per quarter in 2007, it reached 50 million tweets per day in February this  year. Given Twitter’s unprecedented success, it was inevitable that the enterprise looked at it to be used as a collaboration tool on the Intranet. And thus, Enterprise Microblogging was born. I had the opportunity of being a part of two such social networks in my current and previous organizations (powered by Yammer and Socialcast) from their infancy to  relative maturity.  I’m no social media expert but I jotted down some of my random observations from a user’s perspective.

Growth Cycle:-

Like many software products and technologies, a user’s perception of Enterprise Microblogging also follows the hype cycle.

Most new users sign up to Microblogging via an invite from their colleagues. Those who dont have much experience in using twitter are usually bewildered by this new tool. Expectations are high thanks to that over-enthusiastic colleague who promised the world for signing up. But disillusionment quickly sets in, for many users there is too much of wildly varying content to make any sense out of. They send out a couple of messages, maybe like a few posts but thats just about it.  Not many users are patient enough to ride out the disillusionment phase to actually start deriving value out of the system. The user retention is quite low in enterprise networks and according to my rough estimates only about 30-40% users are active in the system.

The Champion

Though social networks are touted to grow horizontally without the need of any central direction, I have observed the opposite in Enterprise Microblogging. People adapt to the network much faster if there is someone who quickly welcomes new users, understands their interests and introduces them to like minded people, routes queries to individuals/groups who can answer them faster. Maybe in a mature network, this function could be taken over by a group of power users. But those that are still growing, require an individual who can be the champion. In my previous organization I observed the quality of the network content going down considerably when the champion scaled down his participation and no one else took over.

Signal to Noise ratio

Image courtesy http://www.socialresearchmethods.net

The public timeline/ Home Stream is the holy grail of user experience in any social network. Facebook took this serious enough to patent its NewsFeed feature to decide which friends’ updates appear on your news feed.  If our close friends updates appear on the news feed, then the chance of responding becomes much higher. In the absence of a personal connection on microblogging sites, this translates into users whose updates would be interesting to us. It could be determined by similarity in content, common connections etc. Nothing would confuse a new user more than completely random content which is not in the least bit appealing.

Groups, Communities and Networks

Continuing with my previous point about the Signal to Noise ratio, the content in the Home Stream is often too amorphous to be of interest to a newbie. Here is where Groups or Communities help in the Microblogging network. When like minded people come together, the content they create is more likely to be homogeneous and easier for a person to relate too. In Yammer I saw that some groups had more posts and replies than the general stream itself. Socialcast also has a great feature which allows users to tag the post to a group in the comment even if the original author misses to do so. This is where power users and the Champion can contribute to the network by making sure all content is properly categorized according to the groups in which it belongs.

Will Microblogging kill email?

Microblogging has as much chance of killing email as Google docs has of replacing MS Office. Regardless of the merits or demerits of the two platforms, the adoption of one at the cost of the other requires a fundamental shift in the thinking of the users – not too dissimilar from the time when typewriters gave way to computers. Whether users will be willing to make that quantum leap and ditch email entirely, only time will tell. However it is not deniable that Microblogging would eventually reduce the amount of email being sent in the organization. Organization wide updates which are cascaded to a large group of users would be far easier through social media than conventional email.

Knowledge Management Tool?

Knowledge Management in today’s organizations are not restricted to churning out thousands of documents and then taking pride in number of documents we have in our knowledge repository. To be of any real value, the more useful resources must be separated from the ordinary Copy-paste jobs from the internet. Microblogging provides us a measurable mechanism to gauge the response of the community towards any particular content posted in the form of the replies / likes it receives. Documents which elicit larger response are more likely to be of use. Also since its all text based, all of the content is indexable and searchable from various other point of access like Intranet Portals, Sharepoint sites etc.

Democratizing the Organization

Even though almost all the higher management if any company works with an open-door policy, in practice the direct communication between employees engaged in actual operations and the management team is quite rare, and mostly in scenarios with set agenda and little room for a free discussion. Microblogging gives every employee the chance to put forth his direct questions to the management team. Frank opinions and suggestions can be given on the strategy of the company.

Tagging and Trending

This one’s a no-brainer. Hashtags allow us to categorize information according to topic and makes it easily searchable later. Trends give us a picture of the so called hot topics being discussed at the very moment. However though trends are suited for a large user base like Twitter, microblogging networks have far less users and this feature isn’t used that much.