Jan 202012
 

I wrote a small application to extract post and user data for groups in the popular enterprise microblogging portal, Socialcast. It lets users access data instantly about the activity going on in their groups and the number of likes/comments being posted by group members.

The details about the API can be found here and their side demo.socialcast.com can be used for testing. There are three main calls in the application

  • Message Data for the group (Using the parameters Group_Id and since to filter by the data)
  • To find the group Id using the group name using the Groups.xml API call.
  • Then finding the group members using groupname/members.xml.

The method to retrieve group_id from the groupname needed improvement. Most organizations contain upwards of a 1000 groups. Making two heavy API calls for finding out the group id from the name is a bit of a drag on the performance of the application. To avoid this, I just cached all the group Ids and names in a Dictionary<string, int> object and used that to lookup the id. For groups that were created after the tool was distributed, the API call was a fallback.

The UI of the tool is fairly simple. Just a textbox to enter the group URL and two datagridviews to display the data along with two listboxes to display the stats. Since data retrieval is a very slow process, a backgroundworker with a progress bar make sure the application doesnt freeze and the user is shown some progress.  There is also a dropdown to select the time for which the user wants to retrieve messages and whether the member information is required or not. The quickest options are selected by default.


The API access code is in a library which hides the webservice calls from the client application. Each method contains the standard page and number of records per page argument. Apart from them, there are other parameter which are needed to make the call e.g GroupName, member id etc. To simply the URL construction for the web service call a class contains the skeleton urls for each call like given below

public static class ObjectType
 {
 static string defaultFormat = ".xml";
 public static string Users = "users" + defaultFormat;
 public static string Streams = "streams" + defaultFormat;
 public static string Messages = "messages" + defaultFormat;
 public static string StreamMessages = "streams/{0}/messages" + defaultFormat;
 public static string MessagesById = "messages/{0}" + defaultFormat;
 public static string SearchUsers = "users/search" + defaultFormat;
 public static string Groups = "groups" + defaultFormat;
 public static string GroupMembers = "groups/{0}/members" + defaultFormat;
 }

public class SocialCastData
 {
 ///
<summary> /// These are the private variables which are configured
 /// as per your socialcast site
 /// </summary>
 string skeletonURL = "https://{0}.socialcast.com/api/{1}";
//Rest of code here
}

The method to construct the service url takes in the Object Type member, and appends it to the skeleton url along with filling in the domain of the socialcast site and any additional parameter (e.g. group id). This approach allows me to add new service calls easily with just one more variable in the ObjectType class. Any additional query string parameters are supplied using the serviceParams which is a list of keyvaluepairs. The list is iterated through and appended after the url. The SocialcastAuthDetails object is a must in every calls since it contains the domain, the username and password all of which are required to be supplied for getting the response.

Here is the method to get the Group ID from the groupName. The GroupQuery.QueryForGroupId accesses the cache and returns immediately if found. If not then another API call is made to get all the groups, iterate through them and identify the id. If the group id is still not found, an exception is thrown.

        private int GetGroupIdByGroupName(string groupName,SocialCastAuthDetails auth)
        {

            int _groupId = 0;
            int _pageNumber =1;
            bool moreItems = true;

            _groupId = GroupsQuery.QueryForGroupID(groupName);
            if (_groupId != 0)
                return _groupId;
            else
            {

                while (moreItems)
                {
                    XmlDocument group = new XmlDocument();
                    var serviceParams = new List>();
                    serviceParams.Add(new KeyValuePair("page", _pageNumber.ToString()));
                    serviceParams.Add(new KeyValuePair("per_page", "500"));
                    group.LoadXml(base.MakeServiceCalls(helper.GetSocialcastURL(ObjectType.Groups, auth.DomainName, null, serviceParams),GetCredentials(auth.Username,auth.Password)));
                    if (group.SelectNodes("//groups/group") == null || group.SelectNodes("//groups/group").Count == 0)
                        moreItems = false;
                    else
                    {
                        foreach (XmlNode groupNode in group.SelectNodes("//groups/group"))
                        {
                            if (String.Compare(GetNodeInnerText(groupNode, "groupname"), groupName, true) == 0)
                            {
                                string id = GetNodeInnerText(groupNode, "id");
                                _groupId = Convert.ToInt32(id);
                                GroupsQuery.AddToDictionary(groupName, _groupId);
                                moreItems = false;
                                break;
                            }
                        }
                    }
                    _pageNumber++;
                }
                return _groupId;
            }
        }

In the front end, a background worker is used to perform the time consuming operation asynchronously. Here is the code (Rough, not refactored)

    class GetStatistics
    {
        APIAccessor _api = new APIAccessor();

        public AppData GetData(string urlEnteredbyUser, string sinceWhatPeriod, bool loadUserData)
        {
            var serviceUrl = UrlReplace(urlEnteredbyUser.Trim());
            List<SocialcastMessage> messages = null;
            List<UserProfile> users = null;
            if (serviceUrl == string.Empty)
            {
                throw new GroupNotFoundException(null, "The group Url you have entered is not valid");
            }
            else
            {
                messages = GetMessageStats(serviceUrl, sinceWhatPeriod);
                if(loadUserData)
                        users = GetAllGroupUsers(serviceUrl);
                return new AppData(messages, users);
            }
        }

        private List<UserProfile> GetAllGroupUsers(string groupName)
        {
            List<UserProfile> Users = new List<UserProfile>();
            int userCount = 500;
            int pageNumber = 1;
            bool moreItems = true;
            while (moreItems)
            {
                var xdoc = _api.GetGroupMembers(groupName, pageNumber.ToString(), userCount.ToString(), GetCredentials());
                if (xdoc.SelectNodes("//users/user") == null || xdoc.SelectNodes("//users/user").Count == 0)
                    moreItems = false;
                else
                {
                    foreach (XmlNode userNode in xdoc.SelectNodes("//users/user"))
                    {
                        string name = GetNodeInnerText(userNode, "name");
                        string userName = GetNodeInnerText(userNode, "username");
                        string email = GetNodeInnerText(userNode, "contact-info/email");
                        int followingCount = Convert.ToInt32(GetNodeInnerText(userNode, "following-count"));
                        int followersCount = Convert.ToInt32(GetNodeInnerText(userNode, "followers-count"));

                        Users.Add(new UserProfile()
                        {
                            Name = name,
                            userName = userName,
                            Email = email,
                            FollowerCount = followersCount,
                            FollowingCount = followingCount
                        });
                       
                    }
                    pageNumber++;
                }
            }

            return Users;
        }

        private List<SocialcastMessage> GetMessageStats(string serviceUrl, string sinceWhatPeriod)
        {
            List<SocialcastMessage> Messages = new List<SocialcastMessage>();
            int messageCount = 500;
            int pageNumber = 1;
            bool moreItems = true;
            long sinceWhen = GetSinceWhenValue(sinceWhatPeriod);
            while (moreItems)
            {
                var xdoc = GetMessagesForTheGroup(serviceUrl, messageCount, pageNumber, sinceWhen);
                if (xdoc.SelectNodes("//messages/message") == null || xdoc.SelectNodes("//messages/message").Count == 0)
                    moreItems = false;
                else
                {
                    foreach (XmlNode node in xdoc.SelectNodes("//messages/message"))
                    {
                        DateTime createdDate;
                        DateTime.TryParse(GetNodeInnerText(node, "created-at"), out createdDate);
                        string ticksLastUpdatedDate = GetNodeInnerText(node, "last-interacted-at");
                        var epoch = new DateTime(1970, 1, 1, 0, 0, 0, DateTimeKind.Utc);
                        epoch = epoch.AddSeconds(Int64.Parse(ticksLastUpdatedDate));
                        string permaUrl = GetNodeInnerText(node, "permalink-url");
                        string commentsCount = GetNodeInnerText(node, "comments-count");
                        string likesCount = GetNodeInnerText(node, "likes-count");
                        string title = GetNodeInnerText(node, "title");
                        string user = GetNodeInnerText(node, "user/name");

                        Messages.Add(new SocialcastMessage()
                        {
                            Title = title,
                            Url = permaUrl,
                            CreatedDate = createdDate,
                            UpdatedDate = epoch,
                            Comments = int.Parse(commentsCount),
                            Likes = int.Parse(likesCount),
                            UserName = user
                        });

                    }
                    pageNumber++;
                }
            }


            return Messages;

        }


        private long GetSinceWhenValue(string sinceWhatPeriod)
        {
            switch (sinceWhatPeriod)
            {
                case "All Messages":
                    return 0;
                case "Since Last week":
                    return GetTicksSince1970(7);
                case "Since Last Month":
                    return GetTicksSince1970(30);
                default:
                    return 0;
            }
        }

        private static long GetTicksSince1970(int days)
        {
            DateTime sinceLastWeek = DateTime.Now.Subtract(new TimeSpan(days, 0, 0, 0, 0));
            DateTime epoch = new DateTime(1970, 1, 1);
            return Convert.ToInt64(sinceLastWeek.Subtract(epoch).TotalSeconds);
        }


        private XmlDocument GetMessagesForTheGroup(string streamName, int numberOfPosts, int page, long sinceWhen)
        {
            string sinceWhenString = null;
            if (sinceWhen != 0)
                sinceWhenString = sinceWhen.ToString();
            var xdoc = _api.GetStreamMessages(streamName, numberOfPosts.ToString(), page.ToString(), sinceWhenString, GetCredentials());
           return xdoc;
        }


        private string UrlReplace(string enteredByUser)
        {
        
            string regexPattern = @"https://demo.socialcast.com/groups/([A-Za-z0-9\-]+)";
            Match match = Regex.Match(enteredByUser, regexPattern,
                RegexOptions.IgnoreCase);

            // Here we check the Match instance.
            if (match.Success)
            {
                // Finally, we get the Group value and return it
                string key = match.Groups[1].Value;
                return key;
            }
            return string.Empty;
        }

        private SocialCastAuthDetails GetCredentials()
        {
            return new SocialCastAuthDetails()
            {
                Password = "demo",
                Username = "emily@socialcast.com",
                DomainName = "demo"
            };

        }

        private string GetNodeInnerText(XmlNode node, string xpath)
        {
            try
            {
                if (node.SelectSingleNode(xpath) != null)
                {
                    return node.SelectSingleNode(xpath).InnerText.Trim();
                }
                return String.Empty;
            }
            catch
            {
                return string.Empty;
            }
        }
    }