Jul 252013
 

Here is a script that I wrote to back up a Trello board to my organization’s Fogbugz wiki. The code is available on GitHub and the script can be set up as a cronjob to take a regular back up of your Trello Board. The main advantage is that you don’t have to maintain the same information in two different places.

If you want to change the HTML formatting, just edit the get_output_html method. The readme.md file has instructions on how to get the API tokens and the list of dependencies that you need to install before running the ruby script.

require 'trello'
require 'open-uri'
require 'htmlentities'
require 'uri'
require 'net/http'
require 'net/smtp'
require 'fogbugz'
require 'json'

=begin
Author : Ganesh Ranganathan
Description: The script copies all the information on a Trello Board to a
fogbugz wiki page. It can be set as a cron job to copy the latest state of 
your trello board and avoid having to duplicate the information
to your organization's fogbugz wiki 
=end

module Constants

	module Fogbugz
		URI = '<Fogbugz_URL>' #The URI Endpoint of your fogbugz deployment
		API_URL = '<FOGBUGZ_API_URL>' #The API url of your fogbugz deployment. Usually ends with api.asp
		FOGBUGZ_TOKEN = "<Enter API Token" #The API Token
		WIKI_ARTICLE_ID = 0 #Wiki Article ID where the information has to be copied.WARNING: Existing info will be deleted
		WIKI_PAGE_TITLE = 'Sample Trello Board Title' #Title of the Wiki page
	end 

	module Trello
		#Fill Trello OAuth Key, AppSecret and token and the board ID
		TRELLO_OAUTH_KEY = '<OAUTH_KEY>' 
		TRELLO_OAUTH_APPSECRET = 'OAUTH_APP_SECRET'
		#only use a read only token for this script. Since we dont want to delete data even by mistake
		TRELLO_OAUTH_APPTOKEN = 'APP_TOKEN'
		TRELLO_BOARD_ID = 'BOARD_ID'
	end

	module Email
		#Email Details to notify user via Email
		SMTP_SERVER = 'Enter SMTP Server'
		FROM_EMAIL_ADDRESS = 'From Email Address'
	end
end 

module TrelloModule

 class Board

 	attr_accessor :title 
 	attr_accessor :members
 	attr_accessor :lists

 	def initialize(trello_board) 		
 		#initialize Arrays
 		@members = Array.new
 		@lists = Array.new 
 		#populate members 
 		@title = trello_board.name
 		trello_board.members.each{ |member| @members.push(User.new(member)) }
  		trello_board.lists.each{ |list|  @lists.push(List.new(list)) }
 	end
 end

 class List
 	attr_accessor :cards
 	attr_accessor :name

 	def initialize(trello_list)
 		#initialize Arrays
 		@cards = Array.new
 		#populate basic variable
 		@name = trello_list.name
 		#Populate Cards Array
 		trello_list.cards.each{ |card| @cards.push(Card.new(card)) }
 	end
 end

 class Card
 	attr_accessor :comments
 	attr_accessor :name 
 	attr_accessor :description
 	attr_accessor :members

 	def initialize(trello_card)
 		#Initialize Arrays
 		@comments = Array.new 
 		@members = Array.new

 		@name = trello_card.name
 		@description = trello_card.description

 		if trello_card.members.count > 0 
 		#populate Users
 			trello_card.members.each{ |member|
 			@members.push(User.new(member))
 		}
 		end
 		
 		#populate Comments
 		trello_card.actions.select{ |action|
 			action.type == "commentCard"
 			}.reverse.each {|comment|
 				@comments.push(Comment.new(comment))
 			}
 	end
 end

 class User
 	attr_accessor :full_name

 	def initialize(trello_member)
 		@full_name  = trello_member.full_name
 	end

 end

 class Comment
 	attr_accessor :text
 	attr_accessor :creator

 	def initialize(trello_comment)
 		@text = trello_comment.data["text"]
 		@creator = User.new(Helper.get_trello_member(trello_comment.member_creator_id))
 	end

 end

 class Helper

 	def self.get_trello_member(member_id)
 		Trello::Member.find(member_id)
 	end

 	#this method generates the output html
 	def self.get_output_html(board)
 		body_html = "<h2>Members</h2>"
 		board.members.each{ |member| body_html << member.full_name << "<br />" }
 		body_html << "<br /><h2>Lists</h2>"
		board.lists.each { |list| body_html << list.name << "<br />" }
		body_html << "<br /><h2>Cards</h2>"
		board.lists.each { |list| 
							  list.cards.each {|card| 
									body_html << "<h3>" << HTMLEntities.new.encode(card.name) << "</h3>" 
									body_html << "Description: " << HTMLEntities.new.encode(card.description).gsub(/\n/, '<br />') 
 		 							body_html << "<br />Assigned To:"
 		 							card.members.each { |member| body_html << member.full_name << ", "  }
 		 							body_html << "<br />Comments:<br /><ul>"
 		 							card.comments.each { |comment|
 		 								body_html << "<li><span style=""font-size:14px; line-height:14px""><b>" << comment.creator.full_name << "</b>: " << HTMLEntities.new.encode(comment.text) << "</span></li>"
 		 							}
 		 							body_html << "</ul><hr />"
 		 						}
 		 					}
 		
 		body_html
 	end

 	def self.write_to_fogbugz(body_html)

 		#The fogbugz-ruby gem doesnt work for larg wiki pages because it tries to send the Body in the URL and fails when the size limit is breached
=begin
		fogbugz = Fogbugz::Interface.new(:token => Constants::FOGBUGZ_TOKEN, :uri => Constants::URI )
		response = fogbugz.command(:editArticle, :sBody => body_html, :ixWikipage => Constants::WIKI_ARTICLE_ID, :sHeadLine => Constants::WIKI_PAGE_TITLE)
		puts response
=end
		uri = URI.parse("#{Constants::Fogbugz::API_URL}?cmd=editArticle&token=#{Constants::Fogbugz::FOGBUGZ_TOKEN}&ixWikipage=#{Constants::Fogbugz::WIKI_ARTICLE_ID}&sHeadLine=#{URI::encode(Constants::Fogbugz::WIKI_PAGE_TITLE)}")
		http = Net::HTTP.new(uri.host)
		http.use_ssl = false
		http.verify_mode = OpenSSL::SSL::VERIFY_NONE
		request = Net::HTTP::Post.new("#{uri.path}?#{uri.query}")
		body = {'sBody' => body_html}
		request.set_form_data(body,';');
		response = http.request(request) 
		puts response.body
 	end

 	def self.send_email(output, recipient)
 		message = <<MESSAGE_END
From: Trello Admin <#{Constants::Email::FROM_EMAIL_SERVER}>
To: A Test User <#{recipient}>
MIME-Version: 1.0
Content-type: text/html
Subject: #{Constants::Fogbugz::WIKI_PAGE_TITLE} backup 
#{output}
MESSAGE_END

Net::SMTP.start(Constants::Email::SMTP_SERVER) do |smtp|
  smtp.send_message message,Constants::Email::FROM_EMAIL_ADDRESS, recipient
  end
 	end
 end

class Main

	def initialize
		init_trello_api
		board =  Board.new(Trello::Board.find(Constants::Trello::TRELLO_BOARD_ID))
		output = Helper.get_output_html(board)
		Helper.write_to_fogbugz(output)
	end

	private 
	def init_trello_api
		Trello::Authorization.const_set :AuthPolicy, Trello::Authorization::OAuthPolicy
		Trello::Authorization::OAuthPolicy.consumer_credential = Trello::Authorization::OAuthCredential.new  Constants::Trello::TRELLO_OAUTH_KEY, Constants::Trello::TRELLO_OAUTH_APPSECRET
		Trello::Authorization::OAuthPolicy.token = Trello::Authorization::OAuthCredential.new Constants::Trello::TRELLO_OAUTH_APPTOKEN , nil
	end
end
end

TrelloModule::Main.new

Dec 212012
 

In my previous post, I blogged about how to access the Socialcast community data without using the API. This is usually necessary when the API doesnt support any particular functionality which is provided by the site.

This is true of the usecase of updating of the user’s profile avatar. Though there is a way to update the user profile in the API, but there is no obvious method of updating the user’s avatar. I asked Socialcast on twitter, but they didn’t answer so I went ahead with trying to use Mechanize to login to the site.

I was finally able to update the profile avatar using the below script. Works like a charm.

require 'Mechanize'
agent = Mechanize.new
agent.agent.http.verify_mode = OpenSSL::SSL::VERIFY_NONE
agent.get("https://demo.socialcast.com/login")
form = agent.page.forms.first
puts "Please enter user email id"
form.email = gets.chomp
puts "Please enter password. caution: it is not masked"
form.password= gets.chomp
form.submit
puts "Please enter username"
agent.get ("https://demo.socialcast.com/users/emilyjames/edit")
form = agent.page.forms.detect{ |f| f.file_upload_with(:name => "profile_photo[data]") }
puts "Please enter file path of the image to replace"
form.file_uploads.first.file_name = gets.chomp
form.submit

Dec 212012
 

The Socialcast REST API provides programmatic access to the Socialcast community data with XML and JSON endpoints. The API provides most of the information one would require to extract out of the site but there are still gaps where the API is not up to date.

This made me look into the possibility of scraping the site directly using cUrl and parsing the generated HTML. However Socialcast is built on Rails and has a security feature which prevents cross site request forgery, using an authenticity token which is a random token generated and sent with every request embedded in a hidden form field. When the form is posted back, this token is checked and an error generated if it’s not found. This makes direct scraping of the page difficult and cUrl fails. Googling gave me a few articles which specified how to use cUrl with sites protected with the authenticity token (Link1, Link2) but unfortunately none of them seemed to work.

Then I came across a suggestion to use Mechanize, a ruby library to automate interaction with websites. Mechanize works like a charm with sites protected by an authenticity token. Here is the ruby script to login to the Socialcast Demo site.

require 'Mechanize'
agent = Mechanize.new
agent.agent.http.verify_mode = OpenSSL::SSL::VERIFY_NONE
agent.get("https://demo.socialcast.com/login")
form = agent.page.forms.first
form.email = "emily@socialcast.com"
form.password= "demo"
form.submit

In Interactive Ruby, we can see that the authenticity token is returned when the first GET is called on the login page. And when the form is submitted the token is posted back to the server and we are redirected to the home page.

login

From here on, we can automate any interaction with the site just as a normal user would do without worrying about the authenticity token restriction. In my next post, I will explain how to automatically update a user’s avatar without relying on the API