The last few days I’ve been seeing a few references to “Mad Libs”. Confused, I quickly punched the term into Google and discovered that it was a reference to something we’ve all done at school, filling in the blanks in a sentence. My google reader(this and this) shared the fact HuffDuffer started using a Mad Libs style form as their sign up form. The question was raised by a few people as to whether this was good design.

Luke Wroblewski (in collaboration with Vast.com) did some research and came up with astounding results that the conversion ratio rose some 25-40% when a Mad Libs style form was used. That’s an impressive, but meaningless number. Don’t get me wrong, I’m very happy that HuffDuffer has found something that works for them. But lets step back for a second and analyse why the results that Vast.com got might be this way.

Here’s a thought; maybe it’s just because its a completely novel approach to a signup form. But can you really see this becoming the standard way to do signup forms? How long do you think it will be before this becomes more annoying than amusing? So quit jibber jabbering about how good it is and take a look at the long term picture. Mad Libs style forms are not something you can take and apply elsewhere with guaranteed results.

To be fair, I’m biased against filling in the blanks. I hated the premise at school and I will probably continue to hate the practice until it, or I, ceases to exist.

I don’t think that I’ve had this much adrenalin pumping through my system on a Saturday morning in quite a while. My girlfriend signed into her email (as she does most mornings), to discover an email from eBay Live Support asking for a code. She called me over and a few seconds later the email vanished. What happened after that point is a blur of password resets, both her and the would be hacker trying to gain control of the hotmail and through it the eBay account.

There were paranoid moments when passwords wouldn’t work, but in the end she’s still in control of the accounts. Just about every account she owns has now had the password and details changed to help protect it.

It’s intrusive though. How exactly did the attacker break into the account? Not phishing, Cherie is well aware of those type of malicious emails. Guessing the security questions? Maybe.

That raises other concerns though, email addresses become repositories of knowledge for our online lives. Just about every account you sign up for online has to have an email account linked to it, that means many details about your online life are there in fragments. We’ll never know exactly what the hacker had access to (albeit briefly).

I’m personally going to be reviewing all my accounts to make sure that they’re secure, and I’d advise you to do the same.

vertis pushed to master at vertis/active_documentum

With JRuby and Warbler it’s possible to get Sinatra, or any WebApp based on Rack, running on a myriad of different Java application servers. There are of course gotchas when it comes to using Warbler with the many different app servers, so this is a definitive guide to everything you have to do to get a simple Sinatra app running on the various application servers.

Why Sinatra?

There are examples of how to get Rails running on Tomcat and Websphere floating around the web, but I find Rails overkill for small projects. With that in mind, it’s worth looking at how to get Sinatra running on java application servers. Besides the weight of rails, Sinatra is a nice, easy to learn framework.

Start by installing Sinatra and Warbler. You don’t have to be using JRuby to install Warbler, the install will download a gem of the jruby jars.

Install Sinatra and Warbler

Lets start by installing the required gems.

$ sudo gem install sinatra warbler haml

Haml isn’t strictly required, but the template I’m going to use has views generated in haml, so if you’re following the tutorial closely you’ll want to install it.

Create a project folder (and structure)

I usually keep a sinatra project template handy. So I’m going to clone my existing template off github. You can create a more minimal example than the one I’ll download, this will get the job done though.

$ git clone git://github.com/vertis/sinatra-example.git deploy_test

TODO: Details about the project

Check that our page is displayed

$ rackup

Go to http://localhost:9292/ and you should see our default page.

Generate the warble config

$ mkdir config && warble config

Lets look at the config file that was generated:

If you tried packaging and installing this now, it would fail miserably because, the ‘init.rb’ file would not be included. The generated warble.rb only includes the following

config.dirs = %w(app config lib log vendor tmp)

In addition to this the gems that we installed above would not be install. Here is same config with the lines we need (and the other cruft removed).

Package up the War file

$ warble

From here on in we’ll be looking at any gotchas, when deploying to the different Application Servers.

Deploy to Glassfish and test (effort: moderate – working: yes)

You can get Suns open source application server from http://glassfish.org. The current stable version of Glassfish 2.1.1, though Glassfish 3 is in active development. The installer comes packaged as a jar file. You can run the installer with

$ java -Xmx256m -jar glassfish-installer-v2.1.1-b31g-linux.jar

After accepting the license it should put all the files in a folder called ‘glassfish’ in the current directory.

$ cd glassfish
You need to run the following commands to finish the setup:

$ chmod -R +x lib/ant/bin
$ lib/ant/bin/ant -f setup.xml

Once the software is installed you can start the domain with:

$ bin/asaadmin start-domain domain1

And use either the admin console or the autodeploy directory to deploy the war file.

Glassfish now has a working copy of our application.

Deploy to JBoss and test (effort: n/a – working: no)

JBoss has a community and an enterprise edition. For the purposes of this test we’ll be using the community edition. The current stable version of JBoss AS is 5.1.0 GA. You can get a copy of JBoss from http://www.jboss.org.

Getting started is as simple as unzipping the archive and running:

$ cd jboss-5.1.0.GA

$ bin/run.sh

You can then use the admin console to deploy the application. One gotcha here, the first time I deployed the application using the console I got the following nasty message:

Application initialization failed: no such file to load — rack
from /opt/application_servers/jboss-5.1.0.GA/server/default/deploy/deploy_test.war/WEB-INF/lib/jruby-rack-0.9.5.jar/vendor/rack.rb:1
from /opt/application_servers/jboss-5.1.0.GA/server/default/deploy/deploy_test.war/WEB-INF/lib/jruby-rack-0.9.5.jar/vendor/rack.rb:22:in `require’
from /opt/application_servers/jboss-5.1.0.GA/server/default/deploy/deploy_test.war/WEB-INF/lib/jruby-rack-0.9.5.jar/jruby/rack/booter.rb:22:in `boot!’
from /opt/application_servers/jboss-5.1.0.GA/server/default/deploy/deploy_test.war/WEB-INF/lib/jruby-rack-0.9.5.jar/jruby/rack/boot/rack.rb:9
from /opt/application_servers/jboss-5.1.0.GA/server/default/deploy/deploy_test.war/WEB-INF/lib/jruby-rack-0.9.5.jar/jruby/rack/boot/rack.rb:1:in `load’
from <script>:1

Turns out that after some digging there is an open jruby bug about the issue -  http://jira.codehaus.org/browse/JRUBY-3935

I also did a bit of digging through the logs and found:

16:27:50,703 ERROR [STDERR] Warning: JRuby home “/opt/application_servers/jboss-5.1.0.GA/server/default/deploy/deploy_test.war/WEB-INF/lib/jruby-stdlib-1.4.0.jar/META-INF/jruby.home” does not exist, using /tmp

I’ve not managed to find a solution to this problem. I will revisit this at some point in the future. After googling a little it may be possible to just revert to a few older versions that seemed to work.

Deploy to Jetty and test (effort: easy – working: yes)

The current version of Jetty is 7.0.1.v20091125, though the version that comes as part of your Linux distro may not be so up to date. You can either install it using your favorite package manager, or if you’re on Windows get it from the homepage at http://www.mortbay.org

Once you’ve installed Jetty copy the generated war file to the webapps folder, and run:

$ bin/jetty.sh

You should be able to go to http://localhost:8080/deploy_test

Congratulations you now have a working copy of your sinatra app on Jetty.
Deploy to Tomcat and test (effort: easy – working: yes)

The current stable version of Tomcat is 6.0.20. You can either install it using your favorite package manager, or if you’re on Windows get it from the homepage at http://tomcat.apache.org

You shouldn’t have to make any changes to get our web app to work on Tomcat. Once you’ve installed Tomcat copy the generated war file to the webapps folder, and run:

$ bin/startup.sh

You should be able to go to http://localhost:8080/deploy_test

Tomcat really is the bread and butter of Java Application Servers, especially outside the Enterprise.

Deploy to Websphere and test (effort: hard – working: yes)

NB: Websphere 6.1.0.11 was the first application server I ever deployed Sinatra too, it failed miserably. I spent a long time debugging and playing with it to make it work properly. The biggest problem stems from the fact that the default way of using rack as configured by warbler doesn’t work.

Start by logging into the administration console, it should be something like – http://localhost:9043/ibm/console

Click on ‘Servers’ and when it expands select ‘Application Servers’. From here you can setup a new server instance that we can use for our testing. Call the instance something like ‘deploy01′. You can follow the default creation steps all the way through.

Once you have a server instance to test on, you can deploy a new application. The big gotcha as mentioned above is that you can’t use filters, the good news is that it’s quite easy to switch out the rack filters for a rack servlet. Rather than duplicate information that already exists, I’ll link to the place I learned to deploy warbler to websphere, http://clint-hill.com/2008/11/26/jruby-on-rails-and-websphere/.

Websphere is not the easiest application server to setup in general, but once you get it all configured it is fairly robust. Worth the effort if you want an application server you won’t have to restart constantly (as can be the case with Documentum on Tomcat).

Deploy to Weblogic and test (effort: n/a – working: no)

Setting up Oracle Weblogic 10.3.2 was nothing short of awesome. The install process is intuitive and speedy, though the size is quite large, at ~600Mb,  compared to smaller cousins such as Tomcat. There is a wizard that walks you through the process of setting up your first domain, what Tomcat would call an instance and Websphere would call a profile, once the software is installed. I chose the default options for everything and had a running Weblogic server in about 20mins (including download).

My previous experience with Weblogic, was the version bundled with Documentum D6SP1. I’ve found both that version and the current fully fledged Oracle version to be a joy to work with.

Once the installation process is finished you can find the administration console at http://localhost:7001/console, you can then login using the username/password you picked during installation.

From the admin console it is a simple matter of clicking on ‘Deployments’ on the left and then when the screen loads clicking ‘install’, browse to the directory with the deploy_test.war file in it and start the install.

You should now be able to access the deployed application at http://localhost:7001/deploy_test…

Application initialization failed: no such file to load — rack from C:/Oracle/Middleware/user_projects/domains/base_domain/servers/AdminServer/tmp/_WL_user/deploy_test/qwtgi/war/WEB-INF/lib/jruby-rack-0.9.5.jar!/vendor/rack.rb:1 from C:/Oracle/Middleware/user_projects/domains/base_domain/servers/AdminServer/tmp/_WL_user/deploy_test/qwtgi/war/WEB-INF/lib/jruby-rack-0.9.5.jar!/vendor/rack.rb:22:in `require’ from C:/Oracle/Middleware/user_projects/domains/base_domain/servers/AdminServer/tmp/_WL_user/deploy_test/qwtgi/war/WEB-INF/lib/jruby-rack-0.9.5.jar!/jruby/rack/booter.rb:22:in `boot!’ from C:/Oracle/Middleware/user_projects/domains/base_domain/servers/AdminServer/tmp/_WL_user/deploy_test/qwtgi/war/WEB-INF/lib/jruby-rack-0.9.5.jar!/jruby/rack/boot/rack.rb:9 from C:/Oracle/Middleware/user_projects/domains/base_domain/servers/AdminServer/tmp/_WL_user/deploy_test/qwtgi/war/WEB-INF/lib/jruby-rack-0.9.5.jar!/jruby/rack/boot/rack.rb:1:in `load’ from

Clearly thats not going to be the case though. This issue is very similar to the error message received for JBoss.

Conclusion

Not a bad scorecard really. Of all the Java application servers that I tested, only JBoss and Weblogic proved to be a problem. I’ll be retesting these two periodically to see if support has been improved (there are open tickets with JRuby). Until then I hope that this has been useful.

I originally published this post to the Early Access Area for Documentum RESTful Services.

In the second part of my exploration of Documentum RESTful Services I promised that we’d delve into browsing around the docbase. Rather do that with ruby I thought I’d grab a copy of JQuery and have a look at what it takes to access the services using what is an increasingly popular javascript library. The most important part of delivering on this is the JQuery call:

$.getJSON(path, ...);

The important thing here is that because of concerns about cross-site scripting we can only call local paths. While getJSON allows remote paths now with the help of JSONP style callbacks, those require server side cooperation to work. I don’t know that they’re not implemented in Documentum RESTful Services, but flipping through the documentation I couldn’t find anything of that nature.

Rather than creating a War for one html file, I decided to show off nginx. Nginx is a superb webserver, and more importantly in this case, a reverse proxy. It has gotten a lot of attention in the Ruby on Rails community, which is where I fell in love with it. After grabbing a copy of nginx it’s simply a matter of doing a minor adjustment to the nginx.conf file.

location /resources {
  proxy_pass   http://127.0.0.1:8080/resources;
}

This will mean that anything below the /resources directory on our webserver is passed off to the tomcat instance on 8080. We can start nginx, and discover it works perfectly. Now that we can use just a path to reference the services, lets get started. We start with a fairly blank html file:

 <html>                                                                 
 <head>                                                                 
 <script type="text/javascript" src="jquery-1.3.2.js"></script>         
 <script type="text/javascript">                                        
   // we will add our javascript code here
   $(document).ready(function() {
        // do stuff when DOM is ready
   });  
 </script>                                                              
 </head>                                                                
 <body>                                                                 
   <div id="results"></div>                                         
 </body>                                                                
 </html>

Notice the included jquery.js, the call the $(document).ready, and the results div. We’re going to use the call I mentioned earlier ‘getJSON’ to populate the results div with the results of calling the ‘folders’ resource with no arguments. Lets look at the code.

$.getJSON("/resources/core/repositories/test_repo/folders.json",
       function(data){
       $.each(data.dataPackage.dataObject, function(i,item){
           $("#results").append(item.properties.object_name+"<br/>");
       });
});

If you put the above html file in the right place and run the example you’ll be prompted for the password, just like you would if you actually went to the resource endpoint itself, then you’ll see a nice list of the root cabinets. Suppose we want to more than just display the root cabinets though, that we want to use the returned results to allow us to click around the docbase. Here is an example that does just that:

It’s very rough, and doesn’t take into account many variables, such as relationships that are returned aren’t necessarily folders. But it does illustrate the ability to access Documentum using libraries like jQuery now.

I’m working on a prototype at the moment that requires me to insert data into offline tables (offline as far as Documentum is concerned). The examples that I’ve found all resort to specifying the exact structure of the table. create table DMI_OBJECT_TYPEx ( R_OBJECT_ID VARCHAR2(16) NOT NULL, I_TYPE NUMBER(10,0) NOT NULL, I_PARTITION NUMBER(10,0) NULL); The example above is smaller than most of the tables I have to create. The weakness with this is that you have to look up the table structure. The writers probably chose this method because the much simpler syntax shown below also brings any data that is in the table. CREATE TABLE DMI_OBJECT_TYPEx AS SELECT * FROM DMI_OBJECT_TYPE; My initial thought was why not copy the table and then just truncate it, but after a bit of searching I stumbled upon the solution. Essentially adding a WHERE clause to the end of the query that never evaluates to TRUE enables us to take the structure without the data. CREATE TABLE DMI_OBJECT_TYPEx AS SELECT * FROM DMI_OBJECT_TYPE WHERE 1=2;

Up until now I’ve been using a fairly standard capistrano deploy.rb. The problem is that as we speak I’m trying to deploy a patch from my Windows work machine and it’s not working. The ethics of fixing one of my project from home aside, this is a problem. When I try and deploy the following error  comes up

can’t convert Net::SSH::Authentication::Pageant::Socket into IO (TypeError)

The change I was trying to deploy stops in its tracks. After quite a bit of searching I found a thread about the error that dates back to Capistrano 2.5.3… from 2008. What a shining example of open source.

To be fair the error isn’t necessarily in Capistrano, it may in fact be in Net::SSH. The sad thing is that we’re over a year later, and nothing has been done to fix the problem. Is the number of people that use Ruby from Windows machines so low that no-one has managed to fix it in a year?

I’m not expecting Jamis to fix it. I get that he has too much to do, and doesn’t have time to give out a bunch of freebie support, but I’m now faced with the choice of either trying to fix the problem, a task I don’t currently have time for, or ditching capistrano.

I’ve considered switching to heroku in the past and just never made the leap. For a start the app in question has to many moving parts for heroku. One thing I did like though was the notion that to deploy all I had to do was ‘git push target master’  and the app would be updated and deployed.

After toying around with rolling my own solution, I stumbled upon mislav’s gem. It lacks some of the features that I’m looking for, but its a good deal closer to the level that I need. It lacks the bloat of capistrano, which is important, because the biggest barrier to me getting in and fixing capistrano would be the size of the library and knowing where to start.

I very quickly migrated my existing application to use git-deploy. It’s not perfect for every problem, particularly if you’re doing multi stage deployments, etc, but at least I’ll be able to do a deployment everywhere I can get access to git now.

I’ve finally moved out of the little apartment I had in Darlinghurst. It’s one of those times when I wish that I’d had a car handy so that I could have loaded everything onto a trailer rather than having to pay a removal company. Not that I was desperate to move all the furniture myself.

Before the move I was quite happy at the thought of letting a removal company carry all our stuff down three flights of stairs. That was before Morgan Removals damaged a bunch of stuff because they didn’t strap my new 3 month old fridge in. They get to the other end, two-three suburbs over and the fridge is on a 45 degree angle, laying across everything in the middle. I was not impressed. Had I known then the extent of the damage I would not have paid them.

The front of the fridge now has scratches and dents that weren’t there. The bed frame, one of the items that it fell on, is unusable, since it broke it in two. The most annoying part about the later damage, is that they snuck it into the house. Didn’t bother to tell me that it had been destroyed, just quietly placed it in the bedroom, and left me to discover it when I went to put the bed together later that day.

When I contacted the company,  I was first palmed off on the driver, no response from the actual company. After two follow up emails, I was told that they weren’t going to replace the broken items, stating:

“Unfortunately no responsibility can be taken from our side. On our confirmation letter we advise all our clients to visit our insurance page for all your insurance needs and it is your responsibility to take out  accidental cover or not, as all removal companies advise.“

Accidental damage insurance is laughable, because the fact is that it wasn’t an accident, the movers didn’t bother to strap the fridge in. That’s not accidental in my books, it’s negligent. Before you ask, yes, I explained in detail to the customer service representative about the fridge not being strapped in and falling over.

I usually try to be rational and calm about these things, but they way they’ve treated my stuff, and their desire to lie and avoid responsibility is going to make me spend tomorrow leaving negative reviews on every site I can find.

If you live in Sydney and are looking for a removal company, then I would encourage you to steer clear of Morgan Removals. I’ve included the pictures showing the damage for your information.

(And I’m hilarious)

Seems that simple lessons don’t get learned. Don’t get get me wrong, its very hard to protect every aspect against hackers who try to pry they’re way into your site. Storing passwords in plain-text is just dumb though. Even if the passwords for your own site are hashed, the proliferation of storing third party login details (which you could still encrypt with a symmetrical key) is a time bomb.

RockYou is just the latest site on the internet to learn this hard lesson. Supposedly the hacker is one of the good guys, but there is no guarantee that someone else didn’t get the information as well. It’s an argument for doing away with passwords altogether, how long will it be until we can use public/private key authentication with websites. It is now accepted best practice with SSH, since the advent of widespread SSH bruteforcing.

Private key authentication solves a lot of the problems with websites storing password information, the hacker would have gained nothing besides the ability to verify users were who they claimed to be.

I’ve finally decided to release ActiveDocumentum.  ActiveDocumentum is a Ruby Gem that I created to bring some of the goodness, learned by pulling apart ActiveRecord, to accessing Documentum. Its nowhere near as mature, but I has been doing the job pretty well so far for the scripts and sites I’ve been using it for. It has a dependency on JRuby because it hooks into the DFS client libraries to facilitate connecting to Documentum. I’m going to post some further examples, but for the time being, here it is.

http://www.github.com/vertis/active_documentum

You’ll need a Documentum repository, JRuby and a copy of the DFS sdk to play with it.

Update: I’ve also done up a quick sample which you can find on my github (right next door to the actual library).

http://www.github.com/vertis/active_documentum_sample