Exploring Documentum RESTful Services – Part 2
I originally published this post to the Early Access Area for Documentum RESTful Services.
In the second part of my exploration of Documentum RESTful Services I promised that we’d delve into browsing around the docbase. Rather do that with ruby I thought I’d grab a copy of JQuery and have a look at what it takes to access the services using what is an increasingly popular javascript library. The most important part of delivering on this is the JQuery call:
$.getJSON(path, ...);
The important thing here is that because of concerns about cross-site scripting we can only call local paths. While getJSON allows remote paths now with the help of JSONP style callbacks, those require server side cooperation to work. I don’t know that they’re not implemented in Documentum RESTful Services, but flipping through the documentation I couldn’t find anything of that nature.
Rather than creating a War for one html file, I decided to show off nginx. Nginx is a superb webserver, and more importantly in this case, a reverse proxy. It has gotten a lot of attention in the Ruby on Rails community, which is where I fell in love with it. After grabbing a copy of nginx it’s simply a matter of doing a minor adjustment to the nginx.conf file.
location /resources {
proxy_pass http://127.0.0.1:8080/resources;
}
This will mean that anything below the /resources directory on our webserver is passed off to the tomcat instance on 8080. We can start nginx, and discover it works perfectly. Now that we can use just a path to reference the services, lets get started. We start with a fairly blank html file:
<html>
<head>
<script type="text/javascript" src="jquery-1.3.2.js"></script>
<script type="text/javascript">
// we will add our javascript code here
$(document).ready(function() {
// do stuff when DOM is ready
});
</script>
</head>
<body>
<div id="results"></div>
</body>
</html>
Notice the included jquery.js, the call the $(document).ready, and the results div. We’re going to use the call I mentioned earlier ‘getJSON’ to populate the results div with the results of calling the ‘folders’ resource with no arguments. Lets look at the code.
$.getJSON("/resources/core/repositories/test_repo/folders.json", function(data){ $.each(data.dataPackage.dataObject, function(i,item){ $("#results").append(item.properties.object_name+"<br/>"); }); });
If you put the above html file in the right place and run the example you’ll be prompted for the password, just like you would if you actually went to the resource endpoint itself, then you’ll see a nice list of the root cabinets. Suppose we want to more than just display the root cabinets though, that we want to use the returned results to allow us to click around the docbase. Here is an example that does just that:
It’s very rough, and doesn’t take into account many variables, such as relationships that are returned aren’t necessarily folders. But it does illustrate the ability to access Documentum using libraries like jQuery now.
ActiveDocumentum
I’ve finally decided to release ActiveDocumentum. ActiveDocumentum is a Ruby Gem that I created to bring some of the goodness, learned by pulling apart ActiveRecord, to accessing Documentum. Its nowhere near as mature, but I has been doing the job pretty well so far for the scripts and sites I’ve been using it for. It has a dependency on JRuby because it hooks into the DFS client libraries to facilitate connecting to Documentum. I’m going to post some further examples, but for the time being, here it is.
http://www.github.com/vertis/active_documentum
You’ll need a Documentum repository, JRuby and a copy of the DFS sdk to play with it.
Update: I’ve also done up a quick sample which you can find on my github (right next door to the actual library).
Exploring Documentum RESTful Services – Part 1
I was chatting to Craig Randall on twitter a little while back and he let me know that they were just about ready to go into Early Access for their new web service platform - Documentum RESTful Services. I was excited then, and I’m even more excited now that I’ve had my first taste.
RESTful web services are hard to explain to people. I usually fall back on explaining them by linking them to Ajax, which is a Web2.0 technology that people are more familiar with. If you wanted to use Ajax to make your Documentum webapps more friendly before Documentum RESTful Services, then you’d be out of luck (short of creating a custom server side component).
RESTful web services are easily accessible because you don’t need any funky client library to access them. Pop open any web browser and type in:
http://localhost:8080/dctm_rest/resources/core/repositories/<RepoName>/folders
And you’ll get back an XML representation of all the root folders (after you’re prompted for a username/password). RESTful web services use only standard HTTP, which means that you’re cutting out the middleman of having to encode requests/responses in Soap (or some other container).
They’re automagically more language agnostic. Gone is the need for a complex client side library. For example, while it should strictly be possible to access DFS in any language, this doesn’t translate into a reality. So Documentum RESTful Services will allow us to step away from relying on Java/.Net and expand into using high level dynamic languages (python, ruby, javascript).
Let’s do a simple example in Ruby to demonstrate:
require 'rubygems'
require 'httparty'
class DctmRest
include HTTParty
base_uri 'http://localhost:8080/dctm_rest'
def initialize(username, password)
@auth = {:username => username, :password => password}
end
def repositories
options = { :basic_auth => @auth }
repository = "test_repo"
self.class.get("/resources/core/repositories.json", options)
end
end
d = DctmRest.new('username', 'password')
puts d.repositories.inspect
The example is probably more complex than it even needs to be. If I look at the output:
{"repository"=>[{"queriesUri"=>"http://localhost:8080/dctm_rest/resources/core/repositories/test_repo/queries.json", "repositoryType"=>"managed", "typesUri"=>"http://localhost:8080/dctm_rest/resources/core/repositories/test_repo/types.json", "foldersUri"=>"http://localhost:8080/dctm_rest/resources/core/repositories/test_repo/folders.json", "checkedoutUri"=>"http://localhost:8080/dctm_rest/resources/core/repositories/test_repo/objects/checkedout.json", "uri"=>"http://localhost:8080/dctm_rest/resources/core/repositories/test_repo.json", "name"=>"test_repo", "objectsUri"=>"http://localhost:8080/dctm_rest/resources/core/repositories/test_repo/objects.json"}]}
I can see a very succinct set of other URIs that I can use to get other information. In the next post I’ll go through how to browse around a Docbase using Documentum RESTful Services. For now, take the time to sign up for the early access program, so you can help give feedback on what is a giant leap forward for Documentum in the web2.0 world.
Installing Documentum Administrator 6SP1 (Patch Release) on IBM Websphere 6.1.0.23
I’ve spent the majority of the last 2 days trying to get DA 6SP1 Patch Release to install properly on IBM Websphere 6.1.0.23. It had been a while since I’d last done the install, and I fell into the same traps as last time, forgetting where to set the class loader and so forth. After getting past these I was left with the application unhelpfully still showing a white screen and error messages that revealed (looking back) very little about the true nature of the problem. Having finally conquered the problem, I thought I would provide the steps necessary so that anyone else trying to get a WDK application working on Websphere as at least one more thing to try.
Grab a copy of da.war from the EMC Powerlink Download site.
- Unpack da.war and add the following files to a directory
- Put dfc.properties into WEB-INF/classes (As described in the deployment guide)
- ibm-web-ext.xmi & ibm-web-ext.xmi goes into WEB-INF – https://solutions.emc.com/emcsolutionview.asp?id=esg91105
- Find a copy of xml.jar on the Websphere App Server in ‘Websphere\AppServer\java\jre’ and put in ‘WEB-INF\lib’ - https://solutions.emc.com/emcsolutionview.asp?id=esg93106
- Install the application using the IBM Administration Console.
- Change the order the classes are loaded in by expanding the Applications node and click the Enterprise Applications node. Select the application called da_war (or equivalent wdk app). Click the Manage Modules link. Click the da.war link. In the Class loader order drop-down list, select the option ‘Classes loaded with application class loader first’ – https://solutions.emc.com/emcsolutionview.asp?id=esg93177
- You can then save and start the application.
The xml.jar file is appropriate for D6SP1 only, previous D6 versions have a slightly different set of instructions (see linked support note). Any feedback, corrections, suggestions greatly appreciated.
ActiveDocumentum teaser #1
Having stepped away from trying to get Documentum running on linux, I’ve been working on bringing the permissions through into our custom Lotus Notes/Documentum client. At the moment when a user imports a document to a given folder it does no permission checking before the fact, just errors nastily when it can’t write the document. A short stop by Prasad’s blog gave me a query to start from.
SELECT i_all_users_names as users FROM dm_group
WHERE group_name IN (SELECT r_accessor_name FROM dm_acl
WHERE object_name IN (SELECT acl_name FROM dm_sysobject
WHERE r_object_id = '0c0xxxxx80009e16'))
ORDER BY i_all_users_names
This is superb, but it doesn’t actually go far enough. If I was to attach it to my application and tell it not to let me import anything into the folder if my name wasn’t on the list I wouldn’t be able to copy anything into my home cabinet. It enumerates the members of any group within the ACL but doesn’t deal with named users & also the owner.
This will give me a chance to show off what you can do with my ActiveDocumentum project. ActiveDocumentum is a working name for a JRuby gem written to provide functionality similar to what can be found in ActiveRecord, with the addition of repeating attributes and other DQL specific items. While the final code for the below function may end up being in Java or .Net. Doing it using ruby allowed me to very quickly scope out the logic of what would need to be done to get the permission that a user had on an object (NB: If there is a way to do this simply with DQL I’m not aware of it):
require 'config/environment'
def get_effective_permission(r_object_id, user = nil)
return 0 if user.nil?
folder = DmFolder.find(:first, :columns => "owner_name, owner_permit, acl_name", :conditions => {:r_object_id => r_object_id})
current_level = 0
if folder.owner_name.eql? user
current_level = folder.owner_permit
end
acls = DmAcl.find(:all, :columns => "r_accessor_name, r_accessor_permit, r_is_group", :conditions => "object_name = '#{folder.acl_name}' ENABLE(ROW_BASED)")
acls.each do |acl|
if acl.r_accessor_name.eql? user
current_level = acl.r_accessor_permit if acl.r_accessor_permit > current_level
elsif acl.r_is_group.to_s.eql?('1') and not acl.r_accessor_permit.nil?
group = DmGroup.find(:first, :columns => 'r_object_id, i_all_users_names', :conditions => {:group_name => acl.r_accessor_name})
current_level = acl.r_accessor_permit if acl.r_accessor_permit > current_level and group.i_all_users_names.include?(user)
end
end
end
return current_level
end
r_object_id = '0c0xxxxx80009e16'
user = 'John Doe'
puts get_effective_permission(r_object_id, user)
As you can see the above function takes a user and returns the highest permission that they have, by checking if they’re either directly listed on the ACL or in one of the groups on the ACL. The bottom line is that with ActiveDocumentum I can write less lines of code for one off scripts that I have to create, and include complicated logic that is lacking in straight DQL scripts. I will be posting more details on this in the near future.
D6.5 Experiments (Part 2)
I’ve been trying to do an install of Documentum using the Linux/Oracle downloads without much success. This may well be more on the Oracle side of the fence, but this started when I put Oracle XE(10.2.0.1-1) on a newly installed CentOS 5.3 virtual machine, and then tried to get Documentum (6.5SP1) installed and configured, only to get the majority of the way through and be told that the minimum is Oracle 10.2.0.3, despite what the release notes say.
Since 10.2.0.1-1 is the latest Oracle XE, I concluded that it must not be supported and moved on to trying 11.1 (Standard One). The biggest downside to this is that this Oracle installer is far less friendly than the XE installer, and at the end of it, I am not left with a TNSNAMES.ORA which Documentum needs. I am not an Oracle expert by any means, I would prefer to use PostgreSQL or MySQL, but they’re not a supported configuration. Even ignoring ‘supportability’, since this project is only experimental anyway, there is no information on whether it is even possible to get it running using these databases. Since there is a large amount of mapping/optimization between DQL and the underlying SQL I would tend towards it not being possible.
Does anyone have any advice or instructions for getting D6.5SP1 installed from scratch on linux? Failing that I will persist with my attempts, and let you know how it turns out.
D6.5 Experiments (Part 1)
Spent most of the night downloading Documentum 6.5 components for Linux & Oracle. While I would prefer to use an open source database like PostgreSQL as the backend, the actual database isn’t going to matter much once its installed (they don’t offer it as an alternative anyway). What I’m looking to do is have a D6.5 environment that I can tune to be as fast as possible. This includes doing things that you wouldn’t be able to do on a System that has to be supported, such as using Nginx as the front end instead of Apache Httpd with Apache Tomcat.
I’m still a bit up in the air as to which App Server I’m going to use. Tomcat is the obvious answer and the starting point for clients that don’t want to invest in IBM Websphere, BEA(Oracle) Weblogic, or Jboss. I do think that Apache Geronimo and Jetty are worth investigating as alternatives. One of the complaints that I’ve heard against Tomcat in production environments is that it is ridiculously unstable when running Webtop. With the app servers needing to be restarted every week (or more frequently). IMO there is more likely an issue with Webtop itself rather than Tomcat. I haven’t heard of stability complaints when running other webapps.
One of the big complaints against Apache Tomcat is that it isn’t a full stack App Server like IBM Websphere, for instance architecturally IBM Websphere has a lot more under the covers. I’ve not installed Apache Geronimo before but it’s supposed to be more of a complete App Server. I’ll let you know how my project goes anyway, got to run off to work now.
Quick Update
It’s been almost a month since my last post, disgraceful. While I have the will to maintain a blog, I certainly don’t have the time. We’re currently in the thick of a large Documentum deployment, which means that there are bucket loads of new users that each have their own issues with the system.
I have learned SO much over the last few months, hopefully I’ll have a chance to sit down over the next week or so and distill some of that wisdom. Until then, keep safe.
The Christmas cleanup
Over the weekend I went to melbourne for a couple of christmas parties, the work one and the extended family one. I’m into my last week at work before the chrismas break and I must say that I am dying to put up my feet and enjoy the a nice sunny and warm christmas. I know there are people that think christmas should be all white and cold, more power to them, but I’m not one of those people. I like the fact that we can have our social occasions outside at the beach or in the sun.
One of the things that has to happen this week is for me to make sure that the system is in the best possible condition. Create documentation about how to keep the system running for the on call person that will be responsible for dealing with any issues that arise and provide as much knowledge based material as possible. Not to much mind since I don’t want my job to disappear over the christmas break, but enough that when I come back on the 5th of january I don’t walk into a disaster zone.
I’m secretly working on a tool that can be used to monitor each of the Documentum components and send me information when one or more of them breaks. It won’t be mature by christmas but it’ll be good if it notifies me of any irritating problems. I also need to finish up as many of the outstanding tasks as possible. All this should make for a hectic week.
Documentum, old world document management (or why to avoid Documentum 6.0SP1)
I’m sick of raising support cases for things that should just work. I’m frustrated that a product as buggy and unstable as Documentum 6.0 SP1 can be proposed as a solution (by EMC). There are a lot of good things about Documentum, but there is a lot of bloat as well. This will be a growing list of stupid problems that I’ve found, I’ll start with one item and go from there.
- Clicking on ‘Save As’ on a permission set with a name over a certain length causes it to bomb out. Why? because it tries to give the permission set a temporary name that is longer than the allowed 32 characters. No chance to override this name, no graceful handling, just a fat error message and the chance to reload DA and try again.
- When using LDAPSync (which you probably need to get hotfixed anyway what with it having so many bugs fixed) ‘Groups and their member users’ does not update the content of mapped attributes even if they change (for instance during a rename).
- My favorite of the last 2 weeks: BOCS and DTS are not supported in the same solution. This problem is related to DTS going looking for content that is still parked on the BOCS server. It’s been fixed in 6.5…the problem is that it was EMC that built the solution that we’re now using. There is some instability in the DTS services, which is something I have to follow up, but EMC/Documentum had better be hoping that the two are not related.
I think part of the problem is that Documentum like a lot of old world enterprise applications has a very tenous hold on some of the techniques that allow for more error free development. I mean was the code even unit tested…