Is there a rationale behind using application platforms for large distributed applications ?

On the software market there are number of software solutions claimed to be software application platforms. I have myself in the past couple years work for a software edition company proposing such a solution. I therefore now have a quite good understanding of what they provide. As any one in the IT business will have seen, there recently is much focus on virtualization and cloud computing. Also ideas such as SOA (service oriented architectures), EDA (Event Driven Architectures) have also started to be generalized at least among software solution architects.

One of the biggest lessons I have learned working with such a platform is probably how to build highly flexible industrial strength workflow-oriented distributed systems. A common erroneous view (that I previously had myself) is that in a workflow-oriented system there is a workflow master which distributes tasks to available workers. This master would have an understanding of who works for him and decide who is to do a given task to distribute. The problem with this vision is that the master is doomed to be overloaded and will eventually become the bottleneck of the whole system. The error is in thinking that the workflow’s job is to distribute tasks while his role is in fact only to keep track of what is going on. The master keeps track of the state of each process and implements the process evolution logic described in the workflow definition. Managing the load is done by determining the number of workers dedicated to a given task.

Also, a great idea I adhere to is that a software system (and in particular an application platform) should not impose a particular development language. One of the nastiest effects of one-language orientation is that it limits the scope of software which can be integrated in to an application developed using the system. This can result in different options none of which are satisfactory in the end:

  • rewriting very good software and often more poorly
  • setting up complex cross-language communication without necessarily having a guru at hand
  • relying on external execution which has its own load of traps (platform specific file paths, command line argument protection, etc.)

Also, contrary to most final user applications or small applications where one computer hosts multiple applications, in large high-volume applications this organization is very often reversed : multiple machines are dedicated to only one application. Each machine is dedicated to host one particular component (e.g. the database, the workflow engine, a worker, etc.)

In the scope of all this, is there any rationale in using an application platform (an more particularly one limited to a specific language) to build such large application ? Can’t the OS itself play the roles usually attributed to application platforms ? In the scope of virtualization restarting a virtual machine (and it’s host OS) can as easily be automated as restarting an application server. Packaging systems have made deploying and configuring a target host very easy and fast. With a well defined target host, installing the virtual machine and the target host OS can be done within minutes.

Leave a Reply