Most web application architectures I encounter during my work as an IT consultant are very traditional monolithic applications and have some very common limitations related to productivity, testability and maintenance. While they pretend to be Service Oriented, in practice they are usually monolithic applications. In this post I’ll try to highlight some of these limitations while also introducing a different, in my opinion better, way to create these applications.
The state of IT in a large organisation
Most enterprise developers have read “Domain-driven Design” (E. Evans, 2004) by now, resulting in a multi-tiered applications focussing on business logic . In my opinion this is a very first important step towards truly customer oriented applications. The IT application is improving and impacting people’s work and/or life and developers grow an understanding of their user’s mindset.
As shown in the picture above interface and infrastructure concerns are cleanly seperated from the domain and application layer. Now, let’s introduce a web interface, mobile client, batch import/export, integration with third party systems and it all seems to start to fall apart (or does it?).
Even in a large organisation applying the best of practices, one will start to see applications like this after a while:
- Good: sharing of existing development to implement common concerns such as business logic
- Bad: multitude of applications doing 80% of the same thing and usually sharing different versions of above common components. Maintenance of these systems is not so easy, especially if multiple teams are involved.
- Bad: true integration testing becomes a horror, because the application under test tries to do too much. For example using Selenium to integration test end to end (web client to database) is usually very fragile and limited. Do you test only the GUI navigation or go as far as verify if records have been correctly created in the database? Do you want to rely on this kind of test for verifying that your DB integration was done correctly? I think not: a simple front-end change can fail this test, resulting in the usual bad practice of @Ignoring tests.
- Bad: front-end (web screens) and back-end (business logic) are treated equally. This has an impact on the entire module life cycle: development, maintenance, scaling in production…. Keep in mind that the two solve fundamentally different problems:
- The front-end should optimise for user experience.
- The back-end should aim for correctness of business logic and integration with other systems.
The SOA way through communication and consolidation
A better way would be to implement your system as a smaller coherent units of logic (Services) that communicate with each other. The scope of this unit is determined by your application domain, but is usually only a small part of the business. In our example we would have at least 4 components to deploy:
- a back-end system that talks to the database and implements all the business logic .This unit would expose functionality through a standard interface (REST, SOAP or some other enterprise standard, but keep it simple). Depending on the complexity, this can be separated into multiple units.
- a web interface talking to the back-end.
- a batch process talking to the back-end.
- a mobile application facade to talk to the real back-end.
The benefits of this Service Oriented Architecture are:
- better consolidation of logic in one place, giving rise to a single version of the truth for a part of the application. New applications can reuse the existing logic and interfaces.
- everyone is using the same version of the backend. The team implementing the backend can easily upgrade code deployed in production.
- easier integration testing of each component as their responsibilities are much better defined.
- more incremental way to build up the system architecture.
- embraces heterogeneity in your system for more flexibility. Different services can follow different architectures and runtime platforms (Java, .Net, Scala…). For example the web application can be implemented as a HTML5 application that is rendered fully on the client but it can just as easily be architected using a server side MVC framework (JSF, Tapestry, Struts…)
There are some pitfalls with this approach though:
- seemingly complexer deployment due to more components to deploy. This is a solved problem given the proper level of automation.
- transactional boundaries are usually determined by the scope of a service. How can we tackle transactions for composite services that consume multiple services where each of them changes records in the database (but they can fail independently)? There are actually multiple solutions to solve this, but the easiest one by far is to let go of the “transactional fear” and be content with eventual consistency of data. Every system will experience bad data at some point in time, so it is better to learn to deal with this than maintain the illusion of control.
- an inherent danger of immobility when trying to maintain services depending on the culture of your organisation. None of these services should be in a state without ownership. Architecting applications like this, means that all services need to be ready to evolve to respond to the requirements of new applications. Teams should talk to each other when introducing breaking changes.
In the previous architecture one could take the existing code, modify it and deploy the new version together with the application. While this approach is more practical, it will become a mess quite quickly.
Is this a radical change in the way we construct web applications? I don’t think so.
It is rather a call to better define responsibilities and simplify the scope of applications to give each service/component the flexibility it deserves. Complexity can still be achieved by integrating multiple, easy to understand services and re-exposing the complex functionality as a new service (composite services).
The careful reader will have picked up on the fact that this philosophy goes further than the technical design of an application. Service Oriented Architecture is something to think about for end users, functional analysts, architects, developers, testers, project managers… Everyone involved in a project should appreciate splitting it in smaller digestible pieces instead of focussing on solving everything at once.
Something that is truly beautiful is often determined by simplicity, not complexity.