Tag: patterns

Architectural Styles and ReST

Introduction

Realising a consistent approach in your implementation projects is quite a challenge. One of the tools I have applied successfully is to introduce some kind of reference architecture or a reference model, or even better: both, in a reference implementation. This document outlines a high-level application architecture that I have found to be very useful in building such a reference implementation.

Goal

I try to provide an umbrella presentation of application architecture principles. Target audience: solution architects, developers and information analysts.

Architectural Styles

Every system is developed according to one or more architectural styles — whether explicitly or implicitly, consciously or not, consistently or not. It is never that there is no architecture. In fact even in organisations investing heavily in modelling architecture most of the architecture will always remain “undiscovered”. A pragmatic approach has shown to be most effective: discovering and uncovering gradually, based on pragmatic and strategic considerations.

The goal is to apply these styles consciously and consistently, thereby reaping the benefits and manage the implications of using a specific style.

Theoretical Foundation

The architectural styles are made explicit based on the dissertation of Roy Fielding (2000): Architectural Styles and the Design of Network-based Software Architectures: https://www.ics.uci.edu/~fielding/pubs/dissertation/top.htm.

This dissertation was the foundation of the definition of RESTful services that have become the de-facto standard for service connectivity, especially for front-end services.

Over the years I have found his overview of architectural styles (not just the RESTful ones!) very useful. Especially for a business-centred architecture it is possible to very succinctly define which style should be used in what part of the architecture (primarily front-end, service layer, business component, back-end connectivity).

Business Services

Business services are about the “how and what?”. Business logic is the core of any system. “Splintering” of business logic is a known and pervasive problem:

  • Functional changes have unpredictable side-effects all over the place
  • Identical functionality is replicated all over the place

Therefore: isolate business logic in one location. This is a logical structuring of the business. It is structured around two aspects:

  1. Business Objects
  2. Business Functions/Processes/Interactions

This isolation is quite pervasive: efforts should be made to avoid including all kinds of “technology” into this component. It should be able to “simulate” the business without needing concepts like computers, networks, databases or such.

Domain Model

The Domain is the component containing the business logic. This component is by itself capable of executing all desired functionality (business processes). It is a relatively isolated component, in that it is able to do that even if completely disconnected from the other parts of the architecture.

It consists of a set of business concepts that together are reflections of the “real” world of the business. If you are a car leasing company, your business domain consists of Car, Driver, Contract, etc. By including the behaviour in this model, you avoid creating this as a data model or in a set of hard-to-integrate business functions.

These domain objects are related 1-1 to Business Services in a service layer, or, as I prefer to call it, a service landscape. The Domain Model is implemented in these Business Services.

Architectural Styles for the Domain Model

Several styles come into play for this component, however the main one I prefer is Peer-to-peer.

  • Architectural Styles: Peer-to-peer (Fielding). Also see Domain Model (Fowler)
  • Distributed Objects (Fielding) for connecting Business Services
    • (Mutual) connections through remoting
    • Light-weight message queues
    • Micro-services to contain the objects
    • Bidirectional relations are only allowed within packages
    • Package dependencies always unidirectional
  • C2 (Fielding, more on this later) for firing events when something relevant happens (i.e. state changes)
    • Objects know nothing about (potential) subscribers on this events

Technical Services

Business Services are Central (satellite model, explained in many other articles).

  • Outside-in: messaging
  • Inside-out: events

Any change entering through a technical service on the perimeter primarily updates a business service. This guarantees any change to be dealt with by the business logic concerned (Larman: Expert Pattern). The business object then propagates the change to subscribers, without explicitly doing that. The change is propagated “indirectly” through firing an event into the blue. The event source has no awareness about any possible event sinks.

Example scenario:

  1. Database (Technical Component) updates Contract::contractDate (Business Service)
  2. Technical service sends a message first to Business Service Contract
  3. Contract fires change event and optionally executes additional business logic
  4. Synchronization services propagate for example to another database

The following sources introduced this architectural style:

Role of Technical Services

Main rule: everything is a service in an SOA. Note that the approach introduced in this article is especially effective in a micro-service landscape, although still very useful in an architecture with more coarse-grained services.

Technical services should be viewed as a synchronization mechanism between the Domain Model and the “real world”. The “real” world is kept in sync with the internal software representation (see: Through the Looking Glass).

For this synchronisation, technical components are created:

  • Web (or other) front-ends
  • Databases
  • Security components
  • Etc.

Also known as Integration Services

Technical Services are (extremely!) simple, parameterised, standard components (framework) and thus are either used/procured (for example Spring, or an Oracle database adapter) or built only once, carefully documented and thoroughly tested to be used intensively within projects. (Extreme) light-weight, performance and bug-free are keywords when thinking about Technical Services.

Business Services ⇄ Technical Services

  • Architectural style: C2 (Fielding)
  • From business to technical (inside-out): notifications (events) – indirect
  • From technical to business (outside-in): requests (messages) – direct
  • Conceptually asynchronous (synchronous is an implementation decision, not a design decision)
  • No direct link with Technical Components allowed (i.e. database, front-end): always go through Technical Services

Technical services are usually chained:

  • Technical Component ⇄ TS1 ⇄ TS2 ⇄ TS3 ⇄ Business Service

Each technical service should responsible for only one integration aspect (for example data aggregation + caching), thereby creating services that are small, easy to understand and maintain, with well-defined interfaces.

Kinds of Technical Services

Below is a model of possible technical services from the Open Group, where they are referred to as Integration Services. Again, these components are used as the glue layer between business services and technology such as front-ends, legacy systems or databases. Other categorisations are possible as well. Note that these components should not be confused with Hohpe’s Integration Patterns.

Integration Services
Kinds of Technical Services

Architectural Style for all Technical Services

The main style here is Uniform Interface (Fielding). Publishing of events and/or subscribing to events by Technical Services is also possible (C2 style). Technical Services are instantiated/parameterised, not programmed. Specific technical services can introduce new styles.

Front-end Connectivity

Front-end should be viewed as a kind of Technical Component, with which you connect (as any Technical C0mponent) through Technical Services. The architectural style here is: Layered-Client-Cache-Stateless-Server (+ Uniform Interface) (Fielding) = REST architectural style.

Spring REST Controllers for example can function as REST resource.

Business Service lookup can be done lightweight (HATEOAS), the original directory services concept has proved to be too heavy-weight.

Framework guidelines

Note that part of the architecture will evolve into a framework, and it should. Framework development should however not be a goal in itself. This has proven to be an anti-pattern in many organisations. But especially because the glue layers are so exquisitely simple, they are the ideal candidates for framework inclusion.

Integration Layer controllers all inherit from a generic Controller. This results in ca. 90% code reuse.

For example (code in generic Controller class):

this.linkBusinessServiceWithTechnicalComponent(businessService, technicalComponent);

This instantiates and parameterizes an integration adapter that takes care of the interface to the technical component. Adapters are only procured, or developed for the framework (not by developers in the projects). Projects only reuse them. Integration Layer Controllers can also been viewed as containers for adapters.

Connecting Persistency

Usually Business Services still fetch data “directly” from databases or legacy (PL/SQL, JPA). Similar to front-end connecting this should be implemented with Technical Services taking care of persistency-as-a-service. This results in an almost complete independence from physical databases and legacy systems, enabling migration and cleanup. For this existing services need to be modified or refactored, which often feels like a daunting task.

Technical Components

What are Technical Components?

Examples:

  • Databases
  • Web front-end
  • Security servers (for example Keycloak)
  • External systems in general (Salesforce, SAP, etc.)

These are isolated or wrapped by providing them with:

  • “well-defined” service interfaces
  • “connectors” (sockets) to connect with

Like domain objects they are completely client-agnostic. The basic strategy is to leave them be as much as possible. If the above-mentioned styles are used you will find you are almost able to completely isolate them, which means that, at least functionally, you are able to migrate them to your leisure.

Add, Don’t Modify Pattern

Intent

We want to be able to grow our domain models. They must be infinitely scalable. Growing an already large model should not be more difficult than a small model. Change comes in two forms: changing existing functionality, or adding new. We want there to be as little difference as possible between the two, regarding testing, validating, and modelling.

Also Known As

No known current variants of this pattern are known to me.

Example

A banking system has functioned for years but now an invasive change is necessary: the format of the account numbering system needs to change from a proprietary 9-digit number to a standardised 20-digit one. Since this change will impact all existing accounts it seems to be a daunting job: all accounts will be touched, and this change may have numerous side effects. It may even be impossible to cover all test cases completely.

Context

Problem

Solution

Don’t change existing objects. Not for changing existing functionality, nor to add new functionality. Existing test cases should not need to be modified. Instead functionality is modified by adding more objects. Since the model has been created using the Active/Passive pattern it is a loosely coupled model that does not hinder us in doing this.

Structure

Existing objects will still be used by the existing part of the system. However new parts, those that need to work with the new format, will “enhance” those existing objects by reusing them. This is done by wrapping the existing bank accounts in a new bank account object, delegating all necessary behaviour to the existing object, but (as it were) enriching the object with the new behaviour.

Variants

The Decorator pattern from the Design Patterns book. In fact this is the possible implementation of this pattern.

Known Uses

Consequences

  • Changes do not lead to mandatory changes to existing components
  • Test cases do not need to be modified (same rule applies: change will lead to new components, in this case test cases, not to changes to existing components, in this case test cases)
  • The system may only grow, this is a possible adversary effect of the applying the pattern. As with any pattern, it is not applied in isolation: you will still need to audit your solution for legacy migration, which will phase-out obsolete parts of the system.

See Also

Global complexity, local simplicity

Models created using the modelling techniques we talk about on this site are extremely scalable. In fact we call them infinitely scalable. This corresponds to the classification in Martin Fowlers Patterns of Enterprise Application Architecture (The Addison-Wesley Signature Series) book of the Domain Model pattern. Maybe it is something readers have overlooked, but the curve for the Domain Model is assumed to be linear.

However there seems to be a price to be paid for this scalability:

  1. models will usually grow to be almost impossible to comprehend
  2. models will contain more and more redundancy
  3. the size of the model will have a negative impact on performance
  4. … and will all of this not lead to a downgrade of scalability since adding new functionality will become difficult?

In this article we will try to explain why we think that, in the context of models endowed with the characteristics of a Domain Model, these problems are not valid or at least much less relevant.

Fowlers three main patterns
Fowlers three main patterns

There is an interesting observation about unit tests within the XP (eXtreme Programming) context. Unit tests are usually extremely local: as a rule there is a set of unit tests for each public interface in each class.
For example if we have a class Person, containing the public interface move(newLocation), there could be tests like testMove. The example warrants only one test, but slightly more complex messages could have one or two more (note that if you need more than three test methods for one message you probably have a candidate for refactoring since the method is too complex anyway).
Now if we evolve our system, we do not write tests for larger or overall processes. Just the combined effect of local unit tests will guarantee that the system as a whole, no matter the size, will behave correctly.

This is an effect of synergy. However to work as intended, we need to learn to work with complexity instead of against it. Especially technically oriented people have an innate fear of complexity. They often talk about reducing complexity. And indeed, in many situations this has proved to be a valuable strategy because it let them avoid paths leading to disaster having to deal with unmanageable code. However the net effect in practice is that they gradually introduce so much code to “reduce” the complexity that they end up accomplishing the exact opposite.

Real-size systems that are deployed can be seen to consist of considerable amount of code that “does nothing”: it is only there to make the system manageable, to “reduce” complexity. It is there because the builders would be unable to find bugs, to repair them, or to add new functionality. It is code that contributes zero to the end user functionality. What if we were able to build systems that consist of relatively independent, small, understandable, testable components? Components that do only what they do, without any overhead? What if we could add these components to our existing, running systems, just by “throwing them in”, as a fish in a pond? And what if our systems are continuously attempting to regain the optimum balance between those components?

Reminds you of something? The biological world operates as such, of course.

Time Inversion Pattern

Inverse Time

Intent

The Time Inversion pattern is used to model dynamic collaborative behaviour between model components. To choose which responsibilities should be endowed upon a component, the modeller considers a goal that should be reached by one or more key components. This goal should be the end-goal. Reasoning backward from this end-goal the modellers searches for other components to delegate behaviour to, but in an inverted time frame: the first modelled object implements the last step in reaching the goal, the second the last step before that, and so on. The choice of candidate objects is from a list of active objects obtained from the Active/Passive pattern.

Also Known As

Example

A model that implements a business process that builds and delivers cars would probably contain an active object Car. The main responsibility, as last in the business process steps, could very well be: deliver yourself to the customer. Sometimes a responsibility named “sell yourself” is used.

How can a car deliver itself? Well, we define a method (=responsibility) in the class Car, called Deliver. We dive into the method that should be executed when the car does this. What is the last thing that should be done in order to satisfy the Deliver responsibility? We might decide that this is the fact that the car dealer drives the car to the customer site. So we need a CarDealer, and we create this class/object with the responsibility DriveToCustomer. How can the car dealer drive the car to the customer site? He needs to have the car. The car, in order to be able to exist on the dealers site, needs to build itself: Build. This is a new responsibility, in the case of aggregates largely delegated to the composing elements. Etc.

Context

Problem

  1. Where do you begin modelling the customers problem domain? This strategy makes this simpler: begin with the goals the business wants to accomplish.
  2. How can you know that the way you model things that are done, are not modelled in such a way as to be almost impossible to change?

Modelling a complex business domain often is very hard, and it is even harder to find out where to begin.

Solution

Walking backwards along the process chain, creating active objects along the way, the car would build itself, parts would build themselves and assemble themselves, factories making the parts would pay themselves, etcetera. This is a backward inferencing process, a demand-chain process. In general, the goal objects generate the process to achieve their goals.

Structure

The resulting object graph is able to perform one or more trees of business processes, where the roots of the trees are the final business results. However it is important to remark here that the object graph structure itself is not at all directly conceived from these processes. Instead, various, sometimes incomplete, processes are taken during the modelling phase to “play out” by existing and newly created objects. This leads to an object graph that can be validated to at least be able to perform the defined processes, but usually is able to perform a superset of these input processes.

Variants

Known Uses

The Time Inversion Pattern is one of the two primary patterns used in eXploratory Modelling (xM). Behaviour of objects is realised by thinking about the final goal of an object, and to decide upon the last previous action needed to accomplish this goal. This final or last action is usually delegated to a new object (usually an Active Object), upon which the modeller jumps to this object, and tries to find out the previous action in this object in the same way.

Consequences

By using the demand-chain modelling strategy models gain in flexibility, especially in the case of uncertainty, vague business requirements, processes that are difficult to made concrete or should be highly optimised.

The resulting solution, the model, may lack structural consistency: models can be redundant and impossible to understand on a global level. This is a potential problem, but we argue that in the case of the kind of model we talk about, this is usually not so. We create models that are complex, difficult to overview or comprehend, but because of the high cohesion this actually works in favour of the model instead of against it.

An important pattern to employ in this case of growing models is the Add, don’t Modify pattern.

See Also

This pattern is usually combined with the Active/Passive pattern, because the choice of where to place the previous process step to is determined by selecting an active object that is usually the passive one in the real world.

This pattern is well known and documented in the supply-chain management theory. A good book introducing this is:

Supply Chains: A Manager’s Guide (paperback)

Active-Passive Pattern

teapotIn a previous post (The Essence of OO) I talked about two patterns in modelling complex systems that I consider to be the most important, the active/passive pattern, and the time inversion pattern. In this post more on the first, and this time the pattern is described in a pattern template.

Intent

A domain model will become large and very complex. The intent of the model is to capture the domain, that is: reflect the real world. This model should be functional when very small, but be willing to grow, indeed to be infinitely scalable. However, in order to avoid a model that is unmanageable we should not make the fundamental error of trying to model the domain as close to the real world as possible. A simple trick is to turn active and passive around: active objects in the real world are modelled as passive, indeed even almost devoid of complexity in the sense of behaviour or responsibilities. Passive objects in the real world on the contrary are modelled as active, taking initiatives. The responsibilities endowed upon them are precisely those activities that, in the “real” world, are done with them. This will help in realising the intent (this is another aspect of turning-around).

Also Known As

  • The World of Roger Rabbit
  • Activity inversion
  • Looking-glass world
  • Anthropomorphising

Example

A person pays a bill. He owns an account to pay bills with. The person is the active object in the real world. He pays the bill by opening or accessing the account, creating a transaction, telling the transaction the amount that he copies from the amount contained in the bill, as some other information. And finally he tells the transaction to go ahead. He is a very complex object, with many responsibilities and many relations with other objects. The objects bill, transaction, account, are passive. In fact they contain only the information (data) necessary for the person and usually contain little responsibilities.

Context

There are many passive, exceedingly simple objects (i.e. an Account), and few active, exceedingly complex objects (i.e. a Customer). Usually the active objects correspond to actors in the use cases, or external agents or triggers. The passive objects are data containers, with information used in a business process.

Problem

The sequence is also rather critical: first do this, then do this. This is called process brittleness. The process is not easily changed, in fact resists change. The few active objects are strongly coupled to a lot of passive objects (typically more than three) with which they perform all kinds of tasks. These tasks usually entail little more than changing values (data) in these objects.

The system can capture the business problem domain reasonably well when still small, but as the domain grows, that is the model of the business domain in the software, it becomes increasingly difficult to manage, until expanding the domain model becomes virtually impossible. All kinds of dependencies arise. The model collapses under its own weight.

Solution

The Active Passive pattern describes the world, or complex systems, by employing two rules:

  1. Objects that are active in the real world (machines, systems, persons, committees) are modelled as passive, that is they do not initiate actions themselves, and any responsibility they have (and usually they have responsibilities) are delegated to other objects.
  2. Objects that are passive in the real world (documents, cars, products) are modelled as active, that is they initiate actions, take responsibilities upon themselves to reach a business goal. They typically delegate most of the actions needed to finish a business process to other active objects.

Inversion of activity turns activity and passivity around. In the example: the person has almost no responsibility. Instead the bill has the one main responsibility in this context, namely to “pay himself”. The bill starts the entire process of paying by contacting the person (the bill has presumably been created by someone, and this someone told the bill: “I want some money from this person.” The bill only needs to ask the person for one thing: an account payable. After receiving this account he proceeds by telling the account to create a transaction for himself. The bill is done now, the chain of responsibility has been transferred to the account. However, account does not hold this for long. He creates a transaction for the bill, and subsequently tells this transaction to go ahead. All the rest is done by the transaction. A simple rule of thumb is that any responsibility of an active object in the real world can be relocated to a passive object in the model.

Structure

Variants

A well-known variant of this pattern is the Expert Pattern, first described as such by Craig Larman in his book on [amazon_textlink asin=’9332553947′ text=’UML and Patterns’ template=’ProductLink’ store=’reflektis-20′ marketplace=’US’ link_id=’4158c5dc-3df8-11e7-9af7-55efd60c4ba7′].

Known Uses

This pattern is usually combined with the Time Inversion Pattern.

Consequences

The distribution of complexity over the model is more balanced. Some objects are rather passive but are usually linked to several other objects. Some objects are rather active but linked to few others. This way there are not complexity hubs in the model. Processes are more demand-chain and more easily changed. Processes are not the source for the structural model, but rather the responsibility of few key objects (like “bill” in the example above). This recurs in the way the domain model structure comes into being, by employing the Time Inversion Pattern.

See Also

The Active Object pattern should not be confused with this pattern. This is a pattern that is mainly used to model objects that contain their own thread of control, to be used in concurrent systems. For the Active Object pattern you can read more here.

This pattern is usually applied together with the Time Inversion pattern.

There is a strong relation between this pattern and CRC sessions. In CRC sessions workshop participants play the role of objects in the customer domain. The facilitator usually emphasises the need to choose objects in the domain that are usually considered passive.