Antifragile Enterprise Microservices Architecture – Impacts on Financial Performance

posted in: Microservices | 4

1  What Is Antifragility?

Antifragility was first elaborated by N. N. Taleb in 2012. Antifragile does not mean that a system is robust or resilient; it is not the opposite of fragile. However, antifragile means the ability to increase the capability of the system so that it becomes stronger or more robust when subjected to volatility, shocks, stressors and errors either from your system or other systems. In other words, what does not kill you makes you stronger or antifragile.

2 Human Body System

If you visit a car factory you will see many different teams each performing a different task. One team will be responsible for the car’s design, another will assemble the car’s engine and another will paint the car’s frame.

Even though each team performs a different task, they all share a common goal: to make cars.

The same can be said about our bodies. Different systems or organs within the human body are responsible for different jobs, yet they all share a common goal: to keep us alive.

2.1 Body Cell

Each system within the human body is made up of trillions of cells. Each cell plays it own simple role and interacts with other individual cells to complete more complex tasks. There is no ‘super cell’ that does everything.

2.2 Cell lifespan cycle and Regeneration

Each cell type has a different lifespan cycle. Some cells live longer than others (Figure 1). Furthermore, each cell has the ability to regenerate by creating a new cell as the old one dies. However, this does not mean that the whole organ dies or fails when old cells die. The organ continues to function normally without impacting the human body.

Cell renewal rates in different tissues of the human body (bionumbers.org, 2017)
Figure 1: Cell renewal rates in different tissues of the human body (bionumbers.org, 2017)

2.3 Strain

The human body responds to strain, it does so by adding muscle mass in the relevant areas. If you have a tendency of using your forearm muscles all the time, they will bulk up. Similarly, your body muscles lose their mass when the strain level is reduced by neglecting to exercise.

2.4 Feedback Loops

There are many feedback loops in biological systems. For instance, we have internal controllers for maintaining our body temperature at 37°C, which is the optimal internal state at which our bodies operate best.

Body Temperature Regulation
Figure 2: Body Temperature Regulation

As you can see in (Figure 2), the body exceeds a certain temperature it takes action to regulate it. This is detected by nerve cells that give feedback to the part of the brain that regulates body temperature, and the brain sends out a signal for the body to cool itself down by sweating.

3 Microservices Architecture

Microservices or Microservices architecture is defined by (Richardson, 2017)as “An architectural style that structures an application as a collection of loosely coupled services, which implement business capabilities”

Microservices share many characteristics similar to those of the human body cells. Each service performs a different task, but together, all services share the same target: to keep the system alive and produce maximum business value. There are different patterns that can be used to implement microservices depending on an organisation’s requirements and capabilities.

3.1 Lightweight and doing one thing

Let’s imagine that you are building a booking system similar to that of Expedia.com or cheapflights.com.au that takes orders from customers to help book your flights, hotel and car renting. The application consists of several components including: booking website interface, mobile application and multiple backend services to orchestrate your booking order as shown in (Figure 3).

Microservice Architecture Example (Online Booking System)
Figure 3: Microservice Architecture Example (Online Booking System)

The system will mainly consist of 4 different layers:

  • User Interface (UI)

UI is an interface for the system with the exception of any server side business logic. It could either be a website, mobile native application or a reporting dashboard system for internal operations.

  • Communication Layer

The communication layer will enable the UI to communicate with different services through a single gateway without the need to directly access any of the services.

  • Service Layer

The service layer will contain all microservices which can be utilised by the organisation. In our example, we only have five services; however, in the real world it could be hundreds of services to handle each aspect of the system.

  • Integration Layer

All services should not have any direct link to other services or share data storages. All communications between services should take place within the integration layer to ensure service decoupling.

Therefore, each of the system services is does one thing and handles a particular task.

3.2 Autonomy

Each service needs to be changed independently without affecting other services and should work as a separate entity. Therefore, each service can be modified and integrated with our system without the need to modify other services.

In addition, each service needs to expose one or many communication interfaces like API (Application Programming Interface) and should collaborate with other services through this interface only. Therefore, each service can control what it wishes to keep open for other services to use and what to hide. If each service shares a lot of functionality with our services, then our consumers will be more coupled to our service and demand more changes to decrease service autonomy.

3.3 Easy to replace

A microservice should be as easy to replace as possible. Replacing a microservice can be sensible when its technology becomes outdated or if the microservice code is of bad quality that it cannot be developed further. The replicability of microservices is advantageous when compared to monolithic applications, which can hardly be replaced.

4 Antifragile Characteristics of Microservices

For years, many enterprises have focused on building robust systems that they invest heavily to avoid failure by handling all known scenarios.

At the same time, business architecture keeps changing over time and enterprise IT systems need to adapt to these changes. The IT system needs to respond to business changes either by changing business rules, extending the IT system or integrating with other systems.

The more changes are made to an IT system, the more the system becomes complex as it needs to handle more predications of failure scenarios when integrated with other systems. This approach often leads to a more complicated system which is hard to be understood by developers. Also, the maintenance cost of the system will become very high, delivery cycles of any changes or extensions are longer and it becomes very difficult to integrate the existing system with other systems.

All these problems lead to a more fragile system that cannot adapt to unpredictable changes. The result is a volatile system that leads to either total system failure or full costly replacement with a brand new enterprise system built from scratch.

Antifragile microservices is a natural design approach for creating antifragile systems that assumes all parts of a system will fail.

4.1 Built for failure

The system needs to handle different types of failures and assume all the parts will fail. Most distributed systems have a higher chance of failure when it comes to network, application issues or even hardware faults. Therefore, each service can be temporarily unavailable to consumers and system architecture needs to minimize the impact of partial outages of any service system.

4.2 Response to stressors

Each service system needs to respond to the stressor by scaling up or down based on the stress level, similar to your body cells. The more you stress your muscles cells by going to the gym, the more your body responds by generating more cells to help build more muscles. Similarly, you will start to lose your body muscles once you stop training and remove the stress on your muscles.

Also, each service needs to handle stress independently without the need to scale the whole system up or down. We never expect to increase our whole body muscle mass by shoulder weight lifting exercises alone.

4.3 Feedback loop

There are many types of feedback that should be gathered from each service to help identify abnormal issues. Another reason is to aid in continuous improvement of each service as well as the whole system in general.

  • Service logging
  • Health monitoring
  • Business feedback

4.4 Randomness

Systems are generally designed to handle known risks and errors. As a result, a system breaks easily when unpredicted shocks hit it. A well built system needs to handle unexpected service unavailability. For example, by throwing random errors or disconnections at the system, it is possible to measure impact on the system. For instance, Netflix has implemented a resiliency tool that helps applications tolerate random instance failures called Chaos Monkey. It will randomly terminate virtual machine instances and services that run within the production environment. Exposing engineers to failures more frequently incentivizes them to build resilient services(NetFlix, 2017).

5 Impact of Microservices on Enterprise Economy

There are many characteristics of Microservices that have direct impact on enterprise revenue and growth.

The four main characteristics of Microservices are:

  • Flexibility to scale
  • Flexibility to change and extend
  • Long term solution complexity
  • Short term design complexity
Microservices Common Characteristics Mapping
Figure 4: Microservices Common Characteristics Mapping https://kumu.io/samuelya/antifragile-microservices

5.1 Flexible Architecture & Delivery Performance

The flexibility of the architecture of Microservices will have a direct impact on the solution delivery speed and cost of each project. For instance, the deployment of a monolithic application starts as a strong structured application. However, over time, more and more dependencies between the individual modules creep in. This leads to the application becoming overly difficult to maintain and update. In addition, the testing strategy of monolithic applications is complicated and slow, as the entire deployment of the system needs to be tested every time a business asks for changes to be done in one of the features of the system.

On the other hand, the system features will be distributed across different services and each service could be divided into more microservices. Further, each team is responsible for each individual service. Therefore, it will be more flexible and cheaper to modify, deploy and test each service individually without affecting other services on the system.

Flexible Architecture & Delivery Performance
Figure 5: Flexible Architecture & Delivery Performance https://kumu.io/samuelya/antifragile-microservices#flexible-architecture-delivery-performance

5.2 Agile Business Architecture & Knowledge Retention

The increasing pace of business innovation continues to push organizations to evaluate business models that will meet stakeholder expectations. Also, IT systems will need to quickly respond to these changes. The response speed of IT development teams to adapt to the new business change is essential to the business’s agility process. Furthermore, it increases the confidence level that exists between the business and IT development team.

The organisation’s employment retention will be improved when the confidence and satisfaction level increases amongst all different parties, businesses, stakeholders and IT development teams. Also, the delivery speed of the new system features will be improved when there is high employment retention and this will reduce the cost of training new employees

Agile Business Architecture & Knowledge Retention
Figure 6: Agile Business Architecture & Knowledge Retention https://kumu.io/samuelya/antifragile-microservices#agile-business-architecture-knowledge-retention

5.3 System Performance and Scalability

With Microservices, everything is more granular, including scalability and management of spikes in demand. Demand may surge in one or more services of the application. Therefore, Microservices architecture enables the system to scale only certain impacted services, rather than the entire application and its underlying infrastructure. Also, the application users will not encounter performance issues during busy times as the application will be able to scale up or down based on demand.

System Performance and Scalability
Figure 7: System Performance and Scalability https://kumu.io/samuelya/antifragile-microservices#system-performance-and-scalability

 

5.4 Short Term Design Complexity of Microservices

Microservices do not come free of charge, and although the individual complexity of a microservice may be reduced, the sophistication of orchestrating a large, distributed system is noteworthy. Consequently, Microservices architecture is shifting the complexity from space of code design and implementation to system operations. Moreover, the cost of third party tools, licences and custom implementations to manage the system operations is high with regards to the required tools for monolithic applications.

Short Term Design Complexity of Microservices
Figure 8: Short Term Design Complexity of Microservices https://kumu.io/samuelya/antifragile-microservices#short-term-design-complexity-of-microservices

5.5 Long Term Solution Complexity of Microservices

Going for the Microservices approach will definitely help the organisation in the long run by providing simple IT solutions that have a high integration ability, extension and low maintenance cost. The sample solution characteristics can be summarised as:

  • Each Micro service can be deployed independent of other services.
  • A new developer needs to understand the functionality of individual services only without extensive training to understand the whole enterprise system and start developing new functionality.
  • The development of a service is confined to one team and this team can work independent of other teams.
  • Improved fault isolation, as only the faulty service can be isolated or rolled back without affecting the rest of the services.
  • Avoid long term commitment to any technology stack and ability to adapt new technologies to increase the business value outcome of an IT system.
Long Term Solution Complexity of Microservices
Figure 9: Long Term Solution Complexity of Microservices https://kumu.io/samuelya/antifragile-microservices#long-term-solution-complexity-of-microservices

 

6 The Overview Effect

Over four decades ago, a group of astronauts saw our planet earth for the first time during the Apollo 8 mission in 1968. They described a cognitive shift in awareness after seeing our planet “hanging in the void” and this state of mentality is called “The Overview Effect”.

The view of earth hanging in the void, shielded and nourished by a thin atmospheric layer, changed their perspective about life and they realized that our planet is very tiny. Astronaut Gene Cernan stated that “There has to be somebody bigger than you, and bigger than me, and I mean this in a spiritual sense, not a religious sense and You do not see the barriers of colour and religion and politics that divide this world” (White, 1987).

 

6.1 Agile and the Big Picture

6.1.1 Agile Methodology

Agile software development methodologies were introduced into the IT business over 15 years ago. However, in the last 7 years we have seen agile methodologies invading many IT businesses around the world.

Primary development method used in organization across projects
Figure 10: Primary development method used in organization across projects (HP, 2015)
Agile adoption over time
Figure 11: Agile adoption over time (number=475 firms with some adoption of Agile methods)(HP, 2015)

The main reason for agile technology success is due to the short feedback cycle that exists between the delivery team and stakeholders. Agile improves the collaboration between all interested elements, has quick delivery to the market, gathers the feedback and adopts these feedbacks within the next delivery cycle.

Heart of Agile
Figure 12: Heart of Agile (Thomas, 2015)

6.1.2 Losing the Big Picture

While many organisations are adopting agile methodologies, it is unfortunate that they are losing focus on the big picture with regards to the enterprise level of the whole system and are focusing more on project delivery.

The reason for losing the big picture is that agile and scrum in particular focuses on iterations or sprints, on the short term. As depicted below.

The Scrum Process (Tal, 2015)
Figure 13: The Scrum Process (Tal, 2015)

Most projects in the enterprise world will be considered successful projects if the team is able to deliver a set of features to solve particular business problems within a set budget and time. However, during the project design and development cycle a lot of adjustments and alterations can be made to the project based on unforeseeable problems or business feedbacks during iteration demos. This design changes and compromises have the ability to deliver the project on time and within budget. However, the big picture of the whole system and integrations of the system with current and future systems are left out.

6.2 Collaborative Enterprise Architecture and the Big Picture

Over the last few decades, IT systems have had a huge impact on how enterprises do business. As the scale and complexity of a system grows, the need for proper planning in designing, building, and operating it becomes more and more important in order to understand and visualise the big picture of the IT system. Enterprise architecture is responsible for controlling the complexity of IT systems as well as identifying the enterprise business needs from IT systems.

The enterprise architecture ensures that the IT system of the organisation is stable, adaptable, agile, efficient and antifragile.

The basic definition of architecture is “Architecture is the illustration of the structure and behaviour of a system and its fundamental parts, plus a set of principles that guide the system’s long-term evolution” (Langade, 2012).

Yesterday’s solutions are the main reason for most of today’s IT system issues, limitation and complexity. Also, to solve today’s problems there has to be collaboration amongst all stakeholders. In the same way that mistakes are made collectively and not individually, solutions need to be found collectively and not individually.

System stakeholders are people who deal with creation, evolution, and operation of the system. For example,

  • The owner who pays for it
  • The strategist who conceptualizes it
  • The planner who plans its creation
  • The designer who designs it
  • The implementer who builds it
  • The subcontractor who provides constituent parts for it
  • The field support staff who maintains or operates it in the field

Conclusion

Antifragile Microservices architecture will have plethora of advantages to reduce the building and maintenance cost of enterprise system in the long term. Also, the Microservices architecture is not the silver bullet for IT system problems and requires more planning and devise a first coarse-grained domain architecture that leads to the first microservices. Therefore, the way to microservices is evolutionary. It is not necessary to start adopting microservices for the whole system from the beginning. Instead, a stepwise migration is the usual way.

References

BIONUMBERS.ORG. 2017. HOW QUICKLY DO DIFFERENT CELLS IN THE BODY REPLACE THEMSELVES? [Online]. Available: http://book.bionumbers.org/how-quickly-do-different-cells-in-the-body-replace-themselves/ [Accessed 26/09/2017 2017].

HP 2015. Agile is the new normal. In: ENTERPRISE, H. P. (ed.). Hewlett Packerd Enterprise

LANGADE, S. B. U. B. S. 2012. Collaborative Enterprise Architecture, US, Morgan Kaufmann.

NETFLIX. 2017. Chaos Monkey [Online]. Available: https://github.com/Netflix/chaosmonkey [Accessed 22/10/2107 2017].

RICHARDSON, C. 2017. What are microservices? [Online].  [Accessed 22/10/2017 2017].

TAL, L. 2015. Agile Software Development with HP Agile Manager, US, Apress.

THOMAS, S. 2015. Did HeartOfAgile emerge from PDCA? [Online]. Available: http://itsadeliverything.com/did-heartofagile-emerge-from-pdca [Accessed 11/10/2017 2017].

WHITE, F. 1987. The Overview Effect: Space Exploration and Human Evolution, New York, Houghton and Mifflin Co.

4 Responses

  1. Randy

    Very interesting. I came to a similar conclusion about microservices.

  2. Alagar Raja

    Very nice article and able to understand micro services architecture in a super simple format..