The middle layer is the new playing field

Enterspeed can be described in many different ways, and honestly, we do. One of the descriptions we have had mixed feelings for over the years is the term "middle layer", but perhaps it is one of the most fitting descriptions? We see that many web projects include a middle layer in one way or the other. In this post, we will place Enterspeed into the current times of web development mega-trends. We want to explain some of our thinking on the middle layer and its role in modern web development and hopefully spur reflections from you, the reader.
We will begin the story way back when CMSes were all the rave.

A bit of CMS history

Since the early 2000s, the CMS has been the technical foundation for most websites. It turned out that the one feature that all websites have in common is publishing content. As the web became mainstream, the days of editing HTML files when your content needed to be updated quickly vanished. Instead, the software for publishing content was the central piece of technology in almost every website. The different CMSes evolved to support more and more features of the website: analytics, email marketing, image processing, search, commerce and many evolved to what became known as a DXP (Digital Experience Platform). But two trends changed the CMS stronghold of the technology stack.

The first trend was the emergence of specialised services. While most DXPs were really good at managing content, at their heart they were Content Management Systems after all, many of the DXPs were only good enough or slightly below average for all of the other features. This makes really good sense, as each of the different disciplines of creating a powerful search engine or world-class personalisation engine are very deep technical undertakings. This realisation created the opportunity for specialised services that honed in on doing one thing really well, e.g. search, commerce or a CDP (Customer Data Platform). The term Best of Bread became a slogan many solutions architects used to describe this approach and now the Composable Architecture is a description of this trend.

The second trend is the rise of the headless CMS. Two things have driven the adoption of headless. First, the need for reusing content across different channels, e.g. a website and a mobile application. Secondly, frontend developers have taken control of the frontend. Nowadays, frontend development is a highly specialised engineering role, with a dedicated technology stack with its own build pipeline, server-side and client-side hosting solutions. Both websites and mobile applications now have their own frontend technology stacks, and the CMSes have been reduced to an editing UI and a set of APIs.

The emergence of a new playing field

The CMS has taken a step back and is today just one layer of the tech stack. This has created a new playing field for web developers: the middle layer.

The middle layer potentially has many responsibilities. Often we have seen the need for developing a layer that exposes the CMS data for the frontend, managing URLs and handling cache invalidation. Another common responsibility is to merge data from the CMS and other data sources and services (e.g. PIM, ERP, commerce etc). Essentially an abstraction is created between the different sources and frontend is created. But not necessarily an abstraction due to architectural decisions, often the middle layer is created to add missing functionality and smooth over the cracks between the different data sources. But nonetheless an abstraction with all its pros and cons is now an essential part of the website build.

Three Approaches

With the necessity of the middle layer established, we can now discuss how you built that middle layer. Generally speaking, we see 3 different approaches:

• Frontend
• DIY
• Use a third-party tool

Bear in mind that these are very broad categories and in the real world, we often see a combination. Also, we want to spur reflection and not necessarily argue for one over the other. We find it valuable to have the software engineering concept of aiming for high cohesion and low coupling when thinking about your approach to building a middle layer.

The frontend as the middle layer

When the frontend is the "middle layer" you have fewer moving parts and hence less complexity. This scenario is often desirable when your system landscape and tech stack are fairly stable. Perhaps you are just figuring out how your business works and you need to try out a lot of different things, and then it makes sense to keep your stack shallow so you can change things in the frontend only and iterate quickly.

In the long term, a frontend "middle layer" could have the opposite effect because a lot of integration code is located in the frontend; replacing the frontend is a relatively big and potentially risky task. This is due to the frontend becoming the central integration point and you risk creating high coupling. This risk can be mitigated to some degree by architectural and technical choices in the frontend code; for instance, some frontend frameworks and hosting options come with the possibility to host server-side functions.

A consideration regarding the frontend is that it is often regarded as one of the areas of differentiation or innovation, so it is expected to change relatively frequently. Therefore, you have to consider what then happens with your integration code and middle layer business logic

Do It Yourself

Injecting a middle layer as an abstraction between the CMS and the other data sources is both an architectural and operational decision. If your business is mature and stable enough to invest in a more architectural infrastructure, you want to create a middle layer that allows you to achieve more technical flexibility while also maintaining business flexibility in the long term. On the operational side, performance and security could be reasons for choosing to build a middle layer.

The main characteristics of the Do It Yourself (DIY) middle layer are high cost and high control. Mentioning high cost is not to hate on DIY; we can absolutely sympathise with building your own software that you fully control. Building your own software makes sense when you use tools that are either uncommon or customised to your specific business. We have often found that every business is unique and unique solutions are required, and often this pushes engineers down the DIY path.

Of course, the question you need to answer is, if the uniqueness and desire for control are strong enough drivers for undertaking the effort of developing, maintaining and operating a middle layer? Enterspeed is actually built on the experience of building a custom middle layer between a specific CMS and many different frontends. We have also heard stories of many similar efforts, where companies have invested large sums in large software projects to build a middle layer.

A third-party tool

Doing similar things again and again leads you to consider if a more efficient solution is available. And in fact, Enterspeed is an answer to this question. Composable orchestration and content federation are some of the keywords that describe Enterspeed and other third-party tools that have emerged in the middle layer playing field.

Before we get to the pros and cons of using a third-party tool, it is, of course, a prerequisite that such a tool meets the business requirements. This prerequisite not only brings back the discussion of uniqueness but also that the question should be more about the mix between the different approaches and not a clear-cut decision on one or the other.

Of course, we are biased, but we believe these tools make the most sense when you don't want to undertake the responsibility of maintaining and operating more software than you absolutely need. But the long term operational perspective is in the only one rational. If you find yourself again and again building a solution that abstracts away the source systems and exposes the data for a frontend, this is a signal that you could save a considerable amount of time and money by utilising a third-party tool.

Which of these 3 approaches fits into your unique situation and how you need to balance between flexibility, control and cost is impossible to answer in this blog post. We hope that we have highlighted some different considerations you can use when you need to scope your next web development project.