Maestro and Open Infrastructure: Open Up for Greater Value

The Cloud migration and adoption keeps on being a hot topic for modern businesses scaling from start-ups to multi-national corporations.  The scope of Cloud usage grows dramatically, and, meanwhile, grow the complexity of requirements, expectations, and the resulting infrastructures.

Moreover, typically, more than one cloud provider is used, which leads to introducing various services, tools, offerings, and integrations that may differ significantly in usage approaches and procedures.  This gives extra complexity, as calls for the need to build an approach when all elements act as a single effective eco-system. 

This is where the Platform Engineering concept comes from. It focuses on designing, building and maintaining platforms or frameworks aimed at supporting applications development and deployment. When it comes to Cloud Migration and adoption, a Platform is a resulting structure that connects all elements, atop which a cloud infrastructure is built. It is aimed to enable unified management, audit, costs control, and security control for all parts, and provide extra value to them, where possible.

Components: Proprietary or Open Source?

The choice of components is a strategic decision, and one of the main stages here is selecting whether we buy a license/subscription for a proprietary one or use Open Source for each element of the platform.

Using proprietary offerings is quite convenient. It allows you to get a ready solution, tested on production, supported and regularly updated by the provider. The vendor also typically takes a big part of the responsibility for the product security and compliance.

On the other hand, when dealing with this type of offerings, you get higher costs, limited customization, updates that may not match your business needs. You also put your infrastructure to the vendor-lock risks, which is especially undesirable when we speak about government, social, and other institutions, that need to work smoothly and without interruptions, and operate with data and processes that affect thousands and millions of people.

This is why, there are regulations and recommendations, that advise on addressing Open Source components. As example, the Open Source Software Strategy 2020-2023, approved by the European Commission, is aimed to encourage software, knowledge, and expertise sharing and reuse to “deliver better European services that benefit to the society and lower cost to that society”.

Although Open Source may be more complicated than just “take and go” paid solutions, it gives lots of benefits to its end users – vendor-lock protection, cost effectiveness, integrability, customization, code and tools visibility. Additionally, it allows to benefit from the expertise of the community, and to prove your own engineering excellence while submitting updates.

Staying Safe with the Open Source 

Still, there are also imposed risks here – the updates brough by the community may include malware functions or security threats. The performance of such tools may differ from the expected one, and the documentation may be not enough.

So, from all the scope of available Open Source tools – which to select to make sure that they are safe and effective?

This question has arisen within the Open Source community for quite long time ago, with  the Cloud Native Computing Foundation focusing on it precisely (along with the proprietary solutions). They did an enormous job which resulted in establishing the process of Open Source solutions review and assessment, as a part of their Cloud Native projects support, oversight, and direction.

In essence, they developed a framework within which the solutions are reviewed, tracked, checked and evaluated, coming the way from sandbox to graduation, and then, once the time comes, to Archive.

Thus, the products that get to the Graduation group are considered to be safe and reliable for using within your software and platform engineering.

Maestro as an Open Source Cloud Management Platform

Maestro was initially built as a platform for unified infrastructure management across public and private clouds within a dedicated enterprise.

Our team has over 10 years of experience in building a cloud management platform, and we invest a lot in continuous Cloud expertise improvement, which includes certifications and close cooperation with partners teams on our skills growth.

Thus, when it came to re-considering the initial approaches we used for Maestro and building it as a product, we took the industry best practices, market expectations and trends, to create a long-term strategy.

And here is where we turned our attention to Open Source, in two directions.

First, we addressed the CNCF recommendations to find the best components to utilize within Maestro framework. Here, we use Terraform, Ansible, Chef behind our Infrastructure as Code mechanism, Zabbix for enabling monitoring, Open Policy Agent and InfraCost for Security and Compliance, and others.

This let us effectively reuse the existing solutions and customize them according to Maestro specifics.  They also added more compatibility for Maestro, as they are standard tools that are frequently used by other enterprises – and this will make introducing Maestro smooth and easy, with minimum need for switching for the new tools. 

Second, we started to Open Source Maestro components (and you can find the repo at https://github.com/orgs/Maestro-Cloud-Control/ ). 

With its modular architecture, Maestro is convenient for sharing specific engines and solutions with the community.

We carefully investigated which modules, or engines, can be interesting and useful, as well as studied the best practices and experience of other vendors in Open Sourcing. As a result, we highlighted a set of access-related tools to be Open Sourced in the first place. 

This was an important decision which made Maestro open for the new offerings and possibilities, both in terms of development and availability. We are still the main contributor to our product, and we are taking the responsibility for its performance and security, meanwhile giving you, as a customer, visibility and possibility to reuse parts of our solution to your own benefit. 

However, managing infrastructure is not enough for building a modern platform. We needed to expand Maestro capabilities with support for microservices and application level, and that is what we did. You will find the details in our next post.