According to HashiCorp, setting up and managing cloud environments requires an industry standard. The company decided not to wait for someone else to come up with one and presented The Infrastructure Cloud prior to the HashiDays last week in London. This is a holistic approach to provisioning, deploying and ultimately decommissioning cloud environments. Important take-away: don’t do everything at once, but do right what you do.
The continued integration of HashiCorp’s product offerings, which we wrote about earlier, did not happen by chance. The company has set itself the goal of providing a new industry standard in cloud management and everything related to it. That includes provisioning, deployment, security, management, and destruction.
A dedicated platform team plays a key role in this approach. It oversees the use of approved resources for all involved and is separate from the cloud infrastructure end users, such as development teams, which use the environment to build their applications.
The platform team handles provisioning and management and provides best practices for app dev colleagues. HashiCorp calls these ‘golden patterns‘. According to the company, this approach prevents unnecessary costs, reduces security risks, and allows each company to mature its cloud policy at its own pace.
All products in a company’s portfolio, including all the components, plugins, and apps accompanying them, must fit into this model, which supports the maturation of companies’ infrastructure as they move to the cloud.
To this end, HashiCorp distinguishes three steps: adoption (the initial move to the cloud), standardization, and finally scaling up. The company provides a matching product for all these steps, but says there is no reason why other parties couldn’t benefit from this model.
Waste of money
According to Hashicorp, many companies struggle to make the most of their move to the cloud because of difficulties managing the enlarged infrastructure footprint. It is simply harder to maintain oversight and control, which slows productivity and poses security risks. To support that claim, Hashicorp points to some recent studies.
For example, research from Gartner in 2023 shows that 35 percent of corporate resources dedicated to the cloud were wasted. This could be for various reasons, including unused test environments, expensive solutions gathering dust because no one is using them, inefficient workflows, unused or obsolete resources, and slow processes.
In addition, IBM, which intends to acquire HashiCorp, calculated in its most recent Cost of a Data Breach report that a cloud environment breach costs a company about 4.5 million dollars on average. This includes costs associated with detecting and resolving the breach, notifying affected parties, cleaning up after the fact, damage control, and possible fines. The report considers both willful theft and accidental leaking of sensitive information a breach.
Yet another report, PwC’s Cloud Business Survey 2023, indicates that barely 10 percent of companies believe they are reaping sufficient benefits from their move to the cloud. The supposed advantages fail to manifest due to inefficient work processes, and ad hoc solutions. Also, there is too little focus on standardization, a step that is sometimes never taken.
Successful cloud transition can be elusive
In other words, a successful transition to a cloud environment, in any form, might remain elusive. HashiCorp cited the above figures to clarify that its concept of a maturity process for cloud environments is not just an interesting thought exercise but a necessity. That it also provides a meaningful framework for its updated, integrated product suite is an added benefit, to be sure.
At least in the adoption phase, companies can still get away with an ‘expeditionary team’ sinking its teeth into their company’s cloud challenge. Ad hoc approaches and tactical solutions (as opposed to strategic) are still possible to make things work initially. However, Meghan Liese, HashiCorp’s vice president of product marketing, says you can’t stop there. “Sometimes, companies forego the necessary next step and remain stuck in the pioneering phase.”
When a roadmap including checks and balances is missing for that next phase, development teams may start doing things other organizational units don’t know about, or vice versa. Consider using an AI without management knowing about it (a compliance nightmare). Or running 11 million resources in the environment, many of which may be deprecated.
Standardization is essential
Therefore, according to Liese, standardisation is the next essential step for a company taking its cloud strategy seriously. That involves a centrally led approach, with the responsible Platform Team running a tight ship, including supplying policies on cloud usage and resources. This team manages, maintains, and aligns with internal stakeholders, app dev teams, and external services.
The next phase involves scaling up, during which the cloud infrastructure matures. This is where e.g. self-service provisioning and automated remediation come in.
In the cloud management value chain, Hashicorp sees a clear role for itself in infrastructure (provisioning, deployment, and management) on the one hand and security (protecting, inspecting, and connecting) on the other. The two pillars form the basis of the so-called Maturity Model. At each stage, this model provides a level of refinement and a corresponding increase in the value of the infrastructure.
Infrastructure-as-code
To be clear, just because a company is in the adoption phase does not mean it’s just mucking about. The decisions made are already robust, for example, providing infrastructure-as-code from Day 1 to provide everyone with a consistent and repeatable code base. That means everyone is provided with the same best practices, and no one has to reinvent the wheel.
At the same time, building the environment is done in a way and at a scale appropriate for such a start-up phase. The time is simply not yet ready to implement all kinds of advanced functionalities. In the adoption phase, the initial focus is still on things like version control, role-based access and encrypted network management.
Letting the cloud mature
In the next standardisation phase, policy-as-code is introduced, meaning users don’t have to constantly think about the policies involved with the infrastructure. They are simply baked into the code. This is also the stage to introduce disaster recovery and automated audits to ensure continuity, as well as integrated and automated privilege-based access. In other words, this is where the cloud matures.
The next phase, scaling up, includes a high degree of self-service for the components or teams that depend on the cloud environment and its services. Think of no-code provisioning and a standardized process for app delivery. Lifecycle management is highly automated in this phase, and encryption as a service appears here. In other words, what was worked so hard for in the previous phases should start to pay off here by allowing for the cloud and its functionalities to grow with relatively less effort.
Each HashiCorp Cloud Platform (HCP) product is woven throughout the model. Regarding the infrastructure side of things, Terraform, Packer, Waypoint, and Nomad play the most prominent roles. When it comes to security at whatever stage, Vault, Boundary, and Consul come into play. Vagrant, which simplifies setting up VMs, somewhat stands apart from this whole story. It did not appear in the model touted by HashiCorp, in any case.
Credentialing is step one
So, how do you get started with such a maturity model? The first things on the list are provisioning, managing the environment properly and securing identity and access management. “Getting your credentialing in order to eliminate that attack vector is actually the easiest problem to solve,” says Senior Developer Advocate Kerim Satirli. Security is also a top priority for many of HashiCorp’s partners, especially in Europe.
Surely the topic of security is important in the U.S., but teams over there tend to emphasize efficiency over security. That is the impression of the development advocacy team, Hashicorp’s company ambassadors led by Satirli. “In the US, they’re more into coding for the sake of coding,” he says. In Europe, especially because of directives like NIS2 and legislation around privacy, there is more concern about security. For this reason, HashiCorp points to provisioning cloud infrastructure via infrastructure-as-code. Not coincidentally what Terraform excels at.
Although the big hype around cloud infrastructures may now be over, Satirli recalls that during those early days, he has seen a company consider a move to the cloud about two or three times, fail to get it off the ground for whatever reason and then just giving up. Missing out on a lot of opportunities as a result. “So we don’t see this approach as purely ours. We think all companies would benefit.”
Open-source alone is not enough for enterprise cloud
As an aside, Vice President of Product Marketing Meghan Liese believes that if companies want to do cloud right (the motto at last week’s HashiDays in London), they cannot depend on open-source solutions alone. While she obviously sees the added value of open-source, she also notices the sheer breadth of available options can deter enterprise customers.
“Many companies are simply looking for a solution that can scale over time. They aren’t looking for as many choices as possible at once.” HashiCorp provides a set of validated designs that the company says will help customers get started, especially in the adoption phase.
HashiCorp has written a white paper on The Infrastructure Cloud that can be downloaded here.