Cloud-driven, fast-paced, on-demand infrastructure consumption models are displacing finance teams, making engineers the new cloud financial controllers. So now the question is: Who, what, and how can I navigate this sea change?

Continue reading to learn how to define a new job, automate it, and implement a cloud cost optimization process with HashiCorp Terraform Cloud.

This guide will:

A RASCI methodology for managing overall cloud posture and expenses

A visual representation of cloud cost management in Terraform provisioning

Terraform cost management and forecasting recommendations

Using Terraform’s cost estimating features

How to leverage third-party cost optimization tools in a Terraform workflow.

Sentinel, Terraform’s policy as a code framework, may automatically restrict overspending based on cost, instance type, and tag criteria.

A Survey of Cloud Waste

These days, you pay for what you use, but you also pay for what you furnish and don’t use. In the absence of a continual governance and optimization process, significant amounts of waste might be generated.

A recent cloud expenditure study revealed:

45 percent of companies reported going over budget on the cloud.

To maximize cloud resources, over 55% of respondents either use tedious manual processes or do nothing.

30% of respondents estimate they can save up to 25% on cloud costs, while 14% say up to 50%. Worse, 27.46% said “I dunno.”

Why Engineers are Becoming the Financial Controllers of Cloud Spend

As most firms go to the cloud, they have considered core governance models that include strategy, design, operations, and cost. Most of these teams are made up of IT management, cloud technical expertise, and finance experts. Foresight and optimization are the main duties of finance.

Finances dictate that they must “get a handle on costs, savings, forecasting,” yet they have no direct control over costs. Engineers now handle infrastructure and costs directly.

The business case is simple:

Engineers are in charge of both operations and expenditures.

Engineers may now automate and manage cost controls directly.

Finance struggles to understand and anticipate costs associated with cloud workloads.

Traditional financial budgeting and on-premise hardware demand planning do not account for cost variability in consumption-based (Cloud) models.

Finance lacks control in two key cost-saving areas:

Pre-providing: Limited resource provisioning governance and control.

No post-provisioning oversight or control over infrastructure changes to save money.

Planning, Optimization, and Governance

To summaries:

• Identify cloud-ready workloads first.
• Configuration of Terraform
• Run terraform plan to estimate costs
• Apply to terraform to provision resources.
• Workloads will run and vendor tools will make optimization suggestions.
• Apply optimization suggestions from vendors to Terraform or your CI/CD process.
• Review and execute Terraform Sentinel policies for cost and security restrictions.
• Plan & apply Terraform updates
• Resources are now streamlined and compliant.

Planning — Pre-Migration and Ongoing Cost Forecasting

Cloud migrations require a multi-point assessment to verify their viability. The main assessing factors are:

• Architecture
• Case study
• Moving Cost Estimate
• Continuing utilisation expenses budgeted/forecasted for 1–3 years

Because engineers are now handling some of these duties, it makes sensible to employ engineering tools. Terraform assists engineers in these new roles by calculating the infrastructure expenses of each provisioning run based on the actual deployment plan.

You can now utilise Terraform Cloud & Enterprise APIs to automatically supply finance with estimated cloud financial data or use Terraform’s user interface to allow finance direct access to assess expenses. This can help speed up many sluggish oversight processes.

Planning Recommendations:

This information is provided via the Terraform API or role-based access controls inside the Terraform user interface to enable finance personas a self-service approach.

Many firms use Excel, Google Sheets, and other web-based applications to plan. We recommend using Terraform’s Cost Estimates API to extract data for these systems.

Predict cloud demand with Terraform modules as standard units of defined infrastructure.

Use Terraform to track application/workload growth and associated costs.

Consistently name resources and workspaces according to the financial budgeting/forecasting procedure.

Basic Patterns for Consuming Optimization Recommendations

We observe several frequent patterns that allow Terraform to access the optimization recommendations:

Manual Workflow — Review provider’s portal optimization recommendations and edit Terraform files. No automation means no automation, yet a feedback loop for optimization must begin somewhere!

File Workflow – Create a scheduled procedure to import optimization recommendations into a local repository (usually daily).

Governance — Ensuring Future Cost Savings

The last and most important component of cloud cost management is guardrails to prevent cost overruns and provide feedback. My experience is that many firms have done optimization exercises only to have their costs rise because they did not include preventative controls from the outset.

Cost Compliance as Code = Sentinel Policy as Code

To write policies that evaluate all data defined in a Terraform file, Sentinel offers a domain-specific language (DSL). Your provided resources are secure, labeled, and cost-constrained.

Customers use Terraform to establish cost policies in three main areas: Get creative):

Cost Control:

Control the amount spent

Provisioned size — Control resource size/usage

Control the resource’s TTL.

The Way Forward for Cloud Cost Management

As more companies employ cloud infrastructure, DevOps is no longer optional. Finance and engineering silos must disintegrate as the developer-operator silos do.

Engineers have far greater freedom to deploy infrastructure quickly. That means more self-control of costs. The automated, finance-monitored workflows that engineering needs to manage expenses and reclaim underutilized resources are now available in technology that most engineers already use. This avoids a tedious ticket-based method, as well as the chaos and waste of Shadow IT gone wild.