devopsy.kinsta.cloud

Category: Infrastructure/Networking

  • Adopting a Low-Code GraphQL Approach to Build a Core Data Platform

    Adopting a Low-Code GraphQL Approach to Build a Core Data Platform

    Our education platform has relied on an outdated monolithic legacy application for years. It’s an outdated system built of a single service layer on top of a database designed for use cases that have evolved over the last decade. But we’re using the wealth of data in the monolith as a foundation for our current projects, thanks to our newly adopted tools.

    With our product line continually expanding, our need for a centralized API that can sit over commonly used data has become even more critical. This includes data related to our customers’ organizational units, purchased products, licensing information, and student, teacher and class rosters, among other things.

    The stakes are high since we have multiple downstream clients depending on us. Our goal is to move some functionality and data out of the main system while making sure our design is flexible enough to adapt to future product changes. We envision a hub-and-spoke model where all our products interact through a central hub to access shared data. Efficiency, security and reliability are all top priorities. Because we’re a small team, we need to move fast and be open to new ideas. That’s why we’re using a low-code backend solution to minimize manual coding and maximize our productivity.

    A few months ago, we adopted a tool called Hasura. Its core value proposition is that it places a production-grade GraphQL API over any database, saving teams months of recurring effort in building, shipping and maintaining their APIs. We put the open source Hasura community edition to the test and found that it lived up to its claims. In this article, we’ll show you how it worked for us and how it can work for you too.

    Converting GraphQL to SQL

    Before a team can convert GraphQL to SQL, they must undergo a data modeling phase. After this data modeling phase, backend teams must complete some unavoidable tasks. One of these tasks is to build an API that allows clients to read and mutate data. Hasura provides this instantly, freeing engineers from the toil of writing basic CRUD APIs. Hasura compiles a GraphQL string into a GraphQL abstract syntax tree (AST), which is then compiled into a SQL AST. This abstraction lets us do powerful things, like using a bulk GraphQL mutation to transactionally and atomically insert or update data. GraphQL provides a flexible way for us to query our data in the form of a graph. This means that, as engineers, we get to focus more on business logic.

    API Federation, Aggregation and Composition

    Hasura’s support for remote GraphQL schemas enables us to design a service-oriented architecture in which Hasura functions as an API federation/composition layer and delegates to other Go microservices that provide specialized business logic for a particular domain. This ability to federate doesn’t just end at remote GraphQL schemas—Hasura also supports remote data sources, opening multiple possibilities for how we traverse through the graph and explore relationships in our data, including how we might join remote databases to each other or to remote APIs (REST or GraphQL).

    In greenfield projects, teams typically spend months crafting a fully-featured API, a common language that all clients can share. With Hasura, we get this in seconds and can spend that saved time writing automated tests or implementing business logic.

    Developer Dashboard

    Hasura’s Dashboard/Console improves developers’ experience. Using the dashboard, developers can craft experimental GraphQL queries and mutations against our local instance of Hasura. The embedded GraphQL playground makes it a breeze for new developers joining the team to introspect and discover our API. We can also visualize our database schema in the UI directly and make changes to it with a few clicks of a mouse (if needed), generating source-controlled SQL migration scripts like you’d find in other web frameworks.

    Observability

    No production-ready system would be complete without observability, or the ability to gauge a system’s internal state based on external output, usually in the form of logs, traces and metrics. Hasura ships logs and traces to our New Relic telemetry platform for each database operation, allowing our team to identify bottlenecks and receive early alerts of unexpected or degraded system performance.

    Authorization

    Hasura’s role-based authentication support allows us to check an additional box on our production-readiness checklist. Clients with certain roles may have permission to execute specific GraphQL API operations or access particular database tables. Hasura can derive a client’s role in one of two ways: Either the client will send Hasura a JWT auth header (which includes a signature containing claims providing the client’s role), or Hasura will refer to an external webhook which, in turn, makes the auth decision. Without Hasura, we would need to write and maintain the code to perform all of this work.

    Change Data Capture

    The last way Hasura enables us to build our core platform is through change data capture (CDC), or the process of tracking changes to a database. Hasura provides this in the form of event triggers, which function like a transactional outbox system. When a record is inserted, updated or deleted, a bona fide database trigger will record this event in a table. Then, a reliable process in Hasura will guarantee that the event is delivered at least once to its intended webhook (one of our microservices). This is tremendously useful for our team since we need to sync data from our new database to the older legacy database.

    There are many tools we could use for CDC, such as AWS SQS or Kafka (some of which we use downstream to integrate our platform with individual products), but having this feature baked into Hasura makes the process simpler, as it frees us from having to provision any new infrastructure or deal with the complexities of bringing on additional tools.

    Conclusion

    For the Teaching Strategies platform modernization team, we’re just getting started. As we continue to evolve and improve our product, we’re considering Hasura for finer-grained authorization, such as attribute-based access control (ABAC) over coarser role-based access control (RBAC). We will also investigate real-time streams so other systems, such as users’ browsers, can see data changes as fast as possible.

    Using the functionality and features of Hasura community edition outlined above, our team is well on the way to building a strong core platform. By leveraging this tool, we can meet our milestones faster and provide even more support for educators and caregivers, making their lives easier.

    Interestingly, Hasura is a combination of the programing language Haskell and the Sanskrit word asura—or demon.  As far as tech choices go, this one feels much more like a guardian angel.

  • How to Keep Your IT Infrastructure Invisible

    How to Keep Your IT Infrastructure Invisible

    IT infrastructure is critical. IT infrastructure enables businesses to run—from day-to-day operational execution to visionary strategic innovation and everything in between. What’s interesting, though, is that people are only aware of it when there’s a problem: HR can’t access the payroll application, customers are complaining about how slow the transactional system is or this month’s cloud bill was double what you expected it to be.

    No one should notice a well-orchestrated environment, but once application performance slows or an outage occurs, people start to grumble. The same is true for IT infrastructure.

    The Top Four Ways Your IT Infrastructure Can Become Undesirably Visible

    Given the complexity of modern hybrid/multi-cloud environments and the applications they support, it’s no surprise that countless things can go wrong—and when they do, your IT team takes the flack.

    If your infrastructure is humming along nicely, that’s a testament to the IT infrastructure team’s hard work and expertise. Sadly, they don’t get the recognition because it’s all invisible to the rest of the organization. It’s only when *BAM!* something happens that the team is scrutinized.

    While there are many ways this can manifest, it usually falls under one of the following four categories:

    • Availability — This may be the most catastrophic because when systems are unavailable, people can’t do whatever it is they need to do.
    • Performance — A system might be available, but that doesn’t mean it’s performing to SLA or user expectations. The result can range from minor annoyance to lost revenue and customer churn.
    • Cost — Runaway expenditures don’t necessarily affect users, but you do not want to be called to account by the finance team or, worse, executive leadership.
    • Compliance — This is another area that may not have a broad impact on user experience, but the repercussions of non-compliance can be financially and reputationally significant.

    Common Infrastructure Culprits and How to Address Them

    To ensure the necessary availability, performance, cost control and compliance to keep your IT infrastructure under the radar, you must understand what can cause problems in these areas. Here are eight common culprits, along with ways you can address them, so your IT team stays invisible.

    Broken or Misunderstood Dependencies
    Your infrastructure is a complex web of interdependent applications and services. Every time you make a change, it could have a rippling effect on the categories mentioned above. Automated discovery and service mapping capabilities enable you to easily trace dependencies throughout your infrastructure to avoid unintended consequences.

    Cloud Waste
    Chances are very high that you’ve got cloud waste. This means that you could be getting billed for a storage block that’s attached to a stopped instance, for compute instances that are not being used, or for unattached load balancers. Finding idle and abandoned resources can keep your monthly cloud bill under control and on budget.

    Un-Breakdownable Cloud Costs
    Cloud bills are notoriously complex, and while cloud providers do offer tools to help you analyze your cloud spend, those tools often don’t answer all the questions your business might have. In addition to adding clarity to such opaque expenses, you need to be able to parse your cloud spend based on what makes business sense to your organization, not what makes technical sense to your cloud provider.

    Poor Capacity Planning
    Your infrastructure needs to meet your business needs, which are continuously evolving. As changes happen, whether it’s increased utilization of existing resources or modified/added resources to support new requirements, workloads could push current capacity and impact performance and stability. You need to be able to forecast infrastructure demands against business objectives to minimize potential issues before they happen.

    Poor Workload Placement Decisions
    Your infrastructure consists of various environments each with its own characteristics. If you migrate a workload without understanding the implications of that particular environment, you could risk performance problems or spiking costs. To prevent this, you need any-to-any migration insights to make better decisions about workload priorities, groups and deployments.

    Poor Reservation Planning
    Reservations (i.e., reserved instances or programmatic discounts) offer discounts for committing to a particular plan. If you plan well, the savings can be significant. But if you don’t use the capacity you’ve bought, the savings might not materialize. To improve cloud resource planning, you need to be able to evaluate your potential savings in varied scenarios.

    Workload Drift
    Over time, your workloads evolve due to changing demand, data volumes, etc. As a result, they can outgrow their current configurations, leading to performance issues or rising costs. This is a natural result of business growth, so the goal isn’t to prevent that drift. Instead, you want to optimize capacity and cost on an ongoing basis to keep your workloads right-sized.

    Slow Problem Troubleshooting and Remediation
    No matter how many precautions you take, something somewhere will inevitably go wrong. And once you become aware of the situation, the clock starts ticking to find the cause and fix it, all of which takes time. The longer the entire process takes, the longer your infrastructure will be visibly and painfully not working as expected. For those cases, you need full-stack visibility with application-centric, AI-based analytics to minimize impact duration as much as possible.

    Keeping the engine running behind the curtain can be a daunting task, but with proper planning and management, your IT team will remain visible to those responsible for managing it and invisible to those in the rest of the organization. Of course, the complexity of today’s cloud infrastructure makes that challenging, and there will be IT fire drills that take place, but it’s about how IT leaders react to the situation and resolve it in the quickest way possible, allowing themselves to return to their preferred place in the shadows.

  • Cloud Orchestration and its Impact on DevOps Teams

    Cloud Orchestration and its Impact on DevOps Teams

    Cloud orchestration refers to the coordination and management of various interconnected cloud services and resources to automate and streamline complex IT processes. In simple terms, it is the process of integrating, synchronizing, and automating various cloud-based components to create a seamless and unified workflow. 

    Cloud orchestration aims to optimize the deployment, management, and use of resources in a cloud environment, improving efficiency, reducing costs, and ensuring that cloud services are delivered in a timely and effective manner. It is related to the concept of CloudOps (cloud operations), which uses cloud orchestration concepts to allow more efficient management of cloud resources. Cloud orchestration is a critical aspect of modern IT infrastructure management, as it enables organizations to manage and control their cloud resources more effectively.

    Difference between Cloud Orchestration and Cloud Automation

    Though the terms cloud orchestration and cloud automation are often used interchangeably, they have distinct differences. 

    Cloud automation refers to the process of automating individual tasks and processes within the cloud environment. It involves the use of scripts, templates and other tools to automate repetitive tasks, such as provisioning resources, deploying applications and configuring environments. The primary goal of cloud automation is to reduce the time and effort needed to manage cloud resources.

    Cloud orchestration is a broader concept that encompasses not only the automation of tasks but also the coordination and management of the entire cloud ecosystem. It involves the integration of various cloud services, tools and resources to create a unified, streamlined workflow. Cloud orchestration ensures that all components in the cloud environment work together seamlessly, automating complex processes and managing dependencies between tasks.

    In essence, cloud automation is a subset of cloud orchestration. While cloud automation focuses on automating individual tasks, cloud orchestration takes a more holistic approach, orchestrating multiple tasks and resources to achieve a higher level of automation and efficiency.

    How Cloud Orchestration Enhances DevOps Practices

    Increased Efficiency and Speed

    One of the main benefits of cloud orchestration in a DevOps context is the ability to increase efficiency and speed in the software development and deployment process. By automating and streamlining various tasks, cloud orchestration eliminates manual processes, reduces the potential for human error and accelerates the overall development cycle. 

    Improved Collaboration and Communication

    Cloud orchestration fosters better collaboration and communication between development and operations teams by providing a unified platform for managing cloud resources. This shared platform enables teams to work together more effectively, breaking down traditional silos and ensuring that all stakeholders have visibility into the development and deployment process. 

    Enhanced Infrastructure and Application Management

    Cloud orchestration simplifies the management of complex cloud infrastructure by providing a single, unified platform for cloud optimization—improving the management of resources, applications and services in the cloud.

     This centralized management approach enables organizations to more easily monitor, control and optimize their cloud environment, ensuring that resources are used effectively and that applications are running smoothly. With cloud orchestration, organizations can more easily scale their infrastructure to meet changing demands, ensuring that they have the resources they need when they need them.

    Improved Monitoring and Troubleshooting

    Cloud orchestration provides improved monitoring and troubleshooting capabilities, enabling organizations to more effectively identify and address issues in their cloud environment. With comprehensive monitoring tools and dashboards, teams can gain insight into the performance of their applications, identify bottlenecks and pinpoint potential issues before they become critical.

    Challenges of Integrating Cloud Orchestration With DevOps

    Despite the numerous benefits of cloud orchestration, integrating it into DevOps practices can present several challenges. Some of the common challenges include:

    • Complexity: Cloud orchestration involves the integration of various cloud services, tools and resources, which can be complex and time-consuming. Organizations may struggle with complexity, particularly if they lack the necessary expertise or resources to effectively implement and manage the solution.
    • Resistance to change: Implementing cloud orchestration requires a change in mindset and approach to software development and deployment. As your organization migrates more systems to the cloud, some team members may be resistant to adopting new processes and tools, making it difficult to fully embrace and realize the benefits of cloud orchestration.
    • Security concerns: As with any cloud-based solution, security is a critical concern when integrating cloud orchestration into DevOps practices. Organizations must ensure that their solution is secure and that they have the necessary policies and procedures in place to protect their sensitive data and resources.
    • Cost and resource constraints: Implementing and managing a Cloud Orchestration solution can be resource-intensive, requiring significant investment in infrastructure, tools, and personnel. Organizations must carefully consider the costs and resources required to implement cloud orchestration and determine whether the benefits outweigh the investment.

    Best Practices to Integrate Cloud Orchestration Into DevOps

    To successfully integrate cloud orchestration into DevOps practices and overcome the challenges mentioned above, organizations should consider the following best practices:

    • Assess your organization’s readiness: Before implementation, evaluate your organization’s readiness for change. This includes assessing your existing infrastructure, processes and tools, as well as your team’s willingness and ability to adopt new practices and technologies.
    • Develop a clear strategy: Develop a clear strategy for integrating cloud orchestration into your DevOps practices. This should include defining your goals and objectives, identifying the tools and resources you will need and establishing a timeline for implementation.
    • Choose the right tools and technologies: Select the appropriate tools and technologies that best meet your organization’s needs and requirements. Consider factors such as ease of integration, scalability, security and cost when evaluating potential solutions.
    • Train and educate your team: Ensure that your team has the necessary knowledge and skills to effectively implement and manage cloud orchestration. This may involve providing training and education on new tools, processes and best practices, as well as fostering a culture of continuous learning and improvement.
    • Monitor and measure success: Continuously monitor and measure the success of your implementation. This includes tracking key performance indicators (KPIs), such as deployment speed, resource utilization and application performance, to ensure that your solution is delivering the desired results.

    Conclusion

    Cloud orchestration is an essential tool for organizations looking to streamline and optimize their IT infrastructure, particularly in the context of DevOps practices. By automating and coordinating complex cloud processes, organizations can improve efficiency, speed, collaboration and overall performance. 

    However, successfully integrating with DevOps can be challenging, requiring a clear strategy, the right tools and technologies and a commitment to continuous improvement. By following the best practices outlined in this guide, organizations can overcome these challenges and fully realize the benefits.

     

     

  • FinOps Gaining Ground, But Progress is Slow

    FinOps Gaining Ground, But Progress is Slow

    A CloudBolt Software survey found that while the majority (98%) of respondents worked for organizations with a formal FinOps team (82%) or are actively considering adding one (16%), realizing value from FinOps is slow. More than half of existing FinOps teams (54%) were formed just 12 months ago or less, the survey also found.

    The trouble is, only one in of 500 respondents claimed to achieve any material value from FinOps to date. More than three-quarters of respondents (76%) said they will have to wait two to three years to see real FinOps-driven results.

    The survey polled 500 executives, engineers and developers at companies with over 5,000 employees in the U.S., Canada, United Kingdom and Australia. It was conducted by Wakefield Research on behalf of CloudBolt Software, a provider of a platform for managing distributed computing environments.

    A total of 45% of respondents also questioned FinOps efficacy in the real world. Specifically, 15% flagged FinOps as being “theoretically easy but harder in practice,” with another 13% regarding it as “much ado about nothing.” Another 10% said it is “nothing more than a suggested framework,” while 7% labeled FinOps as “a necessary evil.”

    Nevertheless, a full 89% also described FinOps as the silver bullet for reining in the complexity of cloud cost management, with 68% noting that FinOps is among their organization’s most strategic priorities for 2023. The average size of a FinOps team is now 4.1 people, with the “director level” being the senior-most leader on the team 53% of the time, the survey noted.

    A total of 71% noted that achieving this year’s IT goals without a FinOps practice would pose a significant challenge, with 74% reporting that FinOps is now on par with ITOps, DevOps, SecOps and other IT disciplines.

    In addition, 71% of respondents said funding for FinOps initiatives increased in 2023, with 58% reporting key performance indicators (KPIs) and metrics to the C-suite or board of directors.

    CloudBolt Software CTO Kyle Campos said it’s clear that when it comes to reining in costs, large enterprises are struggling to effectively manage IT across multiple silos. In many cases, organizations were investing heavily in IT before and during the COVID-19 pandemic to drive digital business transformation initiatives without many cost controls being put in place, he added.

    Now that organizations are encountering economic headwinds, many chief financial officers are now asking IT leaders to be a lot more efficient, noted Campos.

    Developers, in particular, have for years been provisioning cloud infrastructure resources with little to no oversight. Unfortunately, developers are prone to leaving the proverbial lights on by, for example, leaving application development environments running over a weekend or simply forgetting that a virtual machine has been left to run for weeks—sometimes even months.

    Developers are also not especially good at alerting IT leaders of possible spikes in consumption due to, for example, a data-intensive workload being constructed. No finance team likes to be surprised by a monthly bill for a cloud service that is twice as much as the previous month.

    The one thing that is certain is that time is of the essence. FinOps, at its core, may not necessarily be a new idea, but IT organizations that fail to practice it may soon find themselves being held more accountable than any time in recent memory.

  • Survey Sees Value But Limited Understanding of Platform Engineering

    Survey Sees Value But Limited Understanding of Platform Engineering

    Platform engineering is emerging as method for managing DevOps at scale, but a survey suggested that many organizations are still not clear on the concept.

    A survey of 100 DevOps engineers, platform engineers, developers, site reliability engineers and IT managers conducted by Civo, a cloud service provider, found 85% of respondents work for organizations that have partially or fully embraced a DevOps culture, with nearly two-thirds (65%) identifying platform engineering as the next evolution of DevOps. Nearly, a third (30%) of respondents work for organizations that are already currently evaluating whether they should adopt platform engineering.

    Overall, the survey found 50% of the respondents at organizations that have adopted platform engineering reported improved standardization, with 43% also seeing faster product delivery.

    However, only 41% of the respondents in DevOps roles that were familiar with platform engineering said they had a moderate to high level of familiarity with the concept.

    Saiyam Pathak, director of technical evangelism at Civo, said the survey makes it clear that, in terms of understanding how platform engineering best practices should be implemented, there is need for more clarity. Many organizations initially adopted DevOps to spur innovation, so any approach to centralization shouldn’t come at the expense of developer productivity, he noted.

    Historically, DevOps adoption of DevOps workflows in most organizations has been driven from the bottom up. As a result, it’s fairly common for multiple application development teams within an organization to have adopted multiple DevOps platforms to manage workflows. Advocates for platform engineering are making a case for eliminating many of those redundant platforms in favor of, for example, a single continuous integration/continuous delivery (CI/CD) platform that serves as a corporate standard.

    At the core of those efforts is the assumption that centralization will reduce costs by eliminating redundancy. At a time when more organizations than ever are sensitive to the cost of IT, there is always going to be a natural tendency to rationalize platforms. Challenges arise, however, when centralization results in a loss of flexibility. Many of the teams adopt DevOps because they want to be able to add and replace tools as they see fit. If there is a corporate standard that narrows their tool options, those teams may start to look for ways to do an end-run around the corporate standard.

    The philosophical IT battle between advocates for centralization and proponents of decentralization has been around since the first PCs showed up in enterprise IT environments. As always, organizations need to strike a balance between both extremes, said Pathak.

    It’s not clear how much platform engineering might accelerate the pace at which applications are built and deployed, but the total cost of DevOps should decline as various tools and platforms are consolidated. The challenge, of course, is that centralizing DevOps workflows is likely to disrupt the application development workflows on platforms that may about to be rationalized.

    Regardless of what platforms become the corporate standard, the one thing that is certain is the cultural challenges associated with successfully making that transition will loom large within most organizations.

  • Quali Adds Governance and Cloud Cost Control Tools to Automation Platform

    Quali Adds Governance and Cloud Cost Control Tools to Automation Platform

    Quali today revealed it added governance and cloud cost management tools to its Torque platform for automating IT infrastructure management.

    David Williams, senior vice president for market strategy at Quali, said the added governance tools will make it possible to enforce security, compliance and consumption policies as IT environments are launched using the automation blueprints that Torque enables IT teams to create.

    At the same time, IT teams can now also define consumption policies based on cloud runtime or regions employed within a cloud computing environment. In addition to taking advantage of dashboards to visualize consumption of cloud resources, IT teams can also use Torque to identify cloud waste including detached, inactive and overprovisioned instances. They can pause, right-size or tear down environments to reduce costs and eliminate the possibility of a billing surprise at the end of the month, noted Williams.

    Torque also enables IT teams to detect drift and then update live cloud environments to remediate any that fall out of scope, he added.

    Quali has been making a case for an environment-as-a-service approach to managing IT. That approach is based on a set of triggers for controlling infrastructure via a software-as-a-service (SaaS) platform that can be invoked either via an application programming interface (API) or a graphical user interface (GUI).

    In addition to providing a set of infrastructure provisioning guardrails that prevent mistakes from being made, Torque makes it possible to automatically deploy complete application environments on top of infrastructure provisioned as code using blueprints rather than relying on low-level Terraform scripts. The goal is to increase developer productivity by enabling IT teams to create self-service portals that developers can use to spin up environments.

    That approach is becoming more appealing as many organizations begin to adopt platform engineering as a method to centralize the management of DevOps workflows, noted Williams. In fact, as part of that effort, many organizations are starting to unify the management of cloud and on-premises IT environments, he added.

    While many developers routinely provision IT infrastructure using tools such as Terraform, many of them lack the cybersecurity expertise required to ensure cloud computing environments are configured properly. Developers also tend not to remember to de-provision cloud resources when no longer needed; over time, those factors contribute to unnecessarily increased cloud costs.

    Ultimately, the goal is to find ways to centrally manage IT environments to reduce costs and improve cybersecurity without inhibiting the pace at which developers are building innovative applications, said Williams.

    At the same time, as application environments become more complex, it is becoming even more challenging to provision and manage IT environments at a time when many organizations are reluctant to hire additional IT staff, he noted.

    One way or another, IT environments will become more automated. The issue is determining how best to organize the IT teams managing those environments as tasks that once required a lot of manual effort are increasingly automated.

  • Pulumi Embeds Terraform Conversion Tool in Platform

    Pulumi Embeds Terraform Conversion Tool in Platform

    At its virtual PulumiUP event, Pulumi this week announced it embedded an ability to convert instances of Terraform into its infrastructure-as-code (IaC) platform via a simple command line interface (CLI) command. In addition, Pulumi added a Review Stacks capability that automatically creates a dedicated cloud environment to review code every time a pull request is opened.

    Pulumi is also making available a beta release of Pulumi Azure Native Provider 2.0, which decreases the size of the software development kit (SDK) by 60%. This is part of an effort to improve deployment times and reduce resource consumption by, for example, removing support for deprecated application programming interfaces (APIs).

    Finally, Pulumi has added support for property names and values to Resource Search for Pulumi Cloud, a search and analytics tool for cloud computing environments. DevOps teams can now additionally search for instances of, for example, versions of databases or specific availability zone as a result.

    Aaron Kao, vice president of marketing for Pulumi, said previously the company made available a separate tool for converting snippets of Terraform code. For the first time, Pulumi is also making available a converter plugin in the Pulumi engine which allows conversion tools from other IaC tools to be used.

    Review Stacks, meanwhile, makes it simpler for DevOps teams to pick up an unfamiliar codebase, make changes to both application and infrastructure code and share a live environment to review that code.

    Pulumi has been making a case for an alternative to Terraform that centralizes the management and provisioning of infrastructure using a set of embedded guardrails. One of the major issues enterprise IT organizations are encountering these days is the number of misconfigurations created when developers use tools like Terraform to provision infrastructure on their own. Those misconfigurations often lead to, for example, ports being left open and cybercriminals exfiltrating data.

    That approach is gaining traction among enterprise customers that are looking to centralize the management of DevOps processes, said Kao.

    In the wake of a series of high-profile breaches, more organizations are now reviewing the security of their software supply chains, including how infrastructure is provisioned. Many of those organizations will need to revisit those processes as pending legislation that requires more stringent requirements for building software eventually become law, noted Kao.

    At the same time, many organizations are also looking to improve the productivity of DevOps teams during an economic downturn that limits the number of full-time employees they might be able to hire. The pace at which software is being developed has not slowed, so there is a greater urgency to do more with less.

    It’s not clear how quickly organizations are centralizing the management of DevOps workflows, but with the rise of platform engineering, there is clearly more interest in defining consistent backend processes that multiple DevOps teams are currently managing individually. The challenge is finding a way to achieve that goal without impeding the pace of innovation enabled by embracing DevOps in the first place.

  • HashiCorp Releases Slew of Updates to Improve DevSecOps

    HashiCorp Releases Slew of Updates to Improve DevSecOps

    At a HashiDays event this week, HashiCorp launched updates to its Terraform implementation and a software-as-a-service offering, dubbed HCP Vault Secrets, that promise to simplify infrastructure security and ensure that secrets are not leaked.

    In addition, HashiCorp has added a self-hosted edition of its remote access security tool, Boundary, that can now also record SSH sessions. The company is also announcing general availability of Vault Secrets Operator, which provides a high level of abstraction for managing secrets in a Kubernetes environment and which synchronizes with the secrets management interfaces defined by the Kubernetes Technical Oversight Committee.

    Finally, HashiCorp is making available in beta a 1.16 update to the Consul service mesh for containers and virtual machines that adds a Sameness Groups capability to improve the resiliency of microservices by redirecting requests to an identical service. HashiCorp has also added a transparent proxy mode to enable failover between services along with a permissive mutual TLS (mTLS) that enables a service to be configured to accept both mTLS traffic from mesh-enabled services and non-Consul mTLS traffic from external downstream applications.

    Chris Van Wesep, senior director of product marketing for HashiCorp, said Version 1.5 of HashiCorp Terraform adds config-driven import workflow and check blocks capabilities that make it possible to declaratively use code to add existing resources into a Terraform state. Previously, the only way to update a Terraform state was to use an import command that could only be applied to one resource at a time. Terraform 1.5 also adds automatic code-generation capabilities for imported resources.

    In addition, version 1.5 of Terraform also includes a tool to validate that provisioned infrastructure is behaving as intended. That latter capability makes it possible to reduce the number of misconfigurations created by developers that might one day be exploited by cybercriminals to breach an IT environment.

    HCP Vault Secrets, meanwhile, provides a beta release of a SaaS edition of the existing Vault offering specifically designed to enable organizations to better manage application secrets that might inadvertently be exposed to cybercriminals in a production environment, said Van Wesep.

    HashiCorp is moving toward providing a portfolio of tools that enable DevOps teams to securely provision and manage hybrid cloud computing environments using a common workflow, he noted. A Forrester Consulting survey found nearly three-quarters (73%) of respondents are reporting this strategy is helping them reach their business goals, with another 19% adding they expected this strategy to help them achieve their business goals in the next year.

    It’s not clear whether organizations are fully embracing multi-cloud computing, but each additional cloud added to an enterprise IT environment increases the total cost of IT. The only way to mitigate those costs is to define a set of best practices for building and deploying applications that can be applied to any cloud computing environment.

    Unfortunately, many organizations will discover that, as they attempt to define those best practices, they have inadvertently become dependent on a set of proprietary application programming interfaces (APIs) that thwart those efforts.

  • Survey Finds Cloud Management Strategies Shifting

    Survey Finds Cloud Management Strategies Shifting

    Even though organizations continue to consume cloud resources inefficiently, they continue to increase the amount of money allocated. These were among the findings of a global survey of 1,000 technology practitioners and decision-makers conducted by Forrester Consulting on behalf of HashiCorp.

    The survey found a full 94% of respondents reported that their organization struggled with cloud waste. Top factors contributing to avoidable cloud spending included overprovisioning resources (50%), idle or underused resources (50%) and a lack of needed skills (43%). Nevertheless, more than half of respondents (56%) said they boosted their cloud spending in the last year compared to 22% which reduced it.

    At the same time, however, 92% of respondents said they are moving toward adopting, standardizing or scaling platform teams as part of an effort to centralize the management of cloud services, but only 39% are scaling the use of platform teams throughout the organization. The survey also found, however, that organizations are addressing skills shortages by adopting a common operating model/shared automated workflows (58%), reskilling/upskilling (56%) and working with partners (43%).

    The survey also found many organizations are also committed to multi-cloud computing, with nearly three-quarters (73%) reporting this strategy is helping them reach their business goals. Another 19% said they expected this strategy to help them achieve their business goals in the next year.

    The most-cited reasons for adopting a multi-cloud strategy were reliability (51%), cost reduction (48%), security and governance (45%), digital transformation (45%) and scalability (45%). Only 10% had no choice but to go multi-cloud due to acquisitions or other issues.

    Security (88%), availability (84%) and scalability (84%) were identified most often as being important/very important multi-cloud success factors, followed closely by regulatory/compliance requirements (82%), staffing and skill levels (82%), budget (80%), visibility into cloud infrastructure (78%), automated tooling (77%) and platform teams (75%).

    The top cloud initiatives were data protection (77%), followed by secrets management and access control (both at 75%), the survey finds. The top security concerns were password/credentials/secrets leakage (50%) followed by data theft (49%), phishing (46%) and ransomware (42%).

    Overall, respondents said the biggest benefits of having a multi-cloud computing strategy were a stronger security posture (74%) and helping attract, motivate and retain skilled staffers (68%).

    Chris Van Wesep, senior director of product marketing for HashiCorp, said it’s clear organizations are purposely using multiple clouds to reduce costs by pitting one cloud service provider against another. The challenge organizations face when using that approach is they need to define a consistent set of development processes that can be used across multiple clouds to keep total costs under control.

    Organizations that embrace multi-cloud computing are also more likely to implement security best practices because of their need to consistently manage multiple cloud computing environments, he added.

    Of course, there is always going to be a great disparity between high- and low-performing organizations. The survey found that about 25% of survey respondents are high performing, with an equal number falling at the other end of the spectrum. The rest fall somewhere in between those two ends of the spectrum. The issue is that the gulf between the high and low ends of that spectrum appears to be widening as more organizations fail to muster the level of skills and expertise needed to succeed in the cloud era.

  • VMware Streamlines IT Management via Cloud Foundation Update

    VMware Streamlines IT Management via Cloud Foundation Update

    VMware this week updated its Cloud Foundation platform and rolled up recent updates to the company’s virtualization portfolio all at once.

    Version 5.0 of VMware Cloud Foundation (VCF) encompasses the latest versions of VMware vSphere 8.0u1 for workload management, VMware vSAN 8.0u1 for scale-out storage, VMware NSX 4.1 for networking and VMware vRealize LifeCycle Manager 8.10 (Aria) for cloud management.

    In addition, VMware has added a single sign-on (SSO) capability, dubbed Isolated SSO Workload Domains, that enables administrators to configure new workload domains using SSO instances from different providers or platforms via NSX.

    Rick Walsworth, director of product marketing for the Cloud Infrastructure Business Group at VMware, said that as the management of IT continues to evolve in VMware environments, more organizations are opting to update their entire VMware software stack simultaneously rather than attempting to update components individually.

    The primary reason for that shift is that the company ensures that all the latest VCF component releases have been thoroughly stress-tested from an integration perspective, he noted.

    During uncertain economic times, VMware is seeing an increased focus on simplifying IT environments as IT teams are asked to do more with less, said Walsworth. As such, more organizations are looking to standardize on a stack of infrastructure software that can be managed more consistently, he added.

    More organizations are also looking to reduce the total cost of IT by employing the same stack of infrastructure software across hybrid cloud computing environments where workloads are deployed both in on-premises IT environments and public cloud, said Walsworth.

    The company is also seeing increased adoption of DevOps best practices to programmatically manage IT environments that need to respond to rapidly changing requirements, he noted. IT teams are looking to work more collaboratively rather than relying solely on specialists to manage specific functions, said Walsworth.

    Some of that shift is also being driven by Tanzu, a distribution of Kubernetes that the company makes available as part of VCF, he noted. Most application developers don’t have a preference for a specific Kubernetes distribution, so many organizations view Tanzu as an opportunity to extend their existing management framework to a de facto standard for building the next generation of cloud-native applications, said Walsworth.

    VMware is in the midst of being acquired by Broadcom. Nevertheless, VMware this week reported revenues for the first quarter 2023 of $3.28 billion, a 6% year-over-year increase. Subscription and software-as-a-service (SaaS) revenue for the first quarter was $1.22 billion, an increase of 35% year-over-year. That increase in revenue suggests any concerns about the future direction of the company are not impacting the willingness of IT organizations to continue to rely on VMware.

    It may be a while before all the dust settles around Broadcom’s acquisition of VMware. In the meantime, however, it’s clear that the way IT is managed in the cloud era is continuing to rapidly evolve regardless of who owns VMware.