devopsy.kinsta.cloud

Tag: devops

How to Choose an Observability Data Pipeline

A complete full-stack observability platform combines data from multiple sources to gain holistic insights into system behavior. In fact, organizations without a full observability solution could find themselves facing departmental silos since teams with different data sources cannot share insights.

While a key step in observability is the ability to combine logging data with metrics, companies that have developers viewing logging data and their metrics elsewhere are not collaborating on their data in the way observability requires. The solution is an observability data pipeline that best fits your business needs.

What is an Observability Data Pipeline?

An observability data pipeline does two things—centralizes data from a number of different sources and provides teams with the right tools to turn data into business insights. Observability data pipelines are typically user-friendly so that you can easily set up and manage your data.

Observability data pipelines integrate data from a range of sources, including AWS, FluentD and Kubernetes. They have capabilities for searching and viewing log data and notifying engineers through email and even Slack.

An observability pipeline ingests logs, so they can be viewed in a log viewer. Pipelines can also analyze data in real-time, allowing engineers to quickly spot critical security issues before they affect critical systems.

There are a number of full-stack observability platforms, including Coralogix, Splunk, Elastic and even options like Azure Log Analytics and Google Cloud Logging. Coralogix, for example, is a popular observability data pipeline that can analyze data in real-time using Streama and use Kafka Connect to integrate data from a variety of sources and extract trends.

Business Benefits of Observability Data Pipelines

Using observability data pipelines has the potential to transform the way a business does DevOps. Let’s go through a few key benefits of this technology:

● Reducing Cost of Observability Data
Many organizations use systems like Cloudwatch to analyze logging data. However, these often charge per GB of logging data analyzed, raising the question of whether Cloudwatch is really cost-efficient. And it’s not just Cloudwatch. For many systems, logging costs can be increased due to slow query times, noisy logs and the wrong type of storage. Observability data pipelines help organizations manage high volumes of logging data and move away from expensive legacy platforms. This enables enterprises to increase their observability for a modest price.
● Reduced Onboarding, Increased Problem-Solving
Pipelines are straightforward to learn. Unlike options such as ELK, which can be quite specialized and difficult to maintain, observability data pipelines are easy to use and easy to set up. This reduces the time needed to onboard new engineers. Less time onboarding means more time solving problems and serving clients, which increases business revenue.
● Better Data Shaping and Merging
Data from different sources is often stored in multiple formats and structures. Because observability depends so much on well-integrated data, it’s vital to transform this data into standardized formats. Observability data pipelines use log parsing to store key features of logging data for search and analysis.
● Reducing MTTD/MTTR
Having an optimized, user-friendly observability data pipeline reduces downtime by helping engineers solve problems faster.
● Enabling DevSecOps With Observability
We’ve written about the rising prominence of DevSecOps, a paradigm that involves shifting security left in the development life cycle. Key to DevSecOps is the fruitful collaboration between DevOps and Security. This can only be achieved by high levels of observability. Observability data pipelines facilitate this with their ability to integrate data from multiple different sources and present it intuitively. This means security teams can apply their expertise to log analytics that would otherwise have remained siloed.

Wrapping Up

Observability, the ability to use logs, traces and metrics to “picture” the internal state of a system, is vital to any DevOps team. Observability data pipelines can help by integrating and presenting data from different sources. This mitigates the effects of organizational silos and allows DevOps teams to solve problems faster.

May 24, 2023
Five Great DevOps Job Opportunities

DevOps.com is now providing a weekly DevOps jobs report through which opportunities for DevOps professionals will be highlighted to better serve our audience.

Our goal in these challenging economic times is to make it easier for DevOps professionals to advance their careers.

Of course, the pool of available DevOps talent is still relatively constrained, so when one DevOps professional takes on a new role it tends to create an opportunity for others.

The five job postings shared this week are selected based on the company looking to hire, the vertical industry segment and, naturally, the pay scale being offered.

We’re also committed to providing additional insights into the state of the DevOps job market. In the meantime, for your consideration:

LinkedIn

Samsung Electronics America
Mountain View, California
Lead Site Reliability Engineer (SRE) – DevOps Specialist
$180,000 to $210,000

Indeed.com

Federal Reserve Bank of New York
New York, New York
DevOps Engineer
$178,100 to $207,900

CareerBuilder.com

SAIC
Chantilly, Virginia
DevOps Advisor
$130,000 to $200,000

Dice.com

Saicon Consultants
Alpharetta, Georgia
DevOps Engineer
$110,000 to $140,000

Simply Hired.com

Hewlett-Packard Enterprise
Fort Collins, Colorado
DevOps Automation Engineer
$109,400 – $251,500

May 22, 2023
How Test-Driven Methodologies Reduce CI/CD Lead Time
Modern development environments, in which rapid continuous delivery is facilitated by automated continuous integration/continuous development (CI/CD) pipelines, require thorough and automated testing in development prior to integration. This article explains how test-driven development (TDD) and acceptance test-driven development (ATDD) methodologies reduce CI/CD lead times.

Lack of test automation in development impacts CI/CD lead time for subsequent integration, delivery and deployment stages of a CI/CD pipeline in the following ways:
- Increased defect rate: Without automated testing during the development stage, more defects and issues are likely to be introduced into the codebase, increasing the time spent on debugging and fixing issues in later stages.
- Slower feedback loop: Without test automation, the feedback loop between code changes and test results is slower, making it more challenging to identify and resolve issues quickly. This delay can impact the overall lead time in the CI/CD pipeline.
- Reduced confidence in releases: When testing is not performed adequately or isn’t automated, there is less confidence in the quality of the software being released. This may lead to more manual testing and validation, further increasing lead times.
- Inefficient use of resources: Without test automation, developers and testers need to spend more time on manual testing, reducing the time available for developing new features or fixing existing issues.
- Increased merge conflicts: When test automation is not performed early in the development stage, the risk of merge conflicts increases as different developers work on the same codebase. Resolving merge conflicts can be time-consuming and may cause delays in the integration, delivery, and deployment stages.
- Decreased stability and reliability: Inadequate test automation during the development stage can lead to less stable and reliable software. This may require additional time and effort to address issues and ensure that the software is production-ready.
- Inconsistent test environments: Manual testing may not always be conducted in a consistent environment, leading to potential discrepancies between development, staging, and production environments. This can cause unexpected issues and delays in the pipeline.
- Hindered collaboration: Lack of test automation may reduce visibility into the quality of the codebase, making it more difficult for team members from development, QA, security and operations to collaborate effectively and identify issues early.
Although CI/CD pipelines help catch issues and ensure code quality through automated builds and tests, having a strong foundation of test automation during the development stage helps minimize the potential for issues in the pipeline and ensures a smoother CI/CD process.

There are several reasons why many development teams do not automate tests prior to integration, despite the benefits of doing so.
- Lack of awareness or understanding: Some development teams may not be aware of the benefits of test automation or may not fully understand how to implement it effectively in their development process.
- Insufficient skills or experience: Test automation often requires specialized skills and knowledge, and some development teams may lack the necessary expertise to implement it effectively.
- Time constraints: Developing and maintaining automated tests can be time-consuming. Teams working under tight deadlines might prioritize feature development over creating automated tests, believing that it will save time in the short term.
- Limited resources: Test automation tools and infrastructure can require significant investments in terms of cost and resources. Some organizations may not have the necessary budget or resources to invest in test automation, especially for smaller projects or teams.
- Resistance to change: Some teams may be resistant to adopting new practices, particularly if they have been successful with their existing processes. Changing established workflows and learning new tools can be challenging and may encounter resistance from team members.
- Incomplete or rapidly changing requirements: In situations where requirements are unclear or frequently changing, development teams may struggle to create meaningful automated tests, as they may require continuous updates to stay relevant.
- Overemphasis on manual testing: Some teams may place a greater emphasis on manual testing due to a belief that it provides more accurate or thorough results. This perception can lead to a reluctance to invest time and effort into developing automated tests.
- Fear of increased maintenance: Automated tests, like any other code, need maintenance and updates as the application evolves. Some teams may be concerned about the additional effort required to maintain a test suite, particularly if they have limited resources.
The amount of test automation that should be done in development prior to the integration stage of a CI/CD pipeline depends on several factors, including the complexity of the application, team resources and the specific project requirements. However, here are some general guidelines to help you determine the right balance:
- Unit tests: Developers should aim to automate most, if not all, unit tests during the development stage. They are typically fast to execute and provide quick feedback, allowing developers to catch issues early and minimize the number of defects that make it to the integration stage.
- Component and integration tests: While unit tests focus on individual components, component and integration tests verify the interaction between different components or services. These tests should be automated as much as possible to ensure that any issues arising from component interactions are identified and resolved quickly.
- Static code analysis: Automating static code analysis checks during development helps identify code quality issues, security vulnerabilities, and potential performance bottlenecks early in the process.
- Code coverage: Strive for a high level of code coverage with your automated tests. However, it’s important to remember that code coverage is just one indicator of test quality and should be complemented by other testing strategies.
Striking the right balance between automated and manual testing is important. Automate tests that are repetitive, time-consuming or prone to human error. Manual testing should be reserved for exploratory testing, usability testing and other areas where human intuition and judgment are necessary.

Test-driven development (TDD) and acceptance test-driven development (ATDD) are development methodologies that emphasize creating automated tests before writing the actual code, helping teams ensure code quality and catch issues early prior to integration.

Test-driven development (TDD) is a software development methodology that emphasizes writing tests before writing the actual code. The primary goal of TDD is to create clean, maintainable and bug-free code by catching issues early in the development process. TDD follows a short iterative cycle consisting of three main steps: Write a test, make the test pass and refactor the code. The developer starts by writing a test for a specific feature or function, which initially fails. Next, the developer writes the minimal amount of code necessary to make the test pass. Finally, the developer refactors the code to improve its structure and readability while ensuring that all tests still pass.

Acceptance test-driven development (ATDD) is an extension of TDD that focuses on defining and validating the high-level requirements and acceptance criteria for a feature or functionality before development begins. ATDD involves collaboration among developers, testers, and business stakeholders to ensure that the application meets the needs of its users. The process begins with writing acceptance tests, which are derived from the user’s perspective and based on the acceptance criteria. These tests are written in a human-readable language to facilitate clear communication among all team members.

Once the acceptance tests are defined, the development team follows the TDD process to implement the feature, using the acceptance tests as a guide. As the team develops the code, they also create unit tests and other lower-level tests to validate the implementation. The acceptance tests are executed as part of the development process and serve as a measure of progress. When all acceptance tests pass, it indicates that the feature or functionality meets the defined acceptance criteria, and the development task is considered complete. By using ATDD, teams can improve collaboration, ensure that the application meets user expectations and reduce the risk of miscommunication or misunderstanding requirements.

Hurdles to overcome when transitioning to TDD and ATDD include:
- Mindset shift: Adopting TDD and ATDD requires a change in mindset, where testing is considered an integral part of the development process rather than an afterthought.
- Skill development: TDD and ATDD require specialized skills in writing effective tests and implementing test automation. Development teams may need to invest time and resources in acquiring these skills.
- Time and resource allocation: Transitioning to TDD and ATDD might initially lead to a perceived increase in development time, as writing tests before code can seem slower. However, the long-term benefits of improved code quality and reduced debugging time should be considered.
- Collaboration and communication: ATDD relies on effective collaboration and communication among developers, testers, security, operations and business stakeholders. Establishing a culture of open communication and collaboration is crucial.
- Tooling and infrastructure: Implementing TDD and ATDD may require investments in new tools or changes to the existing development infrastructure to support test automation and continuous integration.
In the context of TDD and ATDD, the roles of quality assurance (QA), security and operations teams are crucial for ensuring a well-rounded and robust software development process. Their involvement helps to address various aspects of the application, leading to a more reliable and secure product.
- Quality Assurance (QA): QA team members collaborate closely with developers in both TDD and ATDD methodologies. They can help define test cases, create test plans and provide input on acceptance criteria. In ATDD, they play a particularly significant role in developing and validating acceptance tests, ensuring that the application meets the defined requirements and user expectations. QA team members may also be involved in setting up and maintaining test automation infrastructure, analyzing test results and identifying areas for improvement.
- Security: Security professionals contribute to TDD and ATDD by identifying potential security vulnerabilities and ensuring that the application is developed with security best practices in mind. They can help define security-related test cases and help to embed security into the application, reducing the risk of vulnerabilities and improving the overall security posture.
- Operations: Operations teams are responsible for managing the infrastructure and deployment processes that support the application. In TDD and ATDD, they can collaborate with developers to ensure that the application is designed and developed with operational concerns in mind, such as scalability, performance and maintainability. Their involvement also helps to promote a DevOps culture where development and operations teams work together to ensure the application’s success in production environments.
A practical roadmap for teams to transition to TDD and ATDD while overcoming the hurdles mentioned earlier could look like this:

Build awareness and buy-in:
Provide training and skill development:
Start small and iterate:
Foster a culture of collaboration:
Set up the necessary tooling and infrastructure:
Monitor progress with metrics:
Continuously improve and adapt:
Summary

TDD and ATDD methodologies contribute to reducing CI/CD lead times by automating tests during development. This improves code quality, minimizes defects and promotes collaboration among team members from the outset of the development process. By writing tests before the actual code and focusing on meeting acceptance criteria, teams can catch and resolve issues early, leading to fewer delays and reduced time spent on debugging and fixing errors during the integration, delivery and deployment stages. These practices also help ensure that the application aligns with user expectations and operational requirements, resulting in a smoother and more efficient CI/CD pipeline.

Anyone who is passionate about modern quality engineering processes for building high-quality software to participate is encouraged to join new ShiftSync Community from Tricentis to engage in discussions about this topic and other development methodologies.
May 19, 2023
Infrastructure-as-Code Pitfalls in Platform Engineering
In the context of cloud operations, the last decade was ruled by DevOps and infrastructure-as-code (IaC). But what is true DevOps? Is it developers running their own operations? Is it operators learning development skills to automate infrastructure provisioning using infrastructure-as-code (IaC) tools like Terraform?

Though DevOps is widely understood to incorporate both of these, software developers are far from fully understanding or operating infrastructure. IaC is purely a DevOps tool. When an organization claims to have automated its infrastructure, they mean automation for the efficiency of the DevOps teams, not automation for developers. Developers are still left waiting on DevOps for days for even small infrastructure changes.

While there were efficiencies gained by the current DevOps model, there is an increasing acknowledgment that we require a better approach to enable developer self-service. This has led to the platform engineering discipline.

The high-level goals of platform engineering are:
- Developer self-service for significant parts of infrastructure updates without DevOps subject matter expertise.
- Built-in security and compliance controls.
Infrastructure-as-Code (IaC) Limitations in Platform Engineering

A DuploCloud survey of 40+ enterprises that have made substantial investments in platform engineering showed that the prevalent approach is for DevOps teams to build a DevOps platform with IaC as a core underlying technology. They create templates for the organization’s use cases and publish them in a CI/CD pipeline or a self-service catalog.

Here are the top reasons why platforms that build on top of IaC are failing these platform engineering goals:
1. IaC templates are rigid and not flexible enough to meet changing developer requirements: It is true that DevOps teams can anticipate cloud infrastructure topologies to some extent and have IaC templates for those with a few customizable parameters. But in a microservices world, there are thousands of other workflows and topologies possible based on changing application needs and security controls. Large data centers and edge infrastructure make this even more complex. A manual approach, relying solely on DevOps personnel to constantly build and update myriad combinations in static scripts simply can’t scale.
2. Scripting tools can’t build life cycle management: In cloud operations, people-triggered changes are only a subset of possible use cases. Many asynchronous operations need to be continuously performed. These range from detection of configurations, from desired state to reverting, or complex configuration scenarios where individual components have to be set up asynchronously and brought together later. It could be as simple as a certain component going down and needing to be restored. IaC is a script that runs when triggered to completion. It has no active life cycle to operate continuously in the background.
3. Inability to build a concept of an environment: In any orchestration system, users have a concept of the environment they want to build. When they log in to the platform and navigate to their environment, they expect to update as well as view the state and aspects of resources in that environment, be it the provisioning status, metrics, logs, faults, audit logs, compliance posture, etc. They may want to perform debugging functions, such as restarting services, SSHinginto a VM or accessing a resource’s cloud console (S3 Console, for example) using access control boundaries within that environment. For example, in Kubernetes, you choose the namespace to be the environment and management software like Rancher on top of K8S to provide these functions. If we must replicate this same concept across a broad infrastructure-level platform, we cannot do it using only IaC. We need something with a scope that spans all cloud operations, like Kubernetes. Terraform is a configuration updating and management software, not an orchestration system.
Many platform teams have tried to work around these problems with point solutions by using a disparate set of jobs for certain aspects of life cycle management, building a thin UI shim on top of cloud accounts for visualization of resources while redirecting to other systems like Datadog for logging, metrics and alerts. But for most of the use cases, the DevOps team is still very much in the operations workflow. This completely defeats the concept of developer self-service as well as continuous compliance goals.

Learning From Other Successful Platforms

Two recent examples of successful cloud platforms are Amazon Web Services (AWS) in the context of infrastructure-as-a-service (IaaS) and Kubernetes for container orchestration. These are distributed system implementations using higher-level programming languages like Java and Go. You can’t build such complex systems using scripts and jobs.

Building a true DevOps orchestration platform requires a systems design approach. It also takes expert systems engineers and many years to build and mature, like it did for K8s and IaaS in the Public Cloud.

Conclusion

Since platform engineering teams need systems design expertise that is likely outside of the scope of the typical DevOps role, we need the software development function to intervene with distributed systems programmers and many years of investment. Most organizations trying to build in-house platforms don’t have or don’t want to make this type of investment, as it may not be their core business. Megascale tech companies like Facebook, Uber, Netflix, etc. will likely build an in-house solution, as they have the talent and scale, but for the masses, it is likely that the solution will be an off-the-shelf product from an ISV.

The industry has solved IaaS with public cloud, container orchestration with Kubernetes and observability with platforms like Datadog. The platform engineering category is heading in the same direction.
May 18, 2023
What’s the Difference Between DevOps and Platform Engineering?
The DevOps space is buzzing with a new trend—platform engineering. Platform engineering has been gaining steam over the last few years but has really peaked in interest lately. In fact, according to a study conducted by Puppet Labs, 51% of organizations have adopted platform engineering or plan to do so in the next year. But what exactly is platform engineering, and how is it different than DevOps?

Below, we’ll explore the nuances between these two concepts. We’ll review what DevOps and platform engineering are and see how they’re similar and dissimilar. We’ll also outline the benefits of platform engineering and suggest some best practices to make the most of this emerging discipline.

What is DevOps?

DevOps is a broad topic, but it generally involves efforts to increase software release efficiency and frequency. Some of the hallmarks of the DevOps discipline include reducing siloes between teams and increasing collaboration. DevOps is often directly associated with various tools that automate the process of moving code from a local environment into production, also known as continuous integration and continuous delivery (CI/CD). Monitoring, incident management and configuration management tools also support the DevOps process.

“DevOps reflects more of a philosophy and a way of working,” said David Sandilands, senior solutions architect for Puppet. DevOps enables more of a shift left mentality. Instead of having operations teams “guarding the gates,” developers work in partnership with Ops. In doing so, groups can decrease build time significantly, he said. This style helps engineers incrementally and rapidly release new features, replacing old waterfall development cycles.

The ultimate goal of DevOps is to automate the software life cycle and shorten the feedback loop. Thus, DevOps takes a life cycle approach to shipping applications more efficiently, streamlining phases such as build, release, configure, monitor, plan, create and verify. Microservices architecture and containerization also often play a crucial role in enabling DevOps.

What is Platform Engineering?

Platform engineering is all about developing a self-service internal platform that serves the needs of developer consumers. Although DevOps opened things up and empowered individual developers with advanced automation, it left organizations with scattered tooling and processes, especially at large enterprises. For example, various infrastructure-as-code (IaC) configurations might exist for similar functions within the same organization but be unknown to engineers in different departments. Not to mention, overseeing a growing DevOps pipeline means added responsibilities around continual updates and security.

Platform engineering places more control in the hands of a central platform team rather than individual developers. These teams typically treat the platform as a product. They do so by evaluating the needs of developers across an organization, building out internal self-service tools that suit the needs of most consumers and maintaining the platform.

The 2023 State of DevOps Report found that this self-service approach typically handles areas like deployment, operation of infrastructure, monitoring production applications and security and compliance. Other capabilities, such as abstracting Kubernetes or overseeing secrets management, may also play a role in these efforts.

Benefits of Platform Engineering
- Improve developer experience: Platform engineering can enhance the experience a developer has with their software build and release systems. A better developer experience can do loads to improve morale and create a more productive engineering atmosphere.
- Standardize DevOps practices: By bringing together common DevOps approaches into a shared platform, organizations can begin to standardize with reusable build processes and automated infrastructure. This can unite an organization on common runtimes, observability methods and deployment processes. This can streamline the introduction of DevOps practices to an organization.
- Secure the DevOps pipeline: Various vulnerabilities exist within tooling that comprises modern CI/CD chains and cloud-native open source software. By outsourcing DevOps responsibility to a shared platform team, the organization can coalesce around more secure practices and avoid overlooking important updates or holes.
- Supply additional guardrails: Platform teams might also assist in the acquisition of new tools as well as outlining and enforcing policies for software infrastructure. While this may come at the cost of customization, additional guardrails can help avoid shadow IT and governance around internal development. Letting platform teams handle it could make maintenance easier across an organization.
Platform Engineering: Under the DevOps Umbrella

I don’t see platform engineering as replacing DevOps. Instead, I view platform engineering as falling under the umbrella of DevOps since platform engineering takes DevOps processes and makes them more reusable and self-service. Both use a lot of automation and share many of the same end goals.

However, there are nuances between the two approaches that reflect the recent trends in the IT market. With DevOps, the market was more interested in shifting responsibility left and empowering individual developers to oversee the entire software life cycle. Now, as sprawl and shadow IT emerge, the pendulum seems to be shifting back toward a bit more centralization and governance with platform engineering.

Platform engineering is typically more of an in-house development initiative. So, what are some best practices IT divisions should remember to get the most out of platform engineering? Hope Lynch, senior director of platform at Cloudbees, shared some great strategies to consider that revolve around treating the platform as a product. Doing this involves accepting feedback from developers, not controlling everything and always maintaining a focus on developer experience.

Hopefully, by following these tips, organizations should have a better grasp of the emerging discipline of platform engineering and have the confidence to begin introducing platform-based concepts within their own environments.
May 18, 2023
DevOps Security: Your Complete Checklist

In a fast-paced environment like DevOps, your security strategy needs to be even more agile. While moving fast is great for delivering application features and functionality, speed shouldn’t come at the cost of security.

DevOps security incorporates security strategies directly into your DevOps pipeline, creating a culture of pre-secured software deployments that work seamlessly for the end user. However, this requires a complete shift in your operational policies and security mindset. You need to start viewing security as an integral part of your DevOps lifecycle instead of a separate department.

Here’s a checklist of DevOps security strategies for your development teams to deliver secure and reliable applications faster:

Your Complete DevOps Security Checklist

1. Analyze your development process for security hazards.
The first step in implementing a DevOps security strategy is finding and fixing loopholes in your existing development process.

For instance, continuous delivery and development are key in a DevOps environment. However, if each change doesn’t go through a code review, it has the potential to break your application.

Deploying changes without passing them through a security checkpoint is like driving at high speed on a road with many curves. Each curve (change) is a potential hazard, and the danger increases when the driver does not know a car is coming from the other side (bug).

Therefore, you must turn the spotlight on your development process and find those curves.

2. Create a common goal between development and security teams.
Traditional roles of developers and security teams might look like they are at crossroads — Developers write code, and security teams find fault with their code. However, in a collaborative environment such as DevOps, security and development must work hand-in-hand to deliver a secure and seamless experience to end users.

Your organization should have a common goal of integrating security directly into the development process. Therefore, encourage open and regular communication between development and security teams. Further, establish joint milestones and review them to ensure there are no silos between both teams.

Once these teams get used to this collaborative approach, your deployments will become faster because fewer things will break.

3. Integrate security procedures in the development phase.
Automation is the lifeblood of DevOps security. Integrating scanning tools into development workflows is critical to your entire process.

For example, automated tools can scan the code, network and infrastructure to identify security vulnerabilities and provide recommendations for remediation. With this, development teams can fix a few security concerns themselves before passing them on to the security team for a final check.

4. Add observability to your pipeline.
One of the biggest challenges of the distributed nature of DevOps operations is visibility. How do you efficiently monitor a large number of operation nodes spread across several different locations?

Observability helps you do this by using logs, metrics and traces—i.e., the outputs of your application to determine internal application health. With observability tools, you can see much deeper into the operations of your systems and identify the exact details of any security concerns. You can also easily identify potential bugs and fix them before they cause problems.

5. Make changes in small, incremental units.
It is infinitely easier to test code in small units and deploy them as minor but incremental changes than handling large, unwieldy blocks of monolithic code. Deploying gradual changes means you can resolve vulnerabilities faster.

6. Scan and secure your entire DevOps pipeline.
A large part of the security hazards in your DevOps process comes from unsecured containers and operations nodes. According to a study from ThreatStack, 94% of organizations say containers represent a security risk.

However, the risk is not limited to containers.

Other services like APIs, third-party tools like Docker and imported code pose a potential security risk to your processes. This makes it necessary to scan all your operations nodes periodically for vulnerabilities. And once you’ve cleared them, remember to lock the door behind you by requiring strict credentials for access to your administrative dashboards.

Ticking Items Off Your DevOps Security Checklist

If you implement most of the security guidelines on this checklist, your organization will have a solid security foundation. However, DevOps security is not a one-time process.

You will need to continuously integrate security protocols into each part of your DevOps life cycle. Furthermore, you will need to re-train your employees and invest in new technology that makes your security pipeline smoother, such as observability.

However, since this entire process makes your applications more secure and compliant while shipping them to market faster, investing in DevOps security is well worth it!

May 17, 2023
Boomi Brings Generative AI to Integration Platform

Boomi announced today it is applying generative artificial intelligence (AI) to enable organizations to automate integrations via a natural language interface.

Boomi CEO Steve Lucas said Boomi AI makes it much simpler for professional and citizen developers alike to create integrations at scale using generative AI platforms to invoke the low-code engine that Boomi already provides to create integrations between thousands of applications.

Those generative AI capabilities will complement a wide range of AI capabilities based on machine learning algorithms that are already embedded in the Boomi platform, he added.

Generative AI’s Impact on Integration and Automation – Steve Lucas, Boomi from Techstrong.TV on Vimeo.

Boomi has been widely used to drive business process integration via an integration platform-as-a-service (iPaaS) environment it delivers as a managed service. The low-code platform the company created significantly reduces the need to rely on professional developers to write procedural code using Java, for example, to create those integrations.

Generative AI capabilities make it possible for almost anyone of any skill level to create those integrations across multiple types of application programming interfaces (APIs), noted Lucas.

That capability also makes it possible for Boomi to notify end users when, for example, a data attribute in an application has changed and ask them if they would like to apply that change across their entire application environment.

In addition, Boomi AI can also interpret the intent of a prompt to better orchestrate processes in addition to providing an audit history in natural language that explains how decisions were made and what outcomes occurred as a result. As part of that effort, Boomi algorithms are also trained to avoid biases and unfairness with established ethical guidelines.

Boomi’s AI engine is trained using anonymized data collected from approximately 20,000 customers that have created more than 200 million integrations using the Boomi platform. Those integrations have been exposed to third-party large language models (LLMs) that the company is using to train Boomi AI, said Lucas. Boomi is not storing any customer data to train its AI models.
DevOps teams, of course, spend hours creating integrations that should be much simpler to create, test and validate with the rise of generative AI. In fact, the number of integrations being created to drive a wide range of digital business transformation initiatives should substantially increase.

In the meantime, DevOps teams would be well-advised to start evaluating which tasks they manually perform today that could instead be automated by a generative AI platform. Many of the routine integrations that DevOps teams create today using low-level APIs will be increasingly automated by end users. The ultimate goal should be to identify more complex tasks that previously DevOps teams would not have attempted simply because there wasn’t enough time.

It’s still too early to determine what impact generative AI will have on the way IT is managed, but DevOps teams that are typically committed to automating as many processes as possible should be at the forefront of adoption. The challenge is going to be simply keeping track of the rate of change that might soon be enabled by these platforms.

May 16, 2023
Four Trends Defining the Observability Market in 2023

Observability is a market that has grown exponentially over the past few years. It started as a buzzword in late 2015 and early 2016, and since then we’ve seen dozens of insurgent companies rush in, eager to stake their claim in a new (albeit loosely defined) market. Incumbent companies also rebranded their wares, hoping to capture some of the momentum that comes from the new term.

Venture capital saw the market opportunity, too. By my conservative estimate, observability companies raised nearly $2 billion over roughly the last two years alone (I’m including some cybersecurity companies, which I’ll explain later).

In the seven or so years since its inception, the observability market has fragmented and specialized into categories such as application, data, network and edge observability. On the artificial intelligence and machine learning side, there is model observability and AIOps. Connecting it all are observability pipelines.

While more businesses than ever know about observability and have built it into their tech strategy, category awareness still has a long way to go. And though the dollars were flying fast and furious a couple of years ago, the economic cycle has turned against risk, and growth funding rounds are harder to come by. It’s time to step back and examine the key trends defining today’s observability market.

Staffing Shortages: A Blessing and Curse to Observability

Many companies in the observability space, regardless of their subcategory, represent net-new work for IT and security teams. A new application performance monitoring (APM) platform may offer a marginal improvement around APM’s golden signals, but that marginal improvement is a substantial lift for the end users tasked with implementing and deploying the new platform. While software may get sold, overworked ITOps and SecOps teams may decide the effort isn’t worth the perceived ROI, leaving the software on the shelf.

On the other hand, observability products that reduce toil for operators will continue to outpace market growth rates. IT departments face acute shortages in skilled staff, leaving companies with little choice but to seek out software that automates high-effort, low-value tasks or otherwise offers a force multiplier to existing teams. Subcategories like AIOps and observability pipelines can address those needs.

Consolidation is Needed to Simplify the Market

Another looming hurdle for the nascent observability market is feature similarity between companies in a subcategory. Two subcategories where this is apparent are application and data observability. In the application observability space, several companies were founded in the wake of the availability of extended Berkeley Packet Filter (eBPF) technology. eBPF allows you to extend the capabilities of the Linux kernel at runtime, and a common use case is to trace applications running in user space. While this is a great technology with several uses, there are simply too many companies with the same idea, the same go-to-market strategy and same limited feature set.

Data observability is another subcategory riddled with clones. Data observability started as a reaction to operational brittleness in the modern data stack, and there are only a few companies with marked differentiation here. I agree with Matt Turck’s assessment that data observability products will drift into the well-defined (but arguably less sexy) data quality category.

CFOs are Looking to Reduce Tool Sprawl and Cut Costs

The challenges with raising growth rounds have been well documented. VC funding represents the supply side of the finance puzzle and IT budgets are the demand side. The demand side is getting squeezed as CFOs clamp down on budget growth. Deals are taking longer to close and often involve more parties eager to reject a budget request from another team because it means more money for their team.

A common refrain from CFOs today is that any new spending must be more than offset by corresponding savings somewhere else. This puts the products representing net-new work with little marginal utility at risk. Observability vendors today need a strong consolidation (e.g. can reduce tool sprawl, serve multiple teams, etc.) and cost savings story to survive in a difficult budget climate.

Cybersecurity remains a budget bright spot. According to a recent survey from IDC, 27% of respondents stated cybersecurity budgets were immune from budget cuts. With changing SEC rules highlighting the need for board-level cybersecurity expertise and reporting of breaches, coupled with the new National Cybersecurity Strategy, the priority around preserving these budgets is clear.

Cybersecurity is Pure Greenfield for Observability

A few short years ago, SecOps teams, threat hunters and incident response teams didn’t have enough data to create an accurate picture of their environment. Endpoint data was sparse and incomplete, cloud-based application deployments were still uncommon, the network remained between the office and data center, and so on.

Today’s SecOps teams, threat hunters and incident response teams have too much data to understand their environment. There are too many alerts and too little context, making finding the signal through the noise a difficult, if not impossible, proposition. From a compliance perspective, much of this data must be retained for longer periods, driving up storage costs.

This is where the principles of observability benefit cybersecurity teams. Much like site reliability engineers need to trace an error well after it occurred, cybersecurity professionals need to trace a breach or find threats over huge volumes of data. They need robust exploration tools, rational data storage and routing options, and, most of all, the ability to do this while maintaining budgets. This is where observability and cybersecurity converge.

Seven years young, observability remains a nascent market despite its rapid growth. Though the market will face growing pains, the opportunities and upside are bright.

May 15, 2023
Five Great DevOps Job Opportunities

DevOps.com is now providing a weekly DevOps jobs report through which opportunities for DevOps professionals will be highlighted to better serve our audience.

Our goal in these challenging economic times is to make it easier for DevOps professionals to advance their careers.

Of course, the pool of available DevOps talent is still relatively constrained, so when one DevOps professional takes on a new role it tends to create an opportunity for others.

The five job postings shared this week are selected based on the company looking to hire, the vertical industry segment and, naturally, the pay scale being offered.

We’re also committed to providing additional insights into the state of the DevOps job market. In the meantime, for your consideration:

Dice.com

ERP International
Fort Meade, Maryland
Senior DevOps Infrastructure Engineer
$150,000 to $180,000

LinkedIn

Lark Health
Remote
Lead DevOps Engineer
$127,700 to $183,900

Simply Hired

Shake Shack
Remote
Senior DevOps Engineer
$123,000 to $155,000

Indeed.com

Peraton
Arlington, Virginia
DevSecOps Data Automation Engineer
$112,000 to $179,000

CareerBuilder.com

Booz Allen Hamilton
Fort Meade, Maryland
DevOps Engineer and Release Manager
$73,100 to $166,000

May 15, 2023
The Hidden Configuration Tax Affecting Uptime SLO

This article is a preview of a talk by Greg Arnette for SLOconf 2023 on May 15 – 18. To watch this talk and many more like it, register for free at sloconf.com.

Tax season was just a few weeks ago, and that got me thinking about how frustrating it is to get hit with a surprise tax bill and how that can relate to uptime: Configuration sprawl. DevOps practitioners constantly strive to maintain uptime service level objectives (SLO) while working with complex and ever-evolving infrastructure and applications. Configuration sprawl is a significant challenge many teams face, negatively impacting system performance and reliability.

My company’s recent survey of 900 DevOps and platform engineering leaders revealed a startling (but not too surprising) conclusion: Most teams struggle managing secrets and configs at scale for infrastructure and applications.

Where Does Configuration Sprawl Come From?

As organizations scale their operations, the number of configuration files and settings grows, making it increasingly difficult to manage and maintain consistency across the board. They may have multiple environments, such as development, staging and production environments, often requiring different configurations, leading to duplicated settings and increased complexity.

Configuration sprawl may also lack standardization as teams adopt different naming conventions, file formats and storage locations without enforced configuration standards, resulting in fragmentation and confusion. Relying on manual processes for configuration management increases the likelihood of errors and inconsistencies.

Grinding to a Halt

So you’ve got a lot of configuration–why should you care? Sure, it means more files to manage, but does it really impact your environment? There are a few places where configuration sprawl will negatively impact you. You might see an increase in error rates because as complexity grows, the likelihood of errors and misconfigurations increases, leading to system instability and downtime.

Increased configuration complexity also means slower troubleshooting. Identifying the root cause of issues becomes more challenging and time-consuming when configurations are disorganized and fragmented. Ultimately this means less agility as you are forced to slow the deployment of new features and updates, impeding the organization’s ability to respond to market demands.

How to Overcome Configuration Sprawl

Don’t worry–you can tackle configuration sprawl, keep your SLOs on track and reduce the efforts required to manage and maintain configurations. To mitigate the harmful effects of configuration sprawl, consider adopting these best practices:

Don’t repeat yourself (DRY)
DRY config means variable names are declared once. Then flexible variable values are injected across build, deploy and runtime phases as defaults, inheritances and overrides. DRY avoids errors by allowing single changes to propagate where they are needed.

DRY solves the common problem of forgetting to update critical secrets or parameters and failing to synchronize related changes (e.g., client and server port numbers or rotating a database password.)

Decouple and abstract
Decoupling and abstracting config means externalizing config from source code, separating how code and configuration interact, and isolating the specific values (log level) from the configuration interface (how the log level value is fetched.)

This approach allows the same code to run in different environments using parameterized config. Abstracting config supports a best practice of implementing independent “code and config change” life cycles. Teams that frequently need to change config independently from code should decouple and abstract.

Centralized config
Consolidate configuration complexity into a single location because it’s easier to manage complexity within a single location versus multiple locations. Syncing config to the edge (where it is consumed) mitigates downtime. For example, it’s easier to fetch the configuration for service X versus coding all the logic needed to generate the configuration for service X.

Standardize your configuration management by enforcing consistent naming conventions, file formats and storage locations across teams to reduce confusion and streamline configuration management. Centralizing the configuration and generating that configuration with transformations allows all the places that consume the configuration to be simple. Simple equals easy to understand and troubleshoot.

Version control & monitoring
Use a version control system like Git to track config changes to configuration files, making identifying and resolving issues easier. And continue to monitor changes and identify potential problems before they impact system performance.

Train your team
Ensure all team members know your best practices and patterns for configuration management. The proliferation of microservices and teams operating in parallel, establishing their unique naming schemes and patterns, exacerbates this problem for DevOps teams.

Cutting the Uptime Tax

By understanding the causes of configuration sprawl and implementing best practices to manage and reduce its impact, teams can improve their uptime SLOs and maintain a more reliable, high-performing infrastructure. Remember, investing in the proper tools and processes for configuration management is not only a best practice but also a critical factor in ensuring the long-term success of your DevOps initiatives. Those are tax cuts I can get behind.

May 11, 2023