devopsy.kinsta.cloud

Category: Continuous Delivery

Continuous delivery is a software engineering discipline that allows organizations to consistently develop and release quality software at a rapid pace. It is achieved by automating the software development lifecycle, enabling tasks such as development, testing and release to occur concurrently and continuously.

What is Just-In-Time (JIT) Permission Management, and Why is it Essential?

Have you ever experienced the frustration of discovering an unexpected charge on your bank statement for a forgotten gym membership? Perhaps you kept it active “just in case” you wanted to exercise someday. Similarly, always-on privileged access, also known as “standing privileges,” can lead to similar challenges.

Surprisingly, 77% of organizations provide unrestricted access to employees who do not require it. However, maintaining an always-on approach does not necessarily facilitate their work; instead, it creates opportunities for security breaches that can go unnoticed. In fact, in 2022 alone, 55% of organizations fell victim to cyberattacks where hackers exploited privileged credentials, a critical attack vector as identified by Verizon.

This article explores the concept of just-in-time (JIT) permission management and its significance in providing the necessary security and efficiency for access control in organizations. We will delve into various types of JIT management, including automated JIT, and discuss the best practices for its implementation within your business.

Understanding Just-In-Time (JIT) Permission Management

JIT, also known as just-in-time access, is a cybersecurity practice that adheres to the principle of least privilege. It grants users access to resources, such as applications and systems, only for a limited period when it is specifically needed. Once the designated time expires, users lose access to these resources. This methodology not only limits a user’s access timeframe but also reduces the chances of attackers infiltrating your cloud security perimeter.

With the increasing number of applications, services, users and resources in the cloud, 45% of breaches in 2022 were cloud-based, making JIT an essential requirement. While traditional privileged access management (PAM) processes, such as session management, are effective for on-premises environments, JIT is particularly suitable for controlling access across cloud resources.

Types of JIT Permission Management

JIT permission management can be implemented in different ways, including:

Ephemeral: This approach involves creating a one-time account to fulfill the user’s access requirements, which is then disabled or deleted.

Temporary: Users can request elevated privileges on their existing accounts when necessary.

Justification-based: Users must provide a valid justification for requiring privileged access, based on predetermined policies. A privileged account and credentials are created and rotated using a central vault in this “broker and remove” access model.

The Importance of Automated Just-In-Time Access

Managing JIT permissions manually can be akin to playing a never-ending game of whack-a-mole, with constant access requests popping up across your organization. Can you realistically allocate sufficient time and personnel to grant and revoke access throughout the day, especially considering the resource constraints faced by 59% of organizations that struggle to deploy zero-trust?

In contrast, automated JIT platforms alleviate the challenges posed by manual permission management. These platforms validate, monitor and revoke access without requiring human intervention. By offering features such as auto-expiring permissions and comprehensive reporting capabilities, automated JIT platforms empower users to self-serve permission requests while maintaining your organization’s security posture. Entrusting permission management to an automated JIT platform minimizes human error, reduces the attack surface, eliminates bottlenecks and ultimately preserves productivity.

Benefits of Automated JIT

Automated JIT not only eases the burden on your IT and security teams but also provides several other advantages:

Enhanced security posture: Compromised or stolen credentials were the most common attack vector in 2022, according to IBM’s Cost of a Data Breach report. By eliminating the need for standing privileges, automated JIT significantly reduces the risk of privilege abuse and breached identities.

Improved business productivity: With an automated JIT approval workflow, you can bid farewell to manual review cycles, waiting times and human error. You can grant access at scale, tailored to specific tasks, significantly improving operational efficiency.

Compliance adherence: Automated JIT enables you to meet compliance and customer requirements, such as SOC2, by enforcing zero trust and least privilege access. These platforms include auditing and reporting features that provide visibility into all sessions and privileges, ensuring compliance with regulations like SOC2 and PCI-DSS.

Enabling JIT Access: Best Practices

To successfully enable and implement JIT access, consider the following best practices:

Identify and inventory: Begin by identifying the accounts and assets with the most privileges, particularly those belonging to administrators, which pose the highest risk. Implement JIT access control for these accounts first and then gradually extend it throughout the organization.

Use RBAC and ABAC: Utilize role-based access control (RBAC) and attribute-based access control (ABAC) as supplementary solutions to define granular policies and circumstances for elevated access. Categorize accounts, differentiate their rights, and create control policies that users must satisfy to gain access.

Define and enable temporary access: Apart from justification-based access, establish criteria for users requesting temporary access, including which accounts are eligible and the duration of access. Implement time-based controls, such as granting access to specific resources during predefined days and times.

Record and audit activity: An automated access management solution enables you to log all access activities, receive alerts for suspicious behavior, and record JIT-privileged access. Maintaining a comprehensive digital paper trail is essential for auditing, governance, and compliance with regulations such as SOC2 and PCI-DSS.

Assign responsibility: Delegate responsibilities to employees and determine who will review permission requests. Properly training employees on granting and revoking access, especially during critical incidents like “break glass” and “on-call” situations, minimizes the risk of incidents. Automated JIT facilitates configuring access flows for these scenarios, helping resolve incidents promptly and eliminating bottlenecks.

Use short-lived (ephemeral) credentials: Regularly rotate credentials manually to invalidate them, preventing hackers from exploiting stolen passwords. Employ a centralized vault with the highest level of security clearance to manage these credentials effectively.

Employ an automated access management tool: Simplify cloud access management by leveraging solutions like Apono, which enforces an automated JIT approach. By using such tools, you can streamline processes, reduce over-privileges, and ensure that permissions are never overlooked, safeguarding your organization from unnecessary risks.

October 25, 2023
OpsCanvas Generates Code to Automate DevOps Workflows

OpsCanvas, Inc. today emerged from stealth to unveil a Draw & Deploy platform that generates code to provision cloud infrastructure.

OpsCanvas CEO Brian Kathman said as an alternative to traditional infrastructure-as-code (IaC) tools, this offering makes DevOps practices more accessible to a wider number of organizations.

The OpsCanvas platform analyzes application architecture diagrams to deterministically create known-good code that the company has vetted to replace scripts DevOps teams would otherwise have to create. The platform then, via a single click, makes it possible to automate common tasks for deploying, cloning, updating, rolling back, extending and decommissioning resources.

Unlike generative artificial intelligence (AI) platforms, OpsCanvas does not use a large language model (LLM) to generate code. Instead, vetted code is created based on the diagrams that the platform creates. OpsCanvas also encodes best practices, standardized network topology for applications and provides integration with multiple continuous integration/continuous delivery (CI/CD) platforms.

As a result, DevOps teams are provided with a comprehensive view of diagrams, including the number of stored versions and deployment status, for every application deployed.

The goal is to enable organizations of any size to manage entire DevOps workflows using a platform that can generate more than 10,000 lines of already-tested, custom configurations in less than 30 seconds, said Kathman.

In effect, OpsCanvas aims to democratize DevOps by eliminating the need for specialized skills to provision and deploy applications. In addition to accelerating the pace at which applications are built and deployed, that approach will also make it feasible for more small-to-medium businesses (SMBs) to embrace DevOps, noted Kathman.

Regardless of the path chosen to make DevOps more accessible, the days when software engineers wrote scripts to provision cloud infrastructure are now numbered. Either using code that has been vetted by platforms such as OpsCanvas or relying on co-pilots to generate code using large language models (LLMs), provisioning and deploying applications is about to become both simpler and more secure.

Many of the cybersecurity issues that organizations experience today can be traced back to misconfigured infrastructure provisioned by developers using IaC tools. As generative capabilities become more widely used, the quality of the code used to provision infrastructure should substantially improve. As that transition occurs, DevOps engineers should be able to allocate more of their time to address more complex challenges.

One way or another, DevOps is about to become less tedious in ways that should help reduce burnout that leads to turnover. At the same time, the number of organizations capable of embracing DevOps best practices should increase as it becomes simpler for IT administrators who lack programming skills to provision infrastructure.

In fact, the velocity at which software is built and deployed today is about to exponentially increase in a way that most DevOps professionals never imagined might be possible.

October 24, 2023
5 Reasons DevOps Teams Need Code Mapping

Before we delve into the benefits and importance of code mapping, it’s essential to understand what it is. Code mapping is a visual representation of the structure and flow of your code. It’s like a roadmap, laying out the entire landscape of your code, showing all the paths, intersections and dead-ends.

Just like a physical map gives you a bird’s eye view of a geographical area, a code map gives you a high-level view of your code. It helps you understand the dependencies, connections and interactions between different parts of your code. This high-level overview is crucial for understanding how your code works and identifying any potential issues or bottlenecks.

Code mapping is not a new concept. However, with the advent of DevOps and Agile methodologies, its importance has grown. We’ll discuss why code mapping can be a powerful tool for DevOps teams and how to best use it.

5 Reasons DevOps Teams Need Code Mapping

Let’s explore the top five reasons why DevOps teams need code mapping.

Enhanced Collaboration

In a DevOps environment, collaboration is key. Developers, operations teams and quality assurance all need to work together seamlessly for the smooth execution of projects. Code mapping facilitates this collaboration by providing a clear, visual representation of the code. Everyone on the team can look at the map and understand the structure and flow of the code, regardless of their technical expertise.

Code mapping can also act as a communication tool. Team members can use it to explain their ideas, discuss problems, and propose solutions. It’s like a common language that everyone on the team can understand and use.

Efficient Troubleshooting and Debugging

Troubleshooting and debugging are an integral part of software development. However, they can also be time-consuming and frustrating, especially when dealing with complex codebases. Code mapping can significantly simplify this process by visually highlighting the problematic areas and dependencies in your code.

With a clear map of your code, you can easily trace the flow and identify where things are going wrong. You can see how different parts of your code interact with each other and where potential bottlenecks or issues might be. This makes troubleshooting and debugging much more efficient and less stressful.

Simplified Onboarding Process

Onboarding new team members can be a challenging task, especially when dealing with complex codebases. It can take weeks, if not months, for a new developer to fully understand the code and start contributing effectively. Code mapping can significantly speed up this process.

With a clear code map, new team members can quickly get a high-level understanding of the code structure and flow. They can see how different parts interact with each other, where the dependencies are, and how everything fits together. This gives them a solid foundation to start diving deeper into the code and start contributing sooner.

Improved Code Quality and Maintainability

Maintaining high code quality is crucial for the long-term success of any software project. Poorly written or structured code can lead to numerous problems down the line, from performance issues to security vulnerabilities. Code mapping can help prevent these problems by improving the quality and maintainability of your code.

By visualizing your code, you can easily spot areas of complexity or redundancy that might impact the quality or maintainability of your code. You can identify parts of the code that need refactoring or optimization and make informed decisions about code improvements. This leads to cleaner, more efficient, and more maintainable code.

Efficient Release and Deployment Management

In a DevOps environment, frequent and reliable releases are critical. However, managing these releases can be a complex task, especially when dealing with large, interconnected codebases. Code mapping can simplify this process by providing a clear view of the code structure and dependencies.

With a clear map of your code, you can easily see how changes in one part of the code might impact other parts. This helps you plan your releases more effectively and prevent potential deployment issues. It also makes it easier to track changes and roll back if needed, making your release and deployment process much more efficient and reliable.

Best Practices for DevOps Teams

Here are a few best practices that can help DevOps teams make the most of tools like Microsoft Ignite and CodeSee.

Prioritize Clarity

Clarity should be a priority when it comes to code mapping. A clear, well-structured code map can make it much easier for developers to understand the codebase and how its various components interact with each other. Conversely, a cluttered or confusing map can make the task of understanding and working with the codebase significantly more difficult.

With a clear map, developers can easily identify the parts of the code they need to work on, understand how these parts relate to the rest of the codebase, and avoid potential errors or issues that might arise from a lack of understanding.

To prioritize clarity, it is important to adopt a consistent style and structure for the map. This might involve using a specific naming convention for the different parts of the code or adopting a specific color scheme to differentiate between different types of code.

Version Control

Just like with the code itself, implementing version control for your code maps is highly beneficial. Version control enables developers to keep track of changes made over time. This is important because it allows for easy rollback to previous versions if a newly introduced change leads to unforeseen issues or complexities.

Moreover, version control provides a historical record of the evolution of your codebase. By looking back at previous versions of the code map, developers can gain insights into how the codebase has evolved and grown over time. They can identify patterns and trends and use this information to make informed decisions about the future development of the codebase.

Use Interactivity

Modern code mapping tools offer a variety of interactive features that can greatly enhance the usefulness and usability of your code maps. For instance, some tools allow you to zoom in and out of different parts of the map, enabling you to focus on specific areas of the codebase. Others offer the ability to add notes or comments, making it easier to share insights and observations with the rest of the DevOps team.

Interactivity can make code maps more engaging and intuitive. It can transform them from static, one-dimensional diagrams into dynamic, interactive tools that offer a wealth of information and insights.

Integrate With Documentation

Finally, integrating code maps with documentation can greatly enhance their usefulness. Documentation provides detailed information about the codebase, including the purpose of different parts of the code, how they function and how they interact with each other. By linking this information with the visual representation, developers can gain a more complete and in-depth understanding of the codebase.

Integrating code maps with documentation also makes it easier for new members of the DevOps team to get up to speed. They can use the code map to visually understand the structure of the codebase and then refer to the documentation for more detailed information. This can significantly shorten the learning curve and help new team members to become productive more quickly.

Conclusion

In conclusion, code mapping is a powerful practice that can greatly enhance the effectiveness and efficiency of DevOps teams. By adopting best practices such as integrating feedback loops, prioritizing clarity, implementing version control, leveraging interactivity and integrating with documentation, teams can optimize their code mapping processes and reap the many benefits that this practice has to offer.

October 24, 2023
Maximizing Data Insights: How OLAP Empowers Modern Data Stacks

In the era of big data and cloud computing, online analytical processing (OLAP) has fallen out of favor. As modern data warehouses evolved to accommodate larger volumes of data and claims faster processing capabilities, OLAP fell out of fashion as a notion emerged that it would struggle to manage the combinatorial explosion. The focus shifted toward runtime querying engines. Instead of using predefined or pre-aggregated OLAP cubes, many business intelligence (BI) and analytics tools used their own data modeling applications that worked directly with raw data.

However, in recent times, there has been a realization that OLAP, when implemented intelligently, can be highly beneficial even in the context of a massive scale of data and cloud computing. Modern OLAP leverages advanced technologies and optimizations that overcome any previous limitations. Besides being cost-effective, especially for cloud implementations, it delivers real performance advantages.

Modern OLAP systems are designed to handle large datasets efficiently. One of the key strengths of OLAP lies in its ability to provide real-time or near-real-time insights, enabling businesses to make data-driven decisions with greater agility. The multidimensional data model of OLAP cubes facilitates instant query response, empowering users to derive valuable insights from complex data sets without compromising on query performance.

Consider These Advantages of Using OLAP in Modern Data Stacks

Performance: As data volumes continue to grow, OLAP remains a powerful technology to extract valuable insights and unleash the true potential of modern data stacks and cloud environments. OLAP’s multidimensional data model with pre-aggregation cubes enables efficient data processing and analysis, delivering faster insights and sub-second query responses. Modern algorithms further enhance performance, allowing organizations to handle large-scale data workloads without worrying about data explosion problems.

Unique OLAP Capabilities: Some domains, like finance, have an inherent need to preserve crucial OLAP capabilities even with the growing data storage and compute requirements for which they make a move to modern data stacks and cloud environments. Moving to the cloud enables data scalability, addressing the challenges of handling vast data volumes. However, they cannot afford to compromise on essential OLAP functionalities such as advanced hierarchies, like parent-child relationships, and alternate hierarchies with weighted rollups.

These capabilities are indispensable, especially in the world of financial analytics, regardless of how modern and advanced the new data stack may be. By leveraging OLAP in the cloud, organizations can continue to analyze and explore data with complex hierarchies efficiently, ensuring accurate and in-depth financial insights without sacrificing the flexibility and analytical power that OLAP provides.

Cloud Architecture and Cost Effectiveness: OLAP in cloud architecture presents a powerful and forward-looking solution that combines the benefits of OLAP technology with the scalability and cost-effectiveness of cloud computing. By harnessing the cloud’s resources, OLAP can efficiently process and analyze large volumes of data, thus meeting the growing demands of modern businesses.

Once OLAP cubes are constructed and populated, they provide a highly efficient and aggregated representation of the data. These pre-aggregated structures accelerate query performance, enabling teams to access insights in real-time or near-real-time. BI teams can seamlessly integrate existing tools with the OLAP model, fostering a cohesive and familiar analytics environment. It empowers decision-makers to explore data interactively, drill down into details, and gain valuable insights.

One of the primary advantages of OLAP in the cloud is its cost-effectiveness. OLAP technology, being primarily based on pre-aggregation, drastically reduces runtime querying compute cost, as answers to business queries are already pre-computed and stored in an aggregated structure. At querying time, OLAP tools have to look for right answers from the aggregated structure—not the cloud data storage—and return the query response, which results in much less compute cost as compared to runtime aggregation compute cost. It truly enables build once, query many times culture. Apart from this, businesses can leverage the cloud’s elasticity to scale computing and storage resources up or down as their data requirements fluctuate. This adaptability ensures companies can handle peak workloads without overprovisioning and reducing unnecessary expenses.

Universal Semantic Layer: A USL approach ensures consistent data definitions and analytics across the organization, reducing data discrepancies and improving data governance. A common understanding of the data enables better communication among teams and results in a smooth implementation of a data-driven decision-making process.

At the same time, using a USL means that multiple business intelligence tools and reporting platforms can connect to and leverage the same semantic model. Hence, various parts of the organization can continue to work with their familiar BI interfaces while accessing common underlying data.

OLAP lends itself naturally to delivering on a powerful universal semantic layer. The concept of dimensions, measures, hierarchies and calculations forms the foundation of an OLAP semantic model. OLAP semantic models also accommodate complex calculations, custom expressions or formulas required for complex business-specific logic. These components provide a powerful way to structure and organize data, making it more accessible and intuitive for business users who need to derive insights from the data.

If we consider typical sales data, product categories, geographic regions and time periods could be dimensioned. Measures, on the other hand, would be sales revenue, profit margins, quantities sold or any other KPIs. Hierarchies allow data to be analyzed at different levels of granularity. Users may roll up or drill down to view data at different time intervals such as year, quarter, month, day, etc.

USL and OLAP models are designed to shield business users from the underlying complex data modeling and schemas. They can work with filters, dimensions and measures that are business-focused representations of the data using familiar business terms while abstracting technical intricacies. They can view and explore data through intuitive hierarchies, filters and measures aligned with their business processes and KPIs.

Conclusion

OLAP in cloud architecture represents a forward-looking, fast and cost-effective solution. It capitalizes on the cloud’s scalability and pay-as-you-go model to process vast amounts of data efficiently, while its pre-aggregated structures and real-time capabilities empower business intelligence teams to gain valuable insights seamlessly. With its agility, OLAP in the cloud is well-positioned to meet the challenges of the future, providing organizations with a competitive edge with data-driven decision-making.

October 23, 2023
OpsMX Adds Deployment Firewall to Ensure Compliance

OpsMx today added a deployment firewall to its Deploy Shield portfolio for securing continuous integration/continuous delivery (CI/CD) platforms.

The deployment firewall automatically applies rules to ensure DevOps teams comply with mandates such as the payment card industry data security standard (PCI DSS) or the Health Insurance Portability and Accountability Act (HIPAA). It then evaluates a release against a checklist of policies and blocks any deployments into a production environment unless a specific exception has been defined. In those instances, DevOps teams can also set an alert to address that issue at a later date.

In addition, DevOps teams can extend those rules defined by OpsMX to address specific compliance requirements.

The checklist includes making sure vulnerability scams have been run, tracking when the manifest was last updated, confirming that operational controls are being followed, checking whether images have been modified and evaluating the infrastructure being employed and performing an analysis of the performance impact.

A simulation feature also allows developers to check their release for compliance before it is time to deploy.

OpsMx CEO Gopal Dommety said the goal is to streamline the effort that is required to meet multiple compliance mandates as the pace at which applications are being developed and deployed continues to increase.

Today there are simply too many data silos for DevOps teams to manage, with each one presenting yet another opportunity to run afoul of a compliance mandate, he added. The overall goal is to reduce the cognitive load being placed on developers as more responsibility for compliance is shifted left, noted Dommety.

The deployment firewall currently supports DevOps platforms such as Jenkins, Argo and Spinnaker, with support for GitHub Actions and GitLab to follow.

OpsMX claims its overall customer base has increased by more than 60% in the first half of this year, with customers including Google, Cisco and Western Union. In addition to Deploy Shield, the company provides a Delivery Bill of Materials (DeliveryBOM) tool to capture a consolidated record of every step in the application software delivery and deployment process. Security checks, approvals, policy enforcement and audits are tracked via one tool. In addition, the company provides tools for managing the deployments of software via either the Argo or Spinnaker CD platforms.

It’s now only a matter of time before various regulations that apply to the building and deploying applications become more stringent. Tolerance for sloppy application development processes is dropping as legislative bodies increasingly determine that software development practices are a big cybersecurity problem that threatens national security and the global economy.

Organizations that fail to comply with those regulations will be subject to even heftier fines than the ones that already exist. In many cases, that may slow down the rate at which applications are deployed unless DevOps teams find ways to manage compliance-as-code more effectively as part of a larger DevSecOps workflow.

When it comes to DevSecOps, progress tends to be slow in most organizations, but as more tools become available to automate various tasks, it’s now only a matter of time before software supply chains are more tightly secured. Hopefully, the pace at which those advances are being made will occur faster than cybercriminals can exploit the vulnerabilities that today are too numerous to manually track.

October 23, 2023
Survey Sees Spike in Untested Code Leading to a DevOps Crisis

A survey of 500 developers conducted by the market research firm OnePoll on behalf of Sauce Labs found more than two-thirds (67%) admitted they pushed code into a production environment without testing, with more than a quarter (28%) acknowledging they do so on a regular basis.

More troubling still, 60% also admitted using untested code generated by ChatGPT, with more than a quarter (26%) doing so regularly.

More than two-thirds of developers have also merged their own pull requests without a review, with 28% confessing they do so often or very often.

Finally, the survey found three-quarters of developers admitted to circumventing security protocols, with 39% of developers doing so routinely. A full 70% also acknowledged using a coworker’s credentials to circumvent restrictions for access to data and/or internal systems, with 41% doing so regularly.

Jason Baum, director for community for Sauce Labs, said that while the survey makes it clear there are a lot of instances where best practices are being ignored, much of this behavior can also be attributed to the amount of extra work developers now routinely take on. The survey found more than three-quarters of developers (77%) have, for example, assumed more responsibility for testing in the last year.

The issue organizations need to evaluate is how much of this is due to developers being lazy versus looking for shortcuts in an era where too many tasks have been shifted left to developers and which they are not able to absorb, said Baum. Many organizations might have better outcomes if more tasks were automated within the context of a DevOps workflow to reduce the amount of cognitive load being placed on developers, he added.

The simple fact of the matter is there is a fundamental skills gap, said Baum. The number of so-called full stack developers capable of managing the entire software development life cycle is relatively small, he noted.

In the meantime, the amount of code generated is steadily increasing as developers take advantage of generative AI platforms to improve productivity. The challenge is the large language models (LLMs) being used to create that code were trained using examples of code pulled from across the web. In addition to having known vulnerabilities, it is of varying quality. In fact, those LLM platforms will frequently generate different snippets of code that vary in quality at different times.

DevOps teams that are responsible for managing the overall codebase are now finding their pipelines are becoming overwhelmed. That’s becoming especially problematic at a time when governments around the world are crafting legislation that will hold organizations more accountable for the security of the applications they build and deploy.

Eventually, AI will also be used to increasingly automate DevOps workflows and should enable software engineers to keep pace with the accelerated rate at which code is being written. In the longer term, LLMs that have been trained using vetted code will also generate higher quality code than a general-purpose LLM such as ChatGPT. In the meantime, however, massive amounts of untested code will likely create a series of downstream cascading events that will come back to haunt DevOps teams in the months and years ahead.

October 20, 2023
DevOps and ITIL Integration: Driving Collaborative Agility in ITSM

Businesses face continuous pressure to deliver high-quality IT services efficiently. One emerging integration to meet these goals is between DevOps and IT service management (ITSM), more commonly known as DevOps+.

DevOps+ is becoming more than a trend and has emerged as a powerful collaborative approach to meet these challenges. By combining the strengths of DevOps’ agile software development practices and ITSM’s structured service management approach, organizations can foster better collaboration, enhance agility and improve overall service delivery efficiency.

DevOps and ITSM Symbiosis

DevOps and ITSM traditionally operated as separate frameworks with distinct focuses. DevOps emphasizes rapid software delivery and collaboration between development and operations teams. On the other hand, ITSM-based ITIL (information technology infrastructure library) provides best practices for ensuring service stability, reliability and adherence to governance and compliance.

Integrating DevOps and ITIL creates a symbiotic relationship, allowing organizations to leverage the best of both worlds. DevOps+ encourages a cultural shift, breaking down silos and promoting cross-functional communication. This collaboration enhances the entire software development lifecycle, leading to faster feedback loops, accelerated innovation, and increased responsiveness to business needs.

Enhancing Agility and Speed

DevOps+ integration brings greater agility and speed to IT service management. Organizations can streamline the release process by automating deployment pipelines and implementing continuous integration and continuous delivery (CI/CD) practices, reducing time-to-market for new features and updates. Agile development methodologies enhance the adaptability of ITSM processes to meet evolving business requirements effectively.

Ensuring Stability and Reliability

While agility is crucial, it must not compromise service stability and reliability. ITIL’s best practices provide a strong foundation for maintaining service quality. By incorporating ITIL processes such as incident management, change management and problem management, organizations can effectively manage risks, reduce service disruptions and ensure the stability of critical IT services.

Leveraging Automation and Orchestration

Automation is central to both DevOps and ITIL. Through integration, organizations can leverage automation and orchestration tools effectively. Automated testing, infrastructure provisioning and configuration management improve efficiency and consistency across the ITSM lifecycle. This enables teams to focus on higher-value tasks, reducing manual errors and operational overhead.

Emphasizing Customer-Centricity

DevOps+ puts the end-user experience at the forefront of service delivery. By continuously seeking customer feedback and measuring service performance, organizations can better align their offerings with user expectations. A customer-centric approach fosters higher satisfaction, loyalty, and competitive advantage.

Breaking Down Barriers to Change

Change management is vital in ITIL, ensuring that modifications are planned and controlled. In the context of DevOps+, change management becomes more adaptive and responsive, facilitating a culture that embraces change as an integral part of the development process. This promotes experimentation, reduces resistance to change, and accelerates innovation.

Challenges of DevOps and ITIL Integration

Despite the numerous benefits of DevOps+ integration, organizations may encounter challenges during implementation:

Cultural Shift: Merging DevOps and ITIL requires a cultural shift. Overcoming resistance to change and fostering collaboration between traditionally separate teams can be challenging but is crucial for success.

Process Alignment: DevOps and ITIL have different processes and terminologies. Aligning these methodologies while preserving their individual strengths can be complex.

Skill Sets: Integrating DevOps and ITIL necessitates upskilling teams in both domains to operate effectively in the integrated environment.

Tooling and Automation: Adopting suitable tools and technologies that support DevOps+ practices and integration is vital but may require significant investments.

Metrics and Performance Measurement

Measuring the success of DevOps+ integration is essential to understand its impact and effectiveness. Key performance indicators (KPIs) can help monitor progress and identify areas for improvement.

Deployment Frequency: The frequency of software deployments indicates the efficiency of the integrated approach. Higher deployment frequency implies faster and more reliable releases.

Mean Time to Resolve Incidents (MTTR): MTTR measures how quickly teams can resolve incidents and restore services after disruptions occur. A reduced MTTR demonstrates improved service reliability and faster incident resolution.

Customer Satisfaction Scores: Measuring customer satisfaction provides insights into how well the integrated approach meets end-users’ needs and expectations.

Continuous Improvement and Feedback Loops

Continuous improvement is at the core of DevOps+. Establishing feedback loops and conducting regular retrospectives empowers teams to identify areas for improvement and implement iterative enhancements to processes and practices.

Implementing DevOps+ integration requires a well-structured roadmap. For example, assessment and planning help leaders evaluate the organization’s current ITSM practices to understand how DevOps and ITIL principles align and define integration vision and goals.

Additionally, your continuous optimization efforts allow you to continuously monitor performance metrics, gather feedback, and conduct retrospectives to identify improvement areas, leading to better team collaboration and communication. Fostering cross-functional collaboration between development, operations and other relevant teams is easily accomplished. To meet these goals, encourage regular meetings, schedule joint planning sessions and enable shared documentation to ensure everyone is aligned on goals and progress.

When this happens, leverage automation tools to streamline manual processes and increase efficiency. Implement continuous integration and continuous delivery pipelines to automate software deployment and testing. Automation enhances consistency, reduces the risk of human errors and adds a new layer for measuring and evaluating to continuously measure the performance of the integrated approach using defined KPIs. In this vein, you can regularly review progress against the initial vision and goals to ensure the integration aligns with organizational objectives.

Next, as the DevOps+ integration proves successful in pilot projects, gradually scale and expand it to other teams and departments. Share the success stories and best practices to encourage broader organizational adoption.

Conclusion

In conclusion, the integration of DevOps and ITIL, known as DevOps+, marks a transformative approach to IT service management. By combining the agility and rapid deployment of DevOps with the stability and reliability of ITIL, organizations can drive collaborative agility and achieve efficient service delivery.

DevOps+ integration requires overcoming cultural barriers, aligning processes and upskilling teams, but the rewards are immense. With clear metrics to measure success, organizations can continually optimize their DevOps+ approach to meet customer needs and adapt to evolving market demands.

As technology evolves, DevOps+ integration will remain at the forefront of cutting-edge IT service management, empowering organizations to embark on a journey of innovation, efficiency and excellence.

By embracing DevOps+ as a guiding philosophy, organizations can cultivate a culture of collaboration, continuous improvement and customer-centricity, laying the groundwork for a prosperous future in the dynamic world of IT service management.

October 20, 2023
Preparing for the Shift to Platform Engineering

Platform engineering is one of the hottest trends in the software delivery world and is expected to represent the next evolution of DevOps. According to Gartner’s 2023 study of top strategic technology, 80% of software engineering organizations will establish platform teams as internal providers of reusable services, components and tools for application delivery by 2026. What used to happen in one day in the developer world now needs to happen in five minutes. With high pressure for organizations to scale operations quickly, especially in a marketplace clamoring for the benefits of artificial intelligence (AI), automation is increasingly important in platform engineering. It reduces manual errors, streamlines the development process and boosts growth and efficiency. As technology advances rapidly, it becomes more critical for organizations to proactively navigate newly emerging trends within platform engineering to remain competitive. The keys to successful platform engineering include cloud-native architecture, AI and machine learning (ML), and evolving tools that use intuitive interfaces and pre-built components to create functional applications.

Platform Engineering is the Natural Progression of DevOps

Platform engineering expands on the practices of DevOps by moving development out of silos and taking a collaborative, bird’s-eye view of an organization’s technology. Platform engineering is built on a foundational infrastructure that enables self-service capabilities so developers can deploy code faster, more reliably and more securely. While platform engineering represents the next wave of change in DevOps, it’s essential for leaders and developers to understand that it isn’t a singular technology or approach. The shift to platform engineering also represents a cultural shift for developers and leaders to address the needs, challenges and rapid pace of modern software development by optimizing processes, accelerating delivery and reducing the complexity of operations. For platform engineering to succeed, it’s critical to approach it from the perspective of a technology-driven culture that creates solutions with a developer’s perspective in mind. By seeking to improve the developer experience first, enterprises can achieve better results.

Trends Shaping the Landscape

Platform engineering itself is an emerging trend, but several essential ingredients shape how it will impact businesses today and in the future. For organizations to reap its benefits, understanding these essential trends will keep them ahead of the pack.

Cloud-native architecture is fundamental to modern platform engineering by designing and building applications for cloud environments. Infrastructure-as-code (IaC) is a key component, enabling the automated provisioning and management of resources. Cloud-native platforms like Kubernetes, Terraform or Docker facilitate container orchestration and scaling, while serverless computing isolates infrastructure management entirely. This trend ensures that platform engineering is agile, scalable and cost-effective, with minimal downtime and easier maintenance.

Automation is pivotal in platform engineering, reducing manual errors and streamlining deployment workflows. It allows developers to build, test and deploy software more efficiently, leading to faster and more reliable product releases. In fact, 43.2% of early adopters of platform engineering saw faster product delivery. Automation tools like Jenkins, Ansible and Terraform are commonly used to automate infrastructure provisioning, configuration management and deployment pipelines. Machine learning (ML) models can perform automated troubleshooting by analyzing historical data to identify patterns and predict potential issues, proactively addressing problems and minimizing downtime. This also helps with performance optimization. By using AI that can analyze application performance data, automation takes platform engineering to the next level by suggesting optimizations to improve resource utilization and user experience. AI-powered tools assist in generating code or script, which is vital to reducing development time and human error. ML algorithms empower the security side of platform engineering, a top priority for any developer. It’s possible to leverage automated security tests during development and maintenance, even detecting and responding to security threats in real-time. When AI handles routine tasks, developers are less likely to feel overwhelmed or experience burnout. This allows developers to focus on more creative tasks, complex problem-solving and innovation, resulting in more efficient operations to help them achieve higher levels of productivity, security and consistency.

Evolving development tools that create functional applications using intuitive interfaces and pre-built components increasingly simplify application development and deployment. Low-code and no-code platforms allow developers to create applications with minimal coding effort, utilizing visual interfaces and pre-built modules. Serverless frameworks like AWS Lambda and Azure Functions enable developers to focus solely on writing code. An increasing reliance on and advancement of application programming interface (API) gateways provides a simplified way to manage and expose APIs, enhancing application integration and connectivity. Containerization platforms are becoming similarly powerful tools. Platforms like Docker and Kubernetes simplify application deployment across diverse environments by offering standardized containers and orchestration. Adopting a strategic suite of these evolving tools requires a clear understanding of product goals, organizational goals and developmental capabilities.

How Leaders can Support Platform Engineering

It’s projected that 30% of organizations are currently evaluating the adoption of platform engineering practices, and its prevalence will only grow. While platform engineering’s trends are empowered by digital transformation and new technologies, it’s vital to remember that platform engineering represents a shift in DevOps culture. To effectively support the transition, leaders must commit to a culture of platform engineering. Simply adopting technology isn’t enough. It needs to be backed by a thorough strategy that allows developers to truly benefit from the tools and structures of platform engineering. What does this look like? Success requires leaders and developers to encourage collaboration and break down silos between operations and development teams. It’s possible to build a bridge between developers and operations by committing to cloud migration, creating a centralized platform and investing in collaborative tools and the strategy to back it up.

To engage in platform engineering requires dedication to a collaborative culture instigated from the top, empowered by overall strategic decisions and operations. This includes continued learning for developers to stay on top of new languages, trends, challenges and priorities, internally and externally. Teams are more successful when they utilize performance metrics to track workflows that help them conduct effective maintenance and improve on a consistent and ongoing basis.

Embracing Change is Embracing Progress

In the fast-paced and continuously improving world of software development, it’s easy to think that each wave of technology will replace the last. Of course, experts know this isn’t true. Iteration is vital to advancing technologies, and it’s important to embrace the next iteration of progress to stay competitive. Automation and platform engineering are tools that enable developers rather than replace them. It’s important to note that platform engineering doesn’t mark the end of DevOps. Rather, it takes the core foundations that DevOps has built to the next level to empower developers with self-service capabilities, foster collaboration, enhance scalability, accelerate deployment and instigate sustained business growth.

October 19, 2023
How to Build a Data Platform for Self-Service, Ad-Hoc Analytics

In a digital world, data is often the differentiator between success and failure. Whether it’s defending against cybersecurity threats, improving application performance, or resolving full-blown outages, data is critical to the modern DevOps team.

In fact, there are plenty of situations where DevOps engineers, SRE teams and other observability-focused departments need to explore reams of data rapidly and flexibly. An SRE trying to figure out the root cause of latency may need to dissect the movement of data across endpoints to find the malfunctioning one. Another engineer may need to compare historical and current performance to find anomalous behavior. Lastly, a DevOps engineer may have to dissect user performance metrics to understand the scope of a slowdown—whether it is global or regional.

Every use case requires dashboards that can provide detailed data through a wide variety of visualizations, including choropleth maps, stack areas, pie charts and bar graphs. Ideally, a dashboard will enable teams to isolate dimensions, apply filters, and dive into data for deeper insights.

Unfortunately, not all dashboards can support this type of fast, flexible analysis. Many still utilize outdated technologies, which lack the flexibility, agility and scalability required to seamlessly explore data. Many were also designed without the urgency that today’s data requires—after all, in previous years, most uses for dashboards (such as internal reporting) were not time-sensitive.

Other dashboards are constrained by templates, which offer a finite array of widgets, tools and drill-down capabilities. These dashboards may feel unwieldy and sluggish, unsuited for a rapidly evolving situation like an application outage.

Requirements

To truly fulfill the promise of self-service, ad-hoc analytics across huge datasets, teams need to work in real-time—and so they need a database capable of timely responses.

When an application goes down, an SRE may not know what to look for and, hence, needs to quickly comb through lots of data. While some dashboards rely on workarounds for faster queries, such as pre-aggregations, precomputing, or rollups, this is not possible in this instance because the SRE simply won’t know what to look for. After all, they can’t necessarily predict what will go wrong, and even if they work on assumptions from previous failures, this current issue could be much different.

Therefore, dashboards must provide plenty of functionalities and visualizations. Users should be able to filter data by time, isolate variables like location, and zoom into specific time intervals with a few clicks. Dashboards should also accommodate diverse data types, including intricate parent-child relationships and nested columns.

Further, dashboards must offer a depth of insight. An online streaming media platform may need to assess user metrics (such as latency or load times) across different devices, operating systems, and regions to fine-tune performance. A cloud provider has to monitor their physical hardware for high temperatures, slowed network switches or devices, and other anomalies.

Because data is now so important to success, many more people across a company, including data scientists, product managers and external users, themselves require data-driven insights. In these situations, dashboards must handle the increased user traffic and query activity, maintaining fast responses even under load. This is especially important considering that a single-user operation (such as a zoom) will require multiple queries on the backend to execute.

A database also must scale seamlessly. If an organization’s environment generates millions of events an hour, that equates to billions of events per day or week—challenging for any database to ingest, store, analyze and query. In fact, many databases cannot successfully provide fast response times while managing large datasets and high query volume. As an example, transactional databases (OLTP) can often query rapidly but cannot execute analytics at scale, while analytical databases (OLAP) can analyze massive volumes of data but not at speed.

Apache Druid for Independent, Ad-hoc Data Exploration

This is where open source Apache Druid comes in. Combining the scale and advanced analytics of an OLAP database with the speed of an OLTP database, Druid offers swift, real-time data exploration.

Upon ingesting data, Druid makes it immediately available for querying and analysis, removing the need to first batch or aggregate the data in some way. In addition, Druid natively integrates with streaming technologies like Apache Kafka and Amazon Kinesis, removing the need for workarounds or connectors.

Druid powers interactive visualizations, providing millisecond response times, enabling more versatile exploration, expanding the range of available dimensions and filters, and even maintaining subsecond speeds in the face of surging user and query volumes.

Druid’s unique architecture also enables easy scaling. By separating key duties among separate node types—data nodes for storage, master nodes for data ingestion and availability, and query nodes for executing queries and returning results via the scatter/gather method—Druid ensures that nodes can be independently scaled based on need. Afterward, Druid also automatically rebalances traffic to ensure consistent performance.

Salesforce: A Druid Success Story

Salesforce pioneered the customer relationship management (CRM) space, serves 150,000 customers worldwide and earns billions in annual revenue.

The Edge Intelligence Team is the division of Salesforce that tackles the massive task of ingesting, processing, filtering, aggregating, and querying the entirety of their log data — anywhere from billions to trillions of lines daily. Each minute, Salesforce ingests 200 million metrics, while each day, Salesforce processes five billion daily events globally. In total, Salesforce accumulates dozens of petabytes of data in their transactional store, five petabytes of logs in their data centers, and almost 200 petabytes in their Hadoop storage.

Salesforce teams use Druid to unlock real-time insights into product performance and user experiences, diving into large datasets instantly. Anyone from engineers to account executives can query a wide variety of dimensions, filters and aggregations to better understand trends, troubleshoot any issues that arise and set strategies for the future.

By using Druid’s compaction abilities, Salesforce also decreased the number of Druid rows by 82%, leading to an overall reduction of their storage footprint by 47% and accelerating their query response times by 30%.

October 19, 2023
Atlassian Makes Compass IDP Generally Available

Atlassian this week made generally available a Compass developer experience platform through which DevOps teams can employ a software-as-a-service (SaaS) platform to centralize the management of the environments used to create code.

Taylor Pechacek, head of product for Compass at Atlassian, said Compass provides an instance of an internal developer platform (IDP) that is simpler for DevOps teams to set up and maintain.

At its core, Compass provides access to a software component catalog to track, search and discover the component and approved infrastructure that can be used to build and deploy applications.

In addition, templates enable developers to self-service environment setup using a set of policies that provide guardrails to ensure developers follow best practices for provisioning cloud resources and setting up pipelines.

Finally, Atlassian is making available scorecards based on the DevOps Research and Assessment (DORA) metrics defined by Google.

Compass is designed to be integrated with a wide range of continuous integration/continuous delivery (CI/CD) platforms and third-party observability, testing, collaboration and source code management tools, said Pechacek. The goal is to minimize context switching as much as possible for developers accessing the Compass platform, he added.

That approach also provides a level of consistency that both fosters collaboration and makes it simpler to onboard new members to a development team, noted Pechacek.

While IDPs as a concept have been around for some time, they are now central to any effort to embrace platform engineering methodologies to implement best DevOps practices at scale. The issue platform engineering teams need to come to terms with is determining whether they want to build and maintain an IDP versus invoking one provided as a SaaS application.

Regardless of approach, platform engineering teams are trying to strike a delicate balance between centralizing the management of development environments and the ability of developers to innovate by adding new tools to their development environment. Organizations clearly want developers to become more productive by spending less time managing development environments, but many of those same developers tend to jealously guard their prerogatives when it comes to the tools they prefer.

The challenge, of course, is that spending less time managing a development environment doesn’t necessarily mean more code will be written. All it does is afford developers the opportunity to write more code. The issue is that the writing of quality code only really occurs when developers are inspired, but at the very least, configuring a development environment shouldn’t stand in the way of creativity.

It’s not clear how many organizations are embracing platform engineering methodologies, but it’s apparent that more supervision is being brought to how applications are built and deployed within the context of a set of DevOps best practices. The challenge and the opportunity now is to make sure developers buy into, rather than resist, those efforts.

October 18, 2023