devopsy.kinsta.cloud

Tag: DevOps

  • How to Empower DevSecOps in a Complex Multi-Cloud Landscape

    How to Empower DevSecOps in a Complex Multi-Cloud Landscape

    Organizations are embracing cloud-based application delivery for speed of delivery and scale. However, the proliferation of multi-cloud systems to access and maximize the cloud’s capabilities is driving complexity in IT environments. Many organizations do not consider how multi-cloud adoption is expanding their attack surface—and the risk is growing: The 2023 CrowdStrike Global Threat Report found cloud exploitation cases grew by 95% year-over-year and cases involving threat actors targeting cloud environments were up 288% from a year prior.

    The expansion of cloud infrastructure and services has increased the pressure on organizations to protect assets within their multi-cloud environments. To alleviate this burden, more businesses are turning to DevSecOps, a practice that incorporates security at an earlier point in the development life cycle. This practice of shifting left helps improve the quality and reduces risk of applications, as well as makes the delivery of these applications more efficient since any potential issues are found earlier in the development process.

    DevSecOps is being adopted rapidly by organizations today to address the expanding array of threats and increasingly sophisticated attacks targeting multi-cloud environments. However, securing the cloud can be tricky due to its dynamic nature and multi-cloud makes shift left security even more complex. For DevSecOps to truly be effective, here are three things that organizations should consider doing.

    1. Strive for Complete Visibility in a Multi-Cloud Environment

    It is important that DevSecOps teams have a comprehensive understanding of their multi-cloud environment. Without it, they will not be able to identify a variety of security risks including vulnerabilities, privileged credentials, misconfigurations and other threats that may leave their organization’s cloud assets vulnerable.

    The best way to get full visibility across multi-cloud environments is through a cloud security posture management (CSPM) tool. With a CSPM tool, DevSecOps teams can identify misconfigurations and proactively detect threats. In addition, DevSecOps teams should also use a cloud infrastructure entitlement management tool (CIEM) to manage identities across all of their cloud infrastructure resources. The primary goal of a CIEM tool is to mitigate the risk that comes from the unintentional and unchecked granting of excessive permissions of user credentials.

    2. Visibility and Compliance Do Not Guarantee Security

    Organizations need to develop a comprehensive security strategy to mitigate issues within the development process. This may include implementing tools, automation and standards to enable developers to follow the desired security behavior. These approaches reduce developer friction, as well as diminish the likelihood that unsafe or default configurations will be used. Historically, DevOps teams have pushed back on integrating security since their concern is that it slows down the development lifecycle. However, DevSecOps is gaining traction in recent years because organizations are realizing integrating security sooner in the development lifecycle is more efficient than reactively patching problems as they arise.

    A tool that DevSecOps teams should look to leverage is cloud workload protection (CWP) because it offers breach protection for workloads, containers and Kubernetes, while enabling them to continue to build, run and secure applications with speed and confidence.

    3. Create Consistency With Security Tools

    Ideally, organizations should opt for a single platform that consolidates capabilities such as CSPM, CWP and CIEM. The reason for consolidation is to eliminate security gaps from having multiple security solutions. As a result, DevSecOps teams should turn to cloud-native application protection platform (CNAPP) tools to create more consistency within the multi-cloud environment.

    Securing multi-cloud environments can be complex. By empowering DevSecOps teams using these best practices, organizations will be able to move faster than today’s adversaries and have the agility to quickly adapt to present and future cybersecurity challenges in their multi-cloud environments.

  • How to Avoid Risk When Using Multiple Low-Code Platforms

    How to Avoid Risk When Using Multiple Low-Code Platforms

    Organizations are still increasing their use of low-code/no-code (LCNC). But this adoption isn’t always consolidated around one tool—frequently, multiple low-code/no-code platforms are used under the same roof. In fact, Gartner predicts that by 2024, 75% of large enterprises will be using at least four low-code development tools for both IT application development and citizen development initiatives.

    Utilizing multiple low-code/no-code platforms simultaneously introduces some security concerns. This is partially due to a lack of standard policies to govern different platforms in the same way. Low-code platforms are often incompatible with traditional IT security processes, like vulnerability scanning, manual code reviews and runtime security monitoring. Organizations going all-in on citizen development need safe ways to use low-code/no-code platforms. So, how can they safely enable this innovation without hindering productivity?

    I recently met with Michael Bargury, CTO and co-founder, Zenity, about securing LCNC systems. Bargury, who oversees the OWASP Low-Code Top Ten list, has significant insight into both the benefits and drawbacks of low-code/no-code platforms. Below, we’ll review the potential risks of using multiple low-code platforms and consider ways to mitigate these concerns.

    Why Companies Have Multiple Low-Code Platforms

    Interest in low-code has been rising, and low-code/no-code adoption continues to accelerate across enterprises. According to Bargury, large enterprises often don’t get a choice in the matter—low-code tools are arising organically, and it’s challenging to keep a lid on their proliferation.

    One way this occurs is that low-code becomes embedded into the SaaS platforms everyone already uses. Software platforms integral to business operations, like Salesforce, Microsoft Office and ServiceNow, have arguably mutated into low-code platforms, making them more like cloud providers than SaaS vendors, said Bargury.

    The other way low-code platforms emerge is through bottom-up transitions, says Bargury. This could be a specific knowledge worker working in a subset of the business to automate a domain-specific operation or integrate various tools. Then, there are the platforms, like OutSystems, Mendix or Mulesoft that specifically target professional developers.

    Organizations going all-in on citizen development may build a center of excellence and focus efforts on a single platform. And low-code platforms targeting developers will sometimes encourage more consolidation. Yet, “different platforms solve different use cases better,” explained Bargury. As a result, you will usually have a few converging in the same enterprises.

    Potential Security Concerns of Multiple Low-Code Platforms

    The emergence of low-code is similar to the bring your own device (BYOD) movement. IT and security leaders fought to avoid it at one point, but nowadays, everyone does it. So, what are the security implications if low-code becomes similarly ubiquitous? Well, no matter how it’s generated, low-code-bred programs are still applications susceptible to the same logical problems. And unfortunately, compared to your typical traditional code development life cycle, low-code development is fraught with discrepancies, says Bargury.

    Immature CI/CD Pipelines

    Some platforms do not offer the kind of mature CI/CD process one would expect from a software development platform. For example, there may be no way to create tests or separate your environments. Since there’s no coherent and consistent way to push code from development into production, you lack a way to apply secure development across platforms. “You don’t have a way to ensure every piece of software is up to your standards,” said Bargury.

    Opaque Underyling Codebases

    In traditional software development, you apply your own security reviews to ensure applications don’t contain risks. This includes shift-left scanning, security gates, manual reviews to catch mistakes early on and runtime scanning to catch unknown vulnerabilities in production. But there aren’t always parallels in low-code development. Low-code solutions typically have their own way of doing things with fewer standards, and they avoid exposing the underlying code and Git, said Bargury.

    Lack of Visibility

    As such, with low-code obfuscation, you lose a degree of visibility into how an application runs. Logs may be insufficient or wholly inaccessible. It’s also tricky—or at times impossible—to set up runtime monitoring, let alone scan various platforms. This means that low-code users can’t leverage powerful observability tools to identify errors and remediate security holes.

    Lack of Unified Policies

    Finally, establishing common authorization and permissions across multiple low-code development platforms is another hurdle. Since low-code vendors have their proprietary building blocks, you’re unable to leverage something like OPA or Kyverno to create and unify policies. Lacking shared policies across tools makes it challenging to verify that best practices have been implemented, says Bargury.

    How to Secure Multi-LCNC

    Low-code can grant much creative potential. But, knowing the above risks, what should IT be doing to reduce risks around low-code—especially when using multiple platforms? Well, Bargury shared several steps to help cybersecurity leaders start safely addressing the emergence of increasingly low-code capabilities.

    1. Take low-code development under the application security umbrella. First, low-code must be brought under the security umbrella, if it isn’t already. “It should be part of their mandate,” said Bargury. Shift the responsibility to professionals well-trained in resolving threats.
    2. Understand your surface area. As a basic security tenant, you can’t secure what you don’t know. Thus, Bargury recommends increasing your visibility into each low-code platform by collecting logs when possible and centralizing your analysis.
    3. Create automated guardrails. Next, it’s critical to identify your risks and set guardrails for your low-code users—especially when citizen development is involved. The OWASP framework provides a good overview of common risks and mitigation advice across robotic process automation (RPA), integration and low-code. In general, it’s important to design permissions with the rule of least privilege in mind. This can help reduce the chance of vulnerabilities like LCNC-SEC-03: Data Leakage and Unexpected Consequences and LCNC-SEC-01: Account Impersonation.
    4. Address gaps within the development life cycle (SDLC). Lastly, low-code platforms often provide the building blocks to secure applications, but it’s still up to you to ensure they are safely used. Cybersecurity teams should continually audit the low-code SDLC to ensure holes don’t persist.

    You’re Responsible for Securing Low-Code Development

    Low-code development platforms aren’t 100% safe from harm. It’s not out of the question for hackers to abuse low-code platforms and turn them against their owners. For example, Microsoft’s Detection and Response Team (DART) reported how a hacker group used entry to a low-code tool to remain hidden within an organization for six months, using credential stuffing to access an administrative account. By creating automations, they were able to use internal discovery tools and exfiltrate secrets and data that could have been used for privilege escalation.

    As you can see, there are legitimate concerns for low-code, especially at scale. However, security responsibility is not always that transparent. To date, there isn’t an equivalent of a shared cloud responsibility model in the realm of low-code development. As such, at the end of the day, you need to take the reins to own the applications that you build and ensure everything is secure.

  • Noname Security Looks to Shift API Security Testing Left

    Noname Security Looks to Shift API Security Testing Left

    Noname Security today made generally available an update to a tool for testing application programming interface (API) security that promises to make it easier for DevOps teams to ensure APIs are secure.

    Filip Verloy, a field CTO for Noname Security, said Active Testing V2 is purposely designed to make it simpler to integrate API testing within a larger DevSecOps workflow.

    Version 2 of Active Testing, for example, provides additional integrations with continuous integration/continuous deployment (CI/CD) platforms along with automated API discovery tools.

    Other capabilities included in the latest release are tools to automate API response inspection, troubleshooting, classification and vulnerability discovery.

    Finally, Noname Security has also added support for GraphQL-based APIs alongside REST APIs.

    In total, there are now more than 160 API security tests that can be employed on an API testing tool that Noname Security developed to enable it to build Active Testing, said Verloy. In effect, Active Testing is an extension of that core API testing tool, he noted.

    The biggest issue DevSecOps teams encounter when testing API security is that most legacy tools are not designed to provide any visibility into the business logic used to construct APIs. As such, there is a need for API security tools that can be easily integrated within DevSecOps workflows to discover issues before APIs are exposed in a production environment, he noted.

    While a lot of progress has been made toward securing software supply chains, APIs have generally not received enough attention. Developers typically create APIs, but they don’t typically assume responsibility for managing and securing them. As the number of APIs within organizations continues to proliferate, there is a growing need to centralize API management and security.

    Historically, developers have assumed that cybersecurity teams were making sure APIs are secure. Cybersecurity teams, conversely, assumed that the developers that created APIs would be responsible for securing them. DevSecOps best practices make it possible to now extend responsibility for security further left toward developers in a way that enables cybersecurity teams to define the policies and controls that should be used, noted Verloy.

    It may a while before every organization masters the all the nuances of DevSecOps, but as pending legislation that requires organizations to better secure software supply chains becomes law it’s only a matter of time before organizations revisit API security—if they have not done so already.

    In the meantime, cybercriminals have already discovered that insecure APIs provide a convenient mechanism for exfiltrating data that they can then sell or hold for ransom. The number of rogue and zombie APIs that are unknown or that have been forgotten has become a major cybersecurity issue. Ideally, DevSecOps teams would not only test the security of APIs before they are deployed but would also address a significant amount of technical API security debt that has already accumulated.

    One way or another, API security is going to become a significant issue in the weeks and months ahead. The issue now is how to address it before it gets any worse.

  • Most Productive Days of the Week for Developers – Rob Zuber, CircleCI

    Most Productive Days of the Week for Developers – Rob Zuber, CircleCI

    CircleCI CTO Rob Zuber explains why application developers are more productive on certain days of the week than others depending on the time of year and what impact that has on DevOps workflows.

  • Developers’ Role in Protecting Privacy

    Developers’ Role in Protecting Privacy

    We live in a world where we always expect things to happen seamlessly. Every time somebody walks into a café and signs in to the free WiFi and opens their Instagram or TikTok to stay up-to-date with trends and friends or retrieves a discount from a retailer by joining their newsletter circulation, rarely do they stop to think about how that convenience and value is derived.

    Technology is the key enabler working away in the background, and developers and DevOps teams are making that happen. Underneath the slick interfaces of the apps and websites that provide these highly efficient and personalized experiences is a complex machine that relies on data to fuel it, built by developers and delivered by DevOps teams.

    In many ways, data is a two-way exchange, and those end-users that are aware of this give and take appear to be happy with how the relationship works.

    Indeed, several surveys in recent years suggested that the majority of consumers are happy to share their data with companies if it improves their experience. Furthermore, a significant minority are still comfortable if they have no control over who sees their data, there is no clarity around how data is stored and used, even if the organization in question has a poor reputation for data privacy.

    The Privacy Balancing Act

    So, if most people already accept data trade-offs in some shape or form, should developers still care about building applications and solutions that guarantee data privacy?

    The short answer is yes, but as developers, that involves a tricky balancing act.

    Although sharing data has become commonplace in exchange for benefits and value, consumers are becoming more aware of privacy issues. Take the EU’s General Data Protection Regulations (GDPR) as an example. Over the past five years, awareness has more than doubled in notable European markets such as the UK, Spain, Germany, the Netherlands and France.

    Meanwhile, there is also commercial pressure, as employers rely on developers to innovate to remain profitable. At the same time, customers expect brands to be responsible with their data, and failure to do so at the expense of trying to commercialize a new application could be detrimental. Indeed, while the pandemic may have ushered in significant changes and altered consumers’ attitudes toward data privacy, end users remain unwavering about the importance of security.

    Maintaining this balancing act is becoming increasingly complex to achieve. However, the question of data privacy is becoming a key business priority, and that means developers have a big opportunity to show their commercial value to their organizations. Indeed, companies cannot afford to get the balance wrong, and developers are central to creating the perfect formula.

    In fact, the privacy conundrum has been around for a long time.

    When the internet was created, privacy was not considered an issue because all content accessed online was deemed to be public. Nothing was encrypted; the introduction of email complicated things, not least because anybody on the internet could, theoretically, intercept personal data and read private messages.

    This prompted the emergence of the cypherpunks, a group of cryptographers who began to contemplate what the future of digital privacy might and should look like. They started to create the tools they believed the world needed to safeguard data privacy–today, their mission is carried on by developers who value user privacy.

    Why Developers Need to Be Familiar With FHE

    However, securing the transmission of data is not enough, as many breaches still occur because data isn’t encrypted during processing.

    So, in a world where more and more data needs to be shared to deliver the sort of experiences consumers expect, is it inevitable that privacy and security will be compromised as the volume of data exchanges grows? Not necessarily.

    Fully homomorphic encryption (FHE) offers a solution, allowing developers to build applications and run services without needing to see or secure the underlying data. This is because FHE enables data to be processed blindly without having to decrypt it at any stage. FHE offers a way for developers to maintain that all-important balancing act between maintaining privacy and offering value to both business and customer.

    In FHE, to guarantee a user’s data privacy, all data is encrypted by the user with their own secret key. Users of a particular service, such as an online medical portal, will use a secret key to send their encrypted data to a server where blind processing takes place. The result is also encrypted and sent back, the user again using their secret key to decrypt the information.

    What this means is that, because data is encrypted end-to-end, the company providing the service can do so in a way that keeps the same customer experience, even though they cannot see the data that is being shared and sent back. From a security perspective, this is also critical. Governments, attackers and service providers cannot see the exchanged information because they do not possess the key–they also cannot break it, as the encryption used in FHE can resist attempts made even by powerful quantum computers.

    Privacy can also be protected during larger-scale data exchanges that underpin collaboration between organizations and nation-states on numerous strategic programs, with each contributor able to add their value knowing that their intellectual property and sensitive information is not compromised.

    Looking ahead, if developers can embed FHE into their practices on a broad scale, then a key milestone for privacy is within reach. Rather than being viewed as an objective relating to digital interactions, privacy can be treated as a naturally occurring by-product of every application, piece of software and service that the developer community creates.

  • EP 37: The Rings of Software: DevOps, SRE and Platform Engineering

    EP 37: The Rings of Software: DevOps, SRE and Platform Engineering

    What does the rise of platform engineering bring to the table for DevOps and SRE? Or is it simply the next evolutionary step of DevOps? Despite some confusion about mission and responsibility, there are key differences between these roles, and there’s a need for all three across the SDLC. Platform engineers, SREs and DevOps teams have shared goals and methodologies, and they all play a role in optimizing the software creation and delivery process. Host Mitch Ashley is joined by our panel of experts: Tim Banks (Dell), Adrianna Villela (Lightstep), Bryan Cole (Tricentis) and Taylor Pechacek (Atlassian) as they explore the differences between DevOps, platform engineering and SRE, the most common misconceptions and how to mitigate them and how platform engineering enables DevOps at scale.

  • Revolutionizing DevOps Culture Transformations With AI-Engineered Tools

    Revolutionizing DevOps Culture Transformations With AI-Engineered Tools

    In my recent article Revolutionizing the Nine Pillars of DevOps with AI-Engineered Tools, I explained that DevOps thrives in a culture of openness, transparency and shared responsibility. It’s about breaking down the silos that can exist between different teams (like development, operations, security and QA) and promoting effective communication and collaboration. AI-powered tools can improve collaboration through smart recommendations, fostering more effective communication and knowledge sharing.

    Collaboration and Communication: AI-powered chatbots can facilitate quick and easy communication across teams, enabling faster decision-making and problem-solving. AI-powered project management tools can improve visibility into tasks and dependencies, promoting better coordination among team members.

    Continuous Learning and Improvement: AI-based monitoring tools can help teams learn about system performance in real-time, identifying trends and patterns that can inform better decision-making. AI can also help in creating personalized learning experiences, identifying skills gaps and suggesting relevant training resources, promoting a culture of continuous learning.

    Shared Responsibility and Ownership: AI can improve visibility into the system’s state, helping all team members understand how their work contributes to overall system performance. This can promote a sense of shared ownership and responsibility. AI can also assist in automating the allocation of tasks based on an individual’s skills and availability, promoting a more equitable distribution of responsibilities.

    Automation and Efficiency: AI is pivotal in automating routine tasks, such as testing, deployment and monitoring. This can significantly improve efficiency and allow teams to focus on strategic, high-value tasks. AI can also help automate the detection and resolution of common issues, reducing downtime and improving system reliability.

    Feedback and Adaptability: AI-powered analytics tools can provide real-time feedback on system performance, enabling teams to quickly adapt their approach as needed. AI can also help in analyzing customer feedback and identifying key themes and sentiments that can inform product development.

    Risk-Taking and Innovation: AI can help simulate the potential impacts of different decisions, allowing teams to take calculated risks. AI can also drive innovation by helping teams identify new opportunities, predict trends and develop more creative solutions.

    Roadmap to AI-Assisted DevOps Culture

    A practical roadmap for transforming an organization’s DevOps culture using AI-engineered tools may look something like this:

    1. Define Goals and Evaluate the Current State: Identify what you want to achieve by integrating AI into your DevOps practices. This could be improving deployment times, reducing errors, increasing productivity and more. Evaluate your current DevOps practices and tools to understand where AI could bring the most value.

    2. Identify AI Opportunities and Select Tools: Look for areas where AI can enhance collaboration and communication. This could involve AI-powered chatbots for team communication or project management tools with AI capabilities. Identify opportunities for AI to support continuous learning and improvement. AI-powered analytics and monitoring tools can provide valuable insights for this. Determine where AI can help promote shared responsibility and ownership. AI-enabled visibility tools can assist here, too.

    Choose areas for AI-powered automation. This could include tasks like testing, deployment and issue resolution. Find avenues where AI can support feedback and adaptability. Tools that analyze system performance or customer feedback could be helpful here. Pinpoint where AI can assist in risk-taking and innovation. AI-powered simulation or predictive analytics tools could be useful.

    3. Implement AI Tools Gradually: Start by integrating AI tools into one or two areas of your DevOps process. Monitor the impact to ensure you’re driving the desired improvements. Gradually expand the use of AI tools to other areas of your DevOps practices.

    4. Train Teams on AI Tools: Provide adequate training to your teams so they can use these new AI tools effectively. Encourage a culture of learning and experimentation to help teams become comfortable with AI.

    5. Monitor and Adapt: Regularly monitor the impact of these AI tools on your DevOps practices. Are they improving speed, reducing errors, enhancing collaboration, etc.? Be prepared to adapt your approach based on the feedback and results you’re seeing. The integration of AI is a journey, not a one-time event.

    6. Continuously Improve: Continually look for new ways that AI can enhance your DevOps practices. AI is a rapidly evolving field, and new opportunities are emerging all the time. Remember that this roadmap is iterative and adaptive. AI integration should be seen as a part of the broader DevOps goal of continuous improvement.

    Benefits for Organizations, Teams and Individuals

    With the integration of AI-engineered tools into DevOps processes, organizations can expect significant improvements in efficiency, speed and quality. AI automation can drastically reduce manual, repetitive tasks, thereby lowering the chance of errors and freeing up resources for more strategic endeavors. This can lead to cost savings and a better allocation of resources. Additionally, AI can enable faster decision-making and problem-solving, accelerating the software development life cycle and making the organization more competitive. AI can also enhance product quality by providing valuable insights and analytics, enabling continual improvement.

    For Teams: AI can facilitate improved collaboration and communication, making workflows more seamless and efficient. AI-powered project management tools can provide better visibility into tasks, dependencies and progress, enabling smoother coordination. AI can also automate routine tasks, freeing up team members to focus on more challenging, high-value work. Additionally, AI can provide real-time feedback and insights, enabling teams to continuously learn and improve their processes. These improvements can lead to higher job satisfaction and productivity.

    For Individuals: Individuals stand to gain from AI integration in several ways. AI-powered tools can support skill development by helping individuals learn new technologies and best practices. Real-time feedback and insights can also foster personal improvement. Automation of routine tasks allows individuals to focus more on creative and strategic aspects of their work, leading to a more engaging and satisfying job experience. Furthermore, being part of an AI-driven DevOps environment can open opportunities for individuals to work on cutting-edge technologies and innovative projects, enhancing their career prospects.

    Measuring Success

    Choosing the right metrics is crucial for measuring the progress of your DevOps culture transformation with AI-engineered tools. Here are some important metrics to consider:

    Deployment frequency: This is a measure of how often new features, bug fixes or updates are deployed. An increase in deployment frequency can be a sign that AI tools are improving efficiency.
    Lead time for changes: This metric refers to the amount of time it takes from code being committed to it being deployed. AI tools can help to reduce this time by automating testing and deployment processes.
    Change failure rate: This is the percentage of changes that result in a failure or that require a rollback or hotfix. AI can help reduce this rate by improving testing accuracy and identifying potential issues earlier in the development process.
    Mean-time-to-recovery (MTTR): This is the average time it takes to recover from a failure. AI tools can help to reduce MTTR by quickly identifying the cause of failures and automating the recovery process.
    Customer satisfaction: AI can help improve the quality and reliability of the product, leading to increased customer satisfaction. Customer satisfaction can be measured using surveys or net promoter score (NPS).
    Efficiency metrics: Look at how much manual, repetitive work has been reduced by automation. This could be measured in terms of man-hours saved.
    Learning and development: Monitor the upskilling of your teams to using AI tools and technologies. This could be tracked through training completion rates, skill assessments, etc.
    Innovation rate: AI can free up teams to focus on innovation. Measure the number of new ideas or features being developed over time.

    Summary

    To transform DevOps culture with AI tools, organizations must set clear goals, identify AI opportunities, gradually implement those tools and monitor their impact. Training teams to use AI tools effectively and fostering a culture of continuous learning and improvement will enable this transformation and help DevOps teams thrive. This integration can significantly enhance efficiency, decision-making speed and product quality at an organizational level.

    On a team level, AI can streamline collaboration, automate routine tasks and provide real-time insights for continuous process improvement. For individuals, this transformation provides opportunities for skill development, personal growth and more satisfying job experiences. Monitoring progress with metrics like deployment frequency, lead time for changes, change failure rate, mean-time-to-recovery (MTTR), customer satisfaction, efficiency, learning and development and innovation rate ensures the transformation is delivering the expected benefits and contributing to continuous improvement.

  • Four Ways to Modernize Your Enterprise Monoliths

    Four Ways to Modernize Your Enterprise Monoliths

    In a perfect world, every application would run like Netflix–by which I mean every application would be cloud-native, microservices-based, containerized and, in all other senses of the word, modern. Unfortunately, not every application can be transformed in these ways. Some of the most important enterprise applications–like those powered by SAP or Oracle–don’t lend themselves to architectural transformations simply because the vendors don’t support that strategy. As much as organizations might wish that they could re-architect legacy apps from the ground up, the reality is that they just can’t.

    Fortunately, though, all hope is not lost if you have an enterprise app that you can’t containerize or “microservices-ize.” Even if you can’t “break down your monolith”–to quote the standard advice you’ll find if you read articles about how to modernize legacy apps–there are a variety of practical steps you can take to make legacy, monolithic applications look and feel (in most respects) like modern, cloud-native apps.

    Here are four ways to make your enterprise monolithic apps more cloud-friendly, even if they’re not technically cloud-native.

    Use Cloud Automation to Modernize Legacy Application Management

    Most enterprise apps are compatible with the cloud in the sense that you can host them on cloud-based virtual machine (VM) instances using services like Amazon EC2.

    By taking advantage of cloud hosting for enterprise apps, organizations can also leverage cloud automation tools, such as autoscaling policies and rules that automatically start or stop VMs. In this way, it becomes possible to deploy enterprise apps in a highly scalable, automated way. You get something that resembles Kubernetes-based orchestration, except your apps don’t actually run in containers on top of Kubernetes. They’re just monoliths that benefit from cloud automation tools.

    Deploy Load Balancers to Optimize Performance

    Load balancers, which automatically manage network traffic flowing to and from applications, are another critical tool that organizations can easily leverage when they migrate legacy applications to the cloud.

    Although configuring load balancers for enterprise apps hosted in the cloud is not a strict requirement, doing so can substantially improve application performance and reliability because load balancers help to ensure that requests are directed at application instances best able to handle them. They prevent situations where one instance of your application is overwhelmed with traffic while another sits idle.

    Again, this is something that platforms like Kubernetes also do for microservices apps. But, as I’ve explained, you don’t actually need your app to run as microservices to take advantage of load balancing. You can do it with monolithic legacy apps, too.

    Adopt an Everything-as-Code Strategy

    When your app lives in the cloud and is managed by cloud automation tools, it becomes possible to manage most aspects of application operations via code. Specifically, you can:

    ● Use infrastructure-as-code (IaC) tools to automate infrastructure provisioning and setup.
    ● Automate application configuration management using code rather than applying configuration changes manually.
    ● Codify operational processes, such as application updates and backups, instead of relying on manual operations.

    Practices like these, which enable a strategy known as everything-as-code, significantly reduce the amount of time and manual effort that engineers spend managing applications. They also provide consistency and minimize the risk of errors due to human oversight. And there’s no reason why you can’t leverage these practices for legacy apps as well as modern ones.

    Integrate Legacy Apps With Modern Toolchains

    If your enterprise app is hosted on-premises, you probably manage it using tools and processes designed solely for it. But when you move an enterprise app to the cloud, it becomes possible to manage the application using the same integrated toolchain you deploy for other applications and processes.

    You can, for example, take advantage of cloud-based monitoring tools to continuously collect data about application performance and generate automated alerts when something is wrong. Likewise, you can regularly create image-based backups of your application.

    This toolchain-based operational strategy is another way to streamline enterprise application management, even in situations where you can’t rewrite or containerize the application itself.

    Conclusion: Getting More From Legacy Apps

    Just because you have a legacy enterprise app that you can’t transform into microservices doesn’t mean you have to settle for legacy management and operational practices. Modern automation tools combined with modern operational strategies like everything-as-code make it possible to achieve an enterprise application management experience that closely parallels that of fully modern, cloud-native apps, regardless of which architecture your app actually uses.

  • The Tragedy of the Data Commons: Balancing Innovation With Risk

    The Tragedy of the Data Commons: Balancing Innovation With Risk

    Clean drinking water and fisheries are classic examples of the “tragedy of the commons,” the problem in economics and ecology where an individual is incentivized to consume as much of a shared resource as possible at the expense of the larger society. With fisheries, for example, a logical fisherman would like to make as many catches as possible. However, this can ruin the environment for everyone, as overfishing can endanger an entire species and even threatens whole ecosystems. The modern IT environment has given us a new case to consider related to the tragedy of the commons: Data.

    Data, much like drinking water, is a critical resource. Developers need to use this data and, logically, they want to use it as much as possible to build innovative applications and products. However, too much data flowing around a modern distributed enterprise environment carries risks to the larger company.

    The data developers use often includes customers’ personal data like names and birth dates. The risks of this information being damaged or stolen can be catastrophic. This is where database administrators (DBAs), who are tasked with managing and securing this data, come in. There is an inherent tension between developers, who want as much access as possible, and DBAs, who want to limit that access for security and privacy reasons. How data is used within an enterprise is critical, because it must be used to create new solutions and be kept secure. An overly open data policy could lead to breaches and leaks, while an overly restrictive policy limits innovation.

    Thankfully, there is a solution that is helping ease this tension. Observability can automatically analyze massive amounts of information across a business’s entire IT environment. Users can use it to pinpoint the causes of outages or performance issues and receive actionable insights to resolve these problems quickly. Modern observability solutions even use AI to predict and proactively prevent problems before they occur. DBAs and developers can use observability to strike a balance while boosting innovation and improving productivity. Here are the three biggest ways that observability can solve the tragedy of the data commons and ease the tension between DBAs and developers.

    Ensuring Data is Secure

    One of the biggest concerns that DBAs have regarding data access is security. Sensitive production data are no longer stored exclusively on a company’s on-premise servers, where DBAs have full visibility into how it is being used and who has access. Instead, it is now stored across various private and public clouds as well as on-premises, which has reduced visibility for DBAs.

    This sensitive data is the same information that developers often need to build and launch applications in a continuous delivery model, where developers build software in short cycles so it can be released at any time. This involves working across different clouds, including AWS and Azure, while leveraging complicated cloud-enabled technologies like microservices and Docker containers. When data is deployed across multiple clouds, the attack surface increases while visibility decreases, leading to new security concerns.

    Observability provides DBAs with a better understanding of how data is being used by developer teams so they can more easily ensure that the data remains secure and protected. Additionally, many modern IT environments deploy threat intelligence solutions that observe actions and detect threat signals to help predict security incidents and respond before they happen. Some threat intelligence applications are powered by AI, which can automatically gather and organize complex log data from the entire network. This data is then compared with an out-of-the-box threat database, helping to detect suspicious activity early. Observability and threat intelligence working in tandem provide a comprehensive overview to help relieve some of the security concerns for DBAs and create a more secure environment for data to be shared between teams.

    Remaining Data Compliant

    Ensuring that data is compliant is top of mind for every organization. Compliance regulations such as GDPR, CCPA and HIPAA carry severe penalties for data leaks, meaning that companies must maintain the integrity and availability of sensitive and regulated data.

    Ensuring compliance while managing requests for information from developers working across geographies and clouds can be a daunting task for DBAs. Not only must they remain compliant, but many businesses must demonstrate continuous compliance, which is only possible using complex, dynamic and automated systems.

    Observability allows teams to clearly see and follow the complex flow of data as it moves between DBAs, developers and other teams. Additionally, observability can be deployed to help ensure that any sensitive data that is required to be anonymized is done so before it is shared. DBAs can further use observability to track the lineage of data as it moves between teams to monitor that it is not changed or transformed in any way that risks a compliance violation. Making compliance a regular part of the IT environment should not be viewed as a burden. In fact, it provides many benefits, including improving data quality, which can be used to further innovate and build better products.

    Making Teams More Productive

    Observability has shown it can create harmony between DBAs and developers by helping mitigate security and compliance concerns. Not only is it addressing these issues, but it is being used to make each team more productive on their own.

    For DBAs, this means gaining additional insights that help ensure applications and services are running properly. Observability collects information from across hybrid and multi-cloud environments and presents it in a single view to allow DBAs to respond quickly and resolve issues. When DBAs gain the insights that observability provides, they can better manage critical data and anticipate future problems before they arise.

    Observability is key to helping developers build applications as productively as possible. The ongoing shift left has necessitated the use of observability as a way to gain single-pane-of-glass visibility into the build process. Developers are also provided insights into the performance of web applications and can move quickly to address any problems. Finally, observability is improving the speed at which teams can innovate, as it provides live code profiling that can be used to predict and prevent future problems.

    Observability provides a real solution to ensure that data is not a resource that falls victim to the tragedy of commons, where its use can harm an entire organization. Instead, it is powering teams to use data securely to work more productively and build innovative applications.

  • Avoiding the Pain of a ‘Resume-Driven Architecture’

    Avoiding the Pain of a ‘Resume-Driven Architecture’

    Successful new technologies follow a predictable arc, from initial innovation to technical gold rush and, ultimately, to mass market adoption. As a consequence, they achieve strong standardization and productization. Being highly attuned to where a specific field lies on the maturity arc is how you yield the greatest benefits to a business. But what happens if your developers are not thinking about this, too? You end up with what I call a “resume-driven architecture,” and it can be highly costly to your business.

    A resume-driven architecture occurs when the interests of developers lead them to designs that no longer align with maximized impacts and outcomes for the organization. Often, the developer clings to a technology that provides them a greater level of control and, at least initially, a higher salary. Meanwhile, the organization gets an architecture that only a handful of people know how to manage and maintain, limiting the available talent pool and hindering future innovation.

    To understand why this happens and how to avoid it, it’s helpful to look at this technology arc in more detail. It’s a pattern that has played out up and down the stack, with technologies like big data platforms (now dominated by Snowflake and Databricks), Kubernetes and even the broader move to the cloud. Failing to grasp this concept of a resume-driven architecture can increase costs and hold back innovation.

    Here’s how this technology arc typically plays out, using Apache Spark as an example:

    • Stage One: A technology breakthrough like Spark emerges, providing a significant advantage to early movers.
    • Stage Two: Other companies see the benefits, and the technology receives widespread interest. Specialists are in high demand and command big salaries. Many adjacent professions naturally invest in becoming Spark experts.
    • Stage Three: The technology matures and reaches a level of standardization with an ecosystem of tools and services around it. Specialized knowledge, such as how to tune your Spark job parameters, becomes less valuable.
    • Stage Four: Mass convergence, productization and simplification occurs. As examples, Databricks becomes the primary Spark vendor and launches SQL warehouses for easier data processing. Snowflake launches Snowpark for Python as a simpler model than classic PySpark jobs. Nobody worries about tuning most parameters anymore as the underlying technology becomes part of the stack: Powerful but almost invisible.

    It’s important for businesses to take advantage of this productization stage when it comes along. There’s no sense in investing resources in a bespoke architecture if it’s not providing you with any differentiation—especially when competitors are achieving the same outcome with fewer resources.

    Moreover, getting stuck in a Stage Two mindset when the field moves on to Stage Three (or, worse, Stage Four) and cuts you off from the next wave of innovation. Subsequent technology breakthroughs often build on top of—and interoperate with—the previous technology layers. If you’re stuck with a custom architecture when the industry has moved on, you can miss out on the next wave of innovation and fall further behind competitors.

    The problem is, developers who’ve made a great career from implementing custom architectures are often reluctant to move on. After all, mastery of that previous innovation is what got them to where they are. They may not have realized that the world has progressed around them, or they may worry a standardized architecture with managed services and off-the-shelf products undermines their power and autonomy. This is how organizations fall victim to a resume-driven architecture. I’ve observed this several times when speaking with engineering leaders, and it’s become more common with the rapid innovation happening around data.

    As engineers, our value is in building things and, ideally, in solving innovative, artisanal problems. But this identity and desire can confuse what we want from our own jobs with what our team needs from us. When standard patterns and standard products emerge, we need to adopt them because that’s the way to maximize our impact and ultimately free our time and energy to move further “up stack” and have an even greater impact.

    Developers who would rather stick to a custom architecture often argue that they can squeeze out greater performance, but this is rarely a justification. You may end up with an app that’s 10% faster, but rarely does that come close to making up for the time and energy invested, the loss of flexibility and, ultimately, being held hostage by your own technology stack.

    So how does an organization avoid the trap of a resume-driven architecture? Strong leadership is key. Look critically at the business and ask which areas of your stack are truly differentiators because it’s only those areas that may—just may—justify a unique architecture. If you do go that route, it needs to provide huge benefits to the business to be worthwhile.

    For example, “data” is not in itself strategic to you; every company is now a data company, and the vast majority of things you do with it—how you source, store, process and distribute data—is no different from what everyone else is doing. What does matter are the data products that fuel your business and, most importantly, how fast you can build them.

    That’s why, as a developer or engineer, bespoke stacks are not your friend. Instead, architect your platform in a way that allows other developers to onboard quickly; that allows you to upgrade and swap out components easily over time and that doesn’t leave you held hostage to a developer whose main interest is retaining their value by sticking with an unnecessarily complex architecture.