devopsy.kinsta.cloud

Tag: devops

  • Compuware and CollabNet VersionOne Announce Integration of ISPW and Continuum

    Partnership Enables Value Stream Management with Unified View of Cross-Platform Development and Deployment

    • Application Development and Delivery Professionals are under pressure to track the progress of code deliverables across the DevOps lifecycle based on their value to the business.
    • Large enterprises are additionally seeking increased visibility and control over agile development work across all platforms—from inception to delivery.
    • The integration of ISPW, Compuware’s DevOps-enabled source code management solution for the mainframe, and CollabNet VersionOne’s Continuum release delivery platform, will serve this customer need.

    LAS VEGAS, Oct. 23, 2018 (GLOBE NEWSWIRE) — Compuware and CollabNet VersionOne today announced the completed integration of their respective products, ISPW and Continuum. The companies previously announced they were partnering to help large enterprises unify their Value Stream Management (VSM) by incorporating their critical mainframe software development activities into their digital value chain. Both companies are at DevOps Enterprise Summit (DOES18), featuring the integration in Compuware booth 827 and CollabNet VersionOne booth 219.

    ISPW provides DevOps-enabled source code management (SCM) capabilities for mainframe code while CollabNet VersionOne’s Continuum tracks, automates and orchestrates software value streams across the DevOps lifecycle. Continuum’s visual interface helps users see and understand the progression of what is happening in the process of development.

    The initial integration will enable Application Development and Delivery managers to view and control the status of user stories moving through the mainframe software development pipeline from the same Continuum dashboard as they use to manage those pipelines across other platforms. This dashboard enables managers to quickly identify which stories are progressing on schedule, which have been delayed by defects, and which contain trailing commits.

    Large enterprises are relying increasingly on their core mainframe systems of record as essential back-end support for critical mobile and web front-end customer engagement. They must therefore orchestrate the delivery of new software across all platforms at the same speed and frequency to deliver value to customers.

    “As planned, our partnership with Compuware is already delivering capabilities that make it easier for mainframe users to scale Agile and DevOps. We are helping them connect their mainframe-based development activities into broader cross-platform efforts that can be incorporated into unified value streams,” said Flint Brenton, CEO of CollabNet VersionOne. “Value Stream Management keeps software aligned with business objectives, which is essential for today’s dynamic and continuous application development processes.”

    “Enterprises need greater automation, better visibility and a mainframe-inclusive DevOps tool-chain to increase the quality, velocity and efficiency with which they create new digital experiences for customers,” said Chris O’Malley, CEO of Compuware. “Our partnership with CollabNet VersionOne will provide such capabilities, empowering development and delivery teams to accelerate their DevOps journeys to better meet their customers’ digital demands.”

    About CollabNet VersionOne
    CollabNet VersionOne is the Enterprise Value Stream Management leader that accelerates high value software development and delivery, while improving quality and reducing risk. Our offerings provide global enterprise and government market leaders a cohesive solution, spanning idea through delivery, that enable them to capture, create, deliver and measure the flow of business value throughout their application development lifecycles. For more information, visit www.collabnet.com.

    Follow us on:

    Compuware Corporation
    Compuware empowers the world’s largest companies to excel in the digital economy by fully leveraging their high-value mainframe investments. We do this by delivering highly innovative solutions that uniquely enable IT professionals with mainstream skills to manage mainframe applications, data, and platform operations. Learn more at compuware.com.

  • Delphix and Datical Extend Global Partnership, Bring New DataOps Solutions to Modern Enterprises

     One-half of all Fortune 50 companies work with Delphix, Datical to deliver high quality software – faster and more securely – to close Data Velocity Gap

     REDWOOD CITY, Calif., October 23, 2018 – Delphix, the company accelerating innovation through DataOps, today announced it has extended its partnership with Datical, the leading provider of database release automation solutions, to bring new DataOps solutions to market that unlock the potential of data to drive business transformation. Already in use by  one-half of the Fortune 50, Delphix and Datical are driving the emerging DataOps ecosystem that will enable more Global 2000 companies to open up the flow of fast, secure data to move fast and compete in today’s Digital Economy.

    With worldwide enterprise data expected to grow to 163 zettabytes (ZB) by 2025 (Data Age 2025, IDC), software development and DevOps teams are drowning in data; unable to use it to deliver innovation that keeps pace with ever-changing consumer demands. Delphix and Datical offer proven solutions that eliminate delays through automation to achieve 100x faster high-quality software releases, 10x less cost and no production outages.

    “As large enterprises look to accelerate their digital businesses, two of the biggest challenges they face are the incredible volumes of data being generated and the related lack of velocity from their database layer,” said Stephen O’Grady, Principal Analyst with RedMonk. “While application development has been revolutionized by the DevOps phenomenon, database management has lagged behind. This issue is what’s created the demand for an equivalent ‘DataOps’ style movement, and is why Delphix and Datical are focusing on these needs so intently.”

    “Businesses are feeling the debilitating effects of data friction, challenging their ability to move fast and confidently compete in today’s digital economy,” said Tony Orlando, SVP of Global Field Operations, Delphix. “Through our partnership with Datical and the DataOps ecosystem at large, we are enabling the secure, continuous flow of data within the world’s most complex enterprises, empowering them to accelerate innovation, reduce costs, increase quality and lead in their markets for years to come.”  

    Modern Enterprises Rely on DataOps to Unlock the Potential of Data

     “Data remains locked in legacy systems, and is holding [CEOs] back,” said Salesforce CEO Marc Benioff in a recent earnings call. Over the last three years, Delphix and Datical have worked with some of the most important brands in the world to help them unlock data to accelerate software development and close the costly data velocity gap.

     Together, Delphix and Datical enable enterprises to dramatically accelerate software development and testing. The unique combination of these solutions reduces the amount of time it takes to provision production-quality data from days or weeks to just minutes, while also eliminating manual tasks, such as reviewing and validating database schema and logic changes. Integrating these solutions into a software development toolchain is at the heart of the DataOps approach that transforms how businesses deliver new releases.

    A Top 5 investment management firm based in the US (with over $2 trillion in assets under management) uses Delphix and Datical to increase the speed and quality of application releases that are the backbone of its digital transformation efforts. In a short period of time, the global brand achieved:

    • A 100% increase in rate of application deployments;
    • A 90% reduction in data provisioning times;
    • Dramatic improvement in quality from eliminating data-related defects.

    Pioneers in the DataOps movement, Delphix and Datical are at the forefront of building the platforms and technologies that support this emerging discipline. As Gartner recently recognized in its Hype Cycle for Data Management for 2018, DataOps is the next chapter in the DevOps story that will accelerate business and help create the modern data enterprise.

    “Software is reshaping all industries and the pace of change is accelerating. Companies must speed up innovation in order to survive in the digital era,” said Derek Hutson, CEO and President of Datical. “To achieve their goals of faster time-to-market, companies continue adopting faster, more iterative development methodologies to be able to change and adapt more quickly – but data has become the new constraint slowing them down. Datical and Delphix are in the perfect position to help enterprise leaders remove that constraint and modernize the way data is managed throughout the software delivery lifecycle.”

    About Delphix

    Delphix’s mission is to free companies from data friction and accelerate innovation. Fortune 100 companies use the Delphix Dynamic Data Platform to connect, virtualize, secure, and manage data in hybrid cloud environments. For more information visit www.delphix.com.

    Follow Delphix on Twitter, Facebook and LinkedIn.

    About Datical, Inc.

    Datical’s mission is to transform the way businesses build software so they can deliver better customer experiences faster. Datical solutions deliver the database release automation capabilities technology executives need to get the most out of their Agile, Cloud, DevOps and Continuous Delivery investments. With Datical database release automation organizations can shorten the time it takes to bring applications to market while eliminating the security vulnerabilities, costly errors, data loss and downtime often associated with current database deployment methods. For more information, visit www.datical.com, call 737-402-7187 or connect via @datical.

    Follow Datical on Twitter, Facebook and LinkedIn

  • How GitOps Can (Sometimes) Save DevOps

    How GitOps Can (Sometimes) Save DevOps

    A lot has been written about GitOps since its release by Weaveworks more than a year ago as a “single source of truth for the whole system” for using Git for Kubernetes deployments on the cloud.

    Aptly described by Alexis Richardson, Weaveworks co-founder and CEO, as “90% best practices and 10% cool new stuff,” GitOps largely offers a framework for DevOps CI/CD deployments on Kubernetes. This was something very often seen as missing among the many Kubernetes management distributions on offer—hence, its popularity.

    The tool is obviously not of use for organizations that have not adopted Kubernetes, but in many ways has enabled organizations’ DevOps to more efficiently take advantage of Kubernetes by running Git in a more disciplined and focused way.

    “The idea behind GitOps is to maximize the time developers spend on productive coding versus worrying about working with the latest source code, merging their saved code back into the repository and continuously supporting problems that come up during the application build process,” Torsten Volk, an analyst for Enterprise Management Associations (EMA), said.

    GitOps offers DevOps a way to use its version control capabilities to archive changes to application deployments in a way that allows them to be recalled at a later time. This can come in handy when an architecture fails and it is necessary to access and reinstate a previous version of an application deployment or update prior to the incident. All changes are tagged and archived in a way that each developer knows when and how their changes to an application were changed.

    “Ideally, each developer would check in their code and receive a link to the running application with his or her latest changes applied, as well as with all the other changes that were checked into the repository,” Volk said.

    The Adoption Checklist

    Volk offered the following elements he said DevOps should have in place before adopting GitOps:

    — All infrastructure and application configurations, including databases, middleware and machine learning models, need to be defined as code, Volk said.

    “Everyone must accept the basic concept that code is only complete when all components for a successful build are checked into Git,” he said. “CLI-driven, ad hoc deployments are no longer acceptable.”

    — The DevOps group has to be able to detect changes to application code that previously might have been able to bypass Git. There must be thus be a process in place to bring these changes back under GitOps control, Volk said.

    “The IT Ops/DevOps teams need to make operational changes to the app exclusively via changes to the code checked into Git,” he said. “This means that tasks such as scaling, failover, creating maintenance windows, etc. as well as urgent troubleshooting tasks have to run through Git. The latter is especially tricky, as the enterprise will not accept longer mean time to detect (MTTD) and mean time to repair (MTTR) due to GitOps.”

    — Declarative management practices have to be used in Dev and Ops. “This has proven to be a critical roadblock,” Volk said. “Kubernetes developers, as part of Dev and Ops, still often prefer the control of imperative management over the promise of defining their desired end state and letting the software deal with how to get there.”

    For DevOps teams that have the prerequisites in place as described above to use Git to run Kubernetes deployments on the cloud, then GitOps deserves at least a look. However, for those organizations that adopt GitOps for the right reasons, it is not designed as a one-size-fits-all solution and organizations will need to modify it to fit their needs. Its implementation, for example, will almost certainly different compared to how Weaveworks described its functioning after using it for two years to run Kubernetes and Prometheus telemetry databases on a cloud platform.

    DevOps will, thus, need to be actively involved in GitOps deployment and maintenance to make sure it meets organizations’ individualized needs. When first describing GitOps, Weaveworks’ Richardson wrote: “Fair warning: this is what works for us, and dear reader, you may disagree.”

    — B. Cameron Gain

  • The Truth Behind DevOps on the Cloud

    The Truth Behind DevOps on the Cloud

    DevOps on the cloud may seem simple enough, but in reality it can be very complex

    The ultimate goal of a DevOps administrator is to make sure that everything is running properly and seamlessly—so much so that management is convinced that the DevOps administrator is playing solitaire all day. To that end, a successful DevOps team should have their systems automated such that the level of friction they impose on the organization is minimal.

    Leveraging a cloud environment makes it look easy—almost too easy, making it look as though all DevOps needs to do is press a button. IT can be deceived into thinking that the level of complexity in DevOps on the cloud is trivial, when in reality it is not.

    Running DevOps on the cloud is like renting an apartment in someone else’s building: Even though you don’t own the infrastructure, you’ll have to deal with any issues that may arise. Similarly to a neighbor who’s too noisy, you may experience different levels of performance from the same servers with the same capacity at different times of the day because someone else is running on the same hardware.

    There also are security threats that don’t come into play when running on your own hardware. If the same physical machine that runs your most important computations is also being used by Mr. Nefarious, he can use all sorts of tricks and hacks to gain access to valuable data. The Meltdown Inspector attacks is one example—it impacted all cloud providers with severe fleetwide performance impact, sometimes exceeding 10 percent, and required expensive mitigation steps.

    Making Sense of Your Gold Mine of Data

    The wealth of information provided to system administrators is awe-inspiring, terrifying and frustrating all at once. There is so much information available that, in theory, we should be able to troubleshoot and fix any and all issues. Unfortunately, making sense of the vast amount of information available and nailing down the critical pieces of information necessary can be extremely complex.

    The flat network topology typically expected of an on-premises data center is drastically more complex on the cloud. Beyond just VPN and DMZ, there are multiple isolated networks, containers running VMs and running on segmented networks going through NAT with multiple levels of routing and dispatch at play. Trying to troubleshoot in such an environment can be an exercise in frustration, because you don’t own the entire network.

    Certain tools and abilities you might take for granted in your own data center are not available, such as Tcpdump, a low-level tool to monitor network packets. For example, in VPN packet loss scenario, your cloud VPN server is shared with other clients, so  you won’t be able to capture packets to troubleshoot the connection issues.

    Also in managed services—in particular databases—the debug endpoints and tools that often are used in other environments to troubleshoot are not available, to avoid exposing data from other users. For example, you cannot get the full logs from the server to figure out what it is doing. That can hide critical goings that are too precious to ignore in production.

    Evaluating Cloud Providers

    Since it’s inevitable that problems will occur, one of the important considerations for selecting a cloud provider is the level and quality of support they offer. It’s important to go beyond simple SLAs and evaluate the actual quality and effectiveness of the support you can receive.

    Take, for instance, a situation where a machine would freeze, effectively at random, for long periods of time. All monitoring and metrics have told a simple story. The machine was up, had plenty of resources and capacity to spare, but simply would not respond every now and then for any connection attempt whatsoever. Troubleshooting involved stripping the problem to its core and being able to reproduce this in a minimal fashion, at which point one would contact the cloud provider support to try and understand what is going on. The actual problem turned out to be network port exhaustion in the NAT router, which caused a remote server to reset TCP connections, leading to long timeouts that froze the system. This particular problem took days to resolve and required a major change in the deployment topology.

    Running DevOps on the cloud is not an easy task, and the cloud itself is not a silver bullet. Rather, it’s a way to democratize hardware and operation resources and work effectively at scale. If you can manage the new challenges, you can enjoy a new level of efficiency.

    — Oren Eini

  • Security Through ‘Legisurity’: Impacts of SB-327

    Security Through ‘Legisurity’: Impacts of SB-327

    With more devices connecting to the internet every day, security through obscurity should have died a long time ago. Still, high-profile cases of devices with mass-produced default login credentials and easily cracked crypto keys persist. As a result, say hello to security through “legisurity”: California SB-327—“Information privacy: connected devices”—became law Sept. 28. How will this impact DevOps teams?

    What SB-327 Does and Doesn’t Do

    Here’s a quick look at what SB-327 does, starting with its timing and scope. The provisions go into effect Jan. 1, 2020, giving teams a little more than a year to prepare compliant connected device designs. The legislation defines a connected device as one “… capable of connecting to the Internet, directly or indirectly, and that is assigned an Internet Protocol address or Bluetooth address.” Authentication is “… a method of verifying authority of a user, process, or device to access resources in an information system.”

    The operative passage in SB-327 calls for a “reasonable security feature … appropriate to the nature and function of the device.” (Before you get too excited, it explicitly shuts the door for individuals suing manufacturers over non-compliance.) With implementation left open, the legal standard is very clear. Reasonableness is a device with features meeting either of two requirements:

    “(1) The preprogrammed password is unique to each device manufactured,

    “(2) The device contains a security feature that requires a user to generate a new means of authentication before access is granted to the device for the first time.”

    Also written into SB-327 is a list of things it does not do. The law does not extend these requirements to third-party software a user installs, and it does not force retailers or software providers to review compliance of a device. It doesn’t apply to devices sold only to federal agencies or the military, and it yields to HIPAA requirements. It does not permit manufacturers to hinder law enforcement agency access to a device with these required features.

    What Should DevOps Do Now?

    As with most first attempts at legislation, the unwritten implications of SB-327 may be even more interesting. Here are a few things I see changing for DevOps teams securing devices and infrastructure:

    • SB-327 only applies to devices sold or offered for sale in California. However, setting up California-only design efforts, supply chains and customer support would be costly. (For $20,000 cars? Sure. For $200 devices? Not so much.) Practically, I see these becoming de facto requirements nationwide by the 2020 rollout. Companies should start now to be ready.
    • If you’re a device manufacturer, you should add both requirements into your minimum viable product baseline, if you haven’t already. You’ll obviously need a strategy for assigning and tracking default passwords and a crypto key authentication method for setup connection. Customer support procedures and documentation also need to change. Consider using one-time programmable non-volatile memory or PUFs (physically unclonable functions) to establish a unique device identity.
    • On the deployment side, you’ll need added tracking for unique default passwords in your device provisioning database. Your customer support teams will need device-specific information at their fingertips to get new or factory-reset devices back online quickly. Crypto key authentication methods will be a more important criterion for device evaluation and selection.
    • If you don’t have a good UX person and an experienced pen tester on your team, hire or contract those roles right now. Simplicity with security is always going to be the right answer.

    More Laws Ahead?

    Will the debut of “legisurity” satisfy lawmakers and stem connected device breaches? My guess is this is only the beginning; SB-327 does nothing about requiring stronger passwords or thwarting more sophisticated protocol-based cyberattacks. The main point of SB-327 doesn’t seem about securing a single device, although it should help; rather, it’s trying to prevent rapid compromise of an entire population of that device. (Pro tip: whatever you do, don’t use a device’s MAC address in a simplistic algorithm, or you’ll end up like this sad, not-so-smart lock story. Cracking one device cracked them all.)

    For the more curious, here’s the entire text of SB-327, and some other analysis on legislative proposals at the federal level. What are your thoughts on SB-327, how you see it affecting DevOps, and if legislation is the right path for securing connected devices?

    — Don Dingee

  • Electric Cloud Introduces ElectricFlow 8.5, Unveils Industry-First Support for Both SaaS and On-Premises Delivery

    Flexible Enterprise SaaS Delivery Model, Kanban-Style Pipeline View and Object Tagging with a Continuous Integration Dashboard Allow Teams of Any Size to Start Fast and Adapt Easily

    SAN JOSE, Calif., Oct. 18, 2018 – Electric Cloud (https://electric-cloud.com/), the leader in Adaptive Release Orchestration and Continuous Delivery, today announced a new SaaS consumption option for the industry’s first Enterprise SaaS and on-premises Application Release Orchestration (ARO) offering, ElectricFlow. ElectricFlow now includes a Kanban-style pipeline view, object tagging for custom reporting and enhanced searchability, and CI dashboarding capabilities to track and analyze build processes, failures and successes.

     For more information about ElectricFlow and to download the free Community Edition, please visit: (https://electric-cloud.com/products/electricflow). 

    ElectricFlow SaaS allows DevOps teams of all sizes to get up and running without incurring on-premise implementation costs, and gives teams developing cloud-native or serverless applications a way to keep all their development pipelines and tools in the cloud. Most enterprises also have on-premise assets that they need to manage. ElectricFlow SaaS seamlessly connects to on-premise servers and tools to enable deployment automation, pipeline and environment management, and release orchestration inside and outside of the firewall.

    “More and more, large enterprises are exploring ways to include cloud-native applications and tool chains in their on-premises digital transformation projects,” said Torsten Volk, managing director at Enterprise Management Associates. “It’s important for DevOps tools vendors to make their offerings available on a variety of platforms to support those efforts. I’m glad to see Electric Cloud getting ahead of the market with this new release of ElectricFlow.”

     DevOps Your Way

    The new Kanban view in ElectricFlow 8.5 provides a familiar, easy-to-use management and execution view of pipelines, stages and individual tasks on one screen. Connected directly to the automation and orchestration, users can approve and observe release progress with real-time pipeline and application deployment status. 

    “Moving to a DevOps culture and adjusting to a new pace, new processes, and new toolsets has been a challenge for many of our people,” said Gary McKay, Scrum Master at global toll-free registry service Somos. “ElectricFlow’s new Kanban view coupled with comprehensive pipeline-as-code means everyone on the team gets exactly the kind of working environment they need to help us accelerate our digital transformation.”

    New tagging functionality on all objects allows release managers and product owners to quickly and easily create more granular, real-world reports. Universal data segmentation is now possible across any object type (applications, pipeline stages, environments, etc.) and replaces code-based report creation.

    New CI Dashboarding creates data-driven visibility into CI processes and bottlenecks, so that architects and application teams have clear understanding of builds from within the release pipeline. ElectricFlow works with all CI tools, including Jenkins, Bamboo and Git to give teams unprecedented insight into the value, cost and risk associated with every release, eliminating the need for difficult, extensive custom coding with tools like Hygeia.

    “This release further solidifies our position as the de facto standard for ARO by doubling down on giving customers the flexibility to unleash the power of ElectricFlow for anyone – wherever and however they need it,” said Carmine Napolitano, CEO of Electric Cloud. “This allows us to meet our customers where they are, and simply ‘plug-in’ unified governance and orchestration across an ever-changing array of tools, stacks, and processes. Adapting to change more quickly than your competition is a competitive advantage, and ElectricFlow eliminates much of the anxiety associated with releasing software when adopting new processes or trying new technologies.”

    Share This: .@electriccloud’s Adaptive Release Orchestration Platform, ElectricFlow, Now Available as #SaaS with new #Kanban view, Tagging functionality and #CI Dashboard to let you Do #DevOps Your Way! https://electric-cloud.com/blog/2018/10/electricflow-8-5-devops-your-way/ 

    About Electric Cloud, Inc.
    Electric Cloud’s Adaptive Release Orchestration platform helps organizations like E*TRADE, HPE, Intel and Lockheed Martin confidently release new applications and adapt to change at any speed demanded by the business, with the analytics and insight to measure, track, and improve their results.  

  • Solving the ‘Automagical’ Apps Automation Problem

    Solving the ‘Automagical’ Apps Automation Problem

    The DevOps Revolution started with a handful of  simple principles. A key one was that systems should be managed by code, not by humans, to eliminate complexity and ensure better security, compliance and scalability. To manage compute infrastructure at any scale, the process should be automated with logic captured in code. And, ideally, automation also would apply as you move up to the stack to middleware and applications layers.

    Unfortunately, it’s not that simple. In fact, automating applications turns out to be way more complex than automating server deployments. For this reason, we need a new approach to managing applications and infrastructure. Managing servers and lower-level environments could be simpler, but existing bottom-up methods work fine. Managing applications, however, must come from the top down. That is, applications should come with everything they need to be successful: configuration, packaging, discovery, scaling requirements and more. And that just isn’t how we have historically built software. Applications are built on layers and layers of dependencies that connect all the way down to the system level. So how do we break the mold and make an application “self contained?

    Thousands of Apps with Millions of Dependency Permutations

    By automating the Linux and Windows server ecosystem and a small number of  derivative operating systems we can cover roughly 95 percent of operating systems, and the series of functions that are common across them. The complexity arises when we try to lay that matrix upon the array of applications that exist in the real world of deployment and production. In smaller companies, you usually have at least a dozen applications and often more. In larger companies, you are dealing with hundreds or thousands of applications.

    This melange spans bespoke apps running in legacy languages, proprietary third-party software and more modern open source software. There are literally millions of permutations that must be automated across dependencies, along with security and compliance regimes. In the world of Kubernetes, for example, each container requires its own Sidecar that contains configuration and management information. Kubernetes Sidecars, however, don’t have robust automation capabilities and quickly sprawl out into a mass of human-defined management instructions that can easily conflict or break down under stress.

    The upshot of this? Today we can write a small amount of code that can effectively manage hundreds of thousands of servers for Facebook. This is true to basically infinite scale. But to automate management of Facebook’s software has injected orders of magnitude more complexity.

    Fear of Application Complexity Killing Cloud Migrations

    This dichotomy has led infrastructure automation to go sideways; organizations have adopted  inauthentic automation coupled with departures from the real ideals of DevOps and agile methodologies. Many companies just aren’t realizing the true benefits of infrastructure automation because they are doing something that is needlessly complicated and may have characteristics of automation but in fact is even more complicated. Thus, they have lost the value of the word and merely exchanged one complexity for another.

    Similarly, many application development teams are doing what they call “ad hoc” infrastructure automation as DevOps teams avert their eyes and pray. Translation? A roll-your-own mishmash of tools to manage containers or VMs, applications and all the dependencies from the top to the bottom of the application runtime environment. What this really translates into is a human executing an ever-changing set of narrow paths against an individual server—a recipe for chaos. This chaos is why the majority of our customers believe infrastructure automation is a critical step to achieve agility but a smaller subset have fully embraced it.

    It’s also a major reason why many companies that would love to migrate to the cloud have not yet done so. Managing the repackaging and movement—the “lift-and-shift”—of all their applications is a daunting prospect that they rightly believe will be far more difficult than simply switching from bare-metal or VMs to instances in AWS or Azure or Google Compute Engine.

    Now, we don’t need to throw the baby out with the bathwater. The automation of operating systems and VMs and containers works just fine and is actually a tremendous improvement on past efforts. In fact, we could easily operate there with a smaller set of features and capabilities than we have to date. But applications must be managed from the top-down to accommodate the inherent complexity of managing so many more moving parts.

    Enter The New Paradigm: Top-Down Application Management

    Top-down application management works in a fundamentally different way, Rather than rigid configurations and choices about what an application should do, modern application management tools for DevOps teams should ask what an application needs to succeed and do its job well across the entire life cycle, from development to production to retirement. These tools should automatically discover changes in dependencies of an application stack, from front end to middleware to database and load-balancing and security.

    If there are changes in these dependencies or other critical factors, then those tools need to automatically rebuild and test the applications, and have them ready to deploy either as a canary or directly into production if the changes are minor. For example, if a new SSL patch is published and a DevOps team updates its underlying infrastructure, then the application management toolchain should recognize which applications need to make which changes, and how that might affect both upstream and downstream dependencies. Ultimately, the toolchain should quietly and seamlessly fix all affected applications with the patch but avoid service interruption.

    Once the success criteria of the applications are defined by a human DevOps person, then the application configuration and management tools’ job is ensure that criteria is always met. Collectively, this is a new type of packaging that captures the best of all possible worlds, merging configuration and packaging into intelligent application management that can scale and flex as needed.

    By capturing the dependencies of configuration and the strictures of traditional packaging, top-down application management methodologies and tools allow a CIO to move an aged C or C++ back-office application into a cloud-native environment just as easily as a passel of Node.js microservices backstopping a mobile application, and have them both run with the same level of continuity and flexibility. Which is really what everyone wants—a safer, easier and better way to run applications without having to worry about the underlying infrastructure.

    Conclusion

    In a sense, then, the DevOps Revolution is entering a new phase. As microservices continue to slice and dice the old functionality of monolithic apps into numerous smaller apps, the challenge of managing this app proliferation will only grow—and become more complicated.

    The way we thought about DevOps tools in the days when cloud computing was relatively young was understandably naïve and not yet informed by the vast application cornucopia we would face in subsequent years magnified by the reality that legacy applications decades old would continue to run in corporate data centers until their code keels over. Now we know and we can design an automation methodology and practices that map to our modern realities. We can do this by giving infrastructure the simple rules and tools it needs, while giving apps more leeway to be themselves and succeed regardless of what is happening at lower levels of the stack.

    — Corey Scobie

  • You’re Thinking Software Wrong

    You’re Thinking Software Wrong

    Like any industry, DevOps fosters its own unique culture. And like any culture it resists change, it has inertia. “This is how it has always been done” is an incredible force to reckon with.

    Even though the realities of DevOps have changed dramatically in recent years with the advent of new tools and services, many ideas that have been ingrained in the culture will probably still be with us for ages. Some of these ideas are not only wrong, they border on being dangerous. One such fallacy is the sacred rule cherished by development teams and system administrators: “NEVER deploy on a Friday afternoon.” That is, unless you want to spend your entire weekend fixing bugs. A more pernicious example is the division of labor between us, humans with “root access” to machines, and the robots, the tools teams create to make their processes less manual. These days you’ll still find many development groups that delegate the menial tasks to automation, but keep the most dangerous, infrequent tasks to human operators. These are two prime examples of “thinking software wrong.”

    These ideas are not only incorrect, they can be crippling to an organization, causing delays in software development and leaving the company vulnerable to data breaches, for no good reason. These fallacies are what we refer to as “broken culture”—what happens when culture is deeply misaligned with the interests of the organization, the individuals who compose it and its stakeholders. It usually manifests when historic cultural norms prevent members from embracing new and better ways of doing things.

    Fear of Friday deployments is our first example of broken culture. This idea has taken a strong hold in DevOps culture because developers haven’t trusted their own software and systems enough, and they don’t want to spend their Friday nights (or the whole weekend) debugging a production problem. Fair enough, but there is a better way.

    Let’s look at another example: the Equifax breach. This incident—which cost the company billions of dollars and continues to haunt consumers to this day—happened because a certain procedure mandated that a specific piece of software must always be updated by a human. Since humans are prone to error, this lone simple procedure (designed and supported by years of a broken DevOps culture) resulted in a devastating data breach. Once again, there is a better way.

    Deploying is not the same activity it was a decade ago. Advanced automation tools are drastically streamlining deployments, significantly reducing the chance of error and speeding the time it takes to identify and correct any errors that do occur. This means companies can deploy any day of the week and any time of the day, worry free—and can let the deployment date be dictated by what is best for the company and its customers, not the weekend plans of developers.

    The same concept applies when it comes to updating software. A better way is to leverage the automation tools that are now readily available, and allow organizations to have robust management of compliance requirements without human error or delay.

    In addition to embracing better tools, here are a few other ways companies and developers can rethink software for the better:

    • Software must be valued as more than just the cost it takes to create it, or to replace it. Code should be an asset, not a liability.
    • Software should be thought of as constantly changing—a dynamic system with which people, machines and processes interact continuously. When this never-ending process is supported by the right tools, deployments are easier and managing and updating software is seamless and secure.
    • Code by itself is useless. It needs infrastructure to build it, and data to create something meaningful. Therefore, if you only test the code, but are not managing your infrastructure in the same way, you’re not really testing the software system.
    • Companies can dictate the cadence at which they create new software, but they do not have this same luxury when it comes to fixing broken software. Identifying and fixing problems must be done as soon as possible.
    • Every new update or fix must be version controlled so that everything is auditable. That includes changes to infrastructure configurations!

    Once companies truly understand these fundamental truths about software and embrace the modern tools available to them, they can start to shed the aspects of DevOps culture that are limiting or damaging.

    Companies can deploy new apps and features on Fridays, confidently and without issue. Companies can even rebuild an entire infrastructure from scratch, in minutes, while applying all security updates for every single service and library—and be completely sure nothing will break. If you can’t do that—or don’t believe you can—you’re thinking software wrong.

    — Ori Pekelman

  • DevOps and Containers: A Network Divided

    DevOps and Containers: A Network Divided

    The growth of DevOps and containers has created some interesting split responsibility scenarios. Not that this is necessarily a bad thing, but it is something that bears watching and fixing when it becomes problematic.

    Today’s topic is networking. Plenty of bits have been spewed about the situation, but it persists and is even getting more complex as architectures diversify.

    Specifically, today’s topic is about the dual responsibility built into today’s networking. Generally speaking, network operations staff are still managing the complex hardware and software environments that deliver packets to your systems. What they do and how they do it is changing—but at a much slower pace than areas fully pulled into DevOps. There are many reasons for this, and I won’t delve into them here, but the truth is that managing a production network and dealing with hardware vendors bring unique responsibilities and constraints that aren’t there in a software world.

    That’s experience talking, from a developer who learned network administration. Software is easier and more adaptable, and that is directly reflected in the impact of DevOps.

    But we have drifted into a world in which that infrastructure is still mission-critical and still has those same constraints, while the software layer of networking—driven primarily by containers—has grown up in parallel with a different set of people often responsible for it. And that software network can be mighty complex. I shudder to think about what the iptables look like on some—maybe hundreds—of your machines.

    Cloud has much the same effect, but since cloud instances often are publicly exposed, they tend to have networking people involved to make sure routing is correct and verifiable, and to help InfoSec people determine that the instance is as locked down as possible (and we can all name companies that wish this were true).

    Still, the complexity of cloud spawned companies including CloudCoreo (now a part of VMware), which automate a bunch of network verification and security checks just because there are so many points that require touching. So cloud isn’t the nirvana of shared responsibility either. Yet.

    As DevOps and container adoption has grown, it is natural that responsibility for container/container management networking would have spilled over into the development or application administration side. After all, once a subnet is configured for a cluster and software is managing the subnet, which is integrated with the rest of container management, it feels right to have the Dev/deploy team manage it.

    And it might be. Like all things DevOps, this is a continual improvement exercise. Having someone who knows exactly what is going on between pods/clusters/networks is important, though. Breaks can cause outages that cost money. Knowledge is the key to fixing breaks and container networking is generally super-complex. Even if you are providing standard IP addresses on a corporate subnet to each container, container management networking is still complex and can still break.

    So if you haven’t already, get the networking team involved in inter-/intracontainer communications. Make certain you know what is going where and what should go where.

    You won’t regret it, when things inevitably break that cannot be repaired by dropping the instance and spawning a new one to replace it.

    — Don Macvittie

  • Building Great Cloud Security Guardrails

    Building Great Cloud Security Guardrails

    Security guardrails are an incredible way to keep our cloud deployments safer without slowing things down. Taking this structured approach will minimize friction while increasing protection.

    When I first started working hands-on in cloud (AWS) around nine years ago, I quickly realized both the power and risks of just how darn fast and agile cloud could be. Being, on occasion, an optimist, I decided this capability was a significant upgrade to our security capabilities. Sure, I could accidentally spin up internet-exposed servers, but in exchange I gained levels of central control and visibility unheard of in traditional infrastructure.

    One of the most powerful new capabilities is the ability to leverage cloud APIs and automation to build automated security guardrails without having to spend a gazillion dollars for a big box that actually delivers as promised, but also adds tremendous friction and slows things down. Guardrails are automations that constantly watch your deployments, find deviations from desired baselines and can even automatically remediate issues. Yet, even cloud-native guardrails can become problematic if they aren’t designed properly. Having both built a fair few myself and worked with many organizations as they built out their cloud security programs, here are some of the lessons I’ve learned along the way and some code to show you how it works.

    Building Awesome Guardrails

    My first guardrails were a bit of a hodgepodge in terms of structure and capabilities. And practically speaking, there are a lot of ways to attack the problems. Even today I’ll take different approaches depending on the particular challenge in front of me, but most of my guardrails (including operational ones) tend to follow a consistent pattern:

    • Define the problem: I like to start by defining the business outcome we want, and then move onto the technical problem. I used to start with a narrow technical problem to solve but found that didn’t always lead me to the right outcome. For example, is the problem that we want to stop all internet-facing port 22 access, or is the real outcome we don’t want to expose internet-facing admin servers to the open internet and we want to use a guardrail to restrict to our known corporate IPs instead? This helps prevent a guardrail from becoming a blocker.
    • Set the scope: Multiple cloud accounts (or subscriptions/projects for you Azure/GCP people) are now the standard. The first step is to determine how broad a guardrail you want, which will also define some of the technical specifications. For example, if you want it to apply to multiple accounts in Amazon Web Services, that affects where you run it from and how you manage the required IAM privileges.
    • Build or pick the deployment model: There are a lot of technical specifics regarding where and how to run your guardrail. Today’s example will run in AWS within a single account, but if you manage a larger, enterprise-class deployment you will need to look at how and when to balance centralized and localized guardrails. More often than not, I prefer a serverless approach (FaaS with other supporting PaaS), but I’ve also found longer-running guardrails often need containers as well. Thanks to API gateways, it isn’t bad to build deployments that can support both.
    • Define filters: It’s awesome that you can automatically close all internet-facing security groups with 20 to 40 lines of code—right until you break a production application. Filters allow you to tune your guardrail for a particular environment or project’s needs. Basic filters might work on a simple resource tag, while more complex filters might evaluate a combination of include and exclude rules for greater flexibility. This can, for example, treat development and production environments differently from the same guardrail.
    • Determine triggers: We have a rich range of trigger options on the major cloud platforms today. The main categories are time-based assessments and event-based triggers. I tend to use both, with event-based for more deterministic events (an S3 bucket policy was changed) and time-based sweeps to catch conditions I might miss purely on an event trigger.
    • Perform the analysis: This is the meat of our guardrail. Based on the trigger, collect the information we need and, in combination with our scope and filters, make a decision. For example, if we find port 22 open to the internet, if it is a Dev environment we might decide to update the security group to restrict to known corporate IP addresses. If it is in Prod we might delete the rule completely unless it was implemented through a CloudFormation template that went through an approval process.
    • Send notifications: Even when you have a completely automated guardrail, you likely want a parallel notification. These can go straight to email/text/Slack or drive into a ticketing system. For security guardrails, I also recommend sending them to your SIEM or other security alert/management system. If you’re fancy, you can turn these into the first step of a chatOps process.
    • Take actions: Actions are what separates guardrails from all the annoying alerts of the past. Based on the analysis and filters, actions can be fully automated remediations or request human intervention. Actions also can follow different paths based on the conditions to provide the best options for that situation. The key is that even if a human is called to make a decision, the decision should be a simple click to run the action, saving the person from having to fix things manually. Unfortunately, we still sometimes need human beings to make a decision. Then again, the alternative is bowing down to our robotic overlords. It’s a trade-off, I suppose.
    • Validate and log: Don’t assume the action actually did anything. Rechecking is vital, as is logging and sending the results as a second notification to the ticketing system/SIEM/etc. I fully recommend sending both incident open and close notifications to the destination system, even if the guardrail fixed everything within seconds. This keeps a consistent record and also allows you to do nifty things such as measure your mean resolution time and compare automated versus manual.

    Running a Demonstration Guardrail

    One of my favorite guardrails to demonstrate is automatically reversing security group changes in AWS. Here’s how to set it up for yourself and run my demonstration code to see what it looks like:

    • The first step is to make sure you don’t start running this in production or anyplace else where you might break things before you actually tune the filters. If you do, it isn’t my fault. The desired outcome in this case is ensuring we don’t have network exposures due to unapproved security group changes on designated resources.
    • The scope of this guardrail is a single local account, with a corresponding deployment model. We will use CloudTrail, CloudWatch Rules, Lambda and SNS.
    • There are three prep steps to configure your account before building the guardrail:
      • Log into the AWS console and set up a destination for notifications. If you haven’t done this before, you want to use SNS (Simple Notification Service). Create a topic for security alerts and a subscription to wherever you want to send the alerts. I recommend starting with a simple email subscription.
      • Make sure you have CloudTrail turned on and streaming to CloudWatch. For you newbies, CloudTrail records (almost) all the API calls on your account and we need this for our trigger. Sending the logs to CloudWatch allows us to build near real-time triggers in CloudWatch based on events such as the API call we will look for.
      • Create an IAM Role for Lambda to use. It will need the rights to AuthorizeSecurityGroupIngress, DescribeSecurityGroups and RevokeSecurityGroupIngress. To save logs, it also needs PutLogEvents, CreateLogStream and CreateLogGroup. These are also in a JSON template in the project repository for cut and paste.
    • With that in place, create a new Lambda function (Python 2.7) and load in our code located at https://github.com/disruptops/RevertSecurityGroups.
    • Now go to CloudWatch Rules and create a new rule. For the trigger, use the following:
      • Service Name: CloudTrail
      • Event Type: AWS API Call via CloudTrail
      • Specific Operations: AuthorizeSecurityGroupIngress
    • Then set two targets: your Lambda Function and your SNS topic for security alerts. This will revet the changes and simultaneously send you a notification.
    Screenshot of configuring the trigger in CloudWatch

    This should now revert any new ingress rules added to a security group. If you play with the code you can also use this for egress rules. This is just demo code and there are edge cases it will miss.

    The code also includes a number of filter options, including:

    • Restricting to a region or VPC.
    • Restricting based on a tag or a specific security group ID.
    • Allowing the change if it comes from a designated IAM user, and reversing if it was originated by anyone else.

    It takes the action automatically; if you want to manually review first you would modify the code to send a notification instead. We have some alternative examples posted at https://github.com/disruptops/demos-tools. These includes some samples that protect S3 and also show how to embed notifications in the Lambda itself for more-flexibility.

    Hopefully this gives you some ideas to refine your own guardrails to dramatically improve security while still moving blazingly fast.

    — Rich Mogull