devopsy.kinsta.cloud

Category: DevOps Culture

  • A Developer’s Task: Minimizing Challenges Created by Fragmentation

    A Developer’s Task: Minimizing Challenges Created by Fragmentation

    Over the past decade, there has been a wellspring of apps, services and platforms to help developers build custom solutions for businesses. Some are great, some not so much. Regardless of quality, the quantity of options and each one’s varying level of sophistication has made it difficult for developers to separate the wheat from the chaff.

    A recent survey from IDC supports this notion. It contends that the current landscape of software development languages and frameworks is highly fragmented. Having too many options—as opposed to a single, unified environment—leads to holes and disconnects in the development process, not to mention the potential for significant complications in the long-term support of today’s apps.

    Here are some ways developers can navigate the challenges created by this high level of fragmentation. 

    Don’t Be Afraid of Low-Code (or No-Code) Solutions

    The IDC survey found that less than half of developers—44%—have used low-code development tools professionally at one point or another. As one would surmise, low-code programming brings with it a host of advantages, such as easy-to-use and intuitive interfaces, and the ability to deliver applications faster. 

    Some low-code platforms also enable developers to build applications that can be used on any device while also reducing dependency on outsourced IT. A built-in script builder can allow even relatively inexperienced developers to turn complex business logic into functional workflows—such as automating a repetitive task—within a matter of minutes. Even after apps are launched, workflows can be updated and customized to fit specific business needs nearly instantly.

    Achieve Scale with Serverless Applications

    When building on a serverless platform, developers prevent infrastructure maintenance from inhibiting applications from scaling globally. This also allows developers to build mobile apps without managing the backend infrastructure. Cloud computing using public clouds typically involves renting virtualized servers across the technology stack. But when looking to scale, the onus is on developers to deal with the operating system and database while also managing necessary updates. This can lead to a work-intensive process on the app infrastructure level.

    A serverless infrastructure, on the other hand, abstracts databases and file servers, shifting focus to the app logic level. This allows developers to build and manage microservice-based apps that integrate with other services, making it easy to scale applications.

    Make Things Easy on Your Team

    According to the IDC survey, just 20% of developers claim they are “extremely familiar” with containers and microservices. With such expertise hard to find, using flexible tools that allow teams to quickly deploy code in scalable, compartmentalized components is vital.

    Organizations should aim to help their developers focus more on designing top-class user experiences of serverless applications, rather than spending (or wasting) time and resources on setting up and maintaining components on the server side. This gives developers the ability to focus on doing what they love—developing—as opposed to other tedious and mundane activities such as manually managing servers and storage solutions, or tweaking the underlying infrastructure of their microservices.

    Of course, making things easier for developers means deploying a powerful tool built on a stable foundation that can offer SDKs and APIs for both mobile and web apps, the ability to test early and often in a sandbox environment and analytics that monitor and report performance in real-time.

    Developers know just how challenging their jobs can be, and the fragmentation that can occur due to the vast number of options on the market can make it even worse. There is nothing wrong with using multiple solutions, of course, but having a sturdy and unified underlying infrastructure goes a long way toward avoiding unnecessary headaches.

    Finally, organizations should look to those who know best when deciding which company-wide tools to implement: the developers themselves. As the IDC study states, developers should be viewed as key stakeholders in IT purchasing within any organization moving to the cloud accompanied by an internal digital transformation.

    — Raju Vegesna

  • Leadership Role in Adding Test Automation to Existing Environments

    Leadership Role in Adding Test Automation to Existing Environments

    Leadership is the foundation for all agility. I’ve written elsewhere extensively on a model that works well for the new environment in which we find ourselves: the need for tens or hundreds of very smart people, doing things that have never been done before with new tools, working with people they may not be familiar with.  These creative, expert people make dozens of decisions a day that few others see in real-time, turning ideas into code that must run flawlessly and execute complex business processes. Quite a leadership challenge.

    The leadership model I’ve articulated is simple. Leaders at all levels should focus on rigor (good decisions based on options and data), alignment (everyone rowing in same direction) and efficiency (respecting everyone’s valuable time). Getting the right people, with the right skills and the right kinds of interactions, is more valuable than having the right process and tools. 

    For software agility, we know reducing the friction between development and productive use is a powerful leverage point. This general area we now call DevOps. Coming from the application space as I do, my primary focus has been on test automation. How can we at very low cost continually ensure that our evolving software is ready to be deployed to production (or production staging, if we are doing periodic bundled software deploys)?

    I’ve recently watched a team introduce test automation into an existing application environment. Great leadership was demonstrated throughout by multiple team members. Let’s examine the steps taken and point out the leadership that made it happen.  

    Set the Target

    Team and organizational leaders laid out a clear vision of the business benefits and their support for the initiative. This was the culmination of several months of learning, communication and cost/benefit analysis. Leaders gained alignment based on rigorous analysis.

    Ensure Adequate Leadership and Technical Skills Are in Place

    In the run up to the inception of the work, leaders hired an experienced testing leader, who in turn hired a couple engineers to do the work. Because the environment was a common one in the industry (Salesforce was a big part of it), the leaders sought the best expertise they could find, and wound up engaging an external specialist firm to get them going. People over process. 

    Define the Technical Architecture

    The new team leveraged the knowledge of the consultants and some of their engineering partners to examine options and select an approach. They shared the options and analysis with their team and leaders and gained consensus and understanding. In the process the approach was fine-tuned. The team demonstrated rigor and efficiency and gained alignment. One key of their success was the “towering technical expertise,” a lean product development term, brought by the consultants.

    Deliver Incrementally

    For this item, I will describe the solution chosen, since the specifics will be of interest to this audience. The vision was to train the product owner/systems analysts to write acceptance criteria for each user story (which was the unit of development here) in the stylized language Gherkin. As the development was done during a sprint period, the test team would take the Gherkin and write automated tests for the test tool Cucumber, which would drive execution in Selenium. The execution would be automatically connected to the deployment of new code using Jenkins.

    The team wanted to avoid disrupting the existing flow until the risk was reduced based on proven results. Accordingly, they began by building automation for existing software, proving that the vision worked in practice, and gradually completed the regression suite and started to catch up to work in process that hadn’t been released. Excellent leadership for agility was demonstrated again, driving rigor through proof of the concept and laying the groundwork to get the rest of the team aligned.

    Get the Rest of the Team to Change

    Once the test team caught up to being able to do in-sprint automation, they needed to tackle the training and acceptance of a new way of writing acceptance tests (in Gherkin). The great results they demonstrated helped influence the product owners and systems analysts to give it a try.

    Continuously Improve

    As the creative period of designing the test automation approach and getting into production came to an end, a new period of measurement and continuous improvement began. Here we have another foundational element of leadership for agility: understanding where we have repeatable flows where process really is the key, and where we have a creative endeavor in which the focus on people and interactions must rule. In a sense, the team transitioned from a product development effort to a manufacturing-like flow, and they accordingly changed their approach.

    Conclusion

    Sometimes we make the mistake of thinking that getting to efficient DevOps is all about tools and processes. My experience is that to the contrary, leadership is the key. As the first value of the 2001 Manifesto for Agile Software Development states, “People and interactions over process and tools.”

    — Michael K. Levine

  • Best of 2019: The Time Has Come for an Engineering Approach to DevOps

    Best of 2019: The Time Has Come for an Engineering Approach to DevOps

    As we close out 2019, we at DevOps.com wanted to highlight the five most popular articles of the year. Following is the third in our weeklong series of the Best of 2019.

    Do you want to know how to do DevOps? Are you doing DevOps but not satisfied with the results you are getting? DevOps is complex. It does not come with a prescription, installation guide, user’s manual or maintenance manual. It doesn’t even have a standard definition. How can anyone expect to get good results with DevOps without clear and definitive guidance? There are plenty of books that describe different aspects of DevOps and customer user stories, but up until now there has not been a book that frames DevOps as an engineering problem with a step-by-step engineering solution and a clear list of recommended engineering practices to guide implementers.

    My book, Engineering DevOps: From Chaos to Continuous Improvement … and Beyond, provides a step-by-step engineering prescription that can be followed by leaders and practitioners to understand, assess, define, implement, operationalize and evolve DevOps for their organization. It provides a unique collection of engineering practices and solutions for DevOps. By confining the scope of the content of the book to the level of engineering practices, the content is applicable to the widest possible range of implementations.

    This book was born out of my desire to help others to do DevOps, combined with a burning personal frustration. The frustration comes from hearing leaders and practitioners say, “We think we are doing DevOps, but we are not getting the business results we had expected.”

    There are many documented references, courses and other sources of information that explain a multitude of different aspects of DevOps, but there is not a comprehensive step-by-step prescriptive guide that details how to engineer a DevOps solution for any organization given any starting point.

    Engineering DevOps takes a strategic approach, applies engineering implementation discipline and focuses operational expertise to define and accomplish specific goals for each leg of your DevOps journey. This book guides the reader through a journey from defining an engineering strategy for DevOps to implementing The Three Ways of DevOps maturity using engineering practices: The First Way (called “Continuous Flow”) to The Second Way (called “Continuous Feedback”) and finally The Third Way (called “Continuous Improvement”). This book is deliberately intended to be a guide that will continue to be relevant over time as your specific DevOps, and DevOps more generally, evolves.

    Structure of this book

    This book is organized as an engineering reference guide presented in five parts as follows:

    Part I: What Is Engineering DevOps and Why Is It Important? is organized into three chapters. Chapter one, “What is Engineering DevOps?,” explains engineering concepts and terms used in this book that have proven to work with many clients I have encountered during my consulting experiences. My depiction of a DevOps Engineering Blueprint is presented for as a useful, practical “big-picture” reference for discussing engineering the major parts of DevOps. Chapter two, “Nine Pillars of DevOps,” describes a classification of DevOps practices that I have found to be useful characterization of DevOps engineering practices readily applied when engineering DevOps implementations. Chapter three, “Why Is Engineering DevOps Important?,” explains the benefits of taking an engineering approach to engineering DevOps.

    Part II: Engineering People, Processes and Technologies for DevOps provides a comprehensive explanation of recommended engineering practices for the higher levels of the DevOps Engineering Blueprint. DevOps transformations are not normally understood or applied using strict engineering principles. Engineering practices are presented in nine chapters. “How Should DevOps Be Engineered?” includes a discussion of DevOps Engineering Maturity Levels for each of the Three Dimensions (People, Process, and Technology), Twenty-Seven DevOps Engineering Critical Success Factors and Lean DevOps Value-Stream Pipeline Engineering.

    I describe recommended engineering practices for the top layers of the DevOps Engineering Blueprint in the next chapters, which include “Value-Stream Management (VSM),” “Application Release Automation (ARA),” “Version Management,” “Continuous Security (a.k.a. DevSecOps),” “Service Catalog,” “Governance,” “Site Reliability Engineering (SRE),” “Disaster Mitigation” and “Recovery.”

    Part III: Engineering Applications, Pipelines, and Infrastructures Engineered for DevOps provides a comprehensive explanation of recommended engineering practices for the lower levels of the DevOps Engineering Blueprint, presented in six chapters as follows: “DevOps Applications Engineering,” “CI/CD Pipelines Engineering,” “Elastic Infrastructure Engineering,” “Continuous Test Engineering,” “Continuous Monitoring Engineering” and “Continuous Delivery and Deployment Engineering.”

    Part IV: DevOps Seven-Step Transformation Engineering Blueprint provides a description and tools for my approach to realize and evolve DevOps. The seven steps are Visioning, Alignment, Assessment, Solution Engineering, Realization, Operationalize and Expansion. By using these tools, DevOps leaders and practitioners can create, implement, operate and expand their DevOps across the organization. The chapter goes further to explain how to evolve DevOps from a successful First Way DevOps (Continuous Flow) toward realizing more advanced Second Way (Continuous Feedback) and Third Way (Continuous Improvement) DevOps implementations. This part includes a discussion of “Beyond Continuous Improvement”—a look at emerging technologies shaping DevOps in the future, and how you can prepare your DevOps and yourself for the future. This part includes a discussion of how to set up an effective DevOps engineering training program that supports continuous learning of DevOps engineering skills needed to maintain and enhance DevOps.

    Part V: Appendices, Continuous Learning, and References includes materials and sources I have found most useful for engineering DevOps.

    To get a copy of my book “Engineering DevOps: From Chaos to Continuous Improvement … and Beyond,” or start reading for free, click here.

    — Marc Hornbeek

  • Best of 2019: Critical Skills Developers Need to Avoid Getting Left Behind

    Best of 2019: Critical Skills Developers Need to Avoid Getting Left Behind

    As we close out 2019, we at DevOps.com wanted to highlight the five most popular articles of the year. Following is the second in our weeklong series of the Best of 2019.

    Burnout is a recurring theme at DevOpsDays events around the world. The modern developer faces constantly and rapidly-evolving challenges: the agile movement reducing the development lifespan, the DevOps movement shifting production operational responsibilities into development processes and the complexity of cloud offering new architectures from SaaS through to FaaS, as well as microservices.

    As these shifts occur, often ignored are the operational skills needed to support an application running in production. Developers expect to support production and be on call 24/7, but are IT leaders giving them the necessary tools and training? More and more, we need developers to be prepared and able to seamlessly oscillate between focused specialist and broad generalist. This article highlights a few key areas where software engineers need to bolster their skill sets to remain agile and effective in building applications.

    DevOps and SRE Assume Broader Skills

    Non-developers may interpret agile computing as developers creating a world where they get to code rather than document, follow process or deliver to plan. In reality, agile took the lengthy cycle of waterfall releases and replaced it with the frenetic continuous delivery of valuable software. DevOps was born from the need to deliver application updates rapidly and regularly.

    The monitor phase of the DevOps infinite loop originally focused on ensuring application problem detection and diagnosis was possible. Moving from detection to prevention, DevOps practitioners had to shift their testing and quality processes. Developers were asked to code the application, while also creating the testing systems to validate their code and building the production executables. Progress here led to additional responsibilities. As development moved to agile and DevOps, infrastructure transformed into software defined infrastructure (SDI).

    The separation of the operating system from the server hardware through virtual machines started the journey to SDI. Application containers removed the OS requirement and added cross-server mobility. Kubernetes and Docker Swarm allowed developers to encode application requirements in the recursively titled YAML. After containers, microservices is a further refinement that isolates an action into a separate piece of code—which is well suited for workload-based pricing of cloud environments. SDI’s capabilities bring us to where infrastructure definitions require a software development lifecycle (SDLC) of their own.

    The outcome of all this is a massive increase in complexity that developers must consider as they code components of an application. It requires additional considerations of the infrastructure and business environment the application will run in. Developers can define the best YAML definition of needs for their new application, but does not mean the production infrastructure will be able to provide a matching environment.

    Skills Developers Must Enhance

    Here are some key areas where developers would benefit from honing their skills in order to stay relevant amid growing complexities mentioned above.

    Networking

    The single purpose of computer networking is to move data from the memory of one computer into the memory of a different computer. With TCP/IP protocols as the default network stack, the flexibility needed to survive a nuclear attack comes with a lot of variability in configuration. At the basic level, network components must be configured to identify how many bits of an address to use when searching for another network entity via Classless Inter Domain Routing (CIDR).

    Software engineers need networking skills to understand how their software definitions may conflict with the eventual production network infrastructure. For instance, network definitions in the YAML descriptors for Kubernetes/Docker are generally set with a /nn configuration (e.g. /16). Conflicts arise when you overlap the pod addresses with physical server addresses or have a mismatch between development pods and production pods. If your pods-per-node count differs between pre-prod and prod environments, it is possible the CIDR settings will differ as well—as a colleague regrettably learned when using /8 rather than /16, leading to the largest production outage in their career.

    In addition to these basics of network addressing, there are concepts such as micro-segmentation needing definitions of performance and connection requirements that span across containers, virtual machines and bare metal servers.

    Security and Segmentation

    Historically, applications have relied on production infrastructures with their firewalls and DMZs for protection from the outside world. That castle moat approach is insufficient for today’s threat landscape where applications live everywhere. Not only do enterprises need to protect against outside connections and attacks, they also need to prevent accidental connections from production applications to development, QA and testing environments. Crossing the streams can cause exposure of production data in test environments and, even worse, corruption of production data.

    Ensuring that application components only communicate to specific services in specific environments is addressed with micro-segmentation and its ability to manage and monitor secure code-to-code communication. Control with this detail comes with its own challenges as incorrect or insufficient definitions could leave a production application unable to communicate with needed API’s or microservices. Making diagnosis in micro-segmented environments harder is the possibility that traffic for other transactions or applications may be successfully crossing the same environment. This is when you may hear the classic “it is working fine in test” response.

    Micro-segmentation offers significant security benefits yet requires developers to clearly identify the required connections and components of any communication. The best time to note this is while coding, making this an important skill for the modern developer.

    Dev and Ops Synchronicity

    The synchronous arrival of DevOps and SDI provides a mutual opportunity for developer and operations teams. While developers need to learn production on-call skills and operations teams need to learn SDLC skills, both teams also need to remain current in security concepts. Some organizations tackle DevOps by creating merged teams, though members may continue to stay in their lane. Whether merging teams, or maintaining separation, it is beneficial to hold brown bag sessions to share information and help educate each other. Ask your operations peers to talk about their three most common production issues and the three most popular tools they use.

    An important aspect of the application on-call generalist is having the right tool supporting them to effectively diagnose production outages and slowdowns. Whether developed as an in-house solution (with incumbent technical debt) or selected as a product, you will need something that can see from end-to-end and allow the code-centric software engineer to drill into databases, storage, networks and code components from an application point of view without having to learn siloed management and monitoring tools for each aspect of infrastructure.

    While individuals come into a DevOps transformation with I-shaped specialists, they must leave as T-shaped engineers, thus enhancing their value to current employers and helping their career.

    — Marco Coulter

  • Predictions 2020: Key Trends that Will Shape the Autonomous Digital Enterprise

    Predictions 2020: Key Trends that Will Shape the Autonomous Digital Enterprise

    As 2019 comes to an end, and we look toward trends driving innovation in 2020 and beyond, technology advancements in hyper automation, AIOps, edge computing and the convergence of IT service management and IT operations management will be at the forefront. 

    Autonomous Digital Enterprise

    In order to thrive in the 2025 futurescape, organizations will need to evolve to Autonomous Digital Enterprises (ADE), a state where intelligent, integrated, value-creating functions can operate with minimal (ideally zero) human involvement or interference, across every facet of the organization, and its ecosystem of partners.

    This concept is based on the notion that every company today is being disrupted by technology and that’s fundamentally changing their business model. The notion of every company being a technology company means that software becomes more than just a system of record to be the system of action, the system of insights and the system of engagement.

    Edge Computing and IoT

    Over the next couple of years, partnerships will prove to be the pathway to success in realizing value from edge computing and IoT solutions. By using seamlessly integrated, multi-vendor, commercial off-the-shelf solutions, enterprises will experience more success than they would with single-vendor niche products. At the end of the day, what we need is an ecosystem of integrated solutions as opposed to a single silver bullet. This ecosystem will be able to harness data and look at it in meaningful ways and translate information into insight that then translates into actions that businesses can take to optimize the use of IoT assets.

    Convergence of IT Service Management and IT Operations Management

    As IT service management and IT operations continue to converge in 2020, CIOs will rely on AI. More than ever, CIOs will need to address this convergence and work to break down silos between these departments. To obtain a comprehensive view of the IT landscape, CIOs will begin to adopt AI, cognitive and robotic process automation. This will bring end-user support and managed back-end infrastructure more closely together. 

    If we look beyond traditional IT and service management, in the near future we will have the ability to go to market with fully integrated solutions that enable boundary-less operations across AppDev, SecOps, ITOps and service management. This will further enable the use of DevOps in organizations using the power of AI to improve existing capabilities and create new capabilities. This doesn’t mean companies have to start from the ground up on AI efforts, rather they need to look for software solutions that can offer prebuilt AI/ML solutions that drive change.

    AIOps

    The demand for AIOps in the enterprise will continue to rise as AI and machine learning have taken the industry by the jugular. Due to an expansion in the number of workloads—both in public cloud and on-premises—and an increase in application complexity, investment in AIOps will increase and ultimately lead to better business outcomes. Today’s challenges place a premium on differentiated vendor solutions powered by AI/ML and big data analytics techniques that can help modern IT operations evolve from traditional monitoring to observability to actionability.

    All of these trends open realms of possibilities for what businesses can achieve in 2020—as long as they can harness the massive amounts of data enterprises have access to and the technology that will allow them to unlock those insights.

    Want to learn more about what to expect in 2020? Join us Jan. 23 for our Predict 2020 Virtual Summit  featuring discussions from some of the industry’s best and brightest offering up their visions for the future. Sign up today for this free daylong virtual event.

    — Ram Chakravarti

  • Untapping the Potential of DevOps, Part 4: Tips for Launching a DevOps Team

    Untapping the Potential of DevOps, Part 4: Tips for Launching a DevOps Team

    DevOps has become a hot strategy for IT teams across just about every industry—and for good reason. DevOps is built around increasing the velocity of the development pipeline, and in businesses where time to market is a key to success, it’s easy to imagine why DevOps has gained traction. While the tools and processes put in place are certainly essential, building the right team—from top to bottom—is maybe the most important factor for getting the most out of a DevOps strategy.

    To help get the most out of DevOps initiatives—and to set teams up for long-term, sustained success—here are some basic tips for creating the team that will take development pipelines to new heights.

    Change Is Coming

    When kicking off a DevOps strategy, it’s clear that changes will be made. But, the extent of those changes may be underestimated. To get the most out of DevOps, it’s not just about changing up the tool stack or switching up meeting styles. The mindset around DevOps is at least as important as the processes. To squeeze the most juice out of a DevOps initiative, make sure the company and leadership understand what all DevOps entails and create the understanding that it requires a lot of internal changes to the existing culture. Getting leadership on board with the cultural changes that come with DevOps will go a long way toward driving success.

    Talent Talks

    This may go without saying, but it is imperative to build a team with the right skills for the task at hand. It can be easy to try to shoehorn people into roles, especially when trying to move quickly. But make sure to set individual talent up for success—both employees and the company will benefit. This means having DevOps engineers and leadership with the skills—whether they are technical, interpersonal or something else—to perform the work required.

    Get Your Budget Right

    DevOps helps create efficiencies. Leveraging automation, maximizing value delivery and getting products to market faster are all key tenets of a good DevOps strategy. But that doesn’t mean you should skimp on the important items. Like any project worth doing, it’s a lot easier (and more successful) when it is budgeted properly.

    Make sure to have an adequate budget to create a team, perform the work and support the appropriate environments and infrastructure to perform the DevOps tasks and initiatives that need to be done. Doing this in advance will save a lot of headaches down the line and help develop a clear understanding of what is really needed to get from point A to point B—and help keep spending in line while the DevOps train is moving.

    It’s Not Just About IT

    Remember there are two parts to DevOps—dev and ops. It is often easier to get the developer side to buy into the strategy—after all, many of the tenets of DevOps speak to the technical teams. But overall buy-in is necessary for cooperation and involvement of all teams needed to achieve goals. That means development and operations—plus other departments within the organization, such as sales, marketing, the executive team, etc. Getting everyone on board with the process and rowing in the same direction will fundamentally change the mindset around how to design and deliver products. When the whole team is moving in lockstep, that’s when success can really be visualized.

    By keeping the thoughts above in mind when planning a DevOps journey, organizations can help ensure teams are organized and set up for long-term success. But remember, every organization is different and has different requirements. What may work for one organization may not for the next.

    When considering the potential of DevOps for your organization, it can be tempting to dive right in and skip some important considerations for setting up the initiative. But the key points listed above are tried-and-true and carry weight, no matter the size or scope of business. Having a leader who acts as a champion for DevOps efforts is essential, so IT professionals tasked with leading the organization into the DevOps transformation should arm themselves with these tips to be better positioned for success.

    — Jonathan Fries

  • Predictions 2020: The Year of DevOps Empowerment

    Predictions 2020: The Year of DevOps Empowerment

    Every year since mid-decade, it seems, one publication or another has declared it to be “the year of DevOps.” Allow me to make the case that it really will happen in 2020.

    The reason? IT leaders are finally empowering their engineers with full-service ownership.

    In a world in which speed, quality and precision will continue to determine the difference between continued success and unmerciful failure–particularly with regard to delivering best-in-class, end-to-end experiences for all stakeholders–full-service ownership is a matter of reality and necessity. Not to mention common sense.

    Developing and delivering products and services that maintain pace and meet the needs of businesses is dependent on achieving faster release cycles without sacrificing quality. But running reliable systems at ever-increasing speeds presents huge challenges that software teams can meet by changing their service ownership policies. The good news is DevOps practices have been gaining considerable traction because they help teams deliver with increased velocity coupled with reduced risk. This enablement, similar to any change introduced, takes time and practice to make work for your needs.

    Code It, Ship It, Own It

    Full-service ownership grants engineers greater responsibility for the code they write and the services they develop during production. The importance of owning and operating your own code and taking responsibility for all deliverables should not be underestimated, because it is the engineers, after all, who should be–and have earned the right to be–accountable. They are the ones designing architectures and systems, coding software and writing runbooks. They are the ones on whose shoulders the overall quality of service sits. They are the ones who are on call when issues arise (invariably, as Murphy’s Law implies, in the middle of the night).

    If they are the ones who code it, then they are the ones who should ship it and own it. The days of engineers “throwing code over the wall” to operations or depending on site reliability engineering (SRE) teams to guarantee the performance of services in the wild are no longer relevant, much less workable.

    That’s particularly important going forward, considering the DevOps market is predicted to grow 18% to nearly $13 billion by 2025. As DevOps tools and solutions become more ingrained in IT operations, the most logical and proactive way to manage this growth, and the inevitable chaos it will bring, is to empower IT with full-service ownership.

    The Forces Behind Full-Service Ownership

    This is happening for three reasons:

    Accountability

    It’s a simple truth; consumers implicitly expect the services they use to work seamlessly and with minimal latency. Considering that consumers will simply move to another site when a page is slow to load or returns a 404 error, the payoff in terms of increasing business’ competitive advantage is immense.

    Full-service ownership promotes higher-quality work because it gives engineers clear visibility into how the code and services they’ve created are performing and impacting their customers’ daily activities. 

    Reliability

    It’s not rocket science to ask the scientist who built the rocket to fix it when it breaks. Likewise, businesses can reduce the amount of downtime and customer impact when a serious incident occurs by immediately bringing in the engineers who wrote the code or created the service. 

    Full-service ownership offers another important advantage: overall institutional knowledge among IT teams. Individual engineers benefit greatly from code reviews, on-call hand-offs, daily standups, casual information sharing and other exercises that build redundancy in knowledge and expertise.

    Continuous Improvement

    A common misconception in the engineering world is that “on-call” is synonymous with “always-on.” That in itself is counter-productive because it can lead directly to alert fatigue. In an ideal world, teams will build time in their development work to go “off-call,” a practice that alleviates alert fatigue and that full-service ownership encourages.

    It also fits perfectly with an integral element of the culture of engineers: a commitment to continually refine and improve their code, product, services and alerting. The most effective process goes beyond just fixing problems, it also includes getting ahead of them, by automating the work needed to prevent them, freeing engineers up to focus on the work that matters most. 

    Five Benefits of Full-Service Ownership

    1. The quality of the code will increase.
    2. Engineers will know when they are off-call, and alerts will be covered by another developer with equal expertise.
    3. Team knowledge and skills will increase through a deeper understanding of the codebase.
    4. Services will become more reliable.
    5. Operations will become more agile.

    When engineers are empowered to code it, ship it and own it, businesses put themselves in a better position to enhance the customer experience and reduce the impact of disruptions, which enables them to focus more on innovation. By defining and refining roles and responsibilities, full-service ownership also reduces the chaos associated with incidents, eliminates silos and unnecessary layers and, ultimately, promotes a culture of empowerment and accountability.

    Want to learn more about what to expect in 2020? Join us Jan. 23 for our Predict 2020 Virtual Summit  featuring discussions from some of the industry’s best and brightest offering up their visions for the future. Sign up today for this free daylong virtual event.

    — Tim Armandpour

  • Predictions 2020: Application Trends, Opportunities and Challenges

    Predictions 2020: Application Trends, Opportunities and Challenges

    Without wheeling out all the usual clichés, 2019 has been another whirlwind of disruptive innovation and opportunity–with plenty of challenges to tackle and circumvent along the way.

    The threat landscape mutated with predictable unpredictability, multi-cloud app deployments are becoming mainstream fixtures and DevOps methodologies started exerting a newfound influence on business plans.

    The big question is what happens next? What anticipated app-centric trends will change the game and tear up the rule book (again)?

    Digital Transformation Takes Shape

    In 2020, we will see more organizations shift away from aspirational sloganeering to substantively embrace what can, and should be, a seismic step-change.

    Inevitably, business leaders will get more involved in application decisions designed to differentiate or provide unique customer experiences.

    Expect a new generation of applications that support the scaling and expansion of business’ digital models to emerge. This will include taking advantage of cloud-native infrastructures and driving automation through software development.

    Further down the line, digital transformation efforts will likely be AI-assisted, particularly as they leverage more advanced capabilities in application platforms, telemetry, data analytics and ML/AI technologies.

    End-to-end instrumentation will enable application services to emit telemetry and act on insights produced through AI-driven analytics. We anticipate that these distributed application services will improve performance, security, operability and adaptability, without significant development effort.

    The Rise of Application Capital

    Applications are now firmly established as the main conduit for companies to develop and deliver goods and services. They have become modern enterprises’ important assets.

    Even so, most still only have an approximate sense of how many applications they have, where they’re running or whether they’re under threat.

    This will soon change.

    To manage Application Capital effectively, it is essential to establish a company-wide strategy that sets policy and ensures compliance. This includes addressing how applications are built, acquired, deployed, managed, secured and retired. At a high level, there are six distinct and unavoidable steps that need to take place: build an inventory, assess the cyber risks, define application categories, identify the application services needed for specific activities, define deployment parameters and clarify roles and responsibilities.

    The primary aim of an application strategy should always be to enhance and secure all digital capabilities–even as their reach and influence shift and expand.

    DevOps’ Culture Club

    The technical minutiae DevOps methodologies and associated tools got a lot of publicity this year. 

    2020 will be all about getting the culture right, marrying theory with best practice and unlocking new levels of productivity without upsetting the operational apple cart.

    Culture is not optional. Team structure alone dramatically changes pipeline automation, with traditional single-function teams often falling behind their contemporary, DevOps-driven counterparts. 

    Consequently, we will see more collaborative team structures and alignment on key metrics that give NetOps additional means to focus on what the business requires: faster and more frequent deployments.

    DevOps has a 10-year head start on NetOps in navigating and overcoming obstacles around certain types of integration, tools and skillsets. Collaborative teams can explode the status quo by promoting standardization on tools that span from delivery to deployment (such as Jenkins and GitHub/GitLab).

    DevOps should not–and cannot–end with delivery. That means deployment functions–along with a complex pipeline of devices and application services–must be automated. This won’t happen without effective cultural realignment.

    The Data Center Is Alive and Kicking

    Conflating the adoption of SaaS with IaaS caused speculation that cloud was cannibalizing IT. Pundits warned that data centers would disappear. 

    The rumors were exaggerated. Data centers are still being built, expanded and run around the globe. The cloud hasn’t managed to–and likely never will–kill the data center.

    Early in 2019, an IDC executive told channel partners at the IGEL Disrupt conference that over 80% of companies they surveyed anticipated repatriating public cloud workloads. Security, visibility and performance remain common concerns.

    Repatriation-related opportunities include improving availability of multi-cloud operational tools and a push toward application architectures that rely on more portable technologies such as containers. 

    The data center is not dead. It’s just evolving. 

    Application Protection Challenges

    According to F5 Labs, the server-side language PHP–used for at least 80% of websites since 2013–will continue to supply rich, soft targets for hackers. Situational awareness is critical to mitigate both vulnerabilities and threats.

    Businesses are also realizing that applications encompass more than just the code they execute. Attention needs to be paid to everything that makes them tick, including architecture, configurations, other connectable assets and users. The prevalence of access attacks, such as phishing, are an obvious case in point.

    F5 Labs analysis of 2019 breach data confirms the need for risk-based security programs instead of perfunctory best practice poses or checklists. Organizations need to tailor controls to reflect the threats they actually face. The first step in any risk assessment is a substantive (and ongoing) inventory process.

    As ever, the industry will gradually incorporate emerging risks into business models. For example, cloud computing has gradually shifted from a bleeding-edge risk to a cornerstone of modern infrastructure. The risks associated with the cloud have either been mitigated or displaced to contractual risk in the form of service level agreements and audits. 

    API and You Know It

    The word is out. Application programming interfaces (APIs) can transform business models and directly generate revenue. Cybercriminals know this.

    More than ever, organizations need to focus on the API layer, particularly in terms of securing access to the business functions they represent.

    One of the biggest issues is overly broad permissions, which means attacks through the API can give bad actors visibility into everything within the application infrastructure. API calls are also prone to the usual web request pitfalls such as injections, credential brute force, parameter tampering and session snooping.

    Visibility is another major and pervasive problem. Organizations of every stripe–including IT vendors–have a notoriously poor track record of maintaining situational API awareness. 

    API security can be implemented directly in an application or, even better, in an API gateway. An API gateway can further protect APIs with capabilities such as rate limiting (to prevent denial of service attacks) and authorization. Authorization narrows access to APIs by allowing access to specific API calls to only specified clients, usually identified by tokens or API keys. An API gateway can also limit the HTTP methods used and log attempts to abuse other methods so you’re aware of attempted attacks.

    Appy New Year!

    All this is of course the tip of an increasingly interconnected iceberg. Any New Year’s resolution worth its salt should include a commitment to comprehensively master the development, deployment, operation and governance of application portfolio. The best way to do this, and to get visibility into the code-to-customer pathways for all applications, is to leverage a consistent set of multi-cloud application services. Here’s to a safe, innovative and transformational 2020!

    Want to learn more about what to expect in 2020? Join us Jan. 23 for our Predict 2020 Virtual Summit  featuring discussions from some of the industry’s best and brightest offering up their visions for the future. Sign up today for this free daylong virtual event.

    — Vincent Lavergne

  • DevOps at Longtail UX

    DevOps at Longtail UX

    Longtail UX is an Australian SaaS startup that boosts traffic to e-commerce sites by automating the selection of relevant long-tail search terms and the creation of custom landing pages for each of those terms.

    The company has fully embraced the Infrastructure-as-Code approach, according to Troy Jendra, senior systems architect at Longtail UX.

    By using Terraform and Ansible for automation, “we can release whenever we want,” said Jendra. In practice, that can be at intervals between two minutes and as long as necessary at the other extreme.

    Jendra said he had not used Ansible before joining Longtail UX. He used Puppet in a previous role, but found it “painful.”

    “Ansible was a breath of fresh air,” confessed Jendra. It makes it easier to write configurations, and uses fewer CPU cycles because it does not need to run continuously. He explained that Terraform’s role is to spin up sufficient infrastructure so Ansible can run.

    A recent project added support for CJK (Chinese, Japanese, Korean) characters to Longtail UX as part of a push into the Asia market, and that made it necessary to clone part of the infrastructure. Thanks to Ansible and Terraform, that task was easy to perform and completed in just half an hour. Thus, Longtail UX brought the new capability to market more quickly than would otherwise have been possible.

    Ansible and Terraform together form “a cornerstone of our disaster recovery plans,” said Jendra. Fully automating recovery to improve on the current worst-case time of a few hours would take more time and effort than justified.

    Apart from that, “it’s important to automate as much as we can,” said Andreas Dzumla, Longtail UX co-founder and CEO.

    Avoiding Vendor Dependence

    Even though Longtail UX runs on AWS, a deliberate decision was taken to avoid the vendor-dependency that would come from using AWS-specific services, such as AWS CloudFormation, in place of Terraform.

    For the same reason, the company keeps things as simple as possible and sticks to AWS basics, such as EC2. It has recently started using Spot Instances to reduce the cost of running certain batch processes.

    Jendra keeps a watching brief on AWS Lambda, and while the company does not currently have a use case he does see a time when it will jump from EC2 to Lambda.

    “We’re pretty locked into AWS,” he admitted. However, the company uses AWS-specific services only where the benefits clearly outweigh the lack of portability. He likened the situation to 1990s attitudes toward using Microsoft-specific technologies.

    Growth

    The adoption of Kubernetes is not on the cards due to the relatively small size of Longtail UX’s DevOps team.

    “We’ve doubled the size of our development team in the last six months,” said Jendra. It now comprises seven developers, a project manager and himself.

    Dzumla expects it will double again in the next 12 months.

    Jendra predicts that improvements to Longtail UX’s development stack will make it easier to on-board new hires.

    “We currently utilize a Vagrant/VirtualBox dev stack (hooked into the aforementioned Ansible configuration), but find that very rough around the edges, and I only see it get worse as more attention is focused on Docker,” said Jendra.

    DevOps is a philosophy, not a role, he emphasized. “Either everyone is doing DevOps or it doesn’t work.”

    Fortunately, that understanding goes to the top of the company.

    “Everyone needs to pitch in,” said Dzumla. “We do a lot of things that haven’t been done before, so we need senior people who don’t merely expect to do what they’re told.”

    — Stephen Withers

  • Synack: DevSecOps Being Accelerated by Cultural Shifts

    Synack: DevSecOps Being Accelerated by Cultural Shifts

    The 2020 State of Compliance and Security Testing Report from cybersecurity testing platform vendor Synack claims that some of the world’s largest organizations are encountering a significant cultural shift within their development teams, and that bodes well for those seeking to build DevSecOps teams. For the report, Synack surveyed leaders from more than 300 organizations representing a number of industries and verticals, including technology, government, health care, information technology and financial services. 

    According to the report, a large percentage of organizations and institutions are moving toward a rigorous, continuous testing model to ensure compliance. As part of this shift, organizations are utilizing crowdsourced security testing to achieve regulatory compliance and real security, with adoption expected to increase four-fold in 2020.

    Perhaps those companies are being driven by numerous new regulations on the horizon, including the  California Consumer Privacy Act (CCPA), PCI  DSS 4.0, New York’s SHIELD Act and  other proposed data compliance regulations. Or perhaps the shift toward continuous crowdsourced security testing is being driven by the failures some organizations have encountered with traditional security testing methods, which were performed less frequently and failed to keep up with the CI/CD (continuous integration/continuous delivery) pipelines that many organizations have embraced. 

    Simply put, Agile and DevOps application development and delivery methodologies were poorly served by the traditional, siloed security testing methodologies of the past. A realization that has driven many organizations to embrace the DevSecOps model, in which cybersecurity becomes part of the overall team, having equal standing with both development and operations team members. 

    Organizations pushing ahead with the DevSecOps model quickly realized that security toolsets and team member interactions must change to be successful. Synack’s research bears that out, and the report offers that the adoption of continuous security testing platforms is expected to increase by 4X in 2020.

    “The rapid embrace of crowdsourced security testing has happened because it is proven to work better than traditional security testing methods and addresses the ever-growing talent gap within organizations,” said Synack CTO and co-founder Mark Kuhr. 

    The report found security testing is becoming part of an organization’s normal routine, rather than a once-a-year check of the box focused only on compliance. According to the report, 44% of the organizations and institutions surveyed perform security tests on a monthly or weekly basis, suggesting that they are moving toward a continuous model, which is further enabled by crowdsourced solutions.

    The report also revealed that some 63% of organizations commonly use external vendors to identify and reduce vulnerabilities. However, 52% experience unwanted cost and complexity due to overlap in functionality from using multiple security vendors. Those two data points demonstrate that simplification provided by leveraging a single security testing platform could deliver savings. Synack claims crowdsourced security testing solutions provide 147% higher ROI than a typical pen test and can decrease the burden of testing on organizations by reducing the signal-noise ratio. 

    “This shift toward continuous crowdsourced security testing will allow organizations and institutions to have the best of both worlds by procuring technology that offers efficient and effective results while fulfilling best practice standards such as NIST 800-53 to meet compliance objectives,” said Kuhr.

    “Although we are seeing a move toward a 24/7, 365 security culture at organizations in a wide variety of industries and geographies, there is still ample room for improvement,” said Aisling MacRunnels, Synack’s CMO. “Our survey found that on average, most security tests are lasting just 20 hours. As the number of cyber incidents continues to increase, it will be imperative for decision-makers to implement security testing solutions on a continuous basis with 1500-2000 hours of testing a year.”

    Those increased testing requirements will ultimately become the foundation to build a DevSecOps culture in most any organization bound by compliance regulations.

    — Frank Ohlhorst