devopsy.kinsta.cloud

Tag: devops

  • Minor Crimes and Misdemeanors in the Age of Automation

    Minor Crimes and Misdemeanors in the Age of Automation

    Giving automation the power to detect crime and enforce punishment has ramifications, even for minor infractions

    In November, I broke the law. I crossed over a solid white line to make a right turn at a traffic intersection. At the time I was unaware of my violation. I was on my way to a shopping mall in an unfamiliar part of town to buy my wife a gift for her birthday. My only defense is that I was following the instructions emitted from the map app on my cellphone. It told me to make a right turn. So I did. Little did I know I was being watched.

    Automation enhances law enforcement capabilities

    A few weeks later, I received a citation in the mail. The citation had photos of me, my car’s license plate and the car crossing the solid white line. Turns out, there were cameras at the intersection watching all traffic activity. I was busted. I was also a bit shocked: My experience getting a traffic ticket 10 years earlier involved a real police officer writing me up in real time. Now it was as if Big Brother was indeed watching.

    In addition to the eerie feeling that goes with being charged with a crime anonymously well after the fact, I also felt baffled. The traffic citation did not list the amount of the fine I had to pay. The only information it provided was that I needed to go to the traffic court website to learn the details. So I did.

    I logged into the website and entered my citation number. The site had no record of the ticket, but noted that it can take up to 30 days for an automated ticket to make it into the system. I felt powerless; I was charged with a crime, liable for a fine and yet didn’t know how much my transgression was going to cost me.

    I decided to call the telephone number provided on the citation for more information. I dialed the number using the same cellphone that had gotten me into trouble in the first place. I was connected to an automated answering system. After navigating through various Yes/No and “enter the proper number” prompts, I learned that the system had no record of my citation. My only option was to wait 30 days for the systems to get their collective act together.

    A month later, a formal citation did indeed arrive in the mail, noting the amount of my fine ($480) and options for payment. Also, the citation provided instructions to go online and pay the ~$500 required to attend traffic school, should I so desire. (Traffic school is an option in California in which you attend a one-day class about traffic safety and your violation is kept private from insurance companies. The result is, your automobile insurance rates don’t go up. Here in Los Angeles, a lot of professional actors teach traffic school as a side job. As you can imagine, traffic school can be quite entertaining. But, I digress.)

    At this point, I was completely bewildered. I’d been busted for a crime I didn’t know I committed, by machines I didn’t even know were there. The only way I could get the details about my transgression was by interacting with other machines on the internet or on the phone. And, should I decide to plead guilty and settle matters, my first, best option is to do so via automation.

    Still, the technology was failing me. I needed to talk to a person, any person. So I went down to the courthouse in Santa Monica. There I found a special window on the exterior of the courthouse building for processing traffic inquiries. (One of the nice things about living in sunny Southern California is that providing service outdoors throughout the year is a viable option.)

    I waited in line. Finally, I got called forward to the window—which, by the way, was a pane of blackened glass. I had no way of knowing who or what I was talking to; all I saw was my reflection in the black mirror. A voice from the speaker at the bottom of the window asked me to state my business. I did. The conversation continued. The responses from the speaker seemed brusque, if not a bit preformulated. I started to wonder, Am I talking to a machine or a person? So I asked a random question, “How much is two plus two.” The voice in the speaker responded, “Sorry I am not programmed to respond to that information.”

    Now I was really thrown. Was I confronting yet another piece of automation? After a minute or two, the voice in the speaker said, “I’m just messing with you. I get this all the time. They all think I’m a machine.”

    It turns out that the voice did indeed belong to a human, and the human’s name is Dante Pipkins. Dante shared an interesting fact with me: Even though my violation was caught on camera, the actual OK to create the citation was issued by a human police officer behind the scenes. Go figure.

    Thus, having been enlighted about the details of enforcing traffic law via camera, I finished my business by having Dante give me a date to appear in traffic court, before a human judge, to plead my case. I knew there was no way to proclaim my innocence. I had been caught red-handed. My intention was to throw myself on the mercy of the court to get a reduced fine.

    The Power of Automation

    Automation is a great thing. It empowers us. It saves us time and money. Without automation, there’s no AWS, Google, Azure, Digital Ocean, Netflix, Etsy, Shopify, Facebook, Instagram, Slack, Airbnb, New York Stock Exchange … the list goes on. Automation is more than a technological feature. It’s now a way of life. And, for those of us who work in DevOps, automation is fundamental—none of us want to go back to the days of manual deployment and guesswork system monitoring, to say the least.

    Yet, we do walk a tightrope. As my recent travails with a very minor aspect of the legal system demonstrate, automation is becoming a key factor in everyday law enforcement. On face value, this might seem like no big deal, but consider this: Today, charging a citizen with a crime is still subject to some sort of human intervention. There are a lot of details to consider that go beyond the current capabilities of AI and machine automation. Human judgment is still the most reliable form of judgment. Yet, it’s time-consuming and expensive. Eliminating time-consuming, expensive tasks with cheap, fast, accurate machine-driven processes is a key motivator for using automation.

    As automation becomes more familiar with the activities it is processing, it becomes an expert. And, as automation achieves expert status, it replaces human activity. Just think about detecting credit card fraud; these days, practically all of it is done using automation. While we still have a long way to go in terms of totally automated processes, automation is creeping its way into law enforcement. My recent experience is proof.

    Now, imagine a world in which crimes such as tax evasion, financial fraud, embezzlement, regulatory non-compliance and yes, moving traffic violations are detected, reported and, most importantly, prosecuted using automation. Let’s go further: Let’s imagine that automation becomes the default method of prosecution. Imagine that the justice system becomes so automated that the defendant is presented with an offer to plead guilty to a sentence already predetermined according to an AI algorithm. Or, the defendant can choose to opt into a human interaction but will need to bear the burden of the increased court costs, very much in the same way that an automated audio transcription service charges extra for a human to do the work.

    Is such a scenario possible? Well, think about this: My interactions around the traffic violation were conducted without me ever having once had to interact with a human being. I was guilty as charged and automation proved this beyond a reasonable doubt. Not only did the technology detect my transgression, but it also connected me to my car, thus making citation possible. Hence, there’s a good argument that, when it comes to automation-enabled law enforcement, the distance from catching misbehaving drivers in the act to actually going after real crooks is just not that far a leap.

    So, what does this have to do with DevOps? When you scrape away all of the technological marvels, at some point there’s going to be someone in DevOps either creating, deploying or fixing the systems that make this all possible. It’s a big deal to give automation the power and technical wherewithal to enforce “the law.” As the feasibility of putting complex, comprehensive law enforcement systems into play increases, are we in DevOps going to consider such deployments as just another day in the CI/CD pipeline, or are we going to think through the game-changing impact of what we’re about to make?

    For now, I’ll leave it up to you to decide.

    — Bob Reselman

  • Is Your Organization ‘Fit for the Future’?

    Is Your Organization ‘Fit for the Future’?

    Takeaways from a recent conference suggest more work is needed in DevOps adoption among organizations

    This was a question explored by CIO Platform Nederland and the DevOps Agile Skills Association (DASA) in a recent event I attended, hosted by Transavia. Many organizations have already embarked or are about to embark upon Agile transformation initiatives, driven by growing business demands. This event focused on sharing experiences, learning from the practical examples (successes and lessons learned) from Transavia and ABN AMRO, and exploring the way forward to successfully create a high-performing IT culture that is “Fit for the Future.”

    The event was well-attended—unsurprising, really, when you remember the wise words from the late Robert Stroud, former Analyst at Forrester: “To unlock the promise of DevOps, CIOs must lead and support a cultural change within their technology management organization, but as any leader knows, changing institutionalized behavior is the toughest of all management challenges. CIOs must recognize when a trend becomes an imperative for survival. DevOps has become this imperative.”

    DevOps and Agile transformations being seen as imperative for business survival. But not just business survival, according to the “2017 State of DevOps” report, which declared, “By 2020, half of the CIOs who have not transformed their teams’ capabilities will be displaced from their organizations’ digital leadership teams.”

    Rik Farenhorst, CIO of Transavia and chairman of the DASA Leadership Forum, presented lessons learned from Transavia. He stressed the importance of changing mindsets and giving equal, if not more attention, to creating desired “behavior” change and not relying too heavily on structure, processes and automation and hoping that the right culture emerges. “People first,” “live your values” and “lead by example” were his key messages, stressing that values must be translated into corresponding desired behaviors as well as unlearning old ways of doing things. Another important discovery was accepting that not all teams are so far and not all teams can mature at the same speed.

    A great example of “lead by example” was the leadership team’s daily standups and visualization of strategic priorities, gaining more situational awareness and shared understanding and commitment.

    Matthijs Dee and Ivo Carton from ABN AMRO presented their journey from Agile to DevOps, also pointing out the fact that teams mature at different speeds and the need to share lessons learned between teams. ABN AMRO adopted a DevOps playbook that teams could use, containing a kind of toolkit for improving team capabilities. Equally, Dee and Carton noted ABN AMRO’s challenges in moving from “command and control” and traditional hierarchic management models to new leadership styles and behaviors. In ABN AMRO’s case, the organization needed to focus on BizDevOps—it’s not all about code, it’s about delivering business value.

    Farenhorst mentioned “values” in his presentation, while Dee and Carton mentioned “principles.” Both of these concepts need translating into team behaviors and leadership behaviors must help empower teams to make this shift.

    My presentation, “DevOps Will Fail … Unless,” I revealed the findings culled from more than 400 global teams having participated in the Phoenix Project DevOps simulation and gave a Top Three set of takeaways.

    The first was the need to focus on BizDevOps—making the shift from “features” to “outcomes” and from a flow of “code-to-commit to a flow of “idea-to-value.” This was also one of the top challenges for organizations at the most recent DevOps Enterprise Summit.

    My second tip mirrored advice from Transavia and ABN AMRO: The need to define, agree and commit to “desired behaviors” and what is effective collaboration (here is an example of what I mean).

    My third tip was embedding continual learning and improvement as core behaviors. DevOps is new and all about experimenting—it is a long journey and needs to be done as iterative improvements. At the same time, the pace of change coming at us is increasing, as is the amount of change. Teams need to be able to respond to change. I also quoted Mike Orzen, author of “Lean IT”: “Improving your work is just as important as doing your work”; as such, impediments and improvements should also be made visible on the backlog of work for teams and leaders must ensure adequate time is given and reserved for work-in-progress, for teams to improve.

    Following the presentations were three break-out sessions: “Value Creation and Management,” “Organizational Design – Agile/DevOps at Scale,” and “Sustainable Culture Change.”

    These are some of the challenges raised in the Culture break-out session I attended as a follow-up to my presentation:

    • An interesting discussion was around “control” and how to empower the management layers and have them empower the teams. As a CIO, this layer between you and the work floor can diffuse your vision by the time it reaches the shop floor and at the same time leave you unaware of what is happening.
    • Management by walking around was suggested, as a CIO being visible on the shop floor and soliciting feedback. Listen to concerns and enter into a dialogue about vision, goals and what it means.
    • Teams often are faced with fear, uncertainty and doubt (FUD) about new ways of working. What will it mean to their job? To their responsibilities? And will they be able to do it? Leaders must remove the FUD and create a safe environment. Safe to take risk, to experiment, to fail, to give feedback. Managers must foster a “no blame” culture.
    • A study by Google (2012) called Project Aristotle, named in tribute to Aristotle’s quote, “The whole is greater than the sum of its parts,” looked into why some teams work and others don’t. One of the key conclusions was that “psychological safety” was the “it” factor: A sense of confidence that the team will not embarrass, reject or punish someone for speaking up.
    • Change team behaviors. We adopt these approaches and don’t see the behaviors we expect. Not everybody wants to change. Leaders need to be able to identify the silent killers, what is under the water, what are drivers for working against. At the same time, teams need to be willing and able to work effectively as a team and need team training (not just individual training). Team training also needs to address soft skills such as communication, feedback and collaboration.
    • Leaders also need new skills in organizational behavior management to be able to identify behavior drivers and how to create sustainable behavior change.
    • Lead by example was named, but what is that? What behaviors are expected from leaders? Sometimes managers are only one step ahead in this new working. DevOps is new, there is no fixed framework; it is about experimenting what is right for your organization. This is all new.
    • Ask yourself, “Why are we doing Agile, DevOps, Lean? Are we adopting these approaches as a goal in themselves? Or are they to solve a specific set of problems or deliver a specific set of results? And how does this relate to strategic goals?” This must then be translated into, “What is in it for ME?” for the various stakeholders (business, IT management, teams, customers). Success can only be achieved end to end, which means all end-to-end stakeholders must understand the “why” if they are to commit to the journey.
    • Build empathy and trust and asking questions to get open feedback—not telling (command and control).
    • Authenticity: To build trust, be empathetic to FUD; listen to concerns; demonstrate (words and deeds) how these will be taken seriously; and recognize achievements. Empathy must be authentic; if your behaviors are not seen as real, you will lose the trust you are trying to create.
    • Shift from a (real or perceived) “blame culture.” Failure is OK as long as we learn and improve as a result. If there is no trust, then people fear consequences of making mistakes and are less willing or likely to want to experiment or speak up.
    • Leaders should show vulnerability, too. Set an example of having failed and learned. Admit in these turbulent times when new ways of working really are new, managers don’t have all the answers—it is a journey of experimenting, learning and improving and giving feedback.
    • Allow teams time to learn and discover themselves. Recognize that not all teams mature at the same speed, appropriate leadership styles and coaching for teams. Use external (to the team) coaches.
    • Be prepared for the difficult discussions and decisions. Not everybody can and not everybody wants to change. If you are dependent upon heroes, get rid of them—literally or metaphorically. One example noted was a toxic working culture with teams held to ransom by the hero, while another example was having the hero share knowledge, coach and create more T-shaped capabilities.
    • Look for “change champions” in teams—peer level respect. Empower them with the right skills and recognize their contributions and the team successes. “Give a podium” to success from desired behaviors such as experimenting, learning and improving.

    From the discussions it was clear that C-level managers need to learn new skills and demonstrate new behaviors, while management layers must learn more behavioral and organizational change skills. Teams must not only learn DevOps technical skills, but also team-building and soft skills.

    CIO Platform Nederland and DASA will be organizing future events and sharing content and experiences from member organizations to address these needs as a follow up to this kick-off event.

    — Paul Wilkinson

  • SauceCon 2019 Speaker Profile: Melissa Benua

    SauceCon 2019 Speaker Profile: Melissa Benua

    This is the second in a series of articles highlighting speakers at SauceCon 2019. SauceCon brings together the global community of Sauce Labs users and continuous testing experts to learn from each other and level up their automated testing and continuous delivery skills. This year’s event will be held April 23-25 in Austin, Texas.

    SauceCon speaker Melissa Benua loves horses and riding dressage. Derived from a French word that means “training,” dressage is all about creating strength, discipline and precision. It’s fitting, because she is also passionate about training and mentoring junior-level engineers and increasing developer efficiency—which she’ll be talking about in her presentation.

    At SauceCon 2019, Melissa will present “Continuous Deployment for ‘Mature’ Codebases.” I asked her why this topic was important to her. “When starting a new project, it’s easy to take advantage of modern agile development practices, because you’re starting from scratch,” she said. “However, the reality is that isn’t the case for most projects. Our challenge then is how we can take code that may be many years old and transform it into a lean, mean, continuously-deploying machine.”

    In her presentation, Melissa plans to deliver just that: a road map for getting to a release cycle that deploys on-demand from a one that may take weeks or months. She will leverage her personal experience in significantly re-architecting the deployment structure and offer practical advice to anyone managing a more mature codebase.

    Melissa has worked in nearly every software development role there is—dev, test, DevOps and program management—at companies big and small and somewhere in-between. She has created and run high-availability, high-quality services for PlayFab, Bing, Cortana and Xbox One, and now at mParticle as a senior technical lead.

    When she’s not poring over code or training newer engineers, Melissa enjoys spending time with her two small children—and of course, riding her horse. She travels extensively and presents at industry conferences worldwide.

    You can hear from Melissa and more than 30 other expert speakers at SauceCon 2019, the third-annual Sauce Labs user conference. To see a full list of presentations or to register, visit the SauceCon website.

    — Rebecca Cramer

  • SolarWinds Survey Surfaces DevOps Disconnects

    SolarWinds Survey Surfaces DevOps Disconnects

    A survey of 336 developers, web product managers and DevOps practitioners published by SolarWinds, a provider of IT management tools, finds they spend most of their time troubleshooting applications—so much so that there is little time for strategic projects such as transitioning to DevOps practices.

    Joe Kim, executive vice president and global chief technology officer for SolarWinds, said that while senior IT leaders continue to preach about the critical role DevOps will play in changing their IT culture, IT staff still spends too much time dealing with tactical problems to implement any of the goals defined within a longer-term strategic plan. More than half of respondents (53 percent) said troubleshooting application issues is the top task completed on a given day.

    Because most of their time is spent on troubleshooting applications and not on managing and/or deploying applications, Kim said, many employees suffer from low morale. Then, before many IT leaders realize it, their best people are being lured away by the promise of working for an organization that enables them to focus more on the parts of IT process they enjoy, he said.

    In fact, Kim noted, in this situation IT leaders that embrace DevOps are in danger of losing credibility. If IT leaders tout the benefits of DevOps, and yet the internal IT staff is spending most of their time on application troubleshooting, chances are good many of the rank-and-file IT professionals will conclude their leaders are out of touch with their everyday reality. IT leaders would be well-advised to engage more with their IT staffs to make sure their strategies are more informed by that IT reality, he advised.

    The survey also finds respondents are spending less than 25 percent of their time proactively optimizing performance of their IT environments. Being able to spend more time on this aspect of IT is critical, because if more time is spent on proactively discovering issues, less time must be spent on troubleshooting, said Kim.

    IT teams should also be trying to push the boundaries of IT automation to reduce as much of the manual drudgery associated with IT as possible, added Kim.

    To get a better appreciation for real state of the IT environment, Kim advised IT leaders gather and share metrics with their teams. Not only does that provide IT teams with some sense of accomplishment as specific goals are achieved, the metrics make it easier to get IT teams to rally behind transitioning to DevOps best practices, he said. That’s crucial because any transition to DevOps is likely to fail because of cultural rather than technical challenges.

    It’s unlikely the need to troubleshoot applications ever will go away entirely. The developers that create those applications, after all, are only human. But it is possible to make working in IT a lot more fun and rewarding than what most IT professionals currently experience today.

    — Mike Vizard

  • How to Turn Your DevSecOps Shift into Your ‘Pixar Moment’

    How to Turn Your DevSecOps Shift into Your ‘Pixar Moment’

    If you have worked in security as long as I have, you have seen that vulnerabilities persist, despite years of attempts to fix them without changing how the organization operates. One of my all-time favorite leadership quotes comes from the first stockholder meeting after Robert Iger, the CEO of Disney, bought Pixar. “The riskiest thing we can do is just maintain the status quo,” he said. This quote has stuck with me and I feel that the shift to DevSecOps can be many teams’ “Pixar moment.”

    In recent years, there has been a push to shift to a DevSecOps model that has the potential to make a drastic change in the way an organization operates. But unless these shifts are planned well, they often fall apart without having made a meaningful impact.

    The root cause is often the inability or unwillingness to do the in-depth planning and testing required to make sure everything will move to a DevSecOps smoothly. The failure to plan leads to underwhelming returns and low morale with the organization will slowly slide back to the old “status quo.”

    Moving to DevSecOps is often more about changing an organization’s cultural identity than any technical changes. I’ve have seen it happen successfully in several places. Here’s what they had in common:

    They Defined, Discussed and Documented the Old and New Build Process

    A successful move to an end-to-end CI/CD system normally starts with one thing: a whiteboard. Organizations bring representatives from every existing team (systems, networking, security, development) to form the new DevSecOps team and build an end-to-end diagram of what it takes to build and maintain the system immediately. For many, this meeting is often the first time they have seen a true end-to-end view of the system.

    Then the new team commits resources to discuss the standards and policies they need and want the new system to be built to. They discuss methodologies, deployment schedules, SLAs, business requirements and overall system architecture and security. This is where the team must come to an agreement on the right way to build and where the security team is likely to see their first big win on improving the “status quo.”

    This round ends with something that no one on the new team will likely enjoy, but is crucial to the shift moving forward: a document of all of the decisions made that can be easily shared within their organization.

    They Built a Strong Team and Platform

    The next step is normally everyone’s favorite part—at least at the start. The team gets to walk into a greenfield environment and move one application to their new DevSecOps flow. Essentially, each team member has the opportunity to bring their ideas and thoughts about modern security, architecture and deployment processes and see if they can apply them. However, the honeymoon period of this process often ends quickly and becomes the most challenging segment of the whole process. The switch from theory to practice isn’t as clear as many DevSecOps teams think, and it’s common for teams to struggle while working with new products with which they likely have not had much organizational experience.

    Buy-in from leadership is critically important here and without it, these projects can quickly go awry. I have heard executives complain that their DevSecOps team is “just playing around,” “wasting company resources” or “learning skills that will just let them get a better job somewhere” when a new system seems to take longer to be built than originally expected. Supporting this team throughout the build process is one of the most important leadership roles in a move to DevSecOps and plays a crucial part in aiding this team and reinforcing its culture within your company.

    Continual Incremental Improvements

    Once you have your first DevSecOps flow created, it is important to remember that you are not done, as you would be in a traditional build model. Of course, you want to celebrate this success, but you also need to start answering these questions:

    • What can we do to increase the deployment frequency and speed?
    • Can we do anything to lower the failure rate of the new releases?
    • Are there any steps that could be more automated?

    If you continually ask and answer these questions, you will never be done, but you will continue to make incremental improvements that will help your organization become more agile over time.

    Rinse and Repeat

    Once you have built your first DevSecOps workflow, you now have a good baseline on what to do (and not to do) when you move your next application to this model. Hopefully, you’ll be ready to move the DevSecOp culture that has taken hold in your organization to the next team.

    — Jerry Gamblin

  • Hooked on Service Metrics

    Hooked on Service Metrics

    We live in an increasingly “as-a-service” world. From software as a service (SaaS) and platform as a service (PaaS) to functions as a service (FaaS) and SaaS-delivered applications, service delivery has become paramount to business goals and practices.

    In today’s DevOps landscape, microservices—the cloud-native approach to designing scalable, independently delivered services—allow teams to prioritize each individual service rather than many services simultaneously, enabling agility and continuous delivery. However, even with all their benefits, microservices come with their fair share of challenges, specifically with observability, monitoring and security. Service metrics are therefore more important now than ever before, as mitigating the challenges and seizing the opportunities introduced by microservices helps DevOps teams create a high-quality end user experience.

    With service delivery so integrated into the overall business and customer experience, it’s crucial to ensure that service performance always remains at its peak. How can tech pros ensure that service-level agreements are met, services run smoothly and end user expectations are met and exceeded? The answer is simple: become hooked on service metrics.

    Getting Hooked

    In this piece, we’ll explore service metrics in respect to microservices. Due to the observability, monitoring and security challenges that microservices can introduce, managing and controlling microservices and their traffic requires an additional layer of tooling: a service mesh. Implementing a service mesh architecture helps facilitate traffic management, service identity and security, policy enforcement and more, which ultimately brings reliability, security and manageability to container and microservice management. Important metrics such as request volume, request duration, latency and request size can be gathered automatically across a service mesh, which helps facilitate successful microservice deployment.

    However, despite service mesh tooling helping to mitigate risks, failures such as RPC timeouts and exception propagation can occur. Failures in the microservices space often occur during the interactions between services, so a clear view into those transactions enables DevOps teams to better manage architectures and avoid said failures. A service mesh provides observability, which enables DevOps teams to easily see what is happening when services interact with each other, making it easier to build a more resilient, secure and efficient microservice system. Additionally, service meshes provide a developer-driven, services-first network—a network that runs without application developers having to build network concerns into their application codes. Service meshes create a network that empowers operators with the ability to define network behavior, node identity and traffic flow through policy.

    Tips for Success: What You Need to Know

    So, how does a DevOps team implement a service mesh architecture for service metrics? Following a few best practices can help ensure success:

    • Know When to Implement: If a DevOps team does not plan to run an excess of microservices, a service mesh may not be necessary; fewer than 10 microservices likely could be managed with existing tools and security tools. But once a team starts running more than 10, a service mesh becomes essential.
    • Understand Your Use Case: After confirming the necessity of a service mesh architecture, you must understand how it will interact with the other types of infrastructure you’re running, as this will help determine what service mesh to leverage. For example, some work well with Kubernetes or Docker Swarm, but if you’re not running services inside containers, you would only benefit from the security or observability pieces of a service mesh. Depending on your use case, you might want to implement a highly capable service mesh such as Istio or a simpler tooling such as Linkerd 2.0.
    • Focus on the Network: With a service mesh architecture, you have several microservices that are communicating over a new network. As such, it’s important to understand the network’s criticality: What would you want out of a network that connects your microservices? You want your network to be as intelligent and resilient as possible; to route traffic away from failures to increase the aggregate reliability of your cluster; and to avoid unwanted overhead such as high-latency routes or servers with cold caches. You want your network to ensure that the traffic flowing between services is secure against trivial attack. And you want your network to provide insight by highlighting unexpected dependencies and root causes of service communication failure.

    Conclusion

    Becoming hooked on service metrics is key to ensuring successful delivery of services and meeting end user experiences. With the value service delivers to a business, it’s crucial to ensure that services are always running at their prime. For microservices, implementing a service mesh architecture and following corresponding best practices help ensure successful service delivery.

    — Adam Hert

  • DevOps Challenges and Version Control: What the Survey Results Tell Us

    DevOps Challenges and Version Control: What the Survey Results Tell Us

    Putting DevOps into practice is an ongoing experiment for many organizations. Developers often struggle with continuous integration (CI) or continuous delivery (CD) performance, testing delays and other bottlenecks. While not the answer to everything, version control—that long-established foundation of software development toolchains—has become a central part of DevOps, particularly DevOps at scale.

    This is underlined by the results of “DevOps Challenges and Version Control,” a recent survey conducted by DevOps.com on behalf of Perforce Software, which provides some interesting insight into their experiences. This includes the role of version control systems within DevOps in practice, plus the top challenges the companies surveyed face.

    A total of 467 people from organizations across the globe responded, with roles ranging from DevOps engineer to QA analyst and CTO. Before we look at the challenges and issues they face—not just with their version control system but DevOps in general—first let’s look at the good news: Most version control systems are a good fit for their current DevOps initiatives.

    For instance, more than half of all respondents said the most important features of their current version control system are:

    • Improved collaboration.
    • Improved CI.
    • Better support for distributed teams.
    • Quality.
    • Faster commits.

    These were followed by compliance/security, at 48 percent. These jibe well with the main DevOps challenges they also cited:

    • CI/CD issues (a third cite CI/CD workflow streamlining as their top priority).
    • Prioritizing backlog and development projects.
    • Managing build artifacts and containers
    • Developer complaints about slow commits and merges
    • Working with open source software.

    These are all things that VCS can help fix.  For instance, these days, most version control systems transactions are automation for CI, rather than user requests.

    CI Challenges

    So far so good, but users want more from their version control system. Almost a third would like better integration with the software development toolchain. Other improvements on the wish list include security with fine-grained permissions, handling large files or repos and file access for teams in multiple locations or working remotely and improved CI performance.

    CI is critical to good DevOps and giving developers real-time feedback theoretically improves quality, but if the version control system’s speed and performance does not match demand, then bottlenecks are created. The last thing anyone wants is developers sitting around waiting for builds to happen. The survey’s respondents said testing (28 percent) and waiting for code reviews (23 percent) were the two points in CI that slow everything down the most.

    People Want More from their Version Control Systems

    So, what’s the answer? Version control that combines speed and performance, of course, but also built-in automated code review to alleviate the frustration of time spent waiting for tests and other tasks. Given that 64 percent are based in multinational locations, 7 percent work remotely and a mere 29 percent work in the same office, clearly the version control system needs to support dispersed working environments, too. In many situations where the version control system is being used for multiple artifact types and not just code, this needs to be covered, too.

    Likewise, most respondents have Git somewhere in the organization and understandably so: it is open source, users can get started with Git quickly and it is suited to small teams. However, Git struggles to scale, and, given some of the challenges that respondents quoted, this is clearly something many are experiencing. Enterprise strategies are emerging that give developers the Git interface they’re familiar with, but also the scalability, global replication and high performance for moving files and performing builds.

    With so many perceived challenges cited, it is no surprise that almost half of all those surveyed said they plan to replace their existing version control system. The top reasons cited were scalability issues, poor cross-platform integration and lack of DevOps support, plus the perennial pain point of license costs.

    Understandably, switching version control systems can seem daunting, but equally, based on the survey’s responses, many organizations are clearly living with systems that are expensive to maintain and have productivity issues that are getting in the way of, rather than supporting, DevOps. Conversely, choosing version control system strategies and tools can tangibly and positively impact on achieving DevOps success.

    — Tim Russell

  • Accelerating MVP Development Performance Through DevOps

    Accelerating MVP Development Performance Through DevOps

    The core concept of minimum viable product (MVP) is to get maximum learning and understanding regarding what customers want about a product. Learning and understanding is a continuous process requiring a development process that is incremental in nature. That makes DevOps a favorite for CIOs and everyone else who dabbles with MVP development.

    How can DevOps help in accelerating MVP development?

    Continuous Updates

    Any MVP is built with assumptions about customer preferences in mind. Developers may have a deep understanding of what customers want, but they have no solid proof. The MVP “tests the waters” and measures the depth of the customer reaction. Developers then can develop and refine the application further, depending on the customer responses.

    Building an MVP that can be easily updated is no easy task, however, especially if the MVP juggles too many APIs that collect data from heavy databases. Traditional application development tactics may not work here. It is here that DevOps makes a difference. Its continuous integration and continuous delivery (CICD) tools ensure that any application or MVP can be easily updated according to dynamically changing scenarios.

    In other words, DevOps can help accelerate MVP development.

    Microservices

    Microservices is a concept in DevOps where the application is broken down into several loosely held parts. For instance, when a social media app is being built, its various components such as a newsfeed, stories, chabot, etc., are built separately as microservices.

    The prime benefit of microservices is that the updating, maintenance and bug-fixing process is simplified and easy to manage. For MVP, this translates into easier editing of feature sets to meet changing customer preferences. Also, it breeds an organizational culture of collaboration in which developers as well as applications have to mingle with each other to perform better. Such a collaborative environment is key to accelerated development of MVP.

    Extensive Automation

    DevOps and automation go hand in hand. In fact, zero-touch automation is deemed to be the future of DevOps. Right from continuous business planning to feedback optimization, there is plenty of scope in MVP for automation.

    Needless to say, automation drives operational agility. It also quickens the pace at which routine tasks are done. DevOps adoption for MVP development means developers can make more features and deployments faster within a short span of time.

    A Scalable Foundation

    Infrastructure as code, or IAC, is a key DevOps practice. It works by configuring servers automatically, especially cloud-based servers. IAC works by creating a single configuration file for all the servers. It is like a single template of configuration that can be cloned easily for several servers.

    As a result, DevOps allows applications to quickly ramp up their server capacity in a flash. The same goes for MVP: When the MVP gets the nod of consent and is mature for advanced development, the same server setup can be used as a foundation for upscaling the resources. Also, the same configuration file can be used to set up as many as servers as you want. What is the result? Cloud servers up and ready to use without loss of time.

    DevOps and MVP Development — A Formidable Combination

    Deploying a MVP is an integral part of the development process. DevOps, with its microservices and IAC approach, can help accelerate the pace at which applications are developed and deployed. Additionally, CI/CD tools ensure wide-scale collaboration among teams.

    Integrating a DevOps culture right from the MVP stage of an application will help in the development of a final product that is bespoke to customer requirements. Even if the customer requirements are different, microservices help to quickly modify them and update the application without significant downtime. To sum it up, DevOps is the perfect approach to accelerate MVP development.

    — Dhaval Sarvaiya

  • How AIOps Makes DevOps Less Noisy

    How AIOps Makes DevOps Less Noisy

    For DevOps engineers, “noise” is the enemy of productivity. In this context, the noise we’re talking about is unnecessary or low-priority alerts and notifications that distract engineers from identifying serious issues—and ultimately can cause alert fatigue syndrome, in which alerting systems are ignored altogether.

    Without the application of a well-constructed noise reduction plan, alert noise will end up taking away from the focus of solving the actual problem that may be causing a more critical issue.

    So, how can you fight alert noise? You might be tempted to assume that the answer lies in reducing the number of alerting tools you use, but that’s probably unrealistic. Teams today are dealing with complex application stacks and multiple tools and data sources, all of which can produce their own alerts for issues that may be interdependent or isolated. You simply can’t take tools out of your stack.

    What you can do, however, is analyze and manage alerts more effectively. Dealing with alert noise ultimately boils down to developing an effective problem analysis strategy. Let’s explore how, with a specific focus on how AIOps factors into keeping alerts manageable.

    Reduce the Time in Analyzing Alerts

    While reducing noise is important, DevOps teams risk losing valuable time solving critical problems if they also have to create and/or add systems to properly analyze alerts and their sources and priority level. Maintaining this system also would be a source of unnecessary labor that may reduce the alert noise, but also reduces the amount of time that DevOps staff can work on the solution to the root cause— which brings up another consideration. When coming up with a plan, the amount of complexity involved in adding the monitoring tools necessary to cover an entire application and/or service can be equally laborious compared to not having a solution at all. At this point, we may be merely shifting the labor emphasis.

    This is Where AIOps Comes In

    Now, let us envision a way that a team can gain full insight into a root cause without exposure to alert noise. The team doesn’t need to analyze every single alert, which would include:

    • Duplicate alerts.
    • Alerts from multiple sources (tools, microservices).
    • Informational alerts (warnings).

    Instead, the team can let AIOps analyze and classify the stream of alerts and identify critical issues. DevOps no longer would need to spend valuable collaboration time analyzing every link in the toolchain. When a team has to analyze every tool, the team could produce false positives, redirect resources toward the wrong solution, spend too much time trying to identify the root cause and create excessive alert noise. Let us envision a way in which the team does not need to spend excessive time in analysis and can experience more effective collaboration in service of solving the actual problem.

    AI and machine learning can relieve this manual process through AIOps. Automated collection of data across the application stack, alert correlation, identifying root cause and decision-making can be leveraged by a team to get to the problem immediately without all of the messy work of manual analysis, sorting and identifying root cause. This level of automation allows for needed human interaction through collaboration across teams, which can result in faster resolution, decreased downtime and effective communication of key events to company staff and end users.

    Analyzing to Predict Future Problems

    Another benefit of AIOps and the power of automated data collection, analysis, issue identification and decision-making is the ability to predict future events. A DevOps team can leverage these insights into effective preventative measures. The ability of AIOps to produce these insights will help a team find ways to automate and/or improve operations. Reducing alert noise no longer needs to be a goal because it is happening through a smooth interaction between the human team and the AI.

    There is a growing need for DevOps teams to utilize the power of AIOps. While human teams are not replaceable, redundant and time-consuming tasks are. The ability of AIOps to effectively traverse a complex toolchain with isolated issues to analyze alert importance, correlate alerts, collect and analyze relevant data, identify root cause and produce holistic actionable insights eliminates inefficiencies in problem-solving and reduces alert noise.

    Making DevOps less noisy is a goal with its roots in effective use of information. Alerts serve to inform, but thanks to the increasing complexity of applications, alerts become noise. Leveraging the power of AIOps serves to do a lot of the dirty work for a team so that noise can be reduced—but more importantly, it can make operations run smooth and keep customers happy.

    — Phillip Lorenzo

  • Red Hat Adds Support for Nested Workflows to Ansible Automation Framework

    Red Hat Adds Support for Nested Workflows to Ansible Automation Framework

    Red Hat today announced it is adding support for nested playbooks created using the open source Ansible automation framework within one another, which, in turn, will make it easier to automate complex IT processes.

    Bill Nottingham, product manager for Ansible, said the goal is to enable DevOps teams to reuse existing Ansible playbooks to kick off a series of nested IT jobs and workflows using version 3.4 of Ansible Tower, the console Red Hat provides for creating and managing Ansible playbooks.

    Version 3.4 of Ansible Tower also now makes it possible to slice an IT job into a series of smaller jobs that can be distributed across a cluster to accelerate execution. That capability makes it possible to scale out the processing of an Ansible playbook, he said.

    Finally, Red Hat also announced that Ansible Tower. 3.4 is now able to run on Red Hat Enterprise Linux in Federal Information Processing Standard (FIPS) compliant mode.

    Nottingham said Red Hat expects the ability to reuse Ansible playbooks within a set of nested processes will encourage IT organizations to create more playbooks now that it’s easier to reuse them as a set of modular components. IT organizations can even make launching jobs dependent on the completion of other jobs using a scheduling mechanism within Ansible Tower, he added.

    Although adoption of IT automation frameworks has been uneven, the last few years has seen IT organizations rely more on them to manage tasks as IT environments become more complex. That level of complexity will only increase as IT organizations deploy more applications that are based on microservices created using containers that need to be updated frequently. IT automation frameworks also go a long way toward making up for what has become a chronic shortage of skilled IT administrators. That capability is especially critical as both the number of applications and the rate at which they are being updated continue to accelerate.

    Most IT administrators have limited programming skills, which is why a declarative approach to IT automation in the form of an Ansible open source project has been gaining significant traction. In fact, thanks to automation, traditional IT metrics such as the number of servers being managed per administrator increasingly are becoming obsolete.

    It may take a little while longer before IT automation frameworks are the rule than the exception. But as the IT community now approaches the end of the second decade of the 21st century, reliance on manual processes that were developed at the end of the last century is no longer viable in this era of distributed workloads. In fact, regardless of where applications are deployed, the expectation in the DevOps era is that existing IT operations teams will be able to manage them without having to hire additional IT administrators.

    — Mike Vizard