Tag: devops

  • CD 101 Podcast Episode 2: The Agile Enterprise

    CD 101 Podcast Episode 2: The Agile Enterprise

    And we’re back with another episode of the “Continuous Delivery 101 Podcast.”

    In Episode 1, we covered the three waves of IT disruption—and how these disruptions, combined with the move toward digital transformation, are drastically changing every industry.

    This, of course, underlined how accelerating speed-to-market is clearly more important than ever before.

    In this second episode, we talk about Agile methodologies with expert Dan Rice, who addresses how these practices can enable business strategies to be implemented at greater speed.

    Listen closely and you’ll learn more about what Agile actually means across various teams, roles, product owners and scrum masters, along with related points on DevOps and CD tooling.

    All of these topics are great food for thought as you think about how you can move to build software faster and with increased quality. Yet, if enterprises only focus on these issues, they aren’t tackling or talking about the real problem in their organization that Agile can solve.

    It’s time to take the conversation beyond the team level and talk about how work flows to every involved team, enterprise-wide—specifically, about how initiatives can expand across multiple teams, how to manage shared resources and how to pull in other dependent teams.

    For Agile to work, it has to expand universally and be ingrained further at program level, to understand precisely how work is coordinated and flows across every domain.

    You can listen to Episode 2 of the “Continuous Delivery 101 Podcast” via:

    • YouTube
    • iTunes store
    • Or by searching for it in the iTunes Podcast App under “Continuous Delivery 101 Podcast”

    Thanks for tuning in to the “Continuous Delivery 101 Podcast.” We hope you’ll continue the conversation by posting your comments below or in the comments section on YouTube or iTunes.

    Meanwhile, keep tuning in: We’ll be back soon with another episode!

    — Scott Edwards

  • What Happens Post-Deployment: The Rotating Roster

    What Happens Post-Deployment: The Rotating Roster

    DevOps has become that celebrity who’s always the talk of the town but is largely misunderstood. It’s even quoted in the wrong context.

    A few days back, while speaking at a conference, I was interacting with audience and one of the attendees said DevOps is like the backbone of Agile. I think that’s an excellent way of describing DevOps. In an industry where we are still struggling to implement Agile—and even more so to keep people from drifting—is DevOps ahead of its time? Maybe not.

    DevOps is the set of practices necessary to sustain the fast pace of delivery. The best part of DevOps is there are no levels or certifications one needs to achieve. It’s the stack of practices that you pick and implement and reap benefits. The 7Cs—continuous planning, continuous integration, continuous delivery, continuous testing, continuous deployment, continuous monitoring and continuous operations—help maintain sanity, though.

    One teams had to sustain fast-paced releases with a big customer base. We saw our users increase from .1 million to 2.4 million in three years. Agile, continuous integration and continuous delivery are not just fads for us, but needs for survival.

    Using the Rotating Roster Method

    When we talk about continuous delivery but then focus on just delivery, it’s not enough. It’s equally important to work on what happens post-deployment, such as continuous monitoring and continuous operations.

    Rotating Roster is one practice we use in our organization and this has helped us pull together the whole ecosystem of:

    • Resolving customer issues,
    • Feedback system pumping into out technical pipeline/debt for next releases,
    • Knowledge transfer and learning ecosystem to address attrition, and
    • Broken silos.

    Here’s how we do this. First, we define the scope of the Roster. We picked the following:

    • Production Outage
    • Customer issues/P1
    • Ad hoc requests from stakeholders
    • Data extraction (only tasks out of scope of a separate Data Analytics team)

    Next, we publish a schedule of every two to four weeks, designating at least one person to be on the Roster. It’s essential that the Roster rotates to all or most in the team to build a bus factor or a knowledge transfer ecosystem in the team.

    For each request received, we asked the Roster person to log it in JIRA. This was not just for bookkeeping sake, but as an essential step in line with the Roster objective.

    After every Roster period, we sat down and reviewed these JIRA requests for possible targets to:

    • Automate the task or the effort involved, such as account creation requests automated by writing php scripts, for example.
    • Delegate the task to the right team to cut the effort. For example, for a new data extraction we set up SQL slaves and replicate tables on analytics machines, then delegate the task to them for all future instances.
    • Mitigate the reason for the request, such as the root cause of an outage.

    The mitigation/automation steps had to be picked by development team in their Sprints. Otherwise, usually they become technical debt and lay around in the backlog.

    Since the development team was taking turns on the Roster, they understood the priority and necessity of doing the task. Thus, the technical pipeline was chosen and fed actively.

    Another advantage of the Roster was it showed the team the “big picture” and helped catch early trends.

    The Rotating Roster system turned out to be a big success for our teams; thus, we replicated the model through the rest of the organization. It’s a small but essential step to adopting DevOps culture.

    In one definition, Jez Humble explained that DevOps is “a cross-disciplinary community of practice dedicated to the study of building, evolving and operating rapidly changing resilient systems at scale.”

    I could not have put it in better words.

    About the Author / Kiran Chhabra

    Kiran Chhabra is Vice President of Engineering at Infoedge India Ltd. She has 14 years of industry experience designing and implementing highly scalable and available architectures with optimized continuous integration and delivery in technologies including LAMP, Java and mobile technologies.
    Kiran holds a U.S. Patent and has published technical papers. She is certified scrum master and certified scrum professional, and speaks at various Scrum Alliance & technical conferences in India and the United States. Connect with her on LinkedIn and Twitter.

  • In DevOps, Speed Isn’t Always the Point

    In DevOps, Speed Isn’t Always the Point

    The high-tech industry has fallen into a rut and seems to be struggling to get out of it. This rut is most fervently seen when talking about DevOps, but it exists across the board.

    We’ve fallen in love with the idea that faster is always better. That idea is provably false, and the assumption that everything needs to be faster is false, also.

    Early on, time to market and increased responsiveness to user needs were the driving market differentiators for agile development, DevOps and DevOps tools. There were indeed cases where IT was not responsive enough to business needs—I think most of us were acutely aware of this issue, so dwelling there for a while was not a terrible thing. But responsiveness to business needs is one piece of a much larger puzzle. And we seem to be stuck, with some lip service paid to other strengths of DevOps … as long as it’s still delivered yesterday.

    In my career I’ve worked in two industries where speed of delivery was a tertiary concern for core business systems. To be blunt, when you are putting together a system that will administer contracts expected to last a hundred years or more, you really don’t get that worked up about it being out there tomorrow and “good enough,” you worry about it being right when it does go out, so you’re not still paying for mistakes a century from now that were made by rushing today.

    Of course, there are vendors that don’t want to hear people like me say these things. Of course, there are products with other benefits that are more important than speed of delivery. But as an industry, we’re still too obsessed with speed.

    How we too often see priorities

    Most industries do not even fall into a neat category where they are all or nothing. For most, some teams need to be fast, using agile, CI/CD and fully automated deployment to drive out changes fast (not as fast as some talk—most don’t even need weekly, but fast, nonetheless), while other teams need to be more thorough and cautious and the cadence of releases really doesn’t matter as much as the final result.

    So my advice is similar to advice I’ve offered on similar “hot” topics for the last decade or so:

    You are the rock stars.

    No vendor, analyst, pundit or author knows your vertical market, organization, environment or team the way you do. Do not let them proxy their view of what is “right” over you. Yes, far too many act as though, “If you’re not doing X, you’re behind.” But again, you can look at your industry and get a better grasp of what the definition of “behind” is than someone talking in generalities.

    Know what you need, know what is most important, know how to get to what you need. Don’t do what “everyone” says you should.

    But don’t dismiss the need, either. There are other benefits to these technologies that we have only recently started discussing. Infrastructure as code is an astounding way to transfer knowledge should something happen to critical people, for example. You just have to choose the technologies for the right reasons, and what the right reasons are is dependent upon the organization, not what Vendor Y is selling today.

    We’ve come a long way, and are on the cusp of some very cool stuff, but make sure you know what you need, and how to get it, then keep doing your magic.

    — Don Macvittie

  • The Elusive Definition of DevOps

    The Elusive Definition of DevOps

    How do you define DevOps? If you went out and asked IT professionals at other organizations, could you guarantee their answer would be the same?

    In most cases, the answer to this question is no. DevOps, despite its growing popularity, still lacks a widely agreed-upon definition. There are a few fundamental principles that appear consistently in most explanations of DevOps, such as efficiency, collaboration and improved communication. However, when it comes to the details and how exactly DevOps can be implemented in the software delivery process, differences abound.

    Recently, Clutch, a B2B research firm, conducted a survey of IT professionals to gauge knowledge of and opinions on DevOps. The data backs up the notion that a broad definition of DevOps remains elusive—but that fact shouldn’t hold organizations back when it comes to implementing DevOps.

    Clutch gave respondents four distinct definitions from separate organizations and asked them to pick the one they think best defines DevOps.

    Wikipedia’s definition, recorded in November 2016 and since altered, won the largest percentage at 35 percent. It read, “DevOps … is a culture, movement or practice that emphasizes the collaboration and communication of both software developers and other [IT] professionals while automating the process of software delivery and infrastructure changes.”

    The definitions by Rackspace, Hewlett Packard Enterprise and Amazon Web Services each garnered significant support as well, at 24 percent, 21 percent and 20 percent, respectively. Their definitions emphasized different points or excluded elements of Wikipedia’s definition and you can read them in Clutch’s report here.

    The organizations’ names were left out of the survey and the definitions’ presentation order was randomized to discourage bias.

    No definition won a majority and Wikipedia’s definition leads by only 11 percent. We believe that if there was a “correct” DevOps definition out of these four, then it would’ve gathered more support.

    Why Is a Definition So Tough?

    The abstract nature of DevOps’ definition may be rooted in the movement’s history.

    The idea of combining development and operations teams stems from principles in the Agile software movement, which emphasizes greater communication and collaboration within development teams. By breaking down silos on the operations side, DevOps further streamlined the delivery of product to user.

    If the beginning of the DevOps movement could be pinpointed to a specific moment, it would most likely be a talk Flickr employees John Allspaw and Paul Hammond gave at the 2009 O’Reilly Velocity Conference in San Jose, California, titled, “10 Deploys a Day: Dev and Ops Cooperation at Flickr.”

    Belgian IT consultant Patrick Debois, who had already been frustrated by the issues between development and operations teams, watched the talk online and, with encouragement by American software developer Andrew Shafer, created the first “devopsdays” conference in Ghent, Belgium. The movement (and the term “DevOps”) took off from there, gaining traction worldwide.

    DevOps’ grassroots growth and expansion likely contributes to the lack of a singular definition. The concept was not created by a single organization, nor solidified by a written manifesto, like the Agile movement. In fact, a manifesto may have hurt the DevOps movement, writes Christopher Little on devopsy.kinsta.cloud: “If you have a manifesto for something, then all conversation in the domain will, rather quickly, go back to the manifesto for resolution instead of everybody working together. That’s not handy in an ever-fluid, often on-fire, scale-up fast, resilience-oriented, rapidly changing and evolving IT situation.”

    So, DevOps’ history possibly explains its undefined quality, but that could be its strength.

    Who’s Right?

    In response to Clutch’s data, cloud experts gave differing opinions regarding Wikipedia’s definition.

    David Hickman, vice president of Global Delivery at Menlo Technologies, said, “I think that the word culture is misused. It’s more of a methodology trend than a culture, which pertains more to social elements of organizations and how people relate to each other from a professional and business perspective.”

    Brian Dearman, solutions architect at Mindsight, agreed with the culture aspect: “It being a cultural movement is true. Fifteen to 20 years ago, DevOps consisted of two separate things, with operations and development consistently complaining about each other. The culture is being changed, removing the animosity between each side.”

    Meanwhile, Nick Martin, principal applications development consultant at Cardinal Solutions and organizer for the CloudDevelop Conference, criticized Wikipedia’s emphasis on automation: “The definition may be a little over-prescriptive in talking about automating processes. DevOps can be done without such a strong focus on automation. It’s ultimately about collaboration and getting groups working together in order to solve the same problems.”

    Conclusion

    So, DevOps lacks well-defined and universal guidelines. Should that stop you from implementing it in your organization?

    Absolutely not. DevOps’ benefits are clear for most organizations that implement it. However, it’s imperative to clarify what DevOps should mean for your organization before you start implementation.

    The concept of DevOps is broad, but most everyone can agree that increased efficiency, collaboration and communication are positive changes. However, making a point to hammer out the details with your team before implementation is important. Ideas such as increased communication are futile if every employee has a different approach to achieving them.

    Choose DevOps, but choose it thoughtfully.

    About the Author / Riley Panko

    Riley Panko is a Content Developer and Marketer at Clutch, a Washington, D.C.-based research, ratings and reviews platform for B2B services. She conducts relevant research that aims to help consumers enhance their business and select the services and software best-suited to their needs. Connect with her on LinkedIn and Twitter.

  • Security @ the Speed of DevOps Survey: Efforts Still Lag

    Security @ the Speed of DevOps Survey: Efforts Still Lag

    Enterprises today are in a relentless race to digitize or die. If they don’t digitize business processes, services or internal workflows and their competitors do, they will be at a steep disadvantage—and perhaps soon an irrecoverable one.

    In more and more companies these efforts are being driven by DevOps. According to IDC, the vast majority of businesses today think—and rightly so—that it is essential for them to put into place the right resources and IT processes to support a digital business model. Unfortunately, about 75 percent of organizations don’t think that their IT organization is actually scalable enough to deliver and support a comprehensive digital business model.

    There are many reasons why so many organizations have a challenging time with their digital transformation efforts. Many of these are cultural: IT commands tightfisted control over all information technologies used throughout the organization, yet being unable to deliver the infrastructure and digital services enterprises need to succeed, for example. Or, others within the organization still are unable to see how rapidly the world is being changed by digital technologies and so aren’t taking the necessary steps.

    Another reason is security. That’s because this rapid digitization of business processes can have a profound impact on how enterprises secure their data, applications and software development and delivery. In devopsy.kinsta.cloud’s inaugural annual “Security @ the Speed of DevOps” survey, we queried 255 security IT decision makers within organizations currently practicing DevOps or that are on a path to DevOps adoption. The survey found that enterprises of all sizes are embracing DevOps, cloud, continuous integration and deployment. But in securing these processes, however, we found vast differences in maturity.

    When it comes to DevOps practices, larger companies are outperforming. For organizations with 5,000 to 9,999 employees, a full 90 percent have either adopted or started DevOps practices. However, only 38 percent of organizations with 500 or fewer employees have started DevOps or widely adopted such practices.

    As I wrote in our report on the survey, smaller organizations also have a way to go with automated security testing. Only 6.5 percent of those with fewer than 100 employees have automated large parts of their application security testing, while 29 percent have automated some of their application security testing. For firms with 5,000 to 10,000 employees, 38 percent have automated large parts of their testing, and nearly 50 percent some of their testing. For those with more than 10,000 employees, these figures are 20 percent and 48 percent, respectively.

    The challenge here, for many organizations, is the same with enterprise security in general: the view that security slows things down. For the business, it’s about getting more digital capabilities out as quickly as is possible. It’s the same for developers. Security too often is seen as standing in the way of this.

    When this conflict is experienced at DevOps speed, there is the risk not only of development slowing down but also security processes breaking. To succeed, enterprises can’t take the same approaches to security as they did in the past. They must find tools and processes that are designed for the highly automated and continuous world of DevOps.

    Have a look at our survey and report to see how enterprises are tackling this challenge today.

    — George V. Hulme

  • The State of DevOps Adoption and Trends in 2017

    The State of DevOps Adoption and Trends in 2017

    DevOps has come a long way, and we see a tremendous increase in adoption in the recent years. To outrank your competition and be ahead in the growth curve, it has become an essential element for software-powered organizations. DevOps has taken a center stage in the software development field with no signs of leaving.

    Below is an infographic of DevOps and some of the trends we see happening this year in DevOps:

    DevOps trends in 2017

    Sources:

    http://www.rightscale.com/blog/cloud-industry-insights/new-devops-trends-2016-state-cloud-survey

    https://www.storybase.com

    slideshare.net

    https://en.wikipedia.org/wiki/DevOps

    Google.com

    7 DevOps Trends

    1. DevOps will be the mainstream and garner a lot of attention; hence, 2017 is going to be the “Year of DevOps.”
    2. Along with DevOps, the three C’s—Continuous Integration, Continuous Deployment, and Continuous Delivery—are going to gain a huge momentum.
    3. A number of new DevOps automation tools will be released that change the way we do software development.
    4. “Containerization” also will be in the limelight.
    5. Many software firms will move to a microservices architecture.
    6. Automation and continuous testing will become more popular—and more important.
    7. Must-have tools and platforms including Docker, AWS, GitHub and JIRA will become even more popular within developer zones.

    The Names to Know

    Below are the names of the top DevOps experts you need to follow:

    Name Twitter Profile Who is He/She Klout Score Website
    patrickdebois https://twitter.com/patrickdebois bridging the gap between creativity and technology with a #devops mindset; eternal researcher and pragmatic implementor; cto @townheroes; started @devopsdays 50 http://www.jedi.be/blog/
    Avi Cavale https://twitter.com/avinci ceo @beshippable, wildlife photographer, scotch lover 50 https://app.shippable.com/
    Kris Buytaert https://twitter.com/krisbuytaert Everything is a Freaky #Devops Needs Sushi Problem, Open Source, Infrastructure as Code, Consultant, Speaker, Author, Organiser of #devopsdays, #cfgmgmtcamp 53 https://about.me/krisbuytaert
    Gene Kim https://twitter.com/realgenekim DevOps enthusiast, The Phoenix Project co-author, Tripwire founder, Visible Ops co-author, IT Ops/Security Researcher, Theory of Constraints Jonah, rabid UX fan 64 http://www.realgenekim.me/
    Jez Humble https://twitter.com/jezhumble Co-author of Continuous Delivery, Lean Enterprise, DevOps Handbook; CTO devops-research.com, lecturer @BerkeleyISchool. PGP: https://keybase.io/jezhumble 63 https://continuousdelivery.com/
    Gareth Rushgrove https://twitter.com/garethr Software developer, occasional sysadmin, general web, programming and technology geek and curator of Devops Weekly. Engineer at @puppetize. @gdsteam alumnus 56 https://www.morethanseven.net/
    Denise Dubie https://twitter.com/ddubie Strategic Content Creator @CAInc, editor/writer/techie focused on App Economy, DevOps, Mobility, ITSec.& Mgmt., Cloud; former Network World/IDG tech journo. 50 https://www.ca.com/us/rewrite.html
    Sanjeev Sharma https://twitter.com/sd_architect #DevOps #Agile #Cloud #TechSales #CTO #Architect #Author – #DevOpsForDummies #Blogger #Inventor #Disruptor #IBM #DistinguishedEngineer. 52 https://sdarchitect.blog/
    Donnie Berkholz https://twitter.com/dberkholz Research director for development, DevOps & IT ops @451Research. PhD biophysicist. Open-source OG. Emerging tech, craft beer, and zeroth-world problems 59 https://451research.com/analyst-team/analyst/Donnie+Berkholz
    Nathen Harvey https://twitter.com/nathenharvey VP, Community Development at Chef 57 http://nathenharvey.com/
    Damon Edwards https://twitter.com/damonedwards DevOps, IT operations, and improving how businesses operate is my thing. Rundeck is where I work. 49 https://www.rundeck.com/

    We hope DevOps will be a rock star in 2017 and continue to rule the software industry.

    — TeamShippable

  • Intelligent DevOps for New Age Digital Systems

    Intelligent DevOps for New Age Digital Systems

    Transforming DevOps by leveraging the power of analytics and autonomics

    In a previous blog, I have written about how Quality Assurance is getting transformed to Quality Intelligence using the power of big data analytics and autonomics.

    In this blog, we will discuss how DevOps solutions of the future can become more intelligent by taking advantage of these same emerging digital technologies such as autonomics and analytics.

    Technologies such as predictive analytics, autonomic computing (autonomics), artificial intelligence and robotic process automation (RPA) are already driving unprecedented levels of automation in many other businesses. Combined, these three technologies are driving the next generation of automation—what is commonly referred to as “intelligent automation.”

    Intelligent automation goes beyond basic automation and orchestration of manual tasks to incorporate adaptive self-learning—leading to fully autonomous systems. The figure below (courtesy IRPA) illustrates the evolution of automation as it moves toward full autonomy. The Google car (and other similar solutions) is a great example an intelligent automation system, capable of learning from its environment and adapting behavior accordingly.

    Automation in Traditional DevOps

    Digital systems are driven by new age technologies such as social, mobile, big data, analytics, cloud, internet of things (IoT), artificial intelligence, augmented and virtual reality, genomics, etc.  These technologies introduce significant change and complexity to how modern systems are built and deployed. Businesses that leverage these technologies to transform themselves into cognitive enterprises demand new and intelligent approaches to the underlying processes and techniques to support them. Such processes and techniques need to be more than agile; they need to adaptive and capable of reacting dynamically to frequently changing circumstances.

    Let’s look at some examples of current DevOps techniques and how they will need to evolve to support dynamic digital systems. Automation within traditional DevOps solutions is generally limited to scripting and orchestration as shown above.  Automation adoption levels vary, due to various reasons ranging from complexity to skills and organizational challenges. Maintenance of such scripts is itself sometimes a bottleneck, as applications and environments change rapidly, the scripts need to be updated manually and often become obsolete. New age agile digital businesses, by contrast, demand automation that is able to adapt dynamically and self-heal on demand.

    Further, classic DevOps automation techniques are typically driven by static predefined rules. For example, the criteria for promotion of an application build through different stages of the pipeline is often statically defined. This is a limitation for new age systems where the criteria needs to be dynamic and may vary depending on multiple circumstances. The automation system needs to able to look at past data, keep learning from current data and make adaptive intelligent predictions about the right course of action.

    DevOps for internet of things (IoT) systems is very different than that for traditional software systems. The target environment for production deployment in consumer IoT is geographically dispersed, typically not configuration controlled, may have unreliable network connectivity and may even be fragile. Further, IoT systems generate huge amounts of data that necessitate robust data mining and self-learning (adaptive) techniques not provided by traditional lifecycle automation tools.

    Similarly, customer experience (CX) is a key new metric for digital systems that transcends traditional DevOps metrics such as release velocity and quality. CX data is unstructured, fuzzy, voluminous and volatile. CX-driven DevOps (or CX-Ops) is an emerging discipline that requires big data analytics and cognitive approaches (such as natural language processing or NLP) to decode meaningful insight from such data.

    Hence, as digital enterprises evolve and businesses demand greater agility and flexibility, the DevOps function to support such systems will need to transform as well.

    Next-Generation Intelligent Automation Landscape

    Before we jump into what intelligent DevOps would look like, let’s first look at the different types of automation systems being adopted in the industry in general. The following types of automation are described as part of the “intelligent automation continuum”:

    • “Systems that Do”: These are basic automation systems that replicate human keystroke actions and static (pre-defined) rules-based activities. They also take advantage of descriptive analytics that reflect past trends and patterns. Examples of such systems include speech and image recognition.
    • “Systems that Think”: These use algorithms and intelligence to find meaning in data, handle judgment-oriented tasks using diagnostic analytics and make recommendations based on patterns. Examples of such systems include natural language processing and recommendation engines (such as email spam filters).
    • “Systems that Learn”: These understand context, interpret and dynamically adapt based on circumstances; they typically leverage predictive and prescriptive analytics to solve problems independently. Examples of such systems include self-driving cars and neural networks.

    A Peek at Intelligent DevOps

    So, what would transformed intelligent DevOps look like? Intelligent DevOps automation would take advantage of cognitive and autonomics systems to enable smarter adaptive lifecycle automation based on analytics. In my previous blog, I discussed how QA can take advantage of big data analytics. Intelligent DevOps, to a large extent, depends on such capability.

    Based on the above model, let’s look at the different types of DevOps automation we can create:

    DevOps Systems that Do: This includes traditional DevOps automation systems (e.g. for continuous integration and testing, continuous deployment), as well as pipeline lifecycle automation that are based on static rules (e.g. traditional release management automation)

    DevOps Systems that Think: This includes advanced automation systems such as:

    • Automated automation systems, for example, generation of automated test cases from manual tests (or test models) using NLP, generation of virtual services based on request-response data logs
    • Self-healing automation, for example virtual services (or test scripts) that can auto-update based on change in application endpoints (or behavior)
    • Monitoring of IoT systems, for example “smart homes,” which require continuous use of diagnostic analytics to mine huge volumes of data to understand failure modes and prescribe recovery techniques
    • Automated verification of system requirements based on customer experience analytics
    • Automated generation of test scenarios based on analytics on production logs

    DevOps Systems that Learn: This includes sophisticated test automation systems such as:

    • Adaptive continuous delivery pipeline—Learning systems that analyze past data to manage the pipeline based on dynamic rules. For example, correlate code quality and defect detection and slippage patterns to dynamically decide what tests to be run and what gates to be enforced (vs enforcement of statically defined gates) for different teams and products for promoting application builds
    • DevOps process optimization based on insight across the life cycle. For example, correlation of production log data with past code change data to determine the level of failure risk in different application modules. See more on this in the section below.

    We embody the evolution to intelligent DevOps in CA’s maturity model (shown below), which also aligns with the RPA maturity model described above. Beyond the continuous maturity level, Intelligent DevOps is the driving force behind continuous optimization and improvement.

    Cross-Life Cycle DevOps Intelligence

    Intelligent DevOps enables us to perform process optimization based on analytics from data correlated across the system life cycle, from planning through operation. Each process area generates a great amount of data that is typically analyzed within the process (and sometimes organizational) silo.

    While such analytics is useful in itself, correlation of data across these process areas (see figure below) may be used to provide a wide variety of intelligent lifecycle insight (and process improvement possibilities), such as:

    • Correlating configuration data analytics with code change and defect analytics helps us pro-actively identify failure modes related to code and infrastructure changes.
    • CX analytics (from the Operations process) can be used to validate requirements in the plan and define processes
    • Production log analytics may be correlated to test log data to identify missing test cases

    In the spirit of the “Continuous Everything” philosophy in DevOps, we envision a new solution stream around “Continuous Insight” where analytical insights are generated and acted upon continuously (and autonomously) as processes execute.  We will discuss more on the subject of Continuous Insight in a future blog.

    In conclusion, we believe that intelligent approaches described above will be infused into every aspect of DevOps going forward and revolutionize the way DevOps is performed. Stay tuned my friends, and may all our DevOps be intelligent!

    — Shamim Ahmed

  • DevOps Job Titles Emerging in Enterprise IT Organizations

    DevOps Job Titles Emerging in Enterprise IT Organizations

    IT job functions and, by extension, individual titles inside the classic enterprise have been relatively constant. Various hierarchies of developers labored separately alongside IT operations teams that focused mainly on infrastructure. However, the emphasis on continuous integration and continuous development (CI/CD) within the context of a modern DevOps environment is starting to blur the lines separating who does what inside these organizations. In fact, Phil Guido, general manager of IBM Global Technology Solutions in North America, says new titles such as DevOps engineer and Scrum Master, which better reflect new roles many IT professionals taking on, now are emerging.

    At the crux of this shift is the rise of programmable infrastructure coupled with a need for IT organizations to be more agile. Instead of rolling out a few applications and associated updates a year, IT organizations today are being urged by business leaders to roll out more applications and updates at a pace associated more often with web-scale companies than the classic enterprise. Much of that demand emanates from shareholders and Wall Street to accelerate investment in digital business initiatives that generate higher-margin revenue.

    To achieve those goals, enterprise IT organizations are trying to emulate as many of the DevOps processes employed by web-scale companies. For example, web-scale companies make extensive use of CI/CD technologies and processes to roll out as many new applications as fast as they possibly can. Overseeing those processes are developers who now are accountable for not just writing but also maintaining the application. The theory is that if developers are held accountable for the quality of the application, they will focus more on making sure that application attains a level of quality. The trick is to empower those developers to own everything from the code itself to the programmatic infrastructure it runs on. Because of that requirement, many of those developers adopted a “cloud first” mentality because deploying applications on a public cloud such as Amazon Web Services (AWS) eliminated the need for those developers to engage with IT infrastructure professionals altogether.

    However, most traditional enterprise IT organizations have a much smaller number of application developers. Every minute those developers are not writing code reduces the number of applications an enterprise can roll out. Because of that issue, many enterprises have been trying to find a way for existing IT operations teams and developers to work together hand in glove.

    That requirement, Guido says, is creating a lot of demand for advice from IBM about how to manage DevOps. IBM advises organizations to go slow to ensure they have a few quick successes under their belts before propagating DevOps across the entire organization, he notes. If an organization tries to do too much or move too fast, there’s a good chance that employee resistance to change will only increase. Above all, Guido says, organizations should avoid being too prescriptive in how they define DevOps.

    Of course, not everyone agrees that changing job titles is always the best way to go about fostering change. But human nature being what it is within large organizations, some job titles will have to be reclassified to bring about permanent change.

    — Mike Vizard

  • DevOps: State of the Art in 2017, and Why CDOs Should Care

    DevOps: State of the Art in 2017, and Why CDOs Should Care

    Nine years after the term “DevOps” was coined, the following two questions seem still open:

    From the perspective of the industrialization of work labor also in the software industry, the answer to the first question appears very clear to me: DevOps is the third (digital) revolution of software and IT services, where CI/CD was equivalent of the second industrial revolution as elaborated in more detail in this article. Chances are high that further transformation can be expected to merge it with the fourth industrial revolution, as stated here: “DevOps will underpin the 4th industrial revolution.” That is also why chief digital officers (CDOs) should be in contact with DevOps teams in their organizations.

    Now, I see two important things have happened end of 2016: First, the establishment of DevOps Express, as 14 leading enterprise vendors have recognized the importance of interoperability of their tools for leveraging DevOps stack. The demand for this—the second important thing—has been (in my opinion) totally confirmed by CA’s infographic going viral, which maps a broad selection of tools around the software development life cycle. Adding the lifecycle view has been definitely an improvement beyond an overview of tools as proposed in the Periodic Table from XebiaLabs. Still, we are not done. Let me explain why.

    The next challenge I see is there is yet no agreement within the community what is—or could be—a reference DevOps toolchain (meta) model or process (meta) model. Is it:

    • “Plan, Develop, Test, Release, Operate,” as suggested by CA, or maybe
    • “Plan, Create, Verify, Package, Release, Configure, Monitor,” (Wikipedia) or
    • “Requirements, Development, Build, Testing, Deployment, Execution” (Len Bass et al.)?
    Source: Wikipedia, DevOps toolchain

    In the seminal book, “Continuous Delivery,” this life cycle was suggested as “Commit stage, automated acceptance testing, automated capacity testing, manual testing, release.” While we talk so much about culture in DevOps, we should recognize the need to align on a high-level view on the DevOps life cycle that is well-understood and accepted by most stakeholders.

    To have a track of all this and attract other collaborators, I have created an open format (SVG) infographic and published it as a GitHub project. Dear colleagues at the DevOps Express consortium, this is your call! Let’s join forces and accomplish this together.

    Next Steps

    After we will have (I hope so) a community-accepted DevOps meta-model toolchain setup and all important tools in place on the ecosystem, many more interesting questions will appear:

    • Which of these DevOps tools leverage open APIs to reduce risk of vendor lock-in?
    • Which tools belong to which generation of systems, leveraging optimization of human labor?

    Why DevOps is also CDO’s Business

    The fourth industrial revolution builds on digital process data, right? So if we aggregate all that data spread across and produced by the DevOps tools, we could get useful and holistic analytics and insights about IT in much faster time than we can imagine today.

     

    About the Author / Peter Muryshkin

    1Peter studied M.Sc. in Applied Computer Science and works in management consulting roles focusing on complex analysis and customer satisfaction, like IT Business Analyst and Change Manager. His current scientific research focus is digital transformation applied to the field of software engineering in large distributed organizations. Connect with him on LinkedIn.

     

     

     

  • 4 Top Tips for Developing a New Feature

    4 Top Tips for Developing a New Feature

    As a backend developer at CA BlazeMeter, I recently had the chance to work on a big feature for our web app. The feature required a massive change on our backend code, a big change on our frontend and multiple migrations. You know what I’m talking about—it was one of those projects that developers fear.

    But, I gained a lot of valuable experience from the project. Not only did we deliver a kick-ass feature, I also learned the importance of teamwork and patience when developing something new. Armed with those two values, and the right tools and technologies, you can deliver a new feature quickly and efficiently.

    Here are a few more things that I learned that should prove helpful whether you’re developing a major feature or one that only requires a minor code change.

    Tip 1: Plan to understand the development requirements

    The process of development starts way before you start coding, with planning. After the product manager decides what needs to be developed, R&D needs to understand the cost: how many days (or weeks) of development the feature will take, how many people will be needed and which APIs are required.

    To do that, the product manager needs to conduct at least three “grooming” sessions with backend, frontend and QA leaders. The grooming sessions start with the product manager presenting the product flow and requirements.

    Since no one wants to read a lot of text (except perhaps the awesome readers of this article), the product manager should prepare mockups. They don’t have to be designed or colorful, but they do need to show the team what the feature looks like in his or her vision. During our grooming sessions, we used Balsamiq, which lets you create screen sketches for how the product is supposed to look, feel and work.

    Editing screen sketches using Balsamiq

    After the initial presentation, the sessions should include a lot of open discussion—asking questions, checking for answers and examining compliance, etc. —until everyone is satisfied and understands what the feature requires from them and from others.

    This is the time and place for frontend, backend and QA to give their opinion about the schedule and to alert and flag any potential problems in development or extreme use cases. It’s the role of the product team to answer questions and change the flow and requirements until there are no R&D rejections.

    Afterward, the frontend and backend leaders need to create a feature development plan, sitting down to determine what data they need to transform the product vision into a real thing. Or in other words, which APIs are required for each flow or button, how long each step is supposed to take, including buffers, and the order of development. APIs are the agreed “contract” between the development teams. After determining the contract terms, each team can proceed individually.

    I recommend you write your APIs in Swagger, which turns YAML files into detailed HTML pages with all the APIs.

    Swagger Editor

    This plan needs to be transparent to everyone working on the project. In fact, it’s even better if each project participant can add and describe scenarios. Tools such as TestLink and Atlassian do just that. Then, when the code is ready, developers can go over the list and see everything is covered.

    We will need to stick to this schedule later on, especially when new demands unrelated to the feature come in, such as fixing bugs. Sticking to this plan is the best way to distinguish the wheat from the chaff of the development process.

    Start writing automation ASAP

    After planning, we want everyone to start working, and not only the backend. To do that, the backend can prepare mocks, either through outsourcing services such as JSON servers, which show inputs and outputs, or by writing dummy code.

    I mentioned earlier that we used Swagger to write APIs. We also used it to generate a server in NodeJS/Python, like this:

    Generating a JSON Server through Swagger

    From this moment on, frontend can begin writing and building screens and QA can start writing API automated testing, by using tools suh as JMeter. At this point, frontend and QA have the response codes and response bodies; they know which features to put where; and which APIs they need and how to use them.

    QA also know which data they’re missing, such as field names and field data. Sure, the backend code isn’t ready yet and all of the automated exams will return “red” instead of “green,” but the main issue is making sure everyone is as prepared as possible.

    This includes load and performance testing. You can use open-source tools such as JMeter, which is enhanced by CA BlazeMeter. By uploading our test script into, configuring the number of virtual users and ramp-up time, determining geo-locations to test from and analyzing KPIs to determine trends and the health of our system, we can ensure our product is delivered to our customers without crashing.

    This is also true about sending code to QA: Developers should test their code themselves and ensure that when they commit their code, it works 99.9 percent of the time.

    By starting automation as soon as possible, you can minimize future bottlenecks in the development process.

    Ensure the team communicates and works together

    Like I said before, teamwork is the most basic requirement when creating a new feature. This is because new questions arise during development and because requirements change. Therefore, everyone in the team needs to be in sync all the time, from the product manager and the designer to the developers and QA.

    Open communication lines are crucial because everyone is working on the same product at the same time, but each developer or QA should be working on a different part of the product and on different functionalities. Communication ensures each person’s work is compatible with the rest of the team’s work. This saves time, prevents frustration and redundancy and ensures maximum professionalism.

    To make sure everyone knew what was happening, we used Slack and opened a dedicated channel for the creation of this feature. We also had daily (short!) standups where we discussed what we have and what is missing.

    Opening a Slack channel

    Communication also includes team and assignment management. JIRA, our primary issue tracking tool, offers Kanban-style, Scrum and fully customizable boards, which show the progress of each development task, including the developer assigned to it, and his or her exact status or phase.

    This helps you understand what other people are working on, and just as importantly, where people are stuck. If you can, get everyone to work in an open space and put a large screen in the middle of it. This lets you stay in sync at all times.

    We used JIRA to open and manage bugs as well. This was helpful when multiple QAs found the same bug. It’s important to carefully choose which tickets are opened and put them under one epic, so everyone stays focused.

     

    JIRA Dashboard

    Another important aspect of communication is ensuring everyone uses the same terminology, as sometimes people might use the same term to describe different things or different terms to describe the same things. To prevent mixups that could waste your time, make sure everyone is on the same page. For example, do you say “system admin” or “<name (ex. BlazeMeter)> admin”?

    Work on the same development environments

    It’s important that everyone works on identical development environments and even on the same version of the tools and libraries. Otherwise, it could cause an (in)compatibility mess, including a lot of unnecessary back and forth and even might cause bugs to pop up. This environment needs to be as similar as possible to production, but only on a smaller scale.

    Conclusion

    Now look at the feature that came out of all that work here. I’m so grateful that I had the opportunity to work on it with an awesome team.

    I hope these best practices help you the next time you develop a new feature. Don’t forget to keep going agile after development and delivery by running tests in a continuous environment such as Jenkins for every commit.

    CA BlazeMeter works with Jenkins, ensuring you can load test in the cloud and analyze your data while shifting left. You can also use different APM tools, such as New Relic, AppDynamics and CA APM to alert you about errors before QA, which saves time.

    — Dor Atias