devopsy.kinsta.cloud

Category: DevOps Practice

  • Bots: ‘Like a CLI on Steroids’

    Bots: ‘Like a CLI on Steroids’

    Overheard in a Palo Alto coffee shop:

    “… so, we tokenize the utterances, lemmatize the tokens, vectorize with StarSpace embedding, and run the featurized input through the neural net. The softmax activation layer tells us what the user intent was.”

    “Sorry, you what the what and then what?”

    “Or, you can say the user types commands, the bot interprets those and takes the appropriate action.”

    “Oh, now I get it; it’s like a CLI on steroids!” 

    Calling a chatbot “a CLI” is like referring to a mobile phone as “a portable wireless teletype”:  not even close to a complete representation, yet not entirely wrong. Like a smartphone, a well-trained bot will do as much or as little as the user requires of it.

    Historical Perspective

    The command-line interface (CLI) dominated human-computer interaction for a few decades, until the growing number of options software offered made GUI the new favorite way of talking to computers. Both of these interaction models are based on human to human communication: the operator communicates “an intent” complete with some input “entities,” each inserted in its “slot.”  This is exactly how bots operate, so how is a bot not just another UI?

    Fig. 1: User Interface Elements

    Bots as UI

    While a user interface is merely a human-readable catalog of machine’s capabilities and a set of input slots tightly coupled to a specific application, an NLI-enabled bot acts as our agent: it infers the intent (even multiple intents) from a natural language utterance, identifies any entities in that utterance, such as food: pizza, toppings: (olives, pepperoni) and fulfills that intent, be it by talking to Pizza Hut’s API or by calling the local pizza shop on the phone. Or even by making the pizza itself.

    NLI bots have already made quite an impact on the sales and support side, but how suitable are bot agents for more technical work? How would a chatbot perform in the DevOps world, replete with esoteric terms and complicated interfaces?

    Although there is a lot less training material available (after all, more people eat pizza than reconfigure Kubernetes clusters), there are also a lot fewer ways to express a technical intent, so in the end the technical bots are easier to train and make fewer mistakes than their sales brethren. But is a bot’s ability to classify an intent to deploy a new EC2 instance of specific flavor enough to make hardcore DevOps techies give up their favorite CLI?

    Beyond Interface: Advantages of Bot-assisted System Interaction

    Context memory: Whether a bot remembers your favorite pizza toppings, or you have to say “pepperoni and mushrooms” a couple of times a week is of little consequence. However, if your input looks like this:

    I’m sure you’d rather not have to type that ever again. The machine learning models of the leading bot platforms not only remember your slot values, they also remember the context in which those values were entered, so a well-trained bot is always able to suggest an appropriate value when looking to fill a slot.

    Natural language understanding: Just like a human would not be fooled by variations in the word order or use of synonyms, neither would a well-designed bot. Not having to remember the exact syntax or even spelling translates into higher efficiency.

    Fig. 2: Intent classification and slot tagging

    Continuous training and improvement: The more time a bot spends with a team, the better it gets at its task of helping members of that team, not only due to filling its context memory with details about team members and tasks, but also because of reinforcement learning. In addition to the initial training corpus of data, most contemporary bot platforms feature interactive training mode, wherein a human user can provide a real-time feedback on the bot’s performance.

    Seamless automation: Since it’s an agent rather than a mere translation layer, the bot remembers successfully fulfilled intents and chaining those into a sequence, with or without slot elicitation, with or without branching and decision making is significantly simpler and easier than even the simplest of scripts.

    Collaboration platform integration: Teams typically meet in a collaboration platform such as Slack, WebEx or MS Teams, decide what needs to be done, then go and do it. With a bot on their team, they can get right to the “doing” part without leaving the collab platform; avoiding context-switching improves productivity.

    Built-in analytics: A bot’s main job is to classify intents, fill slots and invoke fulfillment. However, in its spare time it could listen in and take notes. At the end of an event (release, incident, outage, etc.) it could use its NLP savviness to provide a linguistic breakdown of the event: How many questions were asked? How many answers? How many statements? Or, even, What was the level of profanity use as compared to the average? This kind of summary provides benefits for parts of the company beyond bot’s original team.

    Conclusion

    Bots can and do increase productivity in a software development or operations environment. Automation is important, but when done through traditional means such as scripting, its high cost often causes it to be prioritized behind other tasks. The use of an NLI-enabled bot as an agent for interfacing with various back-end systems not only ensures the very best in user interface (“CLI on steroids”), but also minimizes the need to switch contexts and significantly lowers the cost of automation.

    More importantly, the emerging agent-facilitated process control prepares us for the next phase in the human-machine interaction: the post-app world.

    Bots will help us navigate the new world of commoditized services that will replace the familiar apps. They will track our experience and base future selections on our satisfaction with their past choices. What more can we ask for? Aren’t most of humanity’s dreams centered on intent (wish) fulfillment? What are the wish-fulfilling creatures from the fairy tales if not expressions of human desire for the perfect bot?

    — Mike Dimitroff

  • DevOps and Collaboration: Fraternizing with the Enemy?

    DevOps and Collaboration: Fraternizing with the Enemy?

    DevOps is all about collaboration, we hear.
    “You must collaborate,” say managers initiating DevOps adoption programs.
    HR make new posters declaring the corporate values “We value Collaboration & Teamwork.”
    Can’t fail!
    Then when it appears not to be as effective as we had hoped we tell teams they are “Empowered” to collaborate.
    Blank stares!
    When things still don’t appear to be working we tell teams they are “self-steering,” “self-organizing, collaborative teams”….
    Still, we don’t get what we were hoping for.
    Luckily the tool vendors come to the rescue and tell us why it is failing. We need “collaboration tools and software” – Throw some “Slack” at it.
    Can’t fail!

    This is when we usually get frustrated managers knocking on our door asking about the Phoenix Project DevOps simulation. They are moaning about communication and collaboration! When we talk to the teams they are frustrated and moaning about “empowerment” and managers not “walking the talk.”

    In my mind these lack of collaboration skills is not really surprising. Why?

    Impression: Well, it doesn’t help when the word has also historically been connected to global conflict situations. “He faces charges of collaboration,” “fraternizing with the enemy.” Dev & Ops. Traditional adversaries?

    Interpretation: The interpretation of the word itself is also part of the problem. I googled an English Language Help Desk:

    Collaboration: “To work together with somebody in order to achieve a single shared goal.”
    Cooperation: “Work with other people by achieving one”s own goals as part of a common goal.”

    In an HBR article titled, “There’s a difference between cooperation and collaboration,” it was argued  “… Managers mistake cooperativeness for being collaborative,” adding, “… most managers are cooperative, friendly, willing to share information – but lack the ability and flexibility to align their goals and resources with others in real time.” If managers don’t get it then what chance do the teams have when told – “You must collaborate”!

    Induction: Part of our failure to be able to effectively collaborate could come from our schooling. We are chastised at school for asking others for help, for looking to see how somebody else is doing it—”That is cheating”—things are changing with more “collaborative assignments.” Yet, my children (now in their early 20s) were not taught what effective collaboration is, other than discovering for themselves. As my daughter explained about her school assignment, “You have to do it yourself if you want it done because others will slink off and not do it, then my individual grade may suffer.” They are told they need to collaborate but individual performance grades are scored. My daughter once came home stressed and almost in tears, declaring “Oh no! I have been given another collaborative assignment for my most important subject!” Children leave schools distrustful of collaborating.

    Indoctrination and Imperative: There is also the historical perspective. When we, the current workforce entered our chosen profession we had decades of functional silos that created a “them” and “us” attitude. This stimulates a sort of “territorial imperative” or “tribalism,” not helped by our own “technology” languages and tribal cultures. We have learned over time that other teams “dump things on us” or other teams “stop us from getting stuff done.” In other words, Other teams frustrate work.  (Territorial imperative—”the drive in animals and man to take, hold and defend a particular area, zone, turf,” —e.g. defending their best practice framework! And attacking others that come near!).

    Inconsistency: We then built walls between these teams and if they wanted something from each other they needed to escalate it to their manager. Managers would collaborate or not, especially when they were given conflicting, inconsistent goals and KPIs. Development teams are given goals to release as fast and as often as possible. Operations are given goals to make sure everything is as reliable as possible and available at all times.

    I had an example of this “them and us attitude,” once with a board of directors. I split them into two teams and gave each team a pack of ABC cards (Attitude, Behavior and Culture) and within 10 seconds heard, “Oh good, our team will win this!” automatically creating an atmosphere of competition and winning and losing, I hadn’t even explained the task yet! Teams compete.

    ‘It’ factor: A study by Google (2012) called Project Aristotle, named in tribute to Aristotle’s quote, “The whole is greater than the sum of its parts,” looked into why some teams work and others don’t. One of the key conclusions was that “psychological safety” was the “it” factor, “… a sense of confidence that the team will not embarrass, reject or punish someone for speaking up.” Because we have not learned to collaborate and because we have a “them and us” history often accompanied by blame, there is fear, uncertainty and doubt (FUD) about speaking up, sharing ideas or giving feedback.

    In the dark: However, there is another significant reason for poor collaboration, as I have discovered from running hundreds of Phoenix Project simulations with teams globally. It is the fact that nobody has defined the “behaviors” that demonstrate effective collaboration. We assume everybody knows what we mean when we say, “You must collaborate.” Fact: We don’t!

    Below are the results of a series of recent Phoenix Project simulations. One of the goals was to learn DevOps practices such as CALMS and the Three Ways, while another goal was to learn effective team-working and collaboration.

    I asked the teams at the start of the sessions, “What is effective collaboration? What behaviors will I see today that demonstrate this”?  The teams came up with a list of behaviors (each team came up with a top five or six behaviors; this is the consolidated list from seven teams).

    • Transparent communication – “open and honest” and “frequent.”
    • Calm and constructive dialogue (no blame, accusations and shouting).
    • Feedback “asking for and giving feedback.” Constructive feedback.
    • Shared team goals (and cross team). What are we aiming for? Impact on others and their goals. Maybe conflicts that need managing.
    • Listen – is it clear and understood? Show that you listen.
    • Being open for dialogue, being approachable –not closing the door or hiding behind.
    • Giving and taking responsibility.
    • Sticking to agreements.
    • Central moment to get together (e-2-e).
    • Knowing the roles and responsibilities, sticking to responsibilities.
    • Mistakes are OK.
    • Be proactive to get information needed.
    • Equal treatment: “Respect” ideas from ALL
    • Don’t HOPE, confirm.
    • “Willingness.” Confirm and agree.
    • Ownership of responsibilities.
    • Clear agenda and purpose of meetings.
    • Pleasure – celebrating success.

    “Are these behaviors you want to see more of in your daily work?” I asked.

    Yes! Was the answer.

    “Do we now all commit to sticking to these behaviors today, in this simulation?

    Yes! Was the answer. Easy! See there’s nothing to this collaboration stuff.

    However …

    During the simulation it soon became evident that roles and responsibilities were not clear or not being carried out; it was not agreed which information was needed by the business nor when; goals were unclear, meaning that priorities were arbitrary; people did not understand procedures and did not ask for help; assumptions were made (“I told you …,” “I thought you meant …”); people taking responsibility were not respected (“Let’s stop and try and sort this out …”)— everybody just carried on running around! “Headless chickens,” said one delegate; people were not listening to each other; there was a lot of “assumicide”; mistakes were made and people blamed—there was no exploration as to why and what can be done to stop these  mistakes. Mistakes were often corrected by somebody else but without giving any feedback to correct future mistakes; agreements were unclear across the team; there was no end-to-end flow—work and information was ping-ponging back and forward.

    “All in all, just like reality!” declared the teams.

    See how easy collaboration is. Within one hour of making a list of desirable behaviors, they were totally, utterly and universally ignored. Nobody appeared to own these behaviors or to confront each other on them.

    After the Experiences in Simulation

    At the end of the simulation round we looked at the team scores (number of releases, number of successful releases, business value realized). It wasn’t a pretty sight. The CEO wasn’t happy. The team felt stressed, unhappy, frustrated and powerless.

    We explored what this had to do with “effective collaboration.” Had anybody used the list of behaviors? No.
    Had anybody consciously practiced these behaviors?
    No.
    Did anybody give feedback on these behaviors when they were clearly not done?
    No.
    “Giving feedback feels strange, it feels like we are criticizing and blaming,” said one delegate. Another assumption!

    “It’s hard,” said one delegate. “The manager needs to manage this,” said another.
    “It was YOUR list and YOU all agreed this, why is this a manager’s job to get you to do what you said you wanted to do?”
    There was silence. “It’s not easy, we fall back into old ways of doing things, we need help!”

    Is it a Bird, Is it a Plane, Is it a Bus? No, it’s a coach!

    As the simulation progressed, the team refined their list of desirable behaviors based upon their experiences. They were coached for two rounds and consciously practiced and experimented with the behaviors. The coach role became a critical catalyst.

    In the final round the team owned, and were executing on these behaviors without coaching.
    Why were they suddenly executing?
    They felt less stressed; they were trusting each other; they felt the positive impact on their work; they saw, felt and shared the value gained by these behaviors. They felt immediate positive consequences from these behaviors.
    Why no more coaching?
    The team were taking it in turns to lead their own stand-ups; they were giving feedback and correcting their own behaviors. These were the additions the teams made to the behaviors as they practiced them.

    • Transparent communication – “open and honest” and “frequent” (Relevant, and timely as agreed with the various stakeholders).
    • Feedback “asking for and giving feedback.” Constructive feedback. (Related to agreed behaviors and responsibilities, calling stop, then swarming to solve an impediment, constraint, or remove waste; immediate fact based feedback).
    • Shared team goals (and cross team). What are we aiming for? Impact on others and their goals. Maybe conflicts that need managing. Understanding strategic portfolio and how we contribute to strategy, show you understand and agree common goals and prioritize accordingly. Goals for both value creation (features and benefits) and value leakage (risks, debt).
    • Leadership within team – team members took turns in facilitating the stand-up and retrospective. This role was respected.
    • Listen – is it clear and understood? Show that you listen, sender: Ask for clarification, ask to confirm understanding, listener: summarize what was understood and agreed. Avoid assumptions.
    • Being open for dialogue, being approachable—not closing the door or hiding behind. All were engaged and equal in the stand-up and retrospective. All were asked for feedback, ideas, suggestions.
    • Giving and taking responsibility. Respecting team behaviors; e.g if somebody calls “stop,” or “respect somebody taking responsibility for an improvement stand-up.”
    • Sticking to agreements. Confront people to understand why agreement not met and what can be done about it in the future (e.g conflicting priorities, lack of understanding).
    • Central moment to get together (e-2-e). Scheduled stand-ups, retrospectives, and improvement prioritization.
    • Knowing the roles and responsibilities, sticking to responsibilities, confirming your responsibilities, confronting people on responsibilities (feedback). Showing that you take ownership.
    • Mistakes are OK – no blame, so long as it is used as an opportunity to learn and improve.
    • Be proactive to get information needed (or ensure information is available, e.g. visible management board, avoid unnecessary “disruptions).
    • Equal treatment: “Respect” ideas from ALL. Ask people for input, engage ALL in ideas generation and decision-making.
    • Don’t HOPE, confirm.
    • “Willingness.” Confirm and agree – “Willingness” and “ability,” empower to be able. Share knowledge, cross train, coach, help.
    • Ownership of responsibilities. Confronting people on responsibilities.
    • Clear agenda and purpose of meetings. My role, what is expected from me? What is in it for me? avoid “wasting” time in meetings.
    • Team coaching to give feedback on behaviors.
    • Create trust by demonstrating team behaviors.
    • Check understanding and commitment to “agreements” and confront people (feedback) on agreements.
    • Improving own work as a team.
    • Asking for and offering help.
    • Having insight, overview, visibility.
    • Limit WIP (reserve WIP for learning, improving, technical debt).
    • Pleasure – celebrating success. Product owner shared results including customer feedback. Team complemented each other.

    The teams were then asked to make a list of behaviors and actions they wanted to take away and apply. Their line managers and assigned team coaches were there when the list was made and gave commitment to help guide, coach and empower teams to experiment and practice these new behaviors in their daily work.

    The team would prioritize one or two behaviors to start with and consistently practice in the coming weeks. When the new behavior becomes the accepted norm and a “habit” they would focus on more behaviors.  This fits in with the Third Way of DevOps: Continuous experimentation and learning. “Understanding that repetition and practice is the prerequisite to mastery.” And to tie this back to Aristotle the philosopher (not the Google project): “Excellence is an art won by training and habituation … We are what we repeatedly do. Excellence, then, is not an act but a habit.”

    Meetings, Including Training Sessions!

    And finally, one of our industry’s most popular, and wasteful, examples of collaboration are poorly run meetings, which are the biggest time-sink in organizations and a source of frustration. I include some training classes as forms of meetings.

    In more than 100 training sessions in the last year, less than 10 percent have started on time.
    Nine minutes late is the average. Delegates say this is normal in their organizations. When I ask people what their own personal learning goals are for sitting eight hours in this “training meeting,” more than 25 percent had not defined this and more than 70 percent had not agreed this with the “sponsor” of the training.

    We need to manage our overload of wasted meeting effort. How? One way is the ask for goal and agenda in advance, asking what value you need to bring in and what value you need to get from the meeting; also to limit your time to the agenda items relevant for you or your team.

    Efficient collaborators decide when they do, or don’t, have unique value to add—“maintaining focus on what matters.”

    The same applies to training, asking, “Why am I going on the training?” “What am I expected to learn?” “What problem is this training expected to help solve?”  “What will I be expected to do differently after the training?” “How will the learning be transferred back to the workplace?” “Will I be mentored and coached?” “Will I be given opportunities to experiment?” “Will I get constructive feedback?” “How can we demonstrate that the new knowledge delivers expected benefits?” This is better than the most common answer, “I don’t know I was told to come to this training!”

    A training is also a form of collaboration between you, the trainer and the person sponsoring the training.

    There is an “I” in Team, after all. “I” am responsible for my effective contribution to team performance.

    — Paul Wilkinson

  • The Third Way of DevOps: From Knowing to Being

    The Third Way of DevOps: From Knowing to Being

    Why is the Third Way of DevOps so difficult to master?

    We all know the feeling. You’ve seen an opportunity to improve things and genuinely want to do something about is. But you soon feel the clarity and anticipation draining out of you. First there’s the pressure of the day job. Then there’s the inertia of your co-workers who are oblivious of, or indifferent or hostile to, your ideas. And finally, as if these concerns weren’t enough, you’re faced with the difficulty of unlearning old habits and developing new ones. You wonder why you bother. In a way, it’s a bit like a lottery: You buy a ticket fully knowing that the odds are stacked against you, but it’s the worth it for the uplifting short and illusory dream that it might just happen.

    This is, of course, all about the Third Way of DevOps—creating a culture that fosters two things: continual experimentation, which requires taking risks and learning from success and failure, and the understanding that repetition and practice are the prerequisites of mastery. It all sounds great in theory, but why is it so difficult to put into practice?

    Information Abundance  

    It’s not due to lack of knowledge and skills. We have more information available to us and more easily accessible than ever in the history of our planet. We can thank technology for that. But has it really made us more effective at what we do? If I know something and don’t apply it, how is that so different than not knowing it at all? If I know how to read but choose not to, how is that so different that not knowing how to read? If I don’t use what I know, how long will I retain it and how will I integrate it with the other things I know? Most importantly, can and will I apply what I have learned to make things better?

    Lifelong Learning 

    We have been learning and studying all our lives in an attempt to make things better: our ability to understand IT, frameworks, methodologies, capabilities, practices, work processes and, of course, tools. Consider how many thousands of hours you invested in school, self-study, read out of curiosity, listen to podcasts and do work-related training and certification. Most of my friends proudly describe themselves as “lifelong learners” and one claims the day they stop learning shall be the day they die!

    Training in Skills

    We’ve also invested considerable time and effort into improving ourselves so that we may work more effectively with others: communication skills, teamwork, shared rituals, structured problem-solving, leadership and coaching all fit into this category. Who (in IT) has not been a part of some Service Management, Lean, Agile, Scrum, Kanban or DevOps training? After all of our training, you’d think we’d be further along than where we find ourselves.

    Knowing and Understanding 

    Knowing and understanding are not the same thing—I can know the Four P’s of IT Service Management (People, Process, Products and Partners), yet not view it as a lifecycle model and understand that to obtain the benefits of this knowledge, my team must determine the roles of people and objective of work of processes and then implement tools to automate the processes enabling people’s roles and tasks.

    Understanding and Doing

    Nor are understanding and doing the same thing—I can know that structured problem solving is based on the scientific method and can be broken down into four stages (Plan, Do, Check, Adjust). I may also understand that structured problem-solving is preferable to the reactive educated guesswork my team engages in whenever it encounters a problem. But if we do not change our behavior when it comes to problem-solving, our understanding will not manifest as action.

    Doing and Being 

    Neither are doing and being the same thing—I can be doing something and still not have internalized it so that it becomes who I am. I can know the core principles DevOps (engage in systems thinking, amplify feedback loops, foster a culture of experimentation and learning). I may understand that all three principles must be applied to foster a sustaining DevOps environment. I may be even be holding initial planning sessions with my team during which we map out the DevOps value stream, identify bottlenecks, create feedback loops and introduce changes to create a create a culture of experimentation and learning.

    Moment of Truth 

    But what happens when we meet with our next P1 incident? Does the knowing, new understanding and behaviors get tossed aside while we fix the problem, or do we hold fast to the new way of doing things as we grapple to not only remediate the incident but to view it within the context of DevOps?

    So, What’s a Person To Do?

    The velocity and degree to which we can move from knowing to being determines how effectively we can apply what we know. Here are a few things you can do to smooth the transition from knowing to being and help put the Three Ways of DevOps into practice:

    1. See and feel the potential impact from moving beyond knowing to being
    2. Use the uplift to get motivated
    3. Take action by changing something within you and your team’s circle of control
    4. Reflect on the fact that you have realized potential (this is very powerful when you share it with your team)
    5. Check and adjust based on the outcomes of step #3
    6. Allow the feedback from the previous step to motivate you to carry on!

    Curiosity and Humility

    This approach requires a certain attitude. One of curiosity and humility. You need to be inquisitive so you keep striving to understand. And you also need to be unassuming and you see yourself as having plenty of room to learn and grow; being deeply (almost obsessively) interested in what’s going on and how things could be improved; trying to be aware of and to set aside any preconceived ideas—adopting a beginner’s mind.

    To Be Continued …

    In our next post, we’ll explore specific steps you can take to leverage The Third Way of DevOps and make tangible progress based on what you know and don’t know, understand and don’t understand, and do and don’t do.

    This article was co-authored by Mark Smalley.

    — Mike Orzen

  • Why Banks Must Ditch the ‘Build vs. Buy’ Mentality and Embrace a Platform Approach

    Why Banks Must Ditch the ‘Build vs. Buy’ Mentality and Embrace a Platform Approach

    The past decade has seen digital transformation change nearly every aspect of our lives, and banking has proven no exception. Indeed, proprietary data from Avoka shows that in mid-2017, mobile overtook desktop as the most popular channel for applying for new deposit accounts at one top 10 U.S. bank. Banks have accordingly ramped up their IT departments and technology investments to adapt to this brave new world and digitize their offerings, with the aim of making their financial products just as accessible and convenient as products offered by tech customer service giants such as Amazon.

    The push to make banking products digitally ready (and very quickly) has spurred the old “buy versus build” debate in bank IT departments: Should we build our own digital onboarding software from scratch in-house, or do we buy off-the-shelf solutions from third parties that may be generic and not customizable to our brand and product offerings? While this dichotomy may have been a suitable mentality years ago at the start of the digital transformation revolution in banking, it no longer fits with the reality of today’s more complex development landscape and is starting to cause significant problems in bank IT departments.

    It’s no mystery that the majority of today’s banking IT projects use elements of both “build” and “buy.” But banks tend to opt for the “build” option in the main, reasoning that in-house development drives better outcomes. But this is a serious misconception—recent research from Forrester shows that more than half of all banking customer experience projects take longer than expected, leading to a blown project budget an overwhelming 88 percent of the time. A critical point: Projects using or re-using internally built components are far more likely to suffer these issues.

    This inevitably leads to a spiraling buildup of technical debt—a concept that banks should familiarize themselves with and treat just as seriously as actual debt (after it, it ends up in the bottom line—total cost of ownership is higher for internal builds around 64 percent of the time, as per Forrester’s research). A snowball effect takes hold: Projects with problematic delivery are far more likely to require endless catch-up fixes and high maintenance costs, draining time and resources from the development team that otherwise could be put to use on the increasingly crucial work of updating the offering to meet the continually evolving needs of the market and customers. On average, one-third of customer-facing software releases are updated only once a year or less, a number that rises closer to half for projects that are over-budget.

    Forrester’s research shows that this isn’t a matter of under-resourcing; teams report being allocated the right amount of time and resources. Rather, stumbling blocks stem from architectural inflexibility and poor technology, as well as changing business priorities sapping stakeholder enthusiasm and project velocity.

    Neither “build” nor “buy” in the older sense of the two approaches provide an adequate strategy for today’s development environment. The aversion to buying mainly took hold during a period when third-party products were fairly self-contained black boxes—off-the-shelf offerings that provided none of the flexibility of in-house and came at a cost to branding. However, this is no longer true of the third-party vendor landscape. The advent of platforms and APIs means it is now far easier for banks to bring in multiple fintech integrations to fill the gaps in their offering in a modular, “mix and match” manner—all fully customizable, extendable, flexible and capable of being brought together on an organized, neat platform.

    Banks need to embrace this change and ditch the old “buy versus build” debate that so often prioritizes internal builds as a knee-jerk reaction. The constraints of the old build or buy choice means that the tendency is still to think of digital transformation as a series of off-one projects, when the most successful efforts treat it as a holistic, continuous evolution. After all, customer needs and expectations are constantly evolving, and so should customer-facing software.

    So, how should bank development teams and their stakeholders approach a new digital transformation project? The answer lies in BBEA: Buy, Build, Extend and Assemble. The rule of thumb here is that where services and capabilities are generic and not unique to the bank, they should be sourced from expert vendors with a proven ability to get the job done quickly and in an expert manner (why incur a longer development cycle and higher TCO by reinventing the wheel and building internally?). This frees up the bank’s own internal resources to work on building elements that are truly unique to the bank’s value offering. While it used to be prohibitive to pursue both of these approaches simultaneously, this is precisely what the new world of APIs and platforms enables.

    Sourcing and extending components in this way reduces technical debt significantly, and spreads the cost of R&D across external players. The platform approach makes it far easier for banks to tailor these external components to their own brand and specific circumstances, and makes adaption to changing future needs far less onerous, removing the tendency to revert to big and costly start-from-scratch projects on a regular basis.

    Banks that want to get ahead in their digital transformation strategies must their ditch old development habits and embrace the new platform approach. Buy vs. build is dead; long live BBEA.

    — Steve Demchuk

  • The DevOps Role: Not a One-Person Job

    The DevOps Role: Not a One-Person Job

    This is actually a follow up of the article I wrote a little while ago on LinkedIn titled “DevOps is NOT Jenkins.” In that article, I ranted about how many people and companies confuse automation tools with DevOps. Now I want to address another misconception that I come across quite often: “DevOps is a role.”

    When managers tell me they have all bases covered because their team consists of developers, designers and a DevOps engineer, many questions come to my mind. Having the awareness to at least assign that role in a team is already something. Quite a few engineering managers think that by giving an engineer a DevOps title is enough, but nothing could be farther from the truth. As I mentioned in the LinkedIn article, DevOps is a culture, a process and a philosophy. Adding a DevOps engineer to the team is like declaring someone on the team to be the scrum master and assuming agile is in place. These days, Agile is widely adopted and usually it is not seen as a role, but as a proper methodology. (Although it never surprises me when I encounter a team that is micromanaged while a scrum master is on duty …)

    Like Agile, DevOps should be practiced by the team or organization as a whole. At the core of all engineering we should not only address features but also how those things will run in production. And let’s not forget quality assurance, security and compliance when designing a feature or fix. Fixing an issue at the beginning is between 10 and 50 times cheaper than leaving it until all the testing phase is done already, according to a NASA study. “The study revealed that costs escalate in an exponential fashion.”

    I suspect that concentrating on just the feature or fix is so much more convenient because it is the immediate business value that is addressed by completing it. Thinking about how a new configuration variable will be injected in production or how this new function is going to be load tested probably takes away from focusing on a adequate and speedy completion. There is even a possibility that taking every possible aspect into account simply is too much to fit in one’s brain. I know I’ve had to force myself to not forget anything with checklists and tools.

    In Practice

    Every change, feature or bug fix should have a checklist to make sure all the bases are covered. Naturally, the requirements for the change have to be captured with user stories. But also make sure there is a user story for each of the non-functional requirements. Here are some examples of user stories addressing the various concerns:

    • As a security officer I want to be aware of any changes in potential security risks so I can adapt my (automated) security checks accordingly.
    • As a quality assurance engineer I want to be able to verify the functionality of this feature/fix in isolation and integrated in the product.
    • As an operations engineer I want the to be aware of any changes in the runtime contract so I can update deployment systems accordingly.
    • As a compliance officer I want to reduce to scope for which compliance regulations are in effect.

    There are plenty of tools out there that allow these user stories to be part of a template so it’s easy to embed into the team’s workflow.

    Runtime Contract

    Defining a “contract” for how the application is configured and run in the target environment when the application or microservices are developed removes many uncertainties at the design phase. The mere fact that such a contract exists forces you to either comply with it or negotiate an amendment, like with any contract. A runtime contract has to define how the runtime is configured and which elements are considered secrets. It also needs to describe how and in which format the service outputs runtime feedback around errors and informational messages. Metrics, startup and shutdown and health checks also must be considered. This is an example of what such a contract could look like.

    Broaden the Role

    After checklists and contracts have been established, a large part of the process is codified. I typically write these things down in a version controlled document structure so amendments can be easily tracked. However, this still doesn’t relieve us from the misconception that DevOps is just a role in the team. In fact, the DevOps culture within a team or organization probably should be guarded and or facilitated by a person or persons, much like the Scrum master is tasked with facilitating that process and methodology. Probably, the actual automation will be built in large part by the person tasked with the DevOps responsibility, but on top of that it is also their responsibility to raise and maintain awareness of all the aforementioned aspects of testing, deploying and running an application.

    As such, I believe that people practicing DevOps are required to be more than average communicators and well-versed in every step of the software delivery life cycle (SDLC). Almost all of the DevOps job descriptions that pass my eyes take great care to mention all kinds of technology and familiarity with clouds and programming languages, but don’t emphasize the communicative requirements. It’s time to take DevOps seriously and start reaping real benefits by embracing it as a methodology.

    I’d love to hear your opinions and feedback. Let’s start a discussion below in the comments or on LinkedIn/elexy.

    — Alex Knol

  • DOES18 Las Vegas: Video Interviews, Day 2

    DOES18 Las Vegas: Video Interviews, Day 2

    DevOps Enterprise Summit 2018 (DOES18) in Las Vegas was a three-day bonanza of all things DevOps, with conversations spanning the entire DevOps ecosystem, from processes to tools to culture. In my two days at the Broadcast Booth (aka “The Fishbowl”), I had the pleasure of meeting with some very smart people and having some highly insightful conversations, all of which  are available for viewing on DevOpsTV.

    The second day’s conversations at DOES18 were as interesting as the first day’s, with topics ranging from DevOps culture to managing DevOps in highly regulated environments. Below is a recap of the Day 2 video interviews at DOES18 (Day 1’s recap is available for viewing here):

    Organizational Change

    Carmen DeArdo of Tasktop and Jon Smart, formerly of Barclays Group, stopped by to discuss tactics in ways to make change efforts possible in the enterprise.

    Capital One’s DevOps Update

    Jamie Specter, Aimee Bechtle and Abbie Gray of Capital One offered the latest updates from legal, tech leadership and product management in its DevOps journey.

    Work Management Systems

    Scott Prugh of CSG, Keanen Wold of Delta Airlines and Rosalind Radcliffe of IBM discussed how to break down inefficiencies for enterprise practitioners and management leaders who have multiple work management systems that their teams deal with every day.

    Project Management

    Laura Fay of TSIA discussed a product-led growth strategy where the buyer and customer journey is actively engineered into the product experience to deliver value and outcomes. She also explained how DevOps can benefit from expanding the definition of lead time from ’code to cloud’ to ‘concept to customer consumption’ and become a more integrated part of the business value flow.

    The Importance of Culture

    J. Paul Reed of Release Engineering Approaches and Dominica DeGrandis of Tasktop discuss what’s important about culture as part of a DevOps transformation journey, and why a DevOps culture should foster an attitude of open-mindedness, cross-pollination of skills and stepping out of comfort zones.

    DevOps in Regulated Industries

    Robin Yeman of Lockheed Martin and Ken Moser and Melinda Solomon of the U.S. Citizenship and Immigration Services discussed how DevOps can be applied in highly regulated environments and highlighted industry best practices.

    Transformational Leadership – Making Change Happen

    Christophe De Boeck of Kiwibank Ltd. and Steve Mayner of Scaled Agile highlighted considerations for the different ways to lead transformation in large enterprises.

    DevOps Trends – What Does the Future Hold?

    In this forward-looking discussion Eric Robertsonof CollabNet VersionOne, Robert Reeves of Datical and Derek Langone of XebiaLabs chatted about what the future holds for DevOps in the enterprise and what to be on the lookout for in the coming year.

    — Charlene O’Hanlon

  • 10 Years of DevOps—Transition, Growth and the Future of AIOps

    10 Years of DevOps—Transition, Growth and the Future of AIOps

    When software development became a cumbersome task of repetitive coding, testing and glitches, organizations had to come up with a solution.

    The solution? An organized approach that would not only eliminate the endless cycle of repetitive tasks but also would establish proper coordination and execution in the software development life cycle (SDLC).

    That very systematic approach with fewer glitches, more production and exemplary coordination today is called DevOps.

    But where did this wonderful process start? How did it make its pathway into our the cycle of IT industry?

    The Beginning

    It all began one day when Andrew Clay Shafer scheduled a session, “Agile Infrastructure,” at the Agile Conference in Toronto.

    However, the DevOps model back then didn’t seem to make much of a difference to those associated with software development. As a result, Shafer closed the session. But something struck developer Patrick Debois that day, who was presenting a paper on “Agile Infrastructure and Operations” in the same vicinity. Shafer’s theme seemed to draw his attention and he went straight to meet Shafer.

    A long talk in the hallway that day ignited the fire of DevOps, which started with Shafer and Patrick three months after their first meeting.

    Together, they started the Agile Systems Group, with an aim to foster greater collaboration between developers and operators.

    Today, the DevOps process continues to survive on the principles of increased collaboration and automation to build, test and release software faster.

    Mobile Device and DevOps Adoption

    Over the years, DevOps began to grow and pick up fame. And with the emergence of mobile phones, the trend picked up rampant speed.

    According to a market intelligence company App Annie, global economic activity concerning mobile applications will be worth USD $6.3 trillion by 2021, five times 2016’s revenue.

    Creating software for mobile devices requires an elevated software development approach. Now, organizations have to deliver new, high-quality software to customers as soon as possible to remain atop the competitive market.

    In this new approach, organizations need to focus on two prime factors: agility and adaptability. While the former defines the ability to code and recode, the other marks DevOps’ ability to include changes along the SDLC.

    Companies are constantly working on getting these qualities into their workflows to promote DevOps concepts of continuous delivery, continuous integration and eliminating bugs to allow users to enjoy a hassle-free software experience.

    DevOps Over the Past 18 Months

    According to Gartner, about two-thirds of organizations were using DevOps to support the certain business process by the end of 2017, which Forrester termed as the “Year of DevOps.”

    Forrester awarded the term to DevOps owing to increasing clients’ questions and discussions around, “How do I implement DevOps at scale?” rather than just, “What is DevOps?”

    Going by DevOps’ potential to support modern business processes, more and more companies are adopting DevOps, leading to a steady rise in the trend.

    As DevOps continues to be successful within and across the organization, the IT industry will continue to see what it looks like to be “DevOps successful.”

    The Emergence of DevSecOps

    While DevOps continued to change the software industry, however, security within the process still remained an unanswered question.

    In 2012, Gartner analysts Neil MacDonald and Cameron Haight introduced the concept of DevSecOps. This meant security’s introduction right from the beginning and continuing throughout SDLC.

    Security’s entry into the model made DevOps a more alluring practice. Six years out, DevSecOps has been a much-discussed concept across seminars, conferences and written content.

    But it is still a question whether organizations are implementing DevSecOps considering the time constraint, the need for accelerated delivery and the understanding of security’s importance in the cycle.

    For many organizations, security is still not a part of their production cycle. But, it is essential for any organization to consider security if they want to remain in the competitive market and importantly at the top.

    Negligence in this area may cost them bankruptcy or even elimination from the industry. According to the “Global State of Digital Trust Survey and Index,” one-third of British customers stopped using an organization’s services owing to a security breach.

    10 Years of DevOps: What’s Next?

    The emergence of AI and automation have made some waves in the IT industry, and DevOps is no exception.

    DevOps holds high predictions for the future. In the years to come, AI and automation will change the software development process. Apps will be self-written and coded. Tasks such as integration, security, testing and deploying will be dictated by AI, data and machine learning in the all-new AIOps.

    A data-driven approach will have software dynamically adapt its behavior on the basis of what is in its environment, much like other AI-driven technology including Apple’s Siri and Alexa.

    Moreover, this significant shift from programmed to non-programmed apps will call for improved security and functionality management and allow developers to focus on value-addition activity.

    DevOps is evolving. How about your organization?

    — Veritis

  • Cisco Looks to Build DevOps Community

    Cisco Looks to Build DevOps Community

    Cisco Systems is making a concerted effort to create an era of détente between networking professionals and the rest of the IT operations teams that have embraced DevOps practices.

    Susie Wee, senior vice president for the DevNet community at Cisco, said the goal is to provide network operations (NetOps) teams the skills required to programmatically expose a range of self-service capabilities to developers.

    At the same time, Cisco is encouraging developers to deploy applications directly on routers and switches that now incorporate general-purpose processors. Those processors would run applications in a way that significantly reduces network latency, while the custom ASICs that Cisco develops would continue to handle all the networking functions.

    Wee said that dichotomy is now driving Cisco to classify two classes of developers within its DevNet community. The first is aimed at traditional application developers and the second community is more focused on managing infrastructure as code.

    Cisco is now extending that latter initiative into the realm of software-defined wide area networks (SD-WANs). The company has added integrated firewall, intrusion prevention and URL-filtering technologies to its SD-WAN platform. In addition, there are code samples, videos, learning labs and sandboxes available on Cisco DevNet that are intended to make easier to learn how to programmatically manage Cisco SD-WANs using open application programming interfaces (APIs).

    Cisco claims to now have 530,000 developers participating in DevNet. One the one hand, the company is trying to entice NetOps teams that have been reluctant to give up legacy command line interfaces to modernize their networking environments. On the other hand, Cisco is trying to attract more application developers who increasingly find themselves encountering latency issues as they move to deploy applications in highly distributed computing environments, such as an Internet of Things (IoT) application.

    Over time, Cisco and other providers of network operating systems also will be moving away from monolithic architecture to embrace microservices, which would make it possible to programmatically invoke networking services on a more granular level.

    Of course, there may be some developers who would prefer to programmatically take control over networking along with the rest of the IT infrastructure. But Cisco is clearly betting most organizations will want to make it easier for their NetOps and DevOps teams to work more closely together. Today, most enterprise IT organizations can spin up a virtual machine in a matter of minutes; provisioning the network connections to that virtual machine, however, is still measured in days and weeks.

    Other networking infrastructure companies besides Cisco have realized the need to make networking more programmable. But in terms of the resources the company is making available to solve that problem, it would appear to ready to make a significantly larger commitment.

    — Mike Vizard

  • Stop the Domino Effect by Joining Build Engineering and DevOps

    Stop the Domino Effect by Joining Build Engineering and DevOps

    As DevOps has become more common, these teams are tasked with uniting everything from coding to deploying and monitoring applications. Build Engineering forms the foundation for all those processes and is essential for building code. Its tools vary according to the operating system you’re building on and by programming language. DevOps needs to work with Build Engineering, but the two don’t usually connect. And that’s a problem.

    Over the last 10 years, there’s been a shift toward DevOps. Previously there was a discrete division between coding, done by developers and engineers, and deployment, done by Ops. However, it became apparent that for companies to optimize benefits such as productivity, development and operations shouldn’t be so disparate. The result of which is DevOps.

    Along with the shift to DevOps have come several trade-offs—namely trading security for speed, certainty for agility and proprietary code for open source:

    • Speed: Getting to market quickly has become critical to success. Speed now takes greater priority over fixing vulnerabilities, which can always be addressed in the next release (hopefully). And retrofitting for security and vulnerabilities after the fact becomes a big blocker for development and engineering teams. At the end of the day, the risk of lost profits is greater than the risk of potential unknown threats.
    • Agility: Agility trumps certainty in the race to market. For instance, iterative pushes to production do away with fixed, known road maps in favor of implementing whatever this week’s most important feature/functionality is. The trend toward immutability is also important here; it’s generally simpler and quicker to throw away something that’s broken or outdated rather than trying to fix or update it.
    • Open source: Due to the greater innovation, reduced costs and transparency it delivers, open source has won over proprietary code. Rather than focusing on creating proprietary code, developers can quickly iterate on a solution by assembling open source code, accelerating time to market. Plus, the ability to take from what has already been created can free up your dev teams to innovate and build IP for your enterprise efforts.

    For most, the tension between fixing something and knowing if it will break something upstream hasn’t been eliminated. You may be able to quickly update and patch a single solution, but how will this impact you once you push to your CI stream? Even for mature DevOps organizations, there’s a hidden opportunity cost associated with two factors that are overlooked: open source programming languages and Build Engineering.

    Overhead is added to your DevOps teams when you retrofit your open source language of choice with new versions, dependencies, security patches and so on adds. Teams should be focused on the things that matter—creating and delivering the latest release.

    On top of that, Build Engineering takes a hit each time the underlying open source language is retrofitted. You need to rebuild all your developer environments, CI/CD systems and production environments. All of this build engineering work is manual, slows down time to market and eats up valuable engineering resources. Plus, you’re not certain your rebuild and update will provide performance improvements that warrant the update.

    Since DevOps already relies on automation, it makes sense to apply this tool to the retrofitting and building of your open source languages and seamlessly connect to the rest of your DevOps cycle. You could ensure build reproducibility, dependency management and compliance with your corporate security and license criteria. For example:

    • Build impact –  What if you could understand the performance improvements (if any) that could be gained by taking the latest build?
    • System update – Consider the benefits of being able to update all of your impacted systems across dev, test and even production environments with the latest build using a single command.
    • Packaging – If each continuous build could be packaged in multiple ways—any form factor for any operating system—how would it change your workflow?
    • Build validation – Imagine that every continuous build could be vetted against your smoke test, inclusive of compliance with your security and license criteria, so you knew whether it was ready to deploy.
    • Continuous builds  – What if every time a new version of a library or patch for a vulnerability was found, a new build was kicked off and you were notified when it was done? Then you could keep up with your continuous deployments.

    The domino effect is a phenomenon that DevOps must deal with on an all-too-often basis. Dev and Ops came together for greater productivity, but unforeseen dependency issues regularly gum up the works. That’s the reason automation is such a useful tool, one that you can used to stop the domino effect as you automate your build pipeline for open source languages. It’s truly possible today to benefit your business by ensuring build reproducibility, dependency management and compliance with your corporate security and license criteria.

    — Bart Copeland

  • IBM Allies with ServiceNow to Bridge DevOps Divide with ITIL

    IBM Allies with ServiceNow to Bridge DevOps Divide with ITIL

    IBM and ServiceNow today announced an alliance under which IBM Global Technology Services (GTS) will collaborate with ServiceNow to make DevOps processes and events managed via the IBM Cloud Migration Factory visible to a cloud-based IT service management platform from ServiceNow based on the ITIL framework.

    Jim Comfort, general manager for cloud services at IBM GTS, said that from a governance and compliance perspective it has become clear IT organizations need more visibility in to DevOps processes. In general, enterprise IT organizations are increasingly adopting DevOps processes to accelerate application development and deployment, while still relying on ITIL to manage governance and compliance, he said.

    IBM GTS, via the alliance with ServiceNow, will be able to bridge that divide by integrating ServiceNow with a set of tools and services IBM provides across hybrid cloud computing environments that are based on DevOps process.

    The IBM Cloud Migration Factory is based on a methodology IBM developed to move applications to cloud. IBM Services claims it has completed more than 100,000 migrations. In addition to allying with ServiceNow, a new toolset is being added to IBM Cloud Migration Factory that employs algorithms based on the migrations IBM has completed that improve planning accuracy and uncover application dependencies. Included in that portfolio is cloud brokering software that IBM gained via the acquisition of Gravitant in 2015.

    Comfort said the alliance with ServiceNow will make it possible to have DevOps processes and events register automatically in the change management database (CMDB) provided by ServiceNow. That’s critical because it should allow IT organizations to maintain governance and compliance without having to impose an overhead that otherwise would slow down application development and deployment using DevOps processes, he said, adding there’s no need to impose an artificial construct on top of a DevOps process.

    As hybrid cloud computing continues to evolve, Comfort said IBM GTS expects IT organizations will be relying on a mix of ITIL and DevOps processes for the foreseeable future. In fact, he noted that while managing one to ten DevOps processes is relatively simple, keeping track of hundreds of DevOps processes requires a more static set of processes to maintain governance and compliance. The IBM approach is to create a unified approach to integrating DevOps and ITIL processes that doesn’t require organizations to integrate a variety of point products on their own.

    Proponents of DevOps processes have been making a case against ITIL processes that they tend to view as being too rigid to support agile application development. Comfort said the next iteration of the ITIL framework will incorporate many of the concepts advocated by DevOps proponents. But the two frameworks will remain distinct.

    The alliance between IBM and ServiceNow represents a clear signal that DevOps is going mainstream in enterprise IT environments. The challenge now is finding a way for organizations that have spent more than a decade adopting ITIL processes to embrace DevOps processes with a minimal amount of friction possible.

    — Mike Vizard