Category: Blogs

  • Enterprises still struggle to leverage IT as a tool to innovate

    Enterprises still struggle to leverage IT as a tool to innovate

    When trying to build great organizations today, it’s no longer sufficient to simply be agile enough to respond to meet today’s market demands. No. While that may have been good enough a decade ago. Today, enterprises need to be able to anticipate market changes and use technology to innovate ahead of the market.

    As readers of DevOps.com know, DevOps, continuous integration, and continuous delivery are all approaches that enterprises use to drive the business outcomes they need through the use of technology.

    This ability to respond more rapidly to market dynamics is one of the big reasons why enterprises have embraced cloud services. It’s also why they are increasingly embracing APIs, mobile, big data analytics, and more. But it’s not always easy to get these moves right.

    Consider the demands enterprises face building their mobile applications. Despite best efforts, these apps too often don’t solve the business need they intended, and don’t deliver the new abilities initially sought. They build applications that attempt to accomplish too much, and end up being overly complicated and largely unused as a result. Or they take old habits, such as desktop and mainframe apps and squash them onto mobile devices without giving any thought to how the app may be used on that form actor.

    I started thinking about all of this when recently reviewing the results of a survey conducted by the Business Performance Innovation (BPI) Network, an executive advocacy organization, which found that there is a big disconnect between the understanding business leaders have when it comes to their knowing that they need to innovate – and the ability to actually do so.

    This brought me back to a discussion I recently had with Brian Katz, then the head of end user experience and innovation at Sanofi. He spoke about how good app design is one of those things that are crucial to driving business innovation today, but enterprises don’t often don’t get their apps built right. This is because they end up developing complicated apps that people don’t actually use instead of developing apps that their users really need.

    That observation parallels the findings in the BPI Network Study Accelerating Business Transformation Through IT Innovation, which found that from the C-suite out to operational units, senior managers want to implement technologies to increase agility, improve customer experience, and make their companies more competitive. The 250 respondents said:

    • 65% say technology has become “far more important” to their organizations in the past five years. Another 28% see it as “somewhat” more important.
    • 47% say they’re spending more time on understanding strategic implications of new technologies, and another 40% say they’re “working on it.”
    • The three most-pressing IT imperatives were: improve responsiveness to ever-changing business requirements (37%); Focus on digital experience as a competitive advantage (30%); and delivering applications faster, better, and at a low cost (30%).
    • Three areas that would benefit most from transformative technologies: Operational efficiency and effectiveness (52%); Customer experience, monetization and retention (33%); and Organizational alignment, culture and collaboration (28%).

    When it came to data center and cloud transformation, they found:

    • Top three benefits of data center and cloud transformation: Increased agility and responsiveness to business changes (70%); Greater cost efficiencies (57%); Faster time to innovation for new applications (46%)
    • How executives plan to change their data centers: Migrate to hybrid IT that blends traditional data centers with the cloud (45%); Modernize and upgrade (45%); consolidate servers (32%)
    • Three most transformative technologies to give a competitive advantage: Real-time intelligence from embedded sensors – IoT (34%); Always on, highly scalable and web based models (33%); Social media data mining and more efficient engagement (29%)

    However, business managers are not satisfied, generally, with what they see as the slow pace of innovation in their IT organizations:

    • Only 14% rate the level of innovation in IT organizations as “very high” vs. 15% that chose “poor.” Most gave middle ratings: “making progress” (37%) or “good” (32%).
    • How well is the IT organization executing mandate to transform and become a more strategic, responsive and a valued business partner? Only 15% said “very well” while 10% said “poor.” Most were in the middle: 28% said “good” and 44% said “making progress.”
    • What business metrics would you use to evaluate IT performance and effectiveness (top 3): Reliability, scalability and security of IT infrastructure (43%); Ideas and solutions for furthering business performance (40%); Quality and timeliness or app delivery (35%).

    The lackluster view of the role of technology and the ability to innovate at levels business leaders expect was surprising, to me. It’s not as if using technology to differentiate is something that is new to business. It’s not. It’s something we’ve been doing for decades now, and one would expect enterprises would be generally better at it than they are. With that in mind, I reached out to Dave Murray, head of thought leadership at the BPI Network regarding these survey results. You can find that interview here.

  • The IT mandate: Be a catalyst of business innovation

    The IT mandate: Be a catalyst of business innovation

    On the heels of their recent study, Accelerating Business Transformation Through IT Innovation: Getting the Business Leader Take on the IT Change Mandate, we had a conversation with Dave Murray, head of thought leadership at the BPI Network. The study found that IT organizations increasingly want to use technology to drive innovation and business growth, and a big part of that is the ongoing re-tooling their data centers with the public and on-premises and managed cloud services.

    The survey found, as covered in our Enterprises still struggle to leverage IT as tool to innovate, that while about 70 percent of those surveyed by BNI Network believe technology has become “far more important” to their business, 52 percent of respondents believe that their IT organizations are poor or just making progress. Only 42 percent say their IT groups are doing a good job of becoming a more strategic.

    To get a sense of what is going on behind the survey, here is our Q & A with BNI Network’s Dave Murray.

    DevOps.com: Your recent survey had interesting findings regarding how business leaders today view technology as important when it comes to innovation.  

    Murray: We found that 7 in 10 feel that technology is far more important to their business today than it was just five years ago. And we think that’s very, very significant.

    DevOps.com: It is very interesting. How do you compare this to earlier times when technology was also viewed as very important to business innovation, such as the late 1990s when eCommerce was taking hold and enterprises were opening up all of their backend data to the Web? Do business leaders have more of a sense of urgency now, or do they feel the pace of change is more rapid?

    Murray: I believe what they’re telling us is that they see technology today as more essential to business growth, to new business ventures, to the way they interact with customers than ever before in their experiences. And so the movement towards mobility, the movement towards digital business, the movement towards big data are all delivering real strategic value, and is really driving and enabling new competitors in their markets. So they see the need to get on top of technology as never before.

    We did a study not long ago that looked at the impact of new business starts and new digitally driven companies, and we found that there was just a very significant recognition that these new ventures, technology-driven ventures, were disrupting traditional markets where business executives were operating. So I think there are a lot of components to why this is becoming more and more important to business executives.

    DevOps.com: Technology is changing so fast now with mobility and cloud computing, and it looks like IoT is going to be a real thing, that the pace of change of technology is accelerating, so that would probably be a catalyst.

    Murray: Absolutely. I think that’s part of that urgency. There’s a real sense that the game is changing more rapidly than ever before, and that they need to get on top of it as executives. And one of the findings is that business executives say they’re spending more of their time trying to understand the strategic implications of technologies than ever before. Which new ventures and new business opportunities within companies are virtually always now reliant on a technology to make them work. And so they’re trying to get on top of those strategic needs.

    DevOps.com: It also seems that the survey picked up on a long-standing challenge in enterprises and that’s having a tight coupling between the need for innovation and technological transformation, and the ability to actually execute.

    Murray:  I think that one of the questions we asked was, “What do you see as the key metrics for measuring the performance of IT within your organizations?” And this idea of reliability, scalability, and security, the execution of the infrastructure is still the number one issue. But it’s closely followed by strategic, more business-oriented issues like being able to bring new ideas and new thinking to the table about how IT can improve the business and create new opportunities. And then the speed with which they can deliver on those new ideas with applications and faster development of new services.

    So I think there’s a critical need for the expertise to maintain that infrastructure and obviously security and scalability are critical. But then also to more effectively collaborate with business, more effectively understand customer and customer needs, and come up with the architectural designs and applications that can meet those needs.

    DevOps.com: Do you see this as kind of a do-or-die situation for companies that don’t master how to innovate with technology, or will they pay a heavy price for not being able to do so?

    Murray: I think companies that do not figure out how to make their technology and their IT organizations and partnerships more effective at driving innovation and creating a more agile and responsive IT organization and infrastructure that can respond to business change are going to be in big trouble in the years ahead, without a question.

    I think what we’re seeing is in virtually every industry, their technology is becoming, truly becoming part of the business and a major strategic driver of customer acquisition, the business model, the way the company operates, and I think that that’s going to continue to escalate. I think we’re always ahead of ourselves in terms of what we know technology can do, and then actually making it happen.

  • DevOps Chat: Gary Gruver, Leading the Transformation

    DevOps Chat: Gary Gruver, Leading the Transformation

    I recently had a chance to sit down with Gary Gruver, well known DevOps leader and author. Gary’s new book, Leading the Transformation, Applying DevOps and Agile Principles at Scale is a great how to for managers looking to lead the transformation at their respective teams.  Gary put on a very informative webinar which you can listen and watch that goes into much greater depth than this short conversation.

    I have posted both the audio file to listen to and a transcript of our conversation below.  Enjoy!

    [soundcloud url=”https://api.soundcloud.com/tracks/224098337″ params=”auto_play=false&hide_related=false&show_comments=true&show_user=true&show_reposts=false&visual=true” width=”100%” height=”450″ iframe=”true” /]

    Here is the transcript of the conversation with Gary Gruver. You can follow along with the audio file or just read if you like:

    Alan Shimel:         Hi. This is Alan Shimel, Editor in Chief of DevOps dot com, and welcome to another DevOps chat in our leadership suite. Today’s guest is DevOps thought leader, and well-known author, Gary Gruver, who has just published his book, Leading the Transformation, Applying Agile and DevOps’ Principles at Scale.

    Gary, welcome to DevOps Chats.

     

    Gary Gruver:                Hi Alan. Good to be here.

     

    Alan Shimel:                Thank you. Gary, first of all, let me just commend you. The new book is great. I think I mentioned to you offline, I read it on the way out to California, and back. I went back and actually highlighted a bunch of stuff, which I don’t often do with books anymore, but it was that good and that important to me.

    So great job on the book. For those of our readers who may not be familiar with both yourself and the book – why don’t you give us just a quick background and synopsis of Gary and the book?

     

    Gary Gruver:                Well I’ve been in high tech for a long period of time and the last, probably eight years, leading large scale transformations of software development projects in different organizations, first at HP as the Director of the Imbedded Firmware Organization, over about – we started with 800 developers.

    And I wrote my first book because the results and benefits that we saw with that were so dramatic, and I learned so much from the organization and the transformation that I just felt that I had to share it with everybody else, do that’s a practical approach to large-scale actual development. It’s kind of a case study at HP.

    And that kind of gives you a feel, a short, quick, easy to read, not very technical book for executives that just – here’s what a transformation looks and feels like. I’ve gotten more actively involved in the Agile and the DevOps community and learned a lot by going to a large retailer, and was head of QA Release and Operations at that retailer, and led a large transformation towards continuous delivery there.

    And what I found through the process is, it’s easy for engineers to go to different conferences and understand how to develop differently, but if the executives don’t engage in leading the transformation, they’re going to come back, and the organization is going to throw a wet blanket on it. So what I tried to do with the second book, which Leading the Transformation is, is write down and capture in a book that was short enough – 100 pages – so that executives would read it, but had everything I wished I knew before I started on my transformation. So look, there’s going to be uniqueness and differences with different organizations.

    But the more they can avoid the bumps on the heads and the learnings that I’ve already gotten to on my own, and moved forward, and kind of moved the process and the technology forward, I think the more effective they’re going to be. So I created a lot of passion around trying to get some great business benefits.

    And I really wanted to share with others and try to figure out how I could help others to be more successful too.
    Alan Shimel:                Absolutely. So, Gary, one of the interesting things I got out of your book was – you know, many executives, they hear of the benefits of Agile and DevOps, and where do they sign up, right? Sign me up for that. I’d love to see those kinds of exponential improvements in all of these metrics.

    But one of the things you say is – executives who are just looking to do sort of Agile in and of itself aren’t going to see that sort of success, correct? It’s not just doing Agile, if you will. As a matter of fact, if you say that you do just Agile, you’re probably not going to be very successful. Do you want to expand on that?

     

    Gary Gruver:                Yeah. I would say it’s too large of a transformation and too large of a turmoil. If your current development processes are meeting all the needs of your business, you really shouldn’t go through this much effort and turmoil in your organization and cultural change.

    But I think what most organizations are seeing is – they’ve got larger, and software has become more and more important to how they compete in the marketplace, that their current software development processes just aren’t able to react and respond to where they need to add value as a business.

    And so, start with what is not being delivered to your business from your software development processes and your business objectives. You know at HP, it had been two decades that firmware had been the bottleneck for the business, and so we really started by saying – we want to be able to add a new product anytime we want without firmware being the bottleneck, and we wanted to free up capacity for innovation.

    And it was with those business objectives that we were able to prioritize the things we went after, and I think if you start with your business objectives and fundamentally looked at the rest of your toolbox, whether it’s to add to laying DevOps, sort of combined automated – just any of those things are just tools in your toolbox that are there to solve a problem.

    And then you go on a journey of continuous improving. That’s where you get the business results. It’s not going after one of those principles and hoping that you’ll get business results. It’s driving for business results and using the tools as appropriate.

     

    Alan Shimel:                Got it. And Gary, you use the term a lot, continuous improvement. In a world where continuous is the new black, and we hear about continuous delivery, and continuous integration, and continuous testing, and continuous security – briefly, what is continuous improvement?

     

    Gary Gruver:                Well it’s really wanting to go out and learn what’s between you and your organization delivering software better and more effectively. And then it’s constantly just chipping away at those barriers and learning and getting better involved. You’re not going to know everything.

    You know, at HP, we got two to three X improvements in productivity, but we didn’t go off on a three year plan to make that work. We took it month by month, and assessed some objectives, and tried to get better. And you’re going to learn so much as you really start getting out there in trying to figure it out that, what you’re going to find is, if you create a three year plan, or even if your could, it’s going to inhibit you from getting better because you’re going to learn something new that was getting in the way.

    And so, don’t try to lock in too much. Software developed and improved effectively can be very flexible and responsive. Learn and adjust, and learn and response, and just take on the biggest barriers to your productivity one at a time. And when you improve that, you’re going to learn what’s next and continue to improve that.

    But it’s getting the executives to come together once a month to review what they said they were going to get done, what got done, and what didn’t get done, and what you’ve learned in the process, to set the objectives for the next month. You know, at HP, we went off trying to get 10X better, and we did it a month at a time.

    And when we looked back three years later, we had gotten, you know – my co-author would say, 10X better in terms of frequency, and builds, and those things, but business results – easily 2 to 3X better, but it wasn’t by executing a four year plan, it was by getting up after continuous improvement, and learning and adjusting along the way.

     

    Alan Shimel:                Sure. You know Gary, in the book, one of the things I got out of it is, you sort of refer to CD, continuous delivery, as a lot of the tools and maybe processes that we follow along this Agile and DevOps journey, if you will, or process. But DevOps specifically is largely cultural in the book anyway. Is that correct?

     

    Gary Gruver:                Yeah. It depends on whether you’re talking to Gene or Jez. Jez will focus more on the technologies, and Gene will rely on the culture. I can’t really separate the two. You can’t get the cultural changes without the tools, and you can’t implement the tools and get the benefits without the cultural changes.

    I like the definition where you just lump them together and say – it’s the cultural and technological changes that enable us to release code on a more frequent basis, in a more stable way, and get feedback from our customers on an ongoing process. And so that’s the definition that kind of resonates with me, and I have a hard time getting between the two.

    But you know, in a large organization where you’ve got tightly coupled code that requires people to develop, qualify, and deploy in unison, the technologies of continuous delivery, whether you deploy everyday or not, is the forcing function that ensures all your code is going together and stays aligned.

    And so the first order of effect in a large organization is to help it come together to deliver code, or deliver value to their customers, on an ongoing basis. And the second order of effect is how the individual teams work, continuous delivery, and some of those principles are the forcing function that allows working code to provide that alignment across a larger organization, and not develop code that either won’t work together or won’t work in a production like environment.

     

    Alan Shimel:                So Gary, another sort of debate – and I don’t know if it’s a Gene and Jez debate, but it’s certainly within the community – is, do you do DevOps from top down, bottom up? Do you do it from both and meet in the middle? You have a very unique perspective in having led several transformations now and, obviously, you come at it from a top up perspective. What should leaders, listening to you here today, what should they take out of that?

     

    Gary Gruver:                I think if they’re going to be successful, the leaders need to lead it because you can’t have some groups decide to do it, or some groups decide not or do it in a different way, if their code has to come together and be delivered as a unit. But, at the same time, it’s not management.

    It’s more lead it. Set the direction. Set the priorities, and engage with the organization to learn what’s working and what’s not working. And so, you know, the continuous improvement, executive-led continuous improvement process I describe in my book really shows how the executives engage in the organization to lead that because you need all the learnings in everything that’s getting in the way from the bottom.

    And you need kind of the set of priorities from the top, so it’s that constant interaction between the leaders and the organization that gets all the best ideas and all the learnings out on the table, that gets you to prioritize the right things and go after the right improvements.

     

    Alan Shimel:                Yeah. Gary, the last chapter in the book is entitled, Getting Started, and it kind of represents, okay – after you’ve learned what you can from this book, here’s how you can get started in your organization. Of course, people should read it for themselves and learn from it, but what can you take out of that last chapter to give people a jump start on that?

     

    Gary Gruver:                You know, you really need to start where the most pain is. And one of the things that DevOps causes you to do is do things more frequently. And what you’ll find is your organization has been brute forcing its way through the same issues for years. And as you start to increase the frequency, you’ll have to start fixing that.

    And it’s hard to tell. For different organizations, it’s different places, which is why I put that piece at the back end because it depends on what you already have in place. If you’ve got a web micro services architecture that enables teams to work independently, it may be as easy as a cultural shift, and putting an operations person on that team, and letting him own things end to end.

    If you’ve got a more tightly couples system, you’ve got to think about how do you coordinate the work across them? And the first step may be, can you do continuous integration of as much of the enterprise system as you can, with a few simple tests? And make sure you’re automated correctly. So it’s that step of learning and evolving. And then as you try to do it and increase your frequency, you’re going to find different pain points.

    And that’s why I’d say it’s the continuous improvement stuff. And you know I do work shops with different executive teams to really try to bring out, and try to understand their development processes to the point that we can figure out where to start the continuous improvement process.

    We hold a work shop with their technical and manager leaders and everybody in the organization, and try to really nail – what’s that first month’s objectives? And then we do ongoing mentoring calls for the first six months to make sure they’re off on the right path.

    But you know, as opposed to spending forever trying to figure out where to start, if you can think about starting your continuous improvement process, even if you pick the wrong things first, you’re only going to go down that path for a month before you have a chance to learn, and adjust, and go after the things that are really in the way between you and your organization being productive.

     

    Alan Shimel:                Great. Hey, Gary, we’re about out of time.

     

    Gary Gruver:                Okay.

     

    Alan Shimel:                Thank you very much for appearing on today’s DevOps chat and, again folks, the book is, Leading the Transformation, Applying Agile and DevOps’ Principles at Scale. And for any executive or manager out there who is contemplating or perhaps is already engaged in leading a transformation around Agile and DevOps around software development, I highly, highly recommend it.

    You can get the book on Amazon. It is from IT Revolution Press, and just Google it. I think we also have a two chapter download, which we’ll put a link to up on the webinar page for this. Gary Gruver, thank you very much. This is Alan Shimel, of DevOps dot com. Have a great day.

  • Driving Innovation with DevOps

    Driving Innovation with DevOps

     

    DevOps isn’t just about working faster, more effectively, and at a lower cost. A big part DevOps is also about driving business innovation. Sure, DevOps correctly applied is known to cut costs and reduce downtime, but as this Rackspace 2014 survey found, DevOps is also, across many organizations, increasing sales as well as employee and customer engagement.

    But DevOps is also enabling organizations to deploy more capabilities more quickly. As the most recent Puppet Labs DevOps report contends, DevOps organizations are deploying updates 30 times more rapidity and with fewer failures. And they’re recovering 168 times faster from failures and have 60 times fewer failures due to code changes. “What we are seeing is the quality and speed has definitely increased. So people are producing changes that are of higher quality and changes that require fewer rollbacks,” said Nigel Kersten, CIO at Puppet Labs. in this interview with Ericka Chickowski.

    If DevOps is going to positively drive business transformation, all of the associated efforts – collaboration, communication, automation, continuous delivery –  have to be orchestrated toward the right goals, in the right ways, with the right people and processes in place. And that’s exactly what this new section is about. It’s about helping business and technology leaders to manage their enterprises through times of change in technologies and markets.

    During my interviews with technology leaders, since we’ve launched DevOps.com, I’ve identified at least three core areas where DevOps, from a high level, helps to transform organizations beyond IT. Here’s how:

    Find your bottlenecks in the digital supply chain

    Identify where your IT organization is stalling in their processes. Is there a deep application backlog? Is testing and QA a bottleneck? Are teams communicating properly? Are their legacy systems and processes that are holding your organization back? Find these areas, prioritize, and weed them out. DevOps is largely about collaboration and automating anything that can be effectively automated.

    Better process, not better software

    Have your infrastructure and application leaders find all of the bad or broken processes that they can and then eliminate them. Organizations create rules and processes and bureaucracies as ways to enforce standardization, improve quality, and protect the organization. But over time many of these rules become unnecessary, or antiquated as the organization and business environment changes. These, too, are weeds growing within the organization and need to be pulled or streamlined.

    Earlier this year I wrote Five Great Books on DevOps, some books were written recently, others are decades old: but they all highlight the importance of collaboration and good processes.

    And while having quality software and toolsets in your organization is important, it’s secondary to good teamwork and providing that team a vision to work toward.

    DevOps + Business Improved Collaboration = BizOps

    In this context, BizOps refers to a tighter collaboration between technology efforts and the business outcomes the organization needs. When you have all of the benefits of DevOps plus sound business leadership helping to guide the goals of those efforts you will have a company that can better innovate.

    How? The IT teams will know what is most important to focus on, they’ll know better where to invest their assets and time. They will also be able to experiment more quickly and they will know what unexpected opportunities that arise are to be seized upon and which can be let go.

    This alignment between technology and the business is a big part of what we hope to write about here in this new section. And we’d like to hear from you about how your enterprise is using DevOps to help drive business transformation. And if we don’t hear from you, you’ll certainly be hearing from us on these very topics here in the Leadership Suite.

  • DevOpsQA NJ Meetup –  Configuration Management with Puppet

    DevOpsQA NJ Meetup – Configuration Management with Puppet

    DevOpsQA NJ just held our September Meetup on Configuration Management with David Bowser and Jeremiah Sullivan from Puppet Labs and Michael Pereira from Syncsort as speakers.   The venue was provided by  ROKITT at a state-of-the-art office space with breathtaking views of NYC.

    Mi_DSF5099chael gave a great introduction to Environment Automation by giving a live demo of BCMT –  Bash Configuration Management Tool – which he developed to demonstrate 2 critical concepts – Idempotence and Convergence.   Idempotence means that multiple applications of the same action do not have side effect on the system state while convergence means to bring system state in line with a defined polic
    y.  Changes are made on the system only if they need to be made which is one of the key behind the configuration management tools. My favorite was the definition for Configuration Management provided by Michael – “Whatever you need to run your application”  –  short, simple, straight to the point.

    Next, David and Jeremiah gave a rundown of 2015 State of DevOps Report and a history of Puppet Labs.   Jeremiah covered the Open Source Puppet followed by a live demo of Puppet EntIMG_4176erprise.  Puppet Enterprise offers everything available in Open Source edition, plus support, training and a web based GUI.   Puppet allows to automate provisioning for physical machines, as well as VMs, Docker containers, and AWS instances.  One of the reasons Puppet is so powerful is due to the variety of the existing modules that are available from Puppet community.

    At the end we had a great Q&A session ranging from Jenkins integration,  automated validation of the modules,  extending of the modules and future of Chocolatey.       _DSF5103

  • Do You Need Your Ticketing System for Real-Time Incident Management?

    Do You Need Your Ticketing System for Real-Time Incident Management?

    The term “operationally mature” characterizes companies that understand how incident management affects both employee well-being and business impacts. Operationally mature companies have a process in place with built-in reliability and accountability that limits high-severity incidents because of bugs. The more mature a company’s operations, the better it can maintain agility and adapt to change. So how should operationally mature organizations handle incident management? Are ticketing systems still the gold standard?

    Ticketing systems keep track of issues, assign items to other people and order requests. They are a great way to manage a queue of ongoing requests, yet many operationally mature companies shy away from ticketing systems for their real-time incident management.

    Likely, it’s because ticketing systems can be fraught with issues. A clunky workflow, mired in process, means that users can’t always move and adapt quickly. Every minute they spend updating their workflow in the ticketing system equates to less time for dealing with critical issues. Ticketing systems often require a rigorous, structured, manual approach that can slow down work in the heat of the moment.

    Leading DevOps companies are proving that you don’t need a process-heavy system to get work done. In fact, for real-time incident management, the less process, the better. Companies are migrating from a system where a user manually opens a ticket when they see an outage, to monitoring systems that automatically generate alerts as problems arise. As soon as something goes wrong, the system generates an alert. There is no delay waiting for someone to notice the issue and fill out a form. Teams get immediate visibility into what’s going on with the system and can begin work or escalate accordingly.

    Operationally mature companies really need a lighter-weight solution. They need a solution that focuses on automation, allows them to be more agile and fixes incidents faster. The decision is not an either/or proposition, however.

    Many companies still have a place for ticketing systems, even if they aren’t for real-time incident management. Organizations can combine ticketing systems with more agile solutions. A more lightweight system still allows users to open manual alerts if necessary, much like a ticketing system does. Some operationally mature companies use a ticketing system to track incident follow-up and generate work estimates for lower priority tasks. Once the critical incident is resolved and service is restored, companies can create tickets to fix the root causes and make sure the problem doesn’t happen again.

    Ticketing systems are not necessarily outdated or obsolete. Unfortunately, one of the benefits of a ticket system is also a weakness. The plethora of fields, for example, allows users to track various aspects of an issue, but those fields are cumbersome and unnecessary for high-priority items. Ticketing systems can be clunky when DevOps needs something sleek. Combining ticketing with a lightweight, automated, incident management system is a sure-fire way to streamline some of the extra processes.

    Smart companies ask, “How much process is necessary to accomplish our tasks?” Answer that question, and your IT team will thank you. The answer will help them feel less like a help desk and more like an integral member of the DevOps team. And that helps your organization move faster and get closer to achieving DevOps Nirvana.

  • Roll-up your sleeves – Launch a SAFe® Agile Release Train

    Roll-up your sleeves – Launch a SAFe® Agile Release Train

    Most organizations often begin their agile transformation with a series of low risk, stand-alone pilots. While these pilots may prove that teams can adopt small team agile practices such as “Scrum” or “XP”, however these pilots often don’t help prove that your organization can adopt agile at the enterprise level.

    Scaling agile to support larger more complex programs requires gaining alignment across stakeholders, providing program-wide transparency, and delivering end-to-end value with a large number of teams to work together (e.g., 100 – 500+).

    A few months ago, I had the opportunity to launch an agile release train (ART) for a large IT project (110+ resources) at one of the financial institutions that IBM provides agile coaching and mentoring for.   The organization had been following a waterfall approach for most of their larger projects until last year when they were purchased by a larger bank that was already undergoing its own agile transformation.

    IBM was brought in to assess their current progress and provide additional training and recommendations for rolling out to large enterprise programs.   Our mission was to assist them with:

    1. Defining a set of outcomes that supported the expected business benefits and assist them in successfully launching agile programs
    2. Identifying the value streams – the long-lived series of system definition, development and deployment process steps used to build and deploy systems?
    3. Defining the Agile Release Trains – the virtual organizations formed around these value streams?
    4. How to launch an ART and what Support IBM provides for SAFe

    Here are a multi-part blog to help you launch your own train:

     

    1. Launching an ART. Where to start?

    In its simplest form, agile offers a lightweight framework for helping organizations build software using a rapid delivery approach to providing business value. As a result these organizations are able to significantly reduce the overall risk associated with software development while providing high quality products and services.

    Agile is a powerful tool for software development, not only providing benefits to the development team, but also providing a number of important business benefits to the client. In addition, many agile project teams have been able to overcome many of the most common project pitfalls (such as cost, schedule predictability and scope creep) in a more controlled manner.

    The first step in our approach was to conduct a short agile assessment (or feasibility study) to verify the scope, identify the current challenges and recommend an adoption roadmap to support the transformation for the program (110+ resources).

    Having helped a several customers with agile transformation projects along with the challenges and business outcomes associated with those adoptions, I was immediately concerned about our ability to coordinate across multiple teams.

    The organization had several waterfall programs underway and had successfully piloted Scrum on some of their social apps. The leaders wanted our help to adopt a multi-team agile approach where some parts of the organization would stay waterfall.

    After performing a quick assessment we identified the following organizational concerns

    Reedy 1

    After reviewing some of the most common approaches at the time, we concluded given the organizational challenges that the Scaled Agile Framework (SAFe) would be our starting point.   SAFe, created by +Dean Leffingwell‘s Scaling Software Agility, is a proven, publicly-facing framework for applying Lean and Agile practices at enterprise scale.

    Reedy 2In most cases we have found starting with a small to medium size program is the best place to start when attempting to help an organization scale agile in their organization.

    Benefits of starting at the Program Level:

    • A couple of key business stakeholders usually have finally say on budget, delivery teams
    • Can select program within sponsors span of responsibility / influence
    • Requires more organization than going team-by-team, but significantly less effort than a “big bang” enterprise transformation
    • Makes visible the real challenges of agile at scale
    • Gives true, fact-based measures of enterprise agility… the ability to quickly deliver quality software incrementally in a large, complex enterprise

    Here are some of the challenges of starting at the Team level.

    • Have the organization / cultural impacts that a larger adoption would have
    • Provide the visibility to “move the needle”
    • Cover enough ground. Delays dealing with the real challenges of implementing agile at scale

    Here are some of the challenges with starting at the Portfolio level:

    • Working with too many In-Flight Projects
    • Taking on too are a structure to impact (.e.g, Chaotic Architecture and Bloated Technology Stack)
    • Working with teams outside the reach of the team to really impact them.
    • Requires lean-thinking leaders and an Agile PMO who understand lean economics
    • Can be delayed due to the complex organizational and cultural changes needed for a full enterprise transformation
    • Difficult to manage software “assets” in a Lean|Agile portfolio if asset development isn’t agile

    See here for additional information:

     

     

    2. Preparing for first Agile Release Train (ART) launch?

    For any IT project to be successful (traditional, agile or otherwise), it’s vital that it has a clear vision which can be used to guide the project team in the right direction. The team should review the vision regularly to ensure program is fulfilling what it has set out to accomplish.

    Here are 3 key steps that must be done before launching the ART.

    • Identify the initial set of Value Streams

    A Value Stream is a long-lived set of capabilities used to provide a continuous flow of benefits to the business. Value Streams are typically realized by 1 or more ARTs and are the primary organizational concept for understanding how value is delivered in SAFe.

    Reedy 3Here are some questions to help identify the Value Streams

    • What are the larger, ongoing programs or initiatives that will differentiate your business over the next 2 to 3 years?
    • How do external customers perceive the value you provide? For example what are the key products or services that you provide
    • Can you describe the strategic initiatives that have 200+, 100+ or 50+ more developers / testers working together already? Do they have a high degree of interdependence between the features and components?
    • What key business processes do you enable?   What internal departments do you support?
    • What key process, cost, KPI or business improvement initiatives are targeted in this upcoming year?

    For more information checkout:

     

    • Define Vision, Scope and identify Portfolio and Program Epics

    SAFe organizes large agile projects into an scaled agile program structure containing three levels: Teams, Agile Programs (also referred to as Agile Release Trains – ARTs), and Portfolio

    The Agile Release Train is the primary organizational and operational construct within SAFe. Each ART is a self-organizing team of teams, responsible for delivering business value. Ideally each value stream should be mapped to a corresponding ART, however in many cases ART may be needed to support multiple value streams or in some cases a single value stream may require the support from multiple ART to delivered the required business value.

    Reedy 4SAFe defines an artifact hierarchy of Epics – Features – User Stories.  The agile program backlog (a.k.a, ART) containing a prioritized list of features and stories.  Portfolio epics are large initiatives in SAFe that drive business value. These initiatives typically cross organizational and financial reporting (quarters, years) boundaries. Most organizations Business epics are large initiatives in SAFe that drive business value. These initiatives typically cross organizational and financial reporting (quarters, years) boundaries.

    Reedy 5

    Weighted Shortest Job First or WSJF comes from a comprehensive model defined by Don Reinertsen as a way to improve prioritization. In IBM’s solution, WSIF is automatically calculated based on the Job size, Business Value, Time criticality and Risk Reduction/Opportunity Enablement values provided. Then, the Program Roadmap is automatically sorted based on the WSIF score and gives a ranked view of the features’ backlog. WSIF is an useful tool to help with decision making process based on factual assumptions instead of personal feelings.

    Reedy 6

    Features are usually decomposed into User Stories, which flow to Team-level backlogs, and can originate at the Program level, or they can derive from Epics defined at the Portfolio level.  Features are prioritized based on Weighted Shortest Job First (WSJF) economic decision framework.

     

    • Ensure you have established “Whole Teams”

    SAFe is based on Lean principles where the minimal viable product (MVP) epics and features are identified scheduled based on both business needs and technical constraints / dependencies. Like Scrum, SAFe assumes each agile delivery team (typically between 5 to 10) people each have all the cross functional roles necessary to deliver working solution each sprint.

    Unlike Scrum, SAFe assumes some stories may depend on each other or that multiple teams may need to work on Epic before it can be delivered. Prior to launching the ART, important to ensure that each of the teams have been formed with the appropriate set of agile practices (i.e., Whole Team) and know how to execute the ceremonies associate with Scrum / XP.   Whole team is a philosophy that involves significant collaboration between the team mReedy 7embers and stakeholders.

    Each team includes all individual who materially define, develop and deliver software for a project:

    • Any and all contributors
    • Both direct and indirect participants, such as sponsors and key stakeholders
    • Together have the skills to successfully communicate the essentials of a problem and deliver an agreed solution

     

    Here are seven tips for building an effective agile team:

    • Does the team have the skills to get the job done – they are “whole”
    • Is the team as small as possible
    • Are the people dedicated to the project and is there a single product owner
    • Have a single team lead (who is not the product owner)
    • Are team members cross functional, made up mostly of “generalizing specialists”
    • May include specialists, hopefully on their way to becoming generalizing specialists
    • Are self-organizing within an appropriate governance framework

    The greater the deviation from this strategy, the greater your project risk

     

    3. How does IBM Collaborative Lifecycle Management (CLM) support SAFe?

    As part of the engagement, the IBM team led the effort to implement IBM’s Collaborative Lifecycle Management (CLM) solution to support the Agile Release Train (SAFe). IBM DevOps combined with SAFe provides a comprehensive process and tooling platform and framework that will optimize your end-to-end lifecycle, so you can synchronize development, testing and deployment across teams in heterogeneous environments, and collaborate across all roles in the organization in the planning and execution of software delivery.

    Reedy 8

    IBM’s SAFe support provides the following key capabilities:

    • Get up and running quickly with out-of-the-box infrastructureto lead a SAFe Program
    • Improve agility and predictability with dashboardsfor visibility to adjust business goals in real-time
    • Simplify change to culture and process with quick and easy access to SAFe guidance

    Reedy 9

    Each team works in fixed-length time-boxes, called iterations or sprints, which typically last from two to six weeks. At the end of the sprint, there is measurable product increment or (PI).

    Planning an iteration is simple: From the Product Backlog, the Product Owner (PO) and the Scrum team holds a Sprint Planning meeting to select the most important stories that can be delivered in the upcoming iteration. Once the set of stories are identified, the team breaks down the work into smaller tasks and assign it to individual team members.

    Reedy 10

    The Agile Planning capabilities in RTC, can assists you doing this by several means:

    • Plans are accessible to everyone in the team – Be transparent.
    • Plans reflect the team’s position and direction – Know when you are about to fail.
    • Plans are live as they ground on work items – No unnecessary planning artifacts to be taken care of.

    IBM’s approach included a set of accelerators specifically designed to help organizations adopt a Lean mindset to take their current agile projects to the next level. Agile Planning includes a set of development practices that allow teams to plan and deliver high quality software rapidly. For more details see: see the following value-added assets:

    Finally for those teams that are using Scrum methods, here is an article on using Scrum with Rational Team Concert (RTC)

    About the Author/ Reedy Feggins

    Reedy FegginsReedy is a Lead Agile Coach and Software Architect with certifications as a SAFe SPC, Certified ScrumMaster and PMP® certified project manager with over 20 years’ experience leading successful IT projects, employing a combination of Agile, Waterfall, and RUP processes. At IBM he currently works as a Agile Coach leading large and medium agile transformations. Reedy has published papers in several magazines as well as been an invited speaker at conferences. His experience as a trainer has led to the development of several custom workshops on topics ranging from Project Management to Process Adoption.

    Connect with Reedy on twitterlinkedin

  • How do large enterprises do DevOps? Recap of my third video chat with the 2015 DevOps Enterprise Summit Speakers

    How do large enterprises do DevOps? Recap of my third video chat with the 2015 DevOps Enterprise Summit Speakers

    Earlier this morning I had the pleasure of hosting the third video chat with and some of the speakers of DevOps Enterprise Summit 2015.

    This live event featured:

    Paul Reed | Director of Site Reliability Services/Principal Consultant, Salesforce
    Attend Paul’s talk at DOES15: The Blameless Cloud: Bringing Actionable Retrospectives to Salesforce

    Kris Buytaert | CTO @ Inuits.eu
    Attend Kris’ talk at DOES15: Turning Around the Containership

    Michael W. Smith | Technical Director, Hardware & Infrastructure, Raytheon Software
    Attend Michael’s talk at DOES15: Melting the Glacier: Introducing DevOps to a Federal Systems Integrator

    Tapabrata Pal | Product Manager, Capital One
    Attend Tapabrata’s talk at DOES15: Shortening the Feedback Loop: DevOps Dashboard

    Anders Wallgren | Chief Technology Officer, Electric Cloud
    Attend Anders’ talk at DOES15: Continuous Delivery of Microservices: Patterns and Processes, and a panel discussion with Gary Grver, Jez Humble and others on DevOps and Continuous Delivery for Embedded and IoT.

    We discussed how some of the largest enterprises in the world are addressing the biggest challenges facing the DevOps community: from fostering a DevOps culture and leading the transformation, automated testing of legacy applications, security and auditing requirements, managing burnout, fostering an internal consulting capability, and more. Take a look at some highlight tweets and then watch the full recording below.

    The importance of avoiding burnout during a DevOps transformation:

    On security:

    It was such a pleasure to discuss these topics with the best minds guiding DevOps transformations in large, complex organizations. Watch the full recording and hope you join me at DevOps Enterprise Summit 2015 to discuss more!

    The Agile, Continuous Delivery and DevOps Transformation Summit
    October 19–21, 2015
    Hyatt Regency, San Francisco, CA

    Use the code DOES10 to get 10% off your ticket.

  • Continuous test monitoring DevOps ”health-beat”

    Continuous test monitoring DevOps ”health-beat”

    In my prior blog Test Results Analysis at the Speed of DevOps I discussed ways that test analysis features and practices can be optimized to ensure that test results for each CI/CT cycle keep pace with accelerated continuous testing. Beyond optimizing continuous test analysis for each cycle, longer term continuous test monitoring is necessary to assure overall delivery confidence for each release and longer term product performance trend in a positive direction. Individual test results or a fixed number of test campaigns can only tell you so much. Some of the more difficult problems to detect and diagnose are those that only occur occasionally or under “corner case conditions”.

    Unless the results of testing and test analysis are continuously monitored and results aggregated over multiple test and release cycles then there is no way to build sufficient sustainable confidence in the long term health of the DevOps pipeline or products produced by the pipeline. The combination of continuous test monitoring tools with continuous testing and analysis tools can provide a longer term strategic view of test results that is necessary to collect, aggregate and organize test results data to gain confidence in the product for each release and its evolution over multiple releases. Below are some suggestions in a checklist format that have proven useful for continuous test monitoring for sustainable high performance DevOps environments.

    1. Determine continuous test monitoring priorities: Some examples of problems that continuous test monitoring can help with include intermittent failures caused by marginal designs, marginal test designs, environmental condition changes not detected by individual tests, memory leaks, varying starting conditions, interactions with other systems, system topologies, and performance degradation within the margin of a test but accumulate over time. The best practice for continuous monitoring indicates that the problems of most concern to a specific product or DevOps environment will be monitored.
    2. Regression test product areas even though there were no expected changes: One of the practices in high performance DevOps environments is to skip tests for areas that are “known” not to have changed during individual CI/CT cycles. Unintended consequences of indirect changes may impact performance so a regression suite should audit these areas occasionally just to be sure. Typical examples that are often caught by this are features important audit functions, system backwards compatibility features and upgrade/downgrade administration features.

    3. Select continuous test monitoring tools that collect and report trends: Tools that can correlate and report test results across multiple dimensions such as test cases, product features, system performance categories, build versions, releases, and functional tags can find intermittent bugs or problem trends. Thresholds and email alerts and dashboards that highlight short term results views from long term results views are especially valuable.

    4. Use continuous test monitoring results to diagnose and resolve problems: Intermittent failures and problems that only become apparent when test results are presented as trends over a long term trend are easier to diagnose when a large data set is accumulated and filtered by most probable cause tags. Once a diagnosis is determined, the root cause can be verified through targeted retest cases that set all the conditions in accordance with the diagnosis. Once confirmed then the offending design can be refactored to handle or avoid the failure condition.

    The above is a partial list of suggestions for continuous test monitoring that have been proven to support sustainable DevOps. At Spirent we think testing has a bright future in DevOps. You can hear more about our views on this topic by reviewing our joint webinar conducted together with test management product provider Zephyr at: Overcoming-DevOps Challenges

    What do you think of these suggestions and do you have others that should be mentioned?

  • LinkedIn Preps Site Reliability Engineers (SREs) For Exciting Careers

    LinkedIn Preps Site Reliability Engineers (SREs) For Exciting Careers

    Find and Fix, Sink or Swim

    A few LinkedIn NOC engineers including Ariel Casas had a hand in forming what the once fledgling business / professional networking company today calls its Site Reliability Engineering organization. “The main difference between SRE and devops is that SREs are basically developers who happen to be very good at operations. So we call them Site Reliability Engineers,” says Ariel Casas, now Manager, Site Reliability Engineering, LinkedIn, explaining the choice of terminology.

    In those days, circumstances such as outages would thrust Casas and his colleagues into the middle of the excitement, forcing them to devise solutions to the operations challenges at hand. This gave them opportunity to engage developers about how to work together to avoid future issues.

    “Getting thrown into the middle of the storm and having to reverse engineer a service not knowing what it does, not knowing why it’s broken, we basically had to identify the root cause. We had a very, I mean, we still do, but we had a very, very complex infrastructure back when I first started. It was like descrambling an omelet,” says Casas.

    Casas and crew located and scraped through log data, combed through config files, identified service dependencies, and figured out how services worked on the spot in order to understand an unexpectedly broken service. “Those opportunities were a great way for us to learn and figure out how our infrastructure worked,” says Casas, whose early training at LinkedIn was largely on-the-job, experiential learning.

    Early team members like Casas received a lot of hands-on exposure to the various stages in LinkedIn’s application lifecycles, from conception to design to maintenance. Team members touched the application stack at many points and worked with developers to help them understand how the code they wrote would affect the infrastructure that the NOC had to support.

    Playing With Fire

    Speaking of particular fires he and his co-workers had to extinguish, Casas recalls an early example when only a few engineers were on hand as they watched the site, waiting for something anomalous to happen. And something did. “We had a company widget go down,” says Casas.

    The widget served data that LinkedIn collected about specific enterprises that external media outlets were reporting on. In this instance, the CNN news service’s application of the widget stopped working. “We had no idea about the service or how the widget served that data, we simply knew it was broken and causing a poor experience for the customer,” says Casas.

    The team used a Firebug debugging tool to determine the context path for the widget. They connected the dots from there to the service and the port that it was listening on. Then with some log mining relative to that service, they found errors that confirmed that the service could benefit and perhaps fully recover with the aid of some additional memory resources. “We increased the memory for the service, deployed it, restarted it, and it was really rewarding to see the widget behaving the way it should,” says Casas.

    On With the Formalities

    With more than 150 engineers at LinkedIn now, the company has fewer fires to put out and more need for formal training, including external trainings such as conferences and internal trainings at LinkedIn. SREs attend yearly conferences like Velocity, a WebOps-focused event where speakers flock from different organizations to share on topics where they have subject matter expertise. SREcon is another conference and SREs come to seed knowledge and training in their fellow workers there. LinkedIn also sends speakers to each of these conferences as well as attendees. “We also go to Python trainings to learn how to better code and to understand new concepts that we can use to improve our tooling,” says Casas.

    For internal training at LinkedIn the company’s SREs who support select services, such as the internal SaltStack will provide education about developments in those areas. Other training includes Java programming, as the LinkedIn software stack is formed with Java. LinkedIn’s Java SMEs also contribute code back to Java.

    LinkedIn also trains SRE staff on Kafka, the messaging system that supports much of that group’s data streaming. “We have some pretty smart engineers who support, own, and develop Kafka here. They provide good training around how we implement and use it,” says Casas. LinkedIn sends Kafka SMEs to meet-ups to train other enterprises in its use.

    An Open Door to the SRE Career Advancement Ladder

    LinkedIn offers internal trainings with an open door to any high-performing NOC engineer or developer in SRE who wants to attend. “We also have more general trainings called TechTalks and these are high-level and open to anyone in the company,” says Casas.