devopsy.kinsta.cloud

Category: Features

  • Q & A: Puppet Labs CIO Nigel Kersten

    Q & A: Puppet Labs CIO Nigel Kersten

    We recently spoke with Nigel Kersten, CIO at Puppet Labs, regarding high-performance IT and trends he sees in continuous integration in his role at Puppet Labs. Nigel joined Puppet Labs from Google HQ, where he was “Puppet Master” and responsible for the design and implementation of one of the world’s largest Puppet deployments. Kersten is also an experienced Linux and Mac deployment sysadmin.

    In case you aren’t familiar, Puppet Labs provides IT automation system configuration and management software, which is used by start-ups to very large enterprises. Here is our discussion with Nigel, where we talk the recent Puppet Labs DevOps report and high-performance organizations.

    devopsy.kinsta.cloud: In the 2014 State of DevOps Report, there seems to be a relationship between high-performance IT and high-performance companies. Can you see the causation and what correlation is there? Is it high-performance companies driving high-performance IT, or is it high-performance IT driving the high-performance company?
    Nigel: At the moment, it’s a correlation in terms of the amount of data we have. What the data actually show us from the survey is that adopting DevOps practices leads to a higher performing IT organization. This means you’re releasing code more frequently, your recovery time for failures is much, much faster, and all those sorts of things that we know go into what makes a great IT organization.
    We’ve actually been able to correlate that those organizations with higher performing IT teams are outperforming their competitors in the market, and they’re exceeding their own corporate goals. Now, at the moment, we’re just being a little bit tentative in saying that that’s a correlation rather than causative.
    To be clear, statistically we have a really high degree of confidence that it’s actually a causal relationship, not just a correlation. But we’re not quite ready to make that claim yet.

    devopsy.kinsta.cloud: We wonder how you’re seeing continuous integration change the relationships among security teams, application teams, operations, and QA. What are you seeing, broadly, when it comes to how these relationships are changing from the way they were several years ago?
    Nigel: I think that a couple of things are changing those relationships. One has been the slow death, I would say, in some ways, of waterfall approaches in favor of agile. That’s not to instantly label agile as good and waterfall as bad, because I actually think there are some development environments and problem spaces where a really strict waterfall-based approach is the right way to do things.
    But we’re definitely seeing a mass of organizations that are moving to a more agile setup. I think what this has meant is that the demands on those continuous integration systems and QA teams have changed quite a lot. Developers expect much more self-service, and they expect to be able to operate with a much tighter feedback loop. They don’t want to have to commit code and wait till the next day – until a 12-hour set of tests has finished running – before they find out whether their code is any good.
    Prior to this, I think what we saw for a couple of years was developers starting a sort of shadow QA on their own using Vagrant, using local virtualization. But definitely in the last year or two, we’re seeing a trend where QA teams are being forced to provide faster feedback and more self-service to the development organization.

    devopsy.kinsta.cloud: That must give heartburn to those who are used to having strict, steady, sequential processes in place.
    Nigel: It’s interesting, isn’t it? Some of this is a little speculative, because a lot of security folks tend to be very forward-thinking people, and I don’t think that’s necessarily true of everyone in the security industry, just like every industry. But I think it’s forcing people to look at their current security practices and work out what is just security theater and what actually improves the security of the product, or whatever it is that they’re shipping.
    So how can we automate those things that are actually improving security so that security doesn’t become the bottleneck in these fast feedback loops. agree that it can definitely give security people heartburn, especially people who are really used to rigorous code change approvals and the sort of segregation of duties that you put it in one of your pieces.
    What I see is a common compromise here, and I actually think this is a really effective way to do it. For certain environments, you really want to maintain what goes into production and have a strict change approval process. But it should be a fast and sort of appropriately light process.
    We’re seeing people giving developers the ability to run tests, change the testing environment, and have a significant degree of autonomy over the whole stack in pre-production continuous integration. People are focused on giving developers the ability to raise classes of machines, modify the tests that are running on them, changing all of those things so that they get a really fast feedback loop. But then when it comes to the actual merge to production, that’s something that your existing type change control process can take care of.
    This means that we see a lot of people segregating their infrastructure into dev, test, and prod, as far as continuous integration goes.
    And the developers do a lot of dev work locally on VMs. When it comes to test, that’s often on a shared infrastructure that, using something like Puppet, can be configured to be exactly the same as the production system. And that’s good enough for the developers. They’re getting their tight feedback loop. They’re testing code in pretty much the same way as it’s going to be tested in the official continuous integration pipeline, and yet you can still manage tight change control process around the merge from test to production.

    devopsy.kinsta.cloud: And if it works as planned like that, you would think you’d get improved code outcomes because the developers are working much closer to any defects.
    Nigel: Absolutely. And I think that’s the whole promise that this sort of system is meant to provide. I think it does fall down if you don’t have a very tightly disciplined development organization, and it’s just open season on the production infrastructure. In that case, it actually becomes counterproductive, because the feedback loops become longer and longer. Then, if the test fails, you’re never quite sure if it’s due to someone else messing with the system that’s telling you whether it passed or not.

    devopsy.kinsta.cloud: It sounds like they are self-correcting problems over time, because security often uncovers “hidden” or latent problems that you don’t see until something bad happens. But if it were a full onslaught on production, then I would imagine you would have other issues with availability and performance and related quality issues would mount up very quickly.
    Nigel: Absolutely. I think this is one of the reasons why people drive a lot of this work to the cloud, and I’m speaking from experience here, because the sys ops team I manage actually manages the infrastructure for the continuous integration system here at Puppet Labs. We have a pretty complicated testing matrix, and clearly security is a really big concern for us, because Puppet in many ways has the key to the kingdom when it’s deployed on someone’s infrastructure.
    So it’s really hard for a small company to provide enough internal hardware infrastructure to allow for that sort of elastic peak consumption. I think this is one of the big drivers toward people, at the very least, using the cloud to smooth out their internal hardware requirements so that, at times of peak capacity, they can essentially burst out to the cloud.

    devopsy.kinsta.cloud: Organizations that move this route, I assume, need to get to know their own processes very well before they move from what they were doing before – more manually driven processes – to continuous integration?
    Nigel: I think what we tend to see is that it ends up being on a project-by-project basis. If you’re a company that sells on-premise software or software as a service, you’ve got a reasonable amount of investment there, and it’s rare that companies can take the time to just pause the delivery of software. Usually, they need to keep making money; they need to keep shipping releases.
    So, what we tend to see is that people start experimenting with green field environments. This, according to my gut, is often enabled by people moving to more service-oriented architectures, where rather than shipping large, monolithic applications, they’re shipping components that have well-defined boundaries with each other.
    As part of that refactoring goes on, as they build a new component, they might do the testing and continuous integration for that component in a fundamentally different manner, and sort of step through their code base that way.
    Eventually, you have to deal with the brown field deployment. But, ideally, by that point you’ve gotten a whole bunch of huge wins from the green field continuous integration systems that make that sort of return on investment obvious.

    devopsy.kinsta.cloud: I would imagine that knowledge gained from those groups is then shared with small workgroups that experiment and then there’s a lot of knowledge sharing and expansion organically from within.
    Nigel: Yes. And with people being people, this is often where turf wars strike up. The have been a number of times when we’ve sort of walked into organizations and talked to the QA team and the developers and the operations people – and dev and ops have started working together and they can ship something much more quickly by doing things in that way – and yet somehow the QA team feels slighted that it’s somewhat bypassed by this whole process. That can be a rather complicated conversation to be in the middle of.

    devopsy.kinsta.cloud: Do you see companies make common missteps when they move toward continuous integration deployment?
    Nigel: One of the more conceptual mistakes I see people make when they’re considering it is that they confuse continuous deployment with making lots and lots of public releases, and they’re not necessarily the same thing. I often try to tell people that continuous deployment is really about enabling the ability to choose to ship something. It should be a shippable unit that shouldn’t necessarily be shipped, if that distinction makes sense.
    A lot of companies will start stressing out. They’ll wonder about their existing processes and educating their channel resellers. They’ll believe they simply can’t afford the cost of a release every two weeks, or every week. But that’s not necessarily the whole point of continuous delivery. By having artifacts that are shippable, you enable testing internally, you enable easier experimentation, and you still can choose to ship your product only once every six months if you want. But the fact that you might have 20 releases in between those public releases is important. They could all be shipped and could be a bug fix for a certain customer, or enable you to do certain types of development in the future. These are really huge wins, even if end users never actually see those specific releases.

  • Rackspace DevOps automation service broadly available

    Rackspace DevOps automation service broadly available

    Rackspace is the #1 player in managed cloud. While they recently announced they would be withdrawing from the pure IaaS market, they have now double-downed their bet on managed cloud with the general availability of the Rackspace DevOps automation service around managed cloud.

    Rackspace had previously announced their DevOps automation service a few months back, but that was a limited release. Now DevOps automation is one of the two choices Rackspace customers will have in setting up a managed cloud service.

    With this general release Rackspace is offering DevOps automation on both Linux based clouds utilizing Chef among other tools and on Windows using native Windows tools. In addition Rackspace offers enhanced environment stack support. According to the release:

    Rackspace now supports the following environment stacks, which can be deployed in hours on the Rackspace Hybrid Cloud: Chef, Rails, Node.js, PHP, Tomcat and Python. Customers can use recommended stack architectures and will receive updates, security patches and the latest versions of environment stacks, which are constantly maintained and tested by Rackspace DevOps engineers.

    These stacks are not containers per say a la’ Docker, but are self-contained packages that Rackspace can automate installation of and get customers up and running even faster.

    I spoke to Prashanth Chandrasekar, GM of Rackspace’s DevOps Business Segment about this announcement today. Prashanth and his team have been offering DevOps automation to customers since last year.  Today’s announcement represents DevOps going mainstream as one of the two preferred methods of deployment for Rackspace customers. I also spoke to Chris Jackson, the CTO of the DevOps team at Rackspace. Chris wrote a nice blog on the SysOp approach versus the DevOps approach.  It explains much of the thought process behind the Rackspace offering.

    There is also a good, if long Youtube from Rackspace on their DevOps automation service.

    Managed cloud DevOps automation was not the only thing Rackspace announced today. Just automating your cloud infrastructure by using a DevOps managed cloud service does not mean you are a DevOps organization.  Frankly that is just tools being used by a 3rd party. How do you instill a DevOps culture, mindset in your organization? How do you get your enterprise to actually become a DevOps organization?

    Rackspace is offering a full blown professional services offering called “DevOps Advisory Services”. They will come in and evaluate and analyze your needs and present organization. Based upon this they will make recommendations to help you. These include:

    • DevOps Workshops: Rackspace is working with customers to perform a range of services including application assessment, transformation planning and knowledge transfer, plus enablement around key areas like Configuration Management Tools, Provisioning and Deployments, Continuous Integration, Continuous Delivery (CI/CD) and Monitoring.
    • DevOps Strategy Planning: Rackspace DevOps specialists work with customers to understand their requirements and help create a custom project plan for their business needs and timeline.
    • DevOps Professional Services: Rackspace provides access to DevOps professional services from writing infrastructure code to designing structure and building migration plans for applications. This helps customers automate, deploy and test their infrastructure for their main applications, or a set of proof of concepts that the customer helps build.

    With today’s news Rackspace continues to demonstrate how important DevOps is to the go forward planning. Some may question how a managed services model would embrace DevOps, but Rackspace is showing the way. Branching out to DevOps pro services will allow them to compete head to head with IBM/SoftLayer offering cloud hosted and managed services wrapped with pro services.

    What I always find most compelling about Rackspace’s DevOps services though is that they eat their own dog food. Their DevOps services were born from their own internal use before they brought them to customers. It takes a DevOps organization to offer DevOps services.

  • The DevOps Diet (Getting Lean)

    The DevOps Diet (Getting Lean)

    IT has gotten fat – fat silos, fat processes, fat procedures.  Adopting DevOps is a great opportunity to look at some of our unhealthy habits and identify where we can eliminate unproductive waste from our process diets.  DevOps will ultimately teach IT how to exercise more with less effort, trim the fat and get lean.

    According to lean principles, the seven main areas of waste are:

    • Defects –  variations from requirements that result in interruptions and re-work
    • Overproduction – delivering something more or before it is required
    • Inventory  –  carrying excess raw materials, work in progress (WIP) or finished goods
    • Over-processing – doing more work than is required
    • Motion – moving people or equipment more than is required
    • Transportation – moving products from one location to another
    • Waiting – doing nothing or moving slowly while waiting on a previous step

    What are the greatest sources of waste in your IT organization?  Are defects being regularly passed downstream? Are ITSM processes overproducing through complexity and bureaucracy?  Are Agile teams  dealing with excessive work in progress and less finished product?  Are unresolved bottlenecks resulting in frequent delays while waiting for something to get done or someone to be available?  Could automation reduce “motion sickness” and make transportation nonstop? These questions will hopefully facilitate dialog between Dev and Ops about analyzing and streamlining existing workflows.

    While waste is fattening, it is not necessarily ineffective.  Projects and other tasks are still getting done and therefore some areas of waste may be difficult to recognize.  To become leaner, start simple by identifying and eliminating one unhealthy habit from your IT diet.   Engage stakeholders from Dev, Ops, customers and suppliers (if appropriate).  Discuss ways to improve the work with healthier habits.  Agree on what to do next or what not to do anymore.

    Every diet should be supported by recipes for success.  The upcoming DevOps Cookbook by Gene Kim, et al. will provide more tangible guidance and recipes for healthy IT habits.  More on that in future blogs.

    Most of all,  identifying and eliminating waste through Lean DevOps is a great opportunity for collaboration.  After all,  it’s always better to diet with a friend.

  • Microsoft strives to embrace open source culture

    Microsoft strives to embrace open source culture

    In many ways Microsoft is going through the corporate equivalent of a mid-life crisis—it has reached a stage where it longs to drop a few pounds, and turn back the clock a few years. After years of tremendous success, it’s now starting to show its age, and it is struggling to adapt to the new tech landscape and compete with younger, leaner, more innovative competitors.

    The shift began under ex-CEO Steve Ballmer. Ballmer’s approach, though, was misguided. Ballmer’s Microsoft suffered from hubris, and ignored the seismic shift in technology until it was too late. When Ballmer finally recognized the existential threat, and tried to turn the ship around, it was done from within the established bloated corporate culture. There was an effort to rearrange the chairs on the deck with a management reorganization, but it simply wasn’t enough.

    Then Satya Nadella took over. Nadella has a more aggressive vision. It’s still not an easy job, nor a goal that can be accomplished overnight with a behemoth the size of Microsoft, but Nadella has dramatically changed the culture, and the vision for where Microsoft is heading.

    Part of the new and improved Microsoft includes embracing communities that Microsoft has long ignored or alienated, and shifting the way things get done at Microsoft so the company can be more agile. It is evident in how Microsoft partnered with Chef and Puppet Labs to integrate popular DevOps tools into the Azure cloud platform, and it can be seen in how Microsoft is adopting open source.

    “Recent articles have shown that Microsoft is “open sourcing” code to their internal teams. The fact that that is advertised as innovation is telling,” declared Michael LaVista, CEO and Founder of Caxy. “That’s what smart organizations have always done. In fact, Microsoft has somehow found itself the underdog in this sense and probably needs to do things like this to attract talent.”

    Leveraging open source concepts internally isn’t quite the same thing, though, as actually making Microsoft code open source. Open source projects like Apache, Project Libre, or Docker share source code with the general public, and anyone can review the code, and submit changes to improve or fix the code. Open source communities collaborate to continuously enhance the code. Microsoft appears to be doing something similar, but without exposing its source code to the general public.

    LaVista suggests that Microsoft should take that next step, though, and work collaboratively with the open source community outside of Microsoft’s walls. If it does so, Microsoft will some issues to address. Will it accept code fixes from a 14-year old on the other side of the world? Will exposing source code for the Windows operating system, Microsoft Office, or other popular Microsoft applications open up a Pandora’s Box of security concerns?

    “They will have to embrace what happens to code when it is open sourced. Once the code is out there, everyone can see the flaws,” explains LaVista. “The benefit of the model is that you have so many more eyes looking at it from more perspectives. If Microsoft is used to running their own software and making their own decisions about what to work on and fix, this could pose problem.”

    Perhaps. LaVista makes good points about concerns Microsoft would face, but in and of themselves they’re not a reason not to adapt and evolve. Yes, exposing Microsoft source code to the general public would probably result in a spike in security concerns, but once the dust settles the collaborative approach of an open source community addressing those issues could result in quicker patch releases, and make the software more secure.

    LaVista recommends that Microsoft use companies like Red Hat, IBM, and HP as examples of how a larger corporation can be successful in an open source culture. Only time will tell. We’ll have to wait and see to what extent Microsoft makes the shift to open source, and we’ll have to wait and see how Nadella’s Microsoft adapts, and whether Microsoft can successfully reinvent itself to remain competitive in this new tech landscape.

  • Q&A with Kevin Behr on DevOps and Enterprise Excellence

    Q&A with Kevin Behr on DevOps and Enterprise Excellence

    When we want to get an inside peek at the trends within major corporations when it comes to continuous integration, continuous delivery, and DevOps, we turn to Kevin Behr, the chief science officer for the CXO and Board Advisory Practice at Praxisflow, where he consults, mentors, and coaches IT organizations and executives on how to increase their business effectiveness. Behr also is the of seven IT management books, including the bestselling “The Visible Ops Handbook”, which he coauthored with Gene Kim and George Spafford. His most recent book, The Phoenix Project, also coauthored with Kim and Spafford, has helped to spark discussion about DevOps globally.

    Today, we sparked a discussion with Behr about how smart enterprises are using DevOps to further differentiate themselves and enterprise excellence.

    devopsy.kinsta.cloud: There seems to be a lot of misconception out there about what DevOps is – and I’m increasingly hearing it referred to as a software methodology. Are you seeing this?
    Behr: Definitely. DevOps is not a software development methodology. I think that is a big misconception out there. It was a big misconception in the in WSJ post, DevOps is great for startups but for enterprises it won’t work yet. The author had that completely wrong: DevOps doesn’t tell us how to develop software. DevOps literally is collaboration between IT operations and development. So DevOps could be about developing software. It could be about fixing bugs. It could be about fixing integration issues. It could be a whole bunch of different things they’re collaborating on. But it doesn’t necessarily denote how you would develop software.

    devopsy.kinsta.cloud: But there’s no prescriptive connotation to the word at all, is there?
    Behr: Exactly. My whole point is that the only prescriptive connotation is “Can’t we all get along?” Can’t we work together to do the work we do? I think sometimes that means we’re not developing software that means sometimes we’re doing a lot of different things. In some cases, software developers are helping sys admins write better scripts, and do better sys admining. And vice-versa: sys admins are helping software guys understand requirements better, and understand the end delivery point, the people that are going to use software or service this, better. That’s not telling us how we must develop.

    DevOps is about asking: what can we do to improve, what can we change, and what does this change mean? We’re talking to CIOs like crazy right now about this. We’re trying to figure out how can they leverage DevOps without causing all kinds of organizational trouble. On one hand, we’ve got people in the enterprise that are happy with an ill-defined DevOps. To be honest with you some are saying it’s all kinds of things because that creates a subterfuge where they don’t have to do things to make it work. “If DevOps is confusing, if it’s impossible, or if it’s stupid, maybe we won’t have to actually do it,” they think.

    It’s kind of pathetic to watch. We are watching them and wondering: “Wow. Do you really think that doing these in Ice Age speed and style deploys and old school change management that takes 30 people in a room, once a week, to decide what changes are going to get made is okay today?

    devopsy.kinsta.cloud: That’s just insane to hear and think that it is going on. Outside of IT, how are business people responding to DevOps and continuous delivery.
    Behr: I’m telling you, the coolest part about it for me is the business people have reacted to the DevOps message. They’ve done so because we’re talking about changing the business payouts. They’re drastically different now. Organizations that can do these deploys, not just because they’re putting out more software, but because the organization as a whole is focusing on delivering more software to the end user. Even the commercial software companies are way behind what a lot of DevOps companies are doing now when it comes to driving innovation and better outcomes.

    When you look at these big software companies that are still releasing software every eight months, they’re really slow and terrible. We’re all suffering from the quality and the slow feedback loops, and the disconnectedness of regular commercial software. These smaller companies, a lot of them are start-ups albeit, but not all of them, and a lot of these banks that have embraced DevOps are drastically changing the old-school cycle time game. They’re getting faster. They’re reducing cycle time, and they’re reducing defects. That adds up when it comes to better applications and producing a lot less waste.

    devopsy.kinsta.cloud: What does this mean when it comes to impact, can the deficits be measured?
    Behr: Yes, because it ends up taking you forever to get anything out. You’re now suffering from that nasty cycle of unplanned work. It’s similar to all of the work that they call failure demand, where the failures in the quality of what you build are creating demand for work. You’ve already got competition for that work, because your company wants to release new value features, but you can’t because you’re cleaning up the old messes. It’s ugly.

    Now, and the other thing that’s interesting too, here, is that when you move to SaaS, you’re moving to more release cycles per year. The average SaaS provider is going through four major platform upgrades a year. They’re doing four releases. Think about that change from a traditional model, where you’re adding three more releases a year, essentially.

    And every release requires maintenance releases, so now you’re going to have patches. A lot of these companies have been putting SaaS into place, but there’s not a whole lot of SasS that’s profitable from a business perspective right now. There are a lot of companies doing it, but there are not a lot of companies that are in the black. What’s interesting is that they’re spending a lot of labor to maintain the software. It’s expensive, and you have to build it before the customers come. That’s why we see so many people come and go out of that space.

    devopsy.kinsta.cloud: What does that mean for business then, when it comes to success and IT’s relationship to business success?
    Behr: I think what’s interesting is that you’re hearing business people’s expectations change. The business always used to say, “Well, I need that thing tomorrow.” And IT would say, “Well, you’re going to get it when you get it.” Now the business is coming back, saying, “No, I need that tomorrow, because my competitor already has it.” Those rushed cycles used to be crying wolf. But today there is legitimacy to the pressure. They’re actually crying wolf because the wolf is there.

    The dinosaurs are going to be in big trouble, because they are competing with organizations that can do 10, 15, 20 or 40 deploys a day.

    Some are deploying with feature switches, as dark code, so all of those deploys that they’re doing may not even be active features that people are using. But they’ll turn on a feature, see if it works, and if it doesn’t they will turn it back off. No customer impact. They test it in production. They can also then roll out code that nobody even knows is there, and also be testing it on actual, live infrastructure. What they’re doing is improving so much faster than everybody else this way.

    These organizations are going to separate themselves from the pack because of how efficient they are becoming. This is how DevOps is one of those approaches that a lot of organizations are using to make it harder for other people to catch up with them. If they’re a super-predator, in other words if they have a breakthrough idea that nobody else does in some sort of technology, why not spend the dollars and couple that technology with a service approach, or a design from an organizational perspective, so that you can continue to out-improve everybody? DevOps does this.

    You’d be insane not to. So now you have a dual threat. You’re not just offering a better product, but you’re a better product that’s actually iterating faster and separating at a rate that’s going to be hard for other people to deal with. This is how the slow moving enterprise is losing right now. It’s content to think that it’s not losing that bad. The thing is, the game is changing so fast in organizations that are able to do this that they’re able to separate so much faster, and outcompete those that can’t.

  • 8 tips on how to best manage distributed teams

    8 tips on how to best manage distributed teams

    More and more organizations are using distributed testing teams yet many are hard pressed to know how to manage teams that may be separated by thousands of miles away or on another continent. Too often, companies think about offshore product development only as a means to cutting costs and accelerating time to market. Such an approach can produce poor results with your projects if you don’t take into account certain issues like cultural barriers, metrics and workflows.

    The smart strategy is to focus on the needs and skills of the team and to implement procedures that will help manage them better. Your standard bottom-line goals — cutting costs and accelerating time to market — will take care of themselves if you can follow these 8 tips for managing distributed teams.

    1. Know Your Team

    It’s important to address people correctly. People respond well when somebody knows how to spell and pronounce their name. To infuse a sense of team spirit, attach a face to the person who is communicating via email, phone or IM. Have a kick-off meeting for a project team, where each person introduces themselves with some basic personal information such as a hobby or interest, to help establish a ‘human-side’ to the offshore team. While meeting in person might not be a possibility, the liberal use of video conferencing can really improve the relationship with remote teams.

    2. Eliminate Language Barriers

    Working with remote teams on different continents/countries requires strong communication. It’s critical for the key contacts between teams to familiarize themselves with the pronunciation and grammatical styles of each other, while also ensuring that your team members obtains International English certifications, such as TOEFL. In addition, when possible, I recommend that the team leaders from the vendor side and client side visit each other’s offices to understand the culture of the companies and to instill a sense of partnership.

    3. Clearly Define Roles and Responsibilities

    To ensure that all members of the project team understand their expected tasks, it is vital to establish clear roles and corresponding responsibilities early on. Create a collaborative, pyramid-style team structure for the offshore team, so that the key people responsible for daily communication with the rest of the project team are identified, along with an obvious reporting hierarchy.

    4. Create Strong Communications

    The distance between remote teams working on the same project can be eliminated by use of communication tools such as IMs, VOIP-based phone calls, online meeting tools and other electronic mediums. However, communication is only as good as the information provided. All relevant documentation and domain knowledge must be continuously shared with all team members.

    5. Track Detailed Onshore/Offshore Planning and Tasks

    To ensure that your teams understand the assigned tasks and is progressing as expected, it is very important to have a milestone-driven WBS (Work Breakdown Structure) that identifies the effort associated with each task, and identifies the timeline of when the tasks need to be completed. This level of detail ensures that everyone is clear on their assignments, and that any delays (and corresponding affects) can be identified as early as possible, so that a mitigation plan can be implemented if need be. Team progress on assignments and activities should be followed up with daily, weekly and monthly reporting, leveraging defined templates where possible.

    6. Establish Process Workflows

    A clearly defined testing activity workflow should be established and reviewed with the offshore team, to ensure that all team activities are synchronized. Documenting a process workflow allows the team to streamline, validate and verify expected activities that could impact the project delivery timelines.

    7. Don’t Forget Metrics

    Offshore team deliverables and activities should be gauged with quantifiable metrics such as timeline delays, actual-to-estimated effort, QA-to-production defect ratios, and so on. Metrics should be established under at least three categories: timelines, productivity, and quality. These metrics should be established at the beginning of the project, and tracked either by release or on a monthly basis.

    8. Establish Roadmap for Refinements

    To ensure that the offshore team continues to increase its value to the project team, it is important to leverage the daily, weekly and monthly reports/metrics to identify areas of improvement/streamlining for testing processes and test strategies. These refinements should be prioritized and scheduled for implementation by the team, and tracked for progress along with other project deliverables.

    In summary

    Establishing and managing distributed teams is a complex challenge. Cultural differences, time zone variations and lack of face-to-face time contribute to the intricate nature of this model. In addition, some companies focus too much on cost-cutting objectives. To successfully manage distributed testing teams, a company needs to focus on people, processes and communications.

    About the Author

    dave-keil-QASymphony CEO_cropDavid Keil is CEO of QASymphony, a leading provider of test management platforms for agile development teams. He previously served as CEO at Digistrive and at Integrated Broadband Services. He holds an MBA from The Wharton School and a BS in Applied Mathematics and Economics from Brown University.

     

  • devopsy.kinsta.cloud sponsors FlowCon

    devopsy.kinsta.cloud sponsors FlowCon

    devopsy.kinsta.cloud is happy to be a media sponsor of FlowCon this year. FlowCon is a popular event held in San Francisco that focuses on lean product development, continuous delivery, lean UX, and (our favorite) DevOps. From the show producers:

    FlowCon brings together technologists and industry leaders passionate about innovation through lean product development, continuous delivery, lean ux, and devops. We’ll be exploring the role of culture, technology and design in growing organizations that thrive in an environment of continual change. We will provide inspiring and actionable information for key decision makers responsible for products and services that depend on software.

    Our full program includes speakers from Google, Netflix, Heroku, Nordstrom, Soundcloud, Macy’s, HP, Joyent, ThoughtWorks, and IBM. The second day features an open space unconference, and workshops from Don Reinertsen, Mary and Tom Poppendieck, Whitney Johnson, and Sarah B Nelson.

    This conference is especially relevant for our audience because of the intense focus on environment with “continual change.” DevOps is a necessary ingredient for this environment because when the team is focusing on innovation and improvement, it becomes vital to have tedious tasks automated – not just to save time, but to ensure accuracy and efficiency. The most valuable resource at many organizations is time, and one mistake in a simple process can be critical.

    Our mission is to help build the DevOps community; to help the DevOps movement cross the chasm into the mainstream. We want to unite entire organizations behind a DevOps culture, taking lean and agile from being an exclusive club where only the dev team is invited, to being integrated into the overarching company strategy. Everyone is invited.

    We’re bringing this message to FlowCon. Come join us. If you do use this code:  Devops50,  for $50 off your registration.

    FlowCon Banner 2

     

    This is from last years event:

  • Mobile App Development Will Soon Grow More Enterprise-Focused

    Mobile App Development Will Soon Grow More Enterprise-Focused

    As the global mobile app market begins to swell, many consumer-focused developers inexperienced in the ways of enterprise development will soon follow the money to become better initiated, a new report out this month explained. Until then, the more mature types of development methods and tools that enterprise developers depend upon are still relatively scarce in the mobile app development community.

    According to analysts with VisionMobile, which recently released its Developer Economics State of Nation Q3 report,  mobile developers accounted for 12.6 percent of the global developer population in 2013, but the number of developers in these ranks is expected to grow to close to 5 million in the next three years.  As things stand at the moment, more than two out of three mobile app developers currently target consumers. And even among those 27 percent who do focus on business users, 11 target individual business users—the independent BYOD early adopter type that many an IT department  has struggled to keep up with.

    That leaves just 16 percent of the mobile app developer population focused on the enterprise, with about half of them working in the enterprise environment and the other half in third-party or freelance situations.

    Given those demographics, it is no surprise how few tools the entire class of mobile app coders uses during development. Currently, just 9 percent of these developers use any kind of automated app testing tools, only 16 percent use crash and performance monitoring tools, and just 21 percent use beta testing tools. Even the use of cloud computing tools lags, with just under 30 percent using services such as AWS or Heroku to aid their development.

    It’s a perplexing problem, according to the report’s authors.

    ” Developer tools add value at every step of the developer journey from prototyping through to customer support,” the report stated. “It makes sense that hobbyists with no intention of making money from their apps would avoid tools that might incur costs at some point. However, 26% of developers that are interested in making money don’t use any third party tools.”

    This could potentially change as the economic incentives for more mobile app developers to shift to the enterprise and business users push them to create the more dependable apps features that these users demand.

    According to VisionMobile, overall the app economy stacked up to $53 billion in 2012, and its analysts predict 28 percent growth annually through 2016, when the market will reach $143 billion. Currently, the enterprise market leads the pack in terms of revenue generation opportunity, with developers focused on enterprise audiences far and away leading the pack in revenue from their apps, followed by those focused on individual professionals.

    “Smartphones and tablets were introduced as consumer-first technologies, which led to developers initially building consumer apps. Now a growing number of enterprises are adopting mobile technology; this is all businesses, governmental and non-profit organizations, not just large corporations with IT departments,” the report said. “The platform owners are providing clear signs that there is much greater enterprise adoption and significant opportunities for developers to come.”

    In an earlier report this year, VisionMobile predicted that the business and productivity market will double in size by 2016, reaching $58 billion by then.

  • In Support of DevOps: Kanban vs. Scrum

    In Support of DevOps: Kanban vs. Scrum

    The Kanban software development methodology is getting a lot of attention as of late, particularly for its ability to enable DevOps. Some organizations are even going so far as to move from Scrum to Kanban to improve efficiencies. We use Scrum at OutSystems, since we’re doing feature development that requires stakeholder feedback. The sprint lock and end-of-sprint demos to stakeholders are invaluable to us. But I was curious to see how the two methodologies differ and how Kanban may further enable an organization’s move to DevOps. Here’s what I found.

    There are two key differences between Scrum and Kanban: the rules and the workflow.

    The Rules

    Both software development methodologies provide rules for how work is performed. If your organization uses Scrum, then you know that it is quite prescriptive. Agile Advice lists 23 mandatory and 12 optional rules for implementing Scrum. These include, for example:

    • A product owner creates and manages a product backlog
    • Teams are cross functional
    • Interruptions are prohibited during sprints
    • Work is time boxed
    • Scrum meetings are held daily
    • A burn-down chart is used to measure progress

    Taken together, these rules create a fairly rigid box in which teams must operate in order to be successful with Scrum. As I see it, this presents two big challenges. First, few organizations actually follow all of the rules, leading to what is called ScrumBut – We practice Scrum, but… ScrumBut means a company ignores specific Scrum rules for internal reasons, leading to a suboptimal use of the framework.

    Second, at the core of Scrum is the time box. Time boxes are good for providing distraction-free time for developers to deliver the product, and define regular milestones for stakeholders to evaluate and steer the project. But, from a DevOps point of view, these artificial delivery checkpoints break the flow. Coordinating dependencies between sprints and ensuring their successful move to production becomes a big challenge.

    Kanban, on the other hand, is less restrictive. Its rules are as follows:

    • Visualize the workflow
    • Limit work in progress

     With only two rules, Kanban is an open and flexible methodology that can be easily adapted to any environment. (Some companies are using it across the entire organization – from manufacturing to marketing.) You can add rules as needed, borrowing from Scrum, if you so wish.

    The focus on the flow instead of time boxes makes Kanban an attractive choice to use with DevOps. Deploying work items becomes just another step in the process, and with its emphasis on optimizing the whole delivery rather than just the development process, Kanban and DevOps seem like a match made in heaven. With the contrast in rules understood, let’s look at the workflow.

    The Workflow

    In addition to rules, the workflow differs in Kanban vs. Scrum. This is a direct consequence of the rule sets. In Scrum, you decide beforehand what features will be completed in the next sprint. The sprint is then locked, the work is done over the course of a couple weeks (the usual sprint duration), and the queue is empty at the end of the sprint. Locking the sprint ensures the team has the necessary time to work on a problem without being interrupted with other apparently “urgent” requirements, while the feedback sessions at the end of the sprints ensure stakeholders approve the delivered work and steer the project as the business changes.

    In Kanban, you do not have the time constraints of a sprint, and the focus is on making sure the work keeps flowing with no known defects moving downstream. However, limits are placed on the size of the queues. This means that a maximum number of features or issues can be worked on at a given time, allowing the team to focus on delivering those work items with high quality. Making the flow visible creates a sense of urgency to keep things moving along. Note that Kanban was born in manufacturing, and its big focus is on productivity and efficiency. Kanban has to be extended in order to incorporate fundamental aspects of software success, such as stakeholder participation.

    Enter DevOps

    Organizations that adopt DevOps are looking to increase efficiencies, deploy features more frequently and be more responsive to business demands. You can see, then, how each methodology addresses various aspects of DevOps to greater and lesser degrees. Choosing one over the other isn’t cut and dried. Here’s what I advise:

    If your team is responsible for feature development, which requires stakeholder feedback, and developers need to focus, Scrum is probably a better fit. The sprint lock and end-of-sprint demos to stakeholders can be invaluable.

    If your team is responsible for maintenance and tends to be more reactive, then consider Kanban. There is no need to lock your backlog and you have more flexibility to respond to customer input.

    Ultimately, every team is different. You know where your team excels and where it needs extra guidance. I suggest you borrow from both methodologies to give your team just what it needs to achieve DevOps’ goals.

    About the Author

    Rodrigo CoutinhoRodrigo Coutinho is Principal Software Engineer, OutSystems. A member of the founder’s team at OutSystems, Rodrigo has a passion for web development, great products, and geeky stuff. He spends his time designing future versions of OutSystems Platform and dreaming about the cool future of the web.

  • DevOps by any other name still gets things done

    DevOps by any other name still gets things done

    My version of “A rose by any other name would smell as sweet” is not nearly as eloquent as Shakespeare’s, but the sentiment is the same. Juliet argues in Romeo and Juliet that the name of a thing such as a rose is irrelevant—it’s still a rose no matter what you call it. The same thing is true when it comes to DevOps.

    The term DevOps has achieved industry buzzword status. Most people—at least most people in IT—are familiar with the term even if they don’t have any idea what DevOps is. Even among those who think they know what DevOps is, there may not be consensus on how to define or implement the concept. It is understandable that some people don’t have a clue what DevOps really is.

    The Wikipedia entry for DevOps is a great illustration of the issue. Wikipedia is a crowd-sourced encyclopedia, and as such it is comprised of a variety of contributions. The first few paragraphs of the DevOps Wikipedia entry actually offer up different attempts at a definition. The various explanations don’t necessarily conflict directly with one another, but they also don’t agree completely and reading the Wikipedia entry will leave most people just as confused about what DevOps is or isn’t as they were before they started.

    It’s a “software development method that stresses communication, collaboration and integration between software developers and information technology (IT) operations professionals.”

    The goal of DevOps is “to maximize the predictability, efficiency, security and maintainability of operational processes. This objective is very often supported by automation.”

    It “targets product delivery, quality testing, feature development and maintenance releases in order to improve reliability and security and faster development and deployment cycles.”

    It “aids in software application release management for a company by standardizing development environments.”

    All of these are true, and yet none of them is necessarily THE definition of DevOps. The simple reality is that it doesn’t really matter.

    DevOps is not a single thing per se. It is a culture, and how that culture is embraced or implemented van vary from one organization to the next without altering the fact that both are employing DevOps. DevOps is not a product or tool–it’s a culture that has evolved organically to meet the needs of a more rapid pace of IT. Many organizations are utilizing DevOps processes or tools, and have no idea what DevOps is.

    If your organization has developers who manage operations, or IT ops staff who do development, or developers and IT operations staff who work closely together to get things done, you might be doing DevOps.

    If your organization uses scripts or other tools to automate the provisioning and/or configuration of servers, virtual servers, or entire virtualized environments, you might be doing DevOps.

    If your organization rapidly develops new iterations of an application, or continuously rolls out new features and tools, you might be doing DevOps.

    If your organization continuously monitors for suspicious activity or potential malicious attacks, you might be doing DevOps.

    If your organization makes extensive use of the cloud in order to be more agile, and be able to scale quickly to meet demand, you might be doing DevOps.

    If your organization lacks rigid structure between departments, and functions more like a startup where everybody does everything, and just does what it takes to get things done, you might be doing DevOps.

    If your organization automates simple tasks and functions with technology rather than wasting valuable man-hours doing everything manually, you might be doing DevOps.

    It doesn’t matter if you understand what DevOps is. It doesn’t matter if you call it DevOps within your organization. All that matters is that you take the elements that make sense for you, and adopt them in order to function more effectively and efficiently, and remain competitive.