devopsy.kinsta.cloud

Tag: Management

  • NGINX Delivers New Products to Manage Microservices

    NGINX Delivers New Products to Manage Microservices

    At its annual user conference in Portland, Oregon, in September, NGINX delivered releases focusing on the application platform market and support of delivering microservices architectures:

    • NGINX Plus, an application delivery controller that combines a load balancer, content cache and web server.
    • NGINX Controller, a centralized management and monitoring platform for NGINX Plus, which orchestrates the delivery of applications across multiple environments, enabling companies to continuously deploy and update applications using tested and proven policies.
    • NGINX Unit, a multi-language server for applications, designed to work in highly dynamic environments. It features a full REST API that can be fully automated and used to deploy new application versions with no service disruption. It currently supports PHP, Python and Go with more language support coming soon.
    • The NGINX Web Application Firewall (WAF), powered by ModSecurity, which protects web applications against various Layer 7 attacks and provides DDoS mitigation, real-time blacklisting and audit logging.

    In addition to launching the NGINX Application Platform, the company has added the ability to use NGINX Plus as a Kubernetes Ingress Controller. Based on the open-source NGINX Ingress Controller for Kubernetes, this new feature enables the deployment of applications within Kubernetes and Red Hat OpenShift anywhere across a cluster so they can be reached by outside traffic.

    While these products combine to deliver a complete and manageable microservices solution, the most interesting of these is NGINX Unit. According to Owen Garrett, head of products, the complement of tools facilitates management of both north/south and east/west traffic for cloud-native applications. It’s small, lightweight, fast, polygot-enabled and programmable through an API and works with other Unit instances to deliver a service mesh.

    Key features of Unit that should be of particular interest to businesses deploying microservices relate to the ability to foster continuous delivery. For example, when the router accepts new configurations from the Controller process, the worker threads start to handle new incoming connections with the new configuration, while old connections can continue to be processed by the threads according to the previous configuration. That is: Router worker threads can work simultaneously with several generations of configurations.

    Additionally, Unit uses interprocess memory to communicate with the applications, thus allowing the Unit to provide greater agility in routing of HTTP requests. So, rather than forcing the application to directly listen, they can delegate network handling to the service mesh providing for increased scalability. They accept the clients’ requests, pass the requests to the application processes, get responses back from the applications and send the responses back to the clients. Each worker thread polls epoll or kqueue and can asynchronously work with thousands of simultaneous connections.

    — Alan Shimel

  • With DevOps Culture, It’s Mind Over Matter

    With DevOps Culture, It’s Mind Over Matter

    There is quite a bit of conversation about tools versus culture, and sometimes, those constantly involved in the back-and-forth forget that not everyone is so steeped in the finer points of the conversation.

    Tools is the easy part. There are a ton of tools available to optimize IT operations, from small bespoke products such as the Antsle Private Cloud Server to CA’s broad suite of Dev/Test/DevOps/ARA tools. There are plenty of places to find lists or charts of tools, and I’ll leave you to them, just mentioning the two I’ve written about in the not-too-distant past: Xebia Labs’ Periodic Table of DevOps, and CA/Automic’s Continuous Delivery Landscape.

    My youngest just built a Cube-solving robot, so this picture inspired me.

    But what of culture? When pundits and old hands talk of culture, what exactly are they talking about?

    Well, it still varies a bit from organization to organization, but there are some definite underlying foundations.

    Increased Cross-Team Communications

    Even if an org is full-on DevOps, there is still the requirement of cross-team communications to make sure everyone understands what is going on.

    One of the things that this does not make abundantly clear, I will belabor a bit (sorry, old hands). Communications is not just talking. It is not just email exchanges, it is not just sitting in another team’s meetings while researching your current problem on your phone.

    It can be these things, but it is a breadth of options. People are using Slack channels to keep everyone updated, common notifications or even pub-sub systems to make certain information about the state of IT, operations and builds are getting wide distribution. Increase communication, not chatter.

    Organizational Change

    You hear that a lot, and it can mean complete overhaul of an IT organization, but in an enterprise setting, it rarely does. The reason is simple. There may only be one or two Cisco specialists, or one or two EMC specialists, and these people don’t properly belong on one or two product/project teams in the long term. The same goes for security and a whole further swath of operations team members.

    So in the enterprise, what “organizational change” often means is moving those responsible for development and core deployment closer together on small, product- or portfolio-based teams. Networking staff, security staff and infrastructure staff will join the teams as needed, but are not part of the core team.

    Management Change

    This is another often spoken-of mantra that easily can be misunderstood. In short, managing a small, agile, entire product lifecycle team is not the same as managing a pool of resources that pop from project to project and are either (a) not involved in the operations portion at all, or (b) not involved in the development portion at all.

    For a great set of perspectives on this topic, I’ll recommend our partner The DevOps Institute. It has an eBook and a certification course (DevOps Leader) on this topic.

    Decentralization, Yet Increased Oversight

    This is often a confusing pair of messages we send when talking about what DevOps culture change does. It really is not that complex—pundits just assume everyone already knows, and, honestly, those knee-deep in DevOps do know.

    Decentralization, as taught in the DevOps Leader course mentioned above, is about driving as much decision-making as possible down to the team. When teams are built around a product or group of products, then those with the most pertinent information about what is best for the product quite often can be made within the team, speeding decision-making and implementation.

    Increased oversight is an acknowledgement of two things. First, there is more information available in an agile environment, and that information is available more frequently. Standup meetings offer some, while milestones offer other data. Second, tools are starting to allow rollup of information in a uniform fashion for CIOs, CTOs and business leaders to look at. This means they can quickly see the status of projects they have a vested interest in and provide feedback into the process. To go from, “Oh, this roadmap item is going to take a lot more work than we thought,” to, “Great, we have a plan, let’s keep moving,” is shorter when there is more information available to the decision-makers on both the business side and the IT side.

    There is no doubt plenty I have missed, but these seem to be the core things that those of us discussing this topic a lot assume others know, and that those just looking into DevOps really don’t. Hopefully it will help someone push through the clutter and come to a decision on DevOps in general, or culture changes in particular.

    — Don Macvittie

  • So Long to Configuration Management Systems?

    So Long to Configuration Management Systems?

    When I was an operations architect at Ning way back in 2009, we were struggling to manage our deployments and configurations of more than 50 different application servers, or what the cool kids today call microservices. Applications were deployed across three distinct environments: development, staging and production. None of these environments were exactly the same as another, and differences in these dependencies, together with human errors, frequently broke our deployments.

    Ning was nowhere near the first company to encounter such problems. In the mid-1990s, as the public internet emerged, much of what was understood at the time about IT operations was challenged by the need to scale internet-facing systems. For many years preceding the internet boom, the maximum number of users was strictly limited to the number of employees in a company. That all changed with the World Wide Web, where the need to serve thousands and millions of simultaneous users easily exceeded the computing capacity of even the largest and most expensive computers.

    As three-tier application architectures were broken down into services and distributed across dozens or hundreds of individual servers, this proliferation of servers proved to be far too much for operations teams to handle. The worst problems lived within application deployment processes, where environments and requirements were ever-changing in successive releases of software. The lifetime of a server was a long one. Once deployed, servers were subjected to repeated changes by operations and developers in what would come to be called “configuration drift.”  The combined effects of changes, human errors and unknown states would create disastrous outages that damaged brands and destroyed online revenue. It would take sites days and even weeks to recover from these outages. Some would never recover.

    It was in the midst of this chaos that modern configuration management systems were born. CFEngine was the first to emerge in academia, where the need to manage multiple UNIX systems was felt early on. Later, Puppet, Chef, SaltStack and Ansible emerged directly from web-scale systems. Configuration management systems fulfilled several important needs. First, they abstracted the differences between operating systems, providing a high-level language for the system administrator to describe the desired state. Second, they automatically synchronized the desired state across large numbers of servers, reducing costly human errors. Finally, they provided idempotency to ensure that destructive operations were not applied unnecessarily.

    There is never a free lunch. The benefits of better abstraction were offset to a degree by the need to build and maintain evermore complicated recipes. Recipes frequently broke from time to time as package management features were updated, renamed or otherwise changed from the time the recipe was first written. Even when recipes worked, they were frequently broken by differences in environments or runtime errors. The configuration management systems themselves began to suffer from scale challenges, and evolved message bus architectures to improve their speed and responsiveness. By overcoming many of these obstacles, configuration management systems have now become a first-line tool for managing web-scale systems.

    But back at Ning, modern tools such as Puppet and Chef were still in their infancy and simply could not cope with the scale of our 60,000+ node virtualized environment. Ning had a lot of brilliant software engineers floating around, and they set about creating their own deployment automation system called Galaxy, and later a meta-orchestrator called Cosmos. Galaxy and Cosmos were great tools, but they required constant care and feeding and suffered from severe scale problems. There was just a lot of complexity in dealing with these automation systems while also trying to scale a top-tier social network for a hot startup company.

    As we began to analyze and post-mortem our deployment failures, it became clear that a majority of the problems stemmed from host and environment configuration errors. We needed a simpler way to deploy our software. Virtual machine images were a promising way to ensure consistency between environments with the tried and true “gold master image” approach. The problem with this approach was that building full system virtual machines was difficult in automation terms. Once deployed, virtual machines are also subject to configuration drift. Furthermore, while we were eager to adopt virtualization to get better resource isolation, we were wary of the overhead that hypervisors would create. Our benchmarks showed that if we adopted virtual machines, we would need to increase our physical server capacity to accommodate the overhead.

    At the time, Linux containers were already available in the kernel, but the distributions had almost no support for managing them. At the insistence of our VP of Operations, I cooked up an alternative solution based on jails, writing a small hack of a program called Warden to build and manage them. (Note: Totally unrelated to the Warden in Cloud Foundry). Jails gave us bare metal performance, resource isolation and a way to consistently deploy the entire environment for each application server.  With Warden, we could build jail images on the fly automatically and deploy them consistently with our existing tools, galaxy and cosmos. Hey, it was a Band-Aid for a difficult situation, but my experience with building Warden flipped a switch in my brain. I had seen the way lightweight virtualization could eliminate complexity, and I knew right then this was the way of the future.

    Today, with the success of Docker we are finally experiencing sweeping changes in the processes and tools we use to build, deploy, and manage distributed applications. At the same time, we are about to enter a brief period of containers disillusionment as early stage solutions get washed out. When that is over, what comes afterward is something of a golden era for distributed applications. I call it the container application platform, and its somewhere between what we think of as platorm as a service (PaaS) and Serverless today. In that not-too-distant future, containers will be the default output of the software build process, and where that container gets deployed will hardly be an afterthought. This unicorn magic will come from the ability to introspect containers to determine their environment dependencies, and a fully integrated tool chain that deals with deployment, scaling and dependency resolution automatically. I will give you just one guess which company is in the best position to own that future. “One ring to rule them all.”

    What happens to our beautiful configuration management systems? There has been much debate about this among smart people over the last few years. Most of these arguments boiled down to some kind of coexistence model. Indeed, configuration management systems are useful for dealing with a wide range of operational problems outside of the container platform. While it’s no surprise that Chef, Puppet, Salt and Ansible all moved quickly to include Docker support, I think we will see these tools begin to focus more on what Rob Hirschfeld calls achieving “ready state”. That is to say, everything that still needs to be done before you can lay down your container platform also still needs to be automated. Right now, that is still a great deal of work, and it’s a pretty safe place to draw the line on using the right tool for the right job.

    About the Author / James Thomason

    james-thomason_dsc0492James Thomason is chief technology officer at HyperGrid from Dell, where as CTO he led the vision, technology strategy and product road map for Dell Cloud Marketplace. He joined Dell through the acquisition of Gale Technologies in 2012, where as CTO he headed the product road map, architecture and engineering of the company’s flagship cloud and converged infrastructure automation software, GaleForce. Prior to joining Gale Technologies, Thomason was the CTO and Founder of Virtiv, a San Francisco cloud automation startup focused on Linux virtualization, acquired by Gale Technologies in 2011. For over 17 years, as a specialist in distributed systems and large-scale infrastructures, Thomason has been an entrepreneur and innovator at a number of notable Silicon Valley startups, including Exodus Communications, Digital Island, Netli, NetVMG, Netscaler, 3Leaf Systems and Ning. Connect with him on LinkedIn / Twitter.

  • VictorOps Announces DevOps Roadtrip to Impart Industry Best Practices Across the Country

    New event series provides practical tips for implementing and enhancing DevOps strategies

    BOULDER, Colo.–(BUSINESS WIRE)–VictorOps, the real-time, agile incident management company for DevOps teams, today announced its DevOps Roadtrip, a series of events designed to help software development and IT professionals across the country to better implement DevOps practices. The half-day events will focus on tactics and strategies to streamline processes specific to incident management and resolution.

    The first stop in Denver on March 29 will be held at Sports Authority Field at Mile High Stadium and will feature guest speakers from Forrester,Craftsy, and SendGrid. The half day event includes fast-paced TED style talks, topical breakout sessions, and a keynote presentation all aimed at helping others progress in their DevOps journeys.

    “DevOps can be an intimidating topic to many in IT. VictorOps wanted to create a welcoming forum where technology professionals can engage with DevOps experts in the industry,” said Jason Hand, DevOps Roadtrip MC and DevOps Evangelist, VictorOps. “Throughout the country, we’ll spotlight notable speakers in an effort to share DevOps wisdom amongst those tasked with the hard job of maintaining uptime. Our hope is that attendees can head back to their teams with actionable tactics to not only improve their organizations, but also simplify their own lives.”

    In addition to benefiting from a wide range of DevOps expertise and know-how, attendees will:

    • Hear real-life DevOps implementation stories from local companies
    • Learn about industry trends from renowned analysts
    • Network with others searching to implement real-time, agile incident management best practices
    • Have the opportunity to ask questions during an interactive Q&A session
    • And receive practical suggestions to take back to their teams

    By putting these teachings into practice, attendees can help their own companies become more efficient and ultimately more effective.

    “Organizations face an existential threat: deliver more value to customers faster than their competitors, or die. In the past, faster delivery usually meant lower quality and higher risk,” wrote Diego Lo Giudice and Kurt Bittner, Forrester Principal Analyst and DevOps Roadtrip Denver speaker in the December 2015 Forrester report entitled Agile and DevOps Adoption Drives Digital Business Success. “Leading organizations have shown how applying Agile and DevOps practices enables faster delivery with higher quality and lower risk.”

    Other stops on the 2016 DevOps Roadtrip include Seattle, San Francisco, Minneapolis, and New York.

    For more information about the DevOps Roadtrip, visit VictorOps.com/DevOps-Roadtrip, and to sign up for the Denver event, visitVictorOps.com/DevOps-Roadtrip-Denver.

    About VictorOps
    VictorOps is the real-time, agile incident management platform focusing on incident lifecycle management and collaboration for IT and DevOps teams. The solution combines the power of people and data to solve IT problems in real-time. The VictorOps platform seamlessly orchestrates team situational awareness, incident creation, escalation, notification, and remediation with team members regardless of physical location or time of day. Privately held, VictorOps is backed by Costanoa Venture Capital and Foundry Group. For more information, visit www.victorops.com and follow the company on Twitter @VictorOps.

    Contacts

    For VictorOps
    Mackenzie Kreitler, 978-518-4536
    victorops@matternow.com

  • Three Ways Dev and Ops Teams Can Manage the Integration Lifecycle

    Three Ways Dev and Ops Teams Can Manage the Integration Lifecycle

    Many enterprise organizations did not invest enough money into integration as they built their cloud or hybrid infrastructures. As if integrating on-premise systems and applications wasn’t tough enough, in today’s cloud era with the rise of hybrid environments, connectivity has only gotten more complicated – especially when combined with the increased speed of deployment. Puppet Labs found that high-performing organizations deploy code 30 times more frequently than their lower-performing counterparts, meaning dev and ops teams are struggling to break down data silos while simultaneously managing constant technology upgrades for line-of-business users.

    There’s no doubt that, whether public or private or hybrid, the cloud is now part of most companies’ infrastructures, and its share will only increase. Goldman Sachs estimates that spending on cloud computing infrastructure and platforms is expected to grow at a 30 percent CAGR from 2013 through 2018 compared with 5 percent growth for overall enterprise IT, while Forrester predicts that 25 percent of all software applications purchased will be cloud-based by 2020.

    With the speed and agility the cloud imposes on businesses, dev and ops need to come together to look at data integration throughout its entire lifecycle, from developing new apps or services on top of existing apps, all the way to the ongoing service, maintenance and upgrades of those applications. The two teams must join forces to manage this process as one comprehensive integration lifecycle, where dev and ops play equal roles solving for new data integration challenges brought about by hybrid deployments and siloes created by line of business users’ cloud purchasing habits.

    Today’s dev and ops teams have to look at data integration as a service, not just a one-off implementation they can hand off. Here are three strategies to help bring the two teams together to jointly manage the data integration lifecycle.

    1. Assess Current Integration Challenges Collaboratively
      Without alignment between dev and ops teams, integration lifecycle can’t be managed successfully. Organizations need to align on the people and process portion first, asking questions like: what systems does each business unit need data from? How likely is it that their needs will change, and how can you plan for a flexible integration as a service? What capabilities will ops need after dev implements the integration?

    After reaching agreement, teams will need to jointly assess and select a data integration as a service technology or platform that allows for joint requirement setting, collaboration, testing and ongoing adjustments throughout the lifecycle of the applications. Selecting the right integration platform as a service at the onset is critical to minimize cost, time and wasted dev and ops cycles down the line as business and technology needs change. The most agile integration option typically consists of a plug-and-play, flexible platform that makes use of APIs and also allows for deeper customization down the road.

     2.Understand the Four Key Stages of Integration Lifecycle Management
    Dev and ops teams are now part of the same cycle dealing with frequent updates and agile product development. It’s no longer about developing, testing and then having ops maintaining, but about a seamless and transparent handoff from dev, to ops and back throughout the entire application and data integration lifecycle.

    To capture the most value from disparate organizational data and to minimize ongoing data and application synchronization costs, dev and ops teams need to adopt a joint integration management strategy that accounts for the four distinct phases of data and application connectivity. These include:

     3. Design phase. After understanding business use cases and selecting a platform, dev and ops must collaborate on priorities – what near-term functionality can they design for based on the plug-and-play layout, and how can they tackle that low-hanging fruit while maintaining the flexibility to meet future requirements?

    By starting with the right technology and a common integration framework, dev and ops teams can design connected core systems and data early on with plug-and-play API connectivity for solutions like CRM and ERP, while maintaining the ability to easily expand functionality with custom/deeper integration capabilities with marketing automation, business intelligence and other apps.

    In addition to functional capabilities, the design phase should address performance and scalability by taking into account throughput and latency. This process involves planning to meet performance expectations, even with the customization and extensions users will likely want down the road. Dev and ops should ask themselves questions like: which company systems will be on premise, in the cloud or hybrid? What are the performance, scalability and security tradeoffs of each approach?

    1. Develop phase. Dev and ops need to work together in the development phase to carefully document key aspects of the integration while building connectivity for specific use cases. This documentation will be critical to the seamless handoff between the two departments, so they both need to be involved in the process. Strategic collaboration and thorough documentation can improve the efficacy of maintenance and debugging activities, help companies adjust to staffing changes and more easily transfer integrations to and from the two teams during the ongoing development and roll-out process.
    2. Deploy phase. Given that you’ll likely deploy in a hybrid environment, make sure you test both on-premise and cloud integrations to ensure that data is flowing freely. To help test integrations and move them into a production environment, look into staging and production environment tools that streamline the process. Infrastructure costs should also be considered, as they may increase unexpectedly if you don’t watch them throughout the deploy phase.
    3. Run phase. Continuously monitor to confirm that the data is flowing as designed. If dev and ops selected the right technology platform, the teams won’t have to worry about ongoing API management, because the platform takes care of it.

    The right platform will also lower downtime and maintenance costs with a multi-tenant management environment that allows the expert for any given situation to dive into the integration backend or runtime. The platform should also provide ongoing diagnostic reports and proactive alerting to allow ops to home in on potential performance issues.

    1. Build a Responsive IT model
      With dev and ops coming together to take advantage of the speed and agility offered by the cloud, the resulting just-in-time services for businesses are turning IT from a cost center into an innovation center. The data integration lifecycle is another result of the new cloud-shaped world, helping dev and ops teams create a more responsive IT model and allowing them to better meet business needs while driving innovation. While shared environments bring with them an additional element of risk, a responsive IT model is an important step towards both short-term and long-term success in the modern connected enterprise.

    These three steps allow dev and ops teams to offer the value, speed, efficiency and opportunity that today’s enterprises require. By adding collaboration and agility at every stage of the integration process to plan, implement, track, debug and update continually, dev and ops teams can scale to respond to the data-driven needs of the business more precisely.

    How have you overcome cloud and hybrid application and data connectivity challenges in your organization? Connect with me in the comments below, or on Twitter at @scribesoft.

    About the Author/John Joseph

    John-Joseph head shotJohn Joseph, VP of marketing for Scribe Software, earned a master’s degree in business administration from the MIT Sloan School of Management, a master’s degree in electrical engineering from the University of Southern California and a bachelor’s degree in electrical engineering from MIT.

  • Tips to change culture (at least in your team).

    Tips to change culture (at least in your team).

    DevOps is all about automation, enhancing the Application Lifecycle Management and culture change. In the last few years there have been tons of conferences and articles about the benefits of a cultural change on a project, but I have yet to read a paper about how to actually change culture.

    How do you manage to get people to change the way they think and act towards each other and professionally? This is a huge challenge that most DevOps leads will have to overcome. Here are some of my ideas.

    Create cohesion.

    The first thing to keep in mind when trying to implement DevOps (or on any other kind of project for that matter) is that people who don’t have fun at work will not perform as good as they could. How to make people happy to go to work is up to you, but I find that simple activities such as team exercises and/or after-work beer work best.

    Whenever a new member comes to a team, try to get everyone together to have fun and talk. Take them out to a game of laser tag, get some sumo suits and make people try wrestling amongst other sports. Try to identify a common interest for a sport and have everyone play together. Remember, this is not a competition, do not force people to participate if they don’t feel like it.  There are neither losers nor winners, so try to balance the teams as well as you can.

    Another way to get people to know each other and ease potential tension is to take people out. Take your team out to discover a new restaurant or pub and discuss the place together, what they would have changed in the decoration, which Belgium beer is missing from the menu, why on earth do they serve foie gras on a scallop… The point once again is to try to discover others’ interests and behavior outside of work. This specific activity however needs some rules; try to limit the alcohol intake and by all means – do not talk about religions or politics. While those two topics can be fascinating with your friends, some people might get offended, even if they say they don’t mind.

    Back on the project, the use of Scrum is highly recommended. Even though the benefits are debatable, I find that shy people tend to open up and contribute more during those sessions than they normally would. On top of that, Scrum meetings include everyone on the ladder, from bottom to top. Only one rule here: no matter what, remember to actually do your daily stand-up, if you can’t do it personally, get one of you team members to conduct it and check afterwards that is has been done. Delegating is not a sign of weakness; it just shows trust in your team.

    Create incentive.

    People make mistakes, always. There are, however, ways to get to get people to make fewer of them. Pointing fingers and blaming people is definitely not a solution. Making people pay for their mistakes is more efficient. Say each team member has to pay 50 cents (adapt to your own country and project) for every mistake (new bugs, fail in commit, forgot to write a comment in SVN/Git at commit…) they make, and things might start to change. The point here is not to rob your employees, so you will have to find a creative way to use that money. You could for example gather that money at the end of the month to buy candies for everyone. But then that’s just punishing people, which does not make it fun, so you have to come up with a way to compensate for that too. If the management doubles every penny in game, then everyone is involved and there starts to be some interest. But then again, you might get extra bugs because everyone loves candies, so you could triple the sum of gathered money when there are fewer problems than the previous month.

    At that point of reading, you might think it’s actually costly and doesn’t benefit anyone except dentists. Well you’re wrong. In the worst case scenario we’re talking about tens of dollars a month, and given that you will actually get some accurate metrics about this, this is a no-brainer.

    If candies are not popular in your project (is that possible?), you can also get the good old “employee of the month” award to whomever created the least problems and reward him/her with cinema/theater tickets, a dinner for two… Please note that on high stress periods, dinners for two, even though a lot more expensive than regular rewards, are a lot more appreciated as they might represent a good opportunity to ease work related tension in a couple.

    Develop people.

    Don’t see your colleagues, especially the young ones as just resources, rather think of them as potential wonders. Everyone aspires to shine and it’s all up to you to make it happen. Do not ever take someone else’s ideas and get the glory. That would be idiotic, mean and potentially a good way to get your colleagues to quit or resent you.  Instead, go on, try to encourage people to speak up, make them give technical presentations, make them speak and give them a positive feedback (even on failures, there are always good ways to explain how one can improve).

    Once again, Scrum can help you with that. Identify the elements in your team who need some help and pair them with some others that are more comfortable for some small sessions of peer-programming, peer-presenting, peer-testing… once a week. It can cost some time in the beginning but the benefits are potentially huge in a long term perspective. There is nothing better than a team that has learned to work together and where the first reflex when someone ask for help is to actually help, not to ignore.

    Demonstrate values.

    One of the things that really is important to remember, is where you come from. No matter your position today, you didn’t start at the top and you’ve had to learn all the values that you try to get your team to demonstrate today. So show them every day, what it means to create DevOps; a potentially highly stressful job, especially in the beginning, where tension between people should not exist. Smile as much as you can, take time to help others even if you that means staying later at night, be proactive once again, try to identify potential bombs and defuse them before they are seen by others. All in all, be nice to others, treat them as you would like them to treat you.

    But what if you are faced with a pigheaded team member? Someone who is convinced they are right, no matter how many people disagree? Well once again, do not ignore them; that will only create frustration. Instead, get some documentation or an expert and talk openly about the topic without being aggressive. In the end, the team member should be respectful of your efforts and have won some valuable knowledge in the process.

    The biggest piece of advice I have is this: Do not ever be arrogant, do not ever be pompous. Help others, always, and do it nicely and in a humble way. The leaders we like are not the ones that talk too much but the ones that are the nicest to work with.

    Changing a culture on a project is a tough thing, but it can be done with a few tricks and a lot of time spent helping others. Luckily, smiles and good atmosphere have a tendency to spread fast and the tight links between Dev and Ops can be strengthened quite fast with some goodwill.

  • DevOps and The Goal

    DevOps and The Goal

    Reading, “The Goal” was a pivotal event in my career.  More than a decade ago a friend of mine who was an operations professor at Georgia Tech told me it was his favorite book.  We were discussing the challenges of getting work done in a corporate environment.  I was grumbling about leading a web dev team in a company that did not embrace the software development discipline.  Already a fan of Tom DeMarco and Timothy Lister’s, “Peopleware”, and being very vocal about the lack of ability to focus whether due to environmental distractions or the constantly changing priorities and schedules, I was receptive to the message in “The Goal”.  It gave me a vocabulary and a roadmap for improving our throughput issues.

    Our throughput problems were a textbook case for development disaster.  When I joined the project, we had three source code repositories, from none of which could we build production and when we did produce a build it would work in pre-production environments but not production (this just slightly predates the widespread acceptance of continuous integration).  We also had the classic inconsistent release cycles that would lengthen based upon insistence by the business owners to include more features.  Looking back, it is not surprising in the least that we were not successful.

    Not surprisingly, these issues led to a lack of trust with the business.  Overcoming the technical problems was straightforward using brute-force tactics.  We addressed the source code problem by recruiting a senior developer and empowering him to make improvements.  We overcame the build quality issue by using a primitive form of agile and enforcing knowledge transfer; the operations engineer sat between two developers during the build and release process and he was the only one allowed touch the keyboard.  Our tactics were basic, and they worked.

    The bigger challenge facing us was getting our business owners to accept a different SDLC model.  We were saddled with a classic waterfall process, each phase being dependent upon the last with a lot of idle time built-in at each phase gate.  The shortcomings of the waterfall process were amplified because the business lacked confidence about when, if ever, they would get a feature released to production. This lack of confidence led to a constant rearranging of the priorities to get the most important feature du jour in the next release.  The constantly changing priorities created even more idle-time and a lot of work-in-progress (WIP) which Goldratt’s Theory of Constraints (TOC) tells us is at the root of inefficiency.

    Using TOC as a guide, we redefined our SDLC from requirements through QA.  We focused on developing a process that de-coupled the phases, reduced idle-time and work-in-progress (WIP).  De-coupling the phases meant the work could queue between each phase allowing the teams, (requirements, development, QA, and operations) to vary independently; they didn’t have to wait on the team before or after to complete their work which reduced idle-time.  We then focused on improving the slowest areas, which seemed to be requirements and deployments.

    We convinced our business owners to trial a monthly release cycle (this was a long time ago, even before continuous integration was widely accepted!).  The monthly release cycle helped give the business confidence that they would see their features in production at an expected pace and it allowed requirements to cycle on schedule independent of development and QA.  Some features would get done in a month and some would span several months.  Over a period of three months we got into a rhythm that worked for about 14 months until organizational focus changed and we focused elsewhere.

    Reading “The Phoenix Project” reminded me about those days and the challenges Goldratt helped us overcome.  I really liked “The Phoenix Project” and many of the situations rang true.  I have since moved from development to operations (twice!) which gives me a unique perspective as I see both sides.  I see DevOps as the embodiment of Goldratt’s ideas in IT; reducing work-in-progress, reducing cycle-time and always focusing on the most constrained processes.  Add a consistent release schedule and it looks a lot like DevOps.  Everyone has their own ideas about what is DevOps and how best to implement, I suggest we follow Goldratt’s advice and focus our efforts where they will have the most benefit.

  • Step One Of DevOps Ignores Technology

    Step One Of DevOps Ignores Technology

    I’ve spent a bunch of time talking with Adrian Cockcroft, former cloud tzar at Neflix, and listening to his various talks and presentations. I’ve also spent a bunch of time looking around the various rooms within which he presents and seen the dazed look of attendees who have rightly assumed that just seen a picture of the way organizations should be. Netflix is unquestionably an exemplar for IT reinvented – where innovation, agility and developer enablement are natural parts of what they do.

    But then I’ve seen these new disciples of the “Netflix way” write a list of the technologies that Cockcroft detailed and attempt to shoehorn those same approaches into their own organization. All to often the result is disappointment, failed projects and a swing back to the traditional rigid ways of doing things.

    These thoughts struck me then I hear about this new masthead and its aim of creating a hub for DevLops thought leaders – surely giving IT practitioners a good place to find the latest tools and techniques will help spread the DevOps gospel?

    Well yes, and no. You see I’m a firm believer in the viewpoint that DevOps has very little to do with products, processes and policies, and instead is a direct result of a particular type of culture. On the one hand you have Netflix, clearly a cultural outlier with its openness, willingness to experiment, embracing of failure-driven iteration and aim of planning for failure. On the other hand you have traditional IT shops that are top-down, driven by a rigid policy framework and consider the “operations department” and the “developer team” as two distinct entities.

    This was a topic that Christian Reilly (he of the sartorial elegance and funny accent tendencies) and I discussed a couple of years ago, before the DevOps hype had really started to hit. In the video, he reflected directly on his organization, Bechtel, as an outlier at one end of the spectrum, while Netflix is an outlier at the other. It’s a somewhat dated video, and talks about public/private cloud, but the themes in terms of organizational readiness hold true for DevOps as well.

    [youtube http://www.youtube.com/watch?v=gVG1Pj4cW-g&hd=1]

    So my aim in my regularly irregular DevOps column in here is to talk about everything but the technology. And where better to start that the first step towards DevOps which is, I believe, all about building cultural bridges rather than making technology choices. It should be a natural concluding that, since DevOps is all about bridging the developer/operations span, that perhaps getting people in a room to talk about their particular drivers and metrics would be a good idea. But alas most people go from talking about DevOps to deploying their choice of automation tool.

    There’s so many things to think about in embarking on this process – over on the ScriptRock blog Alan points out that visibility is a key first step before automation is embarked upon. While I agree with him that visibility is key, for me the money quote comes in his post where he writes:

    Don’t automate what you don’t understand

    But to then suggest that the first step should instead be a visibility tool does little to improve the outcomes – unless both sides fo the organizational divide are in the same room and accepting a common lingua franca and viewpoint on the data – visibility achieves little. So takeaway #1 focus on the culture. There’s lots of ways to build the bridges needed to change the way an organization looks – I’m going to cover some of that stuff over time.