Tag: continuous delivery

  • How Continuous Delivery Challenges Security

    How Continuous Delivery Challenges Security

    Security and continuous delivery each have implications in their effects on each other. While continuous delivery can demand new security measures, traditional security can slow continuous delivery or simply not address its security needs at all.

    Micro Case Study Clarifies Security Conflict

    Amrit Williams, now CTO of CloudPassage, shares an experience that exemplifies the challenges continuous delivery poses for security. Williams once worked with one of the largest national HMOs in the United States, which developed and supported a customer healthcare portal to enable customers to access health data and take advantage of other HMO services.

    At the time of his service there, the company sought to enhance its portal’s performance to challenge other HMOs in the marketplace that were popular, in part, due to the strength of their portal.

    “They wanted to add a bunch of new features and make continual improvements by way of moving from a development model of new releases every six-month period to new site releases and updates every couple of hours or about four times per day,” says Williams. But every time the Ops team would bring up new containers and compute resources to deploy new technologies and application code, success would count on repeated manual changes to firewall rules to appropriately pass data, enabling applications to talk to the portal and its technologies.

    “They finally decided to allow the firewall rules to apply to the application at the server level rather than at the underlying network level so that they didn’t have to go changing those rules all the time,” says Williams.

    In DevOps, Security Still in First Gear

    “Traditionally in the enterprise, security has been about speed only when the company is potentially dealing with a compromise, a policy violation or some other threat to the business,” says Williams. The organization is not used to having to respond quickly with security simply to deliver code. Common security tools were not designed to shift and adapt to automated code development and deployment.

    “For example, a lot of times the security teams will rely on endpoint security tools that typically require static IP addressing, which is antithetical to dynamic compute models,” explains Williams. These mammoth resource-hogging security applications ask too much and secure too little if they can even work effectively in DevOps shops at all. When you must stop to adjust these security tools, you may trigger change management processes that can slow the development process even further.

    Why Not Fail Fast?

    While DevOps makes it easier to isolate and fix coding errors quickly, this doesn’t appear to spill over to security. Fixing errors to make the code work doesn’t mean the code is more secure. These changes could even introduce new security flaws that were not there before.

    Securing continuous delivery is about the security team surrounding dynamic processes with protections that keep up with the speed of the continuous delivery pipeline. This can be difficult when these processes are automated, bringing up resources on demand to create and deploy software.

    “It is really the deployment and configuration of these environments that can lead to some of the biggest security concerns and not necessarily the code itself,” he says. “It is that aspect of DevOps that security is usually not very familiar with, which can really result in them being left out of the process because they try to apply a rule or a government policy that is again, antithetical to that process.”

    Continuous Delivery Security Incidents Challenge Legacy Security

    In this case example, Williams shares a particular type of threat that legacy security tools would not prevent in a continuous delivery environment.

    “It was 2012, during Christmas and it happened to customers using a region of Amazon‘s public cloud services. Amazon rebooted a bunch of servers, and some of those servers were running local host-based firewall management tools that were reliant on connectivity back to the management consoles as well as a static IP address,” says Williams.

    When those systems came back online, neither the agents on the hosts or the tools functioned properly. Amazon and its customers had to work to ensure the agent software on the hosts worked and communicated with the firewall management tools and that customer systems were secure. This was important because of the number of sales Amazon customers had to process securely at the time, which was just a few days before Christmas Eve.

    Again, this is an example of the failure of legacy security tools inside an environment that was increasingly adding automation while approaching continuous delivery. The lesson here is that as DevOps and continuous delivery approaches are altogether unique from legacy development, so the security tools that protect these environments need to be new, unique and suited to continuous delivery by design, lest they fail to survive partial outages such the Amazon example.

  • Experts Weigh In on the Evolution of DevOps

    Experts Weigh In on the Evolution of DevOps

    DevOps is an ethereal concept. It means different things to different organizations and individuals. It also is going through an evolution, which means that as quickly as we might think we have a grasp on what it is and how to effectively employ it, DevOps has moved on. There’s no way to accurately predict whether DevOps will even still be a thing in another five years, or if it will have faded into oblivion—replaced by the next hot tech or development model. We can, however, take a look at how DevOps has changed in the past year, and what the immediate future holds for it.

    It isn’t an easy question to answer because first we have to agree on what DevOps is. Are we talking about DevOps as a culture or business model, or are we referring to the evolution of the tools and applications generally used in a DevOps environment? I reached out to the DevOps community to get some expert insight.

    Current State of DevOps

    “The technology question is a common one when it comes to DevOps. We are still faced with the problem of a clear definition of DevOps. So what makes this difficult is … I’m squarely in the camp that believes DevOps is more about culture than tools or tech,” explains Andrew Storms, vice president of Security Services at New Context. “That being said, a well-rounded and mature DevOps organization does require a shift in people, processes and technology. A lot of people will likely point to orchestration of infrastructure as the most important DevOps tool. However, I think the bigger driver of orchestration has been virtualization and cloud.”

    One of the defining characteristics of DevOps from my perspective—aside from the IT culture aspect—is the fact that we continue to raise the bar for automation. Each new automation tool further streamlines the process and puts a spotlight on the next bottleneck. Eventually, a new solution is developed to automate that as well.

    Chef CEO Barry Crist believes that one of the most important evolutions in the world of DevOps in the past year is continuous delivery (CD)—or the process of rapidly developing software and iterating through quick, small-batch releases. “Continuous delivery was previously exclusive to the web innovators like Facebook, Amazon and Google, but over the last year companies have realized it’s a primary key to unleash an organization’s innovation engine. When the whole stack—infrastructure, compliance, security, microservices and apps—are all expressed as code, IT is able to deliver value more quickly.”

    DevOps also has introduced more common ground between developers and security/compliance officers. In high-velocity organizations that have embraced DevOps, IT can deal with security threats more quickly by assessing a vulnerability, creating a fix, testing it and deploying it rapidly before damage is done. DevOps is also starting to automate and define compliance. When compliance is defined as code, the software can be managed and tested like any other piece of code in the organization.

    The Evolution of DevOps

    Storms stresses that the immediate future of DevOps revolves around it becoming more mainstream—the new normal. “We are already seeing the business value of DevOps. Companies are getting code out the door faster and at a more efficient means. Other departments, like those of security, are starting to see the value of DevOps and looking at ways to incorporate key points into their departments,” he says.

    Crist sees DevOps as evolving beyond IT itself. “In the next few years, the DevOps tent will expand beyond ‘devs’ and ‘ops’ to include executives and other business units,” he says. “DevOps is really about building high-velocity organizations. Mature DevOps organizations see both innovation and quality increases that drive more profit and shareholder value, as well as advances in security and safety.”

    Only time will tell how DevOps will actually adapt and evolve. I tend to agree with Chef’s Crist that the natural evolution is to take the core principles and practices of DevOps and expand them beyond development and beyond IT to streamline all aspects of the organization. I guess the real question is whether or not we’ll still refer to it as “DevOps” at that point.

  • 5 Things DevOps is Not

    5 Things DevOps is Not

    For those who have been embracing and championing DevOps for years, the growing popularity of DevOps is as exciting as landing your first job out of college! Okay, maybe not that exciting, but still really cool. More and more companies of all sizes are embracing DevOps by including the adoption of DevOps practices directly into their fundamental strategies.

    On one side, this is giving teams the leverage they have needed all along to either begin their DevOps journey or to increase the maturity of existing DevOps adoptions. On the other side, when DevOps becomes a mandate, some of that grassroots passion can become watered down, as the use of the word becomes increasingly overused or incorrectly used.

    DevOps incorporates a new collaborative culture that embraces numerous practices combined together for a continuous software development methodology that places significant emphasis on feedback loops, collaboration and continuous improvement. DevOps requires fundamental cultural changes and includes so many practices that can be overwhelming to those just starting their journey.

    Instead of highlighting everything that DevOps is, I thought it would be good to take a step back and highlight what DevOps is not instead.

    1. DevOps is not simply combining Development & Operations teams

    We’ve all seen this one when the term DevOps comes up. “Let’s combine the Development & Operations team and, voilà!, we now have DevOps.” DevOps combines a set of processes and practices to be adopted throughout the entire delivery pipeline and spans multiple stakeholders. A couple of the key practices within DevOps adoptions include continuous integration (CI) and continuous delivery (CD). Simply combining two teams and calling it DevOps cannot accomplish those practices.

    2. DevOps is not a separate team

    Setting up a separate DevOps team is another trap many organizations fall into when beginning their DevOps journey. In general, I’m not a fan of the DevOps team as I believe it leads to more silos in the end. I’ve also found the creation of these teams can lead to further confusion when the mission is not clearly defined.

    There are some cases where a temporary DevOps team may make sense to help enable the processes and potential tooling that may be needed for adoption, but the key is making it temporary—which is often better in theory than in practice. There are several great blogs out there discussing DevOps teams such as Matthew Skelton’s blog, “What Team Structure is Right for DevOps to Flourish?”

    3. DevOps is not a tool

    Let me be the first to say I actually love the growing number of tools that enable us to continue maturing our DevOps adoptions, but I’ve noticed the use of one or two tools starts to lead to the perception that DevOps is a tool. How many times have you heard?

    “We’re already doing DevOps. We have Chef.”

    “We do DevOps. We automatically deploy through Jenkins.”

    Keep in mind, I’m a big fan of both Chef and Jenkins, but I think the power of DevOps is drastically underutilized if you’re equating the utilization of a single automation tool to DevOps success. Adopting automation and tooling is, of course, a part of DevOps, but only when combined with end-to-end practices of increased collaboration with continuous integration/continuous delivery, amplified feedback loops and continuous improvement (to name a few)! DevOps is a journey and that journey may start with a tool, but the goal should be to first identify your DevOps strategy and then find a tool or tool chain that meets those goals.

    4. DevOps is not a one-size-fits-all strategy

    Because there are so many different business drivers and technologies to consider when setting up an overall DevOps adoption strategy as well as identifying your DevOps tool chain, it’s important to apply the same tenets of DevOps to your DevOps strategy. Embrace change, gather metrics, understand feedback, fail fast and correct your course quickly. As an example, if you initially identify a tool that no longer works for your technology or environment, abandon it and move on. Just because you used it on the last project doesn’t make it a silver-bullet fix for the next one. We need to first understand our current strategy and environment, then react accordingly.

    5. DevOps is not automation

    That one may have caught the eye of a few people, so I’ll clarify: DevOps is not solely automation. Automation is absolutely a huge part of DevOps; however, it is not the only part, and deploying some degree of automation is often used interchangeably with DevOps. I think understanding the key DevOps practices is a great precursor to ensuring DevOps is viewed as more than just automation. Understanding the core DevOps principles is key to understanding the true benefits of DevOps adoption.

    There are a lot of things that DevOps is and I think I share a widely spread passion for DevOps. There are also a few things that DevOps isn’t—or isn’t solely. If you’re just starting your DevOps journey or still maturing your existing model, making sure everyone on your team has some basic DevOps education and is aware of the DevOps strategy for your project is key. It will go a long way in helping everyone understand that DevOps is about a lot of things. It’s also not about a lot of things.

    The list above isn’t exhaustive and I’m sure you all have a few you’d like to see on the list. I’d love to hear you thoughts and suggestions at @Shelbee_SE.

    You also can download and browse through the DevOps for dummies e-book.

    About the author/Shelbee Smith-Eigenbrode

    Shelbee

    Shelbee Smith-Eigenbrode is a Senior Software Engineer/IT Architect at IBM Cloud Infrastructure Center of Competency. She has worked across varying roles spanning the software development life cycle. Her experience across functional roles as well as performing that work deeply inside traditional silos has contributed to her passion for embracing the DevOps culture and transforming the way teams are delivering technologies by adopting DevOps practices. She is a member of the IBM Academy of Technology committed to driving innovation and technical advancement.

  • In DevOps, Standardize to Minimize Architectural Debt

    In DevOps, Standardize to Minimize Architectural Debt

    I recently reading a great Q&A with the Etsy CTO John Allspaw over on The New Stack, which contained some nuggets of good advice that increasingly are important as we continue the journey toward DevOps adoption. Of particular note were statements on the introduction of “new” things to solve problems. In a nutshell, Allspaw reinforced the importance of standardization as a means to reduce architectural debt.

    If you aren’t familiar with the concept of architectural debt, it’s similar to technical debt, in which choices made today incur longer term investments of time and money when future changes are introduced. If you settle on a caching solution such as Redis, for example, there are technical and architectural implications. You can’t simply switch out Redis for memcached later without significant changes to both code and the infrastructure.

    Even your choice of load balancer today implies architectural debt because of the growing inclusion of scalability as part of the application infrastructure itself. The result is integration with automation and orchestration systems that are not easily changed later on if you decide you want a different solution. The style of service registry you choose is an architectural decision that impacts clients, servers and systems across the microservices architecture. Switching from client-side to server-side will later be costly.

    That’s architectural debt.

    Allspaw noted the need to not reinvent the wheel when possible, and instead encourage the collaborative aspects of DevOps as a means to avoid reinventing the wheel and potentially increasing the architectural debt load on the organization.

    This is not a new concept. There’s a reason enterprises in particular proudly proclaim themselves “Java shops” or “Microsoft shops” and primarily require development in one or perhaps two languages, but no more. Maintainability is a component of technical debt, just as middleware decisions are a component of architectural debt.frameworks in use soad2016

    So as we continue to see wider DevOps adoption, it’s important that we carefully consider the impact of our choices in terms of the automation, orchestration and integration required to implement continuous delivery and deployment. Once you’ve invested time and energy (and probably budget, too) in Puppet or Chef, VMware or Cisco, OpenStack or SaltStack, you’re going to find an unavoidable amount of technical and architectural debt in development as well as production across infrastructure and networking teams.

    Each of these systems, solutions and frameworks comes with its own components. It isn’t just a language, it’s a comprehensive system with management and repositories and agents and reporting components. All of these impose certain architectural requirements on organizations, from needing certain software running to prescriptive details on networking that must be enforced, lest the system break down.

    We’re already starting to see that fragmentation in enterprises that are engaged in automation and orchestration efforts. When asked the number of automation frameworks (Cisco, VMware, Python, OpenStack, Chef, Puppet, etc…) in use, one-third today reported using three or more.

    That’s not necessarily “bad,” as it could be indicative of standardization across silos (networking wants to use Cisco, infrastructure is already on VMware, and Python is being used to cobble it together by ops), but it also could be an early sign of the kind of fragmentation that causes exponential growth in technical and architectural debt.

    Certainly there is room for experimentation in these early days, but it behooves us to maintain a watchful eye on the use of frameworks and systems and languages we employ for automation and orchestration to ensure we don’t end up buried in technical and architectural debt.

    Standardization, even loosely enforced, can help minimize the impact of that debt before it spirals out of control.

  • The Difference Between DevOps and Anarchy

    The Difference Between DevOps and Anarchy

    DevOps is about being more efficient. It’s about breaking down silos and removing traditional business obstacles that get in the way of just getting things done. It’s about enabling teams and individuals to collaborate more seamlessly.

    None of that, however, means that DevOps is about turning your IT into some sort of “Wild West” anarchy, where anyone and everyone can just push code to production. That would be insanity.

    One of the primary catalysts that started the DevOps movement was the frustration of developers trying to deal with an operational bottleneck. As developers evolved from the old-school Waterfall development method to the much faster-paced Agile, developers were able create better software more quickly. That simply resulted in a “hurry up and wait” scenario, though, because the Operations and IT infrastructure side of the house couldn’t keep up.

    Fast forward a bit and DevOps was born. Since its inception, however, there has been confusion about what exactly DevOps is or isn’t. That might be a function—at least in part—of the fact that there was no “inception” of DevOps; nobody developed a thing and then rolled it out to the world with the announcement, “This is DevOps.” DevOps was an evolution that occurred organically and out of necessity. It meant different things to different organizations and was implemented in ways that were largely subjective and often not called anything in particular. Organizations were simply struggling to address challenges and coming up with solutions that worked.

    Suppressing the Anarchy

    Although traditional IT operations were the roadblock that sparked the DevOps movement, DevOps doesn’t mean developers get to simply do their own thing. I’ve worked in the trenches as an IT admin at a dotcom startup—I’ve experienced firsthand the disastrous fallout of a developer pushing untested code onto a production server, and I’ve witnessed the wrath of executive leadership when that happens and the proverbial “stuff” hits the fan. It’s not pretty.

    No, DevOps does not mean eliminating Operations from the equation and functioning as an IT anarchy. You’ll note that DevOps is a very obvious portmanteau of developers and operations—implying that operations is half of the DevOps solution. DevOps is about developers working more closely and cooperatively with Operations—not developers bypassing IT Operations.

    There are two components to successful DevOps: culture and continuousness. First, organizations need to shed the traditional mindset of separate siloed teams and departments and let individuals focus on working together in whatever way makes the most sense for efficiency and effectiveness. Empowering individuals to do what’s necessary to collaborate and get things done is a big part of DevOps.

    The other component is continuousness. When developers adopted Agile and started producing code faster, they discovered that Operations was a bottleneck. When developers and Operations started working together more seamlessly, new bottlenecks were identified, and solutions were developed to automate as much of those tasks and processes as possible.

    The ultimate goal of DevOps is continuous delivery—to continuously deliver new solutions and features. Rather than a lawless void where developers just push code out onto production servers willy-nilly, organizations adopt continuous development, continuous testing, continuous deployment, continuous monitoring and other tools that enable their teams and those developing solutions to do so with as little friction as possible.

  • Continuous is the Engine that Drives DevOps

    Continuous is the Engine that Drives DevOps

    It has been stated time and time (and time and time) again that DevOps is primarily a matter of shifting IT culture. The tools and practices that comprise DevOps play a secondary role. But that is only partially true: In reality, the culture shift revolves around continuity, and the applications and tools that fuel continuous delivery drive the DevOps engine.

    I alluded to this point in a post in early 2015. I stated, “Everything is continuous. Continuous development and continuous testing lead to continuous deployment and continuous delivery, which requires continuous support. Continuous monitoring produces continuous integration and continuous change. Continuous security results in continuous incident response … or vice versa. To top it off, all of the continuous activities continuously feed each other to drive more continuousness in some sort of DevOps Mobius strip of continuity.”

    That post last year was a bit tongue in cheek. I have since refined my position, however, and come to the conclusion that the hint of sarcasm was perhaps unwarranted. That continuous continuousness is the crux of the culture shift. It is the thing that separates DevOps from non-DevOps.

    On the Continuous Journey

    Speaking on the topic of continuous security, Andrew Storms, vice president of Security Services at New Context, explains, “Smaller incremental changes are always easier to handle than a large behemoth of a release that can include hundreds of changes. Try finding the single change in a monthly release that resulted in an unexpected security hole in your database configuration vs the release with only two changes that was just developed and released in the last few hours. Security practitioners cannot continue to rely on manual, human-controlled processes to defend against attacks. Automation is a requirement for the next evolutionary phase of security protection.”

    That sentiment extends beyond security to all of IT. When we talk about the culture shift in IT required for DevOps, we are generally talking about breaking down silos and removing the barriers between different teams. The goal of the culture shift is to shed tedious bureaucracy and red tape and empower employees to freely collaborate and cooperate to get things done. In other words, we want to eliminate barriers and roadblocks so productivity can be more continuous.

    Assuming the DevOps transformation begins at that level, it creates an environment of continuous collaboration which becomes continuous development. What follows is a natural evolution of continuity. Continuous development becomes continuous delivery, which requires continuous testing and continuous deployments, followed by continuous monitoring, which provides the metrics and feedback that take you back to the beginning of the loop.

    All of the “continuous” tools and solutions may seem to be part of the technologies and practices that come after the DevOps culture shift, and to some extent, that is true. However, they are also indispensable and inseparable from that DevOps culture shift because without all of the various gears meshing together smoothly—and continuously—what you’re left with is a more traditional approach to IT where everything comes to a halt.

    Is DevOps more of a cultural shift than a collection of tools? Absolutely. That culture shift revolves around continuity, though. Continuousness is the culture shift that drives DevOps.

  • Automic empowers cloud Continuous Delivery with OpenStack Integration

    SEATTLE, Wash. & VIENNA–(Business Wire)–Automic, the leader in Business Automation today announced integration with OpenStack, one of the leading open-source software platforms for cloud computing. The new OpenStack plug-in integrates with Automic Release Automation, enabling users to streamline deployment and provisioning of OpenStack computing resources alongside application components and new versions, resulting in the Continuous Delivery of applications to the cloud. The OpenStack action pack is available immediately from the Automic Marketplace.

    The Automic OpenStack action pack uniquely supports the efforts of DevOps engineers, developers, and system administrators to help them create, deploy, and run applications by automatically provisioning or de-provisioning OpenStack environments in the cloud.

    The Automic OpenStack action pack enables users to:

    • Automate the server compute pool in an OpenStack cloud environment. Activities include creating, starting, stopping, deleting and configuring compute resources. Some of the typical use cases include setting up and destroying environments on demand and per schedule (for test environments for example).
    • Dynamically scaling up and down an existing compute environment using automation.
    • Manage configuration changes in an existing environment.

    “We are delighted to add OpenStack to the list of integrations that we currently offer our customers on the Automic Marketplace” said John Purrier, Chief Technology Officer of Automic, and a founder of OpenStack. “As organizations drive adoption of both Cloud infrastructures and DevOps practices, Application Release Automation supports the need to orchestrate and automate both sides of the fence while providing a completeContinuous Delivery mechanism for DevOps teams. Adding OpenStack to our existing list of integrations is a natural progression of our offering, driven by the demands of our customers.”

    The DevOps movement is a natural extension of the increase in the enterprise open source and hybrid Cloud adoption. While a big chunk of the DevOps movement is spearheaded by development, operations are usually in the driver’s seat for Cloud adoption at enterprises. DevOps is the critical enabler of the Continuous Delivery pipeline, one that seamlessly connects developers and operations.

    A hybrid Cloud strategy is the predominant choice of most enterprise IT departments, while OpenStack is leading in the open source software space for creating private and public Clouds. With hundreds of the world’s leading organizations relying on OpenStack and a projected CAGR of 31%, OpenStack is core to many organizations’ hybrid Cloud strategy.

    Tweet this:

    #DevOps deployment in the cloud. @Automic announces integration with @OpenStack. Available now at https://marketplace.automic.com

    Additional information

    Connect with Automic on Twitter and LinkedIn

    Availability and pricing:

    The plugin is available from the Automic Marketplace. The OpenStack plug-in is free for Automic Release Automation Customers. Pricing is on demand for non-Automic customers.

    About Automic

    Automic, a leader in business automation, helps enterprises drive competitive advantage by automating their IT and business systems – from on-premise to the Cloud, Big Data and the Internet of Things. With offices worldwide, Automic powers over 2,600 customers including Bosch, Netflix, eBay, AMC Theatres, Carphone Warehouse, ExxonMobil, BT Global Services, Société Générale, NHS SBS, General Electric and Swisscom. More information can be found at www.automic.com.

    Automic
    Courtney Glymph
    Courtney.glymph@automic.com
    +447584058886

  • DevOpsQA NJ Meetup:  Pipeline Patterns with ThoughtWorks

    DevOpsQA NJ Meetup: Pipeline Patterns with ThoughtWorks

    DevOpsQA NJ held our January Meetup on Pipeline Patterns, with Max Griffiths (@_maxamg), Max Lincoln (@devopsy) and Gary Stafford (@garystafford) from ThoughtWorks  as speakers. The meetup took place at the sensational ThoughtWorks NYC location, with welcoming, open meet-and-greet area and a huge presentation area with bright, distinctive wall paintings.

    Following speed networking with pizza and beer, the discussion kicked off with a brief intro from the speakers, who each has been in the DevOps space for 10+ years. They gave a high-level overview of ThoughtWorks and some FullSizeRender (1)book recommendations including, “Continuous Delivery” by Jez Humble, “Release It!” by Michael Nygard and “The Phoenix Project” by Gene Kim.

    So what is the issue that we’re trying to solve? We are trying to gain an ability to make decisions early on to impact how our pipeline is going to scale, with the focus on culture, feedback and governance.

    Max Lincoln gave a good definition of a “good” pipeline. It should have incremental QA, fast feedback, visibility of the progress and a consistent, auditable software delivery process.  It’s a good idea to be very selective with how you name things in the pipeline. “The purpose of the pipeline is to kill the build as soon as possible,” he said. As the pipeline gets longer, quick feedback becomes more difficult.

    Next, each speaker addressed how the pipeline can scale in three difference directions: X (more stages between commit and productions); Y (more projects/applications); and Z (concurrent releases). Max Griffiths gave overview of X-Axis patterns which includIMG_6635e Linear, Parallel and Layered (with artifact promotion and release channels). Linear is the simplest pattern, which works for simple applications. With a parallel pattern, you start to parallelize where possible; however, that might create dependencies and make it difficult to reorder. The layered pattern encourages parallelizing but prevents direct dependencies between stages.

    The layered pattern is further enhanced by adding a separate artifact repository for each layer, which makes artifact retention policies a lot easier. You can use bronze, silver, gold as names for the layers to easily understand how release candidate progresses from one stage to another.

    Gary Stafford introduced Y-axis patterns which include joint labeling, aggregate labeling, independent pipelines and VCS labeling. among FullSizeRender (4)others.  Gary used the Microservices example to demonstrate how to do continuous delivery with loosely related applications or services.

    Max Lincoln spoke about concurrent releases with Z-axis patterns. As per Max,   instead of a branching strategy, dev needs a merging strategy, especially if a release branch gets created six months in advance. “If you have Jenkins do 1,000s of builds a day, it does not mean you have CI,” he said.   IMG_6637

    The speakers concluded by each introducing an extra pattern which they referred to as “different dimensions.” The parting word of advice was to keep CI (whether it’s Jenkins or GoCD) in source control and test your pipeline continuously.   At the end we had a great Q&A session and a very warm, open discussion.

    Looking forward to our next session.

    FullSizeRender (2)
    If you would like to learn more about ThoughtWorks and/or Pipeline Patterns, you can continue the conversation on the GitHub page: https://github.com/thoughtworks/PipelinePatterns

  • Performance Testing is a Team Sport

    Performance Testing is a Team Sport

    Performance testing is a vital component of any developer’s working process. The performance of your website or app can be judged by a lot of people in your organization, both technical and non-technical.

    Developers, operators and performance engineers are on the front lines when it comes to the development of a functioning site/app, but there will always be people away from engineering with comments, too.

    “Can we make this faster?”

    “Is the UI making this a little too slow?”

    C-level execs, board members and marketers can often make the journey into becoming amateur web performance testing experts and critics, but that’s okay!

    Web performance is widely accepted as a crucial factor in user experience—for example, when it comes to the e-commerce buyer’s journey. Your application better be fast, or you’re going to end up costing yourself conversions and money.

    Instead of having these conversations after the development and testing has been done, we should definitely include all the stakeholders in the process from the beginning. Sound familiar? (ahem, DevOps, ahem)

    Assemble the Team

    Here are a few simple ways you can get everyone involved in your performance testing cycle:

    Test Early

    In the business of collecting data, getting a baseline is key. Test your minimum viable product (MVP) as aggressively as you would test your “final” product. If the MVP can’t stand up to some pressure early on, it might be in deep trouble moving forward.

    Test Often—and Automate

    The idea here is to create a performance trend to show if you’re moving in the right direction. More data points is always better.

    The best way to generate many data points is to programmatically run tests after each code commit, either using continuous integration software or by having a post-commit hook or something similar in the source code control system that makes an API call to fire off a load test.

    This way, you’re arming yourself with the right information to know when new code is impacting application performance, and you’ll know when to jump in and investigate changes right away.

    Give People Access

    We’re living in the era of transparency in software as a service (SaaS), so it’s always a good idea to give your coworkers access to your testing dashboard data. While some people in the organization might not wholly understand the technical side, it won’t be difficult for them to grasp that an increasing load time can be a bad trend.

    There are all kinds of dashboard SaaS companies out there who can help you do this—assuming you want to stay away from pounding numbers into a spreadsheet or table every day.

    At Load Impact, we’ve received a lot of feedback from engineers who were looking for an easy way to share test information with team members, and from that, we created the “Team Members” functionality.

    Testers simply invite team members to their organization, and the new invitee can see past results and even create scenarios to be tested. That way, everyone’s in the loop, and everyone can get involved to the extent of their expertise.

    Play Nice and Share

    Continuous testing is a mindset as much as it’s a methodology. It fits neatly into the ideals of DevOps, and a big part of that is an inclusive, collaborative culture with your team members.

    So remember, play nice with your colleagues—technical and non-technical—and you’ll create a productive, creative environment for everyone to thrive in.

  • DevOps in 2016: Mainstream Momentum Ahead

    DevOps in 2016: Mainstream Momentum Ahead

    I believe that 2016 will be the year that DevOps takes mainstream foothold in the enterprise. And I base this assertion on seven sweeping changes that — from my vantage point — are taking hold and dramatically changing the digital world.

    1. The Job of IT Will Change From Infrastructure to Innovation. In the past, IT has been about providing compute infrastructure and applications to run the internal workings of a company. But this has been radically shifting as digital transformation penetrates nearly every company of significance across the globe. As a result, IT is increasingly about driving innovation and customer value through software. We see this every day from our work with large enterprises; indeed, almost half of the global 500 use Chef in some manner. And, in our interactions with enterprise customers, over 75 percent cite innovation, velocity and agility as top priorities for IT — and the company. The common ingredients behind this are almost always automation, cloud and DevOps. In fact, while these three elements are, in many ways, distinct, they form a magic IT trinity in service of innovation-led organizations. If you need further proof that IT is on its way to becoming the pointy end of the innovation-creation spear inside large-cap companies and organizations, just look at the auto industry, where CES may actually be more important than the Detroit Auto Show, where GM is investing in Lyft, and where Google is doing state-of-the-art R&D on a self-driving car.
    1. Continuous Delivery (CD) Will Become Ubiquitous. CD is the practice of rapid software development and iteration through fast, small-batch releases. In its purest form, developers can drive small, incremental software releases quickly, sometimes as often as every few minutes for some very well known Web sites. Not long ago, CD was exclusively the domain of big Web innovators such as Facebook, Amazon, Google and Yahoo! Not long ago, making even several releases a day sounded insane. And not long ago, a CIO told me that it was ridiculous to contemplate bringing CD into the enterprise. All this has changed, and today it makes absolutely no business sense to shy away from CD because it’s one of the keys for unleashing an organization’s software development innovation engine. The point is that the whole stack — infrastructure, compliance, security, microservices and applications — is being expressed as code now, and  it’s the same as you deliver with work-flow. In the end, whether your company needs to release frequently all the time, or not, every aspect of IT is charged with a business requirement to do something quickly some of the time. CD is a very important piece of the puzzle.
    1. The Public Cloud Will Triumph in the Enterprise. You may think the cloud is old news, but it’s not. In 2016, the enterprise is going to move beyond public cloud experimentation and project-based work and start migrating entire data centers to the public cloud. We’re also going to see more hybrid cloud initiatives (whatever the composition) because dual adoption helps companies avoid vendor lock-in. As for private cloud projects, there are definitely some good teams and technologies out there. But, having said this, I believe that the public cloud will win out. Many of the lingering public cloud issues, like security and governance, have begun to evaporate. And the ease of use, affordability and effectiveness of the public cloud will simply triumph in the enterprise over the next 12 months at the expense of the private cloud.
    1.  We Will Enter a New Era of System Administration. The world of systems administration has changed forever. In the IT world that I grew up in, a human being logged onto a system to manage, update or fix it manually. Since then, compute scale has grown exponentially for most organizations, thanks, in part, to both server virtualization as well as the cloud. Most of our enterprise customers have, excluding desktops, tens of thousands of compute nodes. Some have over 100,000. A few have over 1,000,000. Administering systems manually by having humans act on systems is no longer viable or tenable. In the new era, humans act on code and code acts on machines. In fact, code is the system. In other words, humans act on compute resources through code. And this reality creates a whole new universe for the world of system administration. The bottom line? The job and role of the system administrator will change significantly. Watch this trend play out in 2016.
    1.  Peace Will Break Out Between Developers and Security / Compliance Officers. This has been a long-running battle inside the enterprise, and there are several theories behind the conflict. One is miscommunication — the  security team doesn’t think the dev team gets it, and the dev team doesn’t think the security team gets it. Another hypothesis is that developers just want to be left alone to write code for the business, but the business is increasingly worried about security — not just IT security. And, lastly, there’s this explanation for the hostility — developers want to go fast, and security teams want to slow everything down to a safe speed. Whatever the reasons, I believe that developers and security / compliance officers will find much more common ground in 2016 because we’re learning that compliance and velocity aren’t necessarily in conflict with each other.  In fact, high-velocity IT organizations can deal with security threats more quickly by assessing vulnerability, creating a fix, testing it and then deploying it rapidly before the damage is done. We’re also seeing the early beginnings of what I believe will be a very important trend in our industry: compliance defined as code. Today, compliance is all too often defined in binders or text documents; what’s a developer going to do with this? But we’re seeing a new idea emerge: define compliance — either to your own security standards or external, regulatory standards in code. If you can define compliance as code, you can then use that code to create a test that can simply be moved into the software release process and managed like any other automated test. Forward-leaning compliance officers will shift their gaze and embrace their developers because it’s through their software developers that they’ll be able to take their craft to a new level.  Keep an eye on this in 2016.
    1.  Executives Will Move Into the DevOps Tent. The DevOps tent will expand to include the executive suite in 2016, and executives are excited to join the practitioner-born DevOps movement. “DevOps” makes one think of only “development” and “operations,” but it’s really a movement focused on how we build and operate high-velocity organizations. Frankly, DevOps is meant for the entire enterprise. The results are both clear and definitive. Mature DevOps organizations see increases in both innovation and quality that flow through to business results, including profit and shareholder value — and, quite possibly, security and safety as well. Executives have seen what code can do, and they’re fervently embracing the far-reaching organizational changes that have accompanied this technology revolution. The good news for DevOps practitioners is that they’re going to find strong support in the executive suite and many will see good things happen to their careers in 2016
    1.  Container Technology Will Be Improved and Grow Even More. Containers — which bundle entire run-time environments into packages that make it much easier to move applications from a developer’s laptop to the test, production and end-user stages — are becoming invaluable, especially in homogeneous green-fields, because they abstract so many problems away. But most enterprises aren’t homogeneous green-fields, so the challenge is how to deal with a variety of legacy technologies without re-writing all the applications that run. I believe that this problem will be addressed in 2016, and container usage will expand by leaps and bounds as a result.

    There will be other big changes in IT during 2016, but, when combined, I believe these seven represent a truly significant tipping point that will catapult enterprises toward the future with a velocity and vigor that can only enhance and enrich growth.

    Abouth the Author/Barry Crist, CEO, Chef

    Barry-Crist-620x930Barry Crist is CEO of IT automation company Chef. Barry has more than 20 years of experience in driving enterprise customer success and adoption with both open source and DevOps software solutions. Prior to joining Chef, he was CEO of Likewise, a company with a strong open source to enterprise go-to-market model that had wide industry reach including OEM agreements with VMware, Dell, HP, Riverbed, and EMC. Likewise was acquired by EMC Isilon in September 2012. Prior to Likewise, Crist served as Vice President of the Application Management group at Mercury, now a division of HP, where he grew North American sales from less than $1 million to more than $75 million. Crist led enterprise sales organizations at Network Associates (now McAfee) and was in various sales, technical, and international roles at Apple.