devopsy.kinsta.cloud

Category: DevOps Practice

  • For DevOps, (Organizational) Size Matters

    For DevOps, (Organizational) Size Matters

    When it comes to DevOps, size matters. The number of people in your organization, as well as the size of its infrastructure, play a huge role in shaping how and why you adopt DevOps.

    The importance of company size may come as a surprise. Few conversations about DevOps mention the role of organizational size in building a strategy.

    That may be in due, in part, to the fact that DevOps engineers tend to be obsessed with the idea of scalability. A healthy DevOps process is one that can support an application with a million users just as effectively as it supports an application with a hundred users.

    In theory, your strategy should be similarly scalable. But in reality, this is unlikely to be the case. The way that a company with a dozen employees does DevOps is going to look quite different from the way a Fortune 500 company does.

    Small Companies

    In a sense, DevOps is easiest to achieve at very small organizations. In fact, it is almost inevitable in companies of this size.

    That’s because, when you have a very small IT staff, your engineers will end up playing multiple roles because that is the only way that all IT tasks can be completed successfully. People you hire as developers will have to help support and test the apps they write because otherwise, those processes might not happen.

    What this means is that at small companies, it tends to be something that happens organically, not deliberately. IT teams at such organizations don’t need to plan a DevOps strategy; they just need to let it come into being of its own accord.

    Large Organizations

    You might expect the opposite to be true within large organizations. But in many cases, it’s not. Large companies may also find themselves building their strategies organically.

    The reason is that, at large enterprises, the IT organization tends to be broken down into many small teams, each working on different apps and processes. Because these individual teams are small, their members also tend to embrace DevOps practices by performing multiple IT roles, whether deliberately or not.

    True, large companies often benefit from planning and implementing a companywide strategy. Encouraging DevOps best practices companywide can help to improve collaboration between different teams within the organization. It can also improve the ability of security or QA teams, which may otherwise operate in silos, to interact with other groups.

    But by and large, if you are an enterprise whose IT staff are broken into a number of small teams, you will likely find that DevOps happens more or less on its own.

    Medium-Sized Organizations

    It’s medium-sized organizations that tend to have the hardest time doing DevOps effectively. That’s because these organizations are large enough to have a sizeable IT organization, but too small to break that organization down into multiple project teams.

    As a result, medium-sized organizations erect the most rigid silos around their IT teams. They might have a single team of developers who focus on building a few applications, and a single IT Ops team whose job is to support those applications. But these two teams may not interact much.

    For this reason, a deliberate DevOps strategy is most important for organizations that are of a medium size. Their silos need to be broken down actively.

    Conclusion

    The bottom line: When it comes to adopting and improving DevOps, take some time to evaluate how the size of your organization impacts it. Is your company sized and organized in such a way that it will more or less happen on its own, or do you need to take deliberate steps to implement it? Answering this question is crucial for achieving DevOps in the most seamless way possible.

    — Chris Tozzi

  • Living in a DevOps World (Part 1)

    Living in a DevOps World (Part 1)

    The concept of DevOps has grown and evolved into a conceptual and working model for more effective and efficient software development and implementation. That said, there are some differences of opinion on the real-world value of any DevOps approach to date, and on the best way to create and implement a real-world DevOps environment. This two-part article will focus on what an agile DevOps approach is meant to address, and what it is not meant to address.

    DevOps sits at the nexus of three essential business technology functions: software development, quality assurance and operations. A short and concise definition of DevOps proposed in 2015 seems as appropriate as any:

    DevOps is a set of practices intended to reduce the time between committing a change to a system and the change being placed into regular production while ensuring high quality.

    The definition was suggested in the book, “DevOps: A Software Architect’s Perspective,” and the authors have hit upon the essence of the practice. The key, of course, is how to put that concept into practice.

    Perhaps the first step on the journey to effective DevOps is the recognition that the concept is the result of the rise of the Lean and Agile software development methodologies. Those methodologies, among other things, emphasize the following:

    • A focus on customer value.
    • The elimination of waste.
    • Reduced cycle time (accomplishing work faster, releasing faster).
    • Shared learning.
    • Avoiding batching (don’t do things until required).
    • Theory of constraints (break things up, focus on individual issues).
    • Continuous integration, testing and delivery.
    • Faster time to market.

    DevOps in Practice

    Adherence to the principles above meant that something had to be invented to accomplish them and that something was DevOps. Over time, an effective DevOps practice should address any number of business technology pain points. The following short list of those pain points and the DevOps response should prove instructive.

    System Downtime

    The developers’ curse since systems were developed, system outages are inevitable as long as systems are designed, tested and implemented—even with increased automation—by imperfect beings. DevOps acknowledges that by changing the focus from trying to create applications that never fail to designing systems that can recover quickly, thus decreasing aggregate systems outage time over the life cycle of any application or system.

    Stagnation

    This was a staple of traditional systems development and is most closely associated with the waterfall methodology for systems development. After requirements were created, the development team would be locked away for weeks, months or, in some cases, years before emerging with “fully” working software that inevitably no longer satisfied rapidly evolving business requirements. DevOps is designed to fit hand-in-glove with the Agile practice of short windows of incremental changes instead of long release cycles, putting working software in the hands of customers as quickly as possible.

    Team Conflict

    Having been borne from the cultural combination of Agile and Lean, DevOps has taken on the problem of functional silos that are often erected between development, operations and the business customers. It follows the methodological approaches of collaboration and teamwork first to understand what others know and to leverage the best of it to solve business problems more rapidly. There is also a cultural bent toward experimentation, continual learning and constant improvement. This leads to blameless post-mortems, where instead of finger pointing when something goes wrong there is collaborative discussion and learning to correct and prevent the problem from occurring again.

    Knowledge Silos

    Functional silos have led to compartmentalized knowledge. If the old game was that knowledge is power, the new game in the DevOps world is that knowledge is freely exchanged as an enabler to solving business problems. DevOps addresses the problem of information being lost in translation between the development and operations functions by eliminating the functional barricades and making knowledge sharing the highest form of collaboration.

    Inefficiency

    Waiting for things to happen used to be a standard operating procedure in the pre-DevOps world. Project plans were created and managed to account for the time it might take for new code to be moved into a testing, quality or even production environment. This was a momentum killer for projects and at times a morale killer for developers waiting to see what changes they might need to make to their code set. The combined Agile and DevOps approach has rewritten the traditional approach to code migration, smoothing and eliminating wait times so that projects flow more seamlessly from start to finish. It also has the benefit of keeping business resources—testers, approvers, etc.—more engaged as a result of a constant flow of new functions and features to test and use. And lest this be viewed as simply a way to keep the technical stuff moving along, it’s important to remember that there is a financial aspect to this as well. Reducing speed to market with new functionality, reducing or eliminating idle hands—be they technical or business—and delighting customers with a steady stream of enhancements and features all go directly to an organization’s top and bottom lines.

    That, after all, is in many ways what the DevOps approach is all about. All of these critical areas become the means to accomplish it. Part two of this article will focus on some more of the benefits of a DevOps approach, and how to achieve them.

    — Dave Farinelli

  • Microsoft Snaps Up GitHub for $7.5 Billion

    Microsoft Snaps Up GitHub for $7.5 Billion

    Recognizing the role GitHub plays in the larger developer and DevOps ecosystems, Microsoft is acquiring the software development platform for a cool $7.5 billion, the two companies have announced.

    GitHub, the world’s leading software development platform, has more than 28 million users who “learn, share and collaborate to create the future,” according to the press release announcing the acquisition.

    “Developers are the builders of this new era, writing the world’s code. And GitHub is their home,” wrote Microsoft CEO Satya Nadella in a blog post. He noted Microsoft is the most active organization on GitHub, with more than 2 million commits made to projects.

    “Microsoft has been a developer-focused company from the very first product we created to the platforms and tools we offer today. Building technology so that others can build technology is core to our mission to empower every person and every organization on the planet to achieve more,” he wrote.

    GitHub CEO Chris Wanstrath was quick to point out the irony of Microsoft, once a bastion of proprietary software and apps, purchasing a repository for open source code.

    “When GitHub first launched ten years ago, I could have never imagined this headline,” he wrote in his blog. “Git was a powerful but niche tool, clouds were just things in the sky, and Microsoft was a very different company. Open source and business, people said at the time, mixed as well as oil and water.”

    That said, there is a certain synergy between the two companies, he noted. “We both believe that software development needs to become easier, more accessible, more intelligent, and more open, so more people can become developers and existing developers can spend more time focusing on the unique problems they’re trying to solve.”

    Under the terms of the agreement, Microsoft will acquire GitHub for $7.5 billion in Microsoft stock. The acquisition is expected to close by the end of the calendar year.

    Keep checking devopsy.kinsta.cloud for updates on analysis on Microsoft’s purchase of GitHub and its potential impact on the DevOps community at large.

    — Charlene O’Hanlon

  • Case Study: Time Series Helps PipelineFX Customers Save Time, Money

    Case Study: Time Series Helps PipelineFX Customers Save Time, Money

    Digital rendering billing and licensing issues have been reduced by a time series solution

    PipelineFX​ ​helps​ ​organizations​ ​of​ ​all​ ​sizes​ better​ ​manage​ the cost of ​rendering​ ​for​ ​digital​ ​media applications​ ​and​ ​programs.​ ​For​ ​more than​ ​16​ ​years,​ ​organizations​ ​of​ ​all​ ​sizes​ ​in​ ​the​ ​film,​ ​visual​ ​effects, post​ ​production,​ ​broadcast,​ ​design​ ​and​ ​education​ ​industries​ ​have​ ​been​ ​using​ ​PipelineFX’s Qube!​ ​to​ ​manage their​ ​render​ ​pipeline and​ ​make​ ​critical​ ​business​ ​decisions,​ ​faster​ ​and​ ​smarter​ ​than​ ​the competition.

    Time Series Solves Customer Problems

    PipelineFX wanted to help its customers​ ​​meet​ ​their digital media deadlines without having to spend the capital to build and maintain their own render farms. The​ ​steps a graphics system needs to perform to render a 3D scene to a 2D screen ​​is​ ​a​ ​very​ ​“bursty​ ​process,”​ so​ ​one customer​ ​may​ ​need​ ​10,000​ ​cores one​ ​night​ ​and​ ​only​ ​a​ ​couple​ ​of​ ​hundred​ ​the​ ​next​ ​night.​ ​In addition,​ customers​’ ​deadlines​ ​are​ ​not​ ​variable, yet ​​the​ ​time​ ​needed​ ​to​ ​render​ ​each​ ​project​ ​is​ ​very different, based​ ​on​ ​the​ ​level​ ​of​ ​computer​ ​graphics,​ ​layers​ ​per​ ​shot,​ ​resolution​ ​of​ ​the​ ​rendered project​ ​and​ ​the​ ​number​ ​of​ ​frames​ ​per​ ​second​.​ ​To​ ​meet​ ​a​ ​deadline,​ ​customers may​ ​need​ ​to​ ​scale​ ​from​ ​500​ ​to​ ​1,500​ ​machines​ ​for​ ​three​ ​days.​ ​But​ ​after​ ​those​ ​three​ ​days,​ ​customers do​n’t​ ​want​ ​to​ ​pay​ ​for​ ​rendering​ ​licenses​ ​that​ ​are​ ​not​ ​being​ ​used.​ ​Their​ ​job​ ​is​ ​finished, so​ ​they​ ​should​ ​not​ ​need​ ​to​ ​pay!

    Real-time view of charges displayed in a monitoring dashboard

    What​ ​makes​ ​the​ ​PipelineFX​ ​offering​ ​so​ ​compelling​ ​is​ ​the​ ​ability​ ​to​ ​measure​ ​utilization​ ​every minute​ ​and​ ​bill​ ​at​ ​a similar time interval.​ ​This​ ​high​ ​level​ ​of​ ​precision ​ensures​ ​that PipelineFX ​​customers​ ​get a​ ​great​ ​return​ ​on​ ​their​ ​investment​​.​ ​With this data, PipelineFX​ ​is able to ​ ​provide​ ​a​ ​very​ ​flexible​ ​pricing​ ​model that​ ​can​ ​accommodate​ ​this​ ​high​-frequency​ ​charging​ ​for​ ​any​ ​license​ ​utilization​ ​over​ ​a​ ​certain per-paid​ ​minimum.​ ​In​ ​addition,​ ​the​ ​system​ ​needs​ ​to​ ​provide​ ​reports​ ​and​ ​insight​s ​into​ ​what​ ​average utilization​ ​rates​ ​are​ ​and​ whether ​a​ ​customer​ ​should​ ​consider​ ​increasing​ ​its​ ​minimums.

    In addition to solving the problem of rendering licenses during bursty processes, PipelineFX wanted to enhance​ ​its​ ​SaaS​ ​infrastructure​ ​to​ ​optimize​ ​customer​ ​demand​ ​and​ ​server​ ​costs. PipelineFX​ customers ​need​ ​to​ ​be​ ​able​ ​to​ ​reserve​ ​the​ ​right​ ​number​ ​of​ ​servers​ ​and​ ​nodes​ ​to​ ​hit​ ​the​ir ​peak demands,​ ​but​ ​they​ ​do​n​’t ​want​ (or need) ​to​ ​purchase​ ​perpetual ​licenses for these short-lived servers.

    A comparison of installed vs. consumed licenses displayed in a monitoring dashboard

    Since​ ​PipelineFX ​​wanted​ ​to​ ​measure​ ​and​ ​bill​ ​at​ ​a​ ​high​-frequency​ ​time​ ​interval, as​ ​well​ ​as​ ​determine​ ​real-time​ ​peak​ ​demands,​ ​its​ ​traditional​ ​relational​ ​database​ ​was​ ​no​ ​longer​ ​proving effective.

    To gather this high-frequency data, PipelineFx deployed a time series solution. ​Customers’ rendering​ ​data​ ​today is​ ​collected​ ​each ​minute​ ​and​ ​transmitted​ ​to​ ​the​ ​PipelineFX​ ​service​ ​every​ ​15​ ​minutes.​ ​If​ ​customers​ ​lose​ ​network​ ​connectivity,​ the application accumulates​ ​the​ ​batch ​and,​ ​once​ ​connectivity​ ​is​ ​restored,​ ​the​ ​whole​ ​batch​ ​is​ ​transmitted.​ ​Often,​ ​eight to​ ​12​ ​hours​ ​of​ ​data​ ​is​ ​uploaded—​some​ ​late​ in ​arriving,​ ​but​ ​the​ ​system​ ​takes​ ​care​ ​of​ ​this as​ ​well. PipelineFX​ ​uses​ ​this​ ​data​ ​store​​ ​to​ ​generate​ ​customer​ ​dashboards​ ​and​ ​provide reporting​ ​of​ ​real-time​ ​usage.

    PipelineFX’s​ ​extensive ​experience​ ​with​ ​monitoring​​ ​tools​​ ​and​ ​​software​ ​development​ ​made​ it ​an​ ​easy​ ​decision​ ​to​ ​use open source projects in its product. The company found early success using open source time series projects to collect and analyze the data needed for its real-time billing offering. Although it was easy to use, PipelineFX was spending valuable development cycles maintaining the solution and decided to review the cost of hosting​ ​the solution​ ​themselves​ ​versus purchasing a​ ​hosted​ version. In its review, PipelineFX found it​ ​made​ ​better​ ​financial​ ​sense to go with a hosted version and rely on vendor expertise​​ ​to​ ​keep​ ​the​ ​software​​ ​up​ ​to​ ​date,​ ​keep the​ ​clusters​ ​fully​ ​optimized and​ ​rely on a 24×7 uptime.

    Results

    PipelineFX​ ​has​ ​been​ ​able​ ​to​ ​help​ ​its ​customers​​ ​achieve​ ​maximum​ ​efficiency​ ​and​ ​optimize their​ ​existing​ ​and​ ​future​ ​infrastructure.​ ​The​ ​combination​ ​of solutions has​ ​allowed for​ ​metering​ ​and​ ​billing​ ​at​ ​minute-level​ ​precision.​ ​In​ ​addition,​ customers now have​ ​a​ ​real-time​ ​view​ ​​of​ ​their​ ​current​ ​usage​ ​and​ ​billing​ ​charges, which enables​​ ​them ​to​ ​maximize​ ​their​ ​investment​ ​in​ ​Qube!​ ​without​ ​having​ ​to​ ​commit​ ​in  perpetual​ ​licenses​ ​that​ ​may​ ​go​ ​unused.

    — Chris Churilo

  • Continuous Testing and Business Assurance

    Continuous Testing and Business Assurance

    As development moves ever faster, continuous testing can help further reduce the risk of introducing errors

    In IT circles, “risk management” implies information security. But in the business as a whole, risk management is a function that spans the organization. The risk that an application will negatively impact corporate image, sales or employee morale is very real in the age of automating everything.

    Once properly identified, these types of risks can be mitigated with continuous testing processes. The ability to set up a test to prove the login system works as expected keeps users able to get to the application.

    Increasing use of agile and DevOps offers the opportunity to receive rapid feedback. Among other things, that includes business (as opposed to InfoSec) risk management. Requirements from the business side can clearly delineate items that create risk for the business unit and the requirements to reach desired outcomes. Continuous testing can turn those requirements into tests that validate the application is not contributing to business risks.

    After all, there are enough risks in the business environment: competition, suppliers, coordination of marketing, legal, etc. Making application development less of an issue is good news for business owners and IT alike.

    By translating business risk management requirements into automated tests early in the development process, stability and risk aversion are built into the application from (or nearly from) inception. This offers stability and provable correctness in the application development cycle.

    As the pace of development continues to accelerate, there is a risk of negatively impacting the user experience. Whether that negative impact be through failures of critical parts of the system or degradation of performance, it harms the application and the business it supports. Continuous testing can help alleviate fears by testing for these scenarios and pointing developers at fixing them early.

    Test automation tools are becoming more prevalent, and the next step is to include business requirements designed to mitigate business risks at every step in the process. It is simple to say and test “valid users must be able to log in,” but as development progresses, if the login prompt takes more than 10 seconds to return, that adds the “in a timely manner” business requirement that is then refined by IT to a number, and a new test is run. Only by having the inputs from the business at each stage can such refinements be identified and implemented.

    Interested in this topic? Check out the ebook here to hear from experts on balancing speed of development with risks to the business.

    And consider it. The purpose of any given piece of software is to serve the business. Speed that introduces unwarranted risk to the business cycle is not the best answer. Continuous testing, and attendant business owner involvement, can mitigate those risks.

    — Don Macvittie

  • Blockchain: A Threat to DevOps—and How Engineers Can Respond

    Blockchain: A Threat to DevOps—and How Engineers Can Respond

    Blockchain is one of the hottest topics in the IT world today—and that creates a challenge for DevOps professionals. Here’s why, and what DevOps engineers can do to remain relevant.

    Blockchain, Fat Protocols and a New Generation of Applications

    To understand why blockchain threatens to decrease the role played by DevOps engineers in software platforms, you have to understand the difference between blockchain-based applications and traditional applications.

    That difference was summed up most famously in an article by Joel Monegro about “fat protocols.” Monegro convincingly argued that prior to the introduction of blockchain technology, most software stacks were composed of “thin protocols” and “fat applications.”

    What he meant was that in most traditional web applications, the majority of the value and functionality lies in the application itself, rather than in the protocols on which it runs.

    In other words, if you write a PHP application, most of the time that you invest and the value you create center around the application code. The protocols that allow the application to work, such as HTTP, are relatively simple and uninteresting.

    In blockchain-based applications, by contrast, the magic happens chiefly at the level of the protocol. The main job of these applications is simply to access that protocol. As a result, the applications themselves are relatively unimportant.

    How It Changes DevOps

    What this means for DevOps is that, if blockchain becomes as central to the software platforms of the future as its proponents hope, application code will become much less important than it is today.

    By extension, the people who write and manage that code—DevOps engineers—will have less of a role to play in software deployment.

    If you take a look at how most existing blockchain platforms operate, it’s easy enough to see how this trend plays out. The amount of effort required to write and maintain a basic “smart contract” application that runs on the Ethereum blockchain, for example, is pretty minimal. You don’t exactly have to be a DevOps genius to churn out some if-this-then-that code that interacts with Ethereum.

    The same is true of an application that writes data to the Bitcoin blockchain to record a real estate transaction, for instance, or one that timestamps a document to a blockchain to create a record of the document’s existence.

    In these use cases—as with almost all use cases associated with the blockchain—the role played by DevOps engineers is relatively small, because there is not much code to write or manage to deliver the platform’s core functionality.

    Blockchain-Proofing DevOps

    Yet, if the blockchain revolution does come to pass, it won’t necessarily be bad news for DevOps experts.

    Even though the core application code required to interact is minimal, crafting platforms that are secure and provide a solid user experience takes more work. This is where DevOps engineers can help.

    One of the most glaring problems with many existing blockchain platforms is that their interfaces tend to be difficult to use. Related software security problems are also well-known.

    In addition, some platforms have performance and scalability problems. Bitcoin advocates like to talk about how digital currency will make transactions much smoother, but the reality is that it can take hours to confirm a Bitcoin transaction. That’s a big problem if you want to buy and sell things with Bitcoin.

    Arguably, these issues originate in part from the fact that many of the people currently designing blockchain platforms are good at recognizing how decentralized, distributed databases can drive innovation, but less adept at writing, maintaining and scaling software and the databases that power it. They are not, in other words, DevOps people.

    By helping to build out blockchain applications so that they are actually usable, secure, efficient and scalable, DevOps professionals can ensure themselves a role to play in the platforms of the future. At the same time, they’ll help the technology transition from its current state of hype into something that can be used to solve everyday problems.

    — Chris Tozzi

  • Appian World Recap: Low Code, BPI, Hackathon & The Woz

    Appian World Recap: Low Code, BPI, Hackathon & The Woz

    I had a chance to attend my first Appian World a few weeks ago. It was right in my backyard, at the iconic Fountainbleau in Miami Beach. I brought a video crew with me and had a chance to do some deep dive interviews with customers, partners and Appian execs. I will highlight those further down. The real story of Appian World, though, was about low code, business process management, hackathons and the Woz. I wrote a recap of the CEO’s keynote from the show, but wanted to follow up with more details.

    shimmy and wozAppian had announced a hackathon with the winners to be announced at Appian World after the judges—John Rymer, Steve Wozniak and myself—selected the winner from five or six finalists. The winner received a $5,000 cash prize. Contestants came from all over the world; some were teams from Appian partners and customers, while others were individuals.

    (Before I tell you more about the winner, let me say that judging this hackathon alongside Steve Wozniak was a personal bucket list moment for me. I did get a little giddy standing next to the Woz and talking with him about the hackathon contestants. But enough about me.)

    Back to the hackathon. After evaluating the finalists, we (the judges) decided that the winner best exemplified the spirit of a hackathon. Brad Cauzza created FRED, a facilities record database. Basically, using Appian low code app building and integrations, Brad built a system that allowed visitors to sign in when visiting a facility. It also tracked incoming packages and more. It was a really nice use of the Appian platform.

    There were some finalists who had grander applications, but we thought FRED was a great example of what you do at a hackathon. Below is my interview of Brad and Malcom Ross of Appian.

    While we are looking at interviews, I also had a chance to sit down for a deep dive with Appian founder and CEO Matt Calkins. Matt has guided Appian from a small group of he and his three co-founders all the way through to public entity. Appian is not an overnight wonder; it has been a 19-year trip for Matt and his team. He shared a lot of his views on this journey, as well as where he sees Appian going in the future.

    As I mentioned, there were actually four founders of Appian and they are all still with the company 19 years later. That is quite unusual, to say the least. I also had a chance to sit and chat at length with another Appian founder, Michael Beckley, who is CTO-founder of Appian, like Matt from day 1 all of those years ago. Here is my discussion with Michael:

    Over the course of the event I spoke with several Appian folks, as well as partners and customers. You can see the whole YouTube playlist of Appian World 2018 interviews on our DevOps TV YouTube channel.

    — Alan Shimel

  • Using CALMS to Assess an Organization’s DevOps

    Using CALMS to Assess an Organization’s DevOps

    Of the tried-and-tested frameworks that allow enterprises to assess DevOps in their organization and how it can be improved, the CALMS model remains particularly useful.

    CALMS, which stands for Collaboration, Automation, Lean, Measurement and Sharing, is particularly helpful for analyzing an organization’s DevOps structure, and ultimately, its utility in any organization. The CALMS framework covers all stakeholders within DevOps, including business, IT operations, QA, InfoSec and development teams, and how they collectively deliver, deploy and integrate automated processes that make business sense.

    “DevOps as a technique is slowly but steadily maturing, so it’s good to see general assessment methods like the CALMS model,” Holger Mueller, an analyst for Constellation Research, said. “The CALMS model gives a good frame of reference to compare the maturity of a DevOps team, and as such,  is invaluable for assessing the state of teams for the transformational change that goes with it.”

    Applying CALMS

    Here are a few ways CALMS can be applied to DevOps for enterprises either thinking about shifting to a DevOps structure or seek to make improvements to an existing deployment.

    Culture

    It is long-established that technology adoption must be to serve a business need rather than investing in technology for its own sake. This shift in mindset applies to how the “culture” component of CALMS supports how the forecasted ROI associated with automating a process must be championed across the DevOps team. This mindset is especially important for getting the non-technical business teams onboard, Mueller said.

    “DevOps must have the backing from the business executives. That’s a crucial factor in the culture part,” he said. “One thing software teams can do is use successful software deployments to help convince the business executives in DevOps to make the investments. Those are the kinds of tribal events that every DevOps organization needs.“

    Automation

    Netflix is well-known as an exemplary case study for DevOps automation. Among other things, it can deploy code and software updates in a matter of minutes across its entire massive infrastructure, many times throughout the day. But Netflix’s example of automation does not take into account what can go wrong at firms that want to follow Netflix’s example when automating their software deployments and updates, Mueller said.

    “The risk is always deploying too fast and you end up deploying bad code,” Mueller said. “Automation, of course, needs to take this into account and compensate. But not everybody can do that.”

    Lean

    Once completed, it is relatively easy to document cost savings by eliminating resources. Within the CALMS framework, the corresponding “lean” component in DevOps is thus relatively easy to assess, Mueller said.

    The good thing about DevOps, Mueller said, is companies can come to a consensus ahead of time regarding what they consider to be a lean operation.

    However, he added, “There is always the risk of being so lean you begin to err on the QA side.”

    Measurement

    The success of software deployments is relatively easy to measure once completed. However, attempting to measure future deployments as part of a forecast model is, of course, much more challenging.

    “It can be surprisingly tricky to measure how good DevOps is ahead of time,” Mueller said. “Measurements should thus allow errors to be pinpointed  quickly during the post-deployment stage to fix the code if needed very quickly.”

    Sharing

    Sharing, in many ways, overlaps with culture. This is where assessing whether all of the different teams within DevOps are working together and communicating comes into play.

    “Thanks to  DevOps, the days of the lone developer are long gone. But sharing is not always easy,” Mueller said. “Even within separate teams, it is easy to fall into the trap of wanting to get your own work done without necessarily thinking about the team as much as you should.”

    The End Result

    While applying CALMS to assess DevOps never hurts, the framework is still not as important as one of the main objectives of DevOps—namely, using technology to make organizations’ operations more agile and reactive to customer needs, Mueller said.

    “At the end of the day, assessment methods such as CALMS are great, but they are never as important as focusing on implementing next-generation applications,” he said.

    — B. Cameron Gain

  • ChefConf 2018: Chef Grows Beyond Infrastructure to Apps

    ChefConf 2018: Chef Grows Beyond Infrastructure to Apps

    chefconf keynoteHere I am in Chicagoland for my third ChefConf. I caught the keynotes today with Nathan Harvey, Barry Crist, Adam Jacob, Corie Scobie and more. The big news at this year’s event is that Chef is focused on more than just infrastructure and compliance—Chef is bringing automation to apps and the entire software delivery life cycle.

    I spoke with Chef CEO Barry Crist about this and he said there is a sea change taking place with software is delivered. Chef has seen this wave coming and, over the last few years, has been retooling its offerings to match this. Today, Chef Automate and Chef Habitat represent the heart of Chef’s arsenal in its mission to offer the premier platform to automate the software development, deployment and security life cycle.

    I personally have seen this shift over the last few years. I think back to when we launched devopsy.kinsta.cloud. Chef was already a popular solution within the DevOps community; however, most people will tell you that they used Chef for configuration management. It was great to be able to make sure that as you spun up new instances in the cloud, your infrastructure was configured exactly as you wanted. By automating and using a Chef-type of solution, you could spin up new instances at-will and their configuration would be exactly the same or exactly as you desired.

    At the time, that was pretty cutting-edge. People were writing Chef recipes for all kinds of tasks—again, usually around configuring infrastructure.  With the advent of microservices, containers, Docker and Kubernetes, this type of automated deployment has become commonplace. But the need to automate, especially with security and compliance and app deployment, is perhaps an even bigger mission.

    Crist acknowledged these facts during his keynote. In speaking about Habitat, Crist noted that not only did many of Chef’s customers not understand the real story behind Habitat, but that half of Chef’s own employees didn’t, either. One person who did was Julian Dunn, director of product marketing, who I interviewed on DevOps Chats about Habitat when it first launched. Tony Bradley wrote that it could be a game changer back then, as well.

    Habitat has come a long way since then and still has a ways to go to grow into its ambitious potential, but it is very functional right now. Chef has several clients here at ChefConf talking about their Habitat deployments and plans. Chef now tags Habitat with Automate Any App Anywhere – Build, Deploy, Manage.

    Perhaps further along than Habitat, however, is Chef Automate. In fact, Chef Automate 2.0 was announced ChefConf, and it seems to be an almost complete rewrite of the program, as it is now built on Microsoft Go. Here is Chef’s take on Automate 2.0:

    Chef Automate has been rebuilt from the ground up to provide performance, scale and advanced analytics required by enterprises using DevOps at scale to meet the demands of post-digital transformation. Chef Automate now provides a single control plane for infrastructure and compliance automation to enable a closed-loop ‘detect, correct, automate’ process to make infrastructure effortless.

    New features include:

    • Enhanced operational visibility and debugging – New tools and visualizations, including a streaming event feed, trend graphs and a rich query language, help users analyze infrastructure and compliance automation data from a unified dashboard.
    • Compliance scanning and reporting in any environment – Integration with cloud compliance capabilities of InSpec 2 enhances scanning, which can now be used to evaluate cloud configurations (AWS, Azure, Google Cloud) via APIs for compliance on either an ad-hoc or ongoing basis. Chef Automate 2 also supports scanning of Cisco IOS equipment as a significant start on extending modern compliance capabilities to network devices.
    • Re-architected for speed and flexibility – A Go-based microservices architecture, with a modern, performant web UI atop a REST API, enables Automate 2 to support tens of thousands of nodes from a single installation. In-place upgrades with automatic data migration from Chef Automate 1 ensure that customers retain all historical information.

    Additionally, Chef has released something called Chef Workstation. From the release:

    Chef Workstation enables DevOps teams and practitioners to complete ad-hoc DevOps tasks using all of the familiar Chef capabilities, without agents. This provides a simple and effective way to use Chef to complete tasks and then seamlessly scale those tasks through Chef Automate.

    Capabilities include:

    • New standard desktop/laptop experience for Chef – Includes a new ‘target mode’ for executing Chef recipes or entire cookbooks remotely and Introduces ‘chef’ command as the single point of entry.
    • Foundation for a better workstation experience – Bundling all the tooling previously included in the ChefDK provides an improved user experience and cohesion between tools to deliver a seamless experience for any infrastructure automation task.

    Finally, new enhancements to Inspec, Chef’s compliance and security solution, were announced. Besides security and compliance being built into most everything Chef does, Inspec now sports:

    • Cloud configuration compliance – Verify AWS and Microsoft Azure policies, with Google Cloud Platform support in beta. More than 10 new AWS resources bring it to CIS Level 1 benchmark completeness.
    • Improved out-of-the-box experience – More than 30 new resources, including support for Cisco IOS network devices as agentless targets, are built in.
    • Drastically improved performance – Users now see up to 90 percent performance gains on Windows and 30 percent on Linux/Unix.

    I will report more as it comes here in Chicago and ChefConf 2018.

    — Alan Shimel

  • Connected Devices and IoT: How the DevOps Approach Can Help

    Connected Devices and IoT: How the DevOps Approach Can Help

    The evolution of connected devices has brought challenges associated with the complexity and managing huge volumes of data. These devices also bring the need for a huge ecosystem of continuous software deployment across numerous application delivery endpoints, including a diversity of devices, mobile, web and others.

    There exists also the opportunity to unify operations and development, thus giving rise to the DevOps environment within the product engineering and IoT space—encompassing tools, processes and people as a single unit for more efficient and coherent working.

    DevOps accelerators and services allow intelligent automation and power the industrial IoT and consumer IoT companies with the capabilities of deployment, testing, build and continuous integration.

    It enables the organizations in connected health care, telematics, networking, security and surveillance, and home and industrial automation to accelerate their product development cycle, deliver significant cost savings via release efforts and reduced outages, enable operational scalability and improve code quality. It will enable the solution and product companies to accelerate their prototype to product cycle and decrease deployment time to days from months.

    Application reliability and availability are the ultimate measures of the success of IT. DevOps applies lean and agile practices all through the software life cycle to attain faster deployment and high-quality application development. Also, DevOps alleviates the rising complexity caused due to new computing trends including mobility, cloud and virtualization via increased automation. Now, let’s dig deeper into the importance of DevOps in the world of IoT and its applications.

    DevOps in the IoT

    The most effective DevOps transformations take a holistic approach, which means taking care of the entire organization in all aspects. This implies including all the constituents of the application, including quality assurance, operations and development teams, line managers and business owners. The constituents may also extend to compliance and security teams. The cross-functional inputs for software functionality and system requirements generates an ongoing feedback loop that helps reduce complexity and promotes engagement.

    Thus, one of the advantages of DevOps is improved collaboration and communications across the company. Deeper collaboration between quality assurance, operations and development teams follows an approach in which they all with collaborate with the client for designing a customized solution to fit their requirements and needs.

    Application testing should reproduce the function of a production system environment. Then, the DevOps teams can locate dependencies, gain knowledge on the functioning of an application when it is in the live environment and, accordingly, make adjustments to the compute environment. These processes become repeatable and iterative with automation and practice, allowing for more deployment, testing and development.

    Analytics and performance monitoring come earlier in the life cycle by moving the testing forward in the process. DevOps can develop performance analytics models which can predict quality and operational problems before deployment rather than waiting to acquire post-production performance data to examine what went wrong.

    These metrics can be utilized to establish KPIs (key performance indicators) and the production environment can be measured against them. The user experience and application performance can be improved as production metrics adhere more consistently to the key performance indicators. At this stage, sharing the data with business teams allows for adjustments to be made with less stress and faster, and accelerates the feedback loop.

    Continuous integration allows for better agility and collaboration to enhance code validation, which in turn reduces risks. The automated processes make for releases of higher quality that are easier to handle once they are in production.

    Treating the infrastructure configurations as code enables DevOps teams to handle provisioning on the fly in software and this sets the stage for the eventual migration towards a completely software-defined environment. With the emergence of the IoT, DevOps will become even more important. The precision of the services and software controlling networks, devices and sensors is critical as every device becomes interconnected in a software-defined environment.

    Conclusion

    Today, DevOps has become the main destructive force in the adoption of many technologies such as blockchain, which is the technology behind cryptocurrencies such as the bitcoin wallet. Owing to its substantial impact on business outcomes, operational efficiencies and deliverables, DevOps has become the major reason behind the evolution of IoT.

    — savaramravindra