devopsy.kinsta.cloud

Tag: devops

  • Successful DevOps Initiatives Start With Company Culture

    Successful DevOps Initiatives Start With Company Culture

    What initially drove the need for embarking on the DevOps journey in your team or at your company in the first place? Was it:

    • Market shift?
    • New leadership or an acquisition?
    • New product or service idea?
    • Cost savings or consolidation?
    • All of the above?

    Whatever drove the need, my guess is there were pre-sales demos, reorganizations and training on new tools that were promised to solve all your DevOps problems along the way. Now, I must break your heart just a little: Buying new tools or apps will not, on their own, ensure positive DevOps outcomes.

    A tool-first approach is ineffective. In fact, it may actually generate backlash from the individuals or groups who are invested in protecting the status quo and result in a wasted implementation effort. Imposing new methodologies or processes across teams through new tools can cause people to dig in, generating responses ranging from passive-aggressive to overtly destructive—both contributing to failure.

    It Starts With Culture

    Your success on the DevOps journey starts with culture. Culture transcends tools, methodologies and processes, making use of all but not relying on any. Culture was, and always will be, present, and much like continuous development—routinely referenced in DevOps—it is continuously influenced by everyone and everything in its environment.

    The successful implementation of DevOps must begin with a conscious effort to reassess and evolve a company’s culture, supported by the tools. For example, continuous deployment as an initiative will only find traction when members of the team all believe, philosophically, it is best for the organization. It requires a certain kind of individual.

    A culture of rapidly evolving and adapting teams focused on continuously discovering, developing and delivering demands a commitment to evolving toward a more contemporary integral DevOps toolset, not the other way around.

    Culture Alignment as a Competitive Advantage

    Cultural alignment is not a buzzword bingo box that gets checked off. One’s sense of alignment with a clearly articulated mission and set of values, coupled with a way of working in a team and supported by the right tools, generates velocity and efficiency. This alignment becomes a competitive advantage.

    But, doing so demands all participants to be more conscious of how what they do affects everyone and everything else, in real time.

    Culture Change is Difficult but Achievable

    It can be very daunting or even scary if your company culture has a long history, producing images of the Borg or Hive, depending on your age bracket. Changing a culture is difficult. Here are some thoughts on where you might start.

    Learning agility: Focus on learning to dance with VUCA (Volatility, Uncertainty, Complexity and Ambiguity) instead of being consumed by it.

    Diversity: Commit to working with people with wide-ranging intellect, backgrounds, titles, CVs, life experiences, ways of learning, gender and energy, instead of working with your mirror.

    Trust: Build a work environment based on psychological safety—a sense of confidence that the team will not embarrass, reject or punish someone for speaking up. Knowing takes time and deliberate effort.

    Leadership: Adapt and promote a leadership style such as the 3 C’s of change leadership–communicate, collaborate and commit.

    Coaching Culture: Consider championing a coaching subculture by applying a methodology such as LACE to work through difficult conversations:

    • Listen to understand.
    • Ask powerful questions.
    • Challenge and support with feedback.
    • Establish next steps and accountability.

    Worth It

    We started with a little heartbreak. But I have had enough conversations about DevOps success and failure to believe that you already knew it was not just about tools. We have all felt the weight of culture when it is not in alignment. Employing some or all of these recommendations will set you down the path of creating your own language that is a requirement for building a culture. Now stop looking at the next great messaging protocol and start working on building an agile, aligned culture.

    — Joshua Platt

  • DevOps Security Champion: Who, What and Why?

    DevOps Security Champion: Who, What and Why?

    In general, DevOps is a process and culture of organizations to get applications out the door faster and with higher quality. To do so, security champions are essential. In DevOps, security champions work as a backup mechanism in various projects and take multiple leadership roles.

    Security champions make effective decisions and take projects forward while strengthening the best security practices. How do you enable security champions in DevOps? In this article, we will describe four practical ways of enabling them in DevOps. But first, we will discuss their role in more detail.

    Security Champions in DevOps

    Let’s suppose you are accountable for creating a mature security process across an organization. To fulfill this project, there are various teams working on different product and services, using multiple technologies and an active approach. However, some team approach security ad hoc security activities while others use existing security technologies and processes.

    After a few weeks of reading documents, learning about calendar releases and suggesting specific knowledge and best practices, you realize that your best efforts around security haven’t addressed myriad issues. You need help, but the central security team isn’t budgeted to help your team. This is where security champions come in.

    Security champions play a vital role in DevOps. They are dynamic members of a team who help in making decisions about the engagement of the security team.

    They act as a fundamental element of the security assurance process for the product and service, and are the main point of interaction within the entire team.

    By taking different leadership roles and decisions inside a smaller team, security champions form a shield of sorts for the team leaders. They give them the freedom needed to drive the project further while supporting and emphasizing the best security practices in the project.

    Enabling Security Champions in DevOps

    Here are four ways to build an efficient team of security champions.

    1. Identifying Teams

    The first and most important thing in building a competent team of security champions is to identify the team you are going to work with. In this phase, distribution of tasks occurs. It is important to note this step should be done before starting the project; otherwise, miscommunication and other problems could occur.

    It is essential to keep a record of the team so the other development departments also know who is reponsible for which of the various tasks.

    To implement this step, you need to talk with the decision-makers and technical managers, such as product owners and company heads to find out information such as the number of people working on different projects and how they can fit into the development teams already working on the project, what programming languages and various frameworks are being used and how they are being implemented.

    2. Role Distinction of Team

    It’s also important to define the role of each team member. Every team must have its security champion assigned to them and the security champions must know what they are supposed to do.

    The security champions should have well-defined goals and purpose, and the rest of the team should communicate with them to ensure best security practices are implemented. The security framework the project follows was selected during its planning phase. As such, the entire team must work with the security champions to ensure the security of the application.

    The security champion should also conduct security reviews regularly, especially at milestones during the project.

    It is important to note that protecting security best practices is not just the security champion. Rather, it is a collective effort of the entire project team.

    3. Select Champions

    Once you have identified the team and defined the roles, the next step is selecting the champions. To accomplish this step, you need approval from managers at all levels, from to-level to product owners and direct team managers. They can help you identify potential champions, choose eligible candidates and conduct interviews to judge their capabilities.

    Because this is the selection stage, you need to describe the role clearly to the selected candidate. You also may explain the strategy you plan to adopt as well as the advantages of becoming a security champion. These include:

    • The chance to attend various security conferences.
    • Being a part of the security meta-team.
    • Increased value in the marketplace.
    • Better product quality.
    • Personal development and the ability to look at things differently.

    4. Stay Connected and Communicate Clearly

    The security champions are required to communicate consistently with their team members and project leaders. They can use any of the various communication channels available to collaborate effectively:

    • Messaging apps
    • Email
    • VoIP apps
    • Mobile phone apps

    Security champions also need to combine as much of the technical data as possible to provide the team safe and secure access, thereby fostering a collective approach toward secure product development instead of keeping everyone in remote smaller groups who don’t exchange knowledge.

    Conclusion

    Security champions play a significant role in DevOps because they introduce a sense of responsibility of security within their department. Their participation starts small and increases gradually. Their presence benefits not only the company but also the application or service itself.

    There are different ways by which you can enable security champions in DevOps to add value to your project. By doing so, you are helping to create a culture of security awareness, which leads to secure applications and improved quality security features.

    — Zehra Ali

  • Chef Extends Automation Reach and Scope

    Chef Extends Automation Reach and Scope

    At its ChefConf 2019 conference today, Chef extended its core platform in ways that collectively serve to automate further the management of applications and the IT infrastructure they are deployed on.

    Extensions to the Chef Enterprise Automation (EAS) Stack include the addition of Application Operations Dashboards, Migration Accelerators for moving applications to the cloud and tighter integration between the Chef Infra and Chef InSpec modules within the Chef Habitat management framework.

    Brian Goldfarb, chief marketing officer for Chef, said these latest additions to Chef Enterprise Automation Stack are part of an ongoing Chef initiative to make it easier to seamlessly unify the management of infrastructure and applications in keeping with best DevOps practices, which Chef refers to as the “Coded Enterprise.”

    Chef DashboardAs part of that effort, Chef Application Operations Dashboards provide visibility into both applications and infrastructure to simplify troubleshooting and compliance, said Goldfarb. For example, IT teams will be able to see how individual security and compliance controls are performing across a heterogeneous computing environment that now includes Microsoft Azure and Google Cloud Platform (GCP) alongside Amazon Web Services (AWS), he said.

     

    At the same time, Chef is trying to make it easier to lift and shift applications to the cloud. EAS Migration Accelerators automate an application migration process that today requires most organizations to spend considerable time and effort to accomplish manually, said Goldfarb.

    An EAS Migration Accelerator for Windows, for example, will scan existing .NET applications running on IIS to discover configuration and dependency details and then package that information as code. That approach significantly reduces the time required to build application packages in Chef Habitat.

    Within Chef Habitat, the integration between Chef Infra, for automating infrastructure configuration, and Chef InSpec, a tool for automation compliance, is now more unified via pre-configured compliance profiles and cookbooks that make it easier to customize and modify which security and compliance controls are tested and enforced.

    Finally, Chef has also added support for high availability to the EAS platform to make sure the environment is always accessible.

    Goldfarb said the goal is to continue to make automation more accessible to IT generalists versus requiring them to hire and train dedicated specialists. One of the primary reasons that DevOps practices have been adopted so unevenly within IT organizations is the bar to master them has been set too high. By making it easier to automate the management of both infrastructure and applications, Chef is moving to lower that bar and make it easier to manage IT by unifying the management of applications and infrastructure, said Goldfarb.

    It’s too early to say to what degree IT automation will advance the adoption of DevOps. What is apparent is that IT environments are becoming too complex for mere mortal IT administrators to manage manually. IT organizations are also expected to be able to respond faster to any issues that arise, which requires IT teams to have a unified view of applications and the infrastructure o which they are deployed. In fact, as IT organizations continue down that IT automation path, many of them one day may discover they have embraced DevOps without even realizing it.

    — Mike Vizard

  • Survey Finds CI/CD Adoption Has Long Way to Go

    Survey Finds CI/CD Adoption Has Long Way to Go

    At the KubeCon + CloudNativeCon Europe 2019 conference, Codefresh published the results of a survey of 130 developers that finds not only are nearly a third (32%) not using any type of continuous integration/continuous development (CI/CD) platform at all, but 60% said their organization has yet to take full advantage of automation to accelerate application development and deployment.

    In fact, according to the survey, it takes more than two weeks for two-thirds of developers (64%) to push a commit into a production environment. Additionally, the survey finds that for 39% of respondents, less than 10% of their company’s processes are automated from Git commit to code to production.

    Not surprisingly, only 19% of developers said they deploy code at least once daily. Roughly a third said they release code into a production environment only once a quarter. At that rate, it may be difficult for those organizations to justify investments in a CI/CD platform.

    Dan Garfield, chief technology evangelist for Codefresh, which develops a CI/CD platform, said that while much progress still need to be made in terms of spurring adoption of CI/CD platforms and best DevOps practices, the shift toward microservices-based applications inevitably will force the issue. The inherent complexity associated with building and maintaining applications consisting of hundreds of microservices based on containers, which regularly are ripped and replaced, will require all development teams to adopt a CI/CD platform. Right now, however, the shift to microservices is even less nascent than DevOps—21% of survey respondents said they are not employing cloud-native architectures because they are overwhelmed by the prospect.

    Garfield said once organizations more aggressively embrace microservices to increase the rate at which applications are developed and deployed, Codefresh is betting most will opt to standardize on a CI/CD platform based on Kubernetes. Most organizations have come to realize they will be building and deploying applications across multiple clouds. A CI/CD platform based on Kubernetes, such as the one offered by Codefresh, enables organizations to consistently apply the same set of best DevOps practices, he said Garfield. As such, competition among vendors that provide CI/CD platforms based on Kubernetes is sure to be fierce in the months ahead.

    In the meantime, a 10-year effort to convince organizations to employ best DevOps practices more broadly appears finally to be on the cusp of becoming mainstream. Organizations of all sizes have embraced DevOps unevenly. In fact, while most organizations have been able to achieve continuous integration with a handful of development teams, very few organizations are anywhere near close to achieving continuous delivery of application code within a production environment.

    The shift to modern applications based on containers, however, will require most organizations to standardize on a set of best practices across the enterprise. Pressure to deliver applications faster in the age of digital business transformation is mounting with each passing day. The only unknown at this point is to what degree will organizations be able to automate those processes.

    — Mike Vizard

  • Security: Are You Doing DevOps Right?

    Security: Are You Doing DevOps Right?

    Phil Kernick, co-founder and CTO of cybersecurity specialist CQR Consulting, has no fundamental problem with DevOps, but asks, from a security perspective, “How many people do it right?”

    If DevOps is going to work and produce secure systems, then developers must take responsibility for security. It’s not something that can be treated as an additional process.

    But abstract responsibility isn’t sufficient. It’s rare for developers to have security tools in their kit and an understanding of their results, said Kernick.

    Learn From History

    As former systems administrator and past president of the Systems Administrators Guild of Australia (SAGE-AU, now the Information Technology Professionals Association), Kernick suggested that DevOps pretends that the discipline of systems administration does not exist and merely treats infrastructure as if it were a software library.

    Spinning up an instance is easy, but do you know that it is properly designed and maintained? This is a real concern, given the number of people working as developers without professional training, Kernick explained.

    Repeatable loops and short development cycles do make sense. But are you confident all the individual pieces being assembled were built properly?

    That raises questions about the software supply chain. Where did that Docker container actually come from? Is it well-maintained?

    The process needs to be managed, yet, people at some organizations are “just grabbing things from anywhere,” Kernick said.

    “It almost integrates the ‘not my problem’ attitude with development,” he noted.

    Developers tend to care about whether a component works from a functional perspective, but an untrustworthy piece of code will compile and run.

    Developers working in a DevOps environment need to understand the tools that help ensure code is secure, and they need to understand infrastructure, said Kernick.

    ‘Code Reuse is Bug Reuse’

    DevOps makes the old joke that “code reuse is bug reuse” even worse, said Kernick, citing the example of a piece of vulnerable code shared on the Stack Overflow site and reused more than 100,000 times in GitHub-resident projects.

    “I don’t want DevOps running in my bank,” he said. What he does want is well-engineered, secure, well-run, certified software.

    Sacrificing security might save some time and get a new product on the market sooner, which might be acceptable if there are no real risks to the user.

    It is easy to tell if a system is really important, as it involves tradable value, safety or privacy. Obvious areas include banking and health. Similarly, it is clear that a website that allows access without requiring any credentials is unimportant.

    But there’s a wide middle ground where it’s “really hard” to determine a system’s importance because it is so difficult to imagine how criminals might exploit it.

    For example, Kernick once bought a DVD cataloging app to help avoid duplicating items already in his very extensive collection. He thought this fell into the “unimportant” category. But—fortunately, as it turned out—he took the usual precaution of using a unique password for the associated service. Sometime later, he received a sextortion email that proffered that password as supposed proof that the criminal had gained control of his computer.

    Worryingly, when he told the developer what had happened, it did not accept that there was a problem. The company said the password quoted in the malicious email wasn’t his password for the service—proving in the process that it was storing passwords in plain text!—even though he explained he changed his password as soon as he received the message. Predictably, he uninstalled the app.

    Clearly, Kernick didn’t know what development methodologies that company used, but that’s not the point. The anecdote shows that a seemingly unimportant system can quickly become important to the user in situations outside the developer’s control. Consequently, security is always important.

    DevOps and Security

    He suggests that as long as an organization has the right security and governance processes in place, it won’t matter what development methodologies are used. So it can safely adopt DevOps or whatever else is most efficient in the circumstances.

    It is possible to use DevOps practices to achieve speed without sacrificing security, Kernick said. But to do that, organizations need to take advantage of advances in software development without throwing away hard-won security experience.

    — Stephen Withers

  • Using Automation Tools to Drive Collaboration, Innovation

    Using Automation Tools to Drive Collaboration, Innovation

    How automation can help business move past the walls of confusion between dev and ops

    In the current state of things, businesses are leaning more than ever on the collective intelligence of their tech teams to drive innovation within their respective organizations. These teams are always comprised of developers and operations professionals. With the foray into automation and the desire for companies to do more with less, it has never been more critical that all individuals, regardless of team, come together to do amazing things. Of all of the services available, Chef is a great example of a suite of tools that want to empower teams to do just that: come together and innovate. It is all about bridging gaps, which can range from simply redefining how teams think about each other to more complex things such as knowledge gaps.

    The first gap that almost always seems to come up when bringing teams together and driving business value is the duties gap. For too long we have sat on opposite sides of the coin. In operations in particular, too often we see ourselves as the ones that have to deal with the aftermath of unfortunate problems within code. But the fact is, we should all care about our systems, we should all care about our code and we should all be ensuring the longevity of both.

    Once you’ve moved past the first hurdle, then comes the gap in knowledge.  Scripting has become pretty common as companies press for automation, but in some instances, it becomes necessary to build up existing team members. Using enterprise automation tools such as Chef, entire teams and entire companies can begin caring about a shared thing: the configuration, health and state of their infrastructure and applications. In doing so, those same teams can begin speaking a shared language. Once this knowledge and shared language are in place, developers no longer view the ops team as an “other,” but rather as collaborators. They can each use their various areas of expertise and speak the same language without having actually having to speak the same language.

    The third common gap is the process gap. Process can represent all of those past processes that a company is reluctant to let go of, which can hinder the ability of the developers and operations humans when they’re attempting to collaborate. The process gap can become particularly difficult to move past without leadership buy-in; as such, it is critical to involve business decision-makers in this transformation. While nearly the entire industry is beginning to move toward more collaborative environments, the entire leadership team must be onboard with the requirements to make changes happen. With the use of automation tools in particular, though, processes can begin to shift more toward collaboration-friendly policies and you can then make the case for further expansion with proof that it can aid in your transformation.

    Empathy is the final gap that must be overcome. The great part about the other gaps is that they work hand in hand to solve each other’s problems. By promoting collaboration, resolving these gaps are also facilitating collaboration. The empathy gap is largely caused by silos of the past that have led to the commonly talked-about “wall of confusion.” Closing the knowledge and duties gap, developers and operations humans can now communicate and understand the same language and processes each is following. This allows the wall to be broken down and operations humans and developers alike can care about both code and stability.

    It’s easy to start getting dev and ops teams working together and on a more even playing field once these gaps have been eradicated. This becomes a big driver for innovation and is, in many cases, a true recipe for success within the company. During my career, I’ve both watched and been involved in these transformations. Working with solutions such as Chef has allowed me to cross the bridge myself, and likewise, be able to share with others how that can be scaled out to entire organizations.

    — Brittany Woods

  • DevOps Chat: We Are Devo, With Walter Scott

    DevOps Chat: We Are Devo, With Walter Scott

    If you have massive amounts of data, Devo wants to help. The Madrid-based company specializes in the handling and processing of massive amounts of data.

    In this DevOps Chat we spoke with Devo CEO Walter Scott about the use cases for Devo in a data-crazy world. Walter is a seasoned CEO in the tech sector. He knows a problem when he sees it and he thinks Devo is the answer. Have a listen to why and check out Devo at devo.com.

    As usual, the streaming audio is immediately below, followed by the transcript of our conversation. Enjoy!

    Transcript

    Alan Shimel: Hey everyone, this is Alan Shimel, DevOps.com. You’re listening to another DevOps Chat. Really happy to have a new company, a new friend, joining us on DevOps Chat today. I have Walter Scott, CEO of Devo. Walter, welcome to DevOps Chat.

    Walter Scott: Thanks, Alan.

    Shimel: So, Walter, you and I met recently in Iceland, of all places, at a Insight IGNITE event that really kinda focused on DevOps, cloud, cybersecurity, if you will, the topics, but it was my first exposure to Devo. I don’t know if our audience is familiar with Devo. Why don’t you–give us the foundation, man. “We are Devo.” What are you guys? What do you do?

    Scott: Well, Devo is a data analytics platform that’s the next gen of technology, that provides a data fabric to allow our customers to take all of the noise that exists in the machines and petabytes, and petabytes of data coming in, for them to really start getting signals so that they can start their automation or their AI or their machine learning process, to make their companies more efficient; to be able to have them act more quickly; as well as to be able to get the people who use our platform, to get them the time back that they spend in managing the underlying data platform and allowing them to add the value to drive IT, to drive that value back to the business.

    Shimel: Excellent. And, you know, Walter, one of the things that impressed me with Devo was this wasn’t your sorta typical Silicon Valley story—a couple of wiz kids come up with a PowerPoint business plan and get funded via VCs, pre-revenue, pre-product even. Devo—the underlying technology here, the company’s been around a while and it’s not a Silicon Valley company, right?

    Scott: That is correct.

    Shimel: Give us a little background on that.

    Scott: Well, you know, over my six stints of being a CEO, I’ve never really run an American-based company. There’s great IT being built all over the world. I’ve done them from Malta to Scotland, Germany to Armenia to now with us here in Madrid, is where the company started. And we had a not Silicon Valley startup, but it was five guys—or four guys and a girl or woman—and started off the product in a research key, to really build something special and change the market because the bank that they worked for had to completely change the way they do business.

    And over the course of three years in research and we finally launched a product in 2014 and have been three-figure type growth every single year, since the date of the company being founded, ’cause it’s such a large market. And it’s really accelerating because of what we’re seeing happening with businesses and trying to get ahead of everything from getting arbitrage out of supply chain to solving their security problems, just to start with basic log management and monitoring to consolidate tools and increase agility.

    Shimel: Sure. So, Walter, you were—as you alluded to, this is your sixth or seventh startup where you’ve come in as CEO and had some tremendous success. And, look, not everyone is a success, right? That’s the mantra in business, but you keep plugging. When we look at Devo—and you and I spoke a little bit about this in Iceland, but so much of what we see as, quote unquote, “AI” today is really machine learning. And so much of machine learning is based, really, on data, right? You’re only as good as your data. In order to find patterns, you have to have enough data to analyze it and make actionable kinda calls out of it.

    But this is not a new problem and there’ve been other solutions. What makes Devo’s solution—I don’t know if you use the term “next-gen” or “new-gen” or something like that, but what makes Devo’s solution different?

    Scott: Well, you know, this has been going around a long, long time and you’ve heard me say, in the absence of data, the machine can’t learn. And I go back, all the way back, to the Walmart days in the ’80s, when they built—I would argue … them as the first person to really take the competitive advantage, coming from their data warehouse, right? And —

    Shimel: I agree.

    Scott: —They actually had the line, they could tell your daughter’s pregnant before you could by looking at the data. Well, today, that’s become necessary, but, over the last 10 years, it’s really—what’s happened with the cloud is being able to get that data manageable. I don’t like the term “AI.” I like more “augmented intelligence” because we’re not at the place—or even narrow AI ’cause I think it can solve very specific business problems that people might have. We have one customer that needed to actually build some ML and machine learning to detect what a bot was, were hitting their site, because, when they launched a new product, the bots would come in and sell it all in the secondary market. They wanted that brand loyalty to go to the end users.

    Or we have another customer that has a bunch of IoT devices integrated into their sporting equipment, where they’re trying to figure out if somebody’s gonna get hurt and using machine learning to be able to define what is that case, in terms of repetitive hits, but they need that data over long scales. And just the data science is actually making that possible. It could be done in the past; the problem was it just wasn’t cost-efficient enough to do it.

    We have customers that are taking literally over a trillion events a day to try to—and building machine learning models to understand which customers may leave their environment, based upon dropped calls or poor support integration. And it’s not just having that one set of data; it’s about having hundreds of different sets of data to help prove those hypotheses, to get those more and more efficient. And the data scientists are spending 70% to 80% of their time getting their arms around the data. We take that problem away from them and allow them to have—to solve that other 80% now, they can put in training their models and getting data to make them more and more efficient.

    Shimel: Excellent. Great answer. Hey, you know, Walter, you and I, this is neither one of our first rodeos. And, when I look at your job, your mission over there, at Devo, one of the things that I guess it’s a double-edged sword ’cause it presents so many opportunities, but, at the same time, there’s an axiom in business—“Put all your wood behind one arrow,” right? There are so many use cases for what you can do here, right? There’s the cybersecurity case, right? Where, “Look, man, we need this data. We need to be logging everything. We need to find if we’re under attack, where the attack’s coming from, what’s happening?,” right? So there’s that. And one can spin up an entire company around just using this kind of technology in cyber.

    You alluded to a few other use cases, right? That had nothing to do with cybersecurity. Better business intelligence. Understanding your customers better. Understanding your product better. There’s only so many targets, though, that you can go after, right? Without a tremendous—you know, you wanna build an IBM kind of force or something or a Dell force, that’s a whole different story. So how do you—Walter, as the CEO, how do you prioritize? What are the best use cases for this technology? What are the right markets for us to pursue?

    Scott: Well, what you say no to is just as important as what you say yes to, in a business.

    Shimel: Yep.

    Scott: And, right now, we are very, very focused in the Americas, as well as in western Europe, right? So we’ve been able to say no to Asia for another year or two. Our value proposition is really about the biggest of the big—the telcos, the banks, the large retailers, people who’re looking at real scale that they’re trying to solve problems with. But the fact is, is where I see the market going over the next three years is more about consolidating the DevOps, the ITOps, the Network Ops, the SecOps. All of those are coming into one. And I’ve heard a couple of people start using the term “fusion center,” which is really helping them drive the business and stop pointing fingers and reducing that mean time to response.

    Well, every security product starts with the logs, right? And every IT monitoring product starts with the logs and the data. And the advantage of us being able to load in all of that relational data, which is about 20% of our data that’s in our systems, across our customer base, is things that is coming out of Xdata or data warehouses or relational or customer application type or weather data or threat hunting data that comes in, and being able to combine all of that. So, for us, it’s the larger customers or companies that have data at scales that we’d see in the global 1,000 and really looking at the people who understand that what Google, and Apple, and Facebook, and Netflix have done is taken a data-first approach to their business.

    And, when we look at customers that are forward seeing—that see forward into the market, they realize there are a whole new set of competitors and being able to ask what their real competitive advantage is in their business and the data that it takes for them to be able to do that, whether that’s a telco to figure out how to launch a 5G network because they have all the 4G, 3G and 2G data, whether it’s in their customer retention rates or net retention of “Can they get more dollars out of their customers each year?” to whether it be their manufacturing process and figuring out that arbitrage in the supply chain.

    So I think we are—because of the ease of use of our product, because it’s so easy to get data in, because we keep data in raw format. So you can have one set of data that’s very easy to get in and use it 100 times, not having to index it or reduce that data because it just makes it more efficient.

    But I see us, in the next couple of years, really focused on the security side of the equation. Looking at people who’re doing the transformation and the weight that it’s gonna take on their DevOps and CloudOps teams and being able to work closely with the network and the DevOps teams to be able to grow their business because the cost of becoming so great on having so many different tools, at different layers of your network, I say that the best and the brightest companies will take control of their own data, consolidate those tools so they have a better competitive advantage.

    Shimel: Good. Good. Good. Walter, I’m gonna throw a term at you and I want your opinion on it, “AIOps.” What do you—this is a term that we, here at DevOps.com anyway, we see it rising with a bullet. And, like you—to me, there’s AIOps with a capital “A” and a capital “I” and it’s like Robby the Robot stuff that’s gonna make us live like The Jetsons, finally. And then there’s AI with a small “a” and a small “i,” like the real, functional, kinda down in the dirt–maybe it’s not as sexy, but it provides real change for us. AIOps is—man, everyone—what’s your story on AIOps? It reminds me of when cloud came out in 2005. What do you think?

    Scott: So let’s be very clear. AIOps is a term that’s been coined by Gartner. Out of the last 100 conversations I’ve had with CIOs, I’ve only heard it referenced once or twice. I think it’s more the vendors that are getting on it. And, when you look at it, log management is a big part of AIOps. You know, user behavior analytics is part of AIOps. The anomaly detection is part of AIOps, right? There’s seven or eight different components to that. I don’t think any vendor right now, in the world, really has defined themselves as an AIOps vendor. I think we’ve probably got three years, through help of M&A and deciding what that is, ’cause incident response or the service management is also there, so I think it’s gotta evolve.

    But, of the seven or eight different components, the monitoring, the dealing with events, the user behavior, the anomaly detection and the log management is there, which is where we focus. And I think the enterprise log management, I thought, five, six years ago, was solved, when we solved who all those vendors are, but now, with the complexity of the cloud, it isn’t solved at the scale customers need to do with it. We had solved router management; now, with SD-WAN, it’s a whole new market.

    Shimel: Yeah. You’re right. I mean, and it’s funny that things that–it’s kinda like the measles, right? We thought we cured measles and here we are back with measles. Walter, we’re both CEOs of companies and I don’t know how many people listening out there are CEOs—many of our listeners are kinda getting their fingernails dirty, practitioners as well as managers at large enterprise—what keeps you up at night about Devo? What do you worry most about, Walter?

    Scott: Well, I think part is we have to manage tens or hundreds of petabytes of data for our customers, right? And I think we have a very large team internally for security, to make sure that the data that we have is protected and we work very closely with Amazon and Google and Azure to make sure that we address that. And that’s something that you’re never gonna sleep well ’cause you have to be passionate about making sure that you protect your customers.

    We were on the phone literally an hour ago with a manufacturer that makes toys for children, and that was the question that’s keeping him up–he says, “I have 10-year-old boys and girls logging into my website. I have to protect that data, making sure in the things that we do with being able to obfuscate data at the row, in the column level, without having to duplicate it and our algorithms that are in there.”

    But my job, as a CEO, is to help people build careers. I’ve been very fortunate in my career and have had some very large exits in part of the business, but it’s making sure that the team around us continues to make sure that we handle all of the things, in dealing with our business and our main offices in Madrid. We also have a big office in Cambridge, Massachusetts, because I love the guys and women coming out of MIT, where we do a lot of hiring. But it’s mostly about the team and then making sure that we continue to have what I call a industry-abnormal or an anomaly in terms of net retention. And our goal is to help our customers continue to grow because it benefits us and wanting to make sure that we make it easier for them to be able to get more value because our mission is about making IT so that they can provide data to the business that is actionable.

    And my belief is is the CISO or the CIO that has control of that data, if they push it out to the business and say, “Hey, did you know that, on this particular case or when this happens, we’re losing customers or we’re gaining customers?” That data is 80% more likely to be leveraged than it is if the customer asks the business. So—

    Shimel: Nice.

    Scott:–I think it’s empowering the IT to accelerate value to the business and most of my nights are, well, we see this new customer is bringing in this type of a data, bringing in our data science teams to help think about use cases that we can drive more value from.

    Shimel: Excellent. So, Walter, as I mentioned to you before we started recording today, the good news and bad news about this audio format is it goes so darn quickly. And we blew past our time. So what we’ll do, though, is we’ll—secret’s out: you live down in Florida, too. We’re gonna need to get you down here and into our studio and let’s do a whole video on this. We’ll continue our conversation in a video format. But, for now, we’re gonna call a—have to pull the plug on this one. Walter Scott, CEO of Devo, thanks for being our guest today on DevOps Chat.

    Scott: Great, Alan. Thanks.

    Shimel: All right. Hey, this is Alan Shimel, DevOps.com, and you’ve just listened to another DevOps Chat.

    — Alan Shimel

  • Adobe Aims to Build Developer Ecosystem

    Adobe Aims to Build Developer Ecosystem

    Adobe today outlined an approach to building a developer ecosystem that revolves around open source code, application programming interfaces (APIs) and DevOps processes.

    Matt Asay, head of developer ecosystem for Adobe, said in the last few years Adobe has increased its reliance on open source code to deliver offerings such as Adobe Experience Manager, a content management system that incorporates open source code for as much as 65% of its modules. In the course of that effort, the company has made reciprocal code contributions to a variety of open source projects being developed under the auspices of the Apache Software Foundation (ASF), including Apache Sling, a web development framework; Apache Oak, a content repository; Apache Felix, an implementation of a services gateway; and Apache Lucene, a text search engine.

    Adobe also recently launched Adobe Experience Cloud, which relies on a variety of open source technologies to create a big data repository on top of which Adobe is encouraging developers to invoke APIs to build applications. Rather than build their own big data repositories, Adobe is making a case for relying on a multitenant big data platform it manages on behalf of customers.

    Based on GitHub data, Adobe currently ranks as the 14th largest corporate open source contributor, up from No. 32 in 2017. One of the primary reasons Adobe was able to move up that list so quickly was its move to acquire Magento, an e-commerce platform based heavily on open source code. But now that the company has come to appreciate what can be accomplished by relying on open source code, organizations should expect to see Adobe accelerating the rate at which it will be making contributions to a wide variety of open source projects.

    At the same time, Asay said, Adobe is making major investments in developing microservices that it plans to make it possible for developers to consume via its APIs. The Adobe Experience Cloud, for example, is a suite of software-as-a-service (SaaS) applications that employ a common data model deployed on a set of Kubernetes clusters running in the Microsoft Azure cloud.

    Asay noted Adobe is trying to make it clear that it has a lot of empathy toward developers committed to open source code. Adobe is a provider of commercial software-as-a-service (SaaS) applications, but it wants to develop a developer ecosystem that consumes the data residing in those applications via its APIs. To foster the development of those applications, Adobe has already signaled it is willing to share best DevOps practices with organizations that commit to invoking those APIs.

    Adobe is one of many providers of commercial software that are increasingly dependent on open source code. It will be interesting to see where the line between the desire to use open source projects to build applications and actual open source applications lies in the years ahead. But the one thing that is clear is that without DevOps processes and a well-defined set of APIs to guide them, there’s not much chance any vendor would be able to build a meaningful developer ecosystem of any size.

    — Mike Vizard

  • Kubernetes Adoption: Are You Ready?

    Kubernetes Adoption: Are You Ready?

    Enterprises are rapidly embracing a cloud-first strategy and microservices play a pivotal role in this journey. But enterprises are achieving various levels of success, measured by many factors includingmaturity of DevOps in the enterprise, ability to identify the right candidates for microservices, selecting the appropriate container management platform, adequate tooling and, finally, cultural shift in design thinking.

    As there is plenty of material around microservices adoption and strategy, this article focuses on the container management ecosystem, which plays a critical role in the success of microservices adoption. Today, Kubernetes has become the de facto platform for container orchestration. Most public cloud providers offer Kubernetes as a service in some form to abstract the complexities of managing Kubernetes as a platform.

    Currently, we see three broad patterns of how enterprises are adopting Kubernetes within their organizations.

    Pattern 1: Kubernetes as a Managed Service

    While Kubernetes makes container management somewhat simple, it is still a complex beast to understand and manage as a platform. Enterprises that have already embarked on a cloud strategy find it easy to test the unchartered waters of Kubernetes by leveraging Kubernetes-as-a-service provided by their cloud providers. It enables them to understand the impact of Kubernetes on their respective application architecture strategy and identify the gaps in their systems (such as the maturity of DevOps management, security management and release management) as well as their monitoring capabilities.

    Pattern 2: Kubernetes as a Turnkey Solution

    Certain enterprises driven by industry regulations and are technology-focused (with considerable experience in other container technology solutions), have shown interest in setting up Kubernetes as part of a private cloud or on–premises with a view to avoid vendor lock-in and limited control associated with managed Kubernetes. There are many such turnkey solutions out there to choose from.

    Pattern 3: Self-hosted Kubernetes Platform

    This category is small compared to the other two categories. However, there are a few enterprises that fully embrace the open source software development model and contribute to Kubernetes community. There are tools and frameworks that can enable the enterprises to setup and manage Kubernetes on their own.

    There are advantages and constraints in adopting any of the above approaches. The decision regarding which strategy to choose depends on the maturity of the enterprise and their business strategy.

    Challenges in Adoption

    Some of the key challenges to be considered, irrespective of the adoption pattern selected, include:

    1. Namespace management: Namespaces play a key role in managing the isolation across application domains and environments, providing controlled management of resources and user management. Bounded contexts can be linked with namespace definitions and standards. It is important to define proper naming guidelines to drive namespace management across clusters.
    2. Resource management: It is necessary to define resource utilization (resource quota) guidelines to ensure the containerized applications co-exist without being eliminated due to resource violations at runtime.
    3. Policy management: It is critical to define appropriate policies including cluster access controls, service access controls, resource utilization controls and secret access controls.
    4. Audit and compliance: It is important to audit the platform for patch levels, secret stores, compliance against the security vulnerabilities, encryption of secret stores, storage volumes, cluster policies, role binding policies and user management controls.
    5. Chaos testing your platform: It is necessary to subject the platform to chaos testing to ensure the robustness of the cluster. It also helps to test the stability of the containerized applications and the impact of crashing these containers. It will also provide vital information about how the platform behaves during high loads or when resources are unavailable for the platform.
    6. Managing the deployment manifests: Kubernetes follows declaration-based management, which means that every object or resource instruction is described through declarative manifests. This leads to versioning complexities, prone to human errors and data corruptions. It is necessary to leverage tools or create custom utilities to manage these manifests.
    7. Observability of the platform: While Kubernetes provides various mechanisms to monitor its own internal components, it does not recommend a standard monitoring approach. It is necessary to introduce well-defined health management standards while designing the microservices. Include appropriate probes for liveness and readiness probes in every containerized application deployed on Kubernetes cluster. Integrate the probe-generated metrics and platform-generated metrics with an external monitoring platform to have comprehensive observability of the cluster.
    8. Continuous deployment of services: It is not recommended to use kubectl-based deployments in a large-scale production setup. Instead, use some of the established open source frameworks specifically built for Kubernetes (such as Helm or JenkinsX) to manage seamless deployments.
    9. Archiving and backup: Kubernetes uses etcd as its internal metadata management store to manage the objects across clusters. It is necessary to define a backup strategy for etcd and any other dependent persistent stores used within the Kubernetes clusters.
    10. Container security management: Kubernetes deploys the applications by downloading the respective images from a pre-defined image repository. It is the responsibility of the repository ecosystem to ensure the image provenance of the images being deployed. Continuous security must be included as part of DevOps pipeline to ensure containers are well-managed.
    11. Service mesh support: Service mesh plays a critical role in streamlining the service to service communication. It abstracts the service level access controls and security management between services through externalized policies. It also provides routing and load balancing features. It is necessary to choose an appropriate service mesh that is compatible with the Kubernetes cluster as well as the underlying infrastructure.

    Conclusion

    Kubernetes makes the container operational aspects simpler, but it requires a methodical approach and best practices to ensure smooth maintainability, as well as seamless developer interaction. For any enterprise which are in a transition journey toward adopting Kubernetes, it is imperative to have a strategy for Kubernetes adoption to ensure success.

    — Sreekanth Nyamars

  • The Bug in Production: What You Don’t Know Can – and Will – Harm You

    The Bug in Production: What You Don’t Know Can – and Will – Harm You

    Despite the risk of unplanned downtime, many organizations that develop software and services push them live without adequately testing for bugs that will manifest against production traffic. This is a huge gamble: Those bugs could lead to errors that bring down the service altogether. Pushing live without sufficient testing can be detrimental to your business.

    The Problem of Going Live With Bugs in the Code

    Software downtime leads to loss of revenue and of reputation. In fact, Gartner analysts have estimated that the average cost of downtime is $5,600 a minute—that’s well over $300,000 an hour. To provide a real-life example of what this looks like, Microsoft Azure suffered a major outage in November 2018 caused by issues introduced as part of a code update, lasting for 14 hours and affecting customers throughout Europe and beyond. With migration from legacy systems to micro-environments in the cloud, outages and downtime pose a growing and serious problem.

    As companies switch to DevOps and CI/CD models to move faster and provide application updates sooner, software developers continually release new features and often push code updates as fast as they’re written. The classic six-month development timelines of dev, quality assurance (QA) and beta testing have been compressed to days and sometimes hours. Gone is the time when teams could beta test with customers for extended periods to flag real-time bugs.

    With current quality testing tools, developers don’t know how a new software version will perform in production—or whether it will even work in production. The Cloudbleed bug is an example of this problem—in February 2017, a simple coding error in a software upgrade from security vendor Cloudflare led to a serious vulnerability discovered by a Google researcher several months later.

    Flaws can lead to serious security issues later, in addition to having the immediate impacts mentioned above. Heartbleed, a vulnerability that arose in 2014 and stemmed from a programming mistake in the OpenSSL library, left large numbers of private keys and sensitive information exposed to the internet, enabling theft which otherwise would have been protected by SSL/TLS encryption.

    Standard QA Testing Isn’t Enough: Test With Production Traffic

    The way QA testing is typically done is no longer sufficient for today’s increasingly frequent and fast development cycles. Traditionally, DevOps teams haven’t been able to do side-by-side testing of the production version and an upgrade candidate.  The QA testing used by many organizations is a set of simulated test suites, which may not give comprehensive insight into the myriad ways in which customers may actually make use of the software. Just because upgraded code works under one set of testing parameters, doesn’t mean it will work in the unpredictable world of production usage.

    As in the case of the Cloudflare incident, the error went entirely unnoticed by end users for an extended period of time and there were no system errors logged as a result of the flaw. Just as QA testing isn’t sufficient, relying on system logs and users also has a limited scope for what can be detected.

    It is estimated that fixing flaws after a software release can be five times as expensive as fixing them during design—and it can lead to even costlier development delays. Enabling software teams to identify potential bugs and security concerns prior to release can alleviate those delays. Clearly, testing with production traffic earlier in the code development process can save time, money and pain. Software and DevOps teams need a way to test quickly and accurately how new releases will perform with real (not simulated) customer traffic and while maintaining the highest standards.

    By evaluating release versions side by side, teams can quickly locate any differences or defects. In addition, they can gain real insight on network performance while also verifying the stability of upgrades and patches in a working environment. Doing this efficiently will significantly reduce the likelihood of releasing software that later needs to be rolled back. Rollbacks are expensive, as we saw in the case of the Microsoft Azure incident.

    Some organizations stage rollouts, which requires running multiple software versions in production. The software teams put a small percentage of users on the new version, while most users run the status quo. Unfortunately, this approach to testing with production traffic is cumbersome to manage and costly, and still vulnerable to rollbacks. The other problem with these kinds of rolling deployments is that while failures can be caught early in the process, they are—by design—only caught after they’ve affected end users.

    This brings more questions, including: How do you know whether the new software is causing the failures? and, How many failures does the business allow before recalling or rolling back the software, since the business does not observe side-by-side results from the same customer? This disrupts the end user experience, which ultimately affects business operations and company reputation. And staging may not provide a sufficient sample to gauge the efficacy of the new release versus the entire population of customers.

    Cost is still an issue as well. If you stage with 10% of customers on the new version and a failure costs more than $300,000 an hour, then a failure affecting 10% of users could potentially still cost more than $30,000 per hour. The impact is reduced, of course, but it’s still significant—not counting the uncertainty of when to rollback.

    Looking Ahead

    Standard QA testing is no longer enough. To reduce the risk injected into the software development life cycle by today’s rapid iterations, DevOps teams can test in production and evaluate release versions side-by-side. This will help prevent costly rollbacks or staging while still releasing a quality, secure product. The old way of doing things is not sufficient, but fortunately, there is a better way.

    — Frank Huerta