devopsy.kinsta.cloud

Author: Alan Shimel

  • DevOps Unbound EP 19 – How AI and ML are being used in DevOps Today – TechStrong TV

    DevOps Unbound EP 19 – How AI and ML are being used in DevOps Today – TechStrong TV

    Alan Shimel: Hey, everyone. I’m Alan Shimel, CEO of MediaOps, devopsy.kinsta.cloud, Container Journal, Security Boulevard, and you’re watching DevOps Unbound. DevOps Unbound is sponsored by our friends at Tricentis. So many thanks to them. And DevOps Unbound is a biweekly show where we cover topics of interest to the DevOps audience. I am the host. My co-host is our CTO and CEO of accelerated strategies group, my friend Mitchell Ashley. Mitchell, welcome.

    Mitchell Ashley: Thank you. Good to be here as always, and with this illustrious panel.

    Alan Shimel: Absolutely. And an illustrious panel, it is. We have two panel members joining Mitchell and I today, let me introduce you to both of them. They’re both great folks in their own and I’m gonna let them introduce themselves. Let’s start with our friend Judith Horowitz. Judith, welcome to DevOps Unbound.

    Judith Hurwitz: Thank you so much, Alan. It’s a pleasure and honor to be here.

    Alan Shimel: It’sour pleasure.

    Judith Hurwitz: So I’m Judith Hurwitz. I’m the co-author of 10 books. Have been in the industry for 30-plus years. Focus on everything from DevOps, security, manageability in cloud, hybrid cloud, and really looking at how you take technology to transform organizations. It’s a complex topic, not easy, but it’s what we’re in the middle of.

    Alan Shimel: Absolutely. Then last, but certainly not least, my friend Brian Dawson. Hey Brian, welcome.

    Brian Dawson: Hey, Alan. Thank you. Good to be on with you again. To tell the audience a bit about myself. As I’ve told you before, Alan, I’ve been in software development and delivery for about 30 years. I consider myself a technologist and even prior to focusing on DevOps, I’ve had a focus on optimizing software development and delivery.

    Excited to talk about AI. I kind of dabbled and dipped my toes in the space and my time that I spent at the company that is now PlayStation. Things have come a long way since, and during that time, I’ve spent the past 10 years of my career focused on identifying, applying and spreading DevOps practices. So I’m excited to discuss the two together here with this group.

    Alan Shimel: Absolutely.

    Participant: Brian, I’m not sure if you mentioned your present position with the Linux Foundation.

    Brian Dawson: I did not actually, because also this organization I’m at today has a big footprint in the AI and ML space. So today I’m with the Linux Foundation, I oversee our developer relations in ecosystem development. And of note within that is the LFAI & Data Foundation which houses a number of impactful products in this space projects. Excuse me.

    Participant: Thank you. I just thought the folks at LF would welcome that.

    [Crosstalk].

    Brian Dawson: Well, appreciate it. Thank you.

    Participant: Yep

    Alan Shimel: All right. So the topic for today’s DevOps Unbound is artificial intelligence and machine learning, how can they help improve performance of DevOps teams? In other words, what’s their role in the DevOps world? And let me preface our conversation by giving you sort of my view of it.

    DevOps for many people was all about automation. What can we automate? Let’s automate everything we can, automate, automate, automate. And obviously in discussions around automation, topics such as artificial intelligence and machine learning, you would tend to think help with automation. And therefore anything that helps with automation is good for DevOps.

    And I should also say that we lump AI and ML together, like their siamese twins joined at the hip, and they’re not necessarily conjoined. You can have AI without ML and you can have ML without AI, at least I think so. I’d be interested in your thoughts.

    But has it lived up to the hype? Will it ever live up to the hype? Was the hype unfounded? Was it unrealistic? What role has AI and/or ML to this point played on DevOps and what role will it play going forward? I think that’s our topic today. I mean, any one of you, if you want to kick it off with your own feelings or respond, go right ahead.

    Brian Dawson: Well, I’d like to jump in as a troublemaker early on and say –

    Alan Shimel: Okay.

    Brian Dawson: – I challenge Alan that DevOps is really about automation is kind of the basis for the tie. If we look at it, DevOps is really a set of cultural practices and tenants that align DevOps and other software delivery stakeholders around the shared objective of delivering quality software reliably, rapidly, and repeatedly.

    Now the reliably, rapidly and repeatedly component, absolutely lends itself to some of the benefits that we could mine from AI. But I actually think it’s a great opportunity if we don’t say DevOps equals automation equals AI, but rather we say DevOps equals a culture aligned around a shared objective and how can AI and ML help support those shared objectives.

    Judith Hurwitz: So I think you make a great point. Brian, I think one of the most important issues, and you mentioned culture. We have been, for what, 50 years through this development and operations perspective on how we get things done. And the promise of AI is, “Okay, can push a button and it will take care of everything.” And, and this promise has been there for, what, 20 years.

    The reality is, it’s just not that simple. There are definitely things that we are doing today, and we are seeing evolve that is definitely helping the developers and the operational professionals. For example, if you have repeatable functions that happen all the time, that for example, if you press a button and a certain task should happen, you can probably use a model and collecting massive amounts of data to do that automatically.

    nd that’s a very good use of AI, and will work today. They call it ML Ops or AI Ops, and there’s a good reason why that’s what we’re focused on today, is because you’re look for predictable patterns and predictable anomalies so you can avoid making stupid mistakes that, “Oh my God, why did I do that? I knew what the right answer was.”

    But AI is not a panacea. And I think if we look back even two, three years ago, you had companies that all were saying, “Okay, we have automated, we’ve put AI into DevOps and all you have to do is press button and all of your problems are over.” That’s just not reality.

    Alan Shimel: Yeah.

    Mitchell Ashley: Yeah, there’s the marketing AI and ML, and then there’s real AI and ML. Marketing, meaning everybody glams onto the term and uses it, meaning it’s a case statement or an if statement in people’s logic in their software. I started doing some work in AI in the 80s and 90s programming and Lisp prologue, and doing some corporate education in that area, some expert systems with triage, things like that.

    So dabbled in a little bit there. And then I was exposed to more recently, about five, six years ago, worked with a gentleman named Dr. Bernardo Huberman, who’s one of the experts in the field. And I asked him a me understand why is machine learning taken off as this sort of, he described it as a subset of AI. So you can know that was his model anyway.

    What he told me really made sense, and it kind of helps me understand where can we apply machine learning? And that being kind of the most popular part of AI. And he said, “Machine learning takes massive amounts of data.” And the fact that we have the cloud and all these applications, creating data, a lot of data exhaust. Now, people are going back and mining that data.

    So we have this whole, Judith, in your writing, in your books about data analysis and leading up to AI, machine learning is great for that, because you can have supervised or unsupervised algorithm.Supervised, meaning that’s a cat, that’s a cat, that’s not a cat, or unsupervised, which is gonna pour through the data and start to look for those patterns and trends and anomalies that you were talking about Judith.

    So when people talk about using AI or ML in our industry, I always think about, well, where’s there lots of data, and can we leverage it in some impactful way? So as I look at products or technologies that claim they’re doing that, that’s at least one sort of criteria to fair it out. Is it real there? Is there something real there, or is it more spin and fluff to help us? We all have case statements in our spot, right?

    Alan Shimel: Absolutely. Actually, Brian, it looked like you were gonna say something, I didn’t wanna jump on you.

    Brian Dawson: No, Alan. I mean, of course I have a lot to say, but nothing in particular, I just, since I’m off mute, I’ll say I absolutely agree with Mitch and Judith and underscore the points that they’ve made.

    Alan Shimel: Yep. You know what, I wanna take a moment though, and explore, Brian, you disagree DevOps isn’t about automation. And I get the whole cultural aspects of DevOps and all of that, but certainly automating as a way to be more efficient, to get more done faster, I think is part and parcel of the DevOps mindset.

    Brian Dawson: Yeah. And if I may and without waiting to see where you wanted to go with that, I’d say, no, it, it is absolutely important, but there’s a line Jez Humble would use in talking about CD. And frankly, I think the book that Dave Farley, Jez Humble and team, Road Around Continuous Delivery is undercelebrated and under referenced.

    And in talking about that, one of the things he said is continuous delivery. And I’m gonna say by extension, some aspects of DevOps don’t require any tools. I can do continuous delivery with a bash script.Catch is it’s not the most efficient in pursuit of effective collaboration of delivery velocity so you can iterate. And as we’ve talked about in this past, establish that control feedback loop that requires things like automation to achieve those goals.

    I just think sometimes when we talk about what we see going on with AI and ML, and in those terms, we overload a principle or practice with expectations that are core to it. And I worry a bit about that, so I wanna call it out. And sorry, Alan, I don’t know if that’s where you wanted to go with the commentary.

    Alan Shimel: No, no, no. I wanna go wherever you wanna go, Brian. It’s a collective. Judith, what about – I’m sorry, go ahead. Oh

    Judith Hurwitz: Oh, yeah. So,Alan, I think one of the key issues is how the whole area of continuous delivery. In previous generations, you would build an application and it would live pretty much as it was written, could be for 10 years. Today, applications are constantly having to be revised because customers change, partners change, the sources of data change.

    So, unless you have the ability to constantly update and change and modify, you lose out. One of the values of using automation and using AI and machine learning models with lots of data, is to support the ability to do this. You don’t have enough hands and enough brains to where problems may occur, because you’ve changed things.

    How many times have we seen problems occur because somebody has added a new service to their environment and somebody forgot to change a configuration file? Now that’s not something that you need a massive brain to be able to do, but people get busy and they forget to do simple things.

    Mitchell Ashley: I think, Judith, that’s really a good point, ’cause I wanted to ask you about this in the software creation process. Given things are so dynamic now and constantly changing, understanding just the environment, the infrastructure’s code all the way up through the application and how much all of that is changing, it seems like the stepping in for humans in certain conditions in that environment is a great application of AI.

    For someone to synthesize all the factors that might go into where a problem exists or what might be causing a problem, seems to be where some of those algorithms might be helpful. Do you agree with that? Has that been your experience of what people are thinking about for AI?

    Judith Hurwitz: Yeah, definitely. I think what we’re seeing is this is where sort of the human factor comes in. You set it up so that if the printers turned off, don’t send me an alarm fast emergency. There’s a problem with the printer. We all have gotten used to that. But when there are problems that you you’ve never seen before that you don’t have data on and the model doesn’t take into account, then the AI Ops or ML Ops then gives you a message, “There’s something strange going on. I don’t know how to fix it. What do you want do here?”

    And so over time, when that appears again, well now you had data, well, this occurred once before, maybe it was anomaly twice. But then when you have enough data and enough experience over time, then you build that into the model so the next time that occurs, you make a fix.

    On the other hand, you don’t want the system to say, “Oh, I know what that is, make a fix,” when it turns out, no, no, no, you don’t understand the context of, of what’s going on here. And just because there’s a correlation, doesn’t mean that there’s a cause for that.

    Mitchell Ashley: It’s good to know, ’cause it’s not always about automating a response to it. I know in the security world, we’re very cautious, we’re skeptical about those things happening, blocking, legitimate traffic and financial world. That’s a huge issue that we run into.

    So you pointed out a good scenario, whereas we see repetitive patterns over time and that can help machine learning algorithms now understand, “Okay, that’s what that pattern looks like,” so you can identify what it is next time instead of it’s just an anomaly.

    Brian Dawson: Yeah. It’s interesting Mitch that you say that, and going back to the original, kind of the starting topic about AI and ML, not really achieving, its its promise yet. And as you called out, getting data is an issue, it’s not necessarily applicable to every space. You need a level of determinism and predictable patterns that you can learn from and build on.

    And I have for a long time been excited that when we look at CI and CD, when we look at, Alan, the automated workflow of software development and delivery, or even if it’s not automated kind of the standard, there’s a couple of things that you do have. You’re doing builds, sometimes thousands of times a day, builds in deployment, getting that out into pro. You could be generating a ton of data.

    It’s also in its nature, you’re striving to achieve a level of consistency and repeatability in how you deliver software. And I’ll say I’m not an expert to the level. Some of the people on this episode are, but I get at a very sort of top level get really excited about the opportunity for AI and ML to help better CI and CD, and in line with the principles of DevOps.

    What I see it really helping with is reducing cognitive loads so developers can focus on coding, innovating and solving problems while helping ensure that quality and stability that’s difficult to maintain while you’re moving fast. So just thought I’d called out.

    And I am curious to see when vendors and others really start to do like what our friend Koski has done with Launchable, and really apply AI and ML to the left hand side or dev side of the process to reduce the load, insure quality and stability

    Alan Shimel: Guys, we’re all lumping AI and ML, AI and ML, AI, and ML. Does it have to be AI and ML? Isn’t it, in my experience, a lot of what we call AI and ML is a lot more on the ML than it is on the AI? So, is it fair to call it truly AI or is it really ML? And maybe ML is for today and AI is it tomorrow. I don’t know.

    Judith Hurwitz: So I think there’s definitely a problem of nomenclature here. What we’re really dealing with primarily now is models and modeling data and creating models from data. That’s the reality of today. I think that there is a lot of misuse of the term AI and there have been some absolutely wild predictions. I can’t remember the name of the computer scientist who predicted that he would be able to replicate the human brain with AI.

    Mitchell Ashley: Marvin Minsky treat.

    Judith Hurwitz: No, this was past Minsky.

    Mitchell Ashley: Oh, was it after? Okay.

    Judith Hurwitz: Yeah, no, I mean, this was in the last five years.

    Mitchell Ashley: Oh, okay.

    Judith Hurwitz: So, over time, you’re always gonna be dealing with models. You’re never gonna get rid of the models. AI is sort of the next – Models are a subset of what you eventually achieve with AI, but AI is really a concept that will probably take decades to evolve. And I think one of the problems we have right now is where you have vendors, and I’ve talked to hundreds of them over the years where they say, “We have an AI application.”

    It’s because it’s a hot buzzword and you can look back and in the history over the last 30 to 40 years and see whatever the hot topic is, all of the vendors say we do that. So I think that that’s one of the problems that businesses are facing right now. What is the difference? What does it really do? How does it make your company better? How do you use this technology to be prepared for change?

    Mitchell Ashley: I think that’s a good way to break it down too, because early days of AI were about emulating human thinking. We’re still long from – I’m not sure – All my intelligence is artificial, by the way, I acquired all of it. I don’t think I was born with any of it. But that was kinda where it started. And then AI became about expert systems largely, and then machine algorithms.

    Machine learning algorithms was really where I think most of the activity is today because of that prevalence of data. It seems like most organizations are faced with sort of one of three strategies, what do we look from our vendors and how they may use AI in a meaningful way, or machine learning in a meaningful way that’s gonna help my business.

    Do I build models myself from the data like you were talking about, Judith, to apply to like a financial analysis kind of situation, more of an expert system, or do I use machine learning algorithms in my own software to do interesting, valuable things that my software can do.

    And maybe you play in all three of those or a subset of those, but that seems to be the question I think most individuals or organizations are at. We’re not at a place where we can go build models, but we’re looking for these capabilities either in our own code or in third party products.

    Judith Hurwitz: So I heard an interesting story from one of my clients a few years ago, maybe five years ago. He had a client that was very gungho about AI. So he went out and he hired five data scientists, paid them a million dollars each, gave them their own space and left them alone. These are the smartest people on the planet. Came back in six months. Didn’t wanna bother them. They’re so smart.

    “All right. What have you found out? Have you written this application?” And they said, “Well, we’ve been discussing this for six months and we have determined what algorithm we’re going to use. The point is, that they worked in isolation. They thought they were the smartest people on the planet. They did not talk to people on the business side about what business problems they had.

    They didn’t talk to the people who understand corporate data. They didn’t ask them, “What data do you actually have? What data do you need?” They didn’t talk to people who knew the business strategy or the business processes that were in place or needed to change. So it really is a team sport.

    And I like Brian’s discussion in the beginning about culture, because it really is about sort of hybrid organizations where you have people, you have leaders that know a little bit about all of these areas, and then a team that’s brought together that can work across these areas.

    Mitchell Ashley: Sure.

    Brian Dawson: Judah, can I ask, do you have any recommendations for DevOps teams on how do we evaluate or investigate where ML is a solution? And I ask this based on observation, we’ll see if we agree, that oftentimes people are looking for a problem to apply the solution to, as opposed to looking for ML as a solution to a problem they’ve identified. I’m just curious if you [crosstalk].

    Judith Hurwitz: I violently agree with you.

    Brian Dawson: Okay.

    Judith Hurwitz: So I think, and a lot of times people get so enamored with a new technology that they look at it as a way to solve all problems.And for a DevOps team, I think we are finally getting to the stage where there’s really reality in DevOps. It’s not the developers who are saying, “That’s not my problems. The operations people have to make this work.”

    There’s really beginning to be this collaboration between development and operations. And this shift left is definitely becoming real. So that’s definitely true, but for these teams to be successful, they have to have a holistic view of, where is your business going? What do they actually need? Why have they come to you and say, “We gotta do something?”

    Is it just because it would be cool to do something and spend money,or is there a real rationale behind this that they need from? What’s the pain that’s out there that they can solve? So, they have to start with that. They have to start not only collaborating between developers and the operations team, but with the business leaders, with the people who understand all of the data, people who understand security. So, all of this comes together in very much a holistic pattern.

    Alan Shimel: Fair, fair. Hey guys, we’re way past halfway through here and I wanted to kind of turn our conversation to the future. We’ve had a discussion on sort of the history of AI and ML and lessons learned, et cetera. But when we look forward, Brian, you mentioned the Linux Foundation has a, I dunno if it’s a daughter foundation or a subgroup, dedicated to AI and technologies.

    What do we see when we look sort of near term future? Forget, yes, one day, we’ll mimic human brain patterns, who knows if we’ll be alive by that, but near term, what do you see? What is Linux Foundation planning for?

    Brian Dawson: Well, looking forward, I – And it’s funny, ’cause you brought me in and I started to think about the impossible dream of what’s gonna happen in the future. I will call out and will mention again, I said at the start, the Linux Foundation is a parent foundation of what we call a Linux Foundation open source project that hosts other project.

    So LFAI & Data, you can call it a sub-foundation of Linux Foundation because they host, I believe at this point, 12 graduated projects with about 20 to 30 projects in incubation. So we’re talking 30 to 50 projects under the LFAI & Data umbrella that are all working on various aspects of shared efforts, multiple, large commercial organizations, as well as standards bodies coming together to drive the future of of AI and ML.

    Brian Dawson: What I do see coming out of LF AI & Data Foundation in the short-term is standards and foundational implementations, i.e. moving sort of beyond the sort of rudimentary discovery around AI, building sort of packaged or gray box implementations that everybody agrees and collaborates on, which I think will help unlock the less initiated, less expert vendors to begin to deliver truly AI based differentiating capabilities.

    So, to put that in short, because there was a lot of words is, I think it’s about establishing a foundation for us to start to build on and accelerate our progress within the AI and data space applied to DevOps.

    Judith Hurwitz: And so, Brian, I think that you are 100 percent right. And that’s when commercialization really happens, is when we have those standards and when everybody agrees to use those foundational services. I think the challenges that we face are, how to get the commercial vendors who don’t necessarily want to –

    They want to give lip service to these standards, but they really want you to only use their version of the “standard” so that customers never leave. So, it’s that stickiness factor that they are looking for. So, I think it’s it’s a hard journey.

    Brian Dawson: Yeah. Well, I think, Judith, you actually nailed the reason the Linux Foundation exists, frankly. And if we look at Kubernetes as a model, it was hard. Press Google could have easily said, “We’re not going to hand this over to the Linux Foundation. We’re gonna dominate this space. We are going to be the modern cloud OS.”

    But they understood for it to gain traction, for it to grow and truly offer benefit industrywide, they had to hand it over to the Linux foundation, a grow manage. They had to bring in Microsoft. They had to bring in Amazon to play in that space. And I see LFAI & Data serving that same role, what I tend to call in some language unlocking innovation, or as our tagline is, decentralized innovation.

    And I would beg to say that that is, we didn’t call that out directly, but one of the challenges that we’re seeing in the AI and data space. If there’s not short-term monetization, we’re not gonna do it. And if there is, then we wanna own it. So can we create a impartial playing field for everybody to come in and innovate together, and then build commercial solutions off of that?

    Judith Hurwitz: Yeah, it’s the challenge. It really is. And I think Kubernetes is a great example. I think data in some ways is more complicated because data’s really the crunch –

    Brian Dawson: Yeah. Well, I know everybody wants to own it. Everybody wants to own the data.

    Alan Shimel: No doubt about it.

    Mitchell Ashley: There’s discussion about AI needs to become an engineering discipline as opposed to this sort of edge specialty. Doesn’t mean that necessarily is gonna be applying everywhere, but I think how we apply DevOps or, I’m sorry, AI to certain kinds of problems, it seems to me the two most ripe areas are that highly complex environments.

    So it’s more and more infrastructure, it’s automated, more things are gone to digital. How do you manage the infrastructure or manage the triage problem-solving? And then of course, I think that the other areas, people will continue to find niche areas where AI can be applied to gain competitive advantage in a certain domain or space.

    And it seems like that’s the trajectory we’re on for quite a while. I don’t think it’s ever gonna be AI takes over everything, and it’s the thing that replaces DevOps or whatever. But it seems to me kinda a tool, but – Sorry, go ahead, Judith.

    Judith Hurwitz: No, what I was gonna say is very interesting. If you look at healthcare, for example, if you look at the spectrum firm, I can automate certain DevOps functions that are repeatable, where I can identify patterns. And as I collect more data, I can automate more things.

    You have something like healthcare, which in terms of orders of magnitude of complexity are just huge. And what I’m seeing is a lot of the vendors who think that I’m going to tackle healthcare with AI and we’re gonna own the industry, a lot of them are getting out of the business because right now, it’s just too hard and it will be too hard for quite a while.

    Mitchell Ashley: It’s true of the medical industry in many ways. It’s a hard nut to crack.

    Brian Dawson: Yeah. If I may start to dream a little and chime in based on something you said Mitch, I do see that in the near term, we are at a point where we talk about modern software development and delivery, the rapid pace which we’re delivering change, we’re building on inventions and progress made over decades. Library reuse is really heavy.

    And we are getting to a point where as we build more complex systems, we have to figure out how can we outsource sort of maintenance and management for one of a better word to use it kind of grossly to solutions like ML? How can ML help us continue to improve, grow and build on what we’ve done, but manage the scale and complexity?

    And I think we’ll move from some standardization, foundational blocks will apply more ML and both the operations and development space to manage that complexity, to maintain stability. But then I also eventually see the next stage being now, how do we apply ML and surface it to a developer at the time of a commit?

    Here’s what the expected outcome of this changes to help provide guardrails to the developer, the end state, or I wouldn’t even say the end state. I think we’d agree existentially there’s never an end state here. That we’re at a point where we don’t even have explicit CICD pipelines. We can commit code to a repository, the language can be inferred.

    We can give queues or signals to infer where we want it deployed, how we want it deployed. And I actually see ML and to an extent, layers of AI, just figuring out what to do with code at rest. So if you flash forward in 10 years, and we’re truly applying these to technologies at scale in the cloud, I just change a line of code.

    That change automatically is delivered to production. And AI/ML is helping us do that. So that’s sort of my dream. I don’t hit compile, I don’t hit build, I don’t build out stages and workflows, I just change code, and that’s running in a system somewhere.

    Judith Hurwitz: So, don’t you also want to have the ability to say before you say, you look at what’s happening, and don’t you wanna sort of at least at this stage say, “Are you sure you wanna do that? Are you sure you wanna delete that file?”

    So until we get to the point, because you have to be able to trust that the system is smart enough to understand if I make this change, what cause will that unleash? Because we’re not dealing with the perfect world in DevOps by a long shot.

    Brian Dawson: Yeah, I do want that. I want protection. I want it to know where the vulnerabilities are and warn me. And I do think, Judith, that would be a prerequisite into this deploying code at rest, or truly applying dev AI.

    Alan Shimel: Fair enough. Hey guys, we are just about out of time. I think this was a great discussion. If there’s one thing I could take out of it though, is that the crystal ball remains cloudy in terms of how this is all gonna play out and we’re gonna have to wait for it to kind of come into focus as we – But it certainly will be an important part of software development, of operations going forward. And it’s gonna be an increasingly important part.

    Brian Dawson: [crosstalk]

    Mitchell Ashley: I think the crystal ball is in Magic 8-Ball, ask again later.

    Alan Shimel: That’s a good one, Mitch. But for now, we’re gonna call an end to this episode of DevOps Unbound. Again, thanks very much to Tricentis for their sponsorship. Thank you so much, Judith. Thank you so much, Brian, for appearing here and hope to see you on a future episode. Mitch, as always, great job, riding shotgun with me.

    Mitchell Ashley: Good to partner with you. Absolutely.

    Alan Shimel: Absolutely. But this is it for this episode of DevOps Unbound. This is Alan Shimel, have a great day, and we hope to see you soon on another DevOps Unbound, as well as don’t forget, every month we do a live round table open to you, our audience, with questions. So stay tuned for that as well. Take care, everyone. Bye-bye.

    [End of Audio]

  • Inclusivity & Diversity in the Workplace – Techstrong TV

    Inclusivity & Diversity in the Workplace – Techstrong TV

    Eamon will discuss ServiceNow’s new cross-functional organization – Center of Excellence for Accessibility – to make the Now Platform more inclusive for a wide range of diverse needs as well as why Eamon was chosen as the new VP & Global Head of Accessibility. The video and a transcript of the conversation are below.

    Moderator:                 This is Digital Anarchist.

     

    Alan Shimel:               Hey everyone. Welcome to another Techstrong TV segment. I have a first-time guest on here I want to introduce you to. His name is Eamon McErlean. I did it so good off camera, and I messed it up. Eamon, say it right for me.

     

    Eamon McErlean:      Eamon McErlean.

     

    Alan Shimel:               Eamon McErlean. Okay. Eamon, now that we’ve got your name, why don’t you give people a little bit of your background, your current position and a little bit of who you are?

     

    Eamon McErlean:      Absolutely. Thanks for having me, Alan. Appreciate it. Yep, Eamon McErlean. As you can probably tell from the name, born and raised in Northern Ireland, just outside Belfast. Been working in the IT industry now for about 25 years, overall, 26 years. Did seven years at Royal Caribbean Cruise Lines, based in Miami, after [crosstalk].

     

    Alan Shimel:               My neck of the woods.

     

    Eamon McErlean:      By your neck of the woods, exactly. Then ended up doing a startup with a couple of business partners, which lasted about four years. To supplement that startup, I did a lot of consulting work. And the last consulting gig I did was with Apple. And after six months, I ended up taking a full-time role at Apple. We moved over to the Bay Area.

     

    Did 7.5 years in Apple global retail under Ron Johnson and his team, which was phenomenal. And then up to about just over four years ago got an opportunity to move up here to Portland, Oregon to join Nike, to run Nike’s quality, automation, analytics. And then that transitioned into deployments and support with Nike’s phenomenal digital team.

     

    And ultimately, about the end of October, I decided to move on to where I’m at now, which is VP and global head of accessibility for ServiceNow. So currently, I’ve been in this role, like literally, this is my three-month anniversary.

     

    Alan Shimel:               Congratulations.

     

    Eamon McErlean:      That’s my history. Yep.

     

    Alan Shimel:               That’s fantastic. You’ve been blessed. You’ve worked with some great companies, some great brands. I live down here in South Florida. There’s a lot of cruise lines down here, but Royal Caribbean has, there’s a certain quality to the Royal Caribbean brands that may be. And I’m not bad mouthing any other cruise lines, please don’t yell at me, but there’s quality within the cruise industry.

     

    But then, obviously, going out to a place like Apple, you want to talk about quality, and Nike. Nike was doing digital transformations before it was cool. And they’ve led the way there too. Even I remember I was involved in the TechStars program. And there was a Nike TechStars, TechStars just for Nike at one point where it was helping to develop.

     

    They were looking for companies that were developing apps for the Nike technology. It was pretty cool stuff. But this role at ServiceNow is a little different, around accessibility. I make a habit of trying to talk to people who have these up-and-coming new roles, especially at an executive level.

     

    For instance, I have a friend I’ve spoken to a number of times, I think it’s at Twilio, where she’s chief of corporate responsibility. Doing good for the world, using the corporation’s assets and roles. Let’s talk about this accessibility, Eamon. In your mind talking to people, what’s really your job? What’s your aim here?

     

    Eamon McErlean:      I think my introduction to accessibility started at Nike working with the retail stores. I had the good fortune of working with many companies or many departments within Nike. But ultimately, we had an initiative going called a career experience. And that career experience enabled our retail store team members to come to WHQ help us test, help us validate new features, gets feedback of product owners.

     

    And we realized when we were doing that that we hadn’t brought in enough adapted users or people with disabilities to follow the path side of it. And when we went to the stores, honestly Alan, to see the challenges that some of our team members at Apple faced with our applications, we jumped on that and said, listen, we need to do better here, we need to improve accessibility.

     

    That was about six, seven years ago. I had limited knowledge of accessibility at that time. But it’s something that really appealed to me. I think when you see it firsthand, the impact that it can have to an end user, I think that empathy instantaneously is awoke. And from my perspective, I knew there was something that I wanted to go after.

     

    Two and a half, three years later, and my transition to Nike and Morgan with a few of the phenomenal team members that followed me came with me to go from Apple to Nike. We realized pretty quickly that we need to address accessibility of Nike. I got phenomenal support from the Nike leadership team, especially from a lady called Holly Newton.

     

    And it was then that we started to build the team around quality and accessibility and automation. But we have a dedicated team towards accessibility. And that grew. We were able to hire a director and build that team up. And literally, what happened about just over a year ago, I hit a certain age website, without going into too many details, and I realized this is something I want to do. This is something that I really want to dedicate myself to.

     

    Alan Shimel:               I know about those ages.

     

    Eamon McErlean:      And rather than doing something that not that I have to do, some work that we have to do, but this is something I truly wanted to do going forward. I was fortunate enough to go into the ServiceNow. The leadership team that I’ve met at ServiceNow and their passion for accessibility and how we want to drive it, which I can discuss, has been phenomenal. It has just progressed smoothly, thankfully, and fairy intuitive and lucky fashion for myself, I would say.

     

    Alan Shimel:               Absolutely. Let me dig in here a little bit. When you’re in a retail environment, an Apple store, or Nike store, accessibility is at a physical level. You’ve got to make sure your doors are wide enough. That there’s [inaudible] or special seating. That’s physical, but there’s also hearing issues, sometimes sight issues, communication issues.

     

    But in a retail environment, it’s had a very physical human to human interface level. Then there’s another accessibility and it is accessibility that I’ve been aware of, and I dealt with on a very periphery old matter. I was designing websites when Netscape was in beta. So going back to 94 maybe, 95 something like that. Shortly after that, was it the 508 accessibilities for screens?

     

    Eamon McErlean:      Right, yep.

     

    Alan Shimel:               Where you had to have, if people couldn’t see images, but it read the alt tag or certain navigational tools for hearing impaired, sight impaired, people who maybe couldn’t control a keyboard or mouse or what have you. And to me, that was machine to person accessibility. You had to have good user design so that a person can interact with that machine with that website. Different than physical in-person kind of stuff.

     

    We think of ServiceNow as a digital company. You don’t have ServiceNow stores in the mall. Not yet, anyway. So let’s talk about what’s the mission at ServiceNow. It’s a different mission than the Nike store or the Apple store.

     

    Eamon McErlean:      It is. Very much so. And it’s as important to make that distinguishment because ultimately, you’re right. When people think about accessibility, that’s the first thing they think of. Can the wheelchair go through the front door? Do we have enough space? Are the restrooms accessible?

     

    But when you go on and start working with a platform company, and as broad as we are, the platform of platforms, it’s a whole different world. And the way that we look at it, is we need to be able to create the optimal consumer journey for every single user independent on their disability. And we work through different companies, W3C, WCAG, the WCAG consortium and conformance that we like to adhere to.

     

    But ultimately, it’s all about working with the people with disabilities. They are the experts, they truly are. And when we start now going forward, we have a multi-tenant approach, meaning it’s not just going in there and fixing issues. Going in there and fixing issues isn’t going to get us to where we need to be. We have to be able to embed accessibility compliance as part of our software development lifecycle.

     

    Because if we don’t do that, we would never going to end up getting to where we want to be. There’s awareness factor, I’m sure you know about unconscious bias training as an example in the HR world. It’s mandatory for every employee. We’re pushing at ServiceNow to make accessibility training for everybody at ServiceNow, for every employee, so they understand the basic principles.

     

    How we communicate with our customers, we want to ensure that we’re going to create a customer hub, a customer awareness board so we really build that overall engagement. The wonderful thing about accessibility community as a whole, it’s the most sharing, giving community everywhere in the word. We’re not sharing it, we put all the work into this, we’re not giving it to you, it’s the opposite.

     

    It’s truly extremely collaborative community. But ultimately, everything that you mentioned, be it from visibility, and being able to use screen readers, cognitive and the way the pages are laid out. Even the color of the screens, it’s everything that we do going forward will have that accessibility mindset in mind.

     

    But one thing I will add sometimes, is when you create something to ensure accessibility compliance, sometimes it actually helps everybody, because it can actually be an ultimate goal for all users. So we never just like thinking, okay, we’re just taking it for regular users and taking it for working with people with disabilities, we want to look at it holistically.

     

    Alan Shimel:               Alright, I think that makes sense. That was the lesson I learned, Eamon, for a long time. I’ve been an Apple user for a while, and I think Windows has it too, they have certain accessibility features that are not turned on by default. And for a long time, I would not even open that folder to see what they were. Because I said thank God, I don’t have any disabilities, and I don’t need them.

     

    And then one day, probably by mistake, I found myself in there. And some of the things in those accessibility options, I found really helpful. I’m not ashamed to say it, and I’ve turned them on ever since, because I like them. Easier on the eyes, easier on the ears, sometimes as you said, you get a little older. I think there is a lesson there that a lot of accessibility options are actually mainstream as well.

     

    I want to talk about a different kind of disabilities, I’ll get your opinion on that. So we have a lot of folks with physical disabilities, whether it’s eye, vision, hearing, cognitive. Then we have a lot of folks who are on the spectrum, for instance, the autism spectrum, or have some other mental kind of, I guess that’s sort of cognitive.

     

    These principles we’re talking about get built into ServiceNow. But they also trickle and find their way into all the companies that are using ServiceNow at their organization. It’s not just confined to the however many 10s of 1000s of people ServiceNow has employed, it’s all the customers of ServiceNow.

     

    Eamon McErlean:      That’s exactly why I took this role, because as phenomenal a company as Nike is and the millions of users satisfied customers Nike has, but ServiceNow will be up to 17,000 employees in a short period of time, but all of their own customers and then their customers have that exponential growth impact that we will have overall.

     

    So basically, from the accessibility standpoint, you have visual, auditory, speech, cognitive breakdowns. And from a cognitive perspective, you were looking at more of a understanding of how things flow but making sure that we’re not making it too busy as well. You don’t want to have a list of 40 items for people to flip through. You want it to be as we talked about before, as intuitive as possible from a user flow.

     

    The platform itself is extremely comprehensive that guys can do. But you can make it as complex or you can make it as simple as you wish, depending on what you want to do and your persona, if you will. Moving forward, I’ve always thought this crazy thought about computers and software and UI, UX. And even as it relates to media and an ongoing pierce.

     

    Right now, we were looking at the screen. Everybody does between six, eight, ten, twelve hours a day. I’ve always thought in the crazy term, how can we make that screentime have a positive impact on people’s lives holistically. And you don’t want to make it in a gamified way, but we should be able to make it, and we’re smart enough to make it in a way that somehow it helps people and a little bit every day.

     

    Again, we know the negative impact that can have while having that much screen time. It could be as simple as like, man, you need to walk away right now, you need to take a break, or here’s, I’m going to play your favorite song while you meditate for three minutes, or it could be anything.

     

    But unless we do that soon, I think we are going on the bad road as it relates to constant screen time and not that personal time. That ties into the cognitive area as well, I really do. And certainly, I think we need to address some of them later.

     

    Alan Shimel:               I tell you my thoughts on that one, Eamon. Who does it affect the most? It’s the opposite ends of our population. It’s the young people. You got to pry their screens out of their cold dead stiff fingers. They don’t put those phones down. I had my boys home from college, we went out to dinner.

     

    And it’s like, hey, man, you are home for the weekend with your parents, we’re out to dinner, put your phone away and be present, be present, be here. There’s definitely a problem there. But I’m involved in another organization that’s looking at technology in seniors. You start getting a lot of people with dementia or memory issues or what have you.

     

    And they can get sucked into that screen. And it could just as easily be watching movies as it is reading things online or playing games. And the beauty of it is there’s actually good things you could do to help improve memory with computer programs and stuff. But doing that versus just mindlessness, it affects them. So it affects the older and the young more than those people in the middle. And those are too at-risk populations, if you will.

     

    Eamon McErlean:      Completely agree. And I think, unfortunately, social media hasn’t thought upon in the most positive light, let’s say. But I think we as a culture have an opportunity to address that. I think we need to address that; I really do. Because I’ve got college-aged kids myself.

     

    And I know what it’s like when it’s just that constant, I don’t want to say attraction, but pretty close to attraction to the screen time. We have to address that. But the more like COVID, is a perfect example, we’re not physically getting together anymore. We’re doing X amount of hours on Zoom on the screen a today, I think it becomes more and more relevant.

     

    Alan Shimel:               I’ll tell you something on that, Eamon, though. And this is my own sherry, I might be crazy. I think what’s going on with COVID and the amount of time we spent on Zooms and stuff like this has taught a lot of people a lesson. That this is not the equivalent of sitting down and having a beer with someone or drinking a coffee or just sitting and talking in person eye-to-eye.

     

    This is good if that’s all you got. But everyone I talk to is dying to get back out to conferences in-person, to interact with real people and see, do they have legs beneath that screen? Real interactions. At the beginning of COVID, everyone was like, oh thank God for Zoom. Look at Zoom stock, it was through the roof.

     

    Thank God For this whole virtual world and the metaverse and whatever else we were calling it. But I think people are fatigued from it. And they’ve learned that this is a poor substitute for real life, for real in-person.

     

    Eamon McErlean:      It’s just not the same, Alan. It’s not. It’s really not the same. And what concerns me is there is a certain age group, back to the age group, there’s a certain age group that when I hear our sons or other kids talking about my friends, I’m hanging out with my friends all the time. Well, they’re not actually physically hanging out with their friends.

     

    Alan Shimel:               No, they’re not.

     

    Eamon McErlean:      They’re not hanging out with their friends here.

     

    Alan Shimel:               That’s same goal with that, yeah.

     

    Eamon McErlean:      Unfortunately, a certain age group, the longer this goes on, it’s a skill set to being able to socialize when in an environment with physical people, if people do have concerns about that or anxious about that. The longer this goes on, the worse that gets overall as well, that anxiety.

     

    To me, I’ve always been a people person. I always love physically meeting people and standing and looking at them and being there. I think there’s health impacts across the board. And I think, to your point, there’s multiple ways we can address it. I just think we shouldn’t ignore it. That’s the biggest thing.

     

    Alan Shimel:               Agreed. Agreed, agreed, agreed. Anyway, hey Eamon, we’re probably way over time, I wasn’t paying attention. But for people who want to find out more about what ServiceNow is doing around accessibility and stuff like that, is there a place on their website that they can get information?

     

    Eamon McErlean:      Under servicenow.com, we should have an area for accessibility. If we don’t, please, through this communication here, I can forward all my contact information, @eamon [crosstalk]

     

    Alan Shimel:               We will try to put it in the notes. But that will always great man. And Eamon, look, this is really good work. As you said, you get to a certain point in life and just getting up to go to a job every day. If you have to, you have to but you die young. Doing something you’re passionate about that you think makes a difference in the world, it’s not working well, it’s following your passion. I’m happy to see you be able to follow your passion, man. It’s a great thing.

     

    Eamon McErlean:      I really, really appreciate that. The last thing about sharing my information, please do because the one thing I’ve learned a bit this role is you have to be humble. You have to understand we don’t have all the answers. The users with disabilities are the experts themselves, and we have to listen to them more than anybody else.

     

    So any advice, feedback, lessons learned, lesson sharing, we are always willing to take as much advice and feedback as possible. So I’ll throw that out there. And Alan, listen, thanks for visibility. We’re trying to do the right thing here.

     

    Alan Shimel:               We appreciate it.

     

    Eamon McErlean:      Cooperation like this help. We really do appreciate it.

     

    Alan Shimel:               Good, man. Eamon, best of luck. Come back and keep us posted.

     

    Eamon McErlean:      I will. Thanks guys.

     

    Alan Shimel:               Thank you my friend. We’re going to take a break here on Techstrong TV. We’re going to be right back with another guest shortly.

     

    [End of Audio]

     

  • Testing Challenges in 2022 – Techstrong TV

    Testing Challenges in 2022 – Techstrong TV

    Stephen and Alan discuss the biggest testing challenges in 2022, along with the tools and methods needed to solve them. The video and a transcript of the conversation are below.

    Recording:              This is Digital Anarchist.

     Alan Shimel:            Hey everyone. Welcome to another Techstrong TV interview here. My guest for this segment is Stephen Feloney of BlazeMeter. I think it’s BlazeMeter by Perforce. Is that correct Steve?

     

    Stephen Feloney:    That’s correct.  

     

    Alan Shimel:            Yep. And for those who aren’t familiar of course Blaze Meter was originally founded by a friend of mine Alon Girmonsky. Great story around one of the pioneers in open source testing and continuous testing. Blaze Meter was acquired I guess it had to be six years ago, maybe more, by at the time called CA or CA Technologies, Computer Associates. Then it always kind of operated kind of semi autonomously. And then I guess it was last year they found a great home with the folks at Perforce who have also been around a while. And Perforce is of course a leader and really expanded their capabilities from code repository to testing and so many other things. So it was a good match, Blaze Meter and Perforce. We’re happy to see it. Steven so I gave the Blaze Meter story but I can’t give the Stephen Feloney story. I need you to do it.

     

    Stephen Feloney:    Oh well we’ll need hours to give my story.

     

    Alan Shimel:            Give us the short version.

     

    Stephen Feloney:    So I’ve been in testing more or less for 20 some odd years. I came to CA must have been seven or eight years ago. And I was part of the team responsible for acquiring Blaze Meter into CA. And I always had plans for Blaze Meter. And as you mentioned it sort of ran autonomous inside of CA. And then CA was acquired by Broadcom and then once we were in Broadcom I was able to expand Blaze Meter to go beyond performance and we added a multitude of features, virtual services, API testing, API monitoring, grew all that. And now as you said we found a very good home with Perforce and that acquisition happened right at the end of October of last year.

     

    But I’ve been doing product management work for, I don’t know, about 15 – 16 years almost exclusively in the testing space, a little bit in the mobile space and a little bit in the monitoring space. And what’s interesting is moving around going from testing to mobile development and mobile testing and then into monitoring I now have all of that with Perfecto and Blaze Meter coming together at Perforce. So I have the monitoring. I have the mobile. I have – it’s all coming together. So everything I’ve learned over the years I can make it all come together and work in one spot.

     

    Alan Shimel:            So give us your best Dr. Evil and tell us isn’t it great when something comes together like that?

     

    Stephen Feloney:    Yeah. It’s fantastic and I hope to make one billion dollars.

     

    Alan Shimel:            Absolutely. Good stuff Steve. So look. Let’s first of all we’ve had the pleasure of talking to several of the Blaze Meter folks since the acquisition by Perforce. And it really does seem to be a good match. But what I wanted to kind of focus in on today is maybe the state of continuous testing if you will and then a little bit of what specifically at Blaze Meter what’s new. What’s coming down the pike that we can be on the lookout for here.

     

    Stephen Feloney:    Sure. Sure. So Blaze Meter historically has been a performance testing solution built upon open source as you mentioned and we were, we play two different roles. We work with centers of excellence who are doing large, massive load tests. And then we work in a shift left world with agile teams. And we were noticing struggles when it came to testing with the agile teams. So when you’re looking at continuous testing, continuous testing is well, a continuum. So it starts off with the development and the agile groups, works along sometimes into a full end to end testing and then continues into production.

     

    And so if we talk about that continuum we look down at the agile teams and we noticed that they were struggling. They weren’t testing as much as they should. And we interviewed dozens of different companies to try to understand what were their challenges with this. And there were a multitude of challenges. One is they didn’t like using multiple different tools to try to get all the testing done, whether it’s a functional test, a performance test, an API test, unit test, all different types of testing, all of these different tools. Some have to be downloaded onto our devices. Some are in the Cloud. They all have different UIs, reporting. That’s a challenge. And then another challenge was we don’t have time to get data. I mean think about it. When you’re testing you need data.

     

    Alan Shimel:            Yeah.

     

    Stephen Feloney:    But it wasn’t just data to run the test because why can’t you do that. Then I need data for the back end system and then I need data if I don’t have my full environments and I’m using mock services or virtual services I now need data for that. How do I get all that data in and make sure it’s all in sync? Because when I’m developing code I have a very limited time to develop that code and get it out the door and if I have to spend time trying to figure out get all this data and sync all this data together, that’s a challenge.

     

    And then well with that challenge as I mentioned I don’t have my full environment. So now I need virtual services. Where do I get those? Who is developing those? And then I have to figure out what data did they put in? So the mock services, the virtual services as well as the data, two big problems that are hindering people from getting their testing done. And then we have other issues around how do I generation my test. I just want to develop. I don’t want to spend my time creating all these tests. So those are challenges there and then on the flip side if I’m trying to do continuous testing and I’m trying to continue the testing into production what tests do I set up.

     

    How do I monitor? How do I test in production when I don’t know what I tested, when there’s not a full collaboration across the entire value stream? How do I know what to test in production? Who is creating those tests? Do you have your ops team and are they monitoring or testing the same things that I wanted to test or that I did test in preproduction? So those are quite a few of the challenges that people run into when they’re trying to do continuous testing and continuous testing at scale.

     

    Alan Shimel:            Absolutely and emphasis on that scale because really that’s where you start seeing the cracks. That’s where you start seeing – when you’re not at scale sometimes you could muscle it. Right? Or fake it even. But when you start hitting scale man, it’s like whatever your weakest points are they’re going to just come right to the forefront.

     

    Stephen Feloney:    That’s correct.

     

    Alan Shimel:            You’re doing to say oh my goodness. This doesn’t scale. Right?

     

    Stephen Feloney:    No. That’s right because you could have like one group, one team, one service that’s able to do something that’s special, unique and they could get that out the door in their own special way. But when you’re trying to scale that either that service or application has grown or you’re trying to scale it in an enterprise as you just said all the cracks show. The whole thing falls apart.

     

    Alan Shimel:            Agreed. All right. Let’s talk about what’s new at Blaze Meter then. What are we doing to help?

     

    Stephen Feloney:    Well surprisingly a lot of the challenges I just mentioned we’re able to solve just coincidental.

     

    Alan Shimel:            I’m shocked.

     

    Stephen Feloney:    I’m shocked as well. I mean I didn’t know this was coming. Yeah. So one of the things we actually just released on February 3rd was our Blaze Meter test data piece of Blaze Meter. And this combined the ability to whether it’s for performance testing, functional testing, API testing we can generate the data that you need. And that data that gets generated is not just to generate the data to drive a test. We’ll also generate the data if you’re using mock services or virtual services, generate the data for that as well and generate data for system undertests.

     

    And the idea is all that stays in sync. And we’ll generate the data at the time you want to run the test. So now you don’t have to worry about managing, maintaining the data sets. You don’t have to worry about that. We will generate the data on the fly and to run the tests and to ensure that all that data is in sync. That takes away a lot of the false positives that you’ve seen in the past where things are failing. Oh but it turns out it was the data in the virtual service or the system under test wasn’t set up the right way. And so all of that goes away and you don’t have to put any thought into it.

     

    Alan Shimel:            Crazy. It’s crazy that no one has thought of this before or kind of implemented this before when it seems rather elementary at some level. Like of course if you want to do this. But I had a conversation with the last guest here on Tech Strong TV and we spoke about the issue is maturation issue. Right? Until your devops processes, until your CICD processes are at such a level of maturity. Right, you don’t even run into this problem because you have problems that pop up before it that kind of take your focus away.

     

    Stephen Feloney:    Correct. Yeah. So in the past what’s happened is when you were handing it off to a testing team the testing team was used to the problem and they had time to get the data. They had time to set up the data. They had a gold copy of the data sitting around that they would use. They were familiar with it. But as you’re saying the CICD process, the shift left process happened the data problem, the virtualization problem is completely exacerbated. ‘Cause they don’t have the time. They don’t have the ability. They don’t have the skill set. They don’t have the desire to do all that. They want to get their code that they developed and they want to get that out the door as quickly as possible. And testing is sometimes seen as a hindrance. It’s a necessary evil – going back to Dr. Evil. A necessary evil if you will.

     

    And so anything that we can do to accelerate that and make it much easier for them that is the challenge. So yeah. You could say why wouldn’t someone have done this whole data thing. There are synthetic data generators. Right? There’s open source synthetic data generators. That’s fine. But that just generates the data in one spot. You still have to figure out how to take that, pull that out, put that here, put that there, put that there. And we’re taking care of all of that for you.

     

    Alan Shimel:            Got it. Very cool. Hey. For people wanting to get more information on that Stephen where can they go?

     

    Stephen Feloney:    They can go to blazemeter.com.

     

    Alan Shimel:            Just straight up right on the front page?

     

    Stephen Feloney:    Yeah. Just right on the front page. It’s our newest part of our release so you’ll see it there. You’ll see blogs. You’ll see help. We have multiple discussions on it. Yep.

     

    Alan Shimel:            Very cool. Hey. We’re coming up on time here. Anything else coming down the pike or you wanted to let our audience know about?

     

    Stephen Feloney:    Yeah. So there’s actually quite a few things that are coming down the pike. But one of the more interesting things that we are working on is one of the challenges that I had mentioned is generating tests. So they don’t want to spend the time to even generate the tests. And there’s two things that come with that. One is just generating your API tests. So we’re going to have coming soon an API test generator to help with that and that is combined with our well synthetic data gen. So those two are combined together to help generate that. The next thing that comes which is a natural evolution is you don’t just want positive tests. You need negative tests as well.

     

    And so that data gen is going to auto generate negative tests for you whether it’s negative data driving a test, putting negative data into your system test or having negative things happen with those mock services or those virtual services. So you want to delay a virtual service. You want to knock a virtual service so it doesn’t respond or just provide different types of data sets to see how well and how robust your system is. And once you do that the next step is all right. So I have negative testing. Then you combine the positive and negative together and now you have chaos testing. And so those are coming down as well.

     

    Alan Shimel:            Cool. Love it. Any – I don’t want to hold your feet to the fire, make you say anything you’re not supposed to say but what’s timeframes on those if you can talk about them?

     

    Stephen Feloney:    Well I’ll say this. By the end of the year you’ll see this. I don’t want to give solid timeframes but this year those will be coming out the door.

     

    Alan Shimel:            Ok. I think that’s fair Stephen. Hey. I want to thank you for coming on and updating, getting us up to speed here.

     

    Stephen Feloney:    Yeah. Thank you very much.

     

    Alan Shimel:            Continued success. We’re hoping maybe at some point we’ll see Blaze Meter at a real live conference.

     

    Stephen Feloney:    Oh I’m hoping to get to real live conference. As much as I enjoy maybe not traveling as much. But the interpersonal interaction.

     

    Alan Shimel:            Yeah, no. We all miss it.

     

    Stephen Feloney:    Yeah. I mean it’s a needed. It’s a must.

     

    Alan Shimel:            Absolutely man. Hey. Steve Feloney. BlazeMeter powered by Perforce here on Techstrong TV. We’re going to take a break and we’ll be right back with another guest.

     

    Stephen Feloney:    Thank you very much.

     

    Alan Shimel:            Thanks Steve.

     

     

    [End of Audio]

     

  • Service Level Objectives – Techstrong TV

    Service Level Objectives – Techstrong TV

    Service Level Objectives (SLOs) are being more widely discussed as they provide a clear path to delivering exceptional results by defining clear reliability goals. Kit provides updates on the community, use cases and best practices. The video and a transcript of the conversation are below.

    Recording:              This is Digital Anarchist.

     

    Alan Shimel:            Hey everyone. Welcome to another Techstrong TV segment. I’m really happy to have my friend Kit Merker all done up in his Seattle Kraken hat and his winter beard for the pacific northwest up there joining us today. Ken of course is the COO of Nobl9, a recent winner of the devops dozen awards as best new company on the devops space. Hey Kit. Welcome to Techstrong TV.

     

    Kit Merker:             Hey Alan. Great to see you.

     

    Shimel:            It’s good to see you again my friend. I’m thinking Kit we may actually see each other in person soon. I’m hoping.

     

    Merker:             Wouldn’t that be great? Hopefully somewhere sunny and I can get rid of the beard.

     

    Shimel:            Well I’ll tell you. I was just looking at airfare for Valencia, Spain for Cube Con in May. I don’t know if you guys are planning on being there for that one, if that’s like a CNCF thing you go to or what or – actually we have a road show coming up in Seattle at some point. I’m not sure when. We’ll talk about that.

     

    Merker:             Well in May I’m not sure about Cube Con but we will be having SLO Con and we’re doing it online again this year in May so that’s what I’m looking forward to.

     

    Shimel:            Yes. I think I saw that. What are the dates?

     

    Merker:             It’s May 9th through 12th. And that will be completely online and asynchronous and attend while you work just like last year. So we’ll have tons of speakers and all the fun from last year that we had and hopefully even bigger. The CFP is open right now so if you go to slowconf.com you can submit a talk. And it’s going to be exciting. And you guys covered the, were a media sponsor last year so hopefully we’ll have you back.

     

    Shimel:            Yes, we were.

     

    Merker:             Not to put you on the spot.

     

    Shimel:            I remember. We’ll talk about it.

     

    Merker:             All right.

     

    Shimel:            So that works out well because I think Cube Con if I’m not mistaken is like May 16th.

     

    Merker:             Yeah. We intentionally didn’t put it the same dates so.

     

    Shimel:            Yeah. So that will work. Cool man. Let’s talk a little Nobl9 though.  I guess we really shouldn’t. I’m assuming everyone knows who Nobl9 is. But I forget you’ve still only been around two, what, two, three years and at most. And not everyone may be familiar. Why don’t we back up and start there?

     

    Merker:             Sure.

     

    Shimel:            Tell people about Nobl9.

     

    Merker:             Of course. Yeah. So Nobl9 we make a service level objective platform, SLOs. And the whole idea of SLOs is about measuring your observability system and bringing new context, shared context so that you can make better decisions and better automation within your organization. So if you think about how you’re pulled in all these different directions, right? Some people on your team say we’ve got to ship features. Some people say if we don’t make it reliable and perfect then we’re going to hurt our reputation. If we don’t fix our tech deck we’re going to burn out all of our employees and we’ll have the great resignation to deal with. They’re all getting pulled in all these different directions.

     

    And what’s interesting about this is that we have actually more data about our software than we’ve ever had before. We have source control. We’ve got our AWS Cloud. We’ve got our Kubernetes cluster. We’ve got all these observability tools. We have so much data and yet we come to these different conclusions about what we should work on. And this is a big problem for so many organizations. So what Nobl9 is doing is we’re bringing some additional context. This is what’s really missing is people need to understand in an organization what are the risks to the system. What are the expectations that customers have and what are the tradeoffs we have to make in order to deliver the right kind of service to our customers?

     

    So what we’re doing with service level objectives in Nobl9 d is we collect all this observability data, we add this context and we drive automation and decisions so that whether you’re on call, you’re planning your future roadmap or trying to justify your investments you can see exactly what the impact is on the business and everybody can start pulling in the same direction. And that’s really what we’re doing. And like you said we’ve been around for a couple years. We now have a product in market. We both have kind of an entry level thing called Hydrogen, an enterprise class product that’s being used by some of the world’s largest enterprises and it’s very exciting and we’re having a great time.

     

    Shimel:            Very cool. Kit when we look over the evolution of the last couple years, right, ‘cause I remember with Nobl9, right, we first thought of it around well this is an SRE kind of thing. Right?

     

    Merker:             Yeah.

     

    Shimel:            And SREs early on were inherited a world of SLAs. Right? Where it was perform or a quick bullet to the back of the head will work.

     

    Merker:             Metaphorically speaking.

     

    Shimel:            But that wasn’t a very forgiving model or a model that would really kind of foster cooperation and team building and a lot of the principles behind devops, right, about how we get stuff done. It’s very hard to have a blameless postmortem when if you blew an SLA someone’s job is on the line. Right?

     

    Merker:             Well it’s really interesting because I think definitely in the sort of earlier I would say IT service model, right, where you have SLAs, especially if you have maybe outsource contracts. Right? SLAs became really a form of punishment. Right? One of the interesting things that I learned from sort of the Google SLA model is that there’s an assumption of reliability at Google, an assumption of infrastructure investment that’s not actually there in most enterprises. The SLOs that are prescribed by the SREs in the Google model are really trying to hold back over investment in infrastructure and this sort of gold plate infrastructure problem. Well most companies in the real world.

     

    Shimel:            It’s the other way.

     

    Merker:             It’s the other way around. Their issue is they can’t keep up with the reliability needs and the feature pressure is really the issue that they have. And so with this SLO model and being able to define them in fine grained ways across multiple data sources, being able to use it with automation. What changes is that you can start to put guardrails around these different conditions that people care about. You start to model the business. And this is really what we’ve got to get to. It’s about building the technology for digital business because all the businesses are digital now. Right?

     

    Shimel:            Right.

     

    Merker:             So you’re point about kind of like how you adopted sort of like devops method and bring this to the table, I mean really devops is trying to get better at building software, become more kind of diligent and efficient and make the right tradeoffs. It’s about ownership, right, a sense of responsibility but not punishment. And I think once people realize – actually I just had this customer from Flexera. We just did this testimonial video together. And he said it’s like once people realize that they weren’t building SLOs to point blame at them or punish them, that we were trying to do it to help them so that the people in the organization could see when they had problems, right, when they needed help.

     

    Then people really got open to this idea. This to me this is the game changer. If we can help organizations become more transparent, to be able to do more through APIs instead of meetings if you know what I mean. Right? Like using the data and the systems to help make these better decisions. This is what changes culture. A lot of people want to talk about reliability culture and devops culture and agile culture but that’s such a scary idea. The way I like to think about this and what we’ve talked about with our customers and community members is it’s about how you make decisions together. Right?

     

    And if you can make lots of decisions in a way that is supportive of the strategy, supportive of the business and also the realities of the pressures that we’re on as humans. And let’s not forget that everybody is going through a hard time and maybe not totally happy being on call and all these other things. Taking all that into account and making a decision together this is what becomes culture, not the other way around. Right? You don’t change culture first to get the decisions. You start making good decisions and then the culture emerges from that. And that’s I think a really important idea.

     

    Shimel:            I agree with you 100 percent. No doubt about it. And it is. But when we focus on devops being about delivering software and clearly that’s the goal. Right? Delivering software better, faster, higher quality. In order to do that you need just that whole culture thing. And some people say oh culture is BS. It’s not. Having a team that works together, having goals that are clearly enunciated, right, we’re all trying to do – you understand what the guy or gal next to you is doing. Right? And why they’re doing it and how you fit into it. That’s all part of that. All of those things help you get software better, faster, higher quality. And that’s where it is.

     

     What’s interesting about Nobl9 Kit from where I sit, right, because I sit – I mean beyond being on different sides of the continent I sit in a different chair than you. I get to look at the whole market without a horse in the race so to speak. You guys have done a really great job of defining and owning SLOs. Right? I think for a lot of people in the marketplace, right, SLOs aren’t necessarily synonymous with Nobl9. But you’ve probably done more to promote SLOs and their use and why they’re beneficial than anybody else in market.

     

    Merker:             Well first of all I appreciate that a lot. Personally I think it’s a reflection of the community. I think it’s a reflection of a nerve that we hit at some level. Because to your point, right, goals that are well enunciated and it’s funny to me because I think oftentimes we think of goals as things that are kind of squishy. Right? We set stretch goals – oh sorry. I thought I muted everything.

     

    Shimel:            It’s never all muted.

     

    Merker:             We’re all muted. We want to set goals. Sometimes they’re a little squishy is what I was saying. So it’s like maybe you have OKRs. Maybe you have stretch goals. Maybe you have some different perception of what the goal is for your service. And what SLOs are doing is turning it into pretty much the most precise type of goal you could imagine which is code, right? Like actually describing it in code and something that’s checked in where it’s the truth. There’s no arguing about what it is. Now we can change it, right? We can always change our minds about what the goal is. But at any given point in time there is a goal that’s described and there’s a set of actions associated with that goal. And to me this is exactly what is so attractive to people in the market about SLOs because they’re no longer guessing.

     

    They’re no longer sending each other Slacks and emails and writing documents and putting it in dashboards. They have a very precise way of describing what they want and expect from their service that takes in account the risks and tradeoffs and customer expectations that they have for the service and then allowing the team to engineer to that goal, allowing the business to understand the tradeoffs. Right? Hey. If you want the service to be faster we’re going to have to spend more to buy more computing power. Right? Hey. If you want these features we’re going to have to put these other reliability concerns on hold and vice versa. This is something that brings visibility to all the little engineering decisions that are made on a daily basis and then aggregates them up to the size of an entire enterprise ‘cause it’s birds eye view of how the enterprise functions and doesn’t limit them to a prescriptive way. ‘Cause I think this is one of the other big challenges.

     

    When I look at the observability space or anybody’s infrastructure space is trying to bring a prescriptive guidance. I think today’s world is about the right tool for the job. Right? You’ve got to fit inside of an ecosystem whether that’s what language you use, what CICD system you use, what package manager you use, what source control you use. All these things are choices that engineering teams have to make for themselves. And you can’t necessarily dictate that Team A and Team B are going to agree on that stack if you will. Right? You give them some flexibility. But what you can get them to agree on is the concept of the service level objective. Like what should my service provide to customers in terms of availability, latency, performance, throughput, whatever the characteristics are that deliver that end to end service.

     

    This is what becomes almost the contract because if I know that I’m using your service and you’re going to give me x number of latency over y number of nines then I can build my system to account for it. I can decide what my retry strategy is, what my caching strategy is. Right? I can design and engineer a system that takes into account what your service says it’s going to do. Well that level of clarity allows engineering teams to operate in a completely fundamentally different way because now they aren’t guessing and they’re not hoping and they’re not wishing. Right? They’re working toward a clearly articulated goal. And I think this is what’s so important. So now the whole idea of us promoting this concept. I mean we’re just like we’re in love with SLOs. I mean I don’t know how else to explain that. Like we’re just like losing our mind about how amazing it is.

     

    But it is really about the community. We now have literally thousands of people that are involved in the SLO, the SLO Conf community. We have the monthly meetups. The conference as I mentioned will be in May. And we’re expecting hopefully about 5,000 people to show up this year. Last year we had 2,200 registrations. So we’re pushing hard for this but based on what we’ve seen in the market and the companies that are literally deciding if they’re going to build or buy an SLO platform right now.

     

    There is a huge groundswell of interest in this concept to take their observability systems, their operation systems, reliability systems to the next level. And all the practices and tools and techniques and tricks everybody wants to share it and learn together which is I think the most exciting thing. And it fits in with so many other areas, incident response and chaos engineering and how we do monitoring and logs. All this stuff kind of fits into this whole story about SLOs. Even finops, right, even like cost management, all this stuff fits into this same story. So yeah. It’s kind of overwhelming to be honest how much the market seems to have appeared and materialized over the last couple of years with SLOs.

     

    Shimel:            Cool man. Hey Kit, we’re up on the clock here. So you mentioned SLO Con and SLOconf with an F dot com. Right?

     

    Merker:             That’s right.

     

    Shimel:            Right.

     

    Merker:             Yep.

     

    Shimel:            For people who want to maybe submit a talk or just find out more about attending this year’s virtual conference in early May. And then for people wanting to get more information on Nobl9 is it Nobl9 dot com or IO? I forget.

     

    Merker:             Nobl9.com. Yeah. N-O-B-L number 9, dot com.

     

    Shimel:            That’s it. Hey Kit, man. It’s great seeing you. I hope to see you in person sooner than later here. But keep doing what you’re doing man and congratulations on the devops dozen award as well.

     

    Merker:             Thank you so much Alan and hopefully we’ll catch a hockey game together.

     

    Shimel:            Come on down. Actually I don’t have Seattle Sounder tickets when they’re playing the Panthers but we’ll look into it. I’ll talk to you later man.

     

    Merker:             All right..

     

    Shimel:            Great seeing you.

     

    Merker:             All right. Take care Alan.

     

    Shimel:            All right. Bye bye.

     

    Merker:             Bye.

     

    [End of Audio]

  • Agile and DevOps for Kiosks

    Agile and DevOps for Kiosks

    Alan Shimel talks with Sam Zietz, the CEO of Grubbrr and Touchsuite, about how software for kiosks can use Agile and DevOps to keep up with a wide range of hardware profiles. The video is below followed by a transcript of the conversation.

    Announcer: This is Digital Anarchist.

    Alan Shimel: Hey, everyone. Welcome to another segment here on TechStrong TV. I’m really happy to be joined by a good friend of mine today. His name is Sam Zietz. Sam is the founder, CEO of GRUBBRR. You may not know the company but Sam’s going to tell you all about it. Hey, Sam. Welcome to TechStrong TV.

    Sam Zietz: Al, thank you for having me on. Excited to see you again.

    Alan Shimel: Yeah, it’s been – well, with COVID it’s been a long, strange trip. Sam, we’ve never had you on TechStrong TV though. We’ve never really talked about the company. Why don’t we start with that. Give people a little background on GRUBBRR and maybe a little of your own background.

    Sam Zietz: Sure. So GRUBBRR is the leader in self-ordering technology. When you talk about self-ordering technology you think kiosks, mobile ordering, online ordering. And in reality, this was going to naturally occur over the next 5, 15 years. That’s before COVID hit. COVID came about and acted as a huge accelerant. So if you think about how we used to go to a bank and there used to be a teller, that teller got replaced over a 5 to 15-year period of time by online banking and ATMs.

    We used to go to the airport, and in the airport we used to see a ticketing agent. That ticketing agent got replaced by a bank of kiosks or QR codes on your smart phone. Same thing was going to happen in many verticals like restaurants and hospitality and retail, and it’s going to be a combination of kiosk, mobile and online, and that online will be the company’s website or a third party delivery service like an Uber Eats, a Grubhub, or a Postmates. What ended up happening through is COVID, and now all of a sudden nobody wanted to interact with a cashier and cashiers didn’t want to interact with consumers either.

    This was all of a sudden a hazardous position and people didn’t want to shop any longer for that type of work. So that accelerated the move towards self-ordering technology. Then Joe Biden was elected president, and one of this four pillars was this $15.00 an hour minimum wage, and regardless of where you fall out politically, whether you think it will be successful or not, he has, and whether it’s passed or not the minimum wage has increased. And right now we’re in a labor shortage on that end of the market for a combination of factors, many of which is people don’t want to come back and do those types of jobs.

    And as a result, you’re seeing especially in restaurants that they can’t even open their doors many times because they don’t have the employees. So for all of these reasons we’ve accelerated the movement towards self-ordering technology.

    Alan Shimel: Yeah. I mean, Sam, this all falls under what we call digital transformation, right? And digital transformation, as you mentioned, was well underway before COVID, but we’ve probably accelerated four to seven years of digital transformation in the last year, year and a half. And the move towards automation, which is what this is, was again well under way. Quite frankly, when we look at should we automate everything – this is a question for academics – should we automate everything, should we automate only what should be automated or should we not automate everything for the sake of preserving jobs, and whether you believe in a Star Trek future where you get a basic, whatever they call it, wage where they send you money or not, quite frankly, if a machine can do it better they don’t take sick days, they don’t steal, purposely anyway.

    Hopefully not. They’re a better employee in some regards. So if there’s a machine that can do the job better, a futurist or technologist will say by all means the machine should do the job then because that frees up people to do hopefully higher-level work, more valuable work. Now are there some people that can’t do higher level, more valuable work and what do we do with them as a society – that’s probably a discussion for another day. But the fact is that today we have the technology, to quote an old TV show, to do this, and that’s really what you guys are doing there at GRUBBRR.

    Sam Zietz: Yeah. And historically this should’ve occurred years ago. It really was restaurant owners said, “Hey, you know what? We’ve been doing it the same way for the last 20 years,” and because labor was so cheap and because the minimum wage was so low, instead of investing in technology like almost every other industry has they said, “You know what? Let’s just throw cheap labor at it,” and that’s what occurred. COVID has exposed those businesses that had not invested in technology, and quite frankly rewarded those companies that had been proactive.

    They were extremely successful during the pandemic, the ones that had digital, they had an online presence, they had these self-ordering applications already in place. They were hitting record numbers. Everybody else either exposed or permanently shut down as a result. And by the way, these aren’t jobs that anybody wants.

    Even pre pandemic, the average turnover for a cashier in this country was 150 percent annually. I think Panera put out a stat before COVID that they were at 100 percent. And they were bragging that they were at 100 percent. So they were proud of the fact that every one of their employees –

    Alan Shimel: It was only 100 percent.

    Sam Zietz: Yeah. So these aren’t –

    Alan Shimel: There’s no doubt. Look, I think we’re both saying the same thing. This is the way it was going. We just had an accelerant take place. So let’s talk now about GRUBBRR. So you guys are filling that void by developing software solutions that allow for self-ordering. And whether people are ordering online via the internet for like a third-party delivery or in person, it’s still a software-empowered transaction. Like everything else, software is leading the world.

    In your case though, Sam, the challenge is that even with self-ordering it comes in a lot of form factors. It comes in a lot of shapes and sizes. And how do you make a software that is shape and size, form-factor independent? I imagine that’s got to be a challenge.

    Sam Zietz: Yeah. Well, we’ve got talented dev people, so from my perspective it’s easy. They do all the hard stuff.

    Alan Shimel: You didn’t write too many lines of code. I get it. But the challenge is there.

    Sam Zietz: We’re agnostic on the hardware side, so we work with all of the leading kiosk manufacturers too. They bring us in for the software. Why? Because nobody’s going to buy a kiosk by itself. A kiosk by itself doesn’t do anything. It’s the software that does it. Just like your Dell computer doesn’t do anything.

    It’s the Microsoft and the other applications that you put on it, that’s the reason you’re buying the computer. So it works the same way on the kiosk. We’re agnostic. So it’s funny, we built this beautiful restaurant of the future in our headquarters here that people can come in and they can see everything in a working environment, but then after they realize we want a kiosk or we want KDS or we want mobile, whatever it is, we take them next door into our showroom and there we have everything.

    In there they can decide do we want a wall mount, do we want a tabletop, do we want freestanding, do we want a 15-inch, a 21-inch, a 24-inch, a 32 – you know, whatever they want. Do we want it in a portable fashion where it’s truly mobile? All of those options are there, but the software at the end of the day, the core that’s running it, it’s the same. It’s the same software. And you’re right, that’s exactly what we spent our time building, was software regardless of form factor that will work the same.

    And the key to this is that somebody can operate it with no training. So you don’t need to – our competitors in the kiosk space, for the most part, fall into two categories. There are POS companies that turn their POS system sideways and say now it’s a kiosk. Well, that doesn’t work. Or they’re writing an app on an iPad, and that clearly doesn’t work either. So really the advantage in which you were talking about online and now going in store with a kiosk, one of the great things about that is the marketing team and the brand owners can now curate that user experience much like they do with the online experience.

    So you can control and provide that consumer the absolute best experience, where before you had a cashier and maybe you taught them at the end to say, hey, at the burger place, do you want fries with that? Okay, and more people buy fries because the cashier asked them if they wanted fries. However, if I’m ordering a salad the likelihood that I’m going to buy fries is pretty low. However, I might take a fruit cup, I might take a healthy treat.

    So being able to, again, pair offerings to the items that that consumer would like increases the average ticket and improves the experience for that consumer. You also eliminate all the errors, right? Because now you’re putting it in yourself. How many times have you gone and said, “Listen, I’d like a hamburger – I’ll have a cheeseburger with extra ketchup, no pickles,” so they go, “Got, it extra pickles, no ketchup.” You’re like, no.

    So there’s a whole host of reasons why that experience is better and could be faster, because maybe you could only afford to put one cashier in there, but you could put a whole slew of kiosks in there for a fraction of the price of one cashier. Well now you eliminate wait times and you can redeploy personnel into the actual prep of food and improving consumer experience.

    Alan Shimel: Yep. There’s another issue I wanted you to touch on, Sam, and that is really one of the inherent advantages of modern software and whether it’s an app, think of it as an app, or it’s just portable software that’s not particularly tied to – hardwired, if you will, into the form factor of hardware, the ability to update on the fly. Right? You change your menu items, you change your price, your running specials, or I mean really, really big functionality – we’re going to start accepting PayPal or Apple Pay or Google Pay or whatever the – we’re going to start taking bitcoin.

    I don’t care how you want to pay for it. But right? We need to update that. By having it all dependent on software, not burned into the silicon, we do that as well. How does GRUBBRR handle that? Like the whole update process?

    Sam Zietz: Yeah. So first off, we can update everything in real time as well as – so we can also be notified before even the client knows that there’s an issue with software. So maybe their internet went down because their provider – there was a problem, whatever. Many times we’re calling them and saying, “Hey, your internet’s out.” Oh, wow. Somebody was in the closet and unplugged the router or whatever and, boom, it’s back up.

    But you’re talking about an entire ecosystem. So because we have the ecosystem instead of a bunch of disparate pieces of technology in there, you’re absolutely right. You make one change. So price would be an example. If we want to change up a price it changes on a kiosk, it’ll change in the POS if they’re using a POS. It will also change on their digital signage, so the menu board above.

    Great. We want to change the price. Now let’s say you’re out of avocados. Boom. Now guacamole comes off the menu, so now it’s not even on the menu board if you don’t want it to be. If you run happy hour, great. At 5:00 we change the price, at 7:00 we change it back.

    There’s none of that whiteout or cardboard being take up with those changes. We’ve also integrated –

    Alan Shimel: [Inaudible comment]

    Sam Zietz: – AI and machine learning into it, so based on weather, time of year, time of day, order frequency, items will move up or down on the menu. Because we all go left to right. So what you want to do is you want to feature the most popular items or the items that that consumer’s most likely to order at that point of time to the left so that they can then make their decision quicker and move on in the ordering process.

    Alan Shimel: Excellent. Sam, we’re running low on time. I did want to mention, I know you guys recently announced a deal with Samsung, I believe, which is probably one of the largest – I’m going to assume one of the largest kiosk producers in the world – to be powered by GRUBBRR. Do you want to tell us a little bit?

    Sam Zietz: Yeah. So Samsung is obviously a very large company. I think they’re the sixth most recognizable brand in the world. They announced on June 30th that they’re entering the kiosk space. I would bet that in short order they will probably be the 800-pound gorilla in the kiosk space. What they’ve really done is they’ve democratized it, bringing that price standpoint down so it’s affordable to everybody.

    So you’re going to have a lower price. But traditionally kiosks were kind of hobbled together. They were a shell with a whole bunch of other pieces of technology kind of plugged in to make it look like it was one thing, but it was really five or six things. Samsung has truly built one slick unit that has it all built, which has allowed them to eliminate points of failure so the unit will be more successful. Obviously nobody builds screens as effectively as they do.

    In fact, they also have it so from touching it you can’t catch bacteria or COVID or any of that. But the exciting thing for us is, again, as great as a piece of software is, no one is going to buy a kiosk unless it does something, and that’s where we come in. We provide the true value proposition that powers it, and by buying the kiosk with our software you end up reducing OPEX or labor or at least solving for labor when you can’t even find it. More importantly, you increase your revenue.

    Typically, our clients see a 12 to 22 percent uplift in average ticket because people will buy more as a result of that curated experience, and we improve customer experience. So those are the value propositions there, but it’s exciting. When you partner with somebody like Samsung it’s not just here in the US. It’s worldwide. And they’ve got unbelievable reach.

    Alan Shimel: Oh, yeah. We’ve dealt with them. I mean, just another example, no pun and no irony intended, but software is eating the world. Hey, Sam, you know what we didn’t tell people? Where can get they go get more information about GRUBBRR?

    Sam Zietz: Our website’s probably the best place, GRUBBRR.com, G-R-U, double B, double R dot com. So, yeah.

    Alan Shimel: Excellent. Alright, man. Hey, say hello to the family. Best of luck with GRUBBRR. Maybe we’ll be ordering somewhere on some GRUBBRR software soon out there. Check it out. Full disclosure. I know Sam a long time, maybe 20 years. Happy to see him doing well with this and playing in the software game. Sam, be well, and we’ll speak to you soon. Come back on and keep us posted.

    Sam Zietz: Alright, Al. Thank you again for having me on.

    Alan Shimel: Alrighty. Sam Zietz from GRUBBRR here on TechStrong TV. We’re going to take a break. We’ll be right back with our next guest.

    [End of Audio]

  • Graph CDN for GraphQL APIs

    Graph CDN for GraphQL APIs

    Alan Shimel talks with Max Stoiber from GraphCDN, the CDN for GraphQL APIs. GraphCDN is now available to everyone with edge caching, query analytics, error tracking and protection from malicious queries for your GraphQL APIs. A video is below followed by a transcript of the conversation.

    Announcer: This is Digital Anarchist.

    Alan Shimel: Hey, everyone. Thanks for joining us on another Tech Strong TV segment. My guest for this segment is Max Stoiber—and if I mispronounced that, Max, I apologize. I did my best, but why don’t you say it correctly?

    Max Stoiber: Stoiber, but it doesn’t really matter. I’m really excited to be here—thank you for having me.

    Shimel: Oh, it’s our pleasure to have you on. But you know what, it does matter—names matter, right? My mom taught me that and somewhere, your mom or dad or both, their name and—they deserve to have it pronounced right, as well.

    But anyway, Max, thanks for joining us. Maybe more important than your name is the name of your company, so why don’t you start off with sharing that with our audience?

    Stoiber: Absolutely. So, I am the Co-founder of GraphCDN, which we just recently announced is now publicly available for everybody. And so, that’s sort of the reason I’m here is to tell you a little bit about that journey and how we got here.

    Shimel: Absolutely. So, GraphCDN—I mean, we’ve seen so much news coming out of kinda graph databases and graph data, and it really gives us a new way of looking at, I guess it’s part of nature, right? Things get more complex and bigger as we go on, and so you need new technologies to help you get your arms wrapped around it. And certainly, graph technology is a real up and comer, if you will, in terms of helping us wrap our heads around stuff like this.

    So, you know, why don’t we start with a little bit of background? Like, what does the company do, what’s the problem you’re solving—that kind of thing.

    Stoiber: Absolutely. So, GraphCDN is a GraphQL CDN. And GraphQL is a technology, in case you’re not familiar with it, that was invented by Facebook back in, I want to say 2014, maybe ’15. It’s been a while. And they use it—it’s basically like an API framework that lets people build very strongly typed APIs to communicate between their servers and their clients. And Facebook invented that, and open sourced it. And a lot of other companies started adopting it, because it feels very nice to use for developers.

    And so, we provide, similar to how other CDNs provide features for websites to make them faster, more secure, think, like, Fastly cloud there, we do the same thing for GraphQL APIs. And we do that for everybody, not just Facebook who has technology like this internally, of course, we do it for all the other companies that aren’t Facebook and don’t really have the time to build all this technology themselves.

    And we came to that whole thing because when we built our last status, me and my Co-founder Tim, we were using GraphQL and we had huge scaling problems. We were growing like mad, but we couldn’t just cache our GraphQL API, for example, because none of the existing CDNs support GraphQL. They can’t cache any of the data that goes through it.

    And so, we struggled with this and we built terrible solutions internally at our past companies, trying to solve this as best as we can. And then now, we got together and we said, “Hey, we just gotta solve this,” right? It’s almost, excuse my language, stupid that nobody’s done this, because it’s such a clear need.

    And so, we sat down, and it turns out it’s quite hard to solve, which is why it took us a while, but we finally got there and now we’ve got GraphCDN, and it does a lot of things for people that have GraphQL APIs. That’s sort of the broad strokes overview, I would say.

    Shimel: Absolutely. Max, you know what, you mentioned a little bit here and there, but let’s talk about your background and your history.

    Stoiber: Absolutely. So, I am, I would say, most widely known in the engineering community for having invented a bunch of open source projects, particularly in the front-end web development space. About 1 to 1.5 percent of all websites on the planet are powered by technologies that I invented, mainly by styled-components, which is one of the biggest ones.

    And so, I’ve sort of amassed a large following of engineers and designers and other tech people following me for my open source work. And then, I’ve previously co-founded a startup with two friends from California where we were building sort of a modern forum, in a way, sort of a mixture between WhatsApp and traditional, you know, the forums of the past where we used to hang out and have friends.

    That was then acquired by GitHub, and turned into GitHub Discussions so that the engineers on GitHub can then sort of hang out. And then I had a stint at GatsbyJS, another startup, that was building a JavaScript framework, or is still building a JavaScript framework, and I left that at the beginning of this year to focus on GraphCDN and really build that out with Tim. And sort of, that’s my background.

    Shimel: Very cool, and congratulations. Very, very impressive.

    Stoiber: Thank you.

    Shimel: Thank you. You know, look, I’ve worked with enough engineers and software developers to know that nothing is ever truly done, right? There’s always more we could do, right? Why GA now, why kind of emerge from Stealth, if you will—you know, why now?

    Stoiber: That’s a great question. It’s funny, because we debated this for a long time internally, because we felt like we were GA ready two or three months ago, right? We sort of had the basic set of features where we thought we could cover common needs, almost all of the common needs that we wanted to cover. We were ready or people, but also, GraphCDN, because it is a CDN, is critical infrastructure. If we’re down, our customers are down. And we take that responsibility incredibly seriously.

    Now, we’re based on Fastly’s edge computing technology. So, the infrastructure underlying GraphCDN is actually very solid. But we wanted to make sure that our part of that infrastructure was as solid as it could be. And so, we gradually on boarded early access customers, including the biggest rugby site in Australia and a bunch of others, to make sure that our infrastructure could really handle the load. And after we proved that out for the last two, three months, we said, “Okay, we are confident in our systems, we’ve found all the bugs we’ll find before with GA.” Of course, you can never find all of them. That would be—I think any engineer that says that is lying. But we found most of them, the ones that really mattered, and then we said, “Okay, it’s good enough, let’s have people sign up and really get this into people’s hands.”

    Shimel: Excellent. And let’s talk about the model for the company. I mean, with your open source background, I’m gonna assume, is there an open source graph?

    Stoiber: There actually isn’t.

    Shimel: No?

    Stoiber: The open source part of it is the GraphQL part, the framework that Facebook open sourced. But our infrastructure is fully proprietary. There’s actually—there’s sort of two schools of thought, right? You have companies like Sentry doing all of their core work in the open source and then providing hosting or other services on top. And I think that model can work quite well, but for us, because we’re critical infrastructure, we knew that we needed to be in people’s critical path and we knew we needed to be reliable, and that just takes a team, right? There’s no way that you can just run this on the side, you know, have it running in the background and not worry about it. It needs to be maintained. Somebody needs to sit there 24/7 on call and pay attention that this thing keeps running, just to make sure nothing goes wrong.

    And so, open source just didn’t feel like the right fit for that, because honestly, we’re gonna sit there 24/7 and make sure that our infrastructure stays up and doesn’t have any down time. And so, we did not go open source and instead have—we have a very generous fee plan, but beyond that, believe it or not, people pay us for the thing we provide. Which, of course, sounds ridiculous, because that’s what businesses do, but as an open source developer, that’s actually less natural to me than you might think. But it feels very nice, I have to say. We’re very appreciative of our customers for helping them scale and secure their GraphQL APIs. And so far, over the past months that we’ve had customers, everybody has been very happy that they have us and that’s very gratifying to see from our side, of course.

    Shimel: Agreed, agreed. When you say GA, I’m assuming GA, people can go buy and use this right now.

    Stoiber: Correct. Absolutely.

    Shimel: But is there a free trial, is there—

    Stoiber: Yes.

    Shimel: You know, because you know how software is consumed today, right? You gotta get it in people’s hands and deliver delight, right?

    Stoiber: Absolutely.

    Shimel: And if it’s good and they like it, they buy it.

    Stoiber: Absolutely. That’s the exact model we follow. We have a very, I would say, generous free plan. You can use us for free for up to, I think, 5,000,000 requests per month, which covers a lot of early stage startups, we’ve realized, maybe even more than we would’ve liked. It’s definitely a very generous free plan, is what we’ve learned, which is a good thing, because we wanna provide that value, particularly to those growing companies. And then eventually, when they hit the scale where they have more than 5,000,000 requests, it starts costing $10.00 per 1,000,000 requests. And so, it’s sort of, the pricing scales with your usage.

    In theory, you can throw as many requests as you want at us, since we’re based on Fastly and they handle a lot of traffic, I’m not very worried about not being able to handle scale. And we have—it was actually funny, after we GA’d and we announced that this is now available to everybody, of course, people signed up and then suddenly started deploying this thing to production and our traffic just 10X’d overnight, and we didn’t even notice. Like, we just looked at our graph, like, we woke up the next morning, we looked at our graphs, at our internal dashboards and we realized, “Hey, a site just enabled us overnight and 10X’d our traffic, and we didn’t even realize,” right? And they’re just—they were just like, “Yeah, this works, we’ll use it—perfectly fine.” And that just kind of keeps happening and it’s very, very interesting to see. I love it, I have to say.

    Shimel: You know, that’s one of the—so, I’ve been in the Internet business since ’96, ’95, and that truly is one of the remarkable things about the Internet is yeah, people, you know, if you put your shop out there, you put yourself out there, people come by. And they could be from anywhere, whether they’re from Australia, Japan, Korea, you know, Amsterdam, London, or next door, it makes no difference, they’re on the Internet. And all of a sudden—boom, you know, it lights up and it’s like, “My goodness, you know? We were hoping for this, but who”—you know, you never know. That’s fantastic.

    So, what are plans going forward? Have you guys announced fundraising, is that in the works, or—you know, where’s the continuing journey, here?

    Stoiber: We raised a round of funding a couple months ago from some well-known industry people, mostly friends of ours, like Guillaume ________, the CEO of Versel; Tom Preston-Warner, one of the Founders of GitHub; Jason Warner, the current CTO of GitHub. A bunch of friends that we knew from the industry that we also knew loved GraphQL and that they knew what it was about and they immediately got what we were working on.

    And so, we raised enough funding to sustain us for a couple years, and now we’re gonna use that funding to build a small team to help us maintain this, because right now, it’s just me and Tim—or has been just me and Tim for the past months, and now we’re starting to hire and on board the first people to help us really take this to the next level. Because it’s a lot of work, it turns out. [Laughter]

    Shimel: Yeah, I know, yeah.

    Stoiber: Building a great product is a lot of work.

    Shimel: Well, you know what they say, Max, if it was easy, everyone would do it.

    Stoiber: Exactly right. [Laughter] Absolutely.

    Shimel: But it’s fun work, right? There are people who really work for a living versus people who love what they do.

    Stoiber: Yeah, absolutely.

    Shimel: You know what we didn’t do, though, Max, we never gave anyone, like, the URL, where could they get more information, where can they download it? Where can they go?

    Stoiber: The URL and the home page is at GraphCDN.io, and it just has—as any website, it has a massive sign up button that you can click, and then you’ll be redirected to GitHub and you can sign up and just try it out. You don’t even need a GraphQL API, we have this sort of flow where we’ll give you a demographical API, just so you can get a feel for what it does and how it works. So, yeah, go to GraphCDN.io and try it out.

    Shimel: That sounds like a plan. Hey, man, you gotta come back and keep us posted of progress here, okay?

    Stoiber: I will absolutely do that. Thank you so much for having me.

    Shimel: Oh, it’s our pleasure. Max Stoiber, Co-founder, GraphCDN, it’s at GraphCDN.io—right, Max?

    Stoiber: That’s it. That’s the one.

    Shimel: Yeah. Go check it out. Seems pretty cool. This is Alan Shimel for Tech Strong TV, we’re gonna take a break and we’ll be right back with our next guest.

    [End of Audio]

  • EP 24: Cloud Performance Testing

    EP 24: Cloud Performance Testing

    The purpose of cloud performance testing is to identify and eliminate performance bottlenecks that might occur within an application. Performance testing allows DevOps teams to check the speed, scalability and stability of the apps to ensure that everything is working as planned under the designated workload. In this episode of DevOps Unbound, Kristen Webb (Tricentis), Donald Lutz (Taos) and Andreas Grabner (Dynatrace) join hosts Alan Shimel and Mitch Ashley to discuss the key benefits of performance testing in the cloud and what to consider when building a cloud performance testing strategy. Join the experts to find out what you can do to ensure that your apps are running smoothly and remain available during critical times. The video is below followed by a transcript of the conversation.

    Alan Shimel: Hey, everyone. It’s Alan Shimel, CEO of Techstrong Group, and you’re watching another Devops Unbound. For those of you who may be new to our audience, Devops Unbound is a biweekly video series sponsored by our very good friends at Tricentis. And we explore a wide range of topics around devops and devops-related technologies, practices, processes and people. As I said, we do this biweekly. It’s every other week, for those who’re a little challenged, and then around once every month we also have a live – what we call roundtable format of Devops Unbound, where we’ll have usually a little bit of larger panel. But we have a really large panel in that we opened it up to a live audience, and you and everyone joining are invited to participate in the roundtable by interacting with the panel and our host, asking questions, giving us your thoughts, and it really becomes sort of a group-hall/town-hall kind of meeting. When it goes right, it’s great, with wide levels of participation. 

    Actually, this is a really good point today, good time to bring this up. Today’s panel version of Devops Unbound is on a holistic approach to performance testing. But the upcoming or the next live roundtable, and I’m gonna have more information on that later in the show, is later this month, on February 24th, which is a Thursday, and it’s gonna feature a performance testing in the cloud. I’ll give you the actual title in just a bit. But once of our panel members here, Kristen, is going to be there, as well as some other great guests. You can sign up for that on devopsy.kinsta.cloud. If go to Upcoming Webinars and check it out, it’s February 24th. And as I said, I’ll bring more information on that in a little bit.

    But for now, let’s talk about holistic approach to performance testing. And in doing so, I think the right place to start is with our audience. I’ve already kinda preannounced Kristen, so I’m gonna ask her to go first, if it’s okay. Hey, Kristen, welcome to Devops Unbound. Please introduce yourself to the audience.

    Kristen Webb: Sure. Thanks, Alan. Kristen Webb here. I’m the director of product marketing at Tricentis. Currently running the product line around neo-loader performance and load-testing solution. I have about 20-something years in technology. I’ve lost count. I’ve worked for a long time on the product-development side, so I remember back when we were testing large software releases at Macromedia and Adobe and trying to get our products out on time as part of a broader release and making sure there was no watermelons, what we used to call the performance bottlenecks at the end of our three-week mean time between failure testing. So performance testing is near and dear to my heart, as is all application development, background, and excited to be here.

    Alan Shimel: Absolutely. Macromedia, that’s a name I haven’t heard in so long. It used to be such a big part of my workplace. And I just totally blanked on it until you mentioned it. But it brought back good memories, so that’s a good thing. Yep, a smile to my face. Next up I wanna introduce you to Andreas. We call him Andy. It’s Grabner, yes, Andreas?

    Andy Grabner: It is, but feel free to pronounce it the way you like, as long as you call me Andy. I always make the joke if you call me Andreas, then I typically did something wrong or rude, because my mom calls me Andreas if I did something not appropriate.

    Alan Shimel: I get it. Our youngest son is Bradley, and everyone calls him Brad, and you say Bradley, and he kinda – he knows he’s in trouble. But, anyway, Andy, welcome. Introduce yourself to the audience, if you don’t mind.

    Andy Grabner: Yeah, thank you. So my first start in performance engineering was, I think, pretty much 20 or 21 years ago maybe now. I was working at Segway Software. I was a performance tester on a performance-testing tool called Performer, still around. But I remember those days, and it was a large load-testing project, and I – with my first kind of customer gig that I did. And I also remember that my performance engagement was scheduled for one week, took exactly ten minutes, because we broke the system with two virtual users, and the virtual user was my left finger and my right finger hitting the Refresh button on two different keyboards. And I really quickly learned that performance testing, while we always think about the magic 10,000 – millions of virtual users, sometimes it’s as simple as just hitting Refresh quite a lot on your website. 

    Over the years I’ve morphed from a performance engineer. Now I’m working at Dynatrace for the last 14 years, actually, in the observability space, and I’m a self-proclaimed devops activist. I really like the idea behind devops, and I try to use channels like yours and my own channels to really advocate for what devops allows organizations to do in terms of tooling transformation, people transformation, process transformation. And I’m very glad to be here.

    Alan Shimel: Excellent. Thank you, and welcome, Andy. Thanks for being here. Our third panel member today is someone I’ve had the pleasure of knowing – I don’t know, Donald. It’s gotta be 20 years, 19 years.

    Donald Lutz: It’s 20 years. Yeah, it’s too long.

    Alan Shimel: Twenty years. On our original team at Still Secure, what became Still Secure, with Mitchell and I – but he is a developer-testing person extraordinaire. I’ll let him introduce himself. I don’t want to embarrass him, our friend, Donald Lutz. Hey, Donald. Welcome.

    Donald Lutz: Welcome. My name’s Donald Lutz. I’ve been doing software architecture for probably about 30 years. I’ve been focused on how to do development right, how to use design patterns correctly. The past ten years I’ve really dropped in the cloud. Right now I’m the senior cloud and software architect at Taos. We are a public cloud company. IBM just acquired us. Sullivan rated us number one for public cloud. We also are on the Gartner Magic Quadrant. So I’m sort of spending a lotta time helping us productize our consulting offering. So the whole idea of testing and how to do performance testing right and chaos engineering, those are really near and dear to my heart. And how do you use the right design patterns in your code to avoid all these performance issues?

    Alan Shimel: Well, an ounce of prevention is worth a pound of cure, as they say, right? And we’ll get to it. Our last member, though, is my cohost and friend, Mitchell Ashley. I’m gonna let Mitch introduce himself, though. Mitch, take it.

    Mitch Ashley: Oh, good to be here with everybody. Thanks for making me follow some really pretty tough people to follow. Oh, my gosh. And, by the way, Macromedia, Dreamweaver, Authorware, remember all that good stuff? And I’ve known Donald – tack on another ten years. So it’s a small world. No, I’m CTO with Techstrong Group working with Alan, and I also am principal with our analyst firm, Techstrong Research. So, great to be here, and I think we could fill up our whole session with war stories about performance testing, Andy. Maybe we’ll save that for another day, right? But good to have everybody here, and thanks to the audience for joining us.

    Alan Shimel: Thank you, Mitchell. Just real quickly, I think back to performance testing at Still Secure, Mitch, if you remember. It broke our budget, ’cause we didn’t have all the virtual – you weren’t able to do it from one machine then. I tried to go out to a place that had enough to do a load of –

    Mitch Ashley: You kept Dell busy.

    Alan Shimel: – Jesus. Man, was that crazy. Anyway, though, let’s jump in. In my mind, one of the most overused words I hear in tech is the word holistic. So let’s jump on that. When we talk about a holistic approach to performance testing, is it really holistic, or are we talking about, “Hey, what’s a great approach to performance testing?” How should we be doing performance testing?

    Kristen Webb: I can take that to start. So building on what Andy was saying earlier about his two fingers being the load test to start, I think what we mean by holistic or just in – the sound approach to performance testing is starting with the code, starting with the APIs, starting at one user and making sure before you check your code in or your API in, your micro-services in that you’re – does this code work for one user? 

    And then as you go through your development process and you start to build the components of your application and you integrate those components, you start to hit those components with maybe 100 or so users. And you’re getting feedback along the way so that by the time you have an application integrated within your whole system that mimics preproduction, by then you’re able to hit the load at hundreds of thousands, potentially. But you already had feedback along the way, feedback loops to say, “What’s performing? What’s not? What do I have to fix along the way so that by the time you get to the really high load, you have fixed a lot of the issues?” And what you’re fixing at the end is, “What do I need to tweak within my system to make sure this application or API is gonna perform?” So that’s one part of holistic, or we need to think of a better word, maybe. 

    The word I tend to use is standardization, is – you need to cover all your bases along the pipeline, but you also need to make sure you’re covering all your protocols and all your use cases. So having a standardized approach for everything from API to monolith like we just talked about, but also what’re all the protocols you’re covering from HTTP to JavaScript to SAP and Citrix? So having an approach that works for all the protocols and across all those use cases, whether it’s from the protocol perspective or even the browser, ’cause those are two different types of performance tests but and also one that’s –the third thing I think about in terms of standardization is the skill set for the people who are going to be doing the performance testing. 

    So you wanna be able to not only have your center of excellence be able to performance-test, but as you move to devops, you want to be able to have a more distributed model for who can participate in the performance-testing process and create more of a self-service approach so that a lot of times – if they can access a browser, they can run a performance test. They can share results. And that encompasses having the approaches both for performance testers who want to build within CI pipelines with an as-code approach or a CLI or for the groups that want to use more of a no-code approach with a GUI tool. So those are the three things that I would say mean – what we mean when we think about holistic.

    Alan Shimel: Love it. Thoughts on that, team, panel? Agree, disagree? You wanna add?

    Andy Grabner: I wanna add something. Again, I’m not a native speaker, so maybe the word holistic means also something different to me. But I wanna add one thing and then add a second thing. But the 1st thing, what you said, Kristen, is I think in the beginning you said developers need to understand actually how their code is doing with 1 user and 5 and 10 and 20 users. I think what we’re talking about here is people understand where are the breaking point and where are the scalability issues, because I think if you are building new code, however small or big it is, if you put it under different sets of load – I mean, you don’t have to go to the millions of users, but if you look at it at 10, 100 and 1,000, you immediately know where are your connectivity issues between your components and issues scaling the load, where things may break apart and where don’t they scale, because obviously we can’t throw a lot of hardware on a problem and we somehow get performance but at a certain cost. So I think it’s very important, as you said, from the beginning on make developers understand where are – where they may have some bad architectural decisions that will later on cost them a lot of money in order to scale this. I think that’s one thing.

    But for me, holistic also means not only looking at it in development and starting early, but for me holistic means I first need to understand what are my real performance goals. If I’m developing a new feature, a new app, I should sit down with whoever comes up with the idea and trying to get their thought on what do you expect from my software? How should it run under certain load? It’s like our nonfunctional requirements in production. And then really take this and then break it down into, what does this mean for me as a developer? To give an example, if we’re building a new mobile app, and let’s say there’s a feature to log in, how long do we expect the login to take? Let’s say a second. Okay, if it is a second under 10,000 users, what does this mean for me as the developer that implements 2 or 3 APIs in the backend? 

    So I think the term of a performance budget is something that I also hear and have heard over the years. So if you break down performance as we agree on with the business, if we have performance goals, and then we can break it down into more technical goals to individual teams. Then we, I think, take a holistic approach, because then everybody can really work on their component and really always make sure that their component is working and operating within the context of their performance budget that they have so that there’s a higher chance that in the end, when all the components are put together in production they’re actually meeting our business and our performance requirements in production. So I think these are kind of the two things, right, that I wanted to add.

    Donald Lutz: Yeah, I’d like to add one thing. The thing I think about a lot is ’cause I’ve been building a lotta micro-services over the past ten years, and thinking about synchronous versus asynchronous communication and how that plays out, because if you have too many APIs, you have to use things like circuit breakers. There’re a lot of things that can affect that. So if you start looking at things like Kafka, where do I split my workload so it’s asynchronous? How does that work? What patterns do we put in there? That makes a big difference, because most developers are always thinking synchronous, and I find that to be a real difficult issue, because you know what? We could put this in the background. When you look at Amazon, just ’cause you place an order doesn’t mean that actually went in and got billed against your card immediately. That whole synchronicity concept has to be embedded in the concept of performance.

    Andy Grabner: Yeah, true. Yeah, it’s also a nice decoupling of components, right? I mean, that’s the event-driven model, as you call it, right? You put a queue in the middle, but it’s basically an event-driven architecture and very important piece. And because the event-driven architecture also allows you to better test and also performance-test components in isolation and then also come up with beta capacity-planning models, right, because you know how many events fall out of the first component, when you put a certain load on it, it means, how do we need to size the queues or wherever it ends up, and then what do we need on the other end to work on the queues? I mean, that’s a very good point.

    Mitch Ashley: Maybe that’s why holistic is being overused, Alan. The topic has not only evolved. I think it’s grown, because I remember one of my first experiences working with Donald was on ISS and TP Gateway server, a third-party product, and he was new to me, and he knew how to tune it. We called it tuning then, right, but the load, the capacity, the scalability, and we’ve added onto that customer experience or user experience goals, metrics. How long does it take to log in, that kind of thing, with the experience to get back a response on a mobile device versus on a Web browser, whatever that might be? What’s our API performance goals to partners and third-party services, things like that? So I think that’s made the world a little bit more complex, maybe a lot more complex to deal with. But it also has brought us closer to what the end result is we’re trying to accomplish with whatever system or software is. Deliver this service or make this function happen for whatever it might be. Kinda help us also think across the system horizontally and vertically of what we’re trying to deliver.

    Alan Shimel: Yep. I think there’s another element, though, that I’m interested in your all thoughts on. When I think back to over my career how performance testing has changed, what’ve been the big factors? Well, I think as Andy referenced, early on, man, it was hardware-dependent, right, because, I mean, if you wanted to scale performance testing, you had to be able to test it on the hardware that was running. And that got expensive quickly, and not only expensive for the actual cost of the equipment but from a people perspective. Right? And then the virtualization of performance testing, wow, what a godsend. We can just have Andy here working with, as he said, his left and his right index finger, and that was enough to break his – the app he was testing or the software he was testing at the time.

    And then I think with devops and dev testing and so forth, the idea of automating our performance testing, right, being able to shift it left into – further left into our development process, right, making – ’cause it used to be you did all your coding, and that took one month of your three-month cycle, and now two months of that three-month cycle were your testing cycles, right? And, man, by shifting left, I really cut that out a bunch. All of these profound changes, right, in performance testing – as we sit here today, can we say that there really is a best practices, holistic or not, approach to performance testing? And where’s the people part of it, right? There’s always people, process and technology. What’s the people part of that? Kristen, would you mind kicking it off, or –

    Kristen Webb: So, I think there is some approaches that are best practices-ish, and one of the first steps is really understanding where your priorities are in your applications and where the risky areas are, because you can’t test everything. So you need to really prioritize what to test first and where the issues are most likely gonna come up. And, two, after you do that, it’s about establishing what your SLOs are, your service-level objections – objectives, sorry – and understanding, what do you need to hit within what level of virtual users hitting that, within what response times, and what you want that end user experience to be like. And building those SLOs within the pipeline so that when you’re running the test in a continuous manner, you’re able to say, “This didn’t hit the SLO when we ran it against XYZ load,” and you send that back, and you have actions based on that result to fail the build, for example, send a ticket to Jira so that it’s integrated within your already-existing development process is a good way to go about it. 

    And then also taking in production feedback is a good part of this, too, to make sure that you’re understanding what’s actually going on within production, and is – are my tests mimicking the real-world experience that my users are having under what load conditions and making sure you’re updating your Kubernetes environments to match that level of code. Do you need a new cluster based on that information that you’re getting from either APM tools like Dynatrace or other monitoring tools like Prometheus or Microsoft Monitoring. And bring that observability data back into the test definitions and updating them continuously to make sure you’re keeping up as things change, because in reality things do change. I mean, we have seasonality, or we have new marketing campaigns that may be a business that went digital. Now they need to keep up with their spikes in load. So you wanna be able to stay ahead of that instead of reacting.

    Alan Shimel: Agreed. Panel, thoughts?

    Donald Lutz: The telemetry idea has become – really the past two years I’ve been doing a lot of open telemetry so we can know what’s happening at every point. We put it in L. We put it in Datadog. We put it all over the place, but also the whole Kubernetes cluster thing, how do you monitor it correctly? What’s the right thing? There’re a lotta tools that I’ve used, but there’s – the whole Kubernetes things opens up a larger can of worms because of the way the control loop works in Kubernetes. So you have to kinda figure out how do you wanna deploy them? What do the environments look like? What’s the appropriate load-testing there? That’s why I’ve got into a lot of chaos engineering, breaking things in production, seeing what happens. I just turned this service off. It doesn’t talk to Slack anymore. Where’s all those messages going? Did we notice this not working? That whole idea I think is really valuable. Surprisingly, it makes developers very neurotic. They don’t like it a lot, because you’re basically saying – it’s sorta like doing improv, because we’re just gonna do it and turn stuff off, and they’re like, “Well, I think it works.” So it’s sort of a very strange response.

    Alan Shimel: It is. Andy, you’re at Dynatrace, right? This is kinda right up your alley there of what you guys do.

    Andy Grabner: Exactly, yeah. And so to add what you already said, both of you already said, on a very mature scale, I think of people at IC, they just go always straight into production and maybe using canaries or future flags and just deploy their new codes and then expose it to, let’s say, a small percentage of users. And I’m saying about the very mature, this is what we would love obviously more people to do, but the reality is just a small number of organizations can really do it. But it’s the ideal scenario. You deploy. Then you use your telemetry to figure out, did that code change now? Does it actually behave better or worse than what’s maybe currently out there?

    What I wanna add I think also what Kristen said. I think what we need to do is we need to get closer to the developers from the beginning. I know that’s different terms, and I’m just throwing one more term out there, performance-driven development, because I think we’ve taught engineers – and I’m an engineer by trade as well, right? We’ve taught them over the last 10, 20 years to do test-driven development, where you write your unit tests and functional tests to cover functionality. I think we should also educate them to do performance-driven development, which means you are – as you write your new micro-service that exposes five APIs, then write your API test that you can run once or can run continuously or with multiple virtual users to actually generate some load. And I think, too, they’ve become much more developer-friendly. Kristen, again, pointing to you, I know you have a great product where you actually allow developers to define their stuff as code, right? And code could be something like JavaScript, whatever it is. But I think as the tools have matured and we are making it easier and more natural for developers to also write these types of tests, I think we will also get more at option. I think that is very important, because we always talk about shifting left. It’s a great term, but shifting left in the end, if you think about it, if you take everything and shift everything left, we put all the burden on the developers. And I think this is nothing that is sustainable. Therefore, we need to make it as easy as possible also for developers to do all the things that we ask them to do. 

    So that means we need to give them tools that are natural to them in their habitat, which means in their ID, right? They need to be able to check in code, and as part of their commit process, they then will automatically not only get feedback on code coverage and functional coverage but also, “Hey, your code change now has slowed down your performance by 20 percent. Are you aware of this, or are you consuming 50 percent more memory? And, by the way, then bringing Donald, what you said also, learning from production – then you can correlate that with your production data and said, “Hey, your service runs in production with a certain amount of instances, and if you are making this code change, it means you have 50 percent more memory consumption. Are you aware of the additional costs?” So there’s a lot of things I think that we need to do, but it starts with really enabling developers just to do more in their natural kind of environment that they’re in, yeah.

    Kristen Webb: I’m gonna build on what you’re saying, Andy, because I wanna make sure I’m clear about what I mean about shifting left, and I do think it is bringing it to the developer but as a service. So I think of it as building out a self-service rather than putting the onus on the developer to build the performance testing within the pipeline. So if it’s automated in the pipeline that they’re operating within, then it’s really up to the performance engineer to build out – and the automation engineer to build out that testing platform for them so that really they’re just kind of functioning, like you said, within their IVEs and within their – whatever they’re using to check in. 

    And, really, they’re just getting the feedback once they do so that they know how to act in the moment about how their code’s performing for that one user to start. And then later, as they get further on, they get more information about an increase in load to a few hundred, maybe, once they have, like you said, a micro-service that might expose more APIs. And once you’re distributing your code like that, how is it going to impact other code, and how is other impact code and endpoints going to impact the new code that you introduce, so really kind of delivering that, and that’s what I was talking about earlier in terms of creating a self-service approach so that more people can participate in a process, but the performance engineers are still the masterminds behind building that.

    Mitch Ashley: I have a question for the panel. I’m curious. How do you think about this aspect of it? It may be for me a better word than holistic is systemic, and I’m thinking about how all parts fit together to contribute to the performance you’re delivering, as well as other things. It’s not just testing our app and our app code. It’s testing the whole stack, vertically and variations of that, different environments that it might run in. And a lot of it’s not our code. Donald was talking about Kubernetes, just as one great example. Is that part of what we mean by holistic as well?

    Donald Lutz: I think so. To me, one of the things I think about the holistic problem is that you get all these CICD pipelines built, but you don’t have a gitops process that keeps the platform aligned. You’re working on that, so fundamentally you don’t have the cluster. You’re not doing infrastructure as code where the cluster is effectively as you need. Therefore, when developers deploy code through their – runs all the tests, but fundamentally part of the cluster didn’t get – so I’ve been thinking a lot lately in my current position, how do we get across our whole organization, gitops deployed in all elements of our groups so we’re using that effectively with everything else we do, ’cause that is the one piece that is happening. Argo CD’s a great example. It works really well, but just getting that to happen and realizing you’re building a platform is also a thing that people need to think about.

    Alan Shimel: Agreed, agreed. You know what? I just wanted to throw another log on the fire. So, cloud. Game-changer? Makes no difference? Easier? Harder? Donald, you’re kinda the voice of the developer here, as well as IT. What do you think? And cloud, you work with a public cloud provider. What do you – 

    Donald Lutz: Yeah, I mean, public cloud is what we focused on fundamentally. That’s what we’re doing for all our customers, and cloud is simpler but more complex at the same time, ’cause they’re – fundamentally companies are used to having their various people within their data center. And when you move to the cloud, it’s upsetting. They have to figure out how to move all the workloads, but it’s also – one of the things that cloud gives that I’ve been thinking a lot about is if you can work with all three public clouds, you start increasing your reliability in everything, but how do you contain the costs so I can run this in GCP, this in Azure, this in AWS? Companies wanna go there, but they’re sort of always going, “How do I do that? Why do we do that? What do all these terms mean, and how do you sorta get the various executives to understand, ’cause we’re trying to sell them on sorta the advisory services. This is why you should do this. This is how you get your devops working. This is all these things. This is what you should do with open-shift.” It’s a really complicated soup, because they feel like it’s sort of like what happens to a lot of industry. They’ve outsourced it to all these people, and what’re all these people doing? Are they really making a Pinto, or are they making a fancy car? What are they doing?

    Kristen Webb: I mean, I’m curious, Donald, how you guys do performance testing for the public. But I will say that cloud is a game-changer in a lotta ways, because it’s not eliminating risk, right? It’s just moving it from one place to another. It’s shifting it from on-prem. to the cloud and also introducing new risk that – new considerations that didn’t have to be made before. And with auto-scaling systems, sometimes people assume that moving to cloud means that implicitly they don’t have to worry about performance anymore because of auto-scaling. 

    And the concern there is that when you’re in the cloud, you’re really bumping up against a lot more services. And you can also run into noisy neighbors and a lot of different scenarios that will introduce bottlenecks from distributed systems. I mean, you could have your SaaS application located in France, your hardware located in China and your middleware in a third location. And you need to make sure that a hybrid system like that is gonna perform, and it can be assumed that it would, just because it’s the cloud. And you really need to make sure that your – whether you’re migrating apps or re-platforming ’em that you have a baseline of performance tests that you know you’re measuring against to make sure that you don’t regress when you move to the cloud and that you’re able to test those again once  you do move there.

    Donald Lutz: And I have another thought I’d like to add to that, ’cause I’ve sorta done it. You run all your services in – pick it AWS, Azure or GCP. And then there’s the other thing where you say, “Well, I wanna run all my services in Kubernetes in all three public clouds. I don’t really wanna consume the services directly from the provider. There’s some new, cool APIs that’re out there, where I just wanna consume it in Kubernetes. So you’ve changed the idea that your hybrid is really Kubernetes, wherever it runs. So then you’re not really consuming it in the public cloud; you’re consuming the services through the cluster. So that also changes the whole discussion.

    Andy Grabner: Maybe what I wanna quickly add – and my first response to your question, Alan, was it doesn’t matter, because it – or it shouldn’t matter, the same with cloud native. When we talk about cloud native, cloud native doesn’t mean your cloud-native apps have to run in the cloud. Cloud native really means how you architect your apps, and they can essentially run anywhere, right? So it’s not where. Cloud native doesn’t define where it runs but how the system runs. And this is why I think, also to Kristen’s point, you don’t get, let’s say, performance for free, or you cannot sign up for performance as a service or for resiliency as a service just by checking a box on your favorite cloud provider, and then all of a sudden you have magic happening. But you still need to factor all of these decisions into your architectural decisions that you make, and you have to do the proper testing in the end. I also think that the cloud was great, because all of a sudden we got a lot of services that we could build upon and really focus on. It’s a business value that we could generate, because we didn’t have to invest time in, I don’t know, maintaining a database and certain stuff like this, because somebody took care of it. 

    So I think overall, the cloud was a great thing. From a testing perspective, to add one more thing – and I think also, Kristen, you said it again, if you have to make smart decisions where your software runs, because you always need to have in mind where your end users are, where your consuming services are, because that also means you need to test from those locations, because you may now have different latency. You may have different costs out of a certain data center, out of a certain through provider. I think these are some of the additional considerations, but in general for performance testing and scalability testing, in theory it shouldn’t matter whether my application that I’ve architecture well runs in one cloud, in three clouds or on premise. That’s kind of my answer. It shouldn’t.

    Mitch Ashley: Alan, too, I think one of the, I think, real game-changers about the cloud is just access to resources. And in the organization, when you do it internally, the two greatest bottlenecks are acquisition of hardware and firewall rules, to be straight about it. And you can kinda control your own environment, whether it be resources or security, whatever it might be. But that flexibility gives you more – not just more options. You can rethink how you can automate how you set up an environment, “Let’s configure it this way,” your permutations, or, “We’re thinking about changing it in some specific way,” and you can automate how you test that. So it seems that it gives you a lot more flexibility. Sometimes flexibility means complexity, too, but that to me was one of the biggest game-changers of moving to the cloud.

    Kristen Webb: Yeah, I mean, with hardware being the most expensive part of performance testing and – like you said, it’s difficult and time-consuming to go through IT to procure resources, to get them provisioned, to make sure you can access them at the right time and there’s no conflicts. So what you’re talking about, really, is more the app development, app testing within a cloud CI system, which is a whole other level, in addition to application deployment in the cloud. And that’s really exploding, too, and it does minimize – with neo-load SaaS, you can get up and start it and run a performance test within five minutes now, because it’s a fully managed service using AWS and Google, and we do that for you. So it is a game-changer in that way, too.

    But also, this is where I do disagree with Andy a little bit. Wherever your app is, it has to perform. It has to run well, but the cloud does introduce more risk, because – well, first of all, it’s shifting the risk from on prem. to cloud, but it’s also – you’re going to hit a lot more services now, particularly where you have a new architecture, where it’s a modernized architecture, and it could be a – what are they calling it, a microlift now? So more services are gonna be affecting you in that architecture. It could be really unrelated, and when you have that kind of a distributed system, you might not know where the bottleneck is coming from.

    Alan Shimel: I mean, I think the cloud does introduce more complexity, if you will, but there’s – I mean, look. The advantages of cloud I think far outweigh the disadvantages of cloud, right? I think that there’s chips on both sides of that scale. But you know what? I mentioned in the beginning that in addition to this prerecorded Devops Unbound episode February 24th – we do have the live roundtable for our audience. So I’d like to invite everyone that’s watching this, if you wanna learn more about what to consider in building a cloud performance-testing strategy, we do have this live roundtable. We have folks from Dynatrace and Tricentis and more joining us on February 24th. The roundtable, funny enough, is called “Strategies for Performance Testing in the Cloud,” and if you go to devopsy.kinsta.cloud under the webinar section, you can sign up for this webinar right there. We’re calling it a webinar. It’s a live roundtable. You will have a chance to have your questions answered, to speak to the panel members directly via chat, and help you with your strategy. So don’t miss that, February 24th, just a few weeks from now.

    We’re running low on time. I’d like to throw one other kinda question out at the three of you for our audience’s benefit, right? Right, okay, you convinced me. I need a holistic approach to performance testing. Other than listening to the three or the five of us on here today, can you point me to some good places, if I’m in the audience, where I can go find out more about this, where I can get smart quick? Donald, we let Kristen lead off everything, so if it’s okay, I’m gonna ask you to go first on this one.

    Donald Lutz: All right. Well, I mean, there’s a couple books I like that aren’t necessarily about performance testing, but one of them, it’s – I always forget the name. It’s over there, Building Micro-services by Sam Newman. It’s a really good book about – it’s a second edition came out. There’s another one that came out by the guys at – oh, God, I can’t think of where they were, but to me that’s a great reference, because it covers everything from culture to how you build your micro-services, how to performance-test it. I like a lotta written things. There’re various videos that’re out there. There’s a couple that Martin Fowler – that talk about this in particular. So those’re kinda the things off the top of my head.

    Alan Shimel: So I think Sam Newman, like Martin, were with ThoughtWorks, if I’m not mistaken.

    Donald Lutz: Yeah, he’s independent now. He went off on his own, and now he consults.

    Alan Shimel: Yeah, yeah, they did leave. I remember. Yep. Andy, how ’bout you?

    Andy Grabner: Yeah. So I would say, and this is a little commercial on what we do on the side of Dynatrace, we have a holistic approach to SLOs, to service-level objectives, with our CNCF Project Kaptn, spelled K-A-P-T-N. It’s a CNCF project, and what we do there, we are providing events-driven orchestration to actually embed performance engineering as part of your delivery process but also as part of your operational processes. And there’s a great integration also with Neutis, with neo-load, so that means we actually provide a framework on top of Kubernetes to enable developers to simply check in their code, and then Kaptn is orchestrating the deployment, the testing like triggering and neo-load test and then automatically evaluating your SLOs and then pushing it forward all the way into production but always evaluating SLOs, so always evaluating your performance criteria, your resiliency criteria, your business criteria. So if there’s one thing, check out Kaptn, our CNCF project, because I think it will definitely contribute towards having a more holistic approach to performance engineering.

    Alan Shimel: Thanks, Andy. Kristen, I saved you for last.

    Kristen Webb: Thanks. So a couple things. One is we have a performance advisory council that is run through Tricentis, but really it’s not about product as much as it’s about approaches, and it organizes all the performance experts from around the world, including – Andy, I think you participated in it a lot. And Henrik, who is gonna be joining me on the webinar later in the month, used to run those performance advisory councils, and he’s at Dynatrace now. So he’s a great resource as well.

    But we have a site for the performance advisory council, where you can see all the recordings of experts around the world telling their stories about what they found worked for them and what their learnings were and how they integrated with various development processes. In addition to that, on Tricentis.com/neoload there is a ton of information about the problems that neo-load is addressing and how we’re addressing ’em, the integrations we have across traditional tech stacks and the devops tech stack and how we work with Dynatrace to help shift right and bring that production feedback, along with all the monitoring tools for that observability we have. And we also have a trial that’s available to learn more.

    Alan Shimel: I love it. Hey, Kristen, thanks very much. Again, just to reemphasize, Kristen mentioned it, February 24th, “Strategies for Performance Testing in the Cloud” with some really great, smart people, including Kristen. Check it out on devopsy.kinsta.cloud. You can register, and you do need to register to attend that live roundtable, though. Hey, Mitchell, I’m gonna let you take it home.

    Mitch Ashley: Well, thanks, Alan. I was just reflecting and really absorbing what our panelists had to say today, and the thing I take away is I’m impressed with the sophistication of how we think about – and the sophistication of the people who work in testing now. Long gone are the days of testers are people who didn’t like to code, right, who were developers that didn’t like being developers. We’ve got some extremely brilliant thought leadership happening in testing, whether it be in the products that’re being created, people that’re trying to solve both complex problems and sometimes fun ones, too. So it was kind of a renaissance for testing time for me. It’s a great time to be a tester. It’s a great time to be a QA person and a software engineer working with test. So it’s a fantastic topic, and I hope people will check out the resources that everybody suggested.

    Alan Shimel: You’re right. It is. Is it an old Irish proverb, “May you live in interesting times,” or something like that? It’s great times to be a tester. Anyway, Donald, Andy, Kristen, thank you so much for being on our panel today. Mitchell, as always, thank you. To our audience, thank you. We hope you found this interesting, and if you did, again, I’ll beat the drum one more time: February 24th, “Strategies for Performance Testing in the Cloud.” Sign up for it. Don’t miss it. I think it’s gonna be a great one. But until then, this is Alan Shimel for Devops Unbound. Many thanks to our sponsor, Tricentis. Thanks to all of you for watching in, and we’ll see you soon on another Devops Unbound.

  • Data-Driven Insights & No-Code Automation – Techstrong TV

    Data-Driven Insights & No-Code Automation – Techstrong TV

    Nishant discusses the importance of data-led engineering leadership. Nishant shares several examples of enterprises that have significantly improved development productivity, speed, quality, and security by bringing together data-driven insights and no-code automation. The video and a transcript of the conversation are below.

    Recorded Voice:     This is Digital Anarchist.

     

    Alan Shimel:            Hey everyone. Welcome to another Techstrong TV segment. My guest for this segment is Nishant Doshi. Nishant is with a company called “Propelo.” Nishant welcome to Techstrong TV.

     

    Nishant Doshi:        Thank Alan, thank you for having us.

     

    Alan Shimel:            Oh it’s a pleasure to have you on. Nishant, you know let’s assume our audience doesn’t know Propelo, not familiar with you, so I think that’s the place to start. Why don’t you give us a little background of both your own personal history, as well as as the company.

     

    Nishant Doshi:        Absolutely Alan. So first of all thank you everyone for joining, really appreciate it. My name is Nishant Doshi. I am the CEO and founder of Propelo.

     

                                     Propelo is a Series A startup based in the valley. What we are doing is we’re building an engineering excellence platform. My own sort of background is I sort of straddle between entrepreneurship and, and engineering leadership. Prior to this I was a VP of Engineering at Palo Alto Networks where I essentially helped them with, with all of their cloud security product lines and, and saw a tremendous growth within four years, from like zero to all the way like three-digit run rates.

     

                                     So I’ve had a lot of experience sort of like actually scaling themes, scaling processes, integrating acquisitions and realized that as an engineering leader I was making a lot of decisions and those decisions were not really based or backed by data. They were more gut-driven and you know there was a lot of different bottlenecks, a lot of different challenges where we really needed that data. We needed some automation and there just wasn’t a great tool or a great product to do that.

     

                                     So that, that sort of led me to start Propelo and sort of like bring you know data-driven insights and, and actionable sort of automation to the engineering to the world and that’s, that’s our mission.

     

                                     A little bit about Propelo. We help a lot of different engineering teams of all sizes, shapes, and varieties across a lot of different verticals. We have Fortune 100 customers, Fortune 500 customers, we have startups, unicons. We have customers all over, you know all over the world.

     

                                     You know I see very consistent theme across all of these different companies and, and what we see is that you know all these themes need to sort of run faster. They need to sort of like shift faster, they need to you know deliver you know quality sort of products in order to compete in order to sort of like capture their own individual markets.

                                       

                                     You know when you have, when you have this challenge of shipping you know faster with great velocity one of the challenges that you sort of grow and, and is that you know you have multiple personas, you have multiple tools, you have different processes within the company, within an organization that sort of need to come together to be able to ship this software, to be able to build and release the software.

     

                                     And often this results in a lot of different bottlenecks and friction points and those bottlenecks sort of keep shifting and it becomes a really big challenge to identify where the bottlenecks are. Sort of like you know you’ve got to understand how do you automate some of those bottlenecks or solve for those bottlenecks? How do you bring automation? And that’s really sort of like you know the common theme that we’ve seen is the lack of intelligence, the lack of sort of actionable intelligence across these different teams.

     

                                     So that’s, that’s the mission we are on. We are helping a lot of different companies with that sort of like mission to sort of improve productivity of their engineering teams by optimizing you know everything from processes to their tools across the entire software lifecycle.

     

    Alan Shimel:            You know Nishant someone sitting here listening to this it sounds simple. It sounds you know very clear, very well enunciated. This is really a case of the devil is in the details though, right? I mean when we’re, when we’re talking about you know I mean it’s classic kind of Phoenix Project, the gold, the book, the goal before that, right? Because what happens is as you find them or clear a bottleneck or a constraint you, you very often expose the one that was behind it and then you know and it’s become it’s like a treasure map where you, you know you’re going around from one bottleneck and one resource constraint or one you know trouble spot to the next. And you know it, it’s, it’s also usually a kind of never-ending story, right, because there’s always something else to improve, something else to do better, something else that popped up, something that’s changed that now needs improvement, right? It’s almost like by the time you get done you know to the end you got to come back to the beginning, because things have changed perhaps. It’s a tough problem to tackle, you know it’s a tough challenge. Let’s talk about how Propelo does that though if you don’t mind.

     

    Nishant Doshi:        Yeah absolutely. So it is a tough challenge but something that we believe definitely you know it’s, it’s worth start taking a step in that journey and that’s why we call it an excellence solution. It’s never done, right, it’s never you know, it’s a continuous improvement process.

     

                                     Now the way sort of you know I sort of guiding principles, right, coming into this is we, we really build a solution around flexibility and adaptability. The reason is you know almost every organization and even within the organization we’ve seen that they operate in sort of slightly different ways. And as a solution if you have to sort of succeed and sort of moving the needle any solution it needs to be flexible and adaptable. Someone might be actually releasing code manually, someone might be actually using GitHub, someone might be using a CI and CD system like Jenkins. So deployment sort of like might be you know following in a different process, right? As a product you need to be able to sort of like have the ability to be flexible and adaptable and that, that’s sort of like you know one of the guiding principles we have.

     

                                     The other guiding principle we have it’s sort of like really talks to some of the, you know some of the challenges you spoke about is really being wholistic. You know I’ve seen companies some of the products take the stance like, “We’re just going to look at this one problem.” But as you accurately pointed out it’s a treasure map. So you need to look at different facets not only within the engineering organization, but also how other stakeholder teams sort of like in track with engineering, right? What is the handoffs? What is that sort of process look like? Are there any deficiencies there between security and engineering, between product and engineering?

     

                                     In fact we’ve seen like product teams that actually have great sort of like hygiene around requirements, acceptance criteria. We see that have a huge impact on engineering delivery and software delivery. So I think like there’s a lot of different facets.

     

                                     And our product the way we’ve sort of architected our product is being able to sort of like use our libraries and our flexible dashboards to actually you know look at different facets, all the way from you know DORA metrics, which is, which is pretty standard nowadays in terms of like measuring overall productivity, but also looking at how teams are collaborating. We’re looking at like how customer success and customer support is sort of flowing into engineering. We look at security, we look at quality. So we’re looking at all these different aspects, because that’s how you stitch a story together, that’s how you understand what’s going on. So it’s really combining that KPIs with those insights and sort of like making it very wholistic.

     

                                     The third thing that we’ve sort of like really architected into our product is actionability first. We really believe like you know insights are great, but you need to sort of take action. Ours sort of like you know learning very early on was that it’s very interesting that everyone sort of has talks a lot… When, when we talk about sort of automation in the you know the Ops world there’s a lot of great tools, there’s a lot of like automation around the CI/CD, but there’s a huge sort of scope of automation outside of the CI/CD, especially as it relates to handoffs between different teams, nudging, sort of like certain actions, certain workloads, with sort of like all the way from release checklist, to sort of nudging the ______, to kind of checking their PRs, to security, engineering, product security, intakes. There’s a lot of different sort of like workloads that need to be automated or, or you know or semi-automated if there’s a human intervention that’s needed.

     

                                     So I think like our view is that there are sort of three sort of fundamental things that are needed to solve this huge challenge, flexibility, sort of being wholistic, and actionability. By sort of putting these three elements together we’ve been able to sort of like work with our customers to, to move the needle and get them into that continuous data-driven sort of improvement process and being able to sort of see examples, a lot of different examples of, of success there.

     

    Alan Shimel:            I love it, excellent stuff. So it’s a new company Nishant. Give us kind of you know what’s, what’s going on new, what you know, what’s, what’s happening?

     

    Nishant Doshi:        Yeah, it’s a new company. We’ve, we’ve grown. Last year was a great year for us, we grew 350 percent year-over-year. So posted some record numbers. In terms of like you know what we’re seeing in the market is as I said you know what was surprising to us was the every company, you know it sounds cliché, but every company is becoming a software company and what we’re seeing is you know a huge demand across the board.

     

                                     Like so we have you know we recently helped a company you know get sort of like their sprints in order, like you know measuring sprint hygiene. We help companies sort of like you know with a customer success, customer support, through engineering journeys. We’ve helped companies with sort of like triaging their Jenkins failure so that they can shift faster.

     

                                     So what we’re seeing is common themes, but across different verticals, across different shapes and sizes of the company. And that’s something that, that sort of like we didn’t expect going in, into this, into this, like how, how widespread the demand is.

     

                                     Engineering leaders more than ever like looking at using data, leveraging data to make decisions. And actually what’s, what’s interesting is with COVID, with remote work what we’re seeing is you know engineers they want to like work in companies that are better organized. They want to work in companies where the processes are flowing smoothly, they’re more productive. They aren’t working on like _____ a lot. And they’re getting sort of like appreciation for the work they’re doing. And that’s something hard to sort of like if you don’t have you know an ______ or it’s hard to recognize that, it’s hard to sort of put that into action.

     

                                     One of the most popular features that we, we saw last year was a feature of ours called “Kudos” where you can sort of actually give kudos to, to a job well done backed by data. And that was sort of amazing and we’ve had customers telling us that it’s really helped them sort of retain talent to be able to sort of like actually increase employee happiness.

     

                                     So, so I think like we’re seeing, we’re seeing a sort of inflection point in how engineering is sort of actually, how engineering leadership – the art of and science of engineering leadership evolve and I think data is a way big part of that story.

     

    Alan Shimel:            Absolutely. Nishant we’re about out of time, but for people who want to get more information is it propelo.com or?

     

    Nishant Doshi:        It’s propelo.ai.

     

    Alan Shimel:            Dot AI.

     

    Nishant Doshi:        Yeah.

     

    Alan Shimel:            And that’s: P-R-O-P-E-L-O?

     

    Nishant Doshi:        That’s correct: P-R-O-P-E-L-O.A-I.

     

    Alan Shimel:            Okay. Then on the website what’s the best way to engage with you via the site?

     

    Nishant Doshi:        Yeah, so we have multiple ways you can engage with us. We have free trials, so there’s you know there’s action, an action button for that. You can also contact us at info@propelo.ai or e-mail, you know e-mail us at or e-mail me at nishant.propelo.ai. So there’s multiple ways to engage. And we do, you know we do work with companies of all shapes and sizes and, and being able to help them sort of actually increase employee happiness and, and be more productive.

     

    Alan Shimel:            You know in the age of great resignations that’s an important thing. Hey Nishant thank you for coming for coming on and telling us about Propelo. Best of luck with it and we’ll have you back on again and we’ll check in on you guys.

     

    Nishant Doshi:        Thanks a lot Alan and thank you.

     

    Alan Shimel:            All right. Nishant Doshi here from Propelo, that’s propelo.ai.

     

                                     We’re going to take a break on Techstrong TV. We’ll be right back.

     

    [End of Audio]

     

     

  • 1.65 Million Devices Enrolled – Techstrong TV

    1.65 Million Devices Enrolled – Techstrong TV

    Fleet is an open-source device management platform Mike founded alongside Zach Wasserman, co-creator of OSquery. The company recently raised a $5M seed round. In this interview, we’ll learn more about what Fleet does, how they got 1.65M devices enrolled in less than a year, and why they believe open source is the future. The video and a transcript of the conversation are below.

    Recorded Voice:     This is Digital Anarchist.

     

    Alan Shimel:            Hey everyone welcome to another segment here on Techstrong TV. I’ve got a first-time guest, new company, open-source project I want to introduce you to. Say hello to Michael McNeil. Michael is a founder of Fleet or FleetO for Osquery is some we know. Hey Mike welcome to Techstrong TV.

     

    Mike McNeil:          Hey Allan, thanks for having me.

     

    Alan Shimel:            My pleasure. So Mike I, I, you know I guess we should jump right into it. Before we even get into who is Mike McNeil, let’s talk about Fleet and, and I guess to talk about Fleet we probably need to start with Osquery and then you know that kind of gave rise to Fleet.

     

    Mike McNeil:          Yeah, for sure. So back in ’14 my cofounder, Zach, he and a guy named, Mike Arpaia and a small team at Facebook built this tool called “Osquery.” They were kind of trying to DIY their own security and they started realizing, “Hey it would be really great instead of running all these little scripts all the time, if we could have a standard way to talk to any computer, whether it’s a Mac, it’s Linux, Windows and it would be great if we could do it in SQL.” And that’s where Osquery came from. So it’s kind of a _____ [crosstalk] utility.

     

    Alan Shimel:            Yep, but I mean look we’re being modest, it’s had a fair amount of success, yes?

     

    Mike McNeil:          Yeah, it’s gotten pretty wide adoption at this stage.

     

    Alan Shimel:            I mean can you give us some numbers to just put it in context? Like how many downloads, users, what have you, whatever you want.

     

    Mike McNeil:          I’ve seen, so I’ve seen a wide number of stats on Osquery itself, but I can say it is used by a lot of the Fortune 100. It turns out the people that have this problem have a lot of computers, right, it’s a scale.

     

    Alan Shimel:            It’s a scaled – you know it’s a problem and it’s at scale.

     

                                     So now, all right so we’ve got Osquery, now let’s talk about Fleet.

     

    Mike McNeil:          So one of the problems with Osquery by itself is you have to go build a server to receive all the data from Osquery. So you can use Osquery to say, “Hey, Linux server” or “Hey, Windows laptop” or “Hey, Mac laptop,” “Hey code machine, tell me some information about you right now.” You can also schedule those queries, right, to run in the background on a certain cadence. But you got to have somewhere to collect that data and send it to where you want it to go. So Fleet is that, it’s an Osquery management server originally and lately it’s become a lot more than that.

     

    Alan Shimel:            Okay. Now we’re going to get into what that “lot more” in one second, but while we’re, while we’re doing foundational stuff, Mike, let’s talk about you. Give us a little bit of your background and how you got to be here.

     

    Mike McNeil:          Yeah, so let’s see a few years ago I started a open-source framework called “Sails.js,” kind of in the early days of no js back in 2012. Got mixed up in open-source that way. Published my first open-source modules. Then lo and behold in like 2013 Sails.js kind of took off. We started to build a community. Went through Y Combinator in 2015. That’s actually where I met Sid Sijbrandij from GitLab. That’s how – that was actually later on the same folks that ended up coming up with the idea of to build a company around Osquery. So, so we kind of got together and with Sid’s help and with and obviously with Zach’s help we build Fleet. We were able to take this kind of stem of a new open source idea. Apply the open-core business model and then run with it.

     

    Alan Shimel:            Excellent. So Sid’s a friend. I mean he’s been on our show many, many times over the years. You know he’s – well since they’ve gone public he’s gotten a little bit more busier I guess. I should mention he’s also an investor in Fleet, right?

     

    Mike McNeil:          That’s right.

     

    Alan Shimel:            Yep. Okay, so now let’s talk about you know what, what’s Fleet doing and I mean recently it’s, it’s doing more than it sort of original sort of scope if you will, right? We’ve, we’ve added to the mission. And, and Fleet is – excuse me, Sid isn’t the only investor, right? You’ve raised some more money at seed rounds, there’s all kinds of stuff going on. But you tell us Mike what’s happening.

     

    Mike McNeil:          Yeah. So we’re up to 22 people now. We just announced the seed rounds led by CRV. We have support from Mike Arpaia, so one of the other creators of Osquery, Nico Waisman, CSO at Lyft, and Jack Naglieri, the CEO of Panther Labs.

     

    Alan Shimel:            Very cool. What, what’s, what’s you know beyond the amount of money what’s the money being used for?

     

    Mike McNeil:          Yeah, yeah so when we started talking to users of Fleet early on and really trying to like understand what were the bigger problems, so what you’re trying to solve. We started to realize that there were these jobs that needed to be done, right, that kind of just were – they weren’t really satisfactory solutions for it yet out there. So a lot of people were using Fleet to kind of sniff out or like spyglass, right, to see like, “What’s going on on this machine right now?” Maybe it’s for incident response, maybe they’re just trying to kind of like diagnose a weird issue with a server. There’s all these different use cases where device data is useful. But one of the strongest is people needed peace of mind that their MDM is working, that their EDR tool’s working, and that the actual agents are – that they think are installed are actually there. So that’s been one of the biggest use cases we’ve seen driving Fleet so far in the adoption we’d seen last year.

     

    Alan Shimel:            Absolutely. What about the go-to-market though, Mike? Like talk to us. How, how you know I mean at some level open source you know once you have an open-source product project like this the go-to-market is, “Hey man give it a try and if you like it you know maybe you’ll want a commercial version or added features or whatever. But here’s our GitHub, you know repo, download it and you know ask the community if you a question.” But is that the deal with Fleet? Talk to us about that.

     

    Mike McNeil:          Yeah, so that’s how it all starts, right? And for a lot of users that’s going to be enough. You know we try to make sure that Fleet is enterprise ready. The free version is low. Like we don’t want anyone that needs to use the product to not be able to use the product there, but we are adding additional features and we’ve added some additional features last year around role-based access control that are, that are especially useful for a larger team to – or it makes sense for them to upgrade. That’s kind of been the, been the pay wall. The other thing is support, so if you need a little bit more support or an SLA that’s included with the paid subscription.

     

    Alan Shimel:            Got it. What about going forward are there other… So this sounds almost like a little bit of an open-core model, right, where you’re going to get an open version and then there will be you know premium features. What other premium features do you see perhaps coming down the pike here?

     

    Mike McNeil:          So a lot of options for that. What’s, what’s currently paid is the team’s feature. So if you want to have multiple, multiple different teams at your organization, like maybe you have one group that manages the laptops, the productivity end points, right, and you got another group managing your production servers, maybe you’ve got another group managing a few of the remaining core servers or maybe some laptops from an acquisition. That segmentation you can do in a Fleet Premium using the team’s feature.

     

                                     You can also do self-hosted auto updates. So if you want to use – you know a lot of folks are using Chef Ansible, something like that to deploy out the latest version of Osquery to all of their hosts. But if you are someone who would rather have that be taken care of automatically Fleet offers auto updates out-of-the-box, they come from fleetdm.com. If you want to host your own version of that and use the register yourself and kind of decide when the update happens that’s currently a paid feature of Fleet Premium as well.

     

    Alan Shimel:            Excellent, very cool stuff. Mike are there any metrics in terms of how many Fleet downloads users or anything? Like give us a sense of, of just how widespread this is now?

     

    Mike McNeil:          Yeah, so not everybody has usage analytics turned on. Obviously, you know some folks it makes sense not to do that, but between, between the data that we have and what we know about our customers, kind of a conservative estimate is 1.65 million devices enrolled.

     

    Alan Shimel:            Wow, that’s crazy, huh when you think about it from a yeah.

     

    Mike McNeil:          Yeah.

     

    Alan Shimel:            From just a project you guys started up. That’s great. You know we didn’t mention for people wanting information where, what’s the URL for the Fleet project?

     

    Mike McNeil:          Yeah, so it’s fleetdm.com and the D-M stands for “device management.”

     

    Alan Shimel:            So it’s fleet device management, fleetdm.com.

     

    Mike McNeil:          That’s right.

     

    Alan Shimel:            Excellent. And then you probably don’t know this, but what, for someone that wants the open-source version they get it from there as well or do they got to go to like a GitHub or some other repo?

     

    Mike McNeil:          Yeah, so there’s a, there’s a few ways to build Fleet. Look if you want to run it locally on your laptop that’s the easiest way to kind of see what it looks like and try it out real quick. We have an easy-to-use installer on the website, you can get going in like five minutes, you’ve just got to have Docker installed.

     

                                     And if you’re going to production we’re actually we just finished a Terraform in fig, so you can take that deploy straight to AWS quickly. We’re working on one for Google Cloud and Azure now.

     

    Alan Shimel:            Yep, that seems to be the trail. Hey man, Mike, I want to thank you for coming onto Techstrong TV and enlightening our audience about Fleet. You know what it’s an open-source world so these are the kinds of projects that may not be household names in everyone’s house, but 1.6 million devices will tell you otherwise, right? Keep up the great work and come back and visit us.

     

    Mike McNeil:          Thanks Alan, thanks for having me.

     

    Alan Shimel:            My pleasure. Mike McNeil here for Fleet, fleetdm.com if you want to check that out.

     

                                     We’ll be going to take a break and we’ll be right back.

     

    [End of Audio]

     

  • Drumroll, Please! The Winners of the 2021 DevOps Dozen² Awards Are …

    Drumroll, Please! The Winners of the 2021 DevOps Dozen² Awards Are …

    We are very proud to announce the winners of the DevOps Dozen² Awards for 2021. For the past seven years, the DevOps Dozen² Awards have recognized the stars of the DevOps community. Over the years, we have expanded the scope of the awards to honor a dozen DevOps tools and services as well as a dozen of the DevOps community’s best and brightest.

    Every year it gets harder to pick just one winner in each category. There are so many worthy contenders that even whittling down each category to six or seven finalists is difficult.

    Once the finalists are selected in each category, things become even more challenging. My fellow judges (Mitchell Ashley, Jayne Groll, Helen Beal and John Willis) and I try to look at the popular vote and use that to guide us. But we are also very familiar with the space and try to make these awards more than just a popularity contest.

    I know that might sound like a cliche, but it really is true—every one of the finalists in this year’s DevOps Dozen² Awards would have been a worthy recipient of the award in their category. Congratulations to all of them. You can see the complete list of finalists here.

    However, as they said in the movie Highlander, “There can be only one.” Without further ado, I am pleased to announce the 2021 DevOps Dozen² Awards honorees:

    DevOps Dozen Community Awards

    Best DevOps Industry Implementation

    City of Gainesville/Gainesville Regional Utilities

    Best DevOps Survey/Analysis/Research

    2021 State of DevOps Report

    Best DevOps Related Video Series (Video)

    DevOps Series (DevOps for IBM Z Media Center)

    Best DevOps Related Podcast of the Year (Audio Only)

    The POPcast, hosted by Dan “Pop” Papandrea

    Best DevOps Books / eBook of the Year

    A Beginner’s Guide to Observability by Splunk

    Top DevOps Evangelist

    Abel Wang

    Best DevOps Transformation (Non-Vendor)

    FedEx

    Best DevOps Virtual Event of the Year

    DevOps World 2021

    Most Innovative DevOps Open Source Project

    Prometheus

    DevOps Executive of the Year

    Shlomi Ben Haim, JFrog

    Best DevOps Presentation of the Year

    The Value of DevOps Transformation in the Digital Enterprise by Rosalind Radcliffe (IBM) and Mark Howell (Lloyds Banking Group)

    Best devopsy.kinsta.cloud Article of the Year

    Measuring Value Streams by Analyzing Flow Metrics by Jeff Keyes

    DevOps Dozen Tools and Services Awards

    Best End to End DevOps Tool/Service

    GitLab 14

    Best DevOps Repo/GitOps Tool/Service

    Red Hat OpenShift GitOps

    Best CI/CD Tool

    CircleCI

    Best Value Stream Management Tool

    CloudBees Platform

    Best Observability Solution

    Honeycomb

    Best DevSecOps Solution

    JFrog XRay

    Best Testing Service/Tool

    Applitools

    Best Kubernetes Platform/Service

    SUSE Rancher 2.6

    Best Cloud Native Security Solution/Service

    Traceable AI

    Best DevOps for Mainframe Solution

    Zowe

    Best DevOps for DataOps/Database Solution

    The BMC AMI SQL Performance for Db2® Solution

    Best New DevOps Tool/Service Provider

    Nobl9

    Winners will receive both digital and physical trophies and badges celebrating their achievements.

    Many thanks to our panel of judges for making the hard calls and thank you to everyone who nominated candidates and cast votes. Thank you for supporting devopsy.kinsta.cloud and the DevOps community.

    For more information, please visit the 2021 DevOps Dozen² Awards website and stay tuned for the 2022 DevOps Dozen Awards which will begin nominations in late August 2022.