devopsy.kinsta.cloud

Category: Features

  • DevOps Chat: Christian Beedgen, CTO and co-founder of Sumo Logic

    DevOps Chat: Christian Beedgen, CTO and co-founder of Sumo Logic

    Christian Beedgen has years and years of experience in collecting large amounts of data and making it useful to Dev, Ops and security teams. I recently had a chance to sit down with Christian and talk about what is driving innovation in the data analysis market and how Sumo Logic is responding. While our conversation is short, Christian gives some great insights that I think you will find useful.

    Below is the streaming audio of our conversation and below that is the transcript so you can follow along.

    Enjoy!

    Alan Shimel:                Hi. This is Alan Shimel, devopsy.kinsta.cloud, here for another DevOps Chat. I’m very happy to be joined by Christian Beedgen, CTO and Co-Founder of Sumo Logic. Christian, I didn’t butcher your name too bad I hope.

    Christian Beedgen:     No, it’s quite fine.

    Alan Shimel:                Welcome. Christian, we had set this up some time ago that I wanted to interview you because there’s so much going on with Sumo Logic, but I realize maybe some of our listeners may not be familiar with the company yet though. So before we dive into some details, can you give us kind of a high level, a 50,000 foot overview of Sumo and what you guys are doing there?

    Christian Beedgen:     Absolutely. Sumo is a machine data analytics company. That’s a highfalutin way of saying that we’re doing log management. We have been around since 2010. The big differentiator that we brought to the market, among other things on a high level, is that we actually offer this as a service. So there’s nothing to install. It’s all focused on actually using the product to make sense out of your logs, out of your machine data for troubleshooting and analysis of application logs and everything up and down the stack basically.

    Alan Shimel:                Excellent. So logging as a service. And you’re right; the idea is a very 50,000 foot view of it. Christian, the dev-ops market of course is exploding and it’s a very hot area for tools vendor. Sumo Logic in particular, though, is always sort of at or near the top in terms of awareness among the dev-ops community. What do you attribute that to?

    Christian Beedgen:     Well, I think we are in a very fortunate position of having good product market fit basically. I come from this space of log management, as does my co-founder. We might not know much, but we certainly know a bit about how to build products that help people managing logs.

    Logs and anything that has to do with monitoring really are at the very center of any kind of operational activity. So it’s not a surprise to me that dev-ops folks are excited about what Sumo is doing, because it’s part and parcel of what they have to be able to do and I think we support it really well, and we end up in their toolbox more often than not.

    Alan Shimel:                Excellent. Christian, sort of the new messaging or new functionality that the Sumo folks seem to be really spreading right now is the idea of unifying logs with metrics. Can you give our audience an idea; what do you mean by that?

    Christian Beedgen:     I’m actually extremely happy to be able to talk about that because that has been cooking for a long time, and as of April we are now able to talk about it publicly. This goes back all the way in my mind to 2010. I’ve been wanting to do this for a very long time.

    From my own observation, of course at Sumo we’re running a pretty large scale distributed system, so we have a lot of insight into just from of our own sort of dog-fooding as to what we need in order to run a large scale system and in talking to customers and observing what they actually need in order to make their applications run better and troubleshoot them better and get them back in line better.

    Logs are certainly a very important part of that, but the other side of the coin when it comes to machine data are metrics. So what are metrics? Metrics are essentially time series data. It usually comes from the hosts. It starts with CPU, I/O metrics, network metrics, and then goes up the stack all the way into the applications, you know, how many logins per second, what’s your API response times, and those kinds of things.

    While those can be potentially captured in logs, logs are not necessarily the most efficient way of processing metrics. But metrics are extremely important and I think they kind of go together with logs in order to give you kind of a full overview as to what your systems are doing.

    We know about this need from our internal use, from our internal operations use cases, obviously, and we’ve heard it from customers as well that they don’t really like to do what we call sort of swivel chair analytics, where you have to like jump back and forth between different tools, one for metrics, one for logs, one for something else. We think we are in a good position to unify those things and this is why we are now basically bringing to market an additional metrics capability as part of Sumo Logic, and instead of building that as a second separate product, we decided that, hey, it’s really part of the same high level use case and sort of our unique proposition, value proposition there is, that we can do it in the same tool and we can kind of amalgamate the two different data sources and drive integrated use cases out of that, if that makes any sense.

    Alan Shimel:                Great. I just want to make clear to our audience. This is not officially released yet or it’s in a beta or early access form still or what is it?

    Christian Beedgen:     That’s correct. We have announced it. We announced it in April. We have obviously been working with design partners for a while. A number of our customers are actually using it as we speak, and we continue to get great feedback from them, and also, partially stuff that we now realize we probably still need to fix, you know, as usual when you’re deciding to roll out new functionality.

    We are currently planning to make this generally available by the end of July. So we’ll cross our fingers that we’ll hit that date. So it’s actually going to be in the market for everybody really, really soon. In the meantime, we allow folks to basically sign up to get onboard as early as possible and we’ve seen great interest in that as well.

    Alan Shimel:                Excellent. Now if any of our listeners who are not currently Sumo customers are interested in this, and I of course can see why they can, Christian, is there any place they can sign up for this or at least get more information, if not actually become sort of an early user or early access?

    Christian Beedgen:     Absolutely. Just to go SumoLogic.com and it’ll lead you there.

    Alan Shimel:                Great. Christian, if you don’t mind, we have a few minutes left and I want to sort of pivot a little bit. It’s basically my own thing. I actually just started writing – or finished writing a survey we finished up on containers and container technology. It certainly seems to be on the tip of everyone’s tongue. What are you doing? What’s your strategy around that? I want to ask a little bit about: does this pose a challenge for Sumo or a log is a log is a log?

    Christian Beedgen:     That’s actually a very good question. It actually turns out that we had to do a couple of things in order to stay ahead of the curve there, and that started in I think 2014. That’s kind of year that Docker broke, the way that I remember it. Then we started looking at what does this really mean, even for our own operations of course. But then there was a major voice coming from our customers, who of course jumped into this right away.

    It turns out that the way that you collect logs changes actually a little bit when you have the additional abstraction layer of a container. So we basically throughout 2015 wrapped our heads around that and figured out how to actually collect the logs from a containerized environment. There’s been a couple of iterations on that, of course obviously moving very fast, and they have also made some advances on how the logging works as far as Docker is concerned.

    But we now have first-party support for what I call sort of a comprehensive monitoring strategy for Docker. So that includes the logs as well as the metrics or what’s called stats in the Docker world, as well as these sort of Docker events and the additional metadata that can be queried out of Docker. So you know not just what the idea of your container is, but also what the images are that it’s running, et cetera. We can pull all of that into the Sumo and, on top of that, we have a really nice extension to Sumo that we call an app. That’s kind of our app for Docker that our customers are using.

    So it did change. The major change really came from the way that – technically speaking, the major change was that we had to do a couple of things in order to be able to collect they logs and the metrics, because that was basically the biggest change in terms of the low level stuff.

    At the higher level, the good news about the Docker stuff is I think that it provides a good amount of metadata, if you know how to get it, and mostly it’s with the Docker Inspect API that makes the analysis of the environment, which is now even more highly abstracted of course. You might not even know anymore which container runs on which source, et cetera. So there’s a lot of additional metadata that makes that actually really nifty. So I like that.

    Alan Shimel:                Very cool. Christian, we only have a few minutes left here, just another minute or two and I have two other things, two other questions on my list for you. Containers are certainly a very hot item. Dev-ops is very hyped. What do you think is next? What’s the next big thing?

    Christian Beedgen:     That’s one I think everybody is always thinking about that. There was a point when I realized what AWS Lambda actually was, and I was like, “Wow. That is really, really cool.” I don’t know yet exactly where it’s going to go, but I think it’s intuitive to me that in many ways Amazon has actually already invented the future after containers, and at the same time they were releasing the elastic containers. So I thought that was _____.

    Anyway, I think the term that has been emerging there is serverless, serverless computing. I think that is actually super-interesting, considering how complicated the stacks have become. It sort of adds yet another layer of abstraction, but basically one that destroys all the other underlying layers of the stack. I think that’s a really interesting thing that’s going on right now.

    Alan Shimel:                Sure, serverless computing for a no-ops age.

    Christian Beedgen:     There you go.

    Alan Shimel:                Let’s get rid of everything.

    Christian Beedgen:     Let’s make a t-shirt.

    Alan Shimel:                Yeah. Hey, once we get a t-shirt some VC will invest.

    Christian Beedgen:     There you go.

    Alan Shimel:                Anyway, Christian, last question because we are just about out of time. I often ask guests this. For people listening in, one book, if you can recommend one book to people that they should read that you find enlightening, helpful, must read, what would it be?

    Christian Beedgen:     The best thing that I’ve read recently is – oh, man, that’s a heavy-duty one. We are actually doing an internal book club on this and it’s been blowing people’s minds. There’s like matter all over the place. It’s called The Principles of Product Development Flow by a guy called Reinertsen. It’s kind of a second generation Lean applied to product development. The guy’s an academic. I love that book. I think it’s kind of mind expanding, so The Principles of Product Development Flow. That’s an awesome book.

    Alan Shimel:                If it’s on Amazon, I’ll try to include it in the show notes, so people can go look at it. Great. That’s a great suggestion and thank you. Christian, we’re over-time actually, but it’s okay, not too bad. I wanted to thank you for joining us on this edition of DevOps Chats. Continued success with Sumo Logic and we hope to check back in on you soon.

    Christian Beedgen:     Sounds good. Thanks for having me, Alan.

    Alan Shimel:                Okay. Christian Beedgen – and I know I always mispronounce his last name – but CTO/Co-Founder of Sumo Logic, thanks for being this week’s guest on DevOps Chat. This is Alan Shimel for devopsy.kinsta.cloud. Until next time, have a great day.

  • Austin & Chef Give ChefConf 2016 That Warm, Fuzzy Feeling

    Austin & Chef Give ChefConf 2016 That Warm, Fuzzy Feeling

    I have written before about the DevOps community in Austin. It is one of the most welcoming and warm groups of people I have ever interacted with. Combine that with the HugOps operation of the friendly folks of Chef and you wind up with a great experience. That pretty much sums up ChefConf 2016 for me. Beyond the warm and fuzzy though, there was also a lot on display on what the future of business is all about. To paraphrase a line from the movie, The Graduate, I have one word for you: AUTOMATION.

    Automation was on full display at ChefConf 2016. And not just your grandpa or dad’s automation either. For those who think Chef is still all about configuration management and automating servers, think again. I am talking about automation extending into every nook and cranny of the business. You only had to see Alaska Airlines CIO, Veresh Sita’s presentation to understand what automation really means. But truthfully that was just one of many talks and sessions that demonstrated the power of automation and how Chef is pushing automation up and down the business.

    There was also the announcements regarding Chef Automate and Habitat. While we have covered Habitat already, a quick word on Automate. Automate which is years in the making, is actually the combination of Habitat, Chef Delivery which was released last year and defines the DevOps workflow, Chef itself and Chef InSpec, the compliance and security project based on the technology from the acquisition Chef made last year of VulcanoSec. All of these are combined in one enterprise class package that Chef CEO Barry Crist called “our flagship product”.

    Beyond that though to really give you a feel if you couldn’t make it to ChefConf this year, there was a palpable buzz around the place. I think the feeling is that so many good things are starting to come together. While many label many of these under the DevOps umbrella, it is automation, DevOps, Agile and more combining to enable businesses to do more, faster and better than they ever have before. In addition, as these trends continue to evolve and accelerate the pace of innovation also continues to accelerate.

    I had a chance to speak briefly with Dr.Nicole Forsgren, who works at Chef but is also one of the key people behind the State of DevOps survey. Forsgren said that is why we are seeing the gap between high performing IT led companies and those who are not continuing to widen. The gulf will continue to grow as companies adopting these practices pull away from those who do not.

    From my point of view it is amazing to see that since I first became involved in IT we have seen IT move from a cost center as viewed by many an executive to the engine that drives the business.  As I remarked to several people at ChefConf, we certainly live in interesting times!

    I was also able to use the Chef Newsroom to do a few interviews there. We have put them up in our DevOpsTV Youtube channel under ChefConf 2016 playlist. There are about 8 or so total but I am embedding two to give you a taste. You should check out the rest if you have a chance, as they are pretty good.

    The first is with George Miranda of Chef and we talk about Habitat and a little about Automate.

    This second interview is with well known DevOps journalist, writer and community member, Ben Kepes, enjoy!

  • Every (Data)dog Has its Day

    Every (Data)dog Has its Day

    Bienvenido, Datadog! You have truly made it to the big leagues, as you join a growing list of reputable companies that have suffered an apparent data breach/security incident. Many people view this as a dreaded moment, one that should always be avoided. However, it’s almost as though nowadays it’s a rite of passage for a company to be hacked at some point in its lifespan. Call them growing pains, if you will.

    Datadog is a software-as-a-service (SaaS) company that helps monitor and collect analytics for customers, which include the likes of Facebook, Spotify, Salesforce and Citrix.

    As with any data breach, Datadog is recommending users change their credentials immediately—a common practice among down-on-their-luck dogs. This recommendation applies only to stored passwords; however, those enlightened few who use Google Auth or SAML should be OK. The notice also directs admin users to revoke or change credentials stored in the Datadog system to prevent fur-ther security breaches and unauthorized access.

    Obviously, data breaches should be avoided at all costs, and can be seen as a hit to a company’s bottom line. But Datadog stands on solid ground here following the attack. Its ruff-and-tuff password storage method, bcrypt, which uses a unique salt, is one of the strongest ways to keep credentials and slow decryption efforts, according to Datadog Chief Security Officer Andrew Bechere.

    Fur-tunately for Datadog, at least right now, its security procedures and bcrypt system have prevented what could have been a major disaster. The company has informed its customers that the “known” attack vectors used by the hackers have been patched.

    It is my utmost opinion that we throw Datadog a bone here as we continue to see hacks and breaches become commonplace among tech companies. Not only has it become quite common, but Datadog cur-tailed the damage of the breach by implanting bcrypt security encryption.

    Oh, you’re a stickler for your information remaining private? I hate to break it to you, but it’s the internet. Nothing is sacred and private once it goes online. However, you can take measures and increase security to make it extremely difficult and lengthy for hackers to obtain information. Still not good enough for you? Let’s just take a look at other companies that have suffered hacks.

    Sony suf-fur-ed a tremendous hack last year that listed executive emails and personal information including addresses and salaries. Home Depot suffered a hack in September 2014 that compromised 53 million email addresses. And let’s not forget about the JP Morgan hack affecting 83 million customers, the eBay hack that compromised 145 million active users (including login credentials and physical addresses) and the 2013 hack of Target, which exposed personal and financial information of 110 million customers.

    So enough pup-arazzi for Datadog, don’t terrier yourself up about it. Every once in a while a company suf-fur-s a data breach. What can I say? The puggle is real out here.

    I’m sorry for all the puns. In related news, Datadog issued a press release, which you can find here.

  • The 2016 State of Software Supply Chain Report is Here

    The 2016 State of Software Supply Chain Report is Here

    Our State of the Software Supply Chain Report has just been released.  Over the past year, we’ve amassed a great deal of data with respect to the staggering volume and variety of open source components flowing through software supply chains into development environments. This year, we assessed behaviors across 3,000 organizations and performed deep analysis on more than 25,000 applications.  

    The results we discovered ranged from staggering to surprising to sobering.  For example, we measured organizations consuming an average of 229,000 components annually.  The good news is, these components help companies accelerate their development and innovation.  At the same time, we saw 6.8% of components used in applications marked with at least one known security vulnerability — adding high levels of security debt.  Not all components are created equal.

     

    The 2016 State of the Software Supply Chain Report: software supply chains by the numbers.
    Software supply chain infographic from 2016 report.  See full image here.

     

    The 2016 State of the Software Supply Chain Report blends public and proprietary data with expert research and analysis to reveal the following:

    • Developers are gorging on an ever expanding supply of open source components.  Billions of open source components were downloaded in the last year.
    • Vast networks of open source component suppliers are growing rapidly.  Over 10,000 new versions of open source components are introduced daily.
    • Massive variety and volume of software components vary widely in terms of quality.  1 in 16 parts include a known security defect.
    • Top performing enterprises, federal regulators and industry associations have embraced the principles of software supply chain automation to improve the safety, quality, and security of software.

    We invite you to read the report and leverage the insights to understand how your organization’s practices compare to others. Then, let us know what you think @sonatype.

    If you would like to join a live discussion on this year’s report, please join us on Wednesday, July 13th. Save your seat here.

  • AppOps: The Next Big Thing or Just the Next Buzzword?

    AppOps: The Next Big Thing or Just the Next Buzzword?

    I’m not sure if you’ve noticed, but the pace of the world seems to continue accelerating. Organizations—and the developers and IT teams that work in them—need to be able to adapt and respond quickly to changing demands. DevOps addresses this challenge for app development and IT infrastructure. However, a new application lifecycle equivalent is emerging, called AppOps, which seeks to bring the same sort of efficiency and agility of DevOps to existing IT services.

    Just as DevOps creates an automated life cycle between software development and IT operations, AppOps streamlines application analysis, deployment and management. Sparked by the emergence of BYOD and the consumerization of IT, AppOps simplifies how IT securely delivers and manages existing enterprise and browser-based apps, while quickly adapting to demands of line-of-business managers.

    “For a long time now, businesses have shifted toward ‘commercial off-the-shelf’ software packages in hopes of saving resources,” says Calvin Hsu, vice president of Product Marketing for Windows App Delivery at Citrix. “However, this shift toward packaged software doesn’t change the operational needs for IT. They need a fast, simple way to securely roll out and deliver new applications in alignment with operations—an AppOps approach.”

    According to Hsu, AppOps provides a variety of potential benefits, including:

    • IT can more quickly respond to demands from line-of-business managers.
    • cloud-based management infrastructure can play a more significant role in accelerating IT deployments.
    • A virtualized secure browser can make browser-based and SaaS apps more secure, and even lock down browser vulnerabilities on apps purchased outside of the IT process.
    • Improve communications between IT organizations and executive teams to improve planning and alignment between business and IT objectives.
    • Improve service quality by automating the application analysis/deployment process to initiate a constant feedback loop (metrics, user experience, stability, etc.), that keeps everyone related to the app life cycle up to date and enables truly proactive management.

    Hsu says that app and desktop delivery simplification and collaboration between vendors is simplifying how IT deploys and delivers apps to employees—virtually, centrally, securely and reliably. He adds that cloud-based management infrastructure makes it easier for IT to offer a menu of services to help business managers determine which apps and data resources will work best for their teams.

    On the one hand, AppOps seems like a natural evolution of the DevOps culture. On the other hand, it also seems like it’s just an extension of DevOps rather than its own separate thing. It is simply taking the DevOps culture—breaking down silos and removing roadblocks to streamline collaboration—and applying it to existing IT services.

    What do you think? Is AppOps the “next big thing” or just a new buzzword being applied to the same old DevOps?

  • DevOps Chat: Habitat by Chef with Julian Dunn

    DevOps Chat: Habitat by Chef with Julian Dunn

    julianI had a chance to speak with Julian Dunn, product manager for Chef ahead of the upcoming ChefConf conference in Austin Texas next week. Julian and I spoke about Habitat, Chef’s new open source application automation project.

    Many in the industry are hailing Habitat as something really special. Our own Tony Bradley says it could be a game changer.  Julian gives us some of the background and reasons why Habitat is important for you. He also gives us an advance look at the three Habitat related sessions scheduled for next week at ChefConf.

    If you are already going to ChefConf, be sure to hit all three of these if you can. If you are in the Austin area next week, try to make the show. We will be there covering it live.  Also the opening keynotes for Tuesday and Wednesday are being live streamed over the web.

    In the meantime here is my conversation with Julian with the printed transcript below, enjoy!

    Alan Shimel:                Hi, this is Alan Shimel, devopsy.kinsta.cloud, here for another DevOps Chat, and I’m happy to have Julian Dunn, Product Manager of Chef joining us today. Good morning, Julian, how are you?

    Julian Dunn:               Good morning, Alan, I’m doing very well. I’m melting a little bit because it’s very hot in New York, but you know, other than that—perfect summer day.

    Alan Shimel:                Yeah, it’s July in New York, it happens. Julian, we could talk about weather, I’ve certainly been traveling enough to see different ones, but let’s focus in a bit on what I wanted to really talk to you about today, and that is Chef’s new Habitat offering. Let’s assume a listener doesn’t know what Habitat is at this point. Give them a quick kind of elevator pitch on it.

    Julian Dunn:               Yeah, absolutely. Now, if you’ve read any of the materials and things like that, we’re using this tagline called Application Automation. And, you know, what on Earth does that actually mean? Well, what is the problem that Habitat solves for people?

    So, you’re an application developer today, and what you end up doing as an application developer is—well, what you want to do is, you want to spend a lot of your time doing the business logic and writing the business code that is actually gonna differentiate you from your competitors, right? So if you’re working at JCPenney, let’s say, you’re trying to write features for JCPenney’s website or mobile application or whatever to differentiate you from Macy’s and Target and Nieman Marcus or whoever your competitors are, right?

    Well, the experience of being a developer today actually ends up being that, before you write any of that program code, you have to spend a lot of time in the weeds and dealing with what I call sort of the plumbing oriented aspects of doing the programming, right?

    So just imagine if you were an interior decorator and you want to simply decorate your bedroom to whatever you wanted in there—you know, damask curtains and what have you—but you’ve spent most of your time dealing with the electrical and the plumbing. Not a particularly nice experience.

    Alan Shimel:                Mm-hmm.

    Julian Dunn:               The other thing is that, when you make those choices about what plumbing and what electrical systems you’re gonna use, that actually changes how your room is decorated. I mean, how absurd would that be, right?

    Alan Shimel:                Mm-hmm.

    Julian Dunn:               And so the notion of trying to let the application develop or be able to concentrate mostly on developing their program code and make a lot of those infrastructural choices later led us to develop Habitat, which is something that takes a little bit of a different approach than how we’ve built IT systems in the past, right?

    It’s kind of back to my analogy of the plumbing. In the past, we’ve built sort of up from the bottom towards the application, right? So we choose an operating system, we choose a bunch of stuff that sits on top of the operating system. You eventually at some point, we reach the application, right?

    And that’s an incredible waste of time for the application developer, who actually wants to start up here and define what their program logic is, right, and how that application should behave in a lot of different scenarios. And if you sort of start at the top and move towards the infrastructure, that’s the surface area that you actually end up owning as the application developer—we believe that that is Habitat.

    Alan Shimel:                Very cool. And hence the name, then, right? When you explain it that way, the name Habitat actually makes a heck of a lot of sense.

    Julian Dunn:               That’s right.

    Alan Shimel:                Yep. So, Julian, this is, some would say, you can see where Habitat is analogous—not analogous, adjacent to kind of the space that Chef plays in already. It’s a much bigger mission, if you will, than just what Chef has done to date. Is that a fair statement, you think?

    Julian Dunn:               It sure it, it sure is. You know, Chef, I think, initially started out, obviously, right, with a configuration management product that was very infrastructure oriented. And, over time, you’ve seen that our product portfolio and what we’re trying to do is a lot larger than just infrastructure configuration management, right?

    We kind of describe ourselves today as an automation company. We solve a bunch of automation problems, wherever automation is needed to help reduce people’s time to market or burdensome activities that they need to do in a regular ________, right? To really free people up, no matter what their problem domain is to being able to do the really intelligent parts of their job—the thinking parts of their job, right? And you’ll see that as we released Chef Delivery, which is a workflow system that allows people to basically implement, if you will, the principles that are in a lot of the books out there like Jez Humble’s Lean Enterprise that tell you, “This is the shape of how you should be delivering the software out to production.” Basically something like Chef Delivery is an automation arrayification or that technology—or those ideas, right?

    And then when we made the acquisition of VulcanoSec, which is the German security company, that gave us in-spec and Chef compliance that’s automating the work of compliance officers, security officers—right, folks that spend a lot of time, their job today and their pay and their experience is they spend a lot of time manually poking around systems and putting up fires manually because they don’t have a good framework and a system for describing the correctness of a particular system, right?

    And so now, as we move into—right, we’re, as a product organization, you want to be one that’s trying to address people’s pain points, right? When we started looking at what the pain was for an application developer, and they spend so much time dealing with the nuts and bolts of actually writing the code, we’re like, “How—why are systems built in this way?” We really went down a path of inquiry and we came up with Habitat as a way more application developer-centric than the perhaps infrastructure-centric things that we’ve built in the past.

    Alan Shimel:                Sure. Just two thoughts on that, Julian, I’d like to explore with you. One is—look, I’m not, I don’t hold myself out to be a Product Manager at Chef or anywhere near that integrated into it. But it feels to me, anyway, that the Chef compliance and the Chef delivery are very tactical pieces that embrace and extend Chef’s core configuration management capabilities, right?

    Where Habitat to me is more of a strategic rather than tactical type of play here, where you’re looking to set the game—the grounds, right? The stadium that we play in, whatever you wanna call it, the field, right? And so it’s a different—and again, I might be wrong, I’ll defer to you—but it feels different to me.

    Julian Dunn:               It is different in the sense that it’s a little bit iconoclastic, you’re right, in that it challenges some of the approaches that folks have been doing to try and deploy applications and get the sort of modern application characteristics that they want out there. And those characteristics are really great. I mean, one thing that’s really—as we developed this product, what really caught for us was to focus in on, “What does a modern application look like? What characteristics does it have?” And those characteristics are things like, it should be deployed immutably. It should be deployed atomically in that it’s either installed or it’s not, right? It should have certain characteristics built in. It should be aware of—it should know how to behave when it’s deployed in different scenarios, right, in different, what we call topologies.

    Alan Shimel:                Mm-hmm.

    Julian Dunn:               So you can think of, on a develop laptop, you might just have one instance. In a production environment, you probably have a number of instances, and they would have some relationship with one another, right? They might have one where you send all the rights two and the other ones are the reads, right? There might be some that are—like, think of a caching system, right? They right be hierarchical where there’s sort of a first, second, and third layer cache, like Squid or something like that, right?

    So it’s relationships between these entities, these application components, and one thing that we realized as we’re developing Habitat is, you know, as folks develop more complex applications and those applications are actually distributed systems, folks haven’t really spent a lot of time figuring out how to describe those relationships in a way that it imbues the applications themselves with the understanding about how they’re related to other components, right? The ways in which people have tried to solve the relationship problem between these components is to create sort of a central system, an orchestrator, if you will, that defines those relationships.

    And the trouble with an orchestrator is that it becomes a single point of failure, and as we know, that’s not really compatible with the cloud, right? Where in the cloud, they tell you, “You really shouldn’t have single points of failure, because you’re on an unreliable fabric,” right? So you really should have a system that knows how to recover and continue to function in the face of failure one or additional components, right?

    And so that’s where we started to come about with the architecture and the design of Habitat and how aware—how self-aware these individual components should be.

    Alan Shimel:                Absolutely. Just one other kind of comment or observation, and that is, you’re right—configuration management and compliance seem to be operational, or at least lead with operational, where Habitat clearly is a pay in the developer specter. But it’s all supposed to be one under DevOps, right? I mean, we don’t want to—I’m not into creating divisions between developers and operators, and I’m sure you guys aren’t, either, Julian.

    Julian Dunn:               Right, but I’ll tell you

    Alan Shimel:                But clearly they’re different focus—go ahead.

    Julian Dunn:               I was about to say, Alan, you know, the dirty of secret of DevOps, my observation, you know, I’ve been doing this for a number of years, and I’d love to hear your thoughts on this. But my feeling about the dirty secret of DevOps, and I go to DevOps Days and these conferences, and you survey the audience and you look at who the folks in the room are, right? And it’s something like, if you’re lucky, it’s still 80 percent Ops and 20 percent Dev, right?

    Alan Shimel:                It is, it’s OpsDev.

    Julian Dunn:               Yeah, right. So DevOps has been very much about Ops taking a lot of the good practices from Dev and learning a little bit of coding and learning that there’s a proper workflow and that you should have tests for stuff, but even if you’re not doing automated tests like Chef does, at least you should have some system for testing stuff before you just go log into a box and make changes in production and whatever.

    It’s almost like there’s this information flow that’s gone one way, but the information flow hasn’t really gone in the other way, even though we’ve had a number of iterations and generations of this technology. And so when we started asking the question, why—why have we kinda gotten from the brick wall to sort of more an English hedgerow, if you will, between Dev and Ops? How can we further develop capabilities to break down even the English hedgerow, if you will?

    Alan Shimel:                Makes a lot of sense. So, Julian, unfortunately, we’ve only got 12 or 14 minutes here and we’re running out of time. There’s a couple other things I wanted to hit. We’re recording this just the week before ChefCon, which is next week down in Austin, Texas—July 12th and 13th, or 11th, 12th, and 13th, technically. And, of course, I’m sure you’ll be there. Will there be a lot of Habitat kind of information and presentation?

    Julian Dunn:               There will. There’s gonna be three major talks, and we’re sort of billing them as the Habitat 101, 201, 301 kinda talks, right? We’re gonna have an intro and then sort of how Habitat fits into the ecosystem is sort of the 201 level talk, and then there’s a deep dive into why we made the technological choices that we did, the system design, and things like that, and that’s the 301 level talk.

    At ChefCon, we’re also gonna have something called the Habitat Zone, and we’ll program that with a few talks as well. The speakers from the 101, 201, 301, they’re gonna answer the Q&A in the Habitat zone, and that’s gonna be near the registration area. We’ll also have sort of community building—that’s one thing I didn’t mention, I don’t know, Al, if you mentioned it, but Habitat is a fully open source project.

    Alan Shimel:                Yeah, under an Apache 2.0 license.

    Julian Dunn:               Under the Apache 2.0 license, and we believed that that, we needed to make it open source to really engender a great community, similar to how created a huge community around Chef and, you know, just ways in which folks can contribute to that Habitat community is by writing plans that can build packages that live within the Habitat ecosystem.

    There’s, you know—just think of how many packages are out there within a typical kinda Linux distribution, right? There’s a lot of surface area that we would want to cover with people creating Habitat plans. That’s a great way for folks to get started. We’ll have some programming in the Habitat Lounge to talk about that with folks.

    Alan Shimel:                Yep. Julian, I’m gonna do my best to at least get the audio of this up before, in the next day or so if I can’t get it today.

    Julian Dunn:               Great.

    Alan Shimel:                And anyone hearing this, if you are in Austin or near Austin and you’re not already going to ChefCon, you really should look into it. You can get information, I guess, at getchef.io. I’m sure it’ll be listed there, just Google ChefCon.

    Julian, unfortunately, we’re just about out of time, but you know what, maybe we’ll pick this up down in Austin and we’ll do a part two or something.

    Julian Dunn:               Great.

    Alan Shimel:                Absolutely. I want to thank you for joining us today on DevOps Chat, and great—you know, I’d love to see Habitat really take off, here, and continued success to Chef and all the gang there.

    Julian Dunn:               Absolutely. Thanks very much, Alan.

    Alan Shimel:                Okay. Julian Dunn, Product Manager of Chef, thanks for appearing on this episode of DevOps Chat. We’ll see everyone next time. This is Alan Shimel for devopsy.kinsta.cloud.

  • Should Citizen Developers be Welcome in DevOps?

    Should Citizen Developers be Welcome in DevOps?

    “We’re all developers now. Citizen developer activities will accelerate.” That’s what Gartner predicted back in 2012. Fast forward more than four years and we’ve seen the DevOps movement explode, while the rise of citizen developers has also marched steadily. The challenge for organizations is to figure out how these two philosophies co-exist and complement one another.

    What are “citizen developers”? The 2015 State of Citizen Development report defines them as, “… empowered problem-solvers from the various lines of business who have the drive and determination to engage in app development even though they lack traditional coding skills.”

    DevOps is about agility and momentum. Automation. Continuous continuousness. The culture and tools that comprise DevOps are all about moving faster—facilitating better collaboration and streamlining processes to accelerate the development and deployment of applications. That more rapid pace relies heavily on automation, in most cases, and still has rules and procedures to manage how things are developed, tested, integrated and deployed.

    The citizen developer movement, on the other hand, is sort of an evolution of rogue or shadow IT. There has always been “that guy” in any given company. The one who is too impatient to follow procedures or wait for IT to deploy a solution, and who knows just enough to be dangerous and brew his own unsanctioned tools. Citizen developers harness that same “maverick” attitude, but in a more acceptable and manageable way.

    Platforms like Intuit Quickbase or Genesis from Metavine provide users with little or no coding experience with the tools to create effective applications without writing a single line of code. It may seem like that encroaches on the work of developers and confuses things, but there are two caveats to that. First, the users are in the trenches and have an intimate understanding of the issues they’re trying to solve—so theoretically they’re best-suited for solving them. Second, if citizen developers can use code their own solutions for the “low-hanging fruit,” then developers are free to focus on solving more complex business problems and developing more innovative solutions.

    Metavine explains, “This approach for generating Cloud and IoT applications places more responsibility in the hands of the people who define business needs, freeing them from typical engineering constraints. Application development becomes more collaborative and highly creative, involving technologists, solutions architects, designers, and line of business people performing cross-functional work to produce agile business results.”

    Craig Sproule, founder and CEO of Metavine, provided a description of Genesis. “It is a true zero code application development platform designed to enable the creation of enterprise-class applications. It allows developers and architects to build applications to meet exacting needs, while delivering unprecedented agility to the business.”

    While at first glance it would seem that having a bunch of rogue citizen developers implementing their own home-grown apps is a bad idea and would complicate and conflict efforts to streamline development through DevOps. But the reality is that they can co-exist peacefully. Using the right tools and platforms, citizen developers can help solve some of their own problems more quickly and let developers focus on solving more complex problems.

  • TASC Powers Up DevOps Pipeline

    TASC Powers Up DevOps Pipeline

    Technology team members at TASC, a leading administrator of employee health benefits, were feeling the chains of manual processes that kept them from moving software out at more than a snail’s pace. Development, operations, quality assurance (QA), project managers (PMs) and business analysts who were pushing code to test, staging and production environments or performing user accepted testing (UAT) tests needed to forward features through the pipeline much more quickly.

    “We were doing one release a month, shutting down the entire website for six to eight hours on a Saturday morning to complete deployments,” says John Gildenzoph, DevOps engineer at TASC.

    The company took to orchestrating its pipeline using Automic Release Automation (ARA) to onboard DevOps and catapult software innovations into production.

    Manual Challenges

    Before ARA, TASC used Jenkins—that was pretty much its first step into DevOps-style automation. Before that, the company’s processes were, for the most part, all manual, implemented with a source code repository.

    “For anyone to deploy the application for development on their workstation, or to a test environment or even to production, they had to go out to the server, check out the source code, build the source code and deploy the artifact,” says Gildenzoph.

    TASC was doing that on up to 30 JBoss servers—which were its main web servers—using only two developers. One developer started on the first node and the other started on the last node as they worked toward the middle, updating each server individually and manually, he says. “We came from the Dark Ages where there were really no other products we were using other than the servers and our version control software. Jenkins was the first iteration where we figured out that we needed to be running builds all the time,” Gildenzoph notes.

    Uniting DevOps Tools, Filling Gaps

    TASC uses ARA to unite a variety of DevOps tools into the pipeline and to fill gaps where there isn’t a specific tool. “ARA is platform-agnostic. You can use it like an assembly line and you can plug the worker applications in along the assembly line as you like,” Gildenzoph says. In one example, TASC uses Jenkins as one of those “worker applications,” i.e., as its build tool: ARA tells Jenkins to build the software, then moves the software along itself to the next tool to do the next job.

    According to Gildenzoph, all a company needs to add best-in-class tools at each step in the development process along the pipeline is a command line interface (CLI) or API that enables them to script that tool in to integrate with ARA. That way, if the company decides to use a new tool for any part of the process, it can simply script that tool into the ARA at that point, rather than replace the entire pipeline with one new tool for everything.

    TASC also uses ARA to orchestrate in its use of Amazon EC2 and the ECS products. At the same time, TASC is working to add the management of Docker containers into ARA in the same way.

    TASC IaaS

    TASC uses ARA to create the entire dev, test, staging and production environments where the company will deploy the software. Using the ARA and its automation engine, which is a sort of complex chron that develops the workflows, TASC interacts with its Amazon cloud presence to create the entire application stack.

    “We see that Automic is working on its next evolution of the ARA product, which is a provisioning product that will interact with various cloud technologies such as Amazon or OpenStack so we will be able to create that environment in less of a homegrown manner. It will be more of a drag-and-drop capability with run books to interact with your cloud,” Gildenzoph says.

    These cloud technologies also will be like the applications/worker setups in Gildenzoph’s ARA assembly-line analogy.

    Revving Up Results

    “We moved from our original manual scenario to the point where in some cases we don’t need downtime to deploy the software. We have the orchestration in place and it manages to bleed off part of the cluster, update that cluster, and then bring those portions back online so that our customer doesn’t notice,” says Gildenzoph. This benefits internal and external customers who use the site for call center processes and submitting flex claims, respectively.

    TASC is more agile since the adoption of Automic, releasing software typically on a weekly timetable. This enables the business to respond to requests from customers for new features, innovations and enhancements to the site more quickly and efficiently.

    When TASC purchases another company to add its products to TASC’s, an accelerated pipeline enables the company to onboard that company and products more swiftly and seamlessly.

  • Chef Has Potential Game-Changer with Habitat

    Chef Has Potential Game-Changer with Habitat

    Chef is an established leader in the DevOps world. Automation is a keystone of DevOps tools and processes, and Chef’s platform simplifies and streamlines automation. Chef is branching out, though, with the recent unveiling of Habitat—an open-source project to automate the applications themselves.

    As companies continue to embrace DevOps and developers continue to create custom applications, it is becoming increasingly difficult to manage it all. Many organizations have infrastructures that span on-premises and cloud—some using multiple cloud platforms—and the applications have to be crafted for the specific infrastructure or platform they’re designed to run on. Habitat seeks to change that. It ostensibly allows developers to focus on the features and capabilities of the app rather than the unique constraints of a specific infrastructure or particular runtime environment.

    “We must free the application from its dependency on infrastructure to truly achieve the promise of DevOps,” said Adam Jacob, co-founder and CTO of Chef. “There is so much open-source software to be written in the world and we’re very excited to release Habitat into the wild. We believe application-centric automation can give modern development teams what they really want—to build new apps, not muck around in the plumbing.”

    A blog post from Chef describes the premise of Habitat:

    Applications packaged with Habitat have the intelligence to self-organize and self-configure. Habitat allows the application to both be portable across independent infrastructure environments and have the intelligence to select the infrastructure features that benefit that application. This makes it easy to run applications across increasingly diverse environments such as containers, PaaS, cloud infrastructure, and on-premise data centers, while also accelerating hybrid cloud environments.

    It sounds promising to Michael Glenney, solutions architect at New Context. Glenney said Habitat—if it lives up to what Chef has promised—may represent a paradigm shift in how applications are managed in a data center where configuration management (CM) is prevalent.

    “Traditionally, you would use CM (Chef, Puppet, Salt, etc.) to not only lay down the bits an application needs to run, but to manage, redeploy, reconfigure and upgrade that application as well,” explained Glenney. “This type of work is not particularly well-suited for a CM and it mixes workloads unnecessarily. For example, an application deployment could fail because you have some data incorrect concerning log rotation on the host system. This is a manufactured relationship that needs to end.”

    Glenney noted that with features such as application-to-container conversion, universal package management, audit and traceability of both buildtime and runtime application dependencies, and advanced management and deployment functionality, Habitat is an exciting addition to the world of DevOps and lean security.

    Habitat combines the automation, configuration and management of an application into a single package. “When used in combination with CM it will deliver an unprecedented level of control over applications in all of your company’s environments and provide what I believe has been missing since the beginning,” Glenney said.

    Habitat is an open source project under the Apache 2.0 license and is available for immediate download. Check it out and let us know what you think of it.

  • SimplyHealth Uses DevOps to Manage Mobile Testing

    SimplyHealth Uses DevOps to Manage Mobile Testing

    SimplyHealth, a private company health insurance plan provider in the United Kingdom, saw its mobile web traffic trending upward with no end in sight. “While this was good news, there was concern whether the customer experience and the company’s ability to provide continuous application delivery on mobile devices might erode as a result,” said Chris Dale, information technology test and release manager at SimplyHealth,

    A good half-dozen challenges in mobile testing and development stood at the crossroads between mobile customer demand and SimplyHealth’s mobile development speed and ability to respond to needs for improvement.

    With some help from Mobile Labs, a provider of internal cloud mobile device testing for software, SimplyHealth allayed its fears of customer dissatisfaction.

    Mobile Web Test, Dev Challenges

    Dale noted SimplyHealth’s challenges to mobile web test and development were many. Among them:

    • Testing came late in development, causing the testing team to find issues at later in the development process and leading to costly rework for developers.
    • The vast array of mobile handsets that needed to be tested against, which affected productivity and increased expenses because of the time spent searching for, ordering and waiting for devices to be shipped to SimplyHealth for testing.
    • The development team’s use of mobile device simulators for testing mobile software on different device platforms—a process that did not represent accurately the behavior of actual devices.
    • The differing behaviors of Apple and Android devices in some functional areas, which made it impossible to do cross-platform testing with a single script.
    • Testing eventually  became a bottleneck, jeopardizing SimplyHealth’s award-winning customer service levels for its mobile customers.
    • The digital marketing team had difficulties in verifying customer experience on mobile devices.

    An all-encompassing solution for the SimplyHealth mobile web testing team would enable early testing and cross compatibility with the needs of all affected departments, including marketing.

    Dale said the tool would have to meet 34 requirements, including:

    • Allow for the highest levels of security and compliance with the Data Protection Act (similar to HIPAA regulations in the United States)
    • Provide the ability to use the existing automation framework and toolset and hook into SimplyHealth’s continuous integration solution for automatic test execution
    • Offer a way to swap devices in and out quickly and easily
    • Provide easy accessibility from any location
    • Have a simple user interface
    • Maintain high levels of performance
    • Offer scalability for future growth
    • Come with backing by dependable, knowledgeable support and a solid development road map

    The Solution

    After looking at four possible testing solutions based on products listed in Gartner’s Magic Quadrant, SimplyHealth decided on Mobile Labs deviceConnect to manage mobile devices for testing software. This on-premises cloud solution provides the accessibility and flexibility of the cloud and the security of on-premises technology, protected by the security measures available to the enterprise for its other assets, such as firewalls.

    The physical installation largely amounts to a mobile cart and rack for plugging in many different mobile devices for testing. “DeviceConnect enabled SimplyHealth to have a product-agnostic virtual lab so we could all easily test software on the devices using our own in-house testing scripts,” Dale said.

    Mobile Labs was willing to configure and demonstrate a proof of concept mobile device web application test setup using deviceConnect to show the SimplyHealth team how it worked to manage apps and devices for testing. “After we thoroughly evaluated this configuration, we concluded that Mobile Labs deviceConnect was the best choice,” Dale said.

    Implementation

    Mobile Labs implemented the entire solution for SimplyHealth, from setup to installation to ensuring it integrated well with SimplyHealth’s existing Hewlett Packard Enterprise testing tools, Dale noted. “We were immediately writing automation scripts to use with it. We had it working in production the next day.”

    All affected teams and departments have fully adopted and use the Mobile Labs tool, including testing, development, branding, user experience and support teams, he said.