devopsy.kinsta.cloud

Tag: dev

  • DevOps Chat: Lucas Carlson, VP Strategy, Automic, Can Ops Thrive In DevOps?

    DevOps Chat: Lucas Carlson, VP Strategy, Automic, Can Ops Thrive In DevOps?

    In this DevOps Chat our guest is Lucas Carlson, newly appointed VP of Strategy at Automic. Of course, Lucas has already had a distinguished career. Here is his bio from Linkedin:

    Lucas Carlson is an author, entrepreneur and executive. He also has a background of 20 years of programming to back it all up.

    He’s written scalable software that has been downloaded millions of times and used by millions of users. As a keynote speaker in the technology arena and innovation leader in the cloud space, Lucas helps companies discover opportunities for growth.

    Lucas wrote a bestselling book on Ruby for O’Reilly called The Ruby Cookbook and has created over a dozen open source libraries, some of which have been downloaded almost a million times.

    Lucas was formerly the CEO and founder of AppFog, a massively popular DevOps startup acquired by CenturyLink in 2013. AppFog signed up more than 100,000 developers, raised nearly $10M in venture funding and has become an industry leading Platform-as-a-Service.

    Before following his passion and writing books, Lucas served as the Chief Innovation Officer for CenturyLink, in charge of building next-generation Docker and Linux Container products for a $20B company.

    Lucas also writes for major tech and entrepreneur publications including Inc, Business Insider, InfoWorld and InformationWeek.

    I spoke with Lucas about why he joined Automic. Our discussion proved to be a provocative one, in which we explored Lucas’s view that DevOps is not a great thing for Ops. Ops needs its “own DevOps” that is more geared to its challenges and world view. Of course, many thought that DevOps is just that. Before you comment, have a listen to our conversation to see what Lucas said. I am interested in your thoughts and comments on this.

    Alan Shimel: Hi, everyone. Alan Shimel, editor of devopsy.kinsta.cloud here for another DevOps Chat. Very happy to be joined in this DevOps Chat by VP of Strategy at Automic Lucas Carlson. Lucas, welcome.

    Lucas Carlson: Thank you so much. Glad to be here.

    Shimel: Thank you. Lucas, I know you’ve been at Automic now, what, about two or three months?

    Carlson: Yep.

    Shimel: But I’ve known Lucas Carlson for years before Automic and I don’t know if our audience is familiar with you, but you’ve had quite a storied career already at a relatively young age.

    Carlson: Thank you. Yeah.

    Shimel: Why don’t you give our listeners a little bit of the Lucas Carlson story.

    Carlson: Sure. Well, I started out as a programmer: for 20 years I wrote code. I started code when I was in elementary school and ever since I’ve been writing code. And I wrote two programming books during my career. I’ve mainly focused on the open-source side of things so Ruby, PHP, Python worlds.

    And then I started a platform-as-a-service startup about seven years ago and I used Cloud Foundry as the foundation for my startup and it was called AppFog. We signed up 100,000 developers, it was a lot of fun. It was a great time.

    We were acquired by a telco and it was really interesting to spend a few years inside of a telco and watching them attempting a digital transformation—trying to move their applications into the cloud was a very interesting story.

    And after that I quit that job, I moved to Paris for a while—it was a life dream of ours—moved my family, and after Paris I looked around for what to do next and this opportunity came at Automic, and I thought it was a very interesting opportunity for reasons we might get into.

    Shimel: Great. Great. And of course, you know, interesting with your background and accomplishments Lucas that Atomic, which is an automation company, is where you wound up and as VP of Strategy there.

    So as someone who was a chief strategy officer at the last company I co-founded, Lucas, I can appreciate the title of VP of Strategy. But for our listeners who are saying, “Well, that sounds like just another corporate title,” give them a little insight. What’s your role as VP of Strategy at Atomic?

    Carlson: Yeah. Well, my role is to take a company that’s been around for almost 30 years that’s had a product that is battle tested, been around the block, really works, works at scale, has thousands upon thousands of enterprise scale customers, has a lot of pedigree and DNA, and yet is a company nobody’s heard of.

    That’s actually why I kind of joined. I thought it was a really interesting challenge to take a company that nobody’s heard of yet has had a product that is—has stood the test of time, is very, very robust, very strong, and see if there wasn’t a way to kind of help that company get recognized for the work that they do, which is actually really great work.

    And so I joined the company with that kind of concept in mind. And it’s been very interesting because fundamentally when I came into the company they called themselves a DevOps company and I come from a background—I’ve been doing DevOps for a long time. I grew up around Portland, I hung out with Luke Kaines who founded Puppet and he was one of my mentors when I was building AppFog and—so I knew DevOps inside and out. I used DevOps to build my own startup.

    And so I came in and I looked at what Automic had and Automic has a very interesting, strong product, but it’s not DevOps, because DevOps is predominantly—you know, having grown up around it I know you know DevOps is not some mystical union of developers and operations into this perfect combination that can do both and is just as adept at writing Ruby as they are debugging MySQL.

    You know, that’s kind of a story—a fairytale that people like to tell, but it’s not the truth. The truth is, DevOps was created by developers for developers to get around operations. They were sick and tired of waiting for operations to get around to provisioning their SQL server so they wrote some code that did it for them.

    They were tired of kind of the delays and lags so they wrote code that got around it. It wasn’t really meant to create this mystical union it was really meant to bypass operations.

    So, I guess, how does that apply to Automic? So, Automic came in saying they were DevOps—they weren’t a tool for developers and yet I thought they have this long track record. They’re doing something right, but what is it?

    And I spent a lot of time thinking about that before I joined and when I joined I became convinced that there is a huge part of the DevOps story that is getting left out: it’s getting forgotten, it is getting sidelined and ignored that doesn’t deserve to be ignored.

    I think that DevOps isn’t enough on its own. Having run a platforms and service company myself, having put platform as a service into major name-brand banks, having seen people trying to adopt these DevOps technologies—what I’m seeing over and over again is that one of the biggest struggles about DevOps is a myth that I’m trying to shatter. And the myth is that DevOps doesn’t need operations: that DevOps outsources operations, that operations isn’t necessary for DevOps.

    In fact, I think the absolute opposite is true and I don’t think ops is becoming part of dev. I think that ops deserves its own practices equivalent to what DevOps brought to developers. Equivalent and yet different, because I think that fundamentally operations was a different world of developers: always will, always has.

    To developers, failing fast is important and breaking stuff is totally acceptable. And going iteratively is great. So if you write some code, it doesn’t work you can fix it really quickly because you failed fast and it makes it better. And that kind of iterative approach is something that works really, really well for developers but it does not translate for operations. Operations cannot fail fast. If you fail in operations and your database goes down in production, that could be millions upon millions of dollars lost for the business.

    So it’s not that operations doesn’t want speed—of course they want speed, it’s just that speed is not the primary. For developers, speed is primary; for operations, reliability, robustness, scalability trumps speed. It’s not that they don’t want speed, it’s that they have a different core value set—a different world view—and instead of trying to mush these things together, instead of making them one idealized person that’s never going to exist, what I say is, let’s raise up the ops side of DevOps, bring it to the same level that developers have been raised. When developers adopted DevOps practices, it raised them up to a new level as a power-up—it was a mushroom in Mario. And what we need is to give the ops side a mushroom.

    We need ops to level up and be able to play at the same level as developers, but with tools built for operations, not built for developers.

    The tools out there for developers in DevOps are not the right tools for operations in DevOps and that’s what –

    Shimel: Hold on, I’m going to stop here because I’ve got to jump in—I’m sorry—I’ve been biting my tongue.

    So this is actually a pretty—I don’t want to say radical, but a very different kind of view of DevOps and what goes on between dev and ops than I think we hear from most people.

    And it actually reminds me I had a conversation—actually a DevOps Chat—with Dave West, who is the CEO of Scrum.org. And you know, obviously, Scrum and part of Agile is really aimed at developers. And though he saw sort of a potential bridge from dev to ops he had a similar view of yours that, you know, we have tools that are written for developers that developers are using that are letting them get more involved in ops but how much is ops really being involved in this DevOps, if you will.

    But yet, you know, let’s look—like configuration management tools.

    Carlson: Yep.

    Shimel: Do you think those are dev tools not ops tools?

    Carlson: Yep, I absolutely do.

    Shimel: Absolutely. And –

    Carlson: I could go into that more but I think it’s self-evident. I think that developers use those tools vastly more than operations. I think that the reasons are obvious. The tools were built for developers by developers.

    Operations looks at the world through a different lens; they use a different way of approaching the same problems and they kind of—the way that configuration management works embraces a development mindset.

    Shimel: Okay, fair enough. Let me ask—so how does this, Lucas, play into Automic and their future toolset? Will we –

    Carlson: Well, I joined Automic specifically because I saw this as the biggest problem in DevOps today and that Automic has solutions on the ops side of DevOps. Automic’s solutions are the level-up for ops teams. They don’t say—like one of my biggest pet peeves right now is this rip-and-replace mentality. You see it … with Mesosphere, with Docker containers, with Cloud Foundry, with OpenShift. All the big vendors out, there they all have the same promise: “Take everything you’ve done, throw it away, import all of your applications into our platform that will do it all for you and it will solve all your problems.”

    Well, I’m sorry, but that’s just not going to happen. Those platforms are generally underdeveloped, undertested at scale, they are a brand new way of managing applications that is untested. They don’t embrace the vast majority of enterprise scale applications architectures. They basically force you to rip and replace. And rip and replace is not just a thing for the code it’s a thing for all the IT processes that have been known to work for the last 10, 15, 20 years.

    So you’re trading a potential promise for a solution to all your problems, but the risks are exponential and vastly untested. So this whole concept of rip and replace I think is very, very dangerous in our industry today and very prevalent.

    And I joined Automic because their approach is to embrace what already exists and make it better. Instead of saying, “We’re going to have a new platform for you to import all your apps to,” they have a way to describe the topology of your existing infrastructure and built on top of that and automate that instead of throwing it out and replacing with something new.

    And so, I think to me, it really levels up the ops side of DevOps—it makes operations a hero, the co-hero of the DevOps story. It doesn’t deny operations their vital role.

    Shimel: Excellent. Excellent. You know what? Lucas, I’d love to see this at a Speak—you’ve got to submit to Speak about this somewhere.

    Carlson: Yeah, for sure.

    Shimel: Or, maybe we can do a webinar and get into it because honestly, our DevOps chat format being just 15 minutes doesn’t give us enough time to really delve in here and I’d love to get a counterview of these things too and let’s really have a discussion around it. I think I’m going to take that as my job. I’m going to go find someone and we’ll go back and forth on this.

    But unfortunately we are just about out of time—we’re at 15 minutes already. As I said, it goes quickly.

    Carlson: No problem.

    Shimel: This has been a very interesting discussion. I would call it truly an ignite or a fire starter in that you’ve opened the door here and we’re not finished with this one Lucas. We’re going to come back to it.

    Carlson: Sounds good.

    Shimel: But for now we’re going to call it a wrap. I hope if you’re listening to this you found the conversation interesting. Of course on devopsy.kinsta.cloud you’ll also see a written transcript of the conversation so you can hold Lucas to his words here and it’s going to be in writing.

    And if you have any questions, comments, feel free to also put them on devopsy.kinsta.cloud we’ll get them out to Lucas, and as I said we’re going to try to continue this discussion at a later date.

    But for now Lucas Carlson, VP of Strategy Automic. Thank you so much for really kind of blowing the doors up today on this and we look forward to seeing what the follow-up’s going to be. And success at the new position.

    Carlson: Thank you so much. I’m very happy to be here.

    Shimel: Thank you. Yeah, no, it’s a great company. We’ve worked with Automic for some time and I’ve met a lot of great people there.

    Anyway, that’s a wrap for this edition of DevOps Chat. Lucas Carlson from Automic was our guest and this is Alan Shimel for devopsy.kinsta.cloud. We’ll see you at the next DevOps Chat. Thanks everyone, have a great day.

    — Alan Shimel

  • First We Fixed Dev, But Let’s Not Forget About Ops

    First We Fixed Dev, But Let’s Not Forget About Ops

    Since its beginning in 2008, DevOps has gone from an underground rebellion by developers to a mainstream movement with representatives in every industry and at every scale. You can see the change just by looking at how people attending DevOps events dress: the black T-shirts are still there, but these days there are lots of suits and ties as well. Another hallmark of this success is companies trying to fit DevOps into their existing conceptual frameworks—hence, the plethora of “ITIL and DevOps” papers out there.

    Despite all this success, though, the DevOps movement is still young. Of course, things move fast in IT, but we aren’t talking about a technology or a technique that can be adopted rapidly and spread very widely. Committing to DevOps is a major cultural shift for organizations, and that takes time. Some organizations have been doing DevOps for five years and have fully re-engineered their processes around this new approach, but most are still only at the beginning of their DevOps journey. As William Gibson said, “The future is here; it’s just not evenly distributed.”

    First Things First: Speed Up Releases

    The early phase of DevOps adoption focuses on activities around releasing code to production, because that was the bottleneck that the DevOps movement was originally created to break. While it still may be too soon to declare victory, it’s safe to say the improvement in the release process is enormous. Today, any new project that does not propose to implement DevOps principles would have to have a very good reason and explanation.

    Notice that qualification about new projects. It’s easy to adopt DevOps methodologies and expectations in a greenfield situation; however, it is much harder in a brownfield project, where previous cultural norms have been expressed in tooling and processes that are hard to change in-flight. The good thing is that over time, the old way of doing things will die off naturally, as code bases get re-engineered, ported and replaced. Eventually, every project will be a DevOps project.

    DevOps image

    Are We Running So Fast That Others Can’t Keep Up?

    DevOps, however, is not just about release and deployment. It also covers operation of what has been deployed. It’s right there in the name, after all! Coming from an operations background, though, there can be a feeling that DevOps is not for “people like me.” Some simplistic interpretations of DevOps focus on the sometimes adversarial relationship between Dev and Ops, forgetting that good Ops people are just as incentivized as Devs to get Devs more involved in Ops.

    One of the difficulties that arise when engineers try to “do the DevOps” without involving Ops is that Ops are not ready to join in. When Dev are doing non-agile releases on an 18-month cycle, Ops have a certain set of assumptions about what a release represents for them. Doing these sorts of monolithic releases is a major pain for Ops for a while, but there will be plenty of time to get ready for it, and other required activities can be scheduled around that release.

    One of those important dependencies is preparing the monitoring and management of the code that is being deployed. What needs to be instrumented, where do those feeds need to go, what needs to be in place to make sense of those feeds when they arrive, who needs to jump when one of the feeds goes red, and so on. There is a lot going on in the NOC that Devs are not always aware of.

    If Devs suddenly show up with a more Agile cadence of a release every couple of weeks, Ops are simply not going to be able to adapt their existing tools and process on the fly to that new cadence. When you have months of lead time, it’s feasible to write and modify extensive sets of rules and filters to make sense of monitoring data, filter out the cruft, and route the remainder correctly. When your lead time on a release is measured in days or hours—not so much.

    Devs should remember that “infrastructure as code” also includes debugging that code. Adopting release automation, containerization or whatever will be the next cool tool will not solve the problem on its own. Bridget Kromhout has a wonderful articulation of this mechanism:

    Containers will not fix your broken culture. Microservices won’t prevent your two-pizza teams from needing to have conversations with one another over that pizza. No amount of industrial-strength job scheduling makes your organization immune to Conway’s Law, which states that “organizations which design systems … are constrained to produce designs which are copies of the communication structures of these organizations.

    Leaving aside the communications barriers that all too often still exist between Dev and Ops, one of the constraints that Ops teams have to deal with is that Ops is segmented into many different teams, each of which deals only with one area or one technology. These days, though, problems tend to have symptoms that cross these domain boundaries, and so operations teams have to communicate with each other.

    The historical ways of doing this are not very efficient, relying on ticket ping-pong, escalations, war-rooms and good old blame-storming. What do you think happens if you suddenly accelerate a system like that? The answer is that it just torques itself into scrap.

    Completing the DevOps Culture Shift

    The good news is that others have been where you are now, and there are both tools and techniques that can help you operate what you have developed. In common with the rest of the DevOps movement, it’s as much a cultural shift as a technological one. Ops teams need to start thinking of operations in a new way.

    One aspect of that change is the ongoing shift to immutable architecture, also known as puppies vs. cattle. Another is to enable efficient communication across support domains. Today, like in the parable of the blind men and the elephant, support teams often get hold of different bits of the problem, misidentifying them because they only have partial information, and therefore cannot truly resolve the problem. Nobody really has a complete view of the whole elephant, except perhaps for some savant who can be reached only after many escalations—and really has better things he or she should be doing.

    If Ops teams can work side by side with Dev, on the other hand, there is a virtuous cycle that becomes possible, with efficient operations removing even more friction from the development process. Communications are the key, whether that means machine-to-machine feeds, human-to-human conversation or machine-to-human interaction. Organizations that get this right reap enormous rewards in both Dev and Ops.

     

    Dominic Wellington serves as Moogsoft’s Chief Evangelist

  • Native Integration of APM Driving DevOps Maturity

    Native Integration of APM Driving DevOps Maturity

    With applications performance monitoring established as one of the fundamental cornerstones of today’s DevOps methodologies, it’s easy to overlook the reality that effective instrumentation of such tools remains extremely difficult. Specifically, the biggest challenge remains creating performance metrics that address both IT and larger business initiatives.

    There’s no question that APM is one of the most widely adopted and deeply embedded elements in today’s DevOps world, serving as the critical source of measurement that enables more transparent and agile applications. Yet, despite the fact that performance monitoring solutions, and their respective providers, are considered to be among the most mature on the DevOps landscape, customers still struggle to optimize the systems to meet their specific requirements.

    However, as the market continues to advance, specifically via direct integration of widely-adopted APM tools with other strategic applications management systems, such as e-commerce platforms, organizations are finding a faster path to success, and bigger-picture DevOps benefits. One such partnership, announced in late August, saw market leader New Relic expanding integration with e-commerce platform provider Magento, owned by eBay.

    Jon Azrielant, director of marketing at online retailer Jewelry.com, a Magento customer, said that his company initially ran into significant obstacles in adapting New Relic’s APM capabilities to its processes, despite the comparative strength of the tools.

    “We had a really difficult time implementing New Relic initially, it was a painful process,” Azrielant said.

    While the APM tools provided Jewelry.com with the data necessary to generate vital alerts, along with more agile sysadmin and operations monitoring, the more silo-based, fragmented approach that it employed could not empower the retailer to circumvent traditional walls between apps support and service support teams, Azrielant said.

    Leveraging the extension of New Relic’s Software Analytics Cloud within Magento, the company has created more advanced instrumentation to inform critical business decisions. Among the most helpful of these measurements are those relating directly to customer experiences, while correlation of such metrics with Jewelry.com’s bottom line has also advanced executive decision-making, he said.

    “The critical result of this integration is more complete visibility across applications and business performance, importantly tied to the actual business ramifications,” said Doug McIver, director of product for Magento. “In the past, merchants had to do a lot of difficult integration to use these systems together, so obviously this is a hugely valuable capability for us to be able to offer, especially to customers with highly performant site requirements.”

    Moreover, Magento claims that it is growing its own customer base by tying APM more closely to its platform. Likewise, New Relic officials concede that to help advance crucial DevOps workflows, it must partner with such platforms to deliver APM data within the context of business requirements.

    “If we’re talking about how to win in world where we’re iterating the end users’ experience on a regular basis, we have to put performance metrics within context of the UI, and the business results; working from the same data sets and getting people on the same page is incredibly important,” said Abner Germanow, senior director of solutions marketing at New Relic.

    While pure performance stats are “near and dear” to New Relic for obvious reasons, the solutions provider is also attempting to advance a broader array of information that drives closer collaboration through new “iteration and experimentation”. New Relic is also focused on further developing the specific types of information most critical to key constituents such as e-commerce platform users. The need to create a common language for those DevOps efforts is amplified by the fact that development and operations officials are often unsure which metrics are critical to the business, and vice versa.

    “Instrumenting code has always been a challenge, and we’ve worked very hard to ease that pain across a lot of different platforms; there’s always another language where work can be done, and we look to our customers to help lead us into the next phase of instrumentation,” Germanow said. “We’re seeing a lot of pressure around the need for constant change and iteration in industries where it’s truly a matter of survival, such as e-commerce, publishing, digital governance and finance – places where the industry is behaving very differently now than it has over the last 50 years.”

    This work, and the subsequent APM integrations offered via platform providers such as Magento, have allowed end user organizations to remain focused on customer experience and sales, the ultimate goal of DevOps, said Jewelry.com’s Azrielant.

    “In terms of how this relates to the DevOps revolution, this is the advancement that is bringing everything together and allowing everyone to look at the same numbers, to share common APIs, to streamline reporting and event response, and define everything in uniform terms,” Azrielant said. “Ostensibly, the most basic tenet of DevOps centers around the unification of these disciplines, and you can’t support that without the unification of this information.”

  • Code Review – Write your code right

    Code Review – Write your code right

    Writing code is definitely subjected to individual thinking, logics and perspective. How one writes code is completely depend on their technical and logical skills. Two different people can write the same code in an entirely different tactic (May be both logics of the code are right in their own way). Even two persons can write bad code as well as good code as per the same standard.

    In the real world it is most likely possible that one writes the code and same will be read multiple times by many other developers to fix the bugs, improve the features, understand the code. That is why each language has its own style of coding and guideline to make it more meaningful and understandable.

    Most of time the Developers struggle to understand the code written by someone else on the team. I have even heard people saying that it’s difficult to read someone else’s code rather writing same from scratch.

    I would like to highlight a few compiled techniques in once place to make writing code better.

    Classical Method

    Classical method is a tools based code review. Developers write their code and submit it for review in some 3rd party tool like review board, Rietveld, or Collaborator for somebody to review the code.

    The tool will be used to track code changes comments, patches and review. Someone can merge code to the active branch once it is reviewed. Reviewers can have a look at the code and provide comments and inputs to the relevant piece of code.

    The good side is that it can be used for almost all sorts of old source code tools and is very much compatible with most of the version control tools.

    The problem with this mechanism is that it needs another tool other then version control system to maintain code review related information.

    Integrated Method

    To overcome the classical method downside, review systems have been introduced with in version control tools; a very good example of this is GIT. Where there is no (or rarely) 2nd tool needed to perform code review activity. Code review can be done via pull request even before merging. Comments, issues, patches can be tracked within Version control system with same workflow.

     As mentioned earlier everything related to review remains with in the same system and that is the biggest benefit of this system.But this mechanist also needs the same time and effort for someone else to go through the code and sign off.

    Extreme Workflow

    Pair programming falls under this category. In this Technique, two programmers work as a pair together on one workstation. One writes code while the other, the observer, pointer or navigator, reviews each line of code as it is typed in. The two programmers switch roles frequently.

    This way coding and its review happen at the same time by 2 programmers simultaneously.

    Some say that 2 developers working on same code at same time doesn’t make any sense, as they will be end up spending double man hours. It may lead to problems with practical implementation of the technique.

    But the good part is that they don’t need to spend extra time to understand and perform code review. It removes to and fro communication between developers. You don’t even need another tool for additional help to submit patch.

    In conjunction with review techniques I would like to add one more point about who can review code.

    • Leaders review the code: – One of the very common processes is that 2-3 leads will review the code always from the entire team (number depends on team size). They are always whole responsible for performing the review, checking the code before it goes for further consumption.
    • Anybody can review the code: – Another technique is that anybody with in team can review the code. Only restriction is that the one is writing code cannot review his own code because this doesn’t make any sense to review your own written code. There is no leader who is responsible for code review. A junior programmer can also review code written by senior, as there is no reason to allot hard role and not allowing others to review code.

    Whatever practice we use it is very necessary to have a look at code by another pair of eyes and have code reviewed.

     

    References-

    https://en.wikipedia.org/wiki/Code_review

    https://en.wikipedia.org/wiki/Pair_programming

    http://www.scmtechblog.net

  • Librato Seeks to Unlock DevOps Application Monitoring

    Librato Seeks to Unlock DevOps Application Monitoring

    As wider adoption of DevOps methodologies continues to foster demand for new tooling to help streamline evolving workflows, at least one vendor is attempting to address the growing need for related monitoring capabilities.

    Librato, a San Francisco-based provider of Amazon AWS applications performance monitoring (APM) services, claims that it is finding favor among the rapidly expanding DevOps community driven by its ability to inject critical visibility into the engineering lifecycle.

    Acquired by Austin, Texas-based SolarWinds in January for a lofty $40 million and renamed Librato SolarWinds Cloud, the company is touting its recent adoption by well-known providers such as Slack for monitoring of DevOps-centric applications performance and health metrics.

    “At its core, DevOps is destroying conventional wisdom around ritualized processes and the need to perform them, mostly in the domain of scale,” said Dave Josephsen, Developer Evangelist at Librato. “By attempting to include monitoring as a key element of the development lifecycle, this has created a whole new range of challenges, and opportunities.”

    By providing a centralized Web interface and open API to help create visibility into critical applications engineering requirements, Librato maintains that it is creating the tooling necessary to empower integrated development and deployment processes, such as continuous change control.

    “There are a lot of disparate tools for monitoring individual engineering tasks, but there hasn’t been any practical way to bubble up that data for efficient monitoring,” Josephsen said. “Within the context of DevOps, change control is a ritual that we’ve abandoned as we’ve embraced test driven development and automated away the human element, so there’s a distinct need for monitoring of feedback and metrics.”

    For example, cloud-based productivity messaging specialists Slack are using the Librato “StatsD” lightweight daemon process to stream business and app-health metrics regarding PHP applications into the monitoring platform. Slack also leverages Librato to chart data regarding connect, query and fetch latencies for its server clusters, and to monitor internal metrics about overall system health.

    “Having those graphs side-by-side lets us clearly see and compare the effects of network latency on one of our new services, and that’s very cool,” Richard Crowley, director of operations at Slack, wrote in a recent testimonial posted to the Librato Solarwinds blog.

    Industry experts agree that broader adoption of DevOps strategies will continue to drive demand for relevant monitoring tools, with solutions such as Librato seeking to deliver “modern stack architecture for looking at the modern stack.”

    “I think that APM is always interesting because it’s always new,” said James Governor, industry analyst with RedMonk. “The pace with which we’re building new architectures to deliver services to end users is increasing all the time, and with every new wave of technology, we tend to get new and associated models of APM tooling.”

    Librato is taking advantage of the need for a system that brings together the elements necessary for cloud native monitoring, and in an agentless manner different from other competing tools such as New Relic, the analyst noted.

    However, whether or not the platform is able to break down traditional engineering siloes to enable more efficient DevOps management, as Librato officials claim, remains to be seen, he said.

    “Each wave of technology has promised to do this, to remove the inherent finger pointing, and each has always made the same claim to break down organizational firewalls,” Governor said. “DevOps hasn’t boiled the ocean in this manner yet, but there are some unique opportunities; unlike previous waves, where automation was intended to replace humans, this is now about augmenting people, and we are seeing that teams are choosing new tools from which developers can truly derive value.”

  • Awkward questions for those getting started with automation

    Awkward questions for those getting started with automation

    This blog asking awkward questions for folks jumping on the microservices bandwagon was enlightening from the perspective that many of the same questions are applicable to operations as they begin to tackle operationalizing all the things in an application lifecycle.

    For example: “What is your deployable unit?”

    Great question, isn’t it? What is your deployable unit of automation? Is it per service? Per device? Per cluster of devices? Per container?

    Wait, but there’s more from this great list of awkward questions:

    • Is it acceptable for another team to take your code and spin up another instance of your microservice?
    • Can team A use team B’s microservice or are they only used within rather than between teams?
    • Do you have consumer contacts for your microservices or is it the consumer’s responsibility to keep up with the changes to your API?
    • Is each microservice a snowflake or are there common conventions?
    • How are these conventions enforced?
    • How are these conventions documented?
    • What’s involved in supporting these conventions?
    • Are there common libraries that help with supporting these conventions?

    All these questions are applicable to devops, in particular, as you go about building out automated scripts, systems and orchestrating processes that operationalize the data center and make it a lean, green deployment machine.

    You are going to have some kind of code, whether it’s a script or a recipe or a template of some kind, and questions regarding the usage of that code is important. Can others use it? Change it? Claim it as their own? Are they useful in a general, “here’s a basic load balancing provisioning and configuration script” or are they specific, as in “this is a piece of code that provisions and configures the networking required to ensure all 18 tiers of this very specific application can communicate”?

    And as you’re developing those scripts (that code) are you documenting it? Are there <gasp> comments? Do you use a standard convention for code formatting (cause let me tell you, where you place your brackets is a matter of great technical religious significance)?

    Most of those jumping into automation head first aren’t thinking of the long term ramifications of the choices they are making right now. Proof of concepts turn into pilots that turn into systems upon which all of IT relies. Rarely are you given the opportunity to go back and put any kind of standard conventions or practices into place. That’s what builds technical debt in software and, no matter what you think you’re building with those scripts, you’re building software.

    Adopting the technical and methodological practices of agile development is a good thing, but let’s not ignore that along the way from waterfall to agile developers have learned a lot of lessons the hard way. That means we should, as practitioners of software-related activities within operations (and that means any one of the four operations groups in IT), pay attention to those lessons and try to avoid the pitfalls that developers have already encountered and, one hopes, overcome.

    To do that, don’t just ask the awkward questions; answer them.

  • Removing the Wall Between Dev and Ops

    Removing the Wall Between Dev and Ops

    Welcome to the new devopsy.kinsta.cloud Q&A series on Enterprise DevOps. In the first post in this series, I answered five key concerns for adopting a DevOps approach in a large enterprise. Now, I want to address a more specific question from an admin at a large enterprise about the silos and barriers between dev and ops …

    Q. How do you work with customers to remove the imaginary wall between dev and ops to create that DevOps environment?

    This is perhaps the core of a DevOps transformation – dev and ops teams working closely together toward common goals, rather than in isolated silos. I have seen several methods have substantive impacts on these entrenched silos, so I will take you through some of the more effective approaches.

    Familiarization

    In many organizations, dev and ops barely know each other – a huge barrier to the empathy at the centre of devops. Breaking this wall down may be as simple as getting both departments together for some social activities. Some successful approaches I have seen include:

    • Sponsor a one-off or regular shared non-work team activity, like paintball, a pool tournament, team picnic, LAN game, or cooking classes. Bring both teams together in a non-workplace environment to work together. Remember to mix up the teams – you do not want to reinforce any us vs them mentality;
    • Schedule sessions for team members to present to each other in short sessions (think lightning talks like Ignite or TedX) beyond technology to any areas of interest – like ‘My Sports Team,’ ‘Steampunk in the 21st Century,’ ‘Restoring Train Carriages,’ or ‘Strategies for TF2 Domination.’ Share common interests to create empathy for each other as people, not just functionaries;
    • Initiate informal knowledge sharing, with regular ‘lunch-and-learn’ or ‘brown bag’ sessions, where team members can explain what their role is, why they do it, how they do it. Offer a ‘Day in the Life…’ presentation, a Q&A session, or a whiteboard over pizza and sodas and your technical staff will come out like vultures, ravenous for … knowledge!

    Co-Location

    Telecommuting has many benefits, but standup meetings are hard to achieve across five time zones, ten locations and six languages. People communicate better in person, so physically co-locate teams as we at CA and others have done, to enable them to connect and talk in person, not just via e-mail or IM.

    Geo Alignment

    In multi-national enterprises central developers often push code to global operations, reinforcing the silo mentality. Align local devs and ops with local business users instead to facilitate customer-centric agile development, and enable smaller, closer, more agile teams with more immediate shared goals.

    Secondment

    Consider seconding team members into the other team e.g. for one day a week, one week a month, or one month in six. When ops learn firsthand the basics of their dev counterparts, and vice versa, they will better internalize how to work together, and the impact of each role on the overall process.

    Restructuring

    Some enterprises have combined dev and ops into one team. While often seen as an ‘anti-pattern,’ for some organizations this is working well. Others have adopted a flexible org structure with hierarchical management for administration (reviews, goals, training, etc.), but loose teaming for functional duties.

    Shared Technologies

    Beyond cultural and attitudinal shift, technology can also help. A shared toolchain for the end-to-end service delivery lifecycle can bring teams together too. Common tools for planning, testing, release, monitoring, or triage will help break down barriers with a common terminology and shared experience.

    Other Approaches?

    While there are no silver bullets, these are some ways I have seen enterprises break down these silos. But perhaps you have seen or been a part of other approaches? If so, other readers and I would love to know more, so please take the time to leave a note in the comments below, or drop me a line directly.

    And remember, if you have questions about Enterprise DevOps, let me know and I will try to answer them for you. If I can’t answer them myself, I will find someone who can! Please leave your questions in the comments below; hit me up on Twitter at @AndiMann; or send me an e-mail to me at andi.mann@ca.com.

  • ChatOps: Communicating at the speed of DevOps

    ChatOps: Communicating at the speed of DevOps

    DevOps is about knocking down barriers between developers and operations teams, and in doing so reducing organizational frictions wherever possible. And that’s certainly the spirit behind ChatOps – the term for the hyper-collaborative way of running DevOps, including operating aspects of systems and infrastructure, through online chat.

    Proponents of ChatOps say it streamlines communication in a centralized and powerful way. Born within Github, ChatOps evolved from its open source chat bot, Hubot, which has changed the way the web-based Git repository runs operations through the automation of deployment, graphing, monitoring, provisioning, and even mitigating security events.

    Think of Hubot, a Node.js app that is commonly run on Heroku, as a robot that helps keep everyone informed in a centralized place and in real-time, or in a near real-time, way. There are many benefits to ChatOps, says David Mortman, David Mortman, contributing analyst at Securosis, including the integration of ops activities with notifications into one place, making it easier for everyone to know what’s going on across the enterprise, as well as everyone throughout the organization having the ability to see error messages on events as they occur, he says.

    Github, as a highly distributed organization, with its several hundred employees spread across 103 cities, can use all of the communication efficiencies it can muster. As Mark Imbriaco, Ops engineer at Github, in his presentation ChatOps, Augmented Reality for Ops, uses Github, email, internal apps, other web systems that they’ve built to help teams communicate asynchronously – and yet they needed an option for the real-time or “semi”-real time communication. “I say semi-real time because, when we talk about chat, the social contract at GitHub around chat doesn’t require you to be immediately responsive to everything you see. Chat is still asynchronous, but it’s more synchronous than GitHub or email or other sorts of notification mechanisms,” said Imbriaco.

    Without enhancement, Hubot doesn’t do a whole lot right away. But when added to chat with one of the dozen or so connectors that include access to Google Talk, IRC, Skype, and Hubot monitors for events that come from within chat or from one’s systems and applications. “The magic happens when you build additional functionality into him to automate, well, everything,” Imbriaco said.

    Hubot will do certain things, such as notify people not present in a chatroom that they were mentioned, phrases can be registered so that notifications to those interested in certain events will be notified. Github has also plugged Hubot into the GitHub service itself. Today, when anyone creates an issue or pull request in GitHub, according to Imbriaco, a link and definition of the event is sent to chat. While Github already has ways to send notifications, by entering events real-time into the chat stream, others can jump in and comment, help, or otherwise advise on events and interact as they’re occurring.

    As for the adoption of ChatOps, Github isn’t the only one: but it’s not yet widespread among DevOps practicing organizations, either, based on our reaching out to numerous organizations. Nor, is the concept especially well known.

    That’s not slowing Owen Zanzal a developer at systems management provider VividCortex down from adopting ChatOps. Zanzal says in his blog post, ChatOps at VividCortex, that, despite ChatOps being off to a slow rise, “ChatOps has really revolutionized how we operate,” he wrote.

    The first Hubot mission at VividCortex was to help improve how code is deployed. The problem Zanzal highlights is that encryption keys and new developers apprehension around code deployment, was slowing down production.

    To streamline deployment, Zanzal moved the execution of Capistrano commands to a Jenkins (a continuous integration server), and created Hubot commands so that jobs could be created using a Jenkins exposed REST API. When done, developers needed only run a chatroom command, such as “hubot stage webapp” or to deploy “hubot release webapp,” Zanzal explained. “Now, no one is blocked by the gatekeeper from pushing code into staging or production, and every developer can push code when they are ready. The unfamiliar Capistrano commands were simplified into much easier to understand Hubot commands,” he wrote.

    VividCortex is also using ChatOps to fetch logs and more quickly remediate issues as they arise, Zanal wrote.

    Chat-as-Ops
    GitHub has also pushed ChatOps much further than simply as a place to stage and deploy code. It’s using ChatOps as a way to help actually run ops. For instance, Imbriaco spoke to how Hubot and ChatOps is used for efforts such as rebalancing shards across its file servers. “Honestly, it took me a little while to get used to some of these. We have some [Hubot actions] that do things like rebalance shards across our file server. And that’s horrifying to me, because the consequences of failure are so bad. But the reality is, I’m either going to run it from a chat room, or I’m going to run a shell script from a terminal. If I ran it here, at least everyone saw it. So at the end of the day, this is repeatable, it’s tested, it’s commonly used,” Imbriaco said. “And if it stopped working we would notice,” he said.

    And one time when Github was fending off a distributed denial of service attack, Imbriaco needed only enter the name of their service provider to direct traffic and the attack was successfully mitigated.

    Moving to ChatOps isn’t without risk, however, Imbriaco explained. A lessoned learned when it became a little too easy to deploy to Amazon Web Services. “We went from zero to way too much spend on Amazon in about two weeks, when I turned up the ability for us to provision EC2 hosts from Chat,” he said. But that issue was resolved by giving users information at the time of provisioning, such as including metadata into the instance that details who created it, as well as informing users how much it’s likely to cost,” Imbriaco said.

    Despite the efficiencies of ChatOps, don’t expect a widespread embracing of ChatOps, but more likely a slow adoption. “GitHub is a flying unicorn. They’re like Netflix,” says Mortman. “People will pick up some of what they are doing but it’s too advanced for most,” he says.

  • Is Ops the quarterback of the DevOps team?

    Is Ops the quarterback of the DevOps team?

    Without stepping too deeply into the charged debate on whether or not developers are at the pinnacle of the IT food chain, I’ve recently observed that many of the DevOps leaders I’ve been speaking with are from operations. For me it begs the question – are the ops folks quarterbacking the DevOps process at most companies? In other words let me put it bluntly, are the Ops folks leading the way forward in DevOps?

    Many DevOps thought leaders have taken great pains to describe and articulate that development, ops, qa, security and many other groups need to work closely together to make DevOps a success inside of a company.

    Some have focused in on the leadership role that developers have taken (and the associated burden that goes along with that role). For larger companies developers are often not the leads when it comes to DevOps. Developers are no doubt a key part of the equation and integral to practicing DevOps within the organization, but the Ops people seem to be calling the plays and leading the teams. This has led me to the question of who really leads the DevOps process within a company?

    Why Ops?

    As with any key initiative within a company, there needs to be a focal point, a champion or leader to advance it. DevOps is no different. So, then, how do companies organize themselves to become and execute a DevOps methodology?  Recently, we’ve spoken with three senior IT leaders – all in ops – from companies that are beyond the initial startup stage, but aren’t yet behemoth enterprises either. All three ops leaders either had the title and role of VP of Operations or VP of DevOps (without trying to get into the “is DevOps a job title controversy). Their job is to deliver code to production as fast and safely as possible. This meant that they are directing, helping even conniving their counterparts in engineering to deliver smaller batches of code faster and on a regular basis. They then push this code out to production. In essence they were playing the role of the quarterback of the DevOps process.

    Just as the quarterback directs the players, calling the play in the huddle and directing the offense, these Ops folks are quarterbacking the process of DevOps.

    As such, these senior leaders were building the overall product delivery pipeline – from ideation all the way through deployment and feedback. While the ops team may not be involved in each part of the pipeline, these leaders were integral in building out the product delivery pipeline. For most organizations that product delivery pipeline is starting with sales and marketing – what needs are the business trying to solve in this particular iteration of the product or service. From there, the various teams are understanding the implications for their particular piece of the pie. Engineering is understanding how to build the features and functions. QA is understanding how to test the functionality as well as automate the QA process. Security is making sure that the functionality doesn’t create vulnerabilities or security holes. And, ops is thinking through how to push all of these features out to the customer. As in any DevOps organization, there are a lot of moving parts with many pieces needing cross functional coordination. And, therein lies the question – we all may practice DevOps and should all practice it; but, who leads the effort?

    Clearly other senior leaders could take the lead on this activity. There is frankly no reason why the VP of Product or VP of Engineering could take the lead, but it has been an interesting observation that I have been seeing more and more ops leaders taking the reigns of DevOps at mid-sized organizations. We would really appreciate hearing who you have seen taking the leadership role in DevOps – in your organization or elsewhere. Let us know by commenting below!

  • Let DevOps be a business process

    Let DevOps be a business process

    I just returned from an amazing time at the first Camp DevOps in Boulder, CO. It was held at University of Colorado, Boulder and in conjunction with Gluecon 2014. This is the first conference I have attended that was singularly focused on DevOps.  I found the three track format at the show had something for everyone. A technical track, a business track and some great DevOps round table discussions with great panel speakers. My takeaway was that DevOps is not just a cultural thing as some claim. DevOps needs to be perceived more as a business process or it will just be a passing fancy. Some may disagree with that view but 35+ years of IT industry experience gives me a bit of a long view.

    Gene Kim enlightened me on some issues in his recent response in the Wall Street Journal with some facts about what is going on within traditional legacy Fortune 500 companies. They are implementing DevOps strategies in a big way. My take is that it is not noticeable within the Fortune 500 because DevOps is viewed by the Fortune 500 as a business process, as opposed to being an implementable organizational structure. Sort of like the DevOps is not really a job title argument.  If it is a business process, no one has DevOps in their title, but it doesn’t mean DevOps is not being done there.

    I think most of us want to see DevOps cross the chasm and become a de facto standard. Since Patrick Debois first coined the term “DevOps” there is no question that DevOps has developed into a global movement. Of course DevOps has its cynics. Pessimists say DevOps is a new name for a Sys Admin,  the flippant say DevOps is just a front for some fanatical developers trying to liberate the world from operations or DevOps is just some foolish operations people trying to make nice with Developers. That it means the extinction of QA job is another familiar mantra.

    I’ve spent a lot of time trying to understand the lack of cohesiveness within the DevOps movement and everyone telling everyone else what DevOps is and what it should be. As with anything else the marketplace will decide the future and fate of Devops. During this process, I’ve begun to notice a common misconception that I think is fueling much of the reactions that some people have to DevOps ideas. Holding DevOps back is not a technology problem or a cultural problem.

    In order for DevOps to happen I realized we all need to keep pushing this philosophy forward.  We cannot worry ourselves with culture, commercialization or technology. Technology plays a critical role in facilitating solutions to DevOps problems. However, DevOps itself is, at its core, a business issue. The most fundamental business process in any company is getting from concept to completion with the end result of profitability.

    If you remove the perspective of the business process, what is left? You’ve got a whole lot of people and groups doing their own thing. Lost is the motivation to battle inefficiency, duplication of effort, turf wars, and disparate groups. It’s a free for all.

    If we don’t let DevOps take its course within the market place and embrace the business process, jobs are lost. We are able to do what we do because of the business of DevOps. If we don’t all push the commercialization of DevOps then we have nothing but a chat room to talk philosophy.

    The sole purpose of DevOps is to enable, big and small alike, business to react to market forces quickly, efficiently, and reliably. Without the commercialization, there is no other reason for us to be talking about DevOps problems, much less spending any time solving them.