devopsy.kinsta.cloud

Tag: microservices

  • DevOps Lessons from the Dyn Attack Fiasco

    DevOps Lessons from the Dyn Attack Fiasco

    Dyn’s DNS servers suffered a major DDoS attack last week, slowing a host of major websites. Does the drive for DevOps automation increase the risk of similar attacks? Maybe, but it doesn’t have to.

    As almost every geek knows by now, a DDoS attack on Oct. 21 overwhelmed Dyn’s DNS servers, and websites that depended on those servers slowed to a crawl. The attack was made possible because a large number of Internet of Things (IoT) devices were compromised using Mirai, a malware tool that breaks into devices using default access credentials.

    While it’s not yet clear exactly why so many IoT devices were configured with default login credentials, it’s likely that at least part of the reason included the following factors:

    • IoT devices are so numerous that securing them all is very difficult, especially when they are configured with weak security out of the box.
    • The need to give multiple users access to the devices encouraged installers and service providers to keep the default credentials in place. For example, if you have a smart thermostat that is installed by a contractor, managed by a utility company and owned by a homeowner, each of those parties could potentially need access to the device. Sticking with default login credentials is a lazy but effective means of providing it to all of them.

    In certain ways, the shift to DevOps makes challenges like these more common. Consider the following:

    • DevOps encourages the use of microservices. That means more services and logins to manage. It increases the risk that some services will not be secured properly because default login credentials will remain in place.
    • Under DevOps, the entire software delivery team is supposed to have access to all platforms and resources that are part of the delivery chain. That’s different from waterfall development, in which developers only had to be able to access developer tools; IT ops only needed to access management tools; and so on. The need for broader access could encourage teams to take the easy way out by using default logins—or even just sharing login information, which is risky even if the credentials are not the product defaults.

    A shorter way of saying this is that to increase agility, DevOps increases complexity. Greater complexity translates to more potential security risks.

    Conclusion

    The Dyn attack was not a DevOps problem, and none of the above means that securing a DevOps environment is not possible. It is, but it takes more work.

    Still, the Dyn outage is a reminder of just how badly things can go if developers and admins make mistakes such as relying on default login credentials. This challenge will become more difficult, not easier, as DevOps continues to reshape the way software is written and deployed.

    — Chris Tozzi

  • Getting Ready for All Day DevOps

    Getting Ready for All Day DevOps

    DevOps Practitioners Unite

    On Nov. 15, we’ll be broadcasting 54 live sessions across 15 hours and 15 time zones. The All Day DevOps conference will be live, online and free. Anyone who registers can watch.

    We have invited some top-notch practitioners from around the world to speak at the event on topics ranging from DevOps transformations, to CI/CD, to automated security, to modern infrastructure (think serverless, microservices and containers). The speaker lineup will include:

     

    www.alldaydevopsy.kinsta.cloud

    Don’t Watch Alone – Host a Viewing Party!

    One of the best parts of attending a DevOps conference is getting to interact with others in the community. We have had many organizations including Capital One, Intuit and Fannie Mae approach us about hosting viewing parties at their offices—and opening the party to the public. Sound intriguing? Here’s how to do it:

    Secure the location. Pick your corporate auditorium, cafeteria or open meeting space. Secure it all day for Nov. 15. Make sure you have at least one internet-accessible viewing screen.

    Promote the session: One you have the approved locations and times, share them here with the All Day DevOps Conference organizers. We will help you promote your party on AllDaydevopsy.kinsta.cloud and through social media postings, directing them to your registration page.

    Invite People to Register: Set up a place for people to register for your location. You’ll want to know how many people to expect because you need to know how much pizza to order. Meetup.com and Eventbrite.com make event registration easy.

    Make it Fun. Order some pizza and drinks. We’ll send along some cool laptop stickers from the conference and some of the sponsors. Share pictures from the event online (@AllDayDevOps). Join the conversation in the conference Slack channels. Set up birds-of-a-feather stations. Play some music during breaks. Set up viewing screens for each of the conferences three tracks.

     

    Just to be clear: the All Day DevOps Conference is not a collection of vendor pitches.  In fact, we have tried our darnedest to ensure the agenda is purely focused on real DevOps practitioners sharing their craft.  Take a moment to look at the speakers participating in each of the tracks and you’ll see what we mean.

    Register today. Invite your colleagues to attend.  Then plan a viewing party at your office.

    — Derek E. Weeks

  • The devopsy.kinsta.cloud Containerization and Microservices Survival Guide

    The devopsy.kinsta.cloud Containerization and Microservices Survival Guide

    Containerization and microservices seemed to have leaped out of nowhere and smack into the center of the enterprise IT scene. But neither leaped from nowhere. Both containerization and microservices have been part of the fabric of enterprise tech for some time—within Linux for containers and service-oriented architecture for microservices.

    Largely, thanks to containerization, both now are moving quickly to become central components of enterprise tech. Containers in recent years—and especially in recent months—have moved from tools used in development environments to production. And the ability to manage containers as discrete services is enabling enterprises to be able to respond quickly and be agile to technology demands.

    According to a recent survey from app performance vendor NGINX, of roughly 1,800 IT professionals, while container and microservices software adoption is limited it’s use is accelerating. The survey found that only 20 percent of respondents said they are using containers in production, while about one-third are doing the same for microservices.

    That survey, not surprisingly, indicated that microservices and container use will continue to accelerate, as roughly two-thirds of respondents said that they are currently evaluating containers and 70 percent microservices.

    Microservices are workloads that perform a single task and they are not dependent on the infrastructure or other applications for their ability to complete their designated task or “service.” Microservices are very effective at providing enterprises the ability to rapidly build software that solves business problems. They also make it possible for development teams to scale their efforts and build a more resilient architecture.

    With all of that in mind, and considering the nature of the inflection point most enterprises find themselves in now, we’ve assembled this microservices and containerization survival guide. The guide consists of some of the most important storage on microservices and container management that we’ve ran in the past year.

    DevOps Teams: Containers Will Help You Become Future Ready

    According to The State of Containers 2015 Docker Adoption Survey, more than 70 percent of respondents are either using Docker or evaluating for future use. Given the overwhelming popularity of containers, this post provides three trends that DevOps teams must be prepared for.

    7 Web Interfaces for Container Management and Orchestration

    Managing containers used to be a command-line affair. But a rich and growing ecosystem of web-based interfaces for containers has now emerged. This post provides an overview of the different web UIs available for container management and orchestration.

    ‘Containers-as-a-Service’ Is a Hot Trend

    Containers are still sort of the new kid on the IT block, but already containers-as-a-service is emerging as a thing. Here’s what you need to know.

    A Closer Look at Container Monitoring

    Enterprises have to ensure that all of their microservices perform well and provide their services as designed. Beyond that, how much do they really need to know about what’s going on inside their containers? The answer to this question varies. In this post Gerald Haydtner provides an overview for container-monitoring options.

    Containers: Not One to Rule Them All

    When it comes to the container platform ecosystem, not all container platforms are created equal. This post will help you sort through the options.

    Building Microservices: A Few Considerations

    An experienced practitioner details the top five design considerations when building microservices.

    5 Docker Best Practices You Should Follow

    Support for Docker has been integrated into many products and platforms and a lot of organizations are either already using Docker containers or trying to understand how to do so themselves. Our Tony Bradley provides the Docker best practices one needs to know to succeed.

    How to Choose a Cloud Platform for Containers

    Containerization has broadened from one or two vendors to a broader, more container-agnostic world and a drive to better scalability, redundancy, interoperability and security. Amazon, Google and Microsoft each have their own unique approach to addressing these cloud container concerns as they battle each other for supremacy. Here’s a look at how to evaluate each.

    Container Security: Why Bother?

    Are containers secure? Probably not. But knowing each approach’s strengths and risks—and available options—could help ease adoption.

    Container Security Challenges: Are Container Scanners Enough?

    How secure are containers? Not very, according to open source company Red Hat, which has published a blog post blasting the container ecosystem for lacking good security processes. Here’s a synopsis of what Red Hat has to say about container security, and some additional perspective.

    For Container Security, Automation Is Key

    Regardless of the IT infrastructure element in question, if it connects to a network—even through proxies or human intermediaries—eventually someone will find a security flaw and exploit it. Compatibility issues with new applications or infrastructure elements will emerge. Because of these reasons, all IT infrastructure elements need to be managed, without exception.

    Containers and Security Q&A: Putting a Lid on Risk

    How do containers affect security? And how can IT teams do everything they need to do in order to ensure the containers in use are as secure as they reasonably can be?

    — George V. Hulme

  • Simplify and Streamline Hybrid Cloud with DevOps

    Simplify and Streamline Hybrid Cloud with DevOps

    The cloud is not new. The virtues of the cloud have been proclaimed incessantly for years, and most businesses have embraced cloud servers and services on some level. The realities of data security and the need to leverage existing investments in IT infrastructure, however, generally result in some sort of hybrid cloud setup, with servers and applications that span both local data centers and cloud providers. Hybrid cloud deployments can be complex and volatile, which is where DevOps comes in.

    I wrote a report recently for devopsy.kinsta.cloud that takes a closer look at the pros and cons of hybrid cloud, and the symbiotic relationship with DevOps. “DevOps Enabling the Hybrid Cloud” examines how DevOps tools and practices streamline management of such an environment. It also looks at some of the security implications and how DevOps can address those concerns.

    The report explains, “Speed and flexibility are imperative for organizations that want to remain competitive. Organizations that strive to lead understand that IT must be viewed as a strategic advantage—a powerful tool for driving innovation and creativity. The hybrid cloud provides a bridge from legacy hardware and assets to cutting-edge concepts such as containers and serverless computing for organizations that do it right.”

    One truism addressed in the paper is that customers—whether they’re external customers of your company or the users within the company that rely on IT services and applications—don’t know or care where your applications or data are delivered from. They know that they want stuff to just work when they need it to work. It’s up to IT to determine the most efficient and effective way to achieve that goal, and to provide a seamless, frustration-free experience. Sometimes that means relying on cloud servers and services, and sometimes it means using local infrastructure.

    I spoke with Justin Foster from Trend Micro about the overlap of hybrid cloud and DevOps, and specifically about unique security issues with hybrid cloud environments. He stressed that legacy tools and traditional approaches to security are not sufficient for the more dynamic hybrid cloud environment. As explained in the report, “Security has to be baked in from the beginning and throughout the development life cycle.”

    A pure cloud infrastructure is unlikely for any established company. It may be achieved eventually through attrition, but there’s no getting around the fact that there already are servers and applications in place. The hybrid cloud approach is the right solution in most cases, and DevOps tools and practices are essential to help manage and secure the hybrid cloud effectively.

    Click here to download the report: “DevOps Enabling the Hybrid Cloud.”

  • Why Is There No DevOps Manifesto?

    Why Is There No DevOps Manifesto?

    Let’s start with a brief story:

    There’s a scene in a movie called “A Few Good Men” in which a lawyer is questioning a cadet. The lawyer directs the cadet to turn to the page in the thick Marine Code Manual that explains where to find the canteen. And the cadet replies, it isn’t in there.

    And the lawyer says, “Well, you eat, don’t you?”

    And the cadet says, “Yes, sir, three times a day!”

    The meals for everyone aren’t served by the book because the canteen isn’t included.

    This isn’t a history for the term DevOps or of the many people who’ve led, presented, written and interviewed about the DevOps movement. I’ve got a footnote below with some pointers for that. This is simpler than that: It’s why today there is no DevOps Manifesto and why I don’t think there will ever be.

    Now, I think I understand the desire for a DevOps Manifesto. I’ve met a lot of folks, company executives and analysts, who still dis DevOps a bit because it lacks a certified definition, an official manual or a definitive statement. Basically, a manifesto; you know, a canonical thing you can put on a shelf.

    Manifestos can be nice. They lay down the vision, the intent, the mission. They can appeal to a wide range of people, from revolutionaries to functionaries to dignitaries. It isn’t for sparking conversations so much as for orderly setting down of intent. A manifesto can bake in the meaning and content of something.

    A manifesto has that “rule of law” book feeling to them. Manifestos also have provenance and pedigree—some group of some renown sat down to discuss some topic for some specific resolutions and wrote the sum of it up. Manifesto wording can be checked to ensure real-world compliance.

    You can almost slap a copy of a Manifesto on a table to settle an argument about the manifesto’s topic. And assuming the words were crafted carefully enough, it will be the end of discussion. And that’s exactly the problem with a manifesto with respect to DevOps.

    I say: That’s not what DevOps is about.

    DevOps begins with stopping the nightmare scenario pitting Ops against Dev—or any other part of the organization. But it comes so much further. It’s for a new age of computing at webscale, using tools, teamwork and techniques to get to better, happier, more engaged outcomes for customers. Some of it is so new we are still figuring out words for it.

    The DevOps movement isn’t slowing down, either. Analysis of the Google results page shows it:

    DevOps searches over time Sep2015

    In all of this time, no one of any significance in the community has attempted to create a “manifesto for DevOps.” I’d say the Godfather of DevOps himself, Patrick Debois, was super insightful on how he went about ensuring no official manifesto could get written. (I’m not going to review the lessons learned from “The Agile Manifesto,” although good lessons were learned from the things created in the wake of having an Agile Manifesto.)

    If you have a manifesto for something, then all conversation in the domain will, rather quickly, go back to the manifesto for resolution instead of everybody working together. That’s not handy in an ever-fluid, often on-fire, scale-up fast, resilience-oriented, rapidly changing and evolving IT situation.

    As Andrew Clay Shafer (@littleidea) said recently, DevOps is about optimizing for people. With the pace of IT and tech ever accelerating, no manual or manifesto could keep up. Which reminds me of one of my favorite quotes:

    Companies die because their managers focus on the economic activity of producing goods and services and they forget that their organization’s true nature is that of a community of humans.

    — Arie De Geus

    One thing you should know for sure about DevOps: The community likes to enhance, educate, enliven, inform and energize people. Basically, everyone is wide open to embracing anything that helps out because webscale is hard. So yeah, DevOps now aligns with “microservices” because microservices can further the DevOps cause. Just like kanban, Docker containers, automated configuration and InfoSec. And anything that enables Dev and Ops to deliver more, better, faster to customers. It’s about serving the customers.

    Meanwhile, if you’re looking for more, you’ll usually find a bunch of folks … figuratively speaking … over in the kitchen at the canteen, serving up awesome and working hard on doing it better and faster.

    And everyone interested is welcome to sit down and enjoy what’s served.

    Seriously. Join in. You do not need a manifesto to.

    P.S.  It was with deep hesitation that I wrote this. There are simply so many awesome people who’ve made all this happen and I am not one of them. Literally the finest collection of technologists—and in many cases, human beings—I have the honor to know who make this community what it is.

    You should check out, in no particular order:

    • The amazing (and amazingly modest) Patrick Debois
    • The writings and teachings of Mark Burgess (one of the most under-recognized geniuses I have encountered)
    • The CALMS write-ups by John Willis (@botchagalupe) and Damon Edwards (@damonedwards)
    • Damon’s five-minute video on DevOps history
    • John’s frank exploration of the issue of burn-out in IT (and that is directly related to why we are having DevOps conversations)
    • Mary Poppendeick’s lean development work
    • John Allspaw and John Hammond’s 2009 Velocity keynote that sorta kicked this whole thing off
    • Ben Rockwood’s (@benr) comprehensive LISA11 keynote
    • The many videos of Andrew Clay Shafer (@littleidea) and his brilliant guidance. Sometimes with katana and sometimes with gattling gun.
    • Jez Humble’s books (particularly “Continuous Delivery” and “Lean Enterprise”)

    And works from folks including:

    • Jeff Sussna (@jeffsussna)
    • James Turnbull (@kartar)
    • Adam Jacob
    • John Vincent (@lusis)
    • Bridget Kromhout
    • Adrian Cockcroft
    • Gareth Rushgrove
    • Kris Buytaert
    • J. Paul Reed
    • Ernest Mueller
    • James Wickett
    • Ines Sombra
    • Karthik Gaekwad

    I mean, this is a global phenomenon, were you expecting a short list? This isn’t even complete!

    Oh, and the VisibleOps guru Gene Kim, who came and showed us all a way to really blow the doors off the DevOps news.

    These people are helping all of us do better Dev, better Ops, better IT, better business so that you can have better customers. It’s amazing. It really is. And it’s wonderful.

    Also, if you get a chance to spend time at a DevOps Days anywhere, you should.

  • Streamlining Cloud Tools, Platforms Management

    Streamlining Cloud Tools, Platforms Management

    DevOps and microservices have transformed the way organizations develop, deploy and manage applications—as well as the underlying infrastructure required to support it, be it cloud on on-premise. On the one hand, DevOps and microservices enable much greater speed and agility. On the other hand, that speed and agility come with a heaping side of complexity.

    It’s a beautiful thing when all of the elements of DevOps development are working in harmony. The diverse array of tools and platforms utilized, though, often are missing key security features and capabilities IT teams need.

    Apcera has a comprehensive portfolio of open source projects that provides customers with the ability to effectively manage and secure applications across various platforms and cloud infrastructures. The Apcera portfolio includes, among other things, NATS, a cloud-native messaging system; Kurma, a flexible container runtime; and Libretto, a virtual machine provisioning library. The company recently announced that it is extending its platform to also support Kubernetes.

    “Apcera … allows customers to securely move to any cloud, and deploy, orchestrate and govern the widest range of workloads on any infrastructure—including on-premises,” said Derek Collison, Apcera founder and CEO, in a press release statement. “Customers and developers are rapidly aligning behind Kubernetes as their API of choice when deploying container-based workloads. With its fast-growing ecosystem and strong market momentum, this was the obvious direction for us to take.”

    Kubernetes is an open source container orchestration platform, originally created by Google and now managed by the Cloud Native Computing Foundation (CNCF). Kubernetes enables organizations to manage large collections of containers as a single system. Embracing Kubernetes expands Apcera’s sphere of influence to include hybrid cloud management. Apcera also will actively contribute to the Kubernetes open source project.

    “We believe delivering trust to the cloud is critical, and Apcera has deep and proven expertise in this area,” said Kelsey Hightower, developer advocate, Google Cloud Platform. “We welcome their support of Kubernetes and future participation in the project.”

    Containers in particular have been a major security concern. A blog post from Gartner noted, “Security properties of containers are a largely unexplored field and there is a lot of controversial discussion about whether containers do contain or not. At times it seems that the discussion is driven by (hidden) business agendas, partnerships and financial dependencies rather than by plain technology. So, leaving all of this aside, can you make your containers contain or not?”

    A lot of progress has been made in the past year toward more secure container technologies. Hyper-V Containers from Microsoft, Clear Containers from Intel, and recent improvements from CoreOS are all examples of the fact that the container world is aware of—and vigilantly working on addressing—the security problem.

    Organizations need to understand how secure their infrastructure is, and be able to answer questions such as, “How can I effectively move into production quickly and with scalability?” Streamlining management of diverse cloud tools and platforms enables IT organizations to balance innovation and speed with safety and stability. Rolling Kubernetes support into the Apcera portfolio may be the key to effectively managing and protecting applications and infrastructure in a hybrid cloud environment.

  • DevOps Debates: Monoliths or Microservices

    DevOps Debates: Monoliths or Microservices

    From the time software came into existence, application software has been booming to leverage the benefit of the computing paradigm. During this cycle, it has transformed decades of design and architectural thinking, and has crossed all aspects of application development and management needs, be it configurability, reusability, maintainability, scalability or security (a good list is of these “ilities” is here).

    While many have been addressed, one “ility” that has turned a lot of eyes and attention in the last few years is deployability. More than that, it is the ability to do so continuously. This requirement has changed the thinking of building software for two main reasons: (a) Businesses have started asking to release software faster than ever, even in verticals unheard of before; and (b) Cloud and infrastructure as code has made faster deployment a easier possibility. When you look at application design with deployability in mind, given the need to be agile, what you end up getting is a microservice.

    Most applications of old style addressed most “ilities” in an architecture but did not consider “continuous deployability” as a core need as well. I call these monolith from a deployment perspective. They are tightly coupled, and for any small change a complete build of the application was necessary.

    Now the question is, Should you really look at microservice as a silver bullet to deploy all software? Should you consider that as the de facto standard to deliver any application functionality? Will applications cease to exist the way we know? Here are my thoughts. As usual, feel free to add yours or write to me personally, as some of you do as well.

    Applications as monoliths will stay as they are

    1. Monoliths have evolved over generations of design and architecture tests and stood the test of time. The focus on modular development is, in fact, a way of creating modules, which are built with separation of concerns in mind.
    2. Monoliths were built to run for a long time. It had addressed some of the “ilities” well before they became considerations.
    3. Applications will remain easier to debug, as most of the debuggable areas are in developer’s hand, instead of having to work with a team.
    4. Applications will perform better, as they can share data in memory; they don’t have to do out-of-process calls to get one task executed.
    5. Applications will be more secure, as they will be a black box from an end user perspective or even a hacker perspective, with little coming out from the system.

    Microservices will be a monolith-killer all the way

    1. Deploy, deploy and deploy to correct your problems away. Fail fast and fix faster is the mantra that you can practice.
    2. It helps bundle two-pizza teams owning microservices and hence embrace better ownership and faster response.
    3. The overall architecture to deliver an end-to-end use-case may become complex, but each piece of the jigsaw puzzle can be handled faster.
    4. In the faster-moving world, business wants to try a concept rather thinking through all the “ilities” ahead of time, and progressively make everything better. Microservices helps abstract all complexities much better.
    5. Microservices will help developers think out of box from day one, while helping business deliver smaller chunks, without having to make the system a “big ball of mud.”

    At the end of the day, irrespective of the approaches, the key is to ensure the applications are managed well, changes are refactored well, the code that is written is readable and maintainable, and significant test assets are created to regress the entire system quickly. So whether it is a large system or microservices, you are able to make any necessary changes quickly and can deploy fast.

    While microservices surely will help this new faster-deployment thinking, I—and I am sure you all—have also seen large monolithic systems that are making delivery at an astounding pace.

    Thoughts?

  • Microservices: Easier Than You Think

    Microservices: Easier Than You Think

    “Microservices are easier than you think and more valuable than you’ve heard.” Those are the words of Ben Gracewood, chief delivery officer at cloud POS and omnichannel retail provider Vend. His company has used microservices to trim a monolithic PHP application, improving performance and changing the way the company approaches updates.

    Approaching Microservices

    Vend is currently used by 15,000 retailers in more than 180 countries. It is aimed at inventory-based retailers—think housewares, fashion, grocery and so on. In addition to the usual POS and payments facilities, Vend handles loyalty schemes, gift cards and online storefronts. The company is based in New Zealand, with offices in London, Melbourne, San Francisco and Toronto. Engineering and DevOps is done in Auckland.

    Its global customer base means there is no maintenance window, and so live updates are essential.

    Vend’s HTML5 and iOS clients include caching and a local database to allow offline operation (the data is subsequently resynchronized), primarily to allow users to keep working in situations where there is no Internet connection. The main system was originally a monolithic PHP application.

    Vend executives had discussed the idea of migrating to microservices for about a year. But when the company’s vice president of Architecture experimented by writing one read­-only microservice that improved performance by orders of magnitude, “This was the ‘a-ha’ moment for us,” Gracewood says.

    Later, when Vend migrated from Rackspace to Amazon Web Services, it used a routing microservice that directed traffic to whichever cloud held that particular customer’s data at the time. That allowed a progressive migration with no disruption for customers.

    Those initial successes showed Vend it should move more seriously in that direction. Nine months later, the company had 12 microservices in production.

    Vend is now adding them at the rate of about two per month, and the monolithic PHP system has been slimmed and surrounded by a cloud of microservices. An Nginix-based routing layer allows existing URLs to be redirected to microservices where appropriate.

    The company has followed DevOps practices from the outset, but its first few microservices were deployed and managed as special cases. Now, both the PHP monolith and all the microservices are within the DevOps tent.

    Adoption Dos and Don’ts

    When it comes to adopting microservices, “It’s never too soon to start,” says Gracewood. “It doesn’t have to be all or nothing,” despite what you might hear from other microservices veterans—everyone has been partway through their journey at some stage.

    Gracewood’s advice is to build your first microservice and do what you have to in the development of your DevOps practice. Then keep iterating.

    Don’t fall into the trap of thinking a microservice has to be 100 percent complete before it can be deployed, he says. Start by developing the minimum functionality required at the outset, and then increasing its sophistication over time. Since a change to a microservice affects a relatively small part of the entire system, the continuous update model is easier to apply to microservices than monolithic applications.

    That point also has an organizational impact: A microservices architecture and DevOps support Gracewood’s philosophy of trusting engineering and development teams to implement the right things at the right time. With monolithic software, a mistake or a poor decision can have wide-ranging repercussions, so considerable oversight is required. But the nature of microservices keeps issues localized, so if an issue does somehow slip through the testing process, it is easier to revert to the previous version.

    Adopting DevOps and microservices does require a certain amount of education, Gracewood notes. Engineering teams must understand the command line, Docker and so on. Making the move calls for “lots of tooling or lots of training,” he says. To that end, Vend has built some custom tools.

    Gracewood’s final tip for organizations thinking of adopting microservices: “Log everything [and] apply metrics to everything,” right from the start. You need to know how people are using your system. In particular, identifying the most heavily used microservices will reveal the areas where efforts to improve performance are likely to yield the best returns.

  • Flash Mob Inflection: Rugged DevOps Revolution

    Flash Mob Inflection: Rugged DevOps Revolution

    Truthfully, I was never a huge fan of the HBO series “The Sopranos.” It’s not that it wasn’t entertaining; I just didn’t agree with the “best ever” label that so many espoused during the show’s halcyon days.

    This had something to do with living in Hoboken, N.J., at the time and feeling the show was a heavily glamorized caricature of the real-life Mafioso I still perceived to be in my midst. More specifically, it’s hard for anything mob-related to stand up next to “Goodfellas” and “The Godfather”—among my favorite films, ever.

    Either way, I currently find myself identifying with the series’ defining sentiment (copped directly from The Godfather, but hammered home as The Sopranos main theme): “Just when I thought I was out … they pull me back in.”

    It wasn’t that I assumed that I’d never delve back into the domain of IT security after transitioning into the DevOps market last year; leveraging DevOps methodologies to improve security, policy compliance and related workflows stands as a core element of the overall movement.

    But, in recent months and weeks it has become increasingly obvious that the security facet of the larger DevOps revolution appears poised for explosion as the buzz around emerging best practices, business models and solutions providers grows from a low hum into a steady roar.

    After spending a few days talking DevOps security with Derek Weeks of Sonatype at CA World ’15 (we were booth neighbors and dinner mates) and giving more consideration to his company’s angle on securing the systems development life cycle (SDLC) supply chain, the internal gears really got spinning.

    Regardless of whether you buy into Sonatype’s security-heavy approach, the concept of aggressively and proactively addressing security across the DevOps spectrum makes infinite sense. The ultimate downfall of applications security has always been the challenge of securing all the underlying layers of code (and today, microservices), especially as applications are assembled using heavier doses of shared, open source componentry.

    As further evidence, nearly every major solutions vendor focused on the DevOps space—whether oriented toward the vibrant application performance management (APM), configuration management and continuous testing segments, among many others—seems to have launched some initiative aimed at integrating security into their products.

    The rise of security capabilities adjacent to—and supporting inclusion into—the DevOps world, such as container giant Docker’s Content Trust initiative, along with the advancement of related startups including Twistlock, Scalock and StackRoxs, offers more proof of this growing momentum. And that’s only the container security piece of the puzzle.

    Of course, there are also a number of emerging startups aimed at addressing the DevOps security opportunity, backed by well-known venture capitalists. A shortlist might include providers such as Immunio, Prevoty and UpGuard (formerly known as ScriptRock, which recently rebranded itself under the guise of a “DevOps security” provider), and we’re sure to see many more. And don’t forget the growing emphasis on DevOps security among established security industry stalwarts such as CyberArk, Snort and Tripwire, among others.

    For a comprehensive list of the various types of DevOps security best practices and tools that we should expect to see gain wider adoption, this October 2015 blog by noted security analyst Adrian Lane of Securosis serves as a basic framework. Each of the areas of focus that he cites clearly offers significant opportunity for both new and existing methodologies.

    Finally, one need look no further than this year’s RSA Security Conference, to be held in San Francisco at this end of this month, as tacit proof of the continued growth of what people now varyingly refer to as DevSecOps or Rugged DevOps (I prefer the latter as it just sounds cooler).

    In addition to multiple conference sessions, notably a “DevOps Throwdown” between security industry veterans Caleb Sima, Chris Wysopal and Gary McGraw, there’s also the 2nd Annual DevOps Connect: Rugged DevOps Edition. If last year’s event is any indication (standing room only, venue literally at max capacity), this daylong DevOps security track at RSA will be an even more high-profile element of the entire week.

    Further, if DevOps security—or, at the very least, the growing focus on adjacent segments such as container security—isn’t the unofficial theme of this year’s entire RSA show, I’d be extremely surprised.

    Meanwhile (self-serving), my employer CA Technologies absolutely has a lot going on that complements and contributes to this growing DevOps Security market momentum.

    For all of those reasons, here I find myself again, drawn closer back into the security space that I’ve called home for roughly the last decade. I guess that’s just how it goes, as DevOps and security are inarguably two of the most significant trends in IT these days and inextricably linked at their respective cores.

    There are definitely worse fates out there. But, just when you think you’re out …

  • DevOps and the Challenge with Services

    DevOps and the Challenge with Services

    At RackN, we’ve been working for years to make DevOps portable across platforms. For us, that means the ability to move between clouds and from cloud to metal. Of the several challenges to portability—composability, API heterogeneity, sequence, cross-system dependencies—making configuration management tools work with services has been particularly important, as significant parts of building systems rely on services that are not directly configurable.

    Let’s look at DNS as an example. Whenever we configure services on a node, there are DNS implications. If we bring up a web server, we want to use a logical name for the node. If it’s a secure web server, that name should match our certificate. If we want to make that service discoverable, we likely want to add a logical name for that node, such as “clustermanager6.example.com” or similar. If you include names for BMC and rack location—not to mention IPv6 naming—it’s easy to image nodes with 10 names. Even cloud nodes have similar issues. That’s a lot of entries to manage!

    The problem with all those DNS names, and similar services, is that they are NOT configurable from the node. They are services with APIs that require restricted authorization.

    Services and Automation

    DNS is just one simple example.  System-level DevOps has many places where the configuration workflow needs to include external service actions. In cloud environments, generally that means either writing pre- or post-configuration steps that talk to the specific cloud API or using one of the cloud-specific template files. In physical environments, generally that means a lot of added manual steps or tightly constraining automation to inventory map files. No matter what, nodes don’t own DNS configuration—it’s a node external service with the API elements that are site specific.

    All of these approaches make DevOps brittle and site-locked. Five years ago that was okay, when choices were more limited and scripts were hand-crafted by DevOps artisans. Today, we need our automation to be robust, portable and transparent.

    We realized system automation needed to treat services interactions as equal to configuration. That means when we’re working on provisioning, we make sure service steps can be woven into the sequence of configuration steps. They must not be assumed correct or delegated to pre- or post-conditions because that makes it impossible to test and re-create working environments.

    What can you do? We’ve found service registration tools (we like Consul.io) are essential and relatively easy to implement to help track services that we manage or proxy. More broadly, we’ve been working in the open on Digital Rebar to blend configuration and services into a single workflow.

    The first step is to re-examine your configuration management scripts (and their prerequisites) for hidden service API interactions. Exposing those external dependencies will help you improve your automation repeatability and portability.