devopsy.kinsta.cloud

Category: Features

  • Gartner gives Docker security an ambivalent thumbs up

    Gartner gives Docker security an ambivalent thumbs up

    As organizations continue to evaluate wider spread use of containerization to enable continuous deployment, security concerns around the wildly popular Docker keep popping up. Last week, an analyst with Gartner published a report that gave Docker security positive marks with some big caveats.

    Written by Joerg Fritsch, the report examined the security properties of Docker. According to Fritsch, Docker is effective at resource isolation and has already just about attained parity in secure operations management and configuration governance with Linux OS and hypervisors . But where Docker containers fall short is on the management end.

    “They disappoint when it comes to secure administration and management, and in support for common controls for confidentiality, integrity and availability,” Fritsch wrote, explaining that this could improve as Docker refines its libswarm and nsenter APIs.

    As things stand, Fritsch recommends that organizations that don’t need to emulate a virtual private system should standardize on nsenter for interacting with running containers, and protect nsenter by limiting the input set. He also recommends that when working at scale, organizations should select a framework for managing resources and deploying containers, and consider using SSL/TLS wrappers for the Docker/Swarm API for the sake of integrity and confidentiality.

    Meanwhile, Fritsch particularly calls out the lack of endpoint protection platform and encryption tools for Docker containers, explaining that the outlook is “grim” on this front.

    “Traditional EPP and encryption vendors have not yet recognized containers as an area that they need to pursue and secure in the future,” he says, explaining that organizations have to mitigate risks through application whitelisting, SELinux or by using a strategic DevOps tool chain to automate and secure their containers, essentially making “containers self-assessing entities.”

    Third party vendors may take more note of Docker as 2015 progresses, particularly if Docker’s growth follows the same trajectory in 2014. From June to October alone, Docker’s downloads grew from 3 million to 21 million. And last fall Docker received venture support for future growth by way of $40 million in additional funding—following up a previous round of $15 million in funding earlier in the year.

    However, security concerns around Docker have started gather like storm clouds. In fact, security objections over Docker were a big reason for the introduction of the Rocket containerization project by CoreOS in early December. That announcement came swift on the heels of the release of the 1.3.2 version of Docker Engine, which took care of a number of vulnerabilities that could be exploited through a malicious Dockerfile, image or registry. It also added added a security feature for validation of images in containers.  However, by late-December, security pros were knocking that verification function.

    “Docker’s report that a downloaded image is “verified” is based solely on the presence of a signed manifest, and Docker never verifies the image checksum from the manifest,” wrote Jonathan Rudenberg, co-founder of Flynn, in a long explanation of the insecurities in Docker’s image downloading process. “An attacker could provide any image alongside a signed manifest. This opens the door to a number of serious vulnerabilities.”

  • Ultimate Checklist for Successful App Deployment

    Ultimate Checklist for Successful App Deployment

    Deploying software releases is a mixture of planning, testing, late hours, and celebratory beers. This checklist is intended to be a guide to help improve how your team achieves this complicated and sometimes hairy task.

    Part 1: Planning and Deployment Preparation

    Collaboration Between Development and Operations

    It is important that operations be in the loop when it comes to planning any software deployment. They need to know when the project starts, not at the last minute about your new deployment. Ensure that operations is aware of any infrastructure changes needed well in advance. If the operations team is your front-line defense for monitoring and troubleshooting your app, help them help you by collaborating on how best to monitor the app and discuss early warning signs of trouble to be alerted to. If you’re thinking of setting up DevOp team you should probably read this.

    Build & Release Automation

    Removing the human factor is important to duplicating software deployment success. How you deploy the code doesn’t have to be state of the art. Utilize a basic build server like Maven, Bamboo, or TFS to automate the process or even a simple script that copies the files across the network can be sometimes sufficient. The key here is to make it easily repeatable by anyone on the team and eliminate the possibility of skipping any manual steps.

    Minimize the Amount of Change

    When things break in IT, typically it is because  something changed. When you introduce small changes in each software update it is easier to roll back those changes, or know exactly where to look for the source of the problem. As each release takes a certain amount of work in planning and testing, the key is finding the right balance of frequency and size of your releases that is optimal for your team to handle.

    Create and Test SQL Change Scripts

    Prior to your app deployment make sure you use tools like Red Gate SQL Compareto know what SQL schema and data changes have to be moved to production. to ensure there are no environmental differences that will cause potential headaches, I recommend never to forget to take a backup of production and run the test scripts on the backup copy before the final release. Production data differences or different database software versions can commonly cause hiccups. Also, have a plan to roll back to a previous state if the release has issues. Code is easy to roll back, but SQL requires some planning.

    Know your KPIs

    Every app should have some form of key performance indicators or metrics that can be used to know if everything is running smoothly. These will vary wildly from one app to another. They can be as simple as server CPU, page load times, or database performance metrics. The best metrics are more custom though. Knowing how many orders are happening per minute, or how many messages are being processed off a queue and how long it takes, or how long a certain piece of code takes to execute is a critical part of your app. Make your KPIs the heartbeat of your application and monitor them at all times.

    Setup Synthetic Transactions Tests

    Create a few tests that test things like your login page, key pages within your apps, APIs, etc. This will allow you to feel very confident that all systems are operational after the app deployment. I highly recommend Selenium, the leading open source framework for web automation.

    Notify Your Users and Colleagues
    Don’t forget to notify your users and others within your company about the upcoming changes. Make sure you have time to update documentation, train internal staff about new features, and coordinate with other departments in your company.

    Part 2: After the Deployment

    Monitor Server Utilization

    Make sure servers are all online, in the load balancer, and still receiving traffic. Monitor server and app CPU, memory, network, and disk usage. Noutilization can be as bad as sudden high utilization.

    Monitor Exception Rate

    After a new release your software is likely to throw some new exceptions you have never seen. Utilize an error tracking program so you can immediately identify new errors introduced in the release and to ensure previously resolved errors haven’t been reintroduced. A comparison of your overall error rates before and after the deployment will give you good indication of the release health.

    Check your KPIs

    Keep an eye on your performance indicators that let you know if your application is running properly.

    Watch your logs

    After an app deployment make sure logging is still working and the volume of logging hasn’t changed dramatically. No logging at all can be as bad as sudden high logging rates. A centralized log management tool makes this easy to monitor. You might also want to check the logs of new features that were released to ensure they are behaving properly.

    Monitor Page Load Times and HTTP Error Rates

    Watch the overall page load times of your application and keep a closer eye on 2-3 requests that are mission critical to your app. Also watch out for the rate of 400 and 500 level HTTP requests to ensure they haven’t dramatically changed.

    Monitor Database Performance

    After your deployment make sure database CPU, IO, and overall traffic look normal. Nothing is scarier than a sudden drop in database traffic after a release… unless it’s a sudden spike in database traffic.

    Monitor Key Database Queries

    Setup monitors in your monitoring system to test key queries to ensure they are loading quickly and returning proper results. For example, if some important background processes work based on SQL queries, run similar queries to make sure it isn’t missing work to be done and is working properly.

    Monitor Application Queues

    Monitor how many messages are in your application queues to ensure everything is flowing properly and not getting backed up.

    Cache Server Changes

    Don’t forget about caching. Depending on how you use caching, you may need to clear your caches or make other configuration changes. Data type changes can commonly cause some weird serialization type issues in cache.

    Have a celebratory beer!

    At Stackify we prefer a local joint with some good craft IPAs or the one brewed by one of our developers. Make sure your team has a good tradition for after releases.

    Post Mortem Meeting

    It can be very valuable to stop for a few minutes and discuss how the last release went and how things potentially can be improved. These kinds of conversations can help identify team weaknesses, allow people to vent, and take note of issues that can try to be prevented in future development cycles.

    Summary

    Deployment success over a long period of time depends on being able to automate and create processes that minimize issues for all stakeholders. Creating some automation or scripts around your build and release processes pay big dividends in the long run. Utilizing various monitoring tools or looking for new ones that  can help monitor your server and application health from every angle. Tracking trends over time can help you show off the improvements you’ve made with each release in availability, performance, KPIs, and user experience.

    About the Author

    mattMatt Watson founded Stackify in 2012 and as CEO provides the vision and leadership for the direction of the company. Prior to founding Stackify, he was the founder and CTO of VinSolutions. VinSolutions’ innovative product offering led to rapid company growth and was even featured on the Inc. 500 (#447) in 2010. Matt is an entrepreneur at heart and excels at product and software development.

    Matt studied Computer Information Systems at DeVry University. He started programming when he was in middle school to automate playing video games while he was at school. He hasn’t stopped programming since and loves anything technology related. Other hobbies include working too much, travel, movies, golf, cars, and of course his family. You can reach Matt at  @Stackify, @mattwatson81,  www.stackify.com, https://www.facebook.com/Stackify

     

  • What are the most critical DevOps Tools?

    What are the most critical DevOps Tools?

    What are the most critical DevOps tools categories? According to the results of a new survey by CA Technologies the answers may be surprising. That is just one of the insights reported on. The survey results and report are chock full of other great insights about DevOps usage as well.

    The survey shows that a staggering 88% of respondents either are already utilizing or expect to be utilizing DevOps within the next 5 years.  The over 1400 respondents to the survey represented IT or line of business executives and is a significant increase over last years 66% response rate. This would certainly indicate that DevOps is crossing over in the mainstream and is no longer solely in the realm of early adopters and visionaries.

    So according to the study what are the most critical DevOps tools?

    tools

    APM as the most critical? It is a popular segment with players like New Relic, App Dynamics, Dynatrace (Compuware) and CA to name a few. But I thought that change/configuration management which would include Chef and Puppet would be number 1. Also performance testing and functional testing were 2nd and 3rd. Could this be a case of categories CA plays in being over emphasized, the “everything looks like a hammer to a nail syndrome”?  I don’t know, but the numbers speak for themselves. I was happy to see enterprise security coming in 4th on the list (another area CA plays in).  Bringing up the rear at only 11% was network performance management. I think that may have a lot to do with the fact that with the rise of the cloud, so many organizations are less invested about network as the cloud and Internet has replaced much of the internal LAN.

    I spoke to Andi Mann, VP in the CTO’s office at CA and a frequent contributor to devopsy.kinsta.cloud about the survey. Andi said that it is re-enforcement that despite all of the talk about DevOps being about culture, we should not forget that it is about tools as well.

    So what are the reasons given for this massive adoption statistic. What are the perceived benefits and reasons driving DevOps adoption?  According to the survey results:

    devops benefits

    As can be seen over 40% cited improved quality and performance of applications as the driver for DevOps.  For those who wonder about whether you can have increased speed only at the expense of quality, this would seem to show that you can have both with DevOps. Perhaps equally as rewarding was the 2nd most popular response, improving the end customer experience.

    In any case with 88% of respondents on board with DevOps, one thing is for sure, DevOps is the worst kept secret in IT. However, let’s not paint an overly rosy picture. Not every DevOps effort succeeds. What are the biggest obstacles to DevOps adoption? Again the survey says:

    devops obstacles

    So while enterprise security is a big benefit for adopting DevOps, security and compliance concerns are the biggest obstacles to DevOps adoption as well. I guess that is the Jekyll and Hyde nature of security? I don’t buy the ROI being difficult to justify. Most DevOps examples I have seen show a clear positive ROI.  Identifying the right DevOps consultant firm?  I don’t know about that one either. But then again I am bullish on DevOps, so I may be as guilty of the “hammer looks like a nail” as anyone.  I am surprised though that under obstacles we didn’t see that our culture is not trusting or some other culture related reason.

    You should go have a look at the survey yourself as there are other good insights there. In the meantime here is another indicator of just how deep DevOps is penetrating the IT market.

     

  • When the bad guys use DevOps, you need DevOps to defend yourself

    When the bad guys use DevOps, you need DevOps to defend yourself

    I came across an intriguing headline the other day: Malware coders adopt DevOps to target smut sites. The article, unfortunately, is fairly vague about what exactly that means, but it got me thinking about the potential implications.

    It’s all just a function of rivals competing. Just as Company A has to find a way to work more efficiently and bring better products and services to market faster than Company B, malware developers need to be faster and more agile than their targets. In either scenario DevOps can give one or both parties a competitive boost.

    Although the article in question simply says malware developers are “now engaged in DevOps,” and claims it’s the first time that behavior has been seen from the dark side, the reality is that malware developers—even those in organized cybercrime groups—have always had an element of DevOps to them. By their very nature they’re more inclined to have shared responsibilities and to exhibit the sort of jack-of-all-trades cross-functionality that we expect to find in young startups.

    “It has always been the case that most attackers embrace much more of a DevOps pattern than defenders. Most of these folks were born of the Internet way of thinking and nowhere in their playbook do they have the traditional data center change controls or any other big enterprise methodology,” declared TK Keanini, CTO of Lancope.

    The problem that many organizations face when it comes to mounting an effective defense is a lack of situational awareness of their own environment. In many cases the attackers actually know more about the organization’s network, and the assets connected to it than the organization itself.

    Effective defense also requires agility. Companies need to be able to adapt quickly and implement changes without impacting business continuity. Keanini says, “DevOps is one way to do this and something like the Rugged Manifesto does a great job of capturing the essence of what it will take to battle this threat day in and day out.”:

    The Rugged Manifesto

    I am rugged and, more importantly, my code is rugged.

    I recognize that software has become a foundation of our modern world.

    I recognize the awesome responsibility that comes with this foundational role.

    I recognize that my code will be used in ways I cannot anticipate, in ways it was not designed, and for longer than it was ever intended.

    I recognize that my code will be attacked by talented and persistent adversaries who threaten our physical, economic and national security.

    I recognize these things – and I choose to be rugged.

    I am rugged because I refuse to be a source of vulnerability or weakness.

    I am rugged because I assure my code will support its mission.

    I am rugged because my code can face these challenges and persist in spite of them.

    I am rugged, not because it is easy, but because it is necessary and I am up for the challenge.

    The question is, “How?”

    Keanini—ever the strategist—references the OODA loop, conceived by USAF Colonel John Boyd. OODA is an acronym for Observe, Orient, Decide, and Act. The concept was originally designed for military combat operations but can also be applied quite effectively in many areas of business. Applying it to network or endpoint security, the idea is to at least raise the cost of observation and orientation for the opponent—in this case the malware developers.

    “DevOps can deliver this because change can be implemented as a defense whereby the adversary never has time enough to make enough of an observation, or orientation accurate enough for effective decision and action,” clarified Keanini.

    There are a few different elements involved here. Continuous monitoring and continuous incident response are both applications of a DevOps mentality that can help organizations mount a more effective defense against sophisticated attacks. The OODA loop Keanini refers to is less about response, and more about applying DevOps philosophies to take away the strategic initiative from the attackers in the first place.

    One thing is certain, if the malware developers implement policies that enable them to craft and deploy attacks faster, businesses will have no choice but to figure out some way to adapt and defend more quickly as well.

  • Continuous Delivery snapshot: a universe of fragmented tools

    Continuous Delivery snapshot: a universe of fragmented tools

    Technology has ushered in an era of real-time information, immediate feedback and instantaneous response. As a result, there is tremendous pressure on development teams to deliver a continuous flow of updates and fixes to software applications. In support of this objective, development philosophies and methodologies abound: agile, lean, scrum, kanban and extreme programming, just to name a few. Development teams are adopting and implementing these new methodologies at a breakneck pace in an attempt to bring efficiency and speed to the development process.

    This increased velocity has placed added pressure on software delivery teams to build, validate and deploy newly developed code faster and more reliably. Over time, the DevOps movement has emerged, injecting robust automation into the delivery process while spawning a diverse collection of hundreds of single-purpose delivery automation tools. As a result, DevOps has improved delivery speed and quality. Even still, continuous development comes with its challenges. Namely, the lack of a real-time “big picture” view,  of the end-to-end software delivery process.

    New Challenges for CD Teams: The Unified Pipeline

    While highly diversified delivery toolchains can be effective and have increased speed and agility, significant challenges continue to plague the software delivery process as a whole.

    Real-time visibility and visualization – Typically, the delivery process begins as soon as newly developed code is considered “dev-complete.” Often, the sequence of steps and stages new code progresses through on the way to potential release is referred to as the delivery pipeline. In a moderately sized organization, delivery pipelines are comprised of dozens of people performing hundreds of tasks. Without a centralized “delivery portal,” team members have little or no insight into the overall upstream or downstream process flow. While delivery pipelines benefit from diverse automation, they are still a “black box,” largely absent of communication, collaboration and audit/tracking capabilities.

    Data fragmentation – Important delivery data is generated at virtually every step along the delivery pipeline. Disconnected toolchains generate siloed, disaggregated and uncorrelated data that is trapped within a dozen or more automation tools. As a result, team members often lack access to a single comprehensive audit trail document that fully describes the functional and technical contents of a specific release along with any relevant data generated during each step of the delivery pipeline. At a macro level, self-organizing delivery teams do not have the real- time and historical analytic data required to quickly identify and react to exceptions or make performance optimizations across pipelines and/or teams.

    Is dynamic pipeline orchestration the answer?

    While the build and integration stages of a delivery pipeline are largely static and predictable, pipeline flow can become much more dynamic once manual testing begins. Defects can be discovered at any point along the delivery pipeline and judgment calls often determine a dynamic process flow. Often tools designed to automate the build/integrate phase of the pipeline are stretched beyond their original intent when it comes to pipeline orchestration. While these tools automate a critical portion of the delivery pipeline, they are far too static to support a real-world pipeline flow through the manual steps that occur in the later stages of a delivery pipeline. As a result, pipeline orchestration is largely a manual process beyond integration and into the quality assurance phases of delivery.

    What’s Missing?

    When it comes to alleviating the challenges associated with delivery pipelines, what will it take to ease the burden on continuous development teams?

    • Detailed Inventory of Each Bundle-of-Change – As multiple revisions get merged together in downstream pipelines to form a release candidate, a clear understanding of exactly what “changes” have been incorporated within any particular change bundle (release candidate) can become crucial.
    • Current Deployed Environment – At any given time, any revision or release candidate can be deployed to one or more environments or locations. The ability to quickly determine which change bundles have been deployed to which environment(s) is a fundamental need of virtually every stakeholder along the delivery process.
    • Broken code lines – Release candidates that contain known material defects are usually considered terminal and/or not deployable. Identifying a release candidate as broken can dramatically reduce the amount of time invested into a terminal release candidate. Subsequent testing within environments that contain a broken release often leads to wasted effort.
    • Detailed release documentation that fully describes the functional and technical contents of a specific release-in-transit along with all relevant data generated during each stage of delivery.
    • Reporting that describes stalled releases awaiting manual intervention with corresponding wait times.
    • Work queues that outline manual tasks or pending decisions/approvals assigned to a specific individual or group.
    • Travel time details on the average time it takes a release to traverse a pipeline from start to finish.
    • Reporting that describes the “base” release (starting version) and all subsequent change that has flowed thru any given development or test environment. 

    What is the Answer? 

    Without a technology that can oversee the entire development process and fill these gaps, teams often try to leverage Continuous Integration (CI) tools. While these tools greatly reduce the effort required to automate the build/integrate phase of the continuous delivery pipeline, this stretches them beyond their practical limits.

    However, a new type of pipeline orchestration technology that further streamlines and advances the effectiveness of the CD process is emerging and can effectively alleviate the challenges associated with fragmented and siloed delivery pipelines. This technology, often called a “pipeline orchestration framework,” expedites movement and raises awareness of a software “release candidate” as it travels through one or more delivery pipelines.

    So how do they work? Pipeline Orchestration Frameworks:

    • Leverage existing tools and processes – Because Pipeline Orchestration Frameworks sit on top of existing delivery automation tools, there is no need to reconfigure current process flow or abandon existing tooling. Pipeline Optimization Frameworks simply enhance and automate what is already in place.
    • Provide audit trail – Comprehensive audit trail documentation of each part of the process delivers highly detailed and valuable information regarding each new potential release. Having this audit trail ensures higher production quality results, increases cross-team awareness and reduces the time and effort required to perform root-cause analysis when defects are found (pre/post-production).
    • Manage deployments – Efficiently managing the cost and friction associated with dynamic development and testing environments over time can dramatically reduce the cost of delivery. Pipeline Orchestration Frameworks are capable of automatically provisioning and/or refreshing fully formed application environments as often as needed while keeping track of environmental drift or change over time.
    • Workflow automation – One of the goals of continuous delivery is to automate as much as possible along the delivery pipeline. However there are often steps that can’t (or shouldn’t) be automated, such as code reviews, exploratory testing, and manual decisions to automatically move a release into late stage test environments or even production. Pipeline Orchestration Frameworks manage and orchestrate these manual workflows as efficiently as possible, ensuring all delivery team members are aware of items in their “work queue,” and identifying stalled releases immediately.

    What will 2015 Hold? 

    The purpose of continuous delivery is to support development teams’ need to release reliable code faster. Therefore, the delivery team must keep up with an ever-increasing development velocity. Managing the delivery process not only requires managing each of the different phases of delivery, but also orchestrating the delivery process as a whole. That, in turn, requires visibility into every part of the end-to-end process, and a 10,000-foot view of the delivery pipeline. Pipeline Orchestration Frameworks offer CD teams a powerful new tool to unlock the full potential for successful continuous development.

    About the Author

    dennisDennis Ehle is the CEO and founder of cloud sidekick, which provides an automated, self-service integration and deployment framework for enterprises seeking continuous delivery. The cloud sidekick Velocity Pipeline Optimization Platform enables agile development teams to push code through multi-stage pipelines, automatically testing in virtual environments and ultimately publishing reliable code faster. For more information, visit www.cloudsidekick.com.

  • Enter at your own risk: DevOps isn’t for everybody

    Enter at your own risk: DevOps isn’t for everybody

    There’s always a sexy new tech buzzword flying around, and it’s easy to fall into the trap of assuming that if a concept or technology is hot you need to hurry up and jump on board so you don’t miss anything. DevOps is hot, and seems to be almost ubiquitous these days. But that doesn’t mean that every organization should rush out and embrace DevOps.

    Defining what it is exactly that you think you’re embracing can be a challenge. The concept of DevOps is a bit broad and ethereal in the first place. The number of vendors calling their products or services “DevOps” just because it’s sexy marketing muddies the waters and makes it more difficult to figure out what DevOps is, and how it’s supposed to benefit your organization.

    Even if you think you have a fair grasp on what DevOps is, though, it still isn’t necessarily right for every organization. It CAN be right for most organizations, but there’s some introspection, and due diligence that has to come up front. More precisely, it isn’t necessarily right for every organization without a solid foundation of the right skills and resources, and an understanding of what you hope to accomplish, and how you plan to get there.

    Nathen Harvey, community director at Chef, shared some insight on this topic. “Through immense feedback from our community, customers and contributors we’ve learned a lot about how and when an organization should begin a DevOps transition. In short, everyone should not immediately jump on the DevOps bandwagon.”

    Harvey added, “DevOps is full of big ideas and big changes and therefore development and operations teams need to be prepared to work together through challenges and to collaborate toward a common goal.”

    James Bottomley, CTO of server virtualization at Parallels, cautions, “DevOps processes are supposed to be accessible to anyone. However, since current DevOps lifecycles are based on virtual machines, they are really only accessible to those with the skills to manage virtual images and, to make use of the live VM, administer base systems and platforms.”

    Bottomley explains that larger organizations often have the necessary skills available, and just have to marshal them toward a DevOps strategy. Small and medium businesses, on the other hand, often lack the appropriate resources.

    “If you are a SMB you should consider carefully whether you have the necessary Ops skills in-house and if not, how you are going to obtain them (whether from a [service provider] or other form of management). Ongoing Ops skills are vital to proper VM maintenance (fixing security issues and the like), so you have to make sure you have an ongoing Ops capability, not just a one-time one when the virtual system was created.”

    According to Chef’s Harvey, there are three specific areas organizations need to consider before embarking on a journey to embrace DevOps. They are the “Three C’s”: Character, Communications, and Community.

    · Character requires trust, respect, empathy, learning and perseverance.

    · Communication is meant to align objectives—such as setting metrics for everyone, and sitting close enough to easily collaborate with one another.

    · Community can take many forms, whether engaging in podcasts, conferences, sharing code and more.

    Don’t make the mistake of jumping on the DevOps bandwagon prematurely just for the sake of jumping on the DevOps bandwagon. It will be a frustrating experience almost doomed to fail and leave a bad taste in your mouth. Instead, take the time necessary to understand what DevOps means to you, figure out what you hope to accomplish by embracing DevOps, consider whether or not you actually have the skills and resources to make it happen, and develop a plan that includes the “Three C’s” so your DevOps journey can be a successful one.

  • Five essential Ops strategies that help make DevOps the real deal

    Five essential Ops strategies that help make DevOps the real deal

    With its roots in Agile and Lean thinking it’s not surprising that DevOps is often looked at through a development lens. After all, it’s proving absolutely essential to support accelerated delivery times; providing a much needed jolt of velocity to protracted development cycles. But of course with the increased need for speed there’s always the danger that quality can be compromised – so much so that faster time-to-value can very quickly become faster-time-to-failure.

    In what seems like a byzantine age of enterprise computing, IT operations tended to call the shots when it came to delivering quality. Sure, a transactional system might suffer a glitch or two, but since constituents were largely internal and customer’s generally one step removed from the application and business process, the impact could often be controlled. Ah, so much for the good old days.

    But as we know this model hasn’t just changed, it’s been completely disrupted.

    Quality is now a different headed beast that bites hard in many places. It bites when customers engage directly with your brand during discrete mobile moments – like when they compare your goods in the shopping aisle with a mobile app, or assess car insurance based on your ability to provide usage-based policies from their in-vehicle software system. Yes, speed is important in the application economy, but it’s also a high quality customer experience that shapes the winners from losers.

    The fabulous thing about DevOps is the fundamental premise that teams can collaborate in real-time to establish, maintain and continuously improve the quality of software systems and applications. For operations this doesn’t mean waiting for applications to reach production stage before enacting performance management processes. Rather, it involves instantiating best practices throughout the development cycle – that way teams resolve quality issues much earlier in development (where they are cheaper to fix), while also preventing a cumulative buildup of crippling technical debt.

    So with application quality and customer experience being essential for DevOps success, what are some of the ops related strategies we can employ to support better business outcomes?

    I see five essential, ‘rubber meets the road’ strategies ops can use to help DevOps become the real deal:

    1. Focus collectively on the user experience – too often organizations become detached from business reality by looking inside-out from a silo’d technology and diagnostic perspective. DevOps practitioners put themselves in the shoes of the customers by constructing, visualizing and managing end-to-end user-centric service views that traverse functional boundaries. Additionally, user-experience management will involve gaining deeper insights into real-time performance of externally facing applications and mobile services, bringing to the forefront analytics to guide decision making and drive continuous improvement.
    2. Give Developers a clear view of how code works in the real world – operational intelligence shouldn’t wait until applications are moved into production. Rather, it should be instantiated across every stage of the lifecycle – whether that’s during regression and unit testing, or pre-production. Many teams recognize this by for example activating monitoring during release automation; giving developers critical insights into both functional and non-functional code behaviors. Others go further, taking user-experience to another level by exposing key behavioral and usage-based intelligence – especially for newer mobile apps, APIs and digital services.
    3. Create a single source of the truth using a common framework – to be fully effective, developers and operations should reference a single version of the truth when it comes to problem triage and resolution. Traditionally this has proved difficult with teams having discrete tools providing diagnostic insights into one aspect of technology. This however falls short due to the complex interdependencies across technologies and results in finger pointing and blames games when problems can’t easily be pigeon-holed. To address this, operations teams use a common unified framework that quickly discovers and exposes infrastructure and application configurations, and importantly the relationships and interdependencies that exist between them. Now, root-cause can be quickly established from either an application or infrastructure perspective, but always prioritized according to business and customer impact.
    4. Define common goals across teams and use shared metrics – traditionally development and operations teams have been incentivized according to separate goals that are often diametrically opposed (e.g. speed of producing code –v- resilience and availability). By using shared customer and business-centric metrics, developers and operations build a ‘common language’ for evaluating progress and understanding the impact of change and continuous delivery. My advice here is quality over quantity; building out metrics that are actionable by all stakeholders. For example, this could include lead and cycle times to pinpoint operational bottlenecks and waste during deployments, or transaction response monitoring in pre-production to pinpoint code deficiencies and avoid technical debt associated with provisioning unnecessary capacity.
    5. Converge around a common understanding to transcend organizational boundaries – a collection of diagnostic based monitoring and alerting provides little more than vanity metrics and feel-good green lights. Sure, they can alert operations to when something is broken, but they do little to drive the more essential element of DevOps success – feedback. Therefore common understanding involves operations automatically correlating performance information end-to-end so as to expose far more beneficial insights that developers can use to improve the code base.

    At the recent Gartner Data Center, Infrastructure and Operations Management Conference in Las Vegas in December, I presented a session – Unified Infrastructure Management is Critical to Enhancing the Ops in DevOps – that further described these strategies together with supporting use-cases. It was certainly a dynamic and lively session, highlighting the viability of DevOps. As organizations look to increase speed to market of new applications while both lowering cost and increasing quality, DevOps will continue to gain traction, and I can’t wait till everyone truly understands its transformative power.

    About the Author

    ravichandran arunaAruna Ravichandran is vice president of DevOps Solution Marketing and Management at CA Technologies, responsible for cross-portfolio solution marketing for DevOps across all product lines. Before joining CA Technologies, Aruna held several management positions in product marketing and engineering at Juniper Networks and HP. Aruna earned an MBA and master’s degree in Computer Engineering from Santa Clara University and holds a bachelor of science degree in computer science from Bangalore Institute of Technology. Ravichandran is  on twitter @aruna13 and on linked in

  • Optimize app performance with the help of real-time monitoring and analytics

    Optimize app performance with the help of real-time monitoring and analytics

    Back in 1980 there was a movie called Nine to Five—referring to the commonly held perception of a standard work day. Of course, that work day evolved to be more like eight to six (or seven), and now it seems like the concept of a work day—or work hours at all—is a distant memory. In order to keep up with the instantaneous, always-on, connected 24/7 pace of today, companies need to be able to understand how their apps are used, and proactively address issues in real-time.

    That’s where companies like MixPanel come in. MixPanel offers an advanced analytics platform for Web and mobile apps so that developers can keep tabs on when, where, and how apps are used, and any problems that might arise.

    I spoke with Tim Trefren, co-founder of MixPanel. He proclaimed, “It’s easy to build a product. What’s hard is building a good product.” I assume that wisdom applies both to MixPanel itself, as well as to the customers that employ MixPanel to monitor and improve their own products.

    Trefren told me that MixPanel provides a very flexible system that enables customers to track and monitor the metrics that matter most to them. It allows for custom metrics, so companies can choose to monitor elements of apps and app functionality that are completely unique. Customers can also implement A/B alternatives and compare metrics between the two to determine which is most effective.

    Monitoring performance metrics, and analyzing that data to fix or improve apps fits into the broader concept of agile development, and rapid iteration that define DevOps. Trefren explained that some customers also employ MixPanel in an innovative way that also applies to DevOps—to actually monitor operational data.

    Trefren says MixPanel may not be the ideal tool for monitoring operations, but it does enable admins to monitor everything from a single display, and it can be used to alert IT personnel when issues arise. For example, if MixPanel starts logging metrics that show an app is not being used at all, that could indicate that a server is offline.

    MixPanel also recognized that it was in a position to do a little more. It offers a service that analyzes captured metrics data, and enables MixPanel to help companies understand who their customers are. Understanding the audience, and the actions they take (or don’t take) with an app provides valuable demographic data that companies can use to target marketing efforts with greater precision.

    Trefren also told me that he is exceptionally proud of the engineering team at MixPanel. While the volume of data being analyzed has tripled in each of the last two years, the engineering team has only doubled over those same periods. That means that the MixPanel engineering team is working more efficiently, and getting exponentially more done in order to keep up with demand, but that’s another story for another day.

    “The combination of power, flexibility, and performance set MixPanel apart from competitors. We’re able to do more, and do it more quickly,” said Trefren.

    That may be true, but MixPanel isn’t alone. I wrote a while back about similar services from AppDynamics, including its DevOps War Room feature. Clearly, the rapid pace of development, and the sheer volume of apps have created a market for being able to monitor metrics, and measure the performance of those apps in real-time.

    MixPanel offers its analytics as a freemium service. Paying customers are billed for metrics per data point, so the overall cost ends up being a reflection of the volume of data being analyzed.

  • Deliver faster, but don’t sacrifice quality

    Deliver faster, but don’t sacrifice quality

    Developing applications, and deploying servers faster are great goals. Being agile and streamlined can give an organization a clear competitive advantage. However, faster does not equal better, and being speedy for the sake of being speedy can lead to avoidable mistakes. Faster is only better if you can achieve greater speed without sacrificing quality.

    At the same time companies are racing to be faster than their rivals, they’re also dealing with an increasingly complex IT infrastructure. IT environments are becoming highly heterogeneous—merging physical, virtual, and cloud platforms, and combining legacy systems with cutting edge technologies. The challenges of managing such a diverse IT infrastructure simply add to the potential issues that can arise as organizations try to work faster.

    That is where DevOps plays an important role. DevOps tools and technologies establish a process chain between development and deployment. It enables organizations and IT admins to reliably track and understand the components in use, and any changes or updates that occur. In order to achieve speed without sacrificing quality, it’s important to maintain as consistent an environment as possible, and to know exactly what has been changed.

    “For hypervisor based virtual images, this depends on the strict process around handling the images, but for container based virtual images depending on a cascading snapshot mount namespace, where there are layers of images built one on top of the other, strict handling becomes unnecessary because everyone can see transparently (via the image cascade) what changes at each stage of the process,” explained James Bottomley, CTO of server virtualization for Parallels. “This gives everyone in the DevOps chain the ability to verify each step and see the changes for themselves, leading to much higher assurance that the resulting deployment is correctly tested and released.”

    Ken Cheney, VP of business development for Chef, describes another way that DevOps helps boost speed without affecting quality. “Automation turns infrastructure into code. For example, Chef recipes automate all facets of management and orchestration—from provisioning bare-metal and cloud servers to automated configuration and application deployment. You can replace manual, slow, error-prone procedures with code that is versionable, testable and repeatable.”

    “When infrastructure is code you can test it just as you would applications—using automated tests that find problems early, before you release to production,” added Cheney.

    In its 2014 State of DevOps Report, Puppet Labs drew an interesting correlation between speed and quality. Essentially, Puppet Labs found that speed and quality become a self-feeding circle.

    “There’s a virtuous circle at work here: As stability improves, IT performance improves. This improved performance helps to create a better-functioning business that can pay attention to the communications and processes that enhance and improve stability.”

    The converse is also true. Poor quality results in issues, and necessitates allocating resources to troubleshoot and resolve problems, which impedes the speed at which an organization can function.

    DevOps tools and technologies enable organizations to streamline development, and work more efficiently without sacrificing quality. Businesses that attempt to work faster, without also working smarter by using DevOps are putting themselves at a disadvantage.

  • DevOps Transforms Copyright Clearance Center

    DevOps Transforms Copyright Clearance Center

    Copyright Clearance Center (CCC, www.copyright.com) works primarily with business, academic, and medical institutions, enabling them to access the vital content that supports their work. CCC manages fee payments to the license holders and content creators who own the content. DevOps approaches have transformed CCC, a global leader in content rights-licensing solutions, resulting in expanded service to an increasing market.

    “In production, CCC is already processing millions of dollars in transactions. This came to life within a year of the inception date for the DevOps project. For a startup that would be fast. For a medium-sized enterprise, I think that is outstanding,” says Babis Marmanis, CTO & VP of Engineering, Copyright Clearance Center.

    The DevOps transformation that took place in the internal structure of CCC’s engineering department continues today, allowing the organization to respond to the massive disruption in the publishing industry, which is due in part to digital content and the Internet. Using DevOps, Marmanis and CCC have accelerated product development, sped time to market, and provided a stronger consumer experience with the product.

    Accelerated Product Development

    “The Copyright Clearance Center vision is to create the most comprehensive portfolio of global licensing and content solutions (products) that make copyright work for everyone. My product platform strategy stems from that,” says Marmanis.

    CCC has multiple software-based products, each performing the same or similar functions and acting on the same or similar data. “There are certain fundamental things that we must abstract appropriately and encapsulate into common components to make development work optimally,” says Marmanis.

    But when Marmanis came to CCC, there were no common components. And what is more the internal systems lacked interoperability, skills were different across internal teams, and there was no cohesiveness to the development approach. These factors slowed software product development.

    A DevOps mindset was crucial to changing the status quo, to treating infrastructure as code, and to bringing in automation, says Marmanis. A DevOps approach would help to speed the delivery of software projects into the marketplace.

    “Now we use Jenkins (www.cloudbees.com) to glue together elements of the continuous integration process. We have Jenkins build jobs that retrieve the source code from GitHub (https://github.com). We publish software artifacts to Artifactory (www.jfrog.com),” says Marmanis.

    For each release, Marmanis’s teams use Jenkins to execute Chef (www.chef.io) recipes. They deploy artifacts from Artifactory onto a pre-specified execution environment. “In addition, and this is key to the overall scheme, we execute the schema updates with Liquibase (www.liquibase.org), and everything is coordinated with Jenkins,” says Marmanis.

    Further, CCC built its cornerstone platform from the ground up using DevOps-related tools, build detection software, and software libraries. The infrastructure that CCC added is based on best-of-breed open source libraries.

    “Using programming languages such as Java (www.java.com) and Groovy (http://groovy.codehaus.org), Gradle (www.gradle.org) scripts, postgreSQL as the relational database, Liquibase for database objects, Tomcat (http://tomcat.apache.org) as a servlet container, Apache Camel (http://camel.apache.org) for messaging and enterprise integration, and countless other tools including Apache Directory Studio (http://directory.apache.org/studio) and Bootstrap (http://getbootstrap.com/)—the list is very long—we were able to create a harmonious ecosystem,” says Marmanis.

    Stronger Consumer Product Experience

    CCC offers its consumers a better product experience through DevOps approaches and through a homogeneity of standards and technologies, which is available when using open source tools.

    “When we’re building the product platform, these initiatives allow us to embody timeless design principles such as crisp abstractions, but also allow us to scale in a number of dimensions including the number of works that we can represent, the number of publishers we can admit and manage, the number of customers that we have, the number of applications that we can build and deploy, and the number of market verticals that we can occupy,” says Marmanis. This is crucial to customer satisfaction.

    One product that is a prime example of how CCC keeps customers happy is RightsLink for Open Access. “Using RightsLink for Open Access, we facilitate fee collection for articles published under an Open Access license, mediating between the publisher and the author to collect those fees,” says Marmanis. Then CCC aggregates those transactions and presents them to the publisher.

    “Obviously, we give the money to the publisher, but we also create reporting for the authors, publishers, and institutions as well as the founding agencies that gave the money to the author or the institution respectively for funding that research,” says Marmanis. The RightsLink for Open Access product keeps transactions simple, integrating with publisher workflows to expedite publication.