devopsy.kinsta.cloud

Category: DevOps Practice

  • Bridging the Gap Between ITSM and Software Engineering

    Bridging the Gap Between ITSM and Software Engineering

     

    Considering the individual roles of IT service management (ITSM) and software engineers, it seems obvious they should work closely together. ITSM tracks internal and external customer issues, and engineering fixes code-related issues (such as defects) that IT cannot fix on its own.

    However, more often than not, coordination between these two departments is poor, which—critically—undermines customer response times and damages application satisfaction. It’s important to remember that the quality of customer response isn’t just about speed, it’s about giving customers insight into the state of the issue.

    The disconnect between ITSM and engineers—caused by individual priorities and semantic differences—makes this real-time insight difficult. Both teams have individual and shared concerns, and separating these while simultaneously improving collaboration is a big challenge.

    Addressing this relationship is an important step in optimizing ITSM and DevOps integration, alongside focusing on IT help desk enhancement requests to product management, and tracking code from build to release and back.

    The Flow of Information on Things that Matter

    Most of the tickets generated by the IT help desk can be handled internally with no help from engineering, but it’s the tickets that do need help that require consideration. For these issues, there needs to be an open flow of information on the things that matter between these two teams.

    IT help desk must give engineering enough information to understand the defect and its impact on the customer. Meanwhile, engineering needs to be able to give the help desk a sense of when to expect the fix, when the fix is done and be able to request more information when required.

    Information that should be shared from the help desk to engineering includes the following:

    • Severity – Engineering needs a sense of the importance of the defect. If everything comes in as “High Priority,” nothing is high priority. The help desk must understand that engineering already has its own internally generated set of defects, as well as defects found by QA and other sources. For these customer defects to be prioritized and fixed in a timely manner, engineering needs to know how impactful they are to the customer.
    • Description – This is perhaps the most important piece of information the help desk can pass along. The more contextual information that can be shared, the better. It’s immensely important to provide engineering with the necessary steps to reproduce the specific defect, which version of your tool the customer is using, the general environment, etc. Sometimes this is best in structured fields, whereas other times it’s important to simply write out everything.
    • Screenshots and other visual information – Along with the written description of the problem, engineering often requires logs and/or screenshots. As the saying goes, “A picture is worth a thousand words.”

    Now that engineering has the vital information it needs to fix the defect, it has a responsibility to provide useful information back to the help desk:

    • Planned release – This should not be a firm commitment, but rather an estimate to give the help desk general timing. While the actual date of the fix may vary, it’s still useful to be able to tell a customer,”This is coming soon” vs. “We realize this is a problem and it’s on our backlog to fix.”
    • Engineering status – While this is a single defect to your customer and to them only has one status, there are really two statuses your organization will care about:
      • Status of the ticket in your ITSM tool
      • Status of the defect in your defect tracking/agile planning tool

    It’s very important that the ITSM tool contains both statuses to provide more information and context to the help desk. Imagine the benefit of knowing that the defect is “In Triage,” “In Progress,” or “In Verification” and being able to provide that information to a customer.

    • Actual release/released – As opposed to the planned release information, this lets the help desk know that the customer’s issue has been resolved. Imagine running a report from your ITSM tool indicating the defects fixed in your most recent release. Your IT help desk can proactively reach out to customers to let them know to upgrade, enabling you to offer more proactive customer communications.

    Finally, and perhaps most important, you allow your teams to talk about the “same thing” even when it exists in different tools and they speak a different language. There are four core ways to do this:

    • Cross tool tracking – Each artifact in each system should have the ID (and ideally URL) of its twin artifact in the other system. If a help desk representative needs to talk to an engineer about a defect, the rep needs to let engineering know which defect it is in the engineering tool, not just in the ITSM tool.
    • Routing – IT help desks often support multiple products, each of which have multiple components that may belong to a different engineering team. The help desk needs some indication which component and/or team should work on the defect. This information will help the IT help desk know which team to reach out to for help.
    • Filtering – Not every ticket in the ITSM tool needs engineering help (that’s a good thing). However, there should be some indication on the IT ticket if it has been sent to engineering. You do not want to send the same defect to engineering twice (confusing) or to assume it’s been sent when it hasn’t (poor time to resolution or issue overlooked).
    • Communication – IT and engineering need a shared way to talk about a specific defect that they both care about. The closer this communication can be to the native commenting that already occurs, the better. Ideally this is not via email as that communication vehicle does not stay attached to the artifacts in question, meaning information can be hard to find or get lost.

    The Benefits

    Customers will love it – Improved communication means happier customers. Customers can plan their upgrades with more certainty, and their trust grows since your organization is able to deliver on its promises. No longer will they feel like their issues are being thrown into a black hole.

    Better engineering focus – Engineering gets pulled in a variety of directions, contending with feature work as well as internally generated defects. The more information it has, the easier it is to prioritize and deliver more value to customer. Without the ability to share the severity of a defect, you risk two undesirable outcomes:

    • The defect is fixed instead of more important work, which hurts all your other customers.
    • Other less important work is prioritized, which can threaten your relationship with this customer.

    Better trust across your organization – Open communication fosters trust and understanding. Providing insight into prioritization and status let’s your help desk feel part of the development process. No longer will staff feel like they’re sending problems into a black hole with no idea when they will be fixed. Enhanced cross-team communication also helps engineering to avoid feeling like work is simply being thrown over the fence. Obtaining more context around a defect is invaluable.

    Next Steps for Total DevOps Integration

    To fully integrate your ITSM and engineering teams, there are still (at least) two other processes to implement:

    • A method for the IT help desk to provide enhancement requests to your product team: These enhancement requests are similar, but distinctly different from the defects described above. Consequently, they need to be prioritized and triaged by a different team.
    • The other process to implement is the traceability from task to code to build and back again. In order to properly inform customers that their problem is fixed in this version of your product, it’s important to correlate features, stories and bug fixes with the actual version of your product that includes them for the first time.

    Bridging the gap between ITSM and engineering teams brings your company closer to customers. While making your organization work as a unified whole is an incredibly complex problem, addressing this relatively small—but vital—piece of the puzzle helps you on your journey to integrating your whole software value stream. This change of mindset—combined with connecting all teams, tools, disciplines and processes in an enterprises’ software value stream— optimizes collaboration, productivity, efficiency and, most importantly, value delivered to the customer.

    About the Author / Trevor Bruner

    Trevor Bruner is a Product Manager for Tasktop Technologies. His background includes experience in financial services, oil and gas and as a Navy submarine officer. Connect with him on LinkedIn and Twitter.

  • Platform9 Brings Workflows to Serverless Computing

    Platform9 Brings Workflows to Serverless Computing

    Serverless computing frameworks are widely regarded as the next big thing in enterprise IT because they are based on an event-driven architecture that makes servers, storage and networking resources dynamically available as needed using a programming construct known as functions. But DevOps teams are challenged in finding a way to orchestrate all those functions in concert with one another.

    To enable the orchestration of those functions, Platform9 has added support for Fission Workflows to its instance of a serverless computing framework. Platform9 CEO Sirish Raghuram says Fission Wokflows will make it possible for IT organizations to craft complex applications made up of thousands of functions.

    On the one hand, serverless computing frameworks promise to make DevOps easier by eliminating the need to provision infrastructure every time a workload exceeds whatever amount of capacity that has been assigned to it. On the other hand, attempting to manage what could easily become thousands of functions in an IT environment is likely to make DevOps even more challenging than it is today. While not a new concept, Raghuram says Fission Workflows provides IT organizations with a familiar construct for managing functions.

    Benefits of Fission Workflows include an ability to reuse functions as well as being able to upgrade entire related sets of functions all at once. There are also fault-tolerant capabilities that make possible to recover and restart a workflow in the event of an infrastructure failure.

    Platform9 is specifically making the case for a serverless computing framework based on Kubernetes clusters that can be deployed on-premises or in the cloud. Platform9 also provides support for containers on an instance of Kubernetes, and Raguham says the company expects that IT organizations will mix and match containers and serverless computing frameworks as they see fit. For example, a container might invoke a function via an API call to run a stateless process while Docker containers are employed for all the stateful processes that require access to primary storage, he says.

    Raguham notes the trouble with serverless computing frameworks put forward by cloud service providers such as Amazon Web Services (AWS) is that they lock an IT organization into a specific platform instead of taking advantage of a portable cluster technology such as Kubernetes, which is backed by the Cloud Native Computing Foundation (CNCF). Kubernetes also provides the added benefit of a common platform for both containers and serverless computing, he says.

    It’s still early days for the use of serverless computing frameworks in the enterprise. Nevertheless, it’s clear they will need to be managed from a DevOps perspective alongside cloud-native applications running on containers and platform-as-a-service (PaaS) environments and legacy applications running on virtual machines and bare-metal servers. Given the fact that none of these architectures is likely to supersede any of the others anytime soon, IT organizations would be well-advised to start crafting a DevOps strategy now on the assumption that the overall IT environment is about to become even more complex.

    — Mike Vizard

  • Scaling DevOps at Pearson

    Scaling DevOps at Pearson

    Pearson might be one of the more influential companies you have never heard of. Its footprint in the educational publishing marketplace is expansive, and its scope was just one challenge when it made the move to DevOps.

    Sean D. Mack (@SeanDMackNYC) is formerly the VP of Operations and Application for Pearson. To give you an idea of the scale of the company’s operations when it began implementing DevOps, Sean oversaw 110 development teams and 3,000 different products across the globe. To further complicate things, Pearson is a mature company with both legacy and new products. This is a case study of how the company approached implementing DevOps with the goal of helping you learn how to do it. Sean presented it at the 2016 All Day DevOps conference.

    As they say, you often learn your biggest lessons with your failures. Sean noted that, ultimately, DevOps was a failure at Pearson due to organizational change. But, he presented the case study with the hope you will learn from what Pearson did and take it forward. He continues to be a passionate evangelist for DevOps.

    Pearson had its challenges. It was a siloed organization operating at a large scale, and had a toss-it-over-the-wall mentality in dealing with problems. Out of sight, out of mind.

    seanmack1.png

    In addition, all of Pearson’s teams were at different levels of maturity, from legacy teams running .Net on Windows 2000 servers while others doing CI/CD to AWS. Obviously, this creates challenges when implementing DevOps, but we will see later how the company tried to tackle that, specifically.

    At the high level, the solution involved several aspects:

    • A DevOps practice area
    • A DevOps steering committee
    • Measuring progress based on global KPIs focused on measuring business impact
    • Measuring teams based on a DevOps scorecard
    • Push from the bottom – plenty of developers and operations who embraced and were eager for the change
    • Push from the top – support from key executives

    Pearson’s DevOps Steering Committee set up several workstreams:

    • Tracking tools
    • Metrics
    • Deployments automation and monitoring practices
    • Test automation
    • App engineering resource contention
    • Operationalization
    • Communications

    While these are specific to Pearson, many may be applicable to your organization. In any case, it is important to step back, look at your organization, and decide what your organization needs.

    Pearson also looked at four core competencies to track and measure: automation; independence (how independent is the feature); operationalization (how easy it is to maintain); and continuous improvement. The below chart outlines each area in more detail.

    Automation

    Continuous Improvement

    Independence

    Operationalization
    • Unit tests
    • Critical Scenarios
    • Dependent Systems
    • Continuous Build
    • Continuous Deployment
    • Test Automation
    • KPI
    • SLA
    • Metric Tracking
    • Metric Reporting
    • Remediation
    • Isolated Configuration
    • Code Dependencies
    • Isolated Infrastructure
    • Monitoring and Alerting
    • Tier 1 Service Desk System Knowledge
    • Tier 2 Service Desk System Knowledge
    • Documentation

    To help each team measure their competencies and improvement, Pearson built a self-reporting tool to empower teams but encourage visibility, and developed business metrics and then a competency model to measure teams against to see how they were progressing. As Sean noted, “We were not trying to get everyone to a 5. Some teams, yes, but we also had legacy products that were running for years, successfully, with little or no maintenance. The investment to bring the teams up wasn’t worth the effort. There had to be a business reason.“

    seanmack2.png

    In the end, Pearson raised DevOps maturity for all of the teams. As noted earlier, though, this was eventually phased out, but Sean presented this case study with the hope you found something to drive DevOps in your company.

    In the end, if you have just a couple takeaways, they are this:

    1. Begin by determining what your desirable outcomes are.
    2. Don’t just do DevOps to be cool. Do it to drive business results.

    You can watch Sean’s entire talk online here. If you missed any of the other 30-minute long presentations from All Day DevOps, they are easy to find and available free-of-charge here.  Finally, be sure to register you and the rest of your team for the 2017 All Day DevOps conference here.  This year’s event will offer 96 practitioner-led sessions (no vendor pitches allowed). It’s all free and online on Oct. 24.

    — Derek E. Weeks

  • DevOps Introduces Complexity. Really

    DevOps Introduces Complexity. Really

    The larger the project, the more resources are generally dedicated to it. That is true of the vast majority of fields of human endeavor. Consider building the San Francisco Bridge versus building a crossing over a stream. Or World War II versus the Spanish Civil War. The resources dedicated scale with the size and complexity they need to support.

    This is true of IT projects also. A four-line script does not require the attention that a highly available web app with dynamic data does. It is a pretty simple concept, we all are aware of it, and methodology doesn’t much change it.

    The problem is that we are often terrible about acknowledging this concept when it comes to internal support projects. We don’t see a compiler as a tool in need of regular maintenance and locking down, and largely it isn’t. There definitely are cases where the newest compiler/interpreter should be implemented, but except for massive change such as Python 2.x to 3.x, there isn’t a driving need to have the newest and shiniest.

    It’s Complex—and Growing More So

    What happens when we apply this relaxed attitude to DevOps? A mess. The reason is simple: The automation side of DevOps is generally implemented with tools. Lots of tools. Tools that require maintenance, troubleshooting, interfacing to other tools … It gets complex fast.

    And where on the calendar is time for these things set aside? Almost always the answer to that question is “nowhere.” But these aren’t compilers and interpreters. If your Bugzilla DB goes bad, you have a major problem. Since compilers and interpreters are generally read-only—changes aren’t made to them except to upgrade—it generally requires a disk-level error or serious human failures for them to stop working as planned. That’s not true for things with constantly changing datasets. And many DevOps tools have constantly changing datasets.

    In an increasing number of organizations, the DevOps infrastructure should be listed as the largest application portfolio the organization manages. And that is where you need to be.

    Make DevOps Tools a Project

    It is that simple. Tooling is a massive bit of overhead that seems to be not so big because it streamlines the development/deployment process. In short, the tools save enough time that their cost seems smaller than it is. But that cost will not go away, and once the pipeline is fully automated with a stable toolchain, keeping it running is a cost IT will have to continue to bear. Don’t just treat it like something you can check into when it fails; give it the same treatment that you use it for. Make certain you have backups—if possible, make your DevOps environment totally dynamic—so a failed instance of X can just be replaced by a new copy. You can’t “just spin up” complete and accurate datasets, but the backups fill that void.

    It’s More Than Just DevOps

    Stay on top of tool updates and improvements, just as you would for any mission-critical software. Because once your software factory is dependent upon the DevOps toolchain, it is mission-critical software. Learn how changes will impact the environment you are using the tools in.

    And it’s more than just the tools themselves. The DevOps environment is highly complex with a lot of variables that can have a negative impact. The move from RHEL/CentOS 6.X to 7.X is a good one. Sure, it’s better by objective standards, but there is enough change that it breaks a lot of things. Mostly the change in networking, but libraries and tools also. In fact, I helped someone work through inter-process communications (IPC) issues caused by the upgrade. This is a stumbling block for the DevOps team in a RHEL/CentOS environment, even though the DevOps team may not be the one choosing to upgrade.

    It’s All Good, Though

    So there is a cost. The cost is not small. But the benefits still far outweigh the costs, and I’m not arguing otherwise. What I am arguing is that it needs to be a cost that is accounted and planned for. “When it breaks, we’ll fix it” is too relaxed for the toolchain that will run your software factory. Once it’s right, take the time each week to keep it right. Set up the policies, processes, and time to keep it clipping along smoothly.

    And keep kicking rear. This is just a cost of doing business, much the same as database maintenance. Make sure you’ve got the time set aside, so your other projects can keep moving the org forward, and all is good.

    — Don Macvittie

  • Bridging the Gap Between ITSM and Software Engineering

    Bridging the Gap Between ITSM and Software Engineering

    Considering the individual roles of IT service management (ITSM) and software engineers, it seems obvious they should work closely together. ITSM tracks internal and external customer issues and engineering fixes code-related issues (such as defects) that IT cannot fix on its own.

    However, more often than not, coordination between these two departments is poor, which—critically—undermines customer response times and damages application satisfaction. It’s important to remember that the quality of customer response isn’t just about speed, it’s about giving customers insight into the state of the issue.

    The disconnect between ITSM and engineers, often caused by individual priorities and semantic differences, makes this real-time insight difficult. Both teams have individual and shared concerns, and separating these while simultaneously improving collaboration is a big challenge.

    Addressing this relationship is an important step in optimizing ITSM and DevOps integrations, alongside focusing on IT help desk enhancement requests to product management, and tracking code from build to release and back.

    The Flow of Information on Things that Matter

    Most of the tickets generated by the IT help desk can be handled internally with no help from engineering, but it’s the tickets that do need help which require consideration. For these issues, there needs to be an open flow of information on the things that matter between these two teams.

    IT help desk must give engineering enough information to understand the defect and its impact on the customer. Meanwhile, engineering needs to be able to give the help desk a sense of when to expect the fix, when the fix is done and be able to request more information when required.

    Information that should be shared from the help desk to engineering includes the following:

    • Severity – Engineering needs a sense of the importance of the defect. If everything comes in as “High Priority,” nothing is high priority. The help desk must understand that engineering already has its own internally generated set of defects, as well as defects found by QA and other sources. For these customer defects to be prioritized and fixed in a timely manner, engineering needs to know how impactful they are to the customer.
    • Description – This is perhaps the most important piece of information the help desk can pass along. The more contextual information that can be shared, the better. It’s immensely important to provide engineering with the necessary steps to reproduce the specific defect, which version of your tool the customer is using, the general environment, etc. Sometimes this is best in structured fields, while other times it’s important to simply write out everything.
    • Screenshots and other visual information – Along with the written description of the problem, engineering often requires logs and/or screenshots. As the saying goes, “A picture is worth a thousand words.”

    Now that engineering has the vital information it needs to fix the defect, it has a responsibility to provide useful information back to the help desk:

    • Planned release – This should not be a firm commitment, but rather an estimate to give the help desk general timing. While the actual date of the fix may vary, it’s still useful to be able to tell a customer,”This is coming soon,” rather than, “We realize this is a problem and it’s on our backlog to fix.”
    • Engineering status – While this is a single defect to your customer and to them only has one status, there are really two statuses your organization will care about:
      • Status of the ticket in your ITSM tool
      • Status of the defect in your defect tracking/agile planning tool

    It’s very important that the ITSM tool contains both statuses to provide more information and context to the help desk. Imagine the benefit of knowing that the defect is “In Triage,” “In Progress,” or “In Verification” and being able to provide that information to a customer.

    • Actual release/released – As opposed to the planned release information, this lets the help desk know that the customer’s issue has been resolved. Imagine running a report from your ITSM tool indicating the defects fixed in your most recent release. Your IT help desk can proactively reach out to customers to let them know to upgrade, enabling you to offer more proactive customer communications.

    Finally, and perhaps most important, you allow your teams to talk about the “same thing” even when it exists in different tools and they speak a different language. There are four core ways to do this: 

    • Cross-tool tracking – Each artifact in each system should have the ID (and ideally URL) of its twin artifact in the other system. If a help desk representative needs to talk to an engineer about a defect, the rep needs to let engineering know which defect it is in the engineering tool, not just in the ITSM tool.
    • Routing – IT help desks often support multiple products, each of which have multiple components that may belong to a different engineering team. The help desk needs some indication which component and/or team should work on the defect. This information will help the IT help desk know which team to reach out to for help.
    • Filtering – Not every ticket in the ITSM tool needs engineering help (that’s a good thing). However, there should be some indication on the IT ticket if it has been sent to engineering. You do not want to send the same defect to engineering twice (confusing) or to assume it’s been sent when it hasn’t (poor time to resolution or issue overlooked).
    • Communication – IT and engineering need a shared way to talk about a specific defect that they both care about. The closer this communication can be to the native commenting that already occurs, the better. Ideally this is not via email as that communication vehicle does not stay attached to the artifacts in question, meaning information can be hard to find or get lost.

    The Benefits

    Customers will love it – Improved communication means happier customers. Customers can plan their upgrades with more certainty, and their trust grows since your organization is able to deliver on its promises. No longer will they feel like their issues are being thrown into a black hole.

    Better engineering focus – Engineering gets pulled in a variety of directions, contending with feature work as well as internally generated defects. The more information it has, the easier it is to prioritize and deliver more value to customer. Without the ability to share the severity of a defect, you risk two undesirable outcomes:

    • The defect is fixed instead of more important work, which hurts all your other customers.
    • Other, less-important work is prioritized, which can threaten your relationship with this customer.

    Better trust across your organization – Open communication fosters trust and understanding. Providing insight into prioritization and status let’s your help desk feel part of the development process. No longer will staff feel like their requests are being ignored with no idea when they will be fixed. Enhanced cross-team communication also helps engineering to avoid feeling like work is simply being thrown over the fence. Obtaining more context around a defect is invaluable.

    Next Steps for Total DevOps Integration

    To fully integrate your ITSM and engineering teams, there are still (at least) two other processes to implement:

    • A method for the IT help desk to provide enhancement requests to your product team: These enhancement requests are similar, but distinctly different from the defects described above. Consequently, they need to be prioritized and triaged by a different team.
    • The other process to implement is the traceability from task to code to build and back again. To properly inform customers that their problem is fixed in this version of your product, it’s important to correlate features, stories and bug fixes with the actual version of your product that includes them for the first time.

    Bridging the gap between ITSM and engineering teams brings your company closer to customers. While making your organization work as a unified whole is an incredibly complex problem, addressing this relatively small, but vital, piece of the puzzle helps you on your journey to integrating your whole software value stream. This change of mindset, combined with connecting all teams, tools, disciplines and processes in an enterprise’s software value stream, optimizes collaboration, productivity, efficiency and most importantly, value delivered to the customer.

    About the Author / Trevor Bruner, Product Manager, Tasktop

    Trevor Bruner is Product Manager at Tasktop Technologies. His background includes experience in financial services, oil and gas and as a Navy submarine officer. Connect with him on LinkedIn and Twitter.

     

  • Configuration Files and DevOps

    Configuration Files and DevOps

    When DevOps appeared on the scene, configuration file management tools already existed. They weren’t used by everyone, but a fair number of organizations saw the power of configuration file versioning/management, and were using them.

    An interesting thing seems to have happened while DevOps evolved. Those tools are more popular, but while we’ve standardized on things like Puppet and Chef for installs, and Jenkins for continuous integration/continuous delivery (CI/CD), configuration file management seems to have gotten less standardized.

    While it is easy to argue (convincingly, IMO) that most people use Git for configuration file versioning, there are a ton of other solutions out there being utilized also. And plenty of places that have no solution in the works.

    There are even a few new vendors cropping up that are basing their existence on the idea that configuration file management should be similar to using the source code git repository, but not the same as using the source code git repository.

    What Are We After?

    That brings us to the question, What is it that we hope to gain from configuration file management? Which should take us back one step further to ask, “Which configuration files?” There is a vast difference between configuration files for a piece of network gear, the configuration files for infrastructure apps such as Docker and the configuration files for your application.

    There is a valid argument that all of the above are relevant and should be stored somewhere, but that argument does not imply they should all be stored in the same manner, nor with the same frequency.

    What does make sense is standardization on a given platform with version control capabilities. The question then becomes, Which platform?

    And That’s Where We Are

    My company uses Git for this storage, right next to application code. But we use regular storage with a naming scheme for switch and router configuration backups because we want them backed up with our other file-based data. Most companies are similarly split.

    The simple answer is, “Throw it all into Git,” and I think that’s where most DevOps teams are. But I don’t think we have the long-term answer in front of us yet. A toolset that can manage and version, and is aware of the idiosyncrasies of popular hardware/software/cloud solution configuration files is likely where we’ll end up. That will offer easier use, ease of management, a centralized store for configurations and some automated check upon the inevitable manual configuration changes.

    Until Then …

    Until then, it is best not to vary too far from the best fit for your organization. We’re content that we know where to get all of our configuration files back from, even if it is split between two platforms. There is one single piece of hardware in our infrastructure that in an emergency would be lost, and that’s because it is massive and only supports backup to attached USB disk.

    For most organizations, Fit is “good enough,” and allows for spawning new copies of servers/apps with the correct configuration during the deploy process, so is probably where you want to be right now.

    For enterprise grade networking gear, the configuration files are complex, and unlikely to be hand-edited these days (though we all know that person who can). So any system that gives you a backup of the configuration file you can re-import is good enough.

    For now.

    — Don Macvittie

  • SlashData Report: Developers Influence Tool Selection

    SlashData Report: Developers Influence Tool Selection

    SlashData has just released its “State of The Developer Nation Report for Q3 2017.” Here at Heavybit, a program that helps developer and infrastructure startups take their product to market, we look forward to SlashData’s reports that allow us to keep up to date on the shifts in the developer landscape.

    In 2013, Stephen O’Grady published, “The New Kingmakers: How Developers Conquered the World,” sending shockwaves through the ecosystem about just how important individual developers are when it comes to making technology purchasing decisions in companies.

    SlashData provides compelling data that backs up O’Grady’s book, showing how influential developers are within an organization and how targeting individual developers at organizations is a viable strategy for enterprise sales. The research only includes professional developers in organizations who buy tools or components and exclude lone wolves—developers who work on their own and where a purchase decision is purely personal.

    Key report highlights Include:

    More than 87 percent of developers with a leadership function, no matter how small, as well as two-thirds of front-line coders, are somehow involved in purchasing decisions.

    For developers that are not in a top leadership position, between half and two-thirds are in a position to make recommendations or influence decisions. In particular team leaders (i.e., senior developers) are big influencers (68 percent).

    Thirty-nine percent of developers acquire tools for their personal use.

    The ability for developers to pick and use their own tools shows that they have the power to make their own decisions and a budget to spend (even if it is small). It’s through this personal use that a tool can get introduced to an organization and is the first step in evangelizing the tool to the greater developer community.

    Up to a third of mid-level developer leaders (team leaders and product managers) hold the pen when writing specs for tools (32 percent and 31 percent, respectively), or even get the final word on which tool to adopt (30 percent and 27 percent).

    This fact is even more proof that individual developers at all levels matter. “Even when in a majority of cases developers still need management to sign off on purchases, the days are over when an inadequate tool could be pushed from the top down because it appeared attractive to CFO’s or top brass,” the report states.

    Developer influence, when it comes to making technology decisions in companies, continues to grow. To harness this shift, companies need to focus on creating tools that developers will come to love and evangelize on your behalf within their organization.  

    — Malia Powers

  • Continuous Deployment and Monitoring of Microservices

    Continuous Deployment and Monitoring of Microservices

    Continuous deployment and continuous monitoring have grown up in DevOps, and microservices have grown up as a delivery platform. While all are relatively stable concepts at this point, there are still a fair number of practitioners that are struggling with the merging of these concepts to create a holistic environment that allows for deep understanding and easy deployments.

    So I thought we could take a look at what’s out there, how it’s going, and even mention a resource or two for you.

    Continuous Deployment

    While continuous delivery (CD) and continuous integration (CI) are well in hand, continuous deployment has lagged a bit behind. There are reasons—both good and bad—why this is true, but true it is. Continuous deployment is the act of actually deploying the results of the CI/CD cycle. The whole point of continuous deployment is to speed the release of new features to market by releasing them as soon as they pass the CI/CD muster, or soon after, or for some industries, on demand. Continuous delivery is a little different from CI/CD in that the rest of the organization—sales, support, marketing, accounting, etc.—have to be ready for the release, so in the slowest case, the point is that IT is not the hold-up to releasing new features under the continuous deployment model.

    Microservices

    Simply put, microservices are decomposition of an application into smaller parts that can be run independently. There are a lot of different ideas for what “smaller parts” means, with some shops breaking their application down to the functional level similar to serverless functions, so an API named calculateSalesTax would do that and only that, given location and dollar information, while others prefer a more broad definition that includes entire groupings of APIs. In this scenario, one company split all database work into a single microservice that was scalable. Either way, you will have more instances (normally containers, but sometimes VMs or even physical servers) running to build the application than historically would be the case.

    Why Containers

    As mentioned above, normally a microservices architecture is built on containers. The portability of a container definition is one of the major reasons why companies lean this direction. The ability to say, “This container can run on our hardware, on our private cloud, on a public cloud, in a public cloud VPC or on a VM,” is a way to ensure that you can target the system to the environment most suited to the organization’s needs.

    Containers are also far lighter than VMs, meaning faster spin-up times, and an easier time expanding the microservices architecture to meet changing user needs.

    Container Management

    Speaking of containers, container management systems are used to make certain that containers are alive and responding. In advanced cases, they can be made to check that a container is doing what it is supposed to do also. While I am a fan of Apache Mesos, the world seems to have gone for Kubernetes, so we’ll focus on it, without further reference to the others out there.

    On top of making certain the correct number of a given container are running, and taking steps to spin up new ones if one disappears, Kubernetes does things such as manage secrets for containers, perform load balancing between instances in a group (these groups are generally called a service in container management speak), scaling up/down of instances as directed, and provide rolling updates to services so the service is not down while being updated.

    This makes a container management tool an important part of any continuous delivery and continuous monitoring effort.

    Taken together

    That gives us tools including Jenkins for CI and CD, and often for release management, though other tools specialize in the release portion, too. Combine Jenkins with an infrastructure-as-code solution such as Teraform or any of the others we’ve discussed in the past. Add in a tool such as Prometheus for gathering monitoring data, and a tool such as Grafana for visualizing the data, and you’re on the road to continuous delivery and continuous monitoring.

    Of course, it isn’t that simple, I’m overviewing here. While researching this blog, the people at Infostretch shared some information with me, and they too would see this as the 50,000-foot view. But that’s where we start. For a more complete picture, here is a simple diagram of continuous deployment that Infostretch shared with me:

    And as long as we’re discussing the information that Infostretch shared with me, let’s also look at a sample of what the company monitors. You’ll see it isn’t “everything under the sun,” but focused points that give them actionable intelligence. A few application-specific monitoring points from within the application, and this turns into an application performance gold mine.

    Classification Metrics
    Cluster Metrics Overall Cluster Memory Utilization
    Overall Cluster CPU Utilization
    Overall Cluster FS Utilization
    Multi-series graph for individual service CPU Utilization
    Multi-series graph for individual service Memory Utilization
    Multi-series graph for individual service FS Utilization
    Application Metrics

     

     

    		 Service Request rate
    Service Error rate
    Service latency

    There is so much more

    There really simply isn’t enough room in a single blog to cover all that is required and possible in continuous delivery and continuous monitoring. I’ll continue to try continuous blogging to help you out, and the other authors here on devopsy.kinsta.cloud have good things to share on the topics also.

    — Don Macvittie

  • OutSystems: Low-Code Apps Require Sound DevOps Processes

    OutSystems: Low-Code Apps Require Sound DevOps Processes

    As more IT organizations embrace low-code application development platforms, many of them are discovering that accelerating the rate at which they develop applications can be too much of a good thing.

    António Barros, product manager for OutSystems, says most of the organizations that have adopted the company’s namesake low-code platform are just now coming up to speed on automated testing. Barros estimates that only 20 percent of the organizations using the OutSystems platform are also employing automated testing tools. The majority of OutSystems customers continue to rely on manual testing even though it’s apparent low-code platforms will produce applications much faster than manual testing can keep pace with. OutSystems has identified five stage of DevOps maturity, he notes, but none of its customers has achieved that level of DevOps maturity.

    To help them get a little further down the DevOps path, OutSystems, at its recent NextStep 2017 conference, announced an extension of its partnership with Indigo to provide tighter integration with the OutSystems platform. In addition, OutSystems plans to make available digital transformation playbooks, including one for DevOps, beginning in 2018. Outsystems has already integrated its platform with the Jenkins automated server to provide continuous integration and continuous development (CI/CD) capabilities.

    Barros says it’s not only critical to engage in continuous testing, but the application needs to be designed to make it easier to isolate user interface, application programming interface (API) and unit testing. Otherwise, automating the testing process becomes more difficult than it should be, says Barros.

    Developers for years have resisted low-code platforms, which automatically generate much of the code that needs to be written for an application. Some fear the platforms can’t build applications that scale. Others have simply decided they prefer to write code themselves. But these days, organizations are under more pressure than ever to deliver applications faster, which has led many organizations to rely more on low-code platforms such as OutSystems.

    It’s not clear just how popular low-code platforms will become. OutSystems revealed at its conference it has signed up 122 new customers in the last quarter alone. On average, it now takes organizations eight weeks to build and deploy an application on the OutSystems platform, with new release cycle updates occurring every 48 hours, says OutSystems CEO Paulo Rosado.

    Michael Facemire, an industry analyst with Forrester Research, told conference attendees that adoption of low-code platforms is now a matter of economics. The starting salary for a developer in the San Francisco Bay area is $145,000, says Facemire. As the salary of a developer with a few years of experience now reaches more than $200,000 in the Bay area, companies in that geography will have no other option but to employ low-code platforms, he says.

    Obviously, developers might be less expensive in other regions. But Facemire notes that no matter what the region, developers are in short supply. That need for speed, he adds, is also why organizations are rapidly adopting containers and serverless computing framework. The challenge IT organizations will face is finding a way to put quality control processes in place to don’t compromise either the speed of the development process or the quality of the end user experience, he says.

    By the same token, however, no one really appreciates code that arrives late, no matter how great it might be.

    — Mike Vizard

  • DevOps Predictions for the Future

    DevOps Predictions for the Future

     Among the few powerful trends we had experienced in the recent times, one is undoubtedly the adoption of DevOps practices. As we are move closer to 2018, it is becoming more obvious that DevOps is becoming mainstream, with many large organizations getting on board with DevOps.

    DevOps Within Enterprises

    DevOps is becoming the standard way of working for businesses. Adoption of DevOps within the organization is rising on a broader scale, and businesses are trending toward it.

    Enterprises of today are well-aware of the trending technology services in modernizing environments. DevOps is growing its recognition due to changes in complexities of today’s software that lies more in the diverse set of operation systems and platforms. It is less about doing things a particular way and more about moving the business forward and turning technology into a strategic advantage.

    DevOps builds upon best practices to help drive enterprise performance in the modern digital economy. It offers organizations a new way to deliver value to their customers quickly. An increasing number of businesses are recognizing the power that DevOps can bring for operational efficiency and overall performance.

    DevOps Key Trends

    As businesses are trending toward DevOps adoption, let’s look at the changing landscape and key trends in DevOps, along with its trajectory.

    Some of the DevOps trends include:

    • Many organizations will challenge agile and recognize DevOps as the new ALM methodology.
    • Many DevOps automation tools are going to roll up.
    • Automation and testing would gain their prominence.
    • DevOps Engineers are highest paid IT practitioners today.
    • Testing and deployment are done more frequently as Ops people are getting their DevOps knowledge in production for testing and automation operations.

    DevOps Predictions

    Based on the market so far, here are my predictions for the future of DevOps.

    Adoption of Microservices and Containers

    Several software enterprises are pushing toward microservices infrastructure. Microservices architect an enterprise solution, independently, over a set of services.

    Microservices are best-suited for DevOps due to its features including:

    • Isolated nature
    • Improvised cycle times
    • Well-defined interfaces
    • Aligned architecture

    Containerization also is growing in popularity. Containers are designed to run one process at a time with minimal deployment, serving as an ideal deployment model. According to one study, 50 percent of enterprises are expected to run containers in production by the end of 2018.

    If an enterprise has decided to leverage microservices and containers, it is highly essential to incorporate solutions that support microservices and containers along with traditional application model.

    Culture, the Top Prerequisite

    Culture is the key differentiator for DevOps. Culture and process are the principal paths for the successful DevOps journey. Culture breaks down the organizational silos and encourages integration across all the groups that need to work together to achieve a great business outcome. Implementing DevOps culture makes an organization achieve success easier. It even enables continuous learning environment that will see bigger benefits.

    Notion of Shifting Left

    DevOps tools and solutions always prefer to shift left. DevOps orchestration tools automate from end to end, making traceability easier. Activities that are performed traditionally, at the later stages of development and release process, are making a shift to the left of the pipeline. The ultimate goal of it is to recognize issues earlier, along with performance and disaster recovery testing.

    Downtime   

    Adopting DevOps holds a lot of benefits. For one, it enables enterprises to ensure uptime and enhance cadence and quality. High-performing IT organizations employing DevOps practices deploy 200 times more frequently than the low-performing enterprises. Similarly, they also enjoy faster recovery from failure and minimize downtime.

    Collaboration of Big Data and DevOps

    Big Data and DevOps are together taking a step forward toward the design of predictive analytics. DevOps has now become an expert in automating the process and automating configuration.

    Consolidation is Critical

    Large and complex organizations are focusing on consolidation and standardization of DevOps processes so as to scale DevOps within the enterprise. A shared and self-service DevOps automation enables end-to-end pipelines. Eventually, consolidation allows reusability, visibility, auditability and security.

    Supporting Hybrid

    “Hybrid-everything” makes DevOps solutions future-ready to support any tool, application and process in the hyper-hybrid state. “Hybrid” state of existence is achieved for infrastructure, architecture, release pipelines and more. 

    DevOps Monitoring                       

    Until now, the focus has been on pre-production provisioning. However, more attention is being given to the post-deploy feedback loop. With the application of new architectures including microservices, new monitoring technologies and pricing models are effectively introduced.

    Expansion of DevOps Across IT

    DevOps is maturing in other areas such as database management, security and testing, all with the goal of application release.

    Business-centric Agile

    The development methodology approach is becoming more critical to the continuous integration and continuous delivery process.

    DevOps Skills 

    People with DevOps skills are in demand in the industry and expected to proliferate as businesses are witnessing great results from DevOps. Organizations incorporating DevOps training practices are extremely high-functioning. If you aim to build your career in DevOps, below are certain notable skills.

    Most common DevOps positions include:

    • Site reliability engineer
    • DevOps engineer

    The core skills of DevOps are:

    • Coding skills
    • Knowledge of automation tools
    • Communication skills

    Roles of a DevOps Engineer

    Many DevOps developers find interest in deployment and network operations. A core set of DevOps skills includes:

    • Ability to code and script using various technologies and tools.
    • Close relation with Systems and IT operations.
    • Strong base of automation tools and its skills.
    • Incremental code testing and deployment.
    • Strong focus on data management skills and business outcomes.

    Final Words…

    DevOps skills will only continue to grow and DevOps is here to stay. Dev and Ops teams are increasingly moving to rapid development and reliable, high-performance services delivery.

    DevOps is being adopted by more and more companies, particularly smaller enterprises. Many industries today are keen on incorporating DevOps and its key points into their respective departments. It will continue to evolve for the foreseeable future, by incorporating executives and other business units into it. Companies that utilize DevOps effectively will experience higher innovation and quality and easier adoption to other changing conditions.

    Adoption of DevOps enables applications and technology solutions. It power businesses with faster and better quality. Investing in DevOps helps organizations better develop and release software.

    About the Author / Sirisha Paladhi

    Sirisha Paladhi is a content writer at TekSlate. During her career, she has written many articles on technology innovations.

     

    LinkedIn URL: https://www.linkedin.com/in/paladhi-sirisha-74987881/