devopsy.kinsta.cloud

Category: Blogs

5 Ways Traceability Is Changing to Bolster the Remote Workforce

Traceability has always been a useful tool in the development process, but it has become especially important since the COVID-19 outbreak and the increased remote workforce.

Where developers and engineers were once working side-by-side, with the ability to discuss their process and keep their team informed, they are now navigating a remote collaboration landscape. According to a 2017 study done by Stack Overflow, there is a correlation for developers between remote work and job satisfaction, and the highest job satisfaction ratings are seen from developers who work remote full time. As technology has grown to make this process easier, remote working is expected to become a norm.

Advanced traceability has the potential to hold teams together by allowing increased visibility into each move throughout an entire project. Without traceability, it would be nearly impossible to keep remote teams aligned and on schedule.

Product development tools have continuously been forced to evolve to keep up with the multi-dimensional nature of requirement, test and risk management. For these processes to be successful, all related variables must work together continuously, at scale, and across teams.

We’ve only just started to see how traceability tools can use updated capabilities to streamline the product development process, but we know enough to discuss two things: what’s happening now and what the future may hold.

Traditionally, traceability could be compared to a map. While not limited to a single view, maps exist to help you navigate your way to a destination. Similarly, traceability leads product developers through every step of their processes, eventually helping them reach their goal. If your map was constantly changing you’d never be able to figure out where you were going, but what if people ahead of you could update it as they went along? This would keep you in the loop of the upcoming twists and turns, and no one would feel blindsided. Traceability makes it easy to share similar production changes with your entire team, all at once.

In the traceability process, links are built automatically which lets major decision points, reviews and approvals be captured in final documents and reports. This allows for faster and more informed decision making and these live references can exist across versions.

With the switch to remote work across the globe, and no immediate end in sight, developers don’t have the option of working side-by-side with their colleagues. Because of this, it is pertinent that they are able to streamline their work in every way possible, and increased traceability is expected to be a pillar of that transition.

At the end of the day, connecting the dots is all about making sense of everyone’s decisions. Why did they follow that process, and what factors led them to make the choices they made?

In the early days of traceability there wasn’t always room to include every piece of information that you might find relevant down the line. Legacy tools such as the act of manual reporting through Word documents left much to be desired. Today, that’s becoming less of an issue, as developers are allowed more meaningful accountability and insight into interconnections as they happen.

As we look ahead, we can expect to see these capabilities grow in five different ways.

Less Manual Effort

Updating your traceability reports with as little manual effort as possible will be essential to a streamlined, remote workforce. As traceability becomes more advanced, the work to create and maintain it should not become more difficult.

More Nuance Captured, Related and Parsed for Meaning

Improvements in software will make data gathering more precise, and increase the number of items a team can consider relevant, traceable information. This will take additional workloads off of the developers as more items are automatically traced and easily accessible.

More Attributes Can Define and Predict a Project’s Success

If you have a more data-rich, detailed record of activities, it is possible to understand the past with more context and less reliance on costly manual documentation or memory.

Dispersed Systems Holding Relevant Information Will Start to Feel Closer and More Interconnected

Through more dense integration and aligned processes, systems feel more controlled and connected, giving users a more seamless experience.

Expansion of Communication and Review Capabilities for Stakeholders Who May Be Impacted by Changes

The availability of a full audit trail of participation will make a big difference in the ease of the review process. The ability to easily see who is impacted by changes will allow teams to proactively manage the effects of said changes.

In response to the COVID-19 pandemic, and the number of companies who are maintaining their remote work policies, we can expect to see more innovations related to product development workflows. These innovations will enable easier remote collaboration between colleagues and across teams, overall streamlining the product development process.

June 24, 2020
COVID-19 Long Tail Budget Impact

Technology budgets have shifted dramatically since the beginning of the COVID-19 pandemic, according to a recent study from online CIO networking and information hub, Pulse. The pandemic, and its resulting shutdown, will likely continue to have a substantial impact on technology budgets for some time.

The initial budgetary impact was substantial. When COVID-19 first arrived, many technology budgets were frozen in place. Today, 23% of CIOs report that their budgets are still frozen with 49% reporting a decline in their technology budgets. Seventeen percent said that they expect to decrease their budgets by more than 10% in the next two months.

Fortunately, it’s not all bad news. Many respondents, 24% of them, predict that within the next two months their budgets will rise, including 9% of respondents who believe their budgets will increase more than 10%. However, most CIOs (27%) say that their IT budget won’t change in the near future.

“And an additional 6% anticipate their budget will become frozen in the next two months — though, on a positive note, this figure is down from 23% two months ago,” stated the report.

Surprisingly, there hasn’t been a change in business-technology priorities over the past two months for 79% of respondents, while 21% said that they now have different priorities than from the beginning of the pandemic.

The priorities of those surveyed include focusing on internal communication and collaboration tools (45%), mobile security (39%), customer communication and collaboration tools (37%), password management tools (28%), virtual private networks (28%), security awareness training (24%), remote desktop tools (23%), encrypted communication tools (23%), and data security tools (15%).

“Many companies are going to allow (or require) employees to work from home after the pandemic, due to health concerns and reduced overhead. The fact that CIOs are investing in security and end-user technology — even though budgets are decreasing — indicates this,” said Brian Sampietro, CIO at TGI Office Automation, in the report.

For those organizations who changed their priorities, the new budget items include virtual private networks (36%), password management (32%), remote desktop tools (30%), data security tools (27%), encrypted communication tools (26%) and mobile security (21%).

How are CIOs making decisions when it comes to technology purchases? The majority said peer recommendations (69%) and industry analysts (52%). However, social media and email fared poorly as influencers, at 23% and 20% respectively. CIOs are determining needs, according to the survey, based on both executive and end-user requests. Only 10% said they depend upon user requests alone, and only 21% said executives solely drive decisions. Interestingly, 35% of CIOs make decisions quickly: Within one week they said they will decide on the right tool to acquire. The rest said they take more than a month (26%) or within four weeks (39%).

When it comes to layoffs, there are still rough patches ahead, as a sizable number of organizations still have layoffs on the planning table. While 48% of organizations have maintained their headcount through the COVID-19 crises, 40% do plan to cut staff sizes in the next six months, with 25% not sure if they will cut staffs.

June 23, 2020
What Developers Really Think About Pentesting

A developer in their natural habitat is often spotted in a state of deep concentration, coding awesome features to tight deadlines. Feature-building is often our favorite part of the job, and really, it’s the fundamental outcome of the software development life cycle (SDLC).

However, as we have discussed before, many of us are still prioritizing features over security best practices. After all, in most organizations, it is set up to be someone else’s job and adequate security training for us is limited. Penetration testing and static analysis scanning tools (SAST) are just part of the overall process to mitigate security risks, operating rather independently from what we do — until the code bounces back to us for hotfixes, of course.

It’s at that moment that many developers think, “Do the pentesters hate me?”

These interactions often define a team and culture. The concerning thing is, the lack of communication, understanding and overall collaboration has created tension, at least on the side of the developer.

Think about it: Imagine you’ve spent a few hundred hours sculpting a marvelous statue, and then someone comes along with a hammer and starts smashing bits off it after they’ve told you its foundations are not up to scratch. That’s the perceived dynamic between a tester and developer — the latter has their software darlings slaughtered by an outsider that hasn’t labored through the process with them; instead, they’ve extended the workload and delayed the satisfaction of shipping code.

Having moved into the security space long ago, I can see both sides of the story. And no, pentesters don’t hate developers. The pentester is in all likelihood, overworked and under a lot of pressure. As such, a constant stream of common security bugs that could be quite easily fixed at the code level take up time, resources and headspace away from the really serious issues.

I always saw pentesters as kind of like parents. They want you to do well, and when you don’t they’re not mad, just disappointed.

Now that I’ve put that (perhaps slightly unfair) image in your mind, let’s explore this a little deeper. What has caused this world view among developers?

Resentment for Being Told What to Do

Nobody likes feeling as though they’ve done a bad job, or that someone doesn’t like their work. Sadly for developers, when static analysis and pentest results come back to them, it can feel like a report card. They’ve been given low grades, but at the end of the day, their bosses assess them on the features they’ve built and the time they’ve delivered them, not whether there were vulnerable elements in the software or not.

For the poor pentester, this is a case of “don’t shoot the messenger.” It’s nothing personal — they are tasked with finding bugs, and they found them. Granted, at a person-to-person level, maybe some pentesters are grumpier than others, but they’re not (or shouldn’t be) out to crucify development teams. It would be far easier for both teams if they were on the same page with what constitutes security best practice. Developers are not expected to be perfect; realistically, the testing team is there to protect them from shipping vulnerable code.

Minor Issues vs. Higher Priorities

It’s true, a developer’s highest priority will always be the building of features, and in this crazy world of rapid digitization, it will have to be done at speed. While some coders have a personal interest in security and secure coding, the general sentiment is that security is “someone else’s problem,” which inevitably includes pentesters.

Most common vulnerabilities are indeed minor issues to remediate — once known, the fixes are simple to execute for things like cross-site scripting (XSS) and SQL injection. The problem is, many developers don’t realize they’re introducing them in the first place, and these seemingly minor issues are the small window of opportunity an attacker needs to cause devastating problems for a company. According to Akamai, between November 2017 and March 2019, SQL injection vulnerabilities accounted for 65% of all web-based attack vectors. For a vulnerability that has had a known fix for more than 20 years, that is a sobering statistic.

Some pentest teams do assist in the remediation of security bugs, but others will provide a report of the bad news and expect developers to work through hotfixes, even if they have moved onto a different project by the time this happens. And in some cases, the development team may be faced with a report that includes bugs they can’t (or shouldn’t be expected to) fix. It still has to be part of the findings, and again, not taken personally.

The “happy medium” for this would be pentesters, security personnel and development managers acting in more of a mentor role to ensure the team has what they need in terms of effective training and tools, giving individual coders the best chance to succeed and code securely from the very beginning of the SDLC. Both teams really should be meeting half-way to ensure security is considered from the start, as part of a healthy DevSecOps practice.

Lack of Knowledge vs. False Positives

Static analysis is an element of the security process in the SDLC, and static analysis scanning tools (SAST) play a fundamental role. And yes, false positives are an issue with these and other types (DAST/IAST) of scanners. It’s an annoyance in what is already a slow process, requiring manual code review and putting pressure on both developers and pentesters alike. Pentesting personnel have taken time to meticulously set up custom rules to avoid inaccurate readings and provided company-specific guidance, yet some false readings slip through and end up in front of a head-scratching developer.

This process isn’t perfect, but the other problem is that many developers lack enough knowledge to mitigate a lot of common vulnerabilities on a consistent basis. With security training rare in tertiary education, and on-the-job training varying in its effectiveness, it stands to reason that there may be some overconfidence at play as well (and it’s not their fault, we as an industry need to get better at equipping them with what they need).

Remediation Tasks

Sometimes, there is an assumption by overworked engineers that pentesters are just hanging in the wings, waiting for the moment to strike by testing an application and raining on the development team’s parade. They are over-testing, they’re nitpicking, they’re creating extra work.

The only problem with that is they, too, are overworked (more so, in fact, the cybersecurity skills shortage is at dire levels and getting worse) and simply don’t have the time to test without reason. They are not the sole decision-makers in prioritizing testing; it could have been requested by senior leadership, a customer, as part of a security audit or even determined as a result of a bug bounty program.

For a developer, being pulled off current feature-building sprints to work on security fixes is annoying — especially if it’s not their work. Perhaps a previous team did the last update, or another vendor. However, security is everyone’s problem. That doesn’t mean every developer has to take ownership of security bugs as though they have made them all themselves, but they do need to come to the party in terms of security being a shared responsibility.

Where to from Here?

Sometimes, a mindset shift can be all it takes to make significant headway in solving a problem. We’ve talked about the rather frosty reaction a developer has to less than favorable pentest results, but what if they could turn it into a challenge? Perhaps they could think of the pentester as a friendly competitor; someone they can beat at their own game. After all, a security-aware developer that can eliminate common bugs as they write code is going to make their job much more difficult. By contrast, a developer with no focus on security is going to be comprehensively bested by their pentester counterparts when they can easily break their software.

Pentesters and developers may not be joined in harmony 100% of the time, but their relationship can be vastly improved when an organization addresses security as a key priority, and empowers teams with the right knowledge and tools to succeed — especially developers. It comes down to whether a company-wide, positive security culture is a priority, and if we are to fight the (currently) losing battle against common vulnerabilities, it absolutely should be.

June 17, 2020
Report: SD-WAN Shift to the Cloud Accelerates

The rate at which software-defined wide area networks (SD-WANs) are being consumed as a cloud service appears to be accelerating in the wake of the COVID-19 pandemic.

A global survey of 750 IT professionals conducted by the market research firm Vanson Bourne on behalf of Barracuda Networks finds over half of respondents (52%) are looking to acquire an SD-WAN solution via a cloud provider, with Microsoft Azure being cited as the most preferred option.

Only 16% said they prefer an SD-WAN from an independent vendor, while 15% would opt for a telecommunications provider. Another 12% said they would rely on a reseller.

Klaus Gheri, vice president of network security for Barracuda Networks, said the survey suggests that as application workloads move into the cloud, more IT organizations want to centralize acquisition of networking services alongside compute and storage. As such, he said, providers of SD-WAN appliances such as Barracuda Networks will need to evolve to extend the basic SD-WAN capabilities provided by cloud service providers.

At the same time, Gheri noted that way applications are being accessed is also evolving as more employees are working from home. Rather than directly accessing applications running in a local data center or office, many organizations are routing access requests to those applications through their cloud service provider. That approach reduces costs by using a public internet connection to access to cloud service, which then routes that request over a virtual private network (VPN) to a local data center.

Given the egress charges a cloud service provider requires, that approach may not reduce costs in the case of every workload. However, it may reduce costs by reducing the amount of network traffic moving across a leased MPLS line, he noted.

Gheri said the rise of the cloud as a networking conduit illustrates how varied SD-WAN implementations have become. Organizations increasingly need some capability at the client level to automatically determine which network path makes the most sense for any request based on cost and latency requirements.

In the meantime, frustrations with VPN clients continue to mount. Most VPN implementations were designed to support a small percentage of employees for occasional access. With more employees working from home more frequently, many organizations are reconsidering their SD-WAN strategies. In addition to being able to scale services more easily, Gheri noted SD-WANs also afford organizations the opportunity to implement zero-trust networking services to better isolate application environments in the event of a security breach.

Over the last few years, SD-WANs have been one of the fastest-growing categories in IT. The Barracuda survey found 23% of respondents have already deployed SD-WAN, with another 51% either in the process of deploying or expecting to deploy one within the next 12 months.

Clearly, competition between cloud service providers, telecommunication carriers and independent providers of SD-WAN services will intensify in the months ahead. The challenge facing IT teams now is determining to what degree they want to be directly involved in managing SD-WAN networks versus simply consuming SD-WAN services on an as-needed basis.

June 15, 2020
Measuring DevOps Performance
As IT becomes increasingly central to our organizations, it is increasingly important to improve our ability to deliver innovations efficiently and safely. DevOps is a movement to reimagine the way we deliver software, with an emphasis on delivering value to end users through automation and collaboration. In the midst of complex changes to complex processes, it’s easy to lose sight of the most important point: Our “improvements” must deliver actual improvements. Measuring the performance of a software delivery team is the basic foundation on which you can assess the impact of changes.

One of the main contributions of the State of DevOps Report has been to focus consistently on the same key metrics year after year. Although the questions in their survey have evolved and new conclusions have emerged over time, the four key metrics used as benchmarks have remained in place:
1. Lead time (from code committed to code deployed)
2. Deployment frequency (to production)
3. Change fail percentage (for production deployments)
4. Mean time to restore (from a production failure)
The book “Accelerate” provides a detailed explanation of each of these metrics, and why they were chosen; those points are summarized here.

The first two of these metrics pertain to innovation, and the fast release of new capabilities. The third and fourth metrics pertain to stability, and the reduction of defects and downtime. As such, these metrics align with the dual goals of DevOps, to “move fast, and not break things.”

These also align with the two core principles of lean management, derived from the Toyota Production System: “Just in time” and “Stop the line.” “Just in time” is the principle that maximum efficiency comes from reducing waste in the system of work; and that the way to reduce waste is to optimize the system to handle smaller and smaller batches, and to deliver them with increasing speed. “Stop the line” means the system of work is tuned not just to expedite delivery, but also to immediately identify defects to prevent them from being released, thus increasing the quality of the product and reducing the likelihood of production failures.

Lead time is important because the shorter the lead time, the more quickly feedback can be received on the software, and thus the faster innovation and improvements can be released. The book “Accelerate” revealed that one challenge in measuring lead time is it consists of two parts: time to develop a feature, and time to deliver it.

The time to develop a feature begins from the moment a feature is requested, but there are some legitimate reasons why a feature might be deprioritized and remain in a product’s backlog for months or years. There is a high inherent variability in the amount of time it takes to go from feature requested to feature developed. Thus, lead time in the State of DevOps Report focuses on measuring only the time to deliver a feature once it has been developed.

The software delivery part of the lifecycle is an important part of total lead time, and is also much more consistent. By measuring the lead time from code committed to code deployed, you can begin to experiment with process improvements that will reduce waiting and inefficiency, and thus enable faster feedback.

Deployment frequency is the frequency of how often code or configuration changes are deployed to production. Deployment frequency is important since it is inversely related to batch size. Teams that deploy to production once per month deploy a larger batch of changes in each deployment than teams who deploy once per week. All changes are not created equal. Within any batch of changes there will be some which are extremely valuable, and others that are almost insignificant.

Large batch sizes imply that valuable features are waiting in line with all the other changes, thus delaying the delivery of value and benefit. Large batches also increase the risk of deployment failures, and make it much harder to diagnose which of the many changes was responsible if a failure occurs. Teams naturally tend to batch changes together when deployments are painful and tedious. By measuring deployment frequency you can track your team’s progress as you work on making deployments less painful and enabling smaller batch sizes.

Change fail percentage measures how frequently a deployment to production fails. Failure here means that a deployment causes a system outage or degradation, or requires a subsequent hotfix or rollback. Modern software systems are complex, fast-changing systems, so some amount of failure is inevitable. Traditionally it’s been felt that there’s a trade-off between frequency of changes and stability of systems, but the highly-effective teams identified in the State of DevOps Report are characterized by both a high rate of innovation and a low rate of failures. Measuring failure rate allows the team to track and tune their processes to ensure that their testing processes weed out most failures before they occur.

Mean time to restore (MTTR) is closely related to the lead time to release features. In effect, teams that can quickly release features can also quickly release patches. Time to restore indicates the amount of time that a production system remains down, in a degraded state, or with non-working functionality. Such incidents are typically stressful situations, and often have financial implications. Resolving such incidents quickly is a key priority for operations teams. Measuring this metric allows your team to set a baseline on time to recover, and to work to resolve incidents with increasing speed.

In 2018, the State of DevOps Report added a fifth metric, system uptime, which is inversely related to how much time teams spend recovering from failures. The system uptime metric is an important addition for several reasons. First of all, it aligns with the traditional priorities and key performance indicators of sysadmins (the operations team). The number one goal of sysadmins is keeping the lights on or ensuring that systems remain available. The reason for this is simple: the business depends on these systems and when the systems go down, the business goes down. Outages are expensive.

Tracking system uptime is also central to the discipline of site reliability engineering (SRE). SRE is the evolution of the traditional sysadmin role, expanded to encompass web-scale or cloud-scale systems where one engineer might be responsible for managing 10,000 servers. SRE emerged from Google, who shared their practices in the influential book Site Reliability Engineering. One innovation shared in that book is the concept of an error budget, which is the recognition that there is a trade-off between reliability and innovation, and that there are acceptable levels of downtime.

According to the Site Reliability Engineering Book, Chapter 3, “Put simply, a user on a 99% reliable smartphone cannot tell the difference between 99.99% and 99.999% service reliability. With this in mind, rather than simply maximizing uptime, Site Reliability Engineering seeks to balance the risk of unavailability with the goals of rapid innovation and efficient service operations, so that users’ overall happiness — with features, service, and performance — is optimized.”

The State of DevOps Report shows how these five metrics are interrelated (See Figure 1). The timer starts on lead time the moment a developer finishes and commits a feature to version control. How quickly that feature is released depends on the team’s deployment frequency. While frequent deployments are key to fast innovation, they also increase the risk of failures in production. Change fail percentage measures this risk, although frequent small deployments tends to reduce the risk of any given change. If a change fails, the key issue is then the mean time to restore service. The final metric on availability captures the net stability of the production system.

Figure 1: Correlation between the five metrics.

How the Five Key Software Delivery and Operations Performance Metrics Tie Together

Together, these metrics constitute a team’s software delivery performance. The goal of any DevOps initiative should be to improve software delivery performance by strategically developing specific capabilities such as continuous delivery and the use of automated testing.

How your team measures these capabilities is another challenge. But “Accelerate” makes a compelling argument for the validity of surveys. Automated metrics can be implemented over time, although the mechanism to do this will depend on how you do your deployments. Salesforce production organizations track past deployments, but it’s not currently possible to query those deployments, so you would need to measure deployment frequency (for example) using the tools you use to perform the deployments. Salesforce publishes their own service uptime on Trust, but that gives no indication of whether critical custom services that customers build on Salesforce are in a working state or not.

Surveys provide a reasonable proxy for these metrics, especially if responses are given by members of the team in different roles. Guidelines for administering such surveys are beyond the scope of this book, but your teams’ honest responses are the most critical factor. Avoid any policies that could incent the team to exaggerate their answers up or down. Never use these surveys to reward or punish; they should be used simply to inform. Allow teams to track their own progress and to challenge themselves to improve for their own benefit and for the benefit of the organization. As it says in the Agile Manifesto “At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly.”

Metrics provide a reliable, long-term indicator of how your software delivery team is performing. They open the door for your team to experiment with different approaches and assess their impact using a common standard. The key metrics described here are important because they emphasize end-to-end performance, and thus incent teams to focus on collaboration towards this common goal. Balancing velocity with reliability is critical, thus these metrics should be viewed together, to ensure that one goal is never emphasized at the expense of the other.
June 15, 2020
Why the Success of Edge Computing Relies on a Linux Legacy

At this point you’ve certainly heard rumblings of the promises of edge computing and may wonder if it’s right for your organization and how to assess its viability. The way we think about edge is as a decentralized extension, or evolution, of cloud computing, enabling many of the same technologies and applications to be leveraged in areas/locations that weren’t possible before.

I believe that the promise of edge computing and the opportunity it unlocks is valuable enough to be a major driver in our industry over the next decade. Here’s how to think about implementing edge computing in your organization.

Getting Started

Since edge is an evolving space, the exploration of edge computing should start as soon as possible, and we’d recommend a phased implementation approach to help deal with change as it comes.

Many of the companies we work with who delayed their cloud initiatives found that it contributed to their falling behind from a technology standpoint. Given the choice, it’s better to deal with some sharp edges — no pun intended — than to fall behind your competitors in reaping the benefits.

Edge requires careful life cycle analysis at the beginning of the journey, rather than an afterthought. The decentralized nature of edge computing raises the stakes and amplifies any potential problems you would have in other computing scenarios. So as you begin, keep in mind issues such as adaptable infrastructure, security and life cycling between hardware and software.

The Value of Adaptability, Linux and Open Source

Twenty-five years ago, innovation was a slow, laborious, bug-filled crawl — with any advances crippled by proprietary technology. The breakthrough that paved the way for the continuous, rapid pace of innovation possible today was Linux.

We need to heed the lessons from the early days around private cloud adoption and, more recently, Linux containers with regards to adaptability. Many man-hours were lost in slash-and-burn style rebuilding of infrastructures. The logistics at the edge make it impractical to have the same learn-as-we-go freedom we had inside the datacenter.

As we’re likely to see edge computing evolving quickly, the only adaptable architectures will be the ones that can avoid a total tear-down. Adaptable architectures will also be well-positioned to take advantage of the newer capabilities as they become available.

Remember, Linux has a strong history in the embedded world and is well adept at tackling a number of the historic challenges in this space. The more immediate impact is being felt at the distribution level. Our distributions, for example, have objectives around minimization and we expect to see more low-level enablement work on the security side for edge happening in the near future.

For edge computing to reach its full potential, we need to be completely driven by open source and open standards. What we’ve learned from previous technology waves is that while most proprietary technologies have their merits, they miss out on the multipliers that can come from a thriving community.

Final Considerations

Security is a longer topic on it’s own, but given that we’re giving up many of the security barriers that are easy to take for granted in the datacenter and cloud (physical access, networking perimeter, security groups/policy, etc.), we are going to rely more and more on lower-level hardware and operating system security.

No company wants to risk a common vulnerability across a hundred thousand nodes. This makes it our job to keep enterprise deployments from following the pattern of some consumer-grade devices that are long out of software maintenance.

For edge computing innovation, we need to be thinking more about how we create sustainable solutions and technologies given how many deployments will require a longer life cycle and are more tightly bound to hardware and equipment refreshes. The path of innovation leads from Linux to and through the network edge. Companies that follow this approach will be better positioned to leverage the promise and power of the edge while avoiding fragmentation and lock-in.

June 15, 2020
5 Business Insights to Gain From APM

My son is a commercial on-demand cargo pilot, and it’s his job to move around auto parts when they’re needed in the supply chain. During a recent auto strike, he ended up flying a few fenders to a factory for something around $20,000. The profit margin on those fenders was blown to bits, but it would have been far more expensive to interrupt the supply chain — auto assembly lines burn millions per hour.

That kind of individual, transactional economics is insane, of course — no actual fender really costs so much. My point is inefficiencies occur when there’s a hiccup somewhere. They can happen at any point — including when the custom applications that work with suppliers aren’t functioning correctly. The result? The item isn’t ordered, or delivered, on time. And there’s a substantial financial impact downstream.

That’s why slow is the new down. The stakes are high. Here’s five reasons why you need a comprehensive performance management strategy to track and measure all digital touchpoints in your business and mitigate the impact to both your business and the user.

Timing Is Everything

Not too long ago, business ran on an analog model. Customers went to the retailer or contacted them in some way and placed an order. It was the same in B2B interactions; parts were ordered and, once in stock, they were delivered.

The old system worked fine, but compared with the way things are today, that analog model was inefficient. It required more warehousing. Delivery took more time. Overhead was higher. Nothing was “Just in time.” You could call it, “Just you wait.”

Now we’re interconnected, often by custom apps and websites (custom, because each company is different, with different needs that can’t be fulfilled by some “one-size-fits-all” app or site). It works well when it’s working well. But when there’s even a blip, there’s disaster.

Our modern, digitally interconnected system is kept running smoothly by monitoring. Everything about the app and the site is monitored and measured, to a microscopic degree. How is the mobile app performing? How long did the customer stay on a page? How quickly did the B2B app answer a parts order? You can find that out — and fix the problem before it grows.

This is called application performance monitoring (APM). The digital world is full of acronyms, but “APM” is a vital one to know because measurement is now table stakes for any digital business, whether it’s retail, financial or B2B.

Customers Are Fast to Leave When You Are Slow

For execs, it’s job number one to protect and increase market share. Salesforce said 52% of holiday shopping occurs on a mobile device. Last year, Google changed its algorithm to make site speed a ranking factor in mobile search. In short: Go slow, and you won’t get found. You won’t make the sale. You lose market share.

If your customers do manage to find you: A recent study found 95% of consumers in the U.S. and U.K. will jump off an e-commerce or brand site if they’re having a frustrating time. Therefore, you need to know where and when they jump. You can find out with APM.

If your customers are loyal, 60% will still leave you to find that product somewhere else. Customers will simply leave any digital touchpoint that doesn’t keep up. How fast is slow? Google said a mobile site should load within three seconds. Take longer than five seconds, and the probability of bounce increases by 90%.

Customers are fast to leave when you are slow. APM gives you continuous visibility into how your application is performing and helps answer the lingering question, “Is my app good?”

Faster Isn’t Necessarily Better

When cloud first debuted, business leaders loved it. They were able to disconnect from messy interactions with on-premises equipment and the tech pros who maintained it. It was elastic, scalable and agile; they could expediently deploy more resources if something slowed down in the cloud.

At least that was the plan, but the trouble is it didn’t automatically guarantee a better customer experience. It also didn’t make those auto parts arrive on time. Don’t get me wrong, Cargo Dogs love it when you urgently need them to deliver parts. But flying fenders around at the last minute is not a sign things are going well at the part vendor or at the plant.

You still need proactive visibility. With continuous monitoring, you can understand the performance of the application and the infrastructure it runs on.

You can stop reacting and start delivering. APM tells you where you’re getting value and where you aren’t. It knows when you’re spending too much on a feature and not getting much ROI. The secret isn’t spending more money than the other guys. It’s spending it more wisely than they do.

I’m not going to claim APM is a digital shaman. But it does give you a perspective of the cost of running your applications and the performance of running those applications — and it does this proactively.

An example: Earlier in my career, I worked with a business leader who had the application business for a large financial services company. This leader was tasked with ensuring digital interactions were operating in a high-performance fashion. Before he implemented APM tools, he ran with the most expensive systems and infrastructure — both on-premises and in the cloud — on the presumption that massively over-provisioning infrastructure was the safe thing to do.

But users complained his pricey infrastructure was slow or unavailable. His IT staff would look at all their technology metrics — not in the context of application performance, just pure technology metrics — and say nothing was wrong; the infrastructure itself was performing flawlessly. Finally, he brought in APM and instrumented it on a key application and immediately got proactive visibility. He was able to fix the problems with his applications — without buying still more infrastructure.

The Importance of Visibility

Visibility means fewer problems that lead to slowness or outages. But if you do have interruptions, end-to-end visibility shows you the linkage between the application and the infrastructure. Your technology teams can rapidly do root-cause analysis and fix the problem much faster because they’re no longer guessing at where the issues might be.

It means IT isn’t wasting time pointing fingers. It is working off a consistent and standard set of information. Additionally, for traditional IT organizations, APM gives next-level visibility into infrastructure, whether it’s on-premises, cloud, hybrid, etc. It gives stakeholders a view into the metrics that matter.

The Right Metrics — Even During App Development

There are many reasons why applications can’t perform well. They could have other applications contending for resources. And here’s something else: that custom application could be poorly written.

APM is a great help in the development stage of applications because it allows people writing the applications to understand how their application code is performing immediately. That way, they can optimize their code before it goes live.

That app can be written in any number of languages, and it might be public cloud, hybrid cloud, or private, on-premises. APM instruments these apps and allows you to see right down to the line of code how those apps are performing. The likelihood of deploying poorly performing code into production thus minimized greatly.

Conclusion

APM allows companies to roll out better performing applications more quickly with a high degree of assurance that they’re not going to be slow in production. Without APM, you have an unknown level of risk associated with availability and performance of your applications, their transactions and your infrastructure. If you’re not monitoring it and measuring it, you can’t manage it in a low-risk fashion — which is ultimately why APM matters and should be adopted as broadly as possible across your application and infrastructure environment.

June 10, 2020
Cisco Moves ACI Strategy Forward

Cisco is advancing its hybrid cloud computing strategy with release of version 5.0 of Cisco Application Centric Infrastructure (ACI), a software-defined networking (SDN) that can be deployed on cloud platforms or networking equipment deployed in an on-premises IT environment.

The latest version of Cisco ACI adds support for a transit gateway between virtual private cloud (VPC) instances on the Amazon Web Services (AWS) cloud to provide a more efficient means of routing traffic across east-west and north-south cloud computing environments.

Cisco is also extending the networking services it provides for the Microsoft Azure cloud to include VNET peering within the Azure Virtual Network, shared service deployments, additional service automation functions.

Via ACI 5.0, IT teams are now provided a centralized view of cloud resource inventory within AWS and Microsoft Azure along with status indicators and a set of wizards to simplify configurations.

Cisco also has deepened the level of integration between Cisco ACI and Kubernetes as well, to increase the level of scale that can be achieved in IT environments based on platforms such as VMware vSphere.

Finally, Cisco has added support for role-based access control (RBAC) in multi-tenancy environments along with two-factor authentication (TFA) capabilities enabled by the Cisco DUO platform. Cisco also added a more flexible policy construct, dubbed Endpoint Security Group (ESG), that can be employed to group endpoints based on L3 attributes and apply contracts between ESGs.

Srinivas Kotamraju, senior director for product management for Cisco ACI, said the latest update to the SDN platform provides IT teams with the equivalent of many of the same networking services that Cisco historically has provided in an on-premises IT environment.

Cisco has been making the case for a hybrid cloud computing approach to delivering network services that enables IT teams to manage those services via a central console. Rather than having to manage networking services provided by each cloud service provider, Cisco ACI makes it possible to deploy the same SDN across multiple clouds and on-premises IT environments. Cisco contends that approach ultimately serves to reduce the total cost of delivering networking services by both centralizing management and making it easier to automate the management of networking services across multiple IT environments.

There are two primary ways Cisco exposes those networking services. DevOps teams can take advantage of application programming interfaces (APIs) to programmatically invoke ACI services. Alternatively, Cisco provides network managers access to a graphical user interface (GUI) through which they can access a command-line interface (CLI) to configure a specific router or switch.

Kotamraju said with the rise of DevOps the number of organizations giving developer programmatic control over networking services continues to rise. It’s too early to say to what degree NetOps and DevOps teams might ultimately converge; however, Kotamraju said finding ways to meld those two groups is being discussed more frequently within many IT organizations as applications become more distributed.

Of course, the biggest challenge when it comes to achieving that goal will be more cultural than technical. However, as networking services become more accessible via APIs, it’s now more a question of when and how than if.

June 3, 2020
DevSecOps vs. Agile Development: Putting Security at the Heart of Program Development

Despite most developers and managers being well aware of the concept of DevSecOps, it is still often confused with a number of related processes and concepts. This is particularly true for the Department of Defense (DoD) contractors because they have long been encouraged to use a related process known as agile development.

Since agile development has been around for more than a decade, there is a tendency for organizations to regard DevSecOps as an extension, or even a synonym, for it. In reality, though, the two approaches are distinct. The objectives of both approaches are similar: balancing speed and agility, detecting risk early and both focus on cloud-native security and performance.

However, while DevSecOps does build on some agile development principles, such as the continuous integration and delivery of software systems in cycles, its key emphasis from the beginning of the process is to integrate security features, while Agile only focuses on delivering software.

In this article, we’ll look at the key differences between agile development and genuine DevSecOps processes, and how each complements the other.

DevSecOps vs. Agile Development

The difference between DevSecOps and agile development methodologies can be understood in reference to one aspect of software development: security. When, where and who implements security in software development varies between the two approaches.

Agile development methodologies focus on iterative development cycles, in which feedback is continuously reintegrated into ongoing software development. However, even in mature agile development processes, security is still often added to software as an afterthought. This should not be read as blaming software developers for often underestimating the potential harm from malware or overlooking the importance of cybersecurity.

Rather, in many firms, it is simply not the responsibility of developers to think about the security implications of their code, because software will be passed to the security team before release.

DevSecOps takes security and puts it on the same level as continuous integration and delivery. DevSecOps methodologies emphasize security at the very earliest stages of development and make security an important part of overall software quality.

In essence, these approaches shift the program manager’s perspective away from making sure that software is in compliance or meets a specification or audit, to ensuring that the code is written correctly and securely and that it’s deployed in a repeatable manner.

The Changing Culture of the DoD

The adoption of DevSecOps methodologies is critical for DoD contractors because these methodologies reflect DoD’s own modernization strategy. Following high-profile vulnerabilities being discovered in all types of systems, DoD seems to have released that cybersecurity weaknesses are not just a potential threat to critical defense data, but also impose a significant operational bottleneck on the continuous delivery of functionality.

At the moment, even those firms who possess a mature agile development framework can find that the multiple security checks and compliance processes that must be performed on software, prior to shipping, can detrimentally impact on their ability to deliver iterative software improvements. This is the issue that the transition to DevSecOps seeks to overcome, by attempting to shift organizations’ approach to cybersecurity away from compliance, and toward genuine security consciousness.

At the same time, analysts are aware that shifting responsibility for cybersecurity onto developers will be uncomfortable, and potentially dangerous, for many DoD contractors. Developers working on the base code for military software do not have – and are not expected to have – an exhaustive appreciation of all of the contexts in which this will be deployed, and exactly which systems their software will be required to interface with.

Automation and New Models

In response to these concerns, DoD contractors are reappraising the DevSecOps model and thinking seriously about how it can be deployed in contexts where continuous service delivery is key. There are three key ways in which this is happening.

The first is the rise of automation. The types of large-scale, multicloud infrastructures that most contemporary DoD projects rely on can often necessitate repetitive, ongoing maintenance and security assessment across both development and security teams. DevSecOps replaces these human checks with automated systems: Instead of requiring a person to go through checklists of hundreds of controls, this is done automatically as part of the software development and supply process pipeline.

The second key element in the transition to DevSecOps is endpoint security. The types of monolithic development processes that DoD contractors traditionally worked with are not suited to the contemporary deployment environment of many DoD projects. In these traditional methods, systems were built as discrete holes, with no expectation that data would be exposed as it moved between components.

This is now an obsolete approach: In DevSecOps processes, virtual private networks are used to provide security and encrypt data as it moves within a network, and the principle of least privilege reduces staff access to small portions of It environments.

Thirdly, a new generation of younger military and civilian personnel in the DoD, many of them trained in the commercial sector rather than entering the military directly from college, are bringing corporate approaches to the development of military software. This is seen in the exponential increase in the utilization of cloud services in DoD applications, and more dynamic and distributed managerial frameworks that distribute security responsibility across entire organizations, rather than having this as the sole remit of a dedicated security team.

Get Support

Moving to a truly DevSecOps methodology is likely to be a challenge for many contractors. However, just as in the last decade many firms have had to redesign their development lifecycles in order to be agile, now they will need to achieve secure development lifecycles in order to stay competitive.

If you are looking to make this shift, you should also recognize that you are not alone. DoD is working actively with its partners in order to facilitate the roll-out of DevSecOps processes and to make commercial tools and techniques available and useful for military contractors because the department recognizes the value of this methodology for the software they commission.

This is seen most clearly in the recent updates to Microsoft Azure Government, though you should also be aware that many of the resources recently released by the Defense Logistics Agency are also focused on the transition to DevSecOps.

In short, whilst the transition to DevSecOps may be challenging, it will be no more so than the earlier shift to agile development. To make this shift, however, firms should draw on the resources available to them, recognize the value of integrating security into their development workflows and build on existing agile frameworks.

June 3, 2020
Report: Debugging Efforts Cost Companies $61B Annually

Undo, a provider of a software failure replay platform, this week published a report in collaboration with a Cambridge Judge Business School MBA project that estimates 620 million developer hours a year are wasted on debugging software failures, at a cost of roughly $61 billion.

The report also notes software engineers spend on average of 13 hours to fix a single software failure.

According to the report, 41% said identified reproducing a bug as the biggest barrier to finding and fixing bugs faster, followed by wiring tests (23%) and actually fixing the bug (23%). Well over half (56%) said they could release software one to two days faster if reproducing failures were not an issue. Just over a quarter of developer time (26%) is spent reproducing and ﬁxing failing tests.

On the plus side, 88% of respondents said their organizations have adopted continuous integration (CI) practices, with more than 50% of businesses reporting they can deploy new code changes and updates at least daily. Over a third (35%) said they can make hourly deployments.

Undo CEO Barry Morris said the report makes it clear organizations need to be able to record software to reduce the amount of time it takes to find bugs. Unfortunately, even then finding a bug is still a labor-intensive process that can involve analyzing millions of lines of code. In the future, software replay systems will be infused with machine learning algorithms to accelerate the bug discovery process, he noted. As software becomes more instrumented, the observability data and metrics that machine learning algorithms require to detect patterns is becoming more accessible.

In the meantime, while CI has been embraced, the continuous delivery (CD) side of the DevOps equation remains a challenge. Each platform software is deployed on is unique, so most organizations find automating software delivery problematic.

Regardless of DevOps approach, it’s clear there’s still lots of room for improvement when it comes to developing software. Even the most advanced practitioners of DevOps are limited by the rate at which software bugs, including security flaws, can be discovered and remediated. Of course, there may come a day when machine learning algorithms automate much of that process, including determining which types of tests to run. For now, however, many organizations could streamline the software debugging process by recording instances of applications when they are built and when they are deployed. After all, each platform that software is deployed on has attributes that inevitably impact how software on the platform runs.

As software continues to become more complex in the years ahead, challenges associated with debugging applications will only continue to grow. In the age of microservices, the issue that software engineers are trying to troubleshoot might not have anything to do with the code they wrote; rather, the issue emanates from a service they invoked via an application programming interface (API). Whatever the source of the problem, it’s apparent to all a lot of time and money is being wasted on debugging software that could be put to far better use.

May 29, 2020