devopsy.kinsta.cloud

Category: Blogs

Survey Surfaces High Reliance on DevOps to Build and Deploy APIs

A survey of 13,586 IT professionals who work with application programming interfaces (APIs) published today by Postman, provider of an API development platform, finds best DevOps practices are now widely employed to both produce and consume APIs.

The primary method through which teams working on APIs collaborate is by sharing URLs (42%). However, publishing API artifacts to GitLab, GitHub, Bitbucket or other repositories came in a close second at 40%, followed by publishing API documentation to an API portal (39%) and instant messaging (28%).

When queried about change management practices specifically, Git repositories topped the list at 63%, followed by versioning APIs (59%), versioning server code (34.6%), versioning client code (27.8%) and semantic versioning (21%).

Respondents said when it comes to deploying APIs, the most widely used option was continuous integration/continuous delivery (CI/CD) pipelines (53%).

A total of 77% of respondents listed their functional area as engineering/development, followed by DevOps/API Ops at 34% and developer relations/evangelism (13%).

Among DevOps practitioners that participated in the survey, Jenkins is the most widely employed platform (41%), followed by AWS DevOps (30.7%) and Azure DevOps (26.1%).

Overall, the survey finds REST is still the dominant architectural style for APIs (93%), followed by webhooks (34%), SOAP (33%) and GraphQL (23%). JSON Schema is the dominant API specification (76%), followed by Swagger 2.0 (44%), OpenAPI 3.0 (28%) and GraphQL (23 %).

When asked to rate the percentage of their organization’s development effort spent working with APIs, 41% said more than half of the effort is spent on APIs.

Kin Lane, chief evangelist for Postman, said the survey also makes it clear that more organizations have adopted an API-first philosophy to application development. Over 60% of respondents rated themselves as 5 or better out of 10 in terms of embracing an API-first philosophy.

Nearly half of respondents said investment of time and resources into APIs will increase over the next 12 months, while another third stated that investments into APIs will stay the same. Eighty-five percent said APIs are playing a significant role in their digital business transformation initiatives, while nearly a third (31%) also noted APIs played a role in their organization’s ability to respond to COVID-19 by, for example, enabling customer communications or remote work options.

APIs appear to have become a lot more stable in recent years, with more than half of respondents reporting APIs do not break, stop working or materially change specification often enough to matter.

When it comes to producing APIs, the No. 1 obstacle by a wide margin is lack of time, while the No. 1 obstacle cited for consuming API is lack of documentation.

The four most important factors individuals consider before integration with an API are reliability (72%), security (71%), performance (71%) and documentation (70%).

Looking to future technologies, respondents appear to be most interested in microservices (49%), Kubernetes (44%) and containers (42%). Given the cloud-native attributes of those technologies, it appears all but certain the number of APIs strewn across the extended enterprise is about to increase exponentially.

September 24, 2020
5 Steps to Avoid Shadow IT Around Low-Code
Shadow IT is far more pervasive than most CIOs think. Cisco Systems estimates that a typical firm has 15 to 22 times more cloud applications than authorized by the IT department.

While shadow IT solves problems in the short-term for a select few, undocumented software ultimately encourages tribal thinking and siloed departments. This can stunt reusability, slow down new developer onboarding and lead to insecure points of failure—all antitheses to a DevOps culture.

Simultaneously, new low-code tools are emerging that promise to reduce the programming burden from developers and enable citizen developers to create business applications on their own. But what happens when advanced users know better? Perhaps they’d rather code a quick function in Python or use another framework they’re more familiar with. If not properly governed, low-code platforms could easily be circumvented and lead to shadow IT.

As many tech leaders can attest to, enforcing companywide standards can be tricky. I recently chatted with Jay Jamison of Quick Base to see what tactics organizations should take to avoid shadow IT when introducing low-code tooling. Here are five general tips CIOs can apply to reap low-code benefits while curbing shadow IT.

1. Earn Developer Trust

Focus on integration and open backends to earn developer respect

Want to win the hearts of developers? Focus on integration. Many engineers don’t care about a shiny user interface and would rather interact with data models programmatically. As I’ve covered before, open API access is the secret sauce for low-code platforms.

Choose a low-code platform that provides high-performant and secure RESTful APIs that use JSON. With open access to data models and low-level mechanics, advanced developers will feel less inclined to subvert the platform. What’s great about HTTP APIs is that they’re agnostic, so developers can interact with them from whatever language or environment they’re comfortable with.

2. Govern and Set Permissions

Establish access management policies with fine-grained permissions

To avoid the emergence of shadow IT, software leaders must apply governance over data models, users and application privileges. These permissions should be fine-grained, to the field level. To do so, managers require an application lifecycle management functionality to edit these permissions. Such a governance layer would grant “tremendous power and control to IT and the business,” said Jamison.

On low-code governance patterns, Jamison notices companies often lean toward one of two extremes:
1. IT-centric: IT is leading the software models and only IT users can make changes.
2. Operation-centric: IT acts as a governance body and building is pushed into business domains.
Selecting which model to use is often based on the organizational culture and individual team member capabilities, noted Jamison.

3. Encourage Collaboration and Knowledge-Sharing

Build an easy path from mock-up to production

Companies adopt low-code tools because they require something “quick and adaptable.” Yet, getting different operating teams at the same table can be difficult. Without high-quality collaboration, low-code could introduce a bottleneck.

“Many teams must come together,” Jamison said. “How will this be productized; how will the workflow be tracked?” Companies truly require an effective collaboration model if they are ever to regularize their data models and assign access roles.

To get everyone on the same page, Jamison recommended starting with a mock-up everyone can use as a reference: “Create an online chart to map out workflows and stitch it together in a collaboration space.” Business strategies can create detailed plans, and then builders can transform this into a prototype.

4. Document and Reuse

Encourage departments to document processes and expose reusable tools

If users are sharing documentation via Excel and email, you know something’s wrong. Undocumented software is unmanageable software. Jamison reinforced the need for a “control layer” to localize documentation and encourage platform-wide governance standards.

“Like buildings in a cityscape, some apps are more significant than others,” Jamison said. CRM applications may be entirely purpose-built for niche settings, while others, such as a team registry application, could be reused across an entire corporation. Increasing tooling reusability, he noted, requires teaching and training of documentation and change management practices.

5. Adopt a Platform-Based View

Standardize on a single platform and embrace a low-code culture

The more teams use low-code, the more they think on low-code terms. As teams become more proficient in low-code, their mindset becomes: “How can low-code solve that problem?”

By adopting a platform-based view, low-code becomes a utility to drive consistent data flows and workflows. To achieve this, it’s probably best to standardize around a single, companywide low-code platform. Otherwise, you risk inviting more unstandardized practices.

For example, Jamison described how a large-scale manufacturer operating hundreds of plants worldwide uses a low-code platform to audit all plants’ quality in real-time. Other traditional non-IT businesses with a global footprint could similarly return a net value from consolidation on a single low-code platform.

Will Low-Code Increase Shadow IT?

It’s difficult to enforce any new tool, and some users will inevitably circumvent standard processes. However, by demonstrating the benefits of shared collaboration and low-code creative potential, leaders could stem shadow IT and the adverse side effects therein.

By talking their language and saving time, low-code tools could win developer faith and encourage platform use. In review, here are some other ways to avoid shadow IT side-stepping around low-code:
- Show some-code: Win developer’s hearts by talking on their terms and opening up the hood.
- Govern: Use a common data model and apply fine-grained security constraints.
- Ignite discussion: Bring groups together for open knowledge forums.
- Unite business and IT: Enable citizen developers to design mockups and builders to execute quickly.
- Use the platform: With more low-code use comes more creative sparks.
Ultimately, low-code won’t replace 100% of your programming needs (especially at software-oriented companies). Low-code should be viewed as a time-saver, not a talent replacer, Jamison said. Low-code enables developers to quickly assemble simple business applications, freeing them to devote more time to core competencies or advanced processes that are far more interesting, such as building machine learning models.

Final Thoughts on Low-Code Transformation

There are many forces driving the current low-code revolution. Major tech hubs lack developer talent and COVID-19 has accelerated digital disruption. In this economy, requirements for business processes are changing “by the hour,” said Jamison. “The awareness around low-code is increasing because the need for automation and more digital solutions is increasing.”

As Jamison noted, many enterprises are undergoing a “dual-track digital transformation.” Companies are investing in expensive, broad-scope cloud technologies while still digitizing many manual processes. To Jamison, low-code is now imperative to meet both needs. Thus, it should be adopted with forethought for all potential use cases (and shadowy non-uses).
September 23, 2020
New ASG Report Looks at COVID-19’s Impact on Software Development

We are all trying to adapt to a new world created by COVID-19. As the coronavirus continues to spread, the economy and businesses everywhere have been severely impacted, and no one really knows what the so-called “new normal” will be. For now, most organizations have gone remote, not knowing when—or even if—employees will be able to return to their offices.

While some organizations have been open to the idea of working remotely, many have also struggled to shift to a remote workforce seemingly overnight. For software developers, however, remote work is nothing new. Many developers have successfully worked remotely before the COVID-19 pandemic even started.

“The Future of Remote Work and Software Development“ research published today by Accelerated Strategies Group dives into the long-term impact of a decentralized workforce on software development and delivery. ASG interviewed 347 industry leaders from around the world to gather and refine data on the impact of COVID-19 on software teams and their organizations’ ability to create software, and the importance of remote work tools and applications during the pandemic.

The ASG report, commissioned by CloudBees, revealed some interesting trends. The survey found 63% of respondents said digital transformation objectives have significantly or somewhat increased in priority. Business automation (61.6%) and the need for investment in creating contactless services (60.1%) have increased somewhat or significantly.

More than half of the respondents said the COVID-19 conditions have increased their focus on DevOps initiatives (51.75%) and increased their progress on cloud migration (52.25%). Also worth noting is the growth in companies’ DevOps and Agile adoption, with 46.24% of respondents stating they are using cross-functional teams, 55.64% practicing daily stand-up meetings and 43.23% automating tasks.

The majority of respondents (59.49%) to the ASG report saw an increase in productivity on software teams since the pandemic started. Forty-two percent of respondents said it has become somewhat easier to complete their work tasks in a timely manner.

However, feedback concerning respondents’ ability to manage unproductive distractions was mixed: 39.7% said managing these distractions was somewhat or much easier, 36.36% said it was somewhat or much more difficult and 23.48% said they had no change in their ability to manage these distractions.

When asked to provide insight into the potential negative impact of their remote workforce on productivity, more than one-third (38.6%) of respondents said there was no change, 27.58% of respondents indicated that teams were slowed down significantly or somewhat and 33.83% said things moved along significantly or somewhat faster.

The majority of remote workers (64.4%) expect to remain working remotely, either permanently or at least three or more days per week, once COVID-19 restrictions are lifted. Only 12.7% said they anticipate returning to an office location full-time.

For the most part, the transition to working from home has benefited software teams. COVID-19 has not only impacted the global economy, but also business priorities and has accelerated business and software strategies. Though the future is uncertain, and no one has a clear picture of what the “new normal” looks like, there is a huge possibility many software and DevOps teams won’t return to the office—at least not full-time.

September 22, 2020
How Logs Can Empower Your CI/CD Delivery Pipeline

Change is the currency of software success. This need to change and grow is the basis for continuous integration and continuous delivery (CI/CD). Each year, new industries embrace this philosophy of constant change, but without the necessary data underpinning their decisions, every deployment is a risk. When you enrich your software delivery with log data coupled with best practice log management, you mitigate this risk and enable rapid, safe change in your organization.

Logs are often the most common source of information about an application’s behavior. They provide context as well as result, opening up far more information than a simple count metric will. Despite the immense insight that they can offer, organizations regularly overlook them. Many leave their logs in files on servers, far away from the CI/CD pipeline that needs them. It doesn’t have to be this way; with some best practices, every part of your business can collect, analyze and utilize the hidden knowledge within your logs.

Add Life to Log Data With JSON

Logging in an unstructured format dramatically increases the complexity of detecting patterns in your logs. By logging your application output in JSON, you open up the possibility of analysis of all of your logs. This gives you a top-down view of your entire system while still maintaining readability. This empowers your CI/CD pipeline by allowing you to query and filter your logs, which then enables you to focus in on the problem and precisely diagnose any unwanted side effects of your latest change.

Create Actionable Alerts

If alerts are defined and channeled correctly, have context and can be interpreted easily, they will be actionable, add context and offer greater value. It is difficult to provide context when a single metric exceeds a threshold, but log lines are more sophisticated. They can include additional information to provide context and allow us to pinpoint the source of the issue fast. This is an essential capability for a company embracing CI/CD.

Prioritize Your Logs

Structured logs often come attached with a severity, which indicates how seriously we should investigate an event. An INFO level log signals business as usual, but an ERROR demands immediate attention. Once your logs have these tags, you can make intelligent, automated decisions that will automatically respond to unwanted changes in your system.

Benchmark Each Version to Understand ‘Normal’

“Normal” is a complicated idea when working in a microservices architecture. It may be safe to ignore a minor slowdown of a single service, but an unfortunate combination of events can lead to disaster. Benchmarking enables us to see these disasters well before our users. Benchmarking is the act of recording what the normal behavior of an application is. After each deployment from your CI/CD pipeline, compare your new behavior with your previous benchmark. If something is out of the ordinary, logs provide the information you need to act decisively. They provide an outstanding baseline signal for your benchmark.

Analyzed Logs Can Level up Your CI/CD Pipeline

Building a CI/CD pipeline is not the most difficult part. Neither is the deployment of new features. In fact, the greatest challenge facing any organization wanting to deliver their changes via a CI/CD pipeline is observability. With a best-practice approach to the preparation, curation and analysis of application and system logs, we can overcome this challenge and confidently change at a pace that propels us to the forefront of our market.

September 22, 2020
DevOps Deeper Dive: IPO Frenzy Heralds Consolidation

A spate of initial public offering (IPOs) has resulted in billions of investment dollars being poured into JFrog, Snowflake and Sumo Logic and is likely to fuel of wave of merger and acquisition activity in the months ahead.

While each of these software companies addresses fundamentally different IT requirements, they are all tapping into a massive amount of liquidity now available, as investors dump the stocks of companies in other sectors that have been adversely impacted by the economic downturn brought on by the COVID-19 pandemic.

In fact, the reason all these IPOs appear to be occurring all at once now is the plans to go public that many software companies had earlier this year were delayed by the onset of the pandemic. The backlog for potential IPOs is already staggering. Currently there are more than 400 private startup companies with valuations in excess of $1 billion, which according to the financial research firm CB Insights officially makes them “unicorns.” Familiar names of software companies on that list include GitLab, Docker Inc., HashiCorp, Snyk, Postman, OutSystems, Digital Ocean, Auth0, Redis Labs, Databricks, Collabra, Cohesity, Tanium, Rubrik, SentinelOne, Illumio, Netskope and Kaseya.

The reasons IT organizations should pay attention to these IPOs are twofold. The first is that it signals the company now has ready access to a stream of investment dollars required to fund ongoing innovation, said Scott Bickley, principal research director at Info-Tech Research Group.

The second, he noted, is that many enterprise IT organizations tend to limit their amount of reliance on a startup company. Venture capitalists that fund these companies often sell them to other companies to recoup their investment as an alternative to launching an IPO. Most enterprise IT organizations don’t want to wake up one morning to discover a key vendor has been acquired by a vendor they would prefer to not do business with, said Bickley.

That issue can be especially problematic when cloud service providers such as Amazon, which has operations that reach well beyond IT, are involved, he added.

In fact, one of the reasons many organizations prefer to rely on IT vendors that are public companies is they have a lot more visibility into the overall viability of the vendor, said Bickley.

“Assessing risk is part of the due diligence process,” he said. “With public companies, there’s a lot more data available.”

Of course, public companies do get acquired. The fact that they are public companies usually makes the effort a comparatively expensive proposition, depending on the stock price and the number of shares outstanding.

However, one of the first things many companies do once they go public is to look to acquire other companies that enable them to expand their overall portfolio, said Mitch Ashley, CEO and managing analyst at Accelerated Strategies Group, a market research firm.

The DevOps sector in particular has seen a wave of acquisitions of late that have been funded in part by IPOs, noted Ashley.

“We’re in a bit of a roll-up period right now,” he said.

Many enterprise IT organizations tend to welcome those acquisitions because it allows them to deal with fewer vendors, Ashley noted, adding support also tends to improve because the larger vendor typically has more resources.

On the downside, he said, large vendors tend to raise prices after acquiring smaller vendors in part to help cover the cost of the acquisition.

Like it or not, however, mergers and acquisitions are now a fact of IT life that organizations just need to accept, Ashley said.

Based on the amount of capital available at the moment, IT leaders should also assume the pace of merger and acquisition activity is about to accelerate, even more as more fuel in the form of capital is added to the fire.

September 18, 2020
Catchpoint Acquires Webpagetest.org Service

Catchpoint this week announced it has acquired Webpagetest.org as part of an effort to expand use of the open source web performance testing service.

Led by Patrick Meenan, the Webpagetest service is already relied on by more than 1 million users per month to instantly run tests of websites and applications from various locations, which are surfaced via a detailed audit that can be shared with developers.

Meenan has been personally supporting Webpagetest as a side project while working for a series of IT organizations and vendors starting with America Online (AOL). Now, with Catchpoint’s acquisition of Webpagetest.org, he will have a staff of contributors as a full-time employee.

Catchpoint CEO Mehdi Daoudi said the company will also incorporate Webpagetest into the tools it provides for monitoring digital experiences. The most appealing aspect of the Webpagetest service is that, rather than focusing on individual backend platforms, it tracks the actual performance that users experience across multiple types of browsers.

Daoudi said that as organizations accelerate the rate at which they are building and deploying digital business applications in the wake of the COVID-19 pandemic, there’s now more focus on web application performance than ever. The experience any user has when accessing these applications now impacts everything from their willingness to complete a transaction to their perception of the corporate brand, he noted.

In fact, a recent survey of 200 CIOs and 200 work-from-home managers conducted in July by ReRez Research on behalf of Catchpoint finds application performance is at the top of the list of concerns (66%), followed by network reliability, work-from-home technologies and security, at 63% each. Application performance is at the top of that list because both online revenue and employee productivity are now increasingly directly impacted by the overall application experience, said Daoudi.

There’s never been a shortage of tools for testing application performance. The issue is how those tools are often employed, he noted. Too many IT organizations are monitoring individual application components rather than the actual user experience.

Organizations that focus on end user experience in the age of digital business will, in time, separate themselves from rivals that merely make applications available online. A zealous appreciation for application performance needs to be ingrained in the culture of the organization for it to ultimately succeed, Daoudi said, noting that every employee, not just the DevOps teams, needs to be aware of the impact application performance has on the business.

That challenge is as IT environments become more complex and distributed, ensuring application performance becomes more challenging than ever. Cloud-native applications based on microservices may be easier to deploy and manage, but the dependencies between all those microservices can conspire in ways that degrade application performance over time. Determining the root cause of intermittent application performance issues can take weeks. The maddening thing is, many of the fixes required to address that issue once it’s discovered can be deployed in minutes.

Performance tools that surface those issues sooner than later are invaluable in a digital age, where time now quite literally is money.

September 18, 2020
Crowdtesting: How to Ensure Quality in Disruptive Times

The practice of crowdtesting has taken on greater importance during the COVID-19 pandemic, for good reason

Entering 2020, digital transformation projects and initiatives dominated tech spending for most companies. But, between a global pandemic and fiscal uncertainty around the world, spending must be shrinking, right?

Think again.

If anything, the pandemic has created a sense of urgency for many companies, accelerating digital transformation in organizations. According to June data from TEKsystems, 56% of digital experience leaders—companies with mature digital transformation strategies—have increased spending on digital transformation. Remarkably, more than one-third of digital experience laggards—those that have historically had limited digital transformation initiatives in place—also increased spending on digital transformation projects.

The rush can be attributed to COVID-19. For businesses, it has changed the future of work. Distributed workforces are nothing new, but very few companies were prepared to deal with a fully distributed model in which all employees are working remotely. The pandemic has changed buyer behaviors, too. In lieu of malls and brick-and-mortar stores, people have turned to e-commerce sites for the bulk of their shopping. Legacy companies, such as Pepsi, for example, are building direct-to-consumer e-commerce sites in response to the evolving market dynamics.

Digital is now the primary means of communication for businesses. Digital quality is no longer a niche issue for the application and DevOps teams; it is a strategic business imperative in the C-Suite.

Apps Everywhere, At All Times

Apps need to be tested by real users, on every device, network and operating system, across the globe and at all times. The typical workday used to be 9 to 5, but that’s no longer the case. With most workforces now remote, the days are longer and the diversions greater (yes, I’m looking at you, juggling Zoom calls and your daughter’s remote learning plan at school). Customers, too, have higher demands for the technology they use. There can be no downtime; downtime means customers will look for alternatives, becoming ex-customers and resulting in lost revenues.

In today’s extremely fragmented device landscape, QA teams face increasing difficulties in keeping up with the myriad device/OS combinations they need to test on. For obvious reasons, purchasing and maintaining a nearly endless pool of devices is not an option—not only is it too expensive but it’s also time-consuming and harms the team’s productivity.

Beyond logistics, QA teams must collect data from real-world testing to make informed decisions. No matter how many test cycles are run, QA teams can’t uncover all unexpected bugs and edge cases unless they include in-the-wild testing in their strategy.

To test anything related to hardware in an efficient manner, leveraging a community of remote testers is the best way to obtain reliable results without burdening internal QA resources, generating disproportionate costs or missing out on important use cases.

Crowdtesting Replaces Offshoring as the De Facto Way to Test Software

With a market greater than $50 billion in India alone, there is no denying the offshoring model worked. But, the pandemic exposed pre-existing cracks in the model, causing many businesses and entire industries to regroup, re-evaluate and find new ways to accomplish tasks that just a few months prior would have been routine.

When the pandemic hit, the supply chain broke. Testing and product development facilities built on product teams working collaboratively in close proximity struggled to keep up with the most basic needs of their customers. Outsourcing companies and their employees lacked the infrastructure to work remotely, negatively impacting the business continuity of their clients.

This new reality leaves companies that are used to the offshoring model for testing and QA with one big question: what is the best way to test software and digital experiences in a post-COVID-19 world?

While crowdtesting has been a factor in application and digital testing for more than a decade, it has taken on increased importance as internal QA teams can no longer maintain business continuity while working remotely and the offshoring model has faltered amid the pandemic.

The reason the crowdtesting model remains a reliable answer even during COVID-19 is that it is always done remotely, via a community of vetted testers who already work in remote environments. This setup allows testing teams to maintain business continuity, while often improving test coverage across devices, operating systems, geographies, demographics and more.

As an example, let’s look at how the Israeli government used crowdtesting to bring its tracing app to the country. The Israeli Ministry of Health developed the app using cellular GPS data to provide alerts on areas with high concentrations of COVID-19 infections. The app is designed to prevent COVID-19 spread by notifying people who were in the vicinity of verified coronavirus carriers. Similarly, healthcare provider K Health has used crowdtesting to help it launch a new COVID-19 diagnostic feature. K Health was already using crowdtesting services prior to the onset of the pandemic to ensure its mobile app functioned properly across all relevant device/OS combinations and timezones but found the scalability and reliability of the model meshed perfectly with the demands on healthcare providers in the new normal. When K Health offered free services to all Louisiana residents, crowdtesters were there to scale testing at a moment’s notice and provide 24/7 test coverage for the K Health app.

It was urgent to get the app out quickly, but it also needed to work properly. Did push notifications go through promptly and to everyone? Would the app work across devices, and in different configurations? Since launching the app in March, the Ministry of Health has been able to leverage continuous crowdtesting to identify dozens of bugs that could have prevented Israelis from using the app or trusting its results.

For example, the testers discovered that the app crashed when it was used in dark mode, and certain situations led to the app showing that it detected data even when data and location services were off. Since this second error was directly related to the main functionality of the app, it represented a very critical bug for the Ministry of Health to fix.

Crowdtesting uses real devices and real people in their daily environments, which accounts for all these variables that are not seen in traditional labs and offshoring facilities. That makes crowdtesting a valuable testing approach even after the COVID-19 pandemic and resulting lockdown ends.

September 16, 2020
The Rise of the Cloud Economist: The Other CFO
The new Cloud Economist job title wrangles runaway cloud costs

Have you ever forgotten about a subscription? Maybe it was that Amazon Fresh upgrade you never used, charging you monthly payments. Or, your phone carrier slyly tacked on a feature and it went unnoticed for years. All these charges may seem insignificant, but they mount up over time.

The same is true for cloud IT tools. Many Ops engineers know all too well the surprising bills that accompany cloud infrastructure. While cloud prices may start low, they often rise unexpectedly with hidden server costs, data replication charges, escalating API fees and other fun add-on expenses.

Conversely, remaining at a low service tier could cause latency and a loss of revenue. Drowning in a sea of hybrid trends and feeling pressure to deploy at lightning speed, it can be difficult to keep track of which cloud models actually advance business objectives.

Some believe an entirely new role is needed to rectify the gap between cloud and business repercussions—a Cloud Financial Officer, or the Other CFO (OCFO). An OCFO could act as a Cloud Economist, a tech-savvy ambassador who both understands the financial implications of a cloud strategy and optimizes service tiers to their necessary levels.

I recently chatted with Leon Adato, Head Geek at SolarWinds, on how businesses could better drive cloud cost optimizations. Like a determined accountant sent in to save a struggling city (h/t to Ben Wyatt in Parks and Recreation), the OCFO may be the star companies need to slash the budget and balance the cloud checkbook.

Why Do We Need a Cloud Economist?

According to the “Flexera 2020 State of the Cloud Report,” 74% of companies report their annual cloud spending exceeds $1.2 million each year. At 20% of companies, it’s more than $12 million. As cloud spending skyrockets, balancing these expenses becomes business-critical.

Yet, forecasting cloud costs is tricky. Unexpected complexities often arise with the cloud, skewing initial cost projections. Maintaining cloud oversight becomes especially complex as COVID–19 accelerates more cloud adoption across the board. Budgets are shrinking, and some companies are undertaking extreme measures to remain lean.

Simultaneously, the ones that advocate for new technology don’t always consider cost implications. “Three clicks and you’re done means a DevOps engineer in the driver’s seat may not be aware of the cost considerations,” said Adato. In recent years, SaaS companies have stressed easy onboarding and developer experience as a top priority. The staying power of developers to adopt new tools is arguably at an all-time high. But while it’s important to improve developer workflows, developers don’t necessarily have business acumen. “Finance is a lexicon of itself,” noted Adato.

Thus, some view a Cloud Economist role as necessary for translation across divisions. “IT divisions shouldn’t be encountering sticker shock when they view their cloud infrastructure bills,” he said. “We need IT folks to be able to pay attention to cloud costs in a meaningful way.”

What Does a Cloud Economist Do?

A Cloud Economist is an expert that specializes in cloud-spending and cloud cost optimizations. Such a role could help teams realize the efficiency benefits of cloud-native architecture while keeping costs lean.

This job title truly requires “hands for code and a head for money,” described Adato. He laid out three simple goals of a Cloud Economist:
1. Increase revenue.
2. Decrease cost.
3. Remove risk.
A Cloud Economist understands the technical underpinnings behind cloud tools and how they support each other within a cloud-native ecosystem. They also appreciate maintenance realities, asking tough questions such as: Will it take 200 code pushes per day to support this microservices integration pipeline? How will server costs change as bandwidth scales? Which plan is most optimized for our needs?

Specific Cloud Economist Duties

A Cloud Economist must manage runaway costs associated with cloud adoption. They see the bigger picture and understand the repercussions behind cloud technologies. In terms of specific duties, the role may wear many hats.

For example, a Cloud Economist may encourage a CSP migration, such as from AWS to GCP. Or, they may foresee the implications of future feature introduction, such as limitations across geographic zones, and advise IT teams accordingly.

An avid Cloud Economist should also interface with monitoring and testing teams. “API testing before launch can mean the difference between a $25 cloud database bill and a $30k bill,” wrote Rapid API’s Chris McCallister in a separate blog post.

Other Cloud Economist duties could include:
- Logging and forecasting rising cloud costs.
- Becoming Terms of Service (ToS) experts.
- Avoiding costly data replications.
- Upgrading servers.
- Discontinuing unused instances and features.
- Ensuring elasticity of applications.
- Database scalability.
- Increasing CPU power or data loads.
- Optimizing subscription plans.
- Considering implications behind feature updates.
- Limiting the intake of new cloud tooling.
- Acting as an ambassador between a company and CSPs.
Traditional IT, or information services, as it was originally called, “has been able to avoid the need to integrate with business,” said Adato. That will likely change drastically as cloud costs escalate and thousands of very small charges amount to “death from a thousand cuts.” It appears that any business size, from startup to enterprise, could benefit from more cloud expense forethought.

Inherently, DevOps is a cultural movement, not a technical movement, Adato noted. Assigning a Cloud Economist encourages specialization and de-siloing between disciplines, freeing up DevOps engineers to work on projects that matter to them. The role of a Cloud Economist “offloads responsibility that has always been there that may have been ignored,” he said.

Avoid Ice Town With a Cloud Accountant

Software architects are often sold on the idea that cloud infrastructure is a cheaper alternative to on-premises. “That statement is false,” Adato said. As an organization cloud evolves out of freemium tiers, where it began cost-wise is often far from where it ends up.

To navigate the opaque cloud, “we need someone that has fluency and familiarity with tech, understands the platform, understands how code is updated and maintained, and can right-size the cloud platform,” said Adato. Cloud-native certainly has its benefits, but if runaway cloud costs aren’t wrangled, they could equate to revenue loss.

As author Bob Lewis wrote: “There’s no such thing as an IT project.” We don’t do technology for the sake of doing technology. Rather, we do it in service to the business. A Cloud Economist could help ensure technology answers the business call, not vice versa.

As of now, the OCFO is mostly a theoretical title; it could materialize as an internal team member or contract hire. Regardless, such a role could advocate for the business and keep operational overhead as lean as possible.

In general, it may require a dash of hard realism to learn how to avoid plunging the books into a debt-ridden Ice Town.
September 16, 2020
Shifting Left With DevSecOps: ESG Report Exposes Difficulties
A recent report asks the tough questions about DevSecOps adoption, and the results are surprising

In a world of increasing development velocity, companies are placing more responsibility on developers to enact quick deployments. Naturally, security is “shifting left” as well.

Theoretically, DevSecOps sounds great, but it opens up many questions. Who exactly is responsible for security knowledge? Are cybersecurity training measures effective? Which tools excel at decreasing risk while maintaining release fluidity? Are AppSec tools aiding or adding unneeded friction?

A report conducted by Enterprise Strategy Group (ESG) sought to answer these pressing questions and expose trends around application development security. The study surveyed 378 IT, cybersecurity and application development professionals at organizations in North America.

I recently met with Chris Eng, chief research officer at Veracode, and Dave Gruber, senior cybersecurity analyst at ESG, to dig deeper into the report’s findings.

Security vs. Development

There is still friction when attempting DevSecOps in practice. Eng recognizes a disconnect between security and development teams.

“Most security teams lack an understanding of modern application development practices,” said Eng. Security may not realize how regularly code is pushed or understand the intricacies of tooling within a CI/CD pipeline. On the flip side, developers may not be well-versed in common container exploits and vulnerabilities.

Organizations are pretty split when it comes to application security testing ownership. Thirty-one percent of organizations dedicate security analysts to manage application security. The development manager and security team jointly own responsibility at 29% of companies. Developer managers follow at 26%, and individual developers at 13%.

To Eng, these figures reflect an evolution from traditional security testing to the way things are changing with faster development cycles. “Go back 10-plus years, and we had a centralized, dedicated security team responsible for managing AppSec,” described Eng.

In modern times, security is shared or even furthered where developers themselves run tools by hand or automate security within CI/CD processes. Large companies, Eng noted, now often utilize a mix of those practices.

Paradox: Secure Yet Vulnerable

Seemingly, the most eye-opening part of the report was a paradox. The report found that most teams give their application security a high rating (36% gave their application security programs a 9 or 10, with a 7.92 mean). Yet, the majority say they regularly push vulnerable code (48% of respondents knowingly push vulnerable code regularly).

Though it’s counterintuitive, organizations may knowingly push out vulnerable code for many reasons. “You’ve promised a release, customers are waiting, ” said Eng. “Revenue is waiting. What are the implications of pushing that release?” Teams may not have time to fix vulnerable code, or it may simply not be worth it.

Publishing vulnerable code may also be a symptom of placing testing too late in the release process. “That’s why there is this trend to shift left,” added Gruber.

Gauging DevSecOps Success

The above findings underscore a reality of IT security: To keep pace, software providers must accept a level of risk. In fact, business success may depend on it.

“The effectiveness of the programs isn’t necessarily measured with how many times you’ve shipped your code with no vulnerabilities,” said Gruber. “Success is that we understand the risk, we have mitigated the risk at the level we believe is right for the organization, and have the right level of control over the process.”

Companies must weigh the cost of improper DevSecOps and mitigate imminent risk to maintain release agility.

Application Security Tools: Finding Balance

At modern companies with quick-release cadences, “CI integrative security tools are way more prevalent and baked into the process,” noted Eng. Yet, 26% of respondents indicate security tooling friction and ineffective tooling usage.

Whereas InfoSec teams are usually the ones dictating policies, developers have much influence in the tooling selection. Thus, tools must appeal to the developer experience, fitting into their established workflows.

“We have to build a habit around it,” said Eng. Frictionless tools that continually scan within an IDE, for example, can enable developers to fix an issue on the spot and foster future behaviors.

At the same time, you must set standards and avoid governance issues. This requires control and observability. “The balance here is that you want developers to be able to integrate something tightly into their toolchain,” said Eng. “But InfoSec needs a bird’s eye view on the organization.”

Developers Lack Knowledge to Mitigate Issues

Though security tools may reveal vulnerabilities, they don’t always correlate with remediation. The ESG report found 29% of developers lack the knowledge to mitigate issues identified by security tools.

There is a clear gap in the market when it comes to developer education around security. This could stem from computer science schooling, in which cybersecurity is rarely taught alongside programming languages.

On the job, hands-on developer security training is also spotty. Thirty-five percent say that less than half of their development teams are participating in formal training. Only 15% of organizations have developers actively involved in security training.

Just giving developers a laundry list of vulnerabilities doesn’t help mitigation. For Eng, what organizations require is habitual, recurring training with hands-on tools. “Try to simulate or provide a real-world environment for learning, identifying and patching these issues,” recommended Eng. Integrating tooling within the daily developer workflow can help with knowledge retention, because “if you don’t use it, you lose it.”

Other best practices will only arise with a security habit. For example, most development environments ship with open defaults. “You need security training and tools to understand to restrict that appropriately,” noted Eng.

That said, a wide variety of security tooling is already advancing this process. For example, 56% of organizations use API security vulnerability (ASV) scanning to identify and mitigate risk associated with API usage, helping secure applications as developers code dependencies.

Final DevSecOps Thoughts

The ESG study found 43% of participants believe DevOps integration is most important to improving AppSec programs. Yet, siloed roles and a lack of security education mean DevSecOps must still mature quite a bit to meet the security-shift-left vision.

Other eye-opening findings included:
- More than half of respondents (54%) use automated controls to identify and quarantine vulnerable images in their image repository.
- The proliferation of AppSec tools on the market is a significant problem for 30% of companies.
- AppSec spending is increasing, yet only 30% will protect more than three-quarters of their codebase 12 months from now.
For more findings, check out the ESG Survey Report: Modern Application Development Security.
September 9, 2020
DevOps: Driving Innovation With Old Habits

DevOps can help drive cost-efficiency, greater productivity and innovation

When software development and IT operations merge and work in a state of harmony, they create a flow that we call continuous product delivery. Ideal for project managers, this flow is the elixir to innovation and enables the agility needed to respond to changing market requirements. It also creates currents of speed, proactiveness and integration within enterprises. However, asynchronous code updates, inadequate IT infrastructure, siloed functioning and inefficient processes can hinder this flow.

In 2009, Patrick Debois and Andrew Clay Shafer devised a framework to break such barriers and coined it “DevOps.” Over the last decade, DevOps has garnered many followers and fans, including myself. Product managers worldwide rely on DevOps to enhance performance, accelerate time to market and innovate with agility. DevOps is more critical today than ever before—when nature mandates a new order of remote working, product managers must rely on DevOps to catalyze innovation and sustain continuous product deployments and updates.

Yet, not all enterprises that have adopted DevOps have unearthed innovation like that of Dyson, which created a new type of ventilator in a mere 10 days to help the UK government scale COVID-19 treatments. Nor have they been able to match the frontrunners of innovation such as Google and Amazon that can deploy code thousands of times per day, compared to low performers that deploy code only about four times per day. The obvious question on everyone’s mind is, “What are these heroic innovators doing differently with DevOps?”

Driving Success Through DevOps

While business executives are aware that implementing DevOps drives business innovation, most are not aware of the old habits that remain, causing unnecessary hiccups within the application cycle and business transformation. As the way forward, organizations must tweak their approach to DevOps to be able to reap more significant benefits of business innovation. They must:

Ensure speed does not outpace stability

Today’s digital-first world has swept through every enterprise. However, to keep up with the rapid dynamics of the digital tide, enterprises need to weed out monolithic architectures and manual processes to turbocharge software development and deployment. Automating the deployment process with the help of microservices and cloud can help enterprises gain speed in software deployments and swim the tide of customer experiences. Speaking from experience, this is a privilege that has allowed me to witness how DevOps practices have enabled enterprises to crunch deployment times from weeks and days to just a few minutes. Needless to say, innovation depends on speed: The faster an enterprise deploys a novel solution, the higher its chance to claim the prize of innovation.

A word of caution here: Many enterprises that have neglected quality and stability while accelerating speed drown in a vortex of defects often. When pushing out a product or an application 30 times faster than the traditional model, enterprises must consider its stability, too. Cloud infrastructure plays a vital role in helping enterprises achieve both stability and speed. On-demand self-service, networks that cater to multiple devices, high scalability and analytics are some of the primary advantages that enable enterprises to deploy faster without impacting the core.

Losing stability for the sake of speed can cause organizations to regress instead of leaping forward. In recent times, British Airways has been in the news for repeated issues in its IT infrastructure’s stability. It has had to cancel several flights and inconvenience passengers because software upgrades and incidents have caused its infrastructure to fail. Enterprises must consider these four metrics—lead time for changes, deployment frequency, time to restore service and change failure rate—to receive the value of their DevOps investment.

Make security equal to agility

The pandemic-driven remote working has seen the video communication solutions provider Zoom soar to 300 million active monthly users last month. The brand showed great agility in meeting the surge in demand but became vulnerable to cyberattacks—500,000 Zoom passwords were available for sale in the dark web’s crime forums last month. This is an excellent example of how security is usually an afterthought when compared to agility; not prioritizing security in the DevOps cycle can place teams and their organizations at significant risk, creating vulnerabilities and exposing organizations to bad actors.

In my experience, putting security checks in place early in the DevOps process can help enterprises shift left more effectively. As a first step, security and development teams need to communicate better through the continuous delivery flow. Security teams need to play more proactive roles in foreseeing risks in innovative solutions and routine software updates. Leaving security for the last will only force organizations to be reactive versus proactive.

Build collaboration, not silos

Gartner predicts that through 2022, 75% of DevOps initiatives will fail to meet expectations due to issues around organizational learning and change. DevOps is a process that provides tools but it is also a philosophy that promotes collaboration among teams in software development. It catalyzes an ecosystem that encourages high collaboration and information-sharing. In a DevOps culture, failure is not shot down but leads to inquiry that drives solutions. Seamless information flow is the critical outcome of a DevOps-driven organization. Proper information flow is timely, answers questions and is understandable while needing the support of tools and technologies to drive success.

The General Motors Freemont Assembly line has been a great case study of how poor workforce culture can cause an enterprise to shut down. As one of the largest automobile factories in the United States, the manufacturer had to shut shop in 1982 due to rising losses attributed to an unruly workforce that was protected by a strong union. Within the next two years, the same factory was re-opened in collaboration with Toyota Motors, and it was called NUMMI.

NUMMI hired up to 85% of the old workforce of GM’s Freemont assembly. The difference was the Toyota culture–one that focused on learning and improvement and set a framework of how leadership interacted with and inspired teams. This change was fundamental in driving success for NUMMI, which reduced the time to manufacture a car to 20.8 hours in 1986—a drastic improvement over 43.1 hours in 1978.

At a time when we are witnessing unprecedented disruptions to supply chains, manufacturing and employment, enterprises need to deep dive into functioning to explore new opportunities for cost-efficiency, greater productivity and innovation. DevOps can help achieve these and build the foundations for continuous innovation. Investments in automation, cloud and culture will help enterprises realize the continuous flow of product delivery and innovation. The wave of innovation in the new normal will be propelled by the forces of time-tested practices and new next-generation approaches to enable enterprises to shift-left.

September 1, 2020