devopsy.kinsta.cloud

Category: Features

Google Analytics Adapts to GDPR, But Questions Remain

The General Data Protection Regulation (GDPR) will have an effect on how Google Analytics collects and processes data and how DevOps uses the tool—at least, that is what is known for sure at this time. These changes also should almost certainly impact DevOps for any organization that relies on the tool to collect and analyze user data. It is also known DevOps teams will have to reconfigure their Google Analytics implementations, even if an organization has no users based in the European Union (EU).

However, while Google has recently offered some guidance about modifications to Google Analytics that impact DevOps, the company has yet to clarify some key points about changes ahead of when the new mandate takes effect May 25. In the immediate, here are some changes Google has communicated so far and actions DevOps will need to take ahead of the regulation’s deadline.

Immediate Action Required

The GDPR’s “right to be forgotten” provision offers users options to demand the deletion of personal data third parties store and share. To meet the terms of this provision, Google Analytics will allow organizations to manage how long their users’ data remains on Google’s servers. In this way, Google Analytics will automatically delete user and event data older than the date specified. Organizations can specify that the data remains on Google Analytic servers for 14 months, 26 months, 38 months and so on, for a maximum 50 months. It is also possible to opt for data to not expire automatically. Google Analytics says it will not delete information based on aggregate user data.

Google Analytics also will allow organizations to delete data for individual users in an EU country that can be used to identify them under the GDPR’s “right be forgotten” clause. This data will include common identifiers that Google Analytics stores—including standard Google Analytics first-party cookies, user ID or App Instance ID.

However, Google Analytics has yet to specify details about how organizations will be able to delete user data. While the process will be animated, Google says more information “will be available shortly,” presumably ahead of the GDPR deadline.

Google says it is also drafting a new “EU User Consent Policy” ahead of GDPR. This policy that customers will have to agree covers organizations’ “responsibilities for making disclosures to, and obtaining consent from, end users of your sites and apps in the EU.”

Another key modification Google has yet to reveal is the interfaces Google Analytics will offer that will allow organizations to make changes to how their users’ data is stored. Whether DevOps will be able to automate these processes or not will depend largely on whether Google Analytics will offer a GUI, an API or both.

“If Google Analytics offers a GUI with no API, there is not much a DevOps team can do to automate it outside the existing functionality, since you need a person to do that. But if it’s an API, DevOps teams can weave the controls into their existing frameworks,” said John L. Myers, an analyst for Enterprise Management Associates (EMA). “If both are available, then you’ve got the best of both worlds.”

Additionally, an API would allow the retention and deletion tools to work within existing DevOps frameworks and to become automated as part of business processes, Myers noted.

The DevOps Challenge

DevOps will obviously need to be prepare for GDPR in other ways outside of the scope of changes to Google Analytics or face fines up to 4 percent of their annual revenues or €20 million ($25 million), whichever is greater. Indeed, GDPR applies to all organizations worldwide that collect data from users in EU countries. In fact, it is often incorrectly reported the new privacy mandate applies to just EU citizens, while instead, the regulation concerns data collected from any user who is physically located in an EU country, regardless of nationality. Regardless, it is also obviously impossible to determine whether user data protected by GDPR including cookies, names, email addresses, social media posts and other information belongs to a user with an EU IP address who might just be visiting the region or is a full-fledged EU national.

Given the far-reaching provision of GDPR that places strict controls on how third parties store and share data and protect their online privacy, GDPR likely will fall under an urgent DevOps project category, requiring the input of the change advisory board (CAB), as well an organization’s IT operations, QA, InfoSec and development teams. The degree to which GDPR will change code deployments and processes will depend on the organization’s business model, of course. GDPR compliance should also require review of qualified legal counsel.

Meanwhile, Google Analytics says its tools can help organizations to comply with GDPR for data collection policies. These features include cookie settings, privacy controls, data sharing settings, data deletion on account termination and IP anonymization.

However, as mentioned above, Google still need to provide DevOps with more clarification about certain changes to Google Analytics affecting organization ahead of the May 25 deadline. Many in the DevOps community are also very likely waiting to learn about other ways DevOps can use Google Analytics for GDPR compliance.

— B. Cameron Gain

April 19, 2018
DevOps and Low-Code/No-Code Tools
Helping out a friend, I had reason to revisit low-code/no-code solutions and poke around a bit. The industry has been relatively quiet on the topic, but changes in systems and marketing made me feel the need to offer an update on the relation of current solutions to DevOps.

Low-code/no-code solutions have been a panacea in the eyes of some both in and outside the industry for decades. 4GLs were the earliest widespread attempt to make an environment that eliminated the need for developers, and while some years are quieter than others, the movement has not slackened.

There are even some things we should consider adjuncts to low-code/no-code environments that normally are not considered. Traditionally the drive for low-code/no-code solutions is to move faster than IT normally turns out solutions. This would force us to consider the spreadsheet in accounting with 50,000 lines of special functions as a companion solution—faster than waiting for IT and more agile.

The thing is that low-code/no-code environments have increasingly realized that IT is going to be their best customer, adjusting their offerings to focus more in that direction. At the same time, agile development and DevOps have made low-code/no-code solutions more palatable to an IT shop that is looking to continue the gains brought about by these technologies.

The future appears to trend toward low-code/no-code solutions being an adjunct of agile and DevOps. I don’t say this lightly, but the signs do seem to point that way. Agile shops, having become more responsive to business needs, will hit a limit on what they can deliver, while the appetite for faster and more responsive systems delivery will not abate. Low-code environments that offer the following will become a very real option to extending the gains of agile and DevOps:
- A clear and understandable way to code when needed in the language of choice for the IT environment.
- A high level of feedback on what is wrong, possibly including suggestions on how to fix it.
- Transparency in what is actually happening behind the scenes with the “no-code” part of the system.
The idea is simple. By handling tasks that we, as developers, do over and over and over, we are freed to spend more time doing the truly innovative and environment specific work. This would seem to offer the next step in the growth of agile.

Those tools that offer options such as automated deployment will step into ARA land also, making the Ops side of DevOps happy—in shops that haven’t invested a fortune in setting up ARA already, anyway. If low-code can offer the ability to say, “Here is a problem,” then the developer can submit a fix, and it is submitted to test, and if tests pass is sent to production (and some of the tools do), then the only questions are, “Does the code generated by the low-code part perform? and, Is it easy to integrate existing libraries, etc.?” If the answers are yes, then we also have an add-on for DevOps that delivers value.

We live in interesting times. It will be very cool to see this all play out, and interesting to see where it heads after this. At least one vendor is betting on artificial intelligence (AI) and machine learning (ML) to offer increased productivity.

Of course, it isn’t a panacea. I read where a vendor said, “We automatically update your queries when a table is modified,” which spawns another dozen questions for me, starting with “Really? What do you do about code that was counting on that field much later in processing? Do you check that screens are displaying defaults, or that variables used to make decisions aren’t always the default value? Do you flag this stuff so devs can ‘fix’ what automated query changes ‘broke’?” I’ll be digging in some more, because now they’ve got my attention. So you’ll probably be hearing more from me about this topic.

Meanwhile, let the future worry about the future, and keep kicking it today. Just stay aware of what else is going on out there that might be handy in your toolbox.

— Don Macvittie
April 9, 2018
Culture Becoming Bigger Issue as DevOps Goes Mainstream

Culture and resistance to DevOps processes appears to be growing at many organizations where DevOps projects have started going mainstream and involving more enterprise groups.

Cloud automation vendor Quali recently surveyed 1,300 DevOps and IT professionals on some of the key barriers to DevOps adoption and found more respondents identifying culture as a top issue compared to last year. Twenty-two percent of the professionals in Quali’s most recent survey cited stagnant organizational culture as the biggest impediment to DevOps—a substantial 8 percent increase over the 14 percent who said the same thing last year.

Quali interpreted the trend as likely the result of more groups getting pulled into broadening DevOps projects at many organizations. Innovation cultures and processes are being impacted in a broader manner as this is has happened. The trend shows why managing culture has to become a top-down responsibility, the firm said.

“As technologies, processes and tools go mainstream, they touch a larger number of people who may need to change the things they do or how they do it,” said Shashi Kiran, chief marketing officer at Quali. “Eventually, change is never easy. It is actually a good thing that culture is there at the top, which means change is touching a lot of people across the organization, which eventually should be net positive.”

An October 2017 report from analyst firm Forrester Research also suggests that more organizations are at least beginning to recognize the need for a cultural shift. Its research shows organizations are making culture a core focus when implementing DevOps projects. The focus is on shifting the mindset of teams to a culture of experimentation, where failure is used as an opportunity to learn and evolve processes instead of as a reason to add more controls and checks. “These mindset transitions are imperative for successful DevOps, and the market trends are encouraging,” Forrester noted.

Other Barriers to DevOps Adoption

While culture is a top barrier to DevOps adoption, it is not the only one. Quali’s survey showed application complexity and legacy processes to be major factors as well, with 20 percent and 21 percent respectively identifying the two issues as among their top barriers to DevOps adoption.

The survey also found that DevOps teams at many businesses are still heavily dependent on their IT organization for infrastructure and continue to use a ticketing system to request and receive required resources.

Nearly 50 percent, for instance said they had to wait for up to one month to get access to infrastructure resources that they had requested from IT, while the wait was even longer at about 24 percent of the organizations in the Quali survey. Only 27 percent—or barely more than 1 in 4—of the DevOps teams represented in the survey get access to requested infrastructure in less than one day.

The numbers are important because slow access to required infrastructure often could have a big impact on the speed and productivity of DevOps teams, Kiran said.

“DevOps teams don’t want to be bogged down by infrastructure. To a large extent, it needs to be invisible to them,” Kiran said.

However, the fact is that environments need to be set up for DevOps and many times there is an infrastructure dependency on the back end. “This may be someone else’s responsibility, but the DevOps team ends up waiting for infrastructure to be set up and that’s frustrating for them,” he noted.

Predictability is important in DevOps environment as is planning and being able to accommodate dynamic change environments, Kiran said. “Self-service is a preferred option where people are involved in environment buildouts, modeling different components and in testing and deploying them,” he said. One-third of the respondents, for instance, said they were strongly inclined to adopt platforms that enabled self-service automation.

The Quali survey found that the degree to which an organization automates processes has a big impact on its ability to enable a self-service capability. More than two-thirds of the organizations surveyed had some form of automation. Of that, 33 percent had implemented a self-service capability for R&D, development and testing initiatives. Generally, organizations with automation and self-service reported improvements in production and agility compared to organizations that had fewer automation and self-service capabilities.

So, where in the development life cycle is most of the automation happening in DevOps environments? The answer isn’t clear because automation is happening in a somewhat siloed manner, Kiran said. “Continuous integration is a starting point and larger organizations adopt continuous deployment as well, based on their internal methodology.” But, for the moment at least, there is no platform that can navigate creation of DevOps environments, expose them for testing and push them into production—all as a continuous life cycle—with full-on visibility, Kiran said.

Kiran’s take on automation is similar to that reported by Forrester last year, which also found silos of automation continuing to be a barrier to DevOps success at many organizations. Forrester has identified “purposeful end-to-end automation” as foundational to DevOps success, especially at scale.

According to the analyst firm, organizations that have implemented complete automation of the CI/CD pipeline have been able to enable rapid deployment. However, despite progress, plenty of opportunity still exists for automation and for deeper collaboration between infrastructure/operations teams and development teams, Forrester has noted.

Quali also created an infographic illustrating some of the finer points of the survey, which can be accessed here.

— Jai Vijayan

April 5, 2018
DevOps: It’s an Odyssey, not a Journey

It has become somewhat of a standard practice to tell people, “DevOps is a journey, not a destination,” which attempts to convey the fact that there is no end point that is set in stone. But a journey is a well-planned, point-to-point trip, while DevOps transformation is a messy, figure-out-the-next-steps-as-you-go trip.

So I propose we quit calling it a journey and start calling it an odyssey. This summer, when my mother takes her 10,000^th trip back to Germany, it will be a journey. She’ll start in Cincinnati, Ohio, and travel through a couple of states before finally making her way to Hamburg, Germany. That’s a journey. She knows where she is going, and the discrete steps along the way.

Her original flight from Germany? That was an odyssey. The trip went Berlin->Naples->Shanghai->Hawaii->California->New Jersey. Of all of that trip, only Berlin->Naples was a choice. Or, rather, the decision to go to Shanghai was a choice that required Berlin to Naples—very much in line with your DevOps choices making decisions for you along the way. And just like her family, you have no idea what your choices will be in a few iterations. They knew they had to go to Shanghai because that was the only port that would allow them in without visas, and they couldn’t get visas. After that? They knew they wanted to get to the United States or Australia (one assumes anywhere far from the insanity that was engulfing Europe and Asia during World War II), but how to get there was a total unknown. “First we start, later we finish,” was her father’s description of the goal.

And that’s pretty much what DevOps is: First you start, and in the fullness of time you will discover things you hadn’t even considered when starting that just make sense and improve your overall position. There will be fits and starts, wrong turns and massive improvements. It is an odyssey, and one that is designed—just like my mothers’ family’s flight—to make things better in the long run.

For example, the choice of Ansible or Puppet will make other choices easier. Giving teams more autonomy and increasing communications will make other choices more obvious. It will evolve into more than the first iteration’s goals would suggest, and that’s a good thing if increased agility and reduced manual intervention are the results.

So keep on that odyssey, but don’t expect a simple end. More likely, the identification and acquisition cost of future improvements will cause more than the benefit they introduce, and you’ll find yourself in a better place, ready to breathe a bit and let the market find new ways to improve all things DevOps.

Which is cool, because that should be your final goal: “We have improved so much that any improvements we could make right now would offer minimal incremental improvement for a significant cost.” At that point, your journey isn’t done, but you’re able to sit and take a rest, consolidate gains and assess future possibilities.

And until you hit that point, be excited that every iteration of DevOps is improving responsiveness and taking you closer to your goal.

Shameless plug: Read about my mothers’ odyssey here.

— Don Macvittie

April 3, 2018
Why Your Software Supply Chain Might Be Your Achilles Heel

Historically, cybercriminals are as lazy as they are innovative. They can come up with clever exploits and attack vectors, but they still generally focus on the low-hanging fruit in the most target-rich environments. Recently, attackers seem to have shifted focus away from directly targeting companies with strong security or a wealth of resources, instead going after weak links in the software supply chain.

No matter how many resources or how much effort you dedicate to securing your networks and protecting your data, it can all be for nothing if you trust a third-party supplier or vendor that is vulnerable. It is also true of your software supply chain, because the process of developing and deploying software offers opportunities that attackers can exploit.

The Case of Xcode Ghost

iOS devices are considered to be exceptionally secure. iOS is not impervious to attack or exploit, but it is more secure than the competing Android platform. Apps for iOS are also considered to be more secure, because Apple’s App Store is a “walled garden” with a stringent review process for an app to be approved.

A few years ago, however, it was discovered that more than 4,000 apps in the Apple App Store contained malicious code. Attackers had figured out how to leverage a weakness in the software supply chain to get past the gatekeepers of the walled garden.

How did they do it? iOS apps have to be written in Xcode. The Xcode software is provided for free, but the Apple servers are often slow—especially when trying to download from China, so developers often just search the web to get Xcode from a third-party site. Cybercriminals developed a malicious version of Xcode, then gamed the systems to ensure their version of the software would show up at the top of online searches.

Apps developed with the malicious Xcode software contained extra code that would phone home and provide the attacker a backdoor—and an opportunity to inject malicious code or execute commands within the compromised apps.

Low-Hanging Fruit

The Xcode Ghost incident is an excellent example of a software supply chain attack, and a perfect illustration of attackers going after the low-hanging fruit. iOS is relatively secure. The Apple App Store is protected and apps are reviewed to ensure they meet strict criteria. So, instead of trying to go after Apple or iOS, attackers figured out how to strike the platform used to develop the apps and sneak in through the back door.

I spoke with Ryan Olson, Intelligence Director for Palo Alto Networks, about the rising threat of attacks on the software supply chain. He explained that attackers know that when they are going after a hard target that would be difficult to defeat directly, there is a better and easier way: Just figure out who they trust and go after the low-hanging fruit.

This becomes a potentially larger issue in the context of DevOps and automation. The recent NotPetya ransomware was propagated through a malicious update. If your systems or software are configured to update and/or deploy automatically, a software supply chain attack can be in the wild and compromising your applications or data before you know it.

There are tools available to scan images, applications or containers before deploying them, but they typically test for known vulnerabilities or overt crashes or conflicts. They are not designed to look for latent backdoors that might be buried in the code.

Olson recommends that you start taking a closer look at the software vendors you rely on. It is important to understand how they are securing and protecting their code—because their weaknesses expose you to threats by virtue of the trust you give them and their apps.

— Tony Bradley

April 2, 2018
DevOps Careers: Report Examines What It Takes To Be an SRE

Do you want to be a site reliability engineer, or SRE? Sixteen years ago, the role didn’t actually exist, but now you can find more than 1,000 opportunities for a career as an SRE on LinkedIn. Before you decide if you want to explore such a career, though, you should check out the new survey from Catchpoint, which examines the role and what exactly SREs do.

Catchpoint launched its survey in January with the goal of determining what exactly it means to be an SRE. The survey takes a closer look at the types of organizations, skill sets and organizational cultures in place where SREs work. Catchpoint wanted to build a profile of the SRE role and determine whether there is a core set of principals or skills common to the SRE role.

The report includes responses from 416 participants representing a range of roles, industries and company sizes. Nearly 4 out of 10 respondents actually identify as SREs. More than half work for companies in technology-related industries, and more than 40 percent indicated that they work for a software-as-a-service (SaaS), infrastructure-as-a-service (IaaS) or other as-a-service (XaaS) provider. More than 50 percent represent companies with at least 1,000 employees, while just under 40 percent are from larger organizations of 5,000 or more employees.

What is an SRE?

The SRE role originated at Google in 2003. The SRE straddles the line between the “Dev” and “Ops” sides of DevOps. SREs foster collaboration, productivity and software performance and—ultimately—gives developers greater responsibility for the performance of the applications they produce.

SREs typically both write code and support existing systems. Nearly two-thirds of SREs report they are deploying code daily—with almost half indicating that they deploy code multiple times per day. According to the report, most SREs report to software engineering despite the fact that a large percentage of SREs actually come from an IT operations background.

Coding skills and a solid knowledge of IT operations are both valuable resources for an SRE, but there are also a variety of “soft skills” that seem to be crucial to success. Problem-solving, teamwork, composure under pressure and strong written and verbal communications regarding incident resolution are all necessary to be effective as an SRE.

Novices Need Not Apply

The Catchpoint survey contains some bad news for fresh college graduates looking for their first gig, and excellent news for experienced IT professionals: The SRE role is not an entry-level job. Eight out of 10 SREs have a college degree and have worked in IT for six or more years. A college degree in computer science or information technology isn’t necessarily a requirement, but 73 percent of SREs did, in fact, get degrees in a technical field of study.

Although a majority of SREs surveyed report to the software engineering department, most come from an operations or infrastructure background. Just under 65 percent of survey participants reported that they held a role as a sys admin, while only 53 percent previously held a role as a developer or software engineer.

To learn more about the site reliability engineer role, check out the full 2018 SRE Report from Catchpoint.

— Tony Bradley

March 30, 2018
DevOps: What a Wonderful World it Would Be

The promises of DevOps are clear. They offer faster iterations, allowing ops to keep up with agile development and make an IT organization more responsive to business needs.

DevOps doesn’t make less code. It doesn’t reduce configuration files. It doesn’t eliminate deployment issues (though it might reduce them). It doesn’t eliminate testing; indeed, it creates more if Agile and DevOps are taken to their logical conclusion: More testing, earlier. DevOps doesn’t reduce the need for security.

In short, it doesn’t reduce workloads. It might automate some of them, freeing up man-hours, which is absolutely a positive result—one of the more positive results, in my opinion. But it also introduces the overhead of the DevOps toolchain, so some of that automated freed-up time is already spoken for.

And automation of testing (for example) is something that’s been around for decades. The first UI-based automated testing system I recall was Microsoft Test. I managed a team that used it extensively to automate test runs of the software we delivered to financial services. It worked well, and there was much gnashing of teeth when Microsoft discontinued the product.

The same is true of most of the time-saving benefits of DevOps: We’ve tried them before, but DevOps approaches them from a different perspective—and, in many cases, a more sustainable perspective.

I’m a DevOps aficionado, make no mistake. The reason I’m stressing what it won’t do here is because misperceptions drag it down. Don’t go into it looking to reduce costs, because at least in the short run you won’t. Don’t go into it looking to save work, because at least in the short term those savings will be minimal. Going into it for the wrong reasons means the things you are measuring will ultimately look like failures, even if all the good DevOps offers is reaped.

Instead, go into it looking to be more responsive. Go into it looking to beat your competition to market. Go into it looking to increase communications across the team. Delve into DevOps for the right reasons, and you’ll be measuring the right outputs.

And remember, it is an iterative process. Don’t overstate expected benefits on the first pass. Low-hanging fruit makes it look as if DevOps will do a lot of things, but low-hanging fruit often has tight ties into firmly rooted trees, making it harder to grab than originally thought. Use a measured approach and look for consistent improvements. In some organizations those improvements might be small at first. But use your knowledge of the organization and application portfolio to create a measured success plan.

Reaping the benefits is worth the change DevOps brings. The ability to tell a line-of-business leader that the new functionality they are considering could be mocked up and ready for review in a week or two is a huge benefit to the organization. And it doesn’t hurt at all that such responsiveness will raise the rest of the organization’s opinion of IT.

So it is a wonderful world, if your goals are the right ones.

— Don Macvittie

March 20, 2018
The ‘Blockbuster Effect’ in the Age Of Automation
Saying that new jobs will always be created to replace those that have been eliminated by automation is a mechanism for avoiding a real problem that’s lurking on the horizon.

In 2004, at the height of its popularity, Blockbuster Video had more than 9,000 stores and employed more than 80,000 people worldwide. About 50,000 of the total workforce was employed in the United States. Going down to Blockbuster to pick up a movie was a Friday night ritual for many, this writer included.

Blockbuster and other video rental stores, along with the advent of affordable Big Screen TVs, were significant contributing factors in the growth of the home theater. People now had an alternative to $10 movie tickets and battling weekend traffic to get to the cineplex. An entire family could view a recent movie in the comfort of their own home for about $5. The only thing that was needed to complete the movie going experience was the popcorn, which frugal families bought at Costco by the caseload. Those who were less thrifty could buy a box of microwave popcorn at the checkout counter of the video store. The $25 cost of a Friday night date was reduced to roughly $6.

Blockbusters kept people on the couches and out of the movie theaters. Television had already put a damper on movie-going. In 1930, 65 percent of the population went to the movies once a week. By 1964, the number dropped to 10 percent. Despite the introduction of 3D projection technology and movie theaters offering added features such as in-seat dining, movie attendance is now at its lowest point since 1992.

Blockbuster was a transformational business on the entertainment landscape. Today, the company is gone. Streaming technology combined with an “all you can eat” subscription model killed it. (See Figure 1.)

Figure 1: As Blockbuster stock declined, Netflix’s grew.

Just as Blockbuster did away with the need to go to a movie house to see a film, Netflix, Amazon and other video streaming services did away with the need to leave the house to get the evening’s entertainment. America stayed home and enjoyed movies on the internet for a fixed monthly fee. Blockbuster went out of business.

Netflix employs around 5,500 people with gross yearly sales of $11.69 billion. Blockbuster’s gross sales in 2004 were $6 billlion. It’s astounding that Netflix does almost twice as much business as Blockbuster did at its height, with less than 10 percent of the Blockbuster workforce. Such is the benefit of automation.

Whereas Blockbuster needed to staff a store in every neighborhood to do business, Netflix and other streaming services work from a central, virtual location without the added expense of redundant staffing. Blockbuster had 9,000 cash registers to man. Netflix has one, which is run by some sort of AI.

So, where did all the out of work clerks and store managers go? Hopefully, they went on to other jobs, maybe ones that were far better than the day-in, day-out tasks that go with processing video cassette rentals. After all, that’s the conventional wisdom: There will always be new jobs to take the place of the ones eliminated by automation. But, let’s face a cruel truth: It takes a lot more knowledge and skill to work on video delivery at Netflix than it does to check out customer purchases at Blockbuster. And Netflix requires fewer people to make the business go. Would a Blockbuster employee in Sioux City, Iowa, have the skills and mobility necessary to go on to to work at Netflix in Los Gatos, California? Seems pretty far-fetched.

Right now, things are pretty rosy in terms of the economy. So the conventional wisdom prevails: As automation eliminates one job, another new type of job takes its place.

Yet, there is another way to look at it. There are good arguments to be made:
- As automation eliminates jobs in the modern world, new jobs for humans will NOT be created at levels required to keep a workforce fully employed. Rather, AI will do more and require less.
- The lead up to large-scale unemployment will not be slow. It will happen fast, over a period of a year or two. And, it very well might happen within a decade rather than decades in the future.
Blockbuster’s demise came about pretty quickly. The company was at its peak in 2004. Six years later it filed for bankruptcy with $1 billion in debt. The music retailer Virgin Megastore closed its last North American store in 2009, a year after music streaming Spotify went public on the internet. (Spotify generated $3.8 billion in revenue in 2016 with an employee count of 2,162.)

Productivity due to automation continues to accelerate. In the early 1980s, it took 10.4 man-hours of labor to make a ton of steel. By 2014, the number was down to 1.9 man-hours per ton. The increase in productivity is due to increased levels of automation (think: robots). Given the choice between buying a machine and hiring a worker, companies are more likely to choose the machine. Thus, it’s entirely possible to imagine that the day when a few humans will be supervising the work of many robots making steel is close. Not too soon after that, a single human will be supervising an entire robotic workforce. Then, finally, as AI continues to improve and further dominate the production landscape, no humans will be required. The robots will supervise themselves.

How far off in the future is this time? Dunno. But I can tell you that robots are now assisting with surgery on humans. How long will it be until one surgeon can use automation to do the work of 10? Then, how long will it be until AI can do surgery unsupervised, five years? Ten years? Only 10 years ago driverless cars were still on the test tracks; today they are roaming the streets of Mountain View, California. An autonomous robot with a scalpel is no longer the stuff of science fiction. Once AI can do surgery unassisted, where will all the displaced surgeons find work? Maybe a few will go into research. What of the others?

At one time there were 200,000 horses in the City of New York. They provided the labor for a good part of the city’s transportation system. Today there are fewer than 1,000 horses. Cars, trucks, planes and trains have taken over. There’s no benefit to employing a horse when a car can do the work better. As General AI continues to develop and improve its ability to connect the dots as only a human can, how long will it be until the labor of most humans is not needed? What will it be like when there is no benefit to employing large parts of the human population because a machine can do the work better? Yes, the ramifications are scary. But, the possibility needs to be considered. The sad fact is that few people are considering such a scenario, let alone coming up with ways to address the impact on the horizon. Saying there will always be employment for humans is an avoidance mechanism, particularly given the fact that modern machines are going beyond purely mechanical labor and doing more thinking work.

In 1990, people thought there would always be movie rental until there wasn’t. In 2000, people thought there would always be a Virgin Megastore until there wasn’t. Today, Sears keeps closing stores. It will soon be gone from the face of the earth. Yes, Amazon is opening brick and mortar stores. This might be good news. But the fact that these stores are highly automated and have few employees cancels out the good feelings of the moment.

These days when a company comes to an end, it’s not a drawn-out process. It happens fast, over a period of a year. Many of the people left behind are confronted with low-wage, “show up and do it” jobs. The more educated have access to a pool of better paying work that comes with an excruciatingly steep learning curve. Whether the job is low-wage retail work or high-paying tech work, each worker is—or will be—competing against AI-powered automation that can—or will—do the job better.

It’s not a pretty picture. But it is one that can avoided if we decide to think about alternatives and put those ideas into action. Sadly, there are more people thinking about how to get to Mars than solving the problem of unemployment on a massive scale due to the proliferation of industrial automation powered by general AI. The problem is that we still seem to be stuck at, “There will always be jobs to replace the ones eliminated by automation.” To say otherwise is deemed Cassandra talk. Maybe so. But to not give the problem consideration is beyond folly. It’s a danger that will exact a heavy price from everyone who has to work to earn a living.

— Bob Reselman
March 19, 2018
DevOps Isn’t Getting Cloud Security Right (and What Can Done About It)

The song title of Billy Joel’s 1977 hit, “Get it Right the First Time” aptly describes what DevOps teams need to take to heart when making the jump to the public cloud. But in most cases, security is lacking in the pipeline when making the shift—and a change in DevOps culture and mindset is necessary to “get it right the first time” for cloud security, according to McKinsey analysts.

It seems commonsensical to make sure adequate security protocols are extended to the DevOps pipeline before migrating to the public cloud. But in practice, statistics tell a different story.

According to analyst firm McKinsey, a full 78 percent of more than 100 firms recently surveyed are not reconfiguring their security tools when migrating to the cloud. Similarly, most security standards in place among the enterprises listed in the survey are non-sustainable for cloud networks.

“There are, of course, many security configurations for a public cloud for DevOps that need to be in place,” said James Kaplan, a McKinsey partner and cybersecurity lead. “Creating your server image in a public cloud can create many vulnerabilities [compared to in-house data centers].”

Security improvements will obviously need to be made on an industrywide scale in the near future as more companies embrace the cloud for DevOps. Currently, only 10 percent enterprises McKinsey surveyed have ported more than 40 percent of their workloads to the cloud. Within three years, that percentage is expected to jump, as 80 percent plan to migrate more than 40 percent of their workloads to a public cloud platform.

Culture Lag

At issue is how DevOps teams’ cloud projects very often remain straddled by security practices dating back to the 1990s.

“Most DevOps security is still with a ticket-based model,” Kaplan said. “They just called Joe in security and asked him to do x, y and z.”

Porting legacy infrastructure and operations to the cloud can also be problematic. A startup, for example, might make the jump more easily than a 30-year-old Fortune 500 company with data centers located in the North America, Europe and Asia. “It’s one thing if you’re building greenfield, such as creating a video streaming service from the ground up,” Kaplan said. “It’s much tougher to go to an agile DevOps cloud platform world if you are dealing with 30 years of legacy applications, eight mergers and all that type of good stuff.”

The Big Meld

Cloud servers can be fired up and ready to use in minutes. But it can take weeks before standalone security departments catch up. This might mean waiting on a security team to manually complete vulnerability scanning, penetration tests and firewall installation before cloud deployment. Or, in the worst case, these steps remain incomplete, leading to relatively easy-to-exploit vulnerabilities on the public cloud.

“DevOps cloud security requires particularly high levels of sophistication and capabilities, as well as agility,” Kaplan said. “There has also traditionally been less focus on speed.”

The solution is to converge security with the DevOps from the outset ahead of cloud development. In this way, security controls, reviews and deployment are integrated with DevOps, to create DevSecOps for the cloud migration process. This heavily involves developing APIs for security automation, Kaplan said.

Implementing DevSecOps also requires investments in training and culture. This consists of ensuring DevOps teams receive additional training to ensure security is integrated into every step of development work and for developers to maintain security support once the switch to the cloud is complete, Kaplan said. Training ensures DevOps team members have working knowledge of security APIs and orchestration technologies, and are eventually able to create new security APIs for DevSecOps, Kaplan said.

The end result is that cloud deployment remains agile, as it should be. Cloud migration is done more swiftly and efficiently compared to traditional security models, once the DevOps team has acquired the security knowledge it needs, Kaplan said.

“DevOps teams really need to expand their purview to think more about security. They must create more rigor around the security services and how the developer journeys should involve security,” Kaplan said. “And they then need to think about how to incorporate those security APIs into the application scripts for infrastructure APIs, capacity scaling and what have you.”

— B. Cameron Gain

March 15, 2018
Enthusiasm for DevOps Greater Than Actual Adoption, Survey Shows

When it comes to DevOps practices, adoption continues to lag intention by quite a bit.

GitLab recently surveyed some 5,300 software developers and engineers from mostly small and medium businesses around the world. The survey examined overall attitudes and perceptions around the state of DevOps, with a particular focus on uncovering any disparities that might exist between developers and IT management.

The responses showed that many organizations are sold on the idea of DevOps, especially those who have implemented it. Sixty-five percent of the developers polled felt that using a DevOps workflow would save development time. An even higher 81 percent of managers shared that same sentiment.

Organizations with an integrated DevOps strategy reported a high correlation between DevOps practices and overall team productivity. Development teams that deploy code on demand and spent more than half their time on new projects, for instance, had twice the rate of DevOps adoption compared to lower-performing teams. Higher-performing teams also generally considered automation of the software development life cycle a top priority and had access to better tools.

Yet, the number of organizations in the survey that have actually implemented DevOps practices was far smaller than the otherwise enthusiastic support for it would suggest. In fact, only 23 percent of the respondents in the survey identified their team’s methodology as following a pure DevOps model. Thirty-five percent described their DevOps culture as being somewhat established.

The low adoption rates are somewhat at odds with Forrester Research’s prediction about 2018 being the “Year of Enterprise DevOps” and may reflect the global nature of the survey base and the fact that many respondents were SMBs. Forrester found DevOps adoption to be around 50 percent when it conducted a survey of its own of 237 DevOps pros last year.

GitLab’s survey found that while developers view DevOps as critical to software development life cycle, many have not implemented a DevOps workflow. Similarly, while over 95 percent of the survey respondents placed a high value on collaboration and 81 percent felt that such collaboration was easy to achieve, 25 percent said they did not have enough visibility into operations, security and production. And 42 percent reported unclear directions as one of their biggest challenges to getting work done.

Culture continues to be a big roadblock. “It’s hard to get Dev and Ops to work together in a cross-functional way,” said Mark Pundsack, head of product at GitLab. “That challenge can be exacerbated by tooling. Disjointed toolchains lead to disconnected processes and barriers to communication that get in the way of working together.” Many developers in the survey reported using at least five tools to get their job done.

Nearly 6 in 10 of the survey respondents (58 percent) pointed to ingrained practices as their biggest roadblock to adopting new tools and practices, while 50 percent blamed an inherent resistance to change.

Interestingly, remote employees tended to have a more favorable outlook on how things were going with DevOps, with 67 percent saying they had good visibility into what others in the organization were doing compared to 57 percent of on-premises teams.

“DevOps culture is clearly important to achieving high performance,” Pundsack said. “For managers … the survey identified the top challenge in getting work done was unclear direction and managers should be able to play a role in fixing that.”

The apparent gap that exists between intent and adoption was not the only surprise in the survey, Pundsack said. Also unexpected was the extent of the discrepancy between how executives think their teams are operating, and what developers are actually saying. When asked how often they deployed code, 47 percent of managers were under the impression their organizations were deploying multiple times a day in contrast to the substantially smaller 39 percent of developers who said the same thing.

That one area aside, though, developers and IT manager seemed better aligned on other issues in the Gitab survey. For example, more than three-quarters of both developers and managers feel they are set up to succeed and about one-third of them felt that they deadlines they have to deal with are fair.

GitLab’s survey showed that most organizations expect to invest in continuous integration and continuous deployment tools in 2018, with nearly half strongly agreeing that CI can help speed up the development process.

“CI can help alleviate blockers in the development cycle such as removing the manual process to verify code before deploying,” thereby allowing development teams to have a faster and more effective feedback cycle, Pundsack said.

— Jai Vijayan

March 13, 2018