The COVID-19 global pandemic has ushered a significant priority shift for the digital workplace. As development teams become completely remote, executive priorities are rapidly adapting to meet the gravity of these unanticipated times.
But, were IT leaders prepared for such a dramatic shift? What are they doing to evolve? To answer these tough questions, let’s turn to the data. A March 2020 study, conducted by Adobe and Fortune, surveyed 200 chief information officers (CIOs) on how their priorities are changing to meet the needs of a global crisis. Here’s what they found.
Remote Work
As it turns out, the majority of enterprises (84%) report they were prepared for remote work, with a slight increase for SMBs, at 94%.
“Like most business leaders, CIOs now have to rethink priorities, or at least reorder them, and we must reinvent ourselves now as virtual leaders,” said Adobe CIO Cynthia Stoddard.
We are living in a golden age of virtual collaboration tools—teams have Slack, GitHub, Zoom, Microsoft Teams and many other options. Yet, 53% of CIOs still cite communication as the biggest hurdle to remote work. Nevertheless, half of the CIOs surveyed reported their organizations are still actively hiring, indicating steady growth for IT.
New Technologies
Regarding technological adoption, some organizations are struggling to keep pace. Twenty percent of CIOs report serious shortfalls in technology tools. Interestingly, less than 25% of CIOs estimated increased financial spending for areas such as public cloud, infrastructure and artificial intelligence/machine learning (AI/ML).
“Indeed, nearly all CIOs (90%) surveyed said they use a public cloud service for at least some of their data, but most is housed on-premises,” said Giselle Abramovich. “Only one in three organizations store half or more of their data in a public cloud.”
One thing most enterprises agree on is the need for better security—seven in 10 anticipated increased financial investments into cybersecurity.
Artificial Intelligence
Only 50% of enterprise CIOs said they use AI in one or more projects (25% for SMB CIOs). However, it seems to be a growing discipline, as 90% of the organizations that have implemented AI have done so in the past year.
Though AI has garnered significant interest, there are hurdles to actual implementation. According to the report, “the biggest issues related to AI include getting data in order, funding and finding the right talent.” With more people online and the importance of distanced communication, it is likely that investments into AI will increase, especially for use in IT support and customer interactions.
“We’re making AI a strategic priority at Adobe. AI implementation is difficult for many organizations because it’s still new to most businesses, and there is a shortage in talent that has a deep understanding of it,” said Stoddard.
Analysis: Impact on DevOps
CIOs are in a unique position. Leaders are reevaluating business practices and also reassessing their technological investments. As the Adobe/Fortune study shows, many companies were prepared for 100% remote work, albeit some more than others.
To endure a COVID-19 economical drought, CIOs are taking action. The report indicates increased investment into remote tools, AI and cybersecurity, while aligning hiring and new infrastructure to reach these goals.
Institutions with a previous investment into cloud infrastructure as well as scalability practices, such as serverless, are likely well-equipped for increased online interaction and unanticipated usages. In this virtual realm, automation is also key to streamline deployments and monitor applications in remote fashions.
Though the report doesn’t differentiate between company types, it’s likely that digitally native enterprises are well-equipped as well. In addition, the trend from physical storefront to “brick and click,” has only been magnified by the current climate, forcing retail and banks to invest heavily in digital technologies and DevOps.
From a DevSecOps standpoint, increased end-user online action means an increased number of threats. Thus, security strategies for containers and service meshes should be reevaluated.
By repositioning priorities, IT can adapt to the new state of interaction. With deliberate execution, CIOs and all executives alike could find themselves on the right side of the river once the economy resumes.
Simply put, data breaches are terrible news for companies. And, the costs associated with such attacks continue to escalate. A recent IBM-sponsored report found an average price of 3.92 million per breach.
Not adapting security automation and vulnerability scanning into development pipelines could have a drastic effect not only on cost but workload efficiency and team morale. With these sorts of headaches, it’s vital to consider the repercussions for not adopting secure (and lean) armaments and auditing procedures.
Can organizations adopt DevSecOps beyond the buzzword it has become? In this article, we’ll uncover some startling evidence on the state of Kubernetes vulnerabilities and discuss what companies can do to rectify.
Trends in Kubernetes and Istio Vulnerabilities
It’s astounding how much in DevOps is left exploitable. Secrets are routinely unknowingly left in the open due to human error. These are the most common gateways for attackers. Plugging these gaps is the first, albeit obvious, step.
There’s proof of an epidemic, too. A recent study conducted by Alcide ran a live production analysis of over 500 cloud-native environments across the globe. According to their findings, “89% of deployment scans show companies are not using Kubernetes’ secrets resources, with secrets wired in the open.”
This means the majority of Kubernetes environments share susceptible to data breaches in some form. Among the 5,000 scans run, the Alcide report found secrets are commonly left open in a few areas, including:
Workloads that had to connect to cloud resources that exist outside the cluster
AWS access keys to connect to those resources
Tokens for workloads that require API access to external databases
Credentials for S3 buckets
Gaps in Best Practices for Kubernetes and Istio
Kubernetes brings many nuances to the table. It’s a sophisticated system, with a wide range of use cases and cloud-native experiences. These require extensive auditing and compliance checks to safeguard deployment.
According to Gadi Naor, CTO and co-founder of Alcide, Kubernetes requires a good level of expertise. However, “there is a knowledge gap that DevOps doesn’t have,“ said Naor.
Much of this gap lies in Kubernetes secrets handling and network policies. On top of Kubernetes, service meshes are a next evolutionary leap. Istio, an architecture layer for creating network policies is ten-fold more complex, according to Naor.
On network policy creation, “75% of the scanned deployments use workloads, which mount high vulnerability host file systems; while none of the surveyed environments show segmentation implementation using Kubernetes’ network policies,” stated the Alcide study.
These are compelling platforms, but best practices must meet the growing number of potential attack vectors.
Why You Should Approach CI and CD Security Separately
Many pundits now link continuous integration (CI) and continuous deployment (CD) together as if they’re synonymous. However, they are separate concepts. Securing them, in the same manner, could decrease overall efficiency.
Scanning all resources created in the CI phase against security auditing could waste time and resources. For example, if you’re standing images for CVEs and only deploy 20% of those images, that is a lot of wasted exercises.
Rather than scanning all environments in the CI phase, Naor believes you should focus on scanning 100% of images for CD. Therefore, security tooling is leaner, and anything deployed will be assuredly safe.
Takeaways on the Cost of Poor DevSecOps
Infrastructure components such as Kubernetes and Istio are powerful allies to the DevOps minded teams. These environments do come with their nuanced security concerns, which must be addressed. Otherwise, the data breach price tag is high.
To intercept system frailties before the black hats do, you must consider the following:
Scan for the Obvious: Utilize automation to discover secrets, keys, tokens, left in public repositories.
Scan in a Lean Way: Maintain a lean security environment that retains your engineer’s attention.
CI vs CD: Recognize the security nuances between integration versus deployment.
Add Auditing Tooling Strategically: Adopt compliance and auditing tooling that doesn’t create friction with existing developer workflows.
How do we define DevSecOps? A combination of DevOps and security is readily apparent, but the philosophy goes much deeper. In a recent eBook, The State of DevSecOps, we asked industry experts to define what DevSecOps meant to them. Below, we’ve condensed their answers into five core attributes.
Following these principles, CIOs or CTOs now have a DevSecOps doctrine applicable to nearly any software development and release environment.
Security Automation
DevSecOps emphasizes security objectives within the automation processes. Ben Newton, director of product marketing and evangelism at Sumo Logic, defines DevSecOps as first making security requirements and objectives a clear part of the continuous integration (CI) and continuous deployment (CD) cycle.
Culture of Security
DevSecOps establishes a team culture that embraces security concerns. As stated by Ben, a security team must provide “clear guardrails for developers as to what is fair game and what is not appropriate from a security perspective.” Building security expertise to integrate security into the entire DevOps lifecycle is critical.
De-Siloing IT
DevSecOps is about eroding boundaries. Just as DevOps has eroded the traditional separation between software engineering and IT operations teams, “DevSecOps further erases the walls between the DevOps team and IT Security,” said Tim Jarrett, senior director of product management at Veracode. “DevSecOps is about building a bridge between the security and DevOps teams,” echoed Dan Hubbard, chief product officer at Lacework.
Security Shifts Left
DevSecOps places security earlier on in the development process. IT security is traditionally viewed from a risk avoidance and compliance standpoint. Rather than viewing security as a gate, Tim noted that within DevSecOps, “security is better positioned to integrate earlier in the development cycle where they can actually make a difference.”
Security Enables, Not Stalls
DevSecOps supports, not stalls, agile development. DevSecOps doesn’t have to be sluggish. As Dan described, DevSecOps must “support the need for DevOps to move fast, but in a way where security is not ignored.” By embracing a security-as-code mindset and involving practices such as automated threat detection, agility is not sacrificed.
I attended Adobe Summit 2019 to research the intersection of DevOps and marketing. Below are insights from my interviews with leading marketing technology experts.
Is DevOps making its way into marketing? In a short answer, a soft “Yes.”
I recently attended the Adobe Summit, held March 26-28 in Las Vegas. This year emphasized digital customer experiences, and I was curious to take a peek into this world of marketing tech to see if DevOps was having any impact.
Many of the people I talked to described how marketing technology is adopting a more agile, DevOps-inspired mindset. But is marketing actually driving product change? While Adobe and proponents would like to have us think so, some are not as sure.
At Adobe Summit, I had some eye-opening meetings with professionals invested in digital marketing, retail and analytics services, most of which are using Adobe products in some form. In this article, we’ll highlight what they had to say on the intersection of DevOps and marketing.
What is the Role of DevOps in Marketing?
The need to break down data silos was a common theme among keynotes, sessions and throughout my meetings at Adobe Summit.
When large digital marketing agencies take on clients, for example, a common disability they find is a disparate and dysfunctional customer data storage system. In our meeting, Alex Kelleher, managing director at Deloitte Digital, noted, “Data needs to be available, and integratable. That is the first thing we typically fix for our clients—we’ll fix the data in silos, get it in the same place, and usually simplify it.”
Deloitte Digital’s process involves three D’s: first, they consolidate data silos, then build data-driven decisions, then deliver agile products. In doing so, Kelleher is seeing more and more developer practices making their way into marketing and sales.
“We’re seeing CMOs manage their teams as agile teams, recruiting agile and scrum masters,” he said. “That’s very cool for CMOs to be involved in … those worlds colliding is super exciting.”
Is DevOps Eroding Traditional Boundaries?
DevOps principles encourage shared knowledge and open communication between teams. Now, it appears such ideas are affecting how marketing technologists architect their teams and delivery cycles.
Rachel Lane, solution principal customer experience at Medallia, sees a trifecta of forces working in tandem. “Development, Ops, and Marketing must work together,” she said. “If one of those doesn’t link up, then we see extended lead time to launch, and it can become problematic.”
Medallia’s goal is to meet customers where they are to gather actionable feedback through push notifications, surveys, embeddable surveys, Net Promoter Score (NPS) systems and other means. Engrossed in this world of customer feedback, Lane reminded us: “You should never ask a question that you already know the answer to.”
Unfortunately, this still happens often. User data is often locked in silos and if multiple teams (sales, marketing, support engineers, etc.) are using separate user data repositories, customer interaction can suffer. Thus, user conversation data must be de-siloed and unified.
For Lane, having tight customer experience-driven feedback is critical for reiterating the overall product. However, customers typically hate filling out surveys. To decrease friction, she foresees more passive voice-driven feedback through home AIs.
Rearchitecting the Digital Footprint for Agility
By now it’s evident that unclear data initiatives and messy data storage can hinder digital transformation. So, what are organizations doing to de-silo and open up data?
Anheuser-Busch (AB) is a huge beer corporation, overseeing brands including Budweiser, Busch, Elysian and many others. Each has unique online eCommerce storefronts to sell ancillary swag, such as T-shirts, drinkware and other products.
With public-facing sites for hundreds of brands, maintaining data standardization across its product lists was a nightmare. Disparate data structures can also inhibit directing change from within, slowing corporate-led growth.
John Faviano, director of demand and integrated marketing technology at Anheuser-Busch, described how AB solved the issue by creating a consistent data structure from which all online stores can pull content from.
“We’ve standardized the tech stack in the last three years to enable the business to run more efficient … You have to think from a platform perspective, not a single siloed technology,” he said.
AB’s web APIs now grant incredible scalability, as the content is now delivered agnostic of client or brand. AB brands retain independent facades yet are fueled by a unified content storage and delivery process. “Our API strategy is a build-once-and-scale mechanism,” Faviano said.
Digitization with IoT and AI
Sreedhar Bhagavatheeswaran, SVP and global head of digital business at Mindtree, believes that every part of the enterprise needs to be digitized. He sees a wealth of untapped data that enterprises can leverage to transform user experiences.
Having built digital presences for big-name consumer brands including Marriott, P&G and Kellogg’s, Mindtree understands the advantages of using a DevOps approach for software cycles. Though it may require a cultural change, “The probability of project success in a DevOps environment is much higher than in a non-DevOps environment,” Bhagavatheeswaran said.
Release automation is integral to Mindtree’s process. Coding checks and CI tools such as Jenkins bring quick releases, enabling business teams and clients to see the impact on a daily basis. They also privilege API-fueled microservices designs for increased agility.
“From an agile program management perspective it gets challenging, but that is the nature of the beast,” he noted.
Bhagavatheeswaran is excited for the potential IoT has to evolve industrial, business and consumer-facing touchpoints. For example, industrial engineers could conduct remote servicing using digital twins of real physical equipment. He also described developing mobile IoT apps for his clients: Push notifications triggered by removing caps from makeup bottles, for example, could empower consumers with advice on how to apply makeup correctly.
To sum it up, Bhagavatheeswaran described our current paradigm with a simple, powerful sentiment: “The experience is at the confluence of edge, humanity and intelligence.”
Approaching the Developer Experience
But digital transformation doesn’t stop with open data, APIs and IoT. Developer experience (DX) has become an important factor in maturing countless digital services.
Jason Woosley, VP of commerce product and platform at Adobe, founded the Adobe-acquired Magento, which began as an open source project on GitHub, welcoming pull requests to alter product features (with the proper vetting in place, of course).
By investing in open source and developer communities, Magento over time reached a critical mass, democratizing how developers build online storefronts.
Woosley estimated that 50 percent of Magento code was built by third-party developers. One such feature is multi-source inventory, which, he said, accounts for 70 percent of the Magento code base. Naturally, he has a deep appreciation for the power of developer communities.
“You can’t buy a developer community. You can’t steer or create a culture,” Woosley said. “Code custodians are actually participants, and to deserve that role DX needs to be delightful and empowered.”
To improve the developer experience, Magento has hosted countless events, increased developer support staff, annotated its APIs, adopted GraphQL for extensibility, invested in optimizing developer portals and more. With the inclusion of Magento in Adobe Experience Cloud, more capital is available to further DX enhancements—something Woosley is excited about.
Wednesday March 27, 2019 in Las Vegas. (Jeff Bottari/AP Images for Adobe)
DevOps is Kicking Martech to Innovate
Deloitte Digital’s Kelleher noted that “nowadays, your competitors are any digitally active application,” regardless of whether they’re even in the same industry.
This means that a consumer will judge a mobile digital experience with an airline with the same scrutiny as calling an Uber or making a PayPal transfer. Users hold the same lofty digital expectations from any business, forcing us all to adapt dramatically.
Fueled by the relentless pressure from users, marketing technologies are adopting DevOps principles to improve their offerings in a few ways:
Continuous releases: Martech uses DevOps strategies to release quickly. User-facing mobile app release cycles, for example, routinely adopt CI/CD.
De-siloing: User data and consumer conversation information are being distributed with greater alacrity across enterprises, improving end UX.
Open data initiatives: With web APIs, new platform-agnostic content distribution architectures have emerged. Development procedures are standardized across an enterprise, quickening the release cycle.
Cloud architecture: By adopting cloud-based management tools, martech is improving the asset management condition of supporting many user-facing touchpoints.
Developer experience: By embracing open source and supporting developer community, leading e-commerce groups are improving their offerings.
While marketing is critical to the whole, it may not be the No. 1 business driver. When I asked Bhagavatheeswaran whether marketing was really driving the creation of new tools, he responded: “Is marketing driving the tools? Are they influencing? Not quite yet. However, it has the ability to influence the developer’s quality. Probably not their job.”
Even if marketing is not directly influencing product development, we’re still seeing DevOps seep into traditionally non-technical environments, which is exciting.
It was eye-opening to enter the world of marketing technology to see firsthand how these institutions—such as Ansira, a 100-year-old company—are using agile means to streamline digital transformation initiatives.
With an emphasis on both creating and managing quality digital customer experiences, Adobe Summit 2019 shed light on many unique strategies being undertaken in the realm of martech.
Whether it’s leading the development of user-facing apps, overseeing expansive brand catalogs, using a cloud-based single source of truth for content management or building developer communities to reap huge R&D benefits, the intersection of marketing and technology has never required more strategic action than it does now.
We’re attending Adobe Summit 2019 to research the intersection of DevOps and marketing. Follow along for more stories.
In a recent article, we described how DevOps is transforming retail from brick and mortar into an omnichannel “brick and click.” At Adobe Summit, we garnered more insights, discovering how other institutions are implementing an innovative convergence of physical and digital spaces.
The result is impacting all sectors, creating new hybrid environments within home tech, banking, apparel, and more. Let’s see which ones Adobe decided to highlight in its opening keynote to discover how companies of all types are reinventing themselves through digital transformation.
Best Buy: Evolving 1 Storefront into 3 Services
Best Buy is a prime example of digital transformation for retail. At Adobe Summit, Best Buy CEO Hubert Joly described how the company is reinventing the “New Blue” to enrich lives with the help of technology; a sense of purpose is a mobilizing factor that affects innovation.
For $200 a year, Best Buy in-home advisers will be your No. 1 on-call tech support. The company is also using AI to monitor seniors to trigger assistance when they are in need. Lastly, an app that compares features is bringing new in-store customer experiences.
Not solely a brick-and-mortar retail anymore, Best Buy is transforming with subscription services and online presences, bringing in multiple new touch points to stay competitive with eCommerce.
Best Buy CEO Hubert Joly sits down with Adobe CEO Shantanu Narayen March 26 in Las Vegas. (Jeff Bottari/AP Images for Adobe)
Intuit: Reinventing the SaaS Platform Model
Atticus Tysen, SVP & CIO at Intuit, described, how, over the years, Intuit has transformed its platform-based SaaS offering. Intuit is now utilizing ML and AI to improve its customer experience process. Doing so requires great data, as well as powerful data structures. As Atticus mentioned: “It all comes back to data…You need to monitor your data pipelines just like you monitor your offerings.”
According to Atticus, a core component of curating data infrastructure is democratizing it. Opening such data to all employees dismantles silos to increase the speed of IT. It’s this sort of horizontal thinking that fosters the collaboration digital transformation requires.
Atticus Tysen, SVP & CIO at Intuit talks with Cynthia Stoddard, SVP & CIO at Adobe, March 26 in Las Vegas. (Jeff Bottari/AP Images for Adobe)
Zumiez: New Zero-Code Methods to Build Online Shopping
Jennifer Small presented how the newly Adobe-owned Magento can be utilized to build and personalize shopping experiences. She walked through a theoretical Zumiez implementation, demonstrating how to make visual edits to develop cross-channel retail applications.
With the rise of eCommerce, traditional retailers such as Zumiez must create lean digital spaces to interact with end users. Not only must such environments be agile, but every point along the customer journey also must be individually personalized.
Online presences are adopting new tools such as AI-driven photo search, mobile A/B testing, user-driven optimization, responsive layouts, instant purchase and more. It’s only with the use of innovative tech that traditional retailers can stay relevant in a competitive landscape.
Foot Locker: AR-Driven Demo
David Nuescheler, Adobe Fellow & VP of Enterprise Technology at Adobe Systems, has been working hard in the lab using Adobe Sensei AI to take physical retail to new heights. In his demo featuring Foot Locker products, he painted an augmented reality that may be closer than we think for storefronts.
Using a catalog of cloud-aware assets, David demonstrated how a storefront could render innovative user-facing views in real-time. Using AR, his live demo rendered a custom shoe to appear on an empty shelf. He scanned and identified shoes on a product stand, and overlaid warehouse inventories on top of products in real-time.
Though demonstrated as a futuristic work-in-progress, such AR-infused digital experiences will likely arise in retail soon, adding a sense of magic to typical storefronts.
David Nuescheler, Adobe Fellow & VP of Enterprise Technology at Adobe, demonstrates some cool storefront AR March 26 in Las Vegas. (Jeff Bottari/AP Images for Adobe)
Chegg: From Textbooks to Online Tutoring
Textbook rental and academic platform Chegg also made an appearance at the Adobe Summit keynote. Chegg CEO Dan Rosensweig detailed the company’s initial fear when its IPO plummeted, and the frantic reorganization that followed. If any digital transformation story has a positive end in sight, it’s Chegg.
Now valued at $4.5 billion, Chegg was pressured to make a complete 180-degree pivot from shipping textbooks to embracing online subscription-based services. Though the company still ships about 5 million books a year, Chegg now has 1 million digital users. Academics from high school through to continuing education are leveraging Chegg’s tutoring and online study communities.
Chegg is deeply tapped into its consumer base, placing great emphasis on responding to “who [the customer] is, and not who they used to be.” For Chegg, using consumer analytics to architect the new digital business is a continuous, and necessary, commitment.
Chegg CEO Dan Rosensweig discusses with Adobe CEO Shantanu Narayen March 26 in Las Vegas. (Jeff Bottari/AP Images for Adobe)
Adobe: An Integrated, Impressive Services Catalog
A sure-fire case study in digital transformation is Adobe itself. Adobe CEO Shantanu Narayen kicked off the Adobe Summit 2019, its annual conference full of 17,000 tech-leaning CMOs and digital transformers in attendance.
Shantanu transparently outlined how, years ago, innovation in the Adobe product line had become stagnant. Compared to the infinite growth of content creation across the world, its static products weren’t doing the trick. This began to change rapidly with the introduction of subscription-based formats including Creative Cloud. Again, Adobe had found a foothold.
He demonstrated the company’s new Adobe Experience Platform, a holistic customer experience management platform. With many of the tools having been used as dog food at Adobe itself, the platform unites a large suite of applications dedicated to advancing the customer journey.
Discover, Try, Buy, Use, Renew
Discover, Try, Buy, Use, Renew. Adobe’s creed for customer experience management was reiterated many times at the Summit’s keynote. Such insights ring true in the agile transformation tactics taken by the businesses showcased above.
Adobe Summit—or, as Marketo’s Steve Lucas calls it, “the Woodstock of marketing”—showcased how successful agile companies placed customer experience as the driver for product design. In tandem, the future of retail holds an interesting future that meshes both traditional physical and digital experiences.
With customer experience now the No. 1 driver, should we place a “Mar” in front of the DevOps to create the triumvirate MarDevOps? The dogs are out.
Brick-and mortar-retail is rapidly transforming with the digital age. As physical storefronts turn digital, it’s safe to say that traditional retail is quickly becoming a mesh of “brick-and-click.”
So, how are retail giants surviving in this new landscape? Interestingly enough, the remaining brick-and-mortar retail are reinventing themselves by applying helpful DevOps principles.
In this post, we’ll peek into cases of innovation and horror stories from historically brick-and-mortar retail chains and brands. From Target, to Balian Group, Sears and other companies, we’ll discover the importance of DevOps in this new brick-and-click multi-channel retail environment.
Target
In an effort to increase agility, Target engineering teams embrace something they call “the Dojo”—immersive, six-week training sessions with hands-on DevOps and Agile coaches. As Target CIO Mike McNamara recently told the Enterpriser’s project: “The Dojo has been fantastic in getting teams engaged with agile and DevOps, removing the natural resistance and fear of change, and then supporting the team through the changes while maintaining productivity. It’s been a huge success for Target.”
The result of these sessions is increased comfort with methods including containerization and tools that enable continuous integration and continuous deployment. McNamara also noted their grassroots approach; they encourage open source activity and listen to their engineers when they petition the use of new tech.
“Our engineers have fully embraced DevOps,” he said.
Bailian Group
For another case study, take Bailian Group, a large Shanghai retailer that operates 6,000 grocery and department stores. By the time Bailian entered the online marketplace, it suffered from a severe lack of agility, non-native web apps and underutilized server assets.
Qichuan and Bailian realized that to stay relevant, they had to cut long development cycles and embrace “new cloud application development tools and processes that foster a CI/CD and DevOps culture and increase innovation and time-to-market,” he said.
They ended up implementing a private cloud using OpenStack. Now, midway through their cloud transition, they estimate to have 10 million virtual machines by the end of 2020.
Sears
While the previous two examples had positive outcomes, digital transformation has not gone over well for other retailers. Sears declared bankruptcy in October 2018.
In its final years, Sears was in a sticky situation. To the chagrin of former CEO Eddie Lambert, surveys found their customers preferred to shop in stores as opposed to online. Disregarding customer desires, however, Sears attempted a big data play with its “Shop Your Way” program, under which Sears gathered user data to then sell. The program floundered, resulting in consumer irritation and loss of sales as reported by Business Insider.
Writing for InformationWeek, Emily Johnson reminded us that evaluating DevOps using the wrong metrics will ultimately lead to failure, citing customer satisfaction as the No. 1 goal.
In its final years, an abrasive Sears leadership halted any sense of DevOps to flourish. A disengaged Sears CEO Lambert created “a climate of fear,” according to Busines Insider reports. It appears the retailers that stay afloat encourage agility and intrapreneurship—this means encouraging innovation, not fear.
DevOps Aids the New Omnichannel Retail Benchmark
Brick-and-mortar are evolving into omnichannel retailers that bridge physical and virtual spaces. Take IKEA’s augmented reality app, which visualizes items in a house. From Disney, to Startbucks, Nordstroms and elsewhere, physical storefronts embrace digital offshoots of their core model to augment the traditional retail experience.
Surprisingly, physical storefronts aren’t completely dead. They still offer conveniences for urban environments with high walking scores. See the emergence of Amazon physical locations for ironic evidence of their benefit. Other storefronts and pop-up shops have taken on a museum-like quality. The flagship Funko store, for example, creates impressive displays and interactive exhibits, pulling in flocks of weekend shoppers that want more than a dull e-commerce shopping cart.
DevOpsGroup similarly describes a “reverse showrooming” movement; consumers research products online to then purchase in person. This enables brands such as ASOS, which is emphasizing social media and free worldwide deliveries, to stay competitive.
“Retailers have been forced to re-evaluate their business strategies and understand how new technologies will affect their offering,” according to the DevOpsGroup website.
DevOps Turns Bricks to Clicks
Online shopping has sufficiently disrupted retail, and brick-and-mortar must reinvent themselves to stay relevant. An open-minded leadership and a commitment to fast development, dynamic scaling, uncompromised availability and open source technology underpin a successful evolution.
Of course, this increased reliance on technology necessitates a hardened look at how development and operations communicate, and brick-and-mortar retailers must adopt the behaviors of tech companies to retain agility.
This may come in many forms: new user-facing digital experiences, microservices for composability in niche omnichannel environments, containerization to package code that scales well, distributed data stores, heightened user data security or smart incident management techniques, to name a few.
Traditional retailers, then, should dissuade their doubts against agile. DevOps can aid retail by reducing redundancy, cutting costs and streamlining production and distribution.
Don’t fall for the marketing jargon: Running legacy infrastructure on a cloud instance is not actually cloud computing. Unfortunately, “cloudwashing” is common among enterprises as they attempt to jump on the cloud trend.
So, how do we dispel the hype to distinguish pure cloud software as a service (SaaS), platform as a service (PaaS) and infrastructure as a service (IaaS)? In this article, we’ll define cloud washed and cloud native, citing NIST standard definitions and industry expert opinions. We’ll add some cues to help decipher cloud washed versus cloud native architecture as you search for new tech. No whitewashing here; prepare for some hard truths.
What is Cloud Washing?
In recent years, services that simply work over the internet were rebranded with the “cloud” buzzword, perpetuating a tenuous, vague perception of the cloud.
Cloudwashed solutions are legacy, on-premises software held in a virtualized data center and rebranded as cloud software. Such tools originally were not built for the cloud and do not satisfy the NIST definition of true cloud computing.
While non-local hosting can reduce maintenance and server costs, by adopting cloudwashed services you sacrifice the benefits of cloud native. As Clint Boulton, writing for CIO describes:
“… the implementation doesn’t provide automated, continuous upgrades, let alone the multitenancy, self-service, auto-scaling and usage-metering capabilities that have come to characterize cloud services …”
“Cloud computing is a model for enabling ubiquitous, convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, servers, storage, applications and services) that can be rapidly provisioned and released with minimal management effort or service provider interaction.”
NIST also defines three service models of cloud computing (SaaS, PaaS and IaaS), and specifies four types of deployment models (private, community, public and hybrid).
As you can see, the NIST definition of cloud computing is specific. Authentic cloud services are configurable, providing hands-on optimization. They are elastic, able to provision and support multiple application requirements. Such cloud architectures are able to efficiently scale to the unique computing workloads of various applications.
Furthermore, resources for cloud services are pooled. According to HiTechNectar, “True cloud platforms are built on a true multi-tenant model, with different physical and virtual resources dynamically assigned and reassigned according to consumer demand.”
A cloud-native application (NCA) is thus an application specifically designed for the cloud that satisfies the definition of cloud computing. A true cloud solution would be a proprietary service built on a platform such as CloudStack or OpenStack. Google Cloud, Amazon Web Services and Microsoft Azure are all examples of authentic cloud providers.
Cloudwashing Warning Signs
Cloudwashing is rampant in the ERP realm. As HiTechNectar reports, “Many enterprise software vendors make claims that their applications are built for the cloud when they are not.”
Thus, to avoid cloudwashed software, take heed of these warning signs:
Overuse of “cloud” in vague marketing jargon.
The need for custom integrations (this is against self-service).
No resource pooling.
Lack of developer tooling.
Lack of configurability.
Tooling not planned with redundancy in mind.
Server arrangement is not distributed.
Good Signs of a True Cloud
Thankfully, finding true cloud computing is made possible with the help from NIST, which lists five essential elements of cloud computing:
On-demand self-service.
Broad network access.
Resource pooling.
Rapid elasticity or expansion.
Measured service.
Another good sign is the vendor’s involvement in the Cloud Native Foundation. Companies also dedicated to open source projects tend to be more progressive and transparent.
Dissipating the Foggy Cloud
A recent CompTIA Report found that half of the companies in 2018 have 31 percent to 60 percent of their IT systems in the cloud. Cloud computing is still driving the rise of cutting-edge trends including AI, blockchain and IoT.
Unfortunately, though, not all purported cloud services are authentic. Ruled by an incessant need to stay relevant, tech marketing adopts and overuses certain jargon, and the “cloud” is not immune. Only by understanding the dichotomy between cloud washed and cloud computing can CIOs make educated decisions regarding their tooling choices.
The NIST cloud computing criteria is valuable because NIST is an impartial standards agency. NIST computer scientist Peter Mell stresses the importance of evaporating this fog to reveal the truth:
“… by adopting an authentic cloud, [businesses] are more likely to reap the promised benefits of cloud—cost savings, energy savings, rapid deployment and customer empowerment.”
To avoid being cloudwashed, seek out multi-tenant architectures with self-provisioning, pay-per-use models, virtual instances and linear scalability. Only then will you reap the rewards of an authentic cloud-native environment.
Earlier this year, Tesla’s Cloud was hijacked and used to mine cryptocurrency, exploiting a vulnerability in the company’s Kubernetes cluster. A mountain of FedEx data was recently exposed, affecting 119,000 individuals. The Equifax breach garnered international attention after an estimated 145.5 million Americans were jeopardized. In other news, we’ve reported on how the Vine Docker registry fiasco was hacked, leaving an embarrassing PR trail in its wake.
What is common across all these scenarios? They arguably could have been avoided with basic safeguards underpinned by a healthy dose of DecSecOps. In hindsight, let’s see what specific strategies developers can adopt to avoid such horrendous leaks.
Basic DevSecOps Can Prevent Vulnerabilities
DevSecOps is described as placing security at the forefront of every action. Within the world of cloud tooling, this means instilling protection at every point of the build life cycle. You would think this equates to advanced container and orchestration security, powerful access management and establishing hardened oversight for internal applications, right? Actually, many modern exploits simply involve a lack of basic password protection.
Case Study No. 1: Tesla Cloud Cryptojacking
In mid–2018, Tesla’s Amazon servers were hijacked by malware and used to mine cryptocurrency for rogue agents. Similar cryptojacking has transpired at Gemalto, Aviva and others. According to RedLock’s report, the hackers infiltrated the Kubernetes console, which was not password-protected. In one pod, they found access credentials to Tesla’s AWS environment. The breach exposed sensitive telemetry data, though was limited to “internally used engineering test cars only,” according to Tesla.
How to Prevent
Password protection for Kubernetes administrative consoles is a readily apparent lesson here. All cloud accounts, even if used internally, must be better-equipped and access credentials sequestered. Surely, instilling a “security by design” mentality can help limit the number of such consoles left accessible.
Case Study No. 2: Vine Registry Breach
In 2016 a white hacker by the alias avicoder was able to infiltrate Vine, the video-sharing social network. The hacker used tools to discover subdomains and behind one—https://docker.vineapp.com—found an open Docker private registry that housed the Vine source code. Such a vulnerability could have been used to collect user details or inject malware for malicious purposes. Thankfully, no users were compromised, and the Twitter Bug bounty program awarded avicoder a handsome sum for the discovery.
How to Prevent
In this particular incident, there was nothing at fault with Docker or Docker containers. The service (which was meant to be private) was simply left public and devoid of access controls. The lesson is that URLs on the world wide web are just that—publicly exposed to the world. It does not matter if no documentation links to them; such subdomains are easy to discover with crawlers and will be exploited if found.
Case Study No. 3: Equifax Exploits
In the Equifax breach, millions of customer records were stolen. The Equifax report notes that hackers exploited a “website application vulnerability.” Further reports detailed that the 2017 breach utilized flaws in Apache Struts, likely through a zero-day exploit, according to the Apache Struts Project Management Committee.
How to Prevent
Some note that the deserialization of untrusted data inherent in Apache Struts applications leaves some major vulnerabilities and potentially malicious code execution.
Containers, due to their short lifespan, are not persistent, therefore making consistent hacking harder. They can also isolate functionalities so as to decrease attack vector. Regarding the Equifax breach, Aquasec speculates that container usage could have lessened the impact:
“… it’s likely that such a breach would have been more difficult with containers, and that if successful, it would have been less persistent, not as widespread, and mitigated sooner.”
Some view containers as a formidable force if armed with behavioral analysis and firewalls for network connections. Still, others note flaws in assuming containers are safer.
Regardless of architecture, In DevSecOps, security for customer data isn’t drafted into the background; rather, it is imbued into the entire tooling process and holds equal footing within each build.
Instilling a DevSecOps Culture
Instead of a build-now, secure-later attitude, DevSecOps seeks to elevate security into every decision. With each build, security is melded with greater forethought regarding the repercussions for how data is treated.
Admin consoles for Kubernetes clusters must be armed and password-protected, and the same goes for private GitHub repositories or Docker registries. Any traversal of data through cloud SaaS services must have authorization in place. A most obvious lesson is to password protect those “hidden” registries with discoverable URLs.
Bounty programs for white hackers continue to lead to successful bug detection. At least for these institutions, there is less evidence of compromised user data and remediation is quickened.
Much of this can be distilled by due diligence and through breeding a culture of security. While that’s a little fluffy for some, it apparently still requires repeating for institutions that continually de-privilege these basic safeguards.
Netflix owes a great deal of its exponential growth to its phenomenal tech stack. Throttling its content through a single internal API, the company was able to deliver content agnostic of device type and quickly disrupt its contemporary competitors in the consumer entertainment industry.
We’re intrigued, then, when Netflix exposes some of its internal architecture for others to use. HubCommander is just one recent addition to Netflix’s growing collection of open source tools. It’s a ChatOps tool for GitHub management.
We’ve previously covered an open source approach to ChatOps, tracking how to bring free operational tools into the conversation for increased transparency and efficiency. In this article, we’ll resume this research by learning how Netflix has utilized HubCommander and delve into its open source iteration to see how others might leverage it to automate its GitHub organization.
Problems
Managing many GitHub repositories while working in a large organization can be frustrating; especially difficult is the task of managing permission levels between various user subsets. Not only are various user permission levels difficult to trace, but there is an intrinsic security problem to address as well, especially when working with external teams.
“One of the biggest challenges with using GitHub organizations is user management.” — Netflix
Project managers need to administer different organizations, and GitHub requires admin privileges to manage such repository settings; if onboarding thousands of employees, editing this manually could be an extremely tedious job.
Benefits!
At Netflix, HubCommander standardizes its user management approach, enabling automation for GitHub repo permissions across development teams.
“Management of many users on GitHub can be a challenge without tooling. We needed to provide enhanced security capabilities while maintaining developer agility. As such, we created HubCommander to provide these capabilities in a method optimized for Netflix.”
HubCommander thereby provides this missing link, granting admins the ability to quickly grant privileges. Easily to embeddable into chat, the HubCommander is made with ChatOps in mind. Therefore, privileged GitHub organization management tasks can be performed from the Slack channel, “without granting administrative or owner privileges to your GitHub organization members.”
Using HubCommander
To use HubCommander, first, you must have Python 3.5+, Slack, a GitHub organization, and a GitHub bot user with ownership level privileges.
Next, Slack channels with HubCommander installed can make the call !help to return a list of bot commands:
Source: Netflix
To create a repo, for example, you can simply use the command !CreateRepo:
Source: Netflix
Utilizing HubCommander also has security benefits as it reduces the number of GitHub API permissions granted per user—reducing attack vectors inherently increases security. Another point on security is HubCommander’s integration with Duo for additional authentication:
Granting outside collaborators specific permissions to repositories
Repository default branch modification
Repository PR listing
Repository deploy Key listing/creation/deletion
Repository topics creation/deletion
Repository branch protection enabling/disabling
Enable Travis CI on a GitHub repo
Safeguard commands with 2FA via Duo
Automating GitHub With ChatOps
If your code resides in multiple Git repositories, like Netflix, having a method to configure permissions across all entities in a convenient way becomes a necessity. With a bot to perform operational tasks on GitHub repositories, Netflix is benefiting from automatic manipulation and a wider team access, adding to its agile goals:
“The reduction in administrative overhead has significantly simplified our open source efforts.”
Netflix operates with three development channels (a core Netflix OSS, Spinnmaker and a skunkworks project for new ideas). Thus, having a consistent permissions model across organizations can decrease friction. Also, since Netflix collaborates with third parties, such as contractors or other open source contributors, HubCommander also acts as an effective way to maintain security.
Final Thoughts
Netflix recognizes a growing popularity in the ChatOps approach and cites other benefits such as team transparency, a timestamp for each event and a self-service nature—all reasons for adopting a ChatOps strategy into its developer tooling.
HubCommander appears like a good ChatOps tool for GitHub repository management. However, GitHub Actions (still in beta at the time of writing) is a new feature that will aid development workflows, offering automation for the GitHub operational duties themselves. This may spark alternative bots to be developed, especially in terms of workflows that respond to events.
For now, with open source utilities such as HubCommander, certain GitHub duties can be quickened, stimulating a culture of more efficient management to decrease redundant tasks. While shorthand for GitHub repository management might not seem at first glance as a priority, it could scale down operational headaches tremendously.
ChatOps is an exciting proposition for organizational Ops communication. Throwing development and operational tooling into the chat conversation can ease the development pipeline, increase transparency and increase security by streamlining bug detection and remediation efforts.
At the heart of ChatOps are bots, scripts and chat clients; many are pre-built and open source. In this article, we’ll review a completely open source method for building out your ChatOps setup. We’ll cover integrating with Hubot, using CoffeeScript commands, and how building custom scripts can ignite ChatOps in your own environment.
Why Open Source for ChatOps?
Many arguments have been given as to the benefits of open source. Building upon open source inherently evades vendor lock-in, enabling more security and service longevity. Furthermore, to avoid reinventing the wheel, forking existing open source ChatOps projects naturally will be more efficient than purely greenfield initiatives.
Open Source Chatbots
There are many Chatbots available for use; each with unique capabilities, varying types of integrations out of the -box, and different ways to extend functionality with custom scripts. Take Lita, for example, an open source chatbot built with Ruby. It works with IRC, HipChat and Campfire. Many other open source chatbot options exist, such as Errbot, Jarvis and Lazlo.
Example: Hubot
By far the most adopted open source ChatOps companion is Hubot. Designed and open sourced by Github, Hubot takes market share in the ChatOps discussion. Described as “a customizable, life betterment robot,” Hubot is written in Node.js and takes commands with CoffeeScript.
Using Hubot, developers can initiate commands with Hubot scripts to perform actions such as post images, translate languages and integrate with maps, but also initiate vital commands to your favorite DevOps cloud tools. A full catalog of scripts can be found here, and it’s quite easy to craft your own CoffeeScript commands as well.
Example Hubot Scripts
The community scripts that support Hubot are similarly open source, meaning that you can repackage these implementations to suit your own needs. Let’s take a look through the catalog of Github-supported Hubot scripts to see what sort of pre-constructed extensions we can seamlessly add to a Hubot environment.
With Hubot, you can ping your major cloud development platforms, such as Heroku. Some example commands for returning status of issues on Heroku include:
hubot heroku status - Returns the current Heroku status for app operations and tools
hubot heroku status issues <limit> - Returns a list of recent <limit> issues (default limit is 5)
hubot heroku status issue <id> - Returns a single issue by ID number
Or perhaps you want to conveniently check PagerDuty from your chat client to see who is on call within your incident response team, or to gather details on current incidents. Hubout has a script for that. Using pagerduty.coffee, some example requests might be:
hubot who's on call - return the username of who's on call
hubot pager me trigger <msg> - create a new incident with <msg>
hubot pager me incidents - return the current incidents
hubot pager me notes <incident> - show notes for incident #<incident>
hubot pager me problems - return all open incidents
Another example could be direct interaction with the Twilio API for sending SMS. Using sms.coffee you can initiate a command with the following syntax:
hubot sms <to> <message> - Sends <message> to the number <to>
Configuring sms.coffee is easy, all you need to insert are:
Aside from using third party scripts, you can always craft your own. To be an executable Hubot script, your code must be in .coffee or .js, and it must be in the correct directory (src/scripts and scripts by default). It must also export a function, as in the following:
module.exports = (robot) ->
# your code here
Essentially, Hubot listens to all chat communication; it will hear commands and respond accordingly. To configure what it must hear in regards to your script, you must specify what to listen for, and what to respond with. For example:
module.exports = (robot) ->
robot.hear /badger/i, (res) ->
# your code here
robot.respond /open the pod bay doors/i, (res) ->
# your code here
For further details, see the documentation on Scripting here.
Getting The Most of Chatbots
Once integrated into a chat platform such as Slack, your chatbot configuration can truly thrive. Given the wide breadth of community generated and Github-maintained scripts, DevOps teams have near unlimited potential for uniting their tooling into a single, human-readable CLI. Another benefit of choosing open source tools for such projects is that you can detach your bot and integrate with whatever chat client you prefer.
ChatOps can add fun and efficiency to the remote workspace. We’ve covered high-level methods for adopting this into your team’s space, and hopefully, a helpful foundation into the power of open source for streamlining the power of ChatOps.
