Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.
This week: Warren Buffett asks too much for wind energy, it’s “completely ridiculous” to say Google’s chatbot LaMDA is sentient, and Microsoft finally kills Internet Explorer. (more…)
Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.
This week: Apple lets devs increase subscriptions “without consent,” Twitter has more than 19% spam accounts, and we dig into the claim that Microsoft is about to double salaries. (more…)
DevSecOps is everywhere, and chances are your organization is shifting toward this convergence of development, security and operations. In an on-premises environment, you build your own DevSecOps process by mixing and matching existing and new tools. However, in the cloud, much of the environment is determined by your cloud provider. Microsoft’s Azure cloud is known for its robust security capabilities. Specifically, Microsoft has developed a set of capabilities and services that can help you ease into a DevSecOps workflow without much effort. The tools are pre-integrated and support everything from CI/CD to identity management. Let’s see how they work.
Security is an important part of DevOps. But it can be challenging to ensure that software developed in a CI/CD process is really secure.
DevSecOps is an organizational pattern that can make this possible, even at high development velocity. It is an approach the DevOps team can use to prevent security issues from the outset of the development life cycle, ensuring their software is not only high quality but also secure and resilient to failure.
DevSecOps is a discipline that combines application development, security, operations and infrastructure-as-code (IaC) with an automated continuous delivery cycle. Here are three practices that can help a team move towards a DevSecOps process:
The Azure cloud provides several built-in capabilities that make it possible to adopt a DevSecOps process.
Azure DevOps provides developer services that enable teams to plan tasks, collaborate on code development and build and deploy applications. Azure DevOps supports a collaborative culture and process in which developers, project managers and contributors work together to develop software. You can use Azure DevOps Services in the cloud or Azure DevOps Server on-premises.
You can integrate Azure Pipelines, the Azure cloud’s CI/CD solution, with your Kubernetes clusters. Use the same YAML document to build multi-stage CI/CD pipelines.
Azure Pipelines lets you track metadata, such as commit hashes and issue numbers, from Azure Boards into container images. This provides direct traceability from any security issue to a specific change made in development. It also provides clear, easy-to-read documentation that can improve the feedback loop between development, operations and security teams.
When developing Kubernetes applications, there is a need to test applications locally and understand how they interact with dependent services. You may need to develop and test multiple services in collaboration with other developers or teams.
Azure provides Bridge to Kubernetes, which lets you run and debug code on your development machine while connecting to your Kubernetes cluster. You can test your code end-to-end, set breakpoints on code running on the cluster and share your development cluster among team members. This makes it possible to test and resolve Kubernetes security issues in a realistic environment before deploying to production.
The Microsoft Identity Platform takes the Azure Active Directory (Azure AD) developer platform one step further. It allows applications to accept logins from any Microsoft identity, and obtain tokens that can be used to call Microsoft APIs or APIs created by other developers. This creates a large, interoperable ecosystem.
Azure Active Directory B2C provides B2C identity services—customers get single sign-on (SSO) access to applications and APIs using their preferred social, enterprise or local account ID. Another option is to integrate Azure AD with on-premises Active Directory for hybrid and Azure migration scenarios.
You can also use Azure Role-Based Access Control (RBAC) to manage access to cloud resources. RBAC lets you manage who can access your Azure resources, what they can do with them, and which regions can access them.
You can also use the Microsoft Identity Platform to protect DevOps tools themselves, including native support for Azure DevOps and integration with GitHub Enterprise.
Exposed secrets are a severe security issue that is very common in modern applications. Azure Key Vault lets you manage the distribution of secrets by storing them centrally in Azure Key Vault. Key Vault greatly reduces the chance of accidentally disclosing secrets. With Key Vault, application developers no longer need to store credentials or other sensitive information in their application code.
Azure Policy lets you specify a default allowed configuration that is automatically applied to all cloud resources. This can avoid misconfigurations that violate security policies. Azure Policy works on the basis of desired state configuration (also known as declarative configuration), letting you specify to what degree resources and services should be secured and whether to alert or block/modify deployments in Azure if they don’t meet the policy.
When using Azure in multi-tenant mode, policies can be applied at the management group, subscription or resource group level. You can define specific policies and enforce compliance requirements for test, staging, and production environments.
The Azure Security Center (ASC) and Advisor can also be easily integrated into the DevOps process. ASC makes it possible to augment event streams in Azure with threat intelligence or analysis by third-party security tools.
Penetration testing is the recommended method for examining your environment for infrastructure or application configuration vulnerabilities that could create vulnerabilities that an attacker could exploit.
An Azure penetration test should check for vulnerabilities on endpoints, apply fuzzing (malformed input) to discover business logic vulnerabilities, and perform port scanning to identify network vulnerabilities.
Microsoft provides specific guidance for penetration testing in Azure with recommended products and penetration testing service providers.
To ensure security for cloud subscriptions and resource configuration across multiple subscriptions, you can use the tenant security solution part of the Secure DevOps Kit for Azure (AzSK).
In addition, you can leverage Microsoft security technologies like Microsoft Defender for Cloud and Microsoft Sentinel. These solutions provide monitoring and security features designed to detect and alert on unusual events or configurations that require investigation and possible remediation.
In this article, I introduced DevSecOps and showed how the Azure cloud provides a range of services that let you easily implement DevSecOps practices. These include:
I hope this will be useful as you take your first steps to practice DevSecOps in the cloud.
Microsoft’s Azure DevOps Server is emerging as one of the most popular cloud application development environments. Like most continuous integration/continuous delivery (CI/CD) platforms, it provides features such as version control, reporting, project management, automated builds, and testing. It covers all aspects of the application life cycle and enables DevOps. Here’s a closer look at this cloud computing environment.
Azure DevOps provides a number of different developer services that allow DevOps teams to plan work, collaborate on code development and build and deploy applications. It is highly collaborative and brings together developers, project managers and individual contributors to develop software. It allows organizations to create and improve products much faster than with traditional software development approaches.
Teams can work in the cloud using Azure DevOps Services or on-premises using Azure DevOps Server.
The cloud offering, Azure DevOps Services, is a scalable, reliable and globally available hosted service. It’s backed by a 99.9% SLA, is monitored by Microsoft’s 24/7 operations team and is available in local data centers around the world.
The on-premises offering, Azure DevOps Server, is built on a SQL Server back end. The on-premises version is preferable when organizations need their data to stay within their network or when they want access to SQL Server reporting services that integrate with Azure DevOps Server data and tools.
Although both offerings provide the same essential services, Azure DevOps Services offers some added benefits:
There are also some other key differences depending on your organization’s or your team’s needs. These can also be helpful if you’re migrating from on-premises to the cloud:
Although Azure DevOps Services is a hosted version of Azure DevOps Server, there are some feature differences. Some Server features aren’t supported in Services; for example, Azure DevOps Services doesn’t integrate with SQL Server Analysis Services to support reporting. Process customization and reporting also differ in their support.
Azure DevOps provides integrated features that can be accessed through a web browser or an IDE client. One or more of the following standalone services can be used based on your organization’s business needs:
Use of the following collaboration tools is also supported:
Azure DevOps supports adding extensions and integrating with other popular services, such as Campfire, Slack, Trello, UserVoice and more. In addition, DevOps teams can also develop their own custom extensions.
Azure DevOps Services supports integration with GitHub and GitHub Enterprise Server code repositories. Azure DevOps Server supports integration with GitHub Enterprise Server repositories.
When you sign up for Azure DevOps, you must first register an organization, project and version control type. DevOps teams can create multiple projects and differentiate them with separate names and access permissions. Projects can be set to public or private. Teams can also use Azure Boards for collaboration and project management, and can also configure use of Agile practices. After setting up your team, you can use the platform to begin deploying your applications to Azure.
In addition to a collaborative environment, Azure DevOps provides access to an Azure Pipeline service that supports multiple languages and platforms. The Azure DevOps pipeline is a set of services that are grouped into individual projects.
Additionally, Azure Test Plans feature facilitates automated testing. It is also possible to set up deployment pipelines with different components for multiple applications.
Azure Pipeline can track work status, manage backlogs and create custom reports. With the help of the Azure DevOps pipeline, developers can create, manage and deploy their projects in a single environment. With this solution, DevOps teams will have a single point of contact and can communicate with each other.
You can find plenty of high-level definitions out on the internet about what site reliability engineering means and what site reliability engineers do.
But if you want to understand what it’s actually like to work as a site reliability engineer, there is perhaps no better source than job descriptions. SRE job ads explain what real companies are looking for and what they expect on a day-to-day basis.
To provide insight on what SRE looks like in practice, we’ll look at some job ads from several leading companies. As we’ll see, each company has a slightly different take on what the site reliability engineer role entails, but it’s possible to identify some core themes that underline what businesses expect.
To understand the content of SRE job descriptions, let’s examine examples from four leading companies.
GitLab’s SRE job description contains most of what you would expect from a typical site reliability engineer job ad. It highlights the importance of having a mix of technical skills and collaboration skills. It also emphasizes expertise in areas like Unix/Linux, infrastructure-as-code and Kubernetes.
What may be slightly surprising, however, is the emphasis that GitLab’s job description places on programming skills. GitLab expects SREs to know how to code in specific languages, like Ruby and Go. The takeaway is that SREs need to know more than just how to use code to solve IT operations problems. They must also be able to develop applications or contribute code to application platforms (the job description mentions “contributing to code in GitLab” as a required SRE technical skill).
Also interesting is the fact that GitLab expects site reliability engineers to be able to write blog posts. Blogging may not be one of the first skills that come to mind when you think of a site reliability engineer’s responsibilities, but at GitLab, at least, it’s part of the job.
LinkedIn provides an SRE job ad template that businesses can use when posting their own SRE jobs on the site.
LinkedIn’s ad is pretty high-level (which is unsurprising, given that it’s a template rather than an ad for a particular job). For the most part, it doesn’t mention specific tools—although the main exception is programming languages, where it says SREs should know how to code in languages like C, Python, JavaScript and Ruby. Like GitLab, then, LinkedIn clearly sees development as a core SRE job responsibility.
The LinkedIn job description also places a heavy emphasis on the ability to “gather and analyze metrics.” It doesn’t use the word “observability,” but that seems to be what LinkedIn is getting at.
Continuing the theme of development-centric SRE work, a recent job ad for a specific SRE opening at Microsoft reads a lot like a DevOps engineer job description. The ability to “design, write and deliver software” is the top responsibility listed in the ad, which also mentions familiarity with CI/CD tools.
The Microsoft job description also mentions the importance of written and verbal communication skills. Unlike GitLab’s ad, this one doesn’t say SREs will need to write blog posts; still, it’s clear that companies expect people in this role to be creative communicators in addition to technical experts.
It’s worth noting, too, that Microsoft wants site reliability engineers who know both Windows and Unix. If you need further proof that Unix skills are absolutely central to SRE work, look no further than this job ad from the company that created Windows.
Even Amazon—a company that is more of a platform provider than a software vendor—places major emphasis on the ability of SREs to code in one of its recent SRE job ads.
Interestingly, the Amazon job also highlights networking skills as a major qualification. That’s understandable given the complexity of networking in the AWS cloud. But it’s a bit surprising because most other site reliability engineer job ads don’t call out networking expertise as a key skill—despite the fact that network engineering is critical to SRE work.
The Amazon job description mentions the importance of an applicant’s willingness to be on-call, too. We suspect that’s critical in virtually any SRE job, but at least Amazon is upfront about the fact that, at time, you’ll be woken up at 4 a.m. to manage incidents.
There are a few key takeaways from these various SRE job descriptions.
This is not to say that all SRE roles are identical or that all companies expect the same things from their SREs, of course. But the jobs ads or templates we’ve surveyed do highlight how the high-level understanding of SRE translates to on-the-ground reality at actual companies.
Simple, safe, reliable software installs and uninstalls are always best, but getting there isn’t easy. Applications break each other—or even the operating system itself.
MSIX, Microsoft’s packaging format, is the newest replacement for the decades-old Microsoft Software Installer (MSI) and setup exe installer technologies. MSIX is gaining acceptance and wider adoption because of the benefits it delivers: security, ease of use and reliability in application installations.
MSIX provides a new, modern runtime environment intended to protect both your application(s) and end user systems. Enterprise users prefer the MSIX format because it provides the most appropriate framework to modernize applications; simultaneously reducing IT departments’ workloads when preparing application package(s) for delivery to hundreds or thousands of end users.
As developers who build Windows desktop applications embrace the concept of running code in an MSIX container, understanding certain key concepts can streamline and strengthen their application installation experience.
Running code in an MSIX container can solve common problems encountered by developers coming from non-containerized deployments. This includes products built and delivered as a Win32, .Net Framework or Universal Windows Platform (UWP) packages—or when starting a new project from scratch.
In addition to providing clean installation and uninstall processes, MSIX enables enterprise customization in independent packages via modification packages. It also supports the ability to deploy applications through enterprise tools (e.g., Microsoft Endpoint manager) or to the cloud (Azure Virtual Desktop, MSIX app attach).
MSIX improves upon the more restrictive model of UWP while still running in a container. It allows developers to code in their preferred way (e.g., unmanaged code using Windows Forms, calling the Win32APIs or managed code using XAML and .NET framework versions of the APIs).
Some tactics must be changed to run inside the MSIX container, however. In addition to minimizing work, it facilitates building modern IT ecosystems, deploying apps and updates faster and more frequently and modernizing applications into the new world of containerized packages.
MSIX also supports deploying applications via enterprise tools (e.g., Microsoft Endpoint Manager) or even to the cloud.
By delivering a more streamlined installation experience for users, MSIX offers benefits to customer and end users, including:
MSIX projects require a different approach than what many developers are accustomed to. To provide customers with the best package file(s) possible, consider the following elements as part of the project. Doing so will strengthen additional project activities, project setup, testing and debugging and supported schemas.
While MSIX continues to evolve, so can your projects. The end results will be application packages that not only streamline work for IT departments but also reduce headaches for end users.
Following its spinout from IBM, Kyndryl has allied with Microsoft to increase the scope of IT services it provides on the Azure public cloud.
Stephen Leonard, leader of global alliances and partnerships for Kyndryl, said that as part of this initiative, Kyndryl will be expanding the scope of the services it provides to both organizations building and deploying applications as well as those looking to migrate existing applications to the Microsoft Azure cloud.
Over time, Kyndryl will expand its cloud services to include additional public clouds as part of a larger effort to enable organizations to embrace a hybrid cloud approach to enterprise IT that spans multiple on-premises and cloud computing environments.
In the meantime, as part of the first partnership established by Kyndryl, the two companies are committing to providing services to address data modernization and governance, security and application workloads infused with artificial intelligence (AI) capabilities. The two companies also plan to establish a co-innovation lab. Microsoft is also committing to establishing a Kyndryl University for Microsoft.
Leonard said given the overall shortage of IT expertise, more organizations than ever are relying on external IT services to, for example, accelerate application development. Kyndryl plans to work with those organizations to best evaluate where specific classes of application workloads might best be deployed, he noted.
It’s become increasingly clear that more organizations are attempting to integrate third-party IT services within DevOps workflows. In some cases, that means standardizing on an opinionated DevOps workflow defined by the IT services provider. In other cases, the IT services provider is focused on a specific task such as securely provisioning infrastructure on behalf of developers.
Regardless of the approach, however, DevOps workflows are evolving into a team sport made up of both internal and external IT professionals that are trying to bridge the cultural divide that always exists between different organizations. Which IT services provider organizations ultimately select often has as much to do with how flexible they are from a cultural perspective as much as it does their technical expertise.
For its part, Microsoft is counting on IT services providers such as Kyndryl to increase the total number of workloads running on its cloud. Now that Kyndryl is independent of IBM, it is free to establish partnerships that previously would have been more complex to navigate.
Of course, it’s up to each organization to determine the degree to which they prefer to work with a larger IT services provider versus a smaller rival that may specialize in one specific area or that is simply a better fit for a smaller company. The one thing that is certain is that as IT continues to become more complex, it’s not feasible for organizations to hire all the IT talent they require on a full-time basis. Organizations will need to carefully evaluate when it makes more sense to consume certain IT services as a function provided by someone else versus attempting to hire and retain talent to maintain that service themselves.
Google this week agitated the bulk of the open source community by launching the Open Usage Commons consortium to protect the trademarks of open source projects, starting with the Istio, Angular and Gerrit projects it launched.
Launched in collaboration with SADA Systems and some independent contributors, the stated goal of the Open Usage Commons is to help open source projects assert and manage their project identity through programs specific to trademark management and conformance testing.
Istio is a service mesh that runs on top of Kubernetes, while Angular is a web application framework. Gerrit is a code collaboration tool.
Unlike previous open source projects that Google played a major role in developing such as Kubernetes, Google has yet to contribute these three projects to an open source consortium. That decision has members of the open source community now questioning Google’s commitment to open source software.
In fact, IBM, which launched the Istio project in collaboration with Google, this week said the Open Usage Commons does not meet the level of open source governance IBM expects. Originally, IBM said the expectation was that Istio would be contributed to the CNCF once it achieved a level of maturity. That contribution would include Istio intellectual property and the trademark. Now IBM is encouraging Google to reconsider its Open Usage Commons approach.
If Google continues to hold on to the intellectual property rights for open source projects such as Istio, those projects will become little more than a community project that Google exercises control over, said Priyanka Sharma, general manager of the Cloud Native Computing Foundation (CNCF), which oversees the ongoing development of Kubernetes, among other open source projects.
“It’s not in the spirit of open source,” said Sharma. “I don’t think you can even call it open source.”
Other open source initiatives will be hesitant to build on top of projects such as Istio if the intellectual property belongs to a single vendor, added Chip Childers, executive director of the Cloud Foundry Foundation (CFF). The trademark is never the main issue, he said.
“The trademark is only part of the consideration,” said Childers.
In the case of Istio, the decision to launch Open Usage Commons might also draw more attention to a Service Mesh Interface (SMI) project being advanced under the auspices of the CNCF. That interface promises to make it easier to rip and replace one service mesh for another.
In the meantime, Google may be trying to maximize its investments in open source projects that until now have tended to benefit rivals more. In terms of consumption of cloud services based on Kubernetes, Google is a distant third at best to Amazon Web Services and Microsoft.
In general, there are more open source projects than ever. However, some of the entities that launch those projects have complained that cloud service providers simply turn those projects into cloud services in a way that consumes most of the revenue opportunities any open source project might generate.
Google now appears to be exacerbating those tensions further by trying to retain control over the open source project it leads, while selling services based on other open source software projects. It’s not clear to what degree this rift might drive a wedge between multiple open source factions. Google with this move may just drive more members of the open source community into the arms of AWS and Microsoft. However, for the moment at least, none of the projects covered by the Open Usage Commons trademarks are so critical that they could not be replaced by any one of several alternative projects that are truly open source.
42Crunch this week announced it has extended an existing alliance with Microsoft to now include support for Microsoft Azure Pipelines for its REST API Static Security Testing Tool. Previously, 42Crunch only provided support for the Microsoft Visual Studio integrated development environment (IDE).
Dmitry Sotnikov, chief product and marketing officer for 42Crunch, said support for the Microsoft IDE assumes that developers will run security tests themselves. By integrating its API security testing tool with the Microsoft continuous integration/continuous delivery (CI/CD) platform, it now becomes possible to make testing APIs for security vulnerabilities part of a DevOps process, he said.
That approach means organizations don’t have to rely on developers to remember to run a security test on their APIs. Instead, security testing of APIs becomes just another gate in the CI/CD process that gets automatically conducted, he noted.
The company has already integrated its API security testing tool with other CI/CD platforms at the request of individual customers. 42Crunch will be moving to make those integrations commercially available in the months ahead, said Sotnikov.
As IT organizations have become more adept at thwarting attacks aimed at end users, cybercriminals are now increasingly focusing their efforts on APIs. Akamai, a provider of content delivery networks, recently reported that from December 2017 through November 2019 it observed 85,422,079,109 credential abuse attacks across its customer base. Nearly 20%, or 16,557,875,875, of those attacks were against hostnames that were clearly identified as API endpoints. Many of those attacks involve, for example, unexpected injections of values to a call made to an API, noted Sotnikov.
The REST API Static Security Testing Tool can locate API contract files in the repository and run more than 200 security checks based on OpenAPI standard requirements, authentication, authorization and data validation. This makes sure that no new or changed API be deployed in production without meeting those standards.
Issues discovered are also prioritized by impact and each issue is accompanied by a knowledge base article explaining the issue, its severity, exploit scenario and suggested ways to fix it.
Sotnikov said the REST API Static Security Testing Tool provides a tangible means for DevOps teams to embrace best DevSecOps processes. Too much of what passes for DevSecOps today amounts to little more than attempting to blame developers for cybersecurity issues, he noted, adding organizations that want to embrace DevSecOps in a meaningful way need to put tools into the hands of DevOps teams that enable them to address vulnerabilities once they are discovered and prioritized.
It’s also critical to provide those tools within the context of existing DevOps processes; otherwise, developers are likely to resist changes to their existing workflow processes, Sotnikov said.
It remains to be seen how DevOps and cybersecurity teams will meld their respective cultures to drive DevSecOps. It is likely cybersecurity teams will continue to define controls and discover vulnerabilities. However, as DevOps teams continue to make security a natural extension of the quality assurance process associated with building any application, the number of vulnerabilities being discovered after an application is deployed in production should decline steadily.
Microsoft scored a huge victory by sweeping in and taking the $10 billion dollar, multi-year JEDI contract for military IT services from AWS (owned and operated by amazon.com, its parent company). The backstory on this is that James “Mad Dog” Mattis, then Secretary of Defense, met with Jeff Bezos who tweeted about it on August 10, 2017 and it appears that Mattis decided to structure a request for IT services (now known as the JEDI contract) that all but led to AWS being crowned the winner by the way the bid was structured.
However, this was not to be. Not only was Mattis gone by January of 2019, but President Trump and Jeff Bezos were having public squabbling matches regarding coverage of the Trump administration in the Washington Post (which Jeff Bezos had purchased in 2013). So tenuous is their relationship that Trump mocked Bezos on Twitter regarding his divorce which was precipitated by the leaked private messages and pictures between Mr. Bezos and his mistress.
What does this all have to do with JEDI? Quite a bit, actually. Amazon has won a perhaps Pyrrhic victory by getting an injunction and getting any work to stop on the process of implementing the JEDI contract on Microsoft’s cloud. Amazon is posting a $42 million bond to cover any costs the government or Microsoft might incur if it turns out that this stoppage is unjustified—not small beans, but not a lot of money to either of these tech titans.
Which way does this jump ball go now? It seems like Microsoft would still have the upper hand, because they as well as the U.S. government would have contemplated this outcome when they decided to switch to a Microsoft solution at what seems like the last minute.
Amazon makes a compelling case as well: Because of the public animosity between two feuding billionaires, one a president, and one the richest man in America, government officials would be afraid to award a contract to Amazon’s AWS because they would suffer the wrath of Trump and his officials. The Trump administration has made it no secret that there is a revolving door for people who don’t get with the Trump administration and its policies. Sounds like a win for Amazon, right?
At the same time, Jeff Bezos’s leaked text and pictures between his mistress bring up a larger question: How can the U.S. government trust some of its most precious data when the CEO not only leaks such personal information, but it is so public that Mr. Bezos himself discussed what he considered ransom demands from a tabloid that threatened to cover it on a blog post on Twitter. Even worse, supposedly the pictures and texts were obtained by the leader of Saudi Arabia, a foreign country, through a bug in WhatsApp that allowed installation of malware on Jeff’s phone that allowed all the data from his phone to be siphoned off to a foreign entity. It’s difficult for AWS to say they are the best solution when their own CEO admits that his most personal data (a secret affair with his mistress) was stolen. What would happen to real military secrets if Amazon’s CEO can’t protect his own?
The fallout from all this is what you’d expect. More arm waving, conjecture and confusion. The reality is this opens up the field to other players such as Google and Oracle. At the same time, there are hundreds of other clouds that may be able to make a case that they are a best fit for certain applications since they may specialize in specific compliances, such as SOX, PCI or HIPAA.
Ultimately, this calls into question whether the government should be putting all its cloud eggs in one basket. The longer-term solution is that the government needs to dictate a series of APIs that its developers and contractors use, and that all compliant clouds would be required to support. This would alleviate the problem of centralizing resources on a single vendor’s solution.
In the medium term, lots of fireworks, lots of drama and most likely Microsoft, the original software empire, striking back and keeping the JEDI contract—one that was Amazon’s to lose. But the story isn’t written so we all need to keep watching for the return of the JEDI to see where this finally ends up.