Tag: google

  • DevOps Chats: 1-Click Vulnerability Scanning on GCP, With Qualys

    DevOps Chats: 1-Click Vulnerability Scanning on GCP, With Qualys

    Sometimes the best way to accomplish something is to choose a path requiring the least friction, or amount of change.

    Qualys customers now have that path available to them in bring vulnerability scanning into Google Cloud Platform. Qualys’ recent announcement means a one-click configuration change enables vulnerability scans in GCP with the results appearing in both Qualys Cloud Center and GCP Security Command Center.

    Join me on this DevOps Chats to explore this announcement with Sumedh Thakar, Qualys President and Chief Product Officer. We discuss Qualys’ and Google’s collaboration and how this benefits DevOps teams.

    Transcript

    Mitch Ashley: Hi, everyone. This is Mitch Ashley with devopsy.kinsta.cloud and you’re listening to another DevOps Chat podcast. Today I’m joined by Sumedh Thakar who’s President and Chief Product Officer with Qualys. We’re talking about an important announcement that just happened with Qualys and with Google about security natively embedded within Google. Welcome to the podcast, Sumedh.

    Sumedh Thakar: Hey, Mitch. Thank you for having me.

    Ashley: Great to have you. Well, let’s start by just having you introduce yourself and, of course, as President and Chief Product Officer we can imagine what you might do at Qualys, but go ahead and tell us a little bit about what you do.

    Thakar: Yes, I’m not sure you can imagine because I’ve been here for 16 years, so I’ve done a lot.

    Ashley: Okay. [Laughs]

    Thakar: Today, as President and Chief Product Officer, I’m responsible for product strategy and implementation of Qualys product and where we have come from where we were many years, mainly focused on just vulnerability assessment. So today I have the engineering product management support operations team as well as the sales team and that kind of gives a pretty unique perspective because of the engineering and the DevOps teams within Qualys. Also, part of the organization for me helps me to really get a very good understanding of today what are the needs for DevOps and what evolution we’re going through with the DevOps team internally within Qualys, and being a massive SaaS platform ourselves I think that that’s something that I’d really like to focus on when we develop security solutions for DevOps teams.

    Ashley: Well, it does give you a very hands-on perspective, both _____ as well as living with DevOps in your own products internally. It’s great.

    Thakar: Exactly.

    Ashley: It’s good for you. Great role to have. Well, let’s dive into this announcement. So, security that’s natively embedded within Google – Google Cloud I think we’re talking about.

    Thakar: Right.

    Ashley: Tell us a little bit about what it is.

    Thakar: Yeah, we’ve had a fantastic partnership with the Google team and, you know, I think as digital transformation, moving things into cloud container and DevOps have been really getting traction for us to _____ differently than what we have done in the past, which has always been after the fact, adding security solutions. So, today with DevOps and DevSecOps the opportunity to embed security upfront into the infrastructure in an automated way so that you can ensure that you’re not missing – you know, that’s always the biggest issue with security is, oh yeah, I did all of that, but then I missed that one thing because I didn’t know that existed.

    So today, Google Cloud has a fantastic solution that really allows DevOps team, as they’re pushing code continuously into the cloud, to have the ability with this new Qualys integration to automatically have that capability of assessment of the security of that virtual machine embedded directly and done transparently so that the end user does not really have to go about installing agents and consoles and all of that. It’s done; completely embedded in the platform.

    Ashley: Excellent. So interesting, and we’d love to dive into this some more, because you’re talking about not bolting things on after the fact. You know, this is the whole shift left idea within DevOps.

    Thakar: Yes.

    Ashley: So, I imagine with the Qualys cloud agent for Google that’s something that you’re embedding within the software platforms that developers test production environments you’re using.

    Thakar: Yes.

    Ashley: Is it that kind of a lifecycle?

    Thakar: Yes, exactly. So, developers today use that solution to first of all ensure that in their – whether it’s Jenkins or whatever it is that they don’t even push solutions out to production that already have vulnerabilities, right? The unfortunate side effect of doing this bolting on after the fact is that the moment you spin up a new machine it already has so many vulnerabilities in the last two years, you spend all your time trying to patch them. So, this DevOps pipeline does give us the opportunity to ensure that first of all in your build process you already eliminate and patch your images so that you don’t go out with images that are already vulnerable, and _____ customers absolutely use the Qualys agent and scanning to really achieve that.

    But then once it gets pushed out to product you also want to have a monitoring to ensure that somebody doesn’t go to an S3 bucket or to a Google storage unit to pull down a vulnerable version of some sort of a software in their virtual image and create new vulnerabilities, so you also need to have that monitoring. And so that native embedding ensures that once you have done your DevOps process and cleaned the image, as the image is being pushed into the production environment in the cloud, in Google cloud, it is going to have that monitoring capability already embedded transparently without having to do additional effort.

    Ashley: Great. I imagine it’s not a heavy lift for developers to install this in the development test environment, right?

    Thakar: Yeah, and that’s the beauty of it is everything is automated, right? So, once you write the scripting it’s not a process that requires you to run commands every time and do all of that. Once you push that into the CD process then it’s really transparent, because in this case the developers don’t even need to run any command to get the agent onboarded. It’s an option that they check in the Google security center and then every single image that spins up will have that sort of one-click integration and automatically embedded in there.

    Ashley: Great. I imagine then you’re also tying back into integrating into the security consoles both within the Qualys environment and also within Google?

    Thakar: Yes, absolutely. That’s a unique ability here, because you have multiple business units for organizations. Different teams have different accounts in Google and they are, of course, only looking at their own application or their own account. And so the second part of this announcement really is also the fact that those vulnerabilities and those issues that are detected are then pushed back into the security center from Google so that the individual owners of the accounts can see all of the vulnerabilities on all of the issues that they need to fix right there within their console and they can create playbooks and use the automation that is available to quickly fix those issues so that they don’t have to know go to another tool and go to another console and try to do all of that.

    For the IT folks and the DevOps folks who are pushing these applications into the cloud, they just see their own individual view directly in Google, but then of course this information also is available in the Qualys console, which is a wider visibility into the overall risk posture because it will also include the security and vulnerability findings from other types of infrastructure that the customer also has, things like laptops and handheld devices and containers and other things as well. And so the security team will leverage the Qualys platform because then they get a holistic view of the entire outside vector. The individual team in Google will get their view within the Google console so that they can really focus on fixing what they need to fix.

    Ashley: Really kind of empowers that idea of SecOps, right?

    Thakar: Right.

    Ashley: Better right upfront.

    Thakar: Exactly.

    Ashley: Now I imagine that the security engineers – you know, usually there’s some kind of a corporate security team maybe involved in operations, maybe not, but also would be super pleased about having this implemented earlier into the dev cycle and also that it’s pre-integrated already into the Qualys platform, Qualys monitoring as well as Google’s cloud monitoring.

    Thakar: Right. And that’s really a very good point because that’s a big struggle for security teams, right, is that just getting – ensuring that the entire infrastructure is covered by the security solution is a challenge when you do it after the fact, because they may not even know that there’s a new instance that was pushed by the dev team overnight into the Google account, the security may not know. So, with this integration of course it ensures that no matter what gets pushed Qualys is always monitoring every single image that goes out. Now the security team is good for them because they don’t have to spend a lot of time as they have been doing so far trying to get the security solution implemented.

    So instead of focusing on the findings of the security solution a lot of security teams just spend time trying to get that actually installed and making sure it’s up and running and it’s connecting and all of that. So, having the DevOps team actually do all of that work is fantastic because now the security team can only focus on what is important, which is your policy. So, they can be the one that says, “Well, this is a failure if you have a _____,” and they don’t have to worry about how the thing is installed or whatever it is. Qualys is then the third party that ensures that the policy that the security team has defined is actually tested and a pass or a fail is given to both sides of the house and then they each can go ahead and do what they need to do in terms of fixing and reporting those issues.

    Ashley: Now, you described in the announcement sort of one-click integration. What does one-click mean? Can you visualize that for us, how that works?

    Thakar: Yeah. So we say one-click because if you have ever deployed security solutions they are absolutely painful, you know, trying to get the multiple commands –

    Ashley: Or like one month or one quarter. [Laughs]

    Thakar: Yeah, it’s a thousand clicks and multiple consoles and whatever it is. So the one-click is really because when you go and look at that integration, once you kind of go in Google security center and define that Qualys account that you connect to, then after that it’s really just checking one box that says that include the Qualys agent in every single image and then you don’t have to do anything after that, like you don’t even have to run a separate command every time you initiate a new image because that is already taken care of in a very easy manner with the automation that is provided by Google.

    Ashley: Wow. So it truly is a checkbox kind of process.

    Thakar: Right.

    Ashley: Just included.

    Thakar: Exactly.

    Ashley: Are there any other things that maybe you’re already doing or that are part of this announcement in terms of what you’re doing to monitor scan and report vulnerabilities in the Google cloud environment?

    Thakar: Yeah. I mean we have a very strong partnership with them. We’re listed, of course, in the marketplace as well, and with this sort of integration it really provides sort of a security reference architecture where, you know, Google can now provide along with Qualys customers the ability to make it significantly easy by defining the architecture of how you want to secure your workload that you’re putting in Google and then have – not only define the right solutions that should be used but then also work with vendors like Qualys to get those directly implemented with a single click so you also remove the resistance at integration and really simply that overall process. And so that will enable existing Qualys customers as well as new customers who already are going through the digital transformation, they may have Qualys already on their on-prem infrastructure; now it makes it – it removes another hurdle for them to really move into Google cloud because they can get the same kind of reports and the same kind of analysis as they have been used to because their auditors are very comfortable and used to having the Qualys reports so now they can get the same reports and don’t have to go through an entire process to recertify another solution, so there’s a lot of these kind of benefits that go across the board, and just having that embedded I think really helps.

    Ashley: Yeah. It certainly lowers the bar to entry –

    Thakar: Right, right.

    Ashley: – into the Google cloud if it’s already integrated into the Qualys and Google platforms. Let’s circle back to talking about what are some of the benefits to the DevOps teams getting the Qualys agent into their environment earlier? What kind of things do they learn in the dev cycle and the test cycle group normally they would’ve skipped and not found out till way later when they’re ready to go to production or in production?

    Thakar: Yeah. So because the way the platform is structured with APIs, the biggest advantage, there is automation of – and that’s what DevOps is all about, right? How can I script? You know, you talk about infrastructure as a code. I talk about security as a code, right? Can you just code in to say – you know, like you do your infrastructure, can you say if vulnerability is greater than this than failed _____ or, you know, can you write that as a thing?

    And with the way the Qualys platform is and being a cloud-based platform ourselves and with AP as available, that makes it extremely easy for the DevOps folks to really create directly these scripts that they can basically say, “In your pipeline I got a new image, bring that image up, Qualys agent gets already embedded, it does its analysis and the findings show back up in the console and now you can really code that thing with a human, never having to go in and verify and look at the scans or whatever it is, right?” The way the architecture of the Qualys platform and the agent is, is that agent is an extremely lightweight agent that is not focused on doing just vulnerability assessment.

    It does inventory assessment. It does vulnerability, also configuration assessment. And so what happens is that single agent can be used to look at a larger picture of the security of the device and not just if the software is not updated, right? You can do everything from is unauthorized software being put on this build, is end-of-life software being put on this build, is there configuration that is incorrect even though you may have patch or all your vulnerabilities, and so on and so forth. With that sort of an integration, what happens is that once you have done that one task of getting that agent on the virtual machine you can now suddenly with just APIs do a lot more to get a broader view and make sure that that overall workload is heavily locked down, because now you really are ensuring that all the configurations are as per the golden images and all of the CIS benchmarks are followed and all the software is updated and unauthorized software is already eliminated from that whole thing. DevOps is all about how do I eliminate the manual work and how do I use automation and reduce the amount of work that I have to do, and that’s exactly what that architecture enables us to do.

    Ashley: I think it’s an extremely good point; with all the automation, the increase in speed, the decrease in manual intervention or tasks that are being performed. It’s easy for something to slip in.

    Thakar: Yeah.

    Ashley: New opensource code we haven’t used before or something now has got a vulnerability that didn’t two hours ago and you’ll find out right away.

    Thakar: And, you know, one of the things that we have seen is the DevOps push to go quickly into production and sort of was that initial thing of, “Oh, I want to bypass my operations team, I want to bypass my security team,” and the flipside of that is like now your code that goes directly to production and has an issue, that’s on you as a DevOps – like you cannot blame the security person because, you know, they are not really involved anymore. And so that also incentivizes the developers up front to ensure that they’re doing all the right things from a security perspective also so it’s not their microservice that goes out that has their name on it that gets compromised, right? Because then you can basically say, “I’ve coded everything, done everything on my side up front so that what I am pushing out to production using automation is not something that is already vulnerable.”

    Ashley: Excellent. So, let’s talk about how do you get access to this with the release of this capability? Is it already native within Google cloud? You just connect to it and say, “Here – you know, I’m already a Qualys customer,” or do you have to go to Qualys to sign up for something or how do you get access to this?

    Thakar: Yeah. So, we – and as Qualys we provide directly – if you go to Qualys.com there’s – you can sign up for a free trial if you’re not a Qualys customer and, you know, we’re a SaaS service so you immediately get credentials that you need to access the platform, but then it’s completely embedded in the Google cloud console, so when you got there and there’s the configuration settings you will be able to now pick Qualys as a vulnerability management capability that you would like to add and, you know, there’s a one-time configuration where you provide your Qualys agent information and from that point on, once you have that done, every single – you know, do that one-click checkbox, after that that embedding will happen automatically and Google cloud will be the one doing that for you. So within a few minutes you can be up and running with this integration and starting to see the results in a couple hours.

    Ashley: Excellent. Is there any incremental cost for this or is it already included, just an added capability in the Qualys product?

    Thakar: Yeah. So, it’s a cost that the license of vulnerability management that customers already have. They can basically just take that license or use that or they can come and purchase additional licenses to run additional infrastructure in Qualys, but it’s the same license that they would otherwise use that they can use – that they get from Qualys and they can use in Google Cloud as well.

    Ashley: Yeah. Just the licenses they would need to manage a Google could environment anyway.

    Thakar: Exactly.

    Ashley: Okay. Excellent. Well, great. Fantastic news. Anything else we should know before we wrap up?

    Thakar: No, I’m really – you know, as heading the product, I’m always curious to get feedback, and so I encourage as many of your listeners to go and sign up and, you know, send us feedback on how they feel it helps simplify, and if there’s opportunities for us to simplify the DevOps process even more with better integration we’ll be happy to hear that.

    Ashley: Great. Well, let us know how it goes and we encourage our listeners, go check out Qualys. Qualys.com. And especially if you’re an existing customer, it sounds super easy to start to use this, and I know – I’ve been a Qualys customer and also worked with Qualys before as a partner and super easy to get set up and going, so I encourage folks to check that out. Well, thanks for joining us, Sumedh. Congratulations on the announcement.

    Thakar: Thank you very much. Thanks, Mitch. Good talking to you.

    Ashley: Nice to talk with you. I’d like to thank my guest, Sumedh Thakar, President and Chief Product Office of Qualys, and of course, thank you our listeners for joining us today. This is Mitch Ashley with devopsy.kinsta.cloud. Have a great day and be careful out there.

    — Mitchell Ashley

  • Oracle v. Google and Its Impact on APIs and Code

    Oracle v. Google and Its Impact on APIs and Code

    What, if any, impact will the Supreme Court’s rulings on Oracle v. Google have on the industry at large?

    TL/DR: It is unlikely to impact the vast majority of us, vendor and enterprise developer alike.

    Disclaimer: I am a technologist, not a lawyer. I have done some research, and talked to lawyers, but all that I say here is opinion—even my direct quotes of lawyers are their opinion. The Supreme Court can and has caught people off guard and taken cases in different directions than expected. Though I believe this path is the one it will follow, nothing is assured until the Court rules.

    Throughout this blog, references to The Case are actually using Harvard’s case document.

     

    Background

    On Nov. 15, the U.S. Supreme Court agreed to hear a dispute between Oracle and Google. The Oracle v. Google case has been going on for nearly a decade. And its roots go further back than that.

    The origins of the case go back to 2005, when Google and Sun Microsystems discussed the licensing of Java for Android. This will likely be important because it shows that Google knew it was someone else’s intellectual property (IP). More on this later.

    The crux of the case falls into three parts:

    1. Eight files that were decompiled and used. In one court case, these were found to never have made it into distribution and only used for testing (which limits liability).
    2. Two source files from Java used directly. These were found to be infringing, but damages were set to zero with consent of both legal teams.
    3. The part at issue: the copying of the entirety of Java’s interface from 37 modules. Most tech writers imply this as 37 APIs. It is not—it is hundreds of classes and thousands of methods. The 37 is the number of Java packages.

    The case has been through four court hearings/rulings so far. To sum up the relevant parts of these cases:

    1. The first case (split between a jury and district court, running at the same time) included patent violations. Those were defeated and Oracle has not appealed that defeat. On copyright infringement, the court found that Google did violate copyright on the 37 packages and the directly copied source code, but not on the eight decompiled files. The jury could not settle the issue of “fair use” of the 37 packages.
    2. The district court decided that the eight decompiled files and the copied source code for one function were violations, but remanded the rest back to the jury trial to settle the issue of fair use.
    3. In the end, the district court determined that APIs are not protected by copyright. Their decision involved two parts:
      1. That there is only one way to write a given API.
      2. The interface code and hierarchy can be replicated by third parties without violating copyright because it is an interface and not content.
    4. On appeal, the finding was reversed, and APIs were deemed copyrightable, sending the case back again.
    5. At this point, Google asked the U.S. Supreme Court to intervene. The court chose not to.
    6. Back at jury trial, the court found that while APIs may be copyrightable, Google’s use was fair use in the legal sense. This is the one topic the jury could not originally agree upon.
    7. Oracle appealed this decision, and the District Court ruled that it indeed was not fair use to copy the structure and exact interfaces.
      1. Where the jury and the district court appear to disagree is in the idea of APIs. Calling a publicly documented API is obviously fair use, and the jury latched onto that idea. The District Court appears to acknowledge the difference between calling an API and copying it.
    8. And after some tumultuous hearings and years in the court, the Supreme Court has agreed to weigh in on the issue.

    The real problem here is that copyright law is not a great fit for software. It never has been. If I rewrite your Java application in Python, using the same exact structure and naming, is that protected? All of the “words” have changed. It is akin to rewriting your Hindi book in English without a direct translation  … except that in software there is the added issue of it behaving exactly the same.

    But we can’t fix the misapplication of copyright law to software—only Congress can. So what to expect?

    A fair number of people, from the open source community and from businesses that see opportunity in a ruling that favors their world  view, are advocating that affording copyright protection to APIs is a horrible idea because it would hurt independent developers. This is not quite true. Independent developers are not generally copying Java. While this case does have bearing on coding practices and what developers can do, it does not have a huge impact on innovation, just on copying others’ work and how that is achieved—much like the “reverse engineering” court cases of the ’80s.

    There are good arguments on both sides of this case, but it seems to me that fair use will lose. They are public APIs in the sense that anyone can call them; indeed, some of these packages are so core to the system that you could argue every app must call them. But calling a function is not the same as re-creating it. In that sense, they are not in the public domain. Anyone can read a book, but that does not make it public domain.

    But no matter how this turns out, it is all good for the industry. This uncertainty has hung over the IT sector for nearly a decade. Just like when a new boss comes along and there is uncertainty in the department until everyone adjusts to the new demands/style of the boss, a solid ruling will allow settling to start. The industry can adapt once it knows what the rules are and we can move along.

    The real long-term answer is to have an equivalent to copyright and patent that is designed around the unique parts of computer code. Clearly, software is not the same beast as a book or magazine. The considerations are half-patent and half-copyright, and trying to bend those two to meet the needs of software has created some serious issues. Oracle quoted a 1995 case to showcase this problem—the company is quoting it to win the appeal, but it should be quoted as the reason the protections need to be designed for this medium that is not a publication and yet consists of words whose sequence matters: “Applying copyright law to computer programs is like assembling a jigsaw puzzle whose pieces do not quite fit.” (attributed by Oracle to the concurring opinion of J. Boudin in Lotus Dev. Corp. v. Borland Int’l, Inc., 1995).

    One interesting bit that the courts thus far have ignored, and I would love to see the Supreme Court take up, is this: If Google entered into licensing negotiations, the company knew it was Sun/Oracle’s IP. To then go and copy parts of it without licensing appears wrong on its face. It would be interesting to understand the process that ended in the copying and hear whether that bears upon the legal standing of the case. I can think of both malicious and innocent routes to get there; it would be interesting to hear testimony about motivations and thought processes.

    But even without delving into the licensing negotiations, this case stands to redefine copyright with regards to software. J. Michael Keyes, a partner at the international law firm Dorsey & Whitney, had this (and more) to say:

    “One of the issues that the court may decide is the scope of copyright protection for software. Google had early success below in convincing the trial judge that the snippets of copied code were not copyrightable. Google may urge to have the court take a look at this issue anew. Another huge implication is the scope of fair use. The jury found that Google’s use was “fair” and that it was not liable for infringement. The appellate court overturned that jury conclusion and found that Google’s use was not fair as a matter of law. The Supreme Court will likely wade into this thicket and provide guidance on the scope of fair use in the context of software code and what role an appellate court plays in reviewing jury decisions in that regard.”

    What we (all of us) do not want to do is confuse “good for the industry” with “legal standard.” It is not the court’s job to do what we think is best for the industry; it is the court’s job to determine applicable law and what that implies needs to happen. Presumably, the Supreme Court will do just that. And it may not please any of us. But if we think the current iteration of the law is flawed, that is something to take up with elected representatives, not ask a court to “fix” on the fly.

    For the vast majority of us, though, it impacts our development not at all. Using APIs or forking open source projects are all still perfectly acceptable, and no ruling by the Supreme Court is going to change that—because copying someone’s APIs is the issue here, not using them. And open source licenses allow for copying and improving projects. So carry on. Get the job done. And read about the Supreme Court’s final decision with a smile, because either way, the industry will adapt to the new normal and continue forward.

    — Don Macvittie

  • DORA Report: DevOps Maturity Levels Rise

    DORA Report: DevOps Maturity Levels Rise

    The annual DORA report on DevOps shows a marked increase among Elite performers compared to last year

    The DevOps Research and Assessment (DORA) arm of Google Cloud today published the results of an annual survey that suggests there have been marked gains in the level of DevOps maturity.

    The “2019 Accelerate State of DevOps Report,” an annual global survey of 31,000 IT professionals, finds 20% of the respondents now consider their organizations to be “Elite” DevOps performers, up from a comparatively paltry 7% a year ago.

    The Elite group reported their organizations routinely make multiple code deployments per day. By comparison, low performers reported deploying code somewhere between once per month and once per six months. The normalized annual deployment numbers range from 1,460 deploys per 365 days for the highest performers to seven deploys per year for low performers. Effectively, Elite performers deploy code 208 times more frequently than low performers, according to the DORA report.

    The report also finds Elite performers need less than a day of lead time to make a change to an application, compared to 2,555 hours for low performers, and Elite performers reported a change failure rate of 0% to 15%, compared to 46% to 60% for low performers. Finally, elite performers are able to recover from an incident 2,604 times faster than low performers.

    Nicole Forsgren, DORA Lead, Research and Strategy at Google Cloud, said the study makes it clear DevOps best practices are now crossing the proverbial chasm in IT. Organizations that are furthest along in making that transition tend to set up “communities of practice” within their organizations through which DevOps best practices are shared across teams, she noted. The DORA report also makes it clear elite performers automate and integrate tools more frequently into their toolchains on almost all dimensions.

    Forsgren said the study also makes it clear that the practice of slowing down the rate at which code is developed to achieve higher-quality application deployments is not nearly as effective as employing lighter-weight change processes to lower the amount of code getting deployed at any one time. Slowing down the rate at which code is developed and deployed is not a good strategy, she said, because as the amount of code that needs to get deployed at any time increases, reliance on heavyweight processes to deploy that code usually results in more errors being made. A lighter-weight approach provides the added benefit of reducing the amount of technical debt any organization is likely to incur at any stage of the DevOps process.

    The increased rate at which organizations are embracing best DevOps practices naturally coincides with the rise of more complex cloud-native applications. Most of those applications are being built using microservices that create a lot of dependencies within that application environment, with different development teams often working in isolation on specific microservices. The only way to manage that level of complexityeffectively is to embrace DevOps processes, noted Forsgren.

    In fact, the DORA report finds Elite performers are 24 times more likely to have met all cloud characteristics as defined by the National Institute for Standards and Technology (NIST) than low performers. Organizations that meet all essential cloud characteristics are 2.6 times more likely to be able to estimate the cost to operate software accurately. They also are twice as likely to be able to identify their most operationally expensive applications easily, and 1.65 times as likely to stay under their software operation budget, the study finds.

    Clearly, best DevOps practices is one of those rare IT gifts that simply keeps on giving.

    — Mike Vizard

  • Acquisitions to Fuel Programmable Analytics

    Acquisitions to Fuel Programmable Analytics

    Now that Tableau Software has agreed to be acquired by Salesforce and Looker is set to become a part of Google, the rate at which analytics will be embedded within almost every application is about to accelerate.

    Tableau today is relied on by more than 86,000 organizations to surface business intelligence via data visualizations created by professional analysts and line of business executives. That user base accounts for why Salesforce is willing to spend $15.7 billion to acquire Tableau.

    Competition within the analytics software space, however, has always been fierce. The rise of the cloud has resulted in several analytics startups challenging incumbents for a piece of one of the fastest growing sectors of the enterprise software market. One of those startups is Looker, which created a cloud-based analytics platform that Google is now set to acquire for $2.6 billion.

    Google has already signaled its intent to make Looker a part of its expanding Google Cloud Platform (GCP). As such, Google Cloud CEO Thomas Kurian last week noted one of the primary reasons for the Looker acquisition is to add a software-as-a-service (SaaS) offering that will enable developers to embed analytics within their applications by invoking a set of application programming interfaces (APIs) exposed via a cloud service. Today that analytics service runs on GCP alongside the Google BiqQuery data warehouse. Google, however, has also signaled its intent to make Looker available on multiple clouds alongside the rest of its software portfolio.

    Tableau, which will continue to operate as an independent arm of Salesforce, is playing catch up in the cloud. Tableau is deployed widely in both on-premises and public cloud computing platforms, including as a managed service provided by Tableau through which the company manages a dedicated instance of its software on behalf of customers. The merger with Salesforce will create an opportunity for Tableau to add a full software-as-a-service (SaaS) application instance to its portfolio that leverages the same infrastructure that Salesforce employs to deliver customer relationship management (CRM), marketing and customer service applications. Salesforce CEO Mark Benioff this week said going forward, Salesforce fully expects analytics generated by Tableau to drive a raft of digital business transformation initiatives.

    Most of the analytics that will be embedded within applications will manifest themselves in three forms. The first is embedded software that will be updated continuously as new data is entered in an application. The second is embedded predictive analytics modules that will invoke big data repositories and machine learning algorithms to surface recommendations on how to optimize a business process or customer experience. The third—and more ambitious—goal is an embedded prescriptive analytics capability that automatically optimizes business processes and customer experiences in real time based on events occurring, for example, within an internet of things (IoT) application.

    Every provider of analytics capabilities, from traditional Tableau rivals such as Qlik Software and Microsoft to TIBCO Software, IBM, MicroStrategy, SAP, Oracle and host of others, is pursuing all these emerging use cases for programmatic analytics. Most recently, Information Builders revealed it has embraced Docker containers and Kafka messaging platform as a means for achieving that analytics goal.

    The arrival of programmable analytics platforms will, of course, make implementing best DevOps practices all the more challenging. It’s hard to think of a business process that will not benefit to some degree from embedded analytics. The challenge DevOps teams will face is embedding and then continuously updating all the analytics modules embedded within the applications they deploy. The good news is that as organizations move to embed analytics everywhere increases, so too does the need to embrace DevOps to manage it all.

    — Mike Vizard

  • SRE: How Do You Get to Blameless?

    SRE: How Do You Get to Blameless?

    Site reliability engineering (SRE) is the companion to DevOps. If we take my normal divisions of DEVops versus devOPS, traditional DevOps is mostly the developer side, and SRE is mostly the Ops side. This isn’t a perfect comparison, but it is a good starting point, and for this blog, I’ll run with it.

    While DevOps focuses on getting releases out consistently, quickly and with minimum disruption, SRE focuses on putting those releases into a robust infrastructure. Tracking application issues, regardless of whether you’re using DevOps, always requires information about the software and hardware infrastructure. SRE takes the proactive steps to try and make sure the environment is stable and designed to host the applications being thrown onto it. When that process fails, SRE attempts to identify the code/architectural problem and resolve it without blaming staff members—hence, the idea of “blameless.” It’s about the environment and improving it, not about the individual and punishing them.

    There are a lot of interesting parts of SRE that most IT groups could look at and see as a good place to start taking advantage of. Closely tracking errors in a cross-section of IT interest areas—by application, by infrastructure, by toolset—means problem areas can be definitively identified and fixed. But there is more behind SRE. In this example, once tracking of where errors occur more frequently is in place, a selective reduction in deployments using that piece of the IT puzzle can go into effect until the issues are resolved. This is not unlike what we are doing today, but it is formalized and has measurements around it that generally are not in place.

    Which brings us to Blameless, a company that just came out of stealth to champion SRE. While some large organizations with large application footprints are using SRE already, Blameless is offering a platform to make SRE more accessible to companies without large staff and application footprint.

    We took a bit of time to talk with Blameless last week, and their vision is a strong one: Address directly what DevOps addresses more indirectly. Build and environment that can measure and improve the entire application ecosystem.

    If nothing else, the idea of error budgets is intriguing. Counterintuitive at first glance, the idea of saying, “Piece X gets Y amount of issues before we start backing off of it or raise the priority to fix the underlying issues,” is a solid one.

    If your organization is looking for a mechanism to improve reliability and stability while reducing time to resolve issues, SRE in general and Blameless specifically are worth a closer look.

    — Don Macvittie

  • Linux Foundation to Host CHIPS Alliance Project to Propel Industry Innovation Through Open Source CPU Chip and SoC Design

    New Linux Foundation Project to Foster Flexible, Next-Generation Chip Design for Diverse Data-Centric Applications and Workloads

    SAN FRANCISCO – March 11, 2019 – The Linux Foundation, the nonprofit organization enabling mass innovation through open source, today announced its intent to form the CHIPS Alliance project to host and curate high-quality open source code relevant to the design of silicon devices. CHIPS Alliance will foster a collaborative environment that will enable accelerated creation and deployment of more efficient and flexible chip designs for use in mobile, computing, consumer electronics, and Internet of Things (IoT) applications.

    Early CHIPS Alliance backers include Esperanto, Google, SiFive and Western Digital, all committed to both open source hardware and continued momentum behind the free and open RISC-V architecture. “The RISC-V community is working to foster open source foundation technologies that will help unlock market innovation to move [artificial intelligence/machine learning and infrastructure composability] forward,” said Eric Burgener, research vice president of IDC’s Infrastructure Systems, Platforms, and Technologies Group, via a recent IDC report.

    The project will create an independent entity so companies and individuals can collaborate and contribute resources to make open source CPU chip and system-on-a-chip (SoC) design more accessible to the market.

    “Open collaboration has repeatedly proven to help industries accelerate time to market, achieve long-term maintainability, and create de facto standards,” said Mike Dolan, vice president of Strategic Programs, the Linux Foundation. “The same collaboration model applies to the hardware in a system, just as it does to software components. We are eager to host the CHIPS Alliance and invite more organizations to join the initiative to help propel collaborative innovation within the CPU and SoC markets.”

    CHIPS Alliance will follow governance practices consistent with other Linux Foundation projects, which will include a Board of Directors, a Technical Steering Committee, and community contributors who will work collectively to manage the project. Initial plans will focus on establishing a curation process aimed at providing the chip community with access to high-quality, enterprise grade hardware.

    Planned Contributions

    Google

    Google is planning to contribute a Universal Verification Methodology (UVM) based instruction stream generator environment for RISC-V cores. The environment provides configurable, highly

    stressful instruction sequences that can verify architectural and micro-architectural corner-cases of designs.

    Western Digital

    Western Digital is planning to contribute their high performance, 9-stage, dual issue, 32-bit SweRV Core, together with a test bench, and high performance SweRV Instruction set simulator. Additional contribution will be specification and early implementations of OmniXtend cache coherence protocol.

    SiFive

    SiFive was founded by the inventors of the free and open RISC-V Instruction Set Architecture, who, together with their colleagues at UC Berkeley, developed the first open-source RISC-V microprocessors and a new open-source hardware description language Chisel. This initial work at UC Berkeley also developed the RocketChip SoC generator, including the initial version of the TileLink coherent interconnect fabric.

    SiFive remains committed to maintaining and improving the RocketChip SoC generator and the TileLink interconnect fabric in open-source as a member of the CHIPS Alliance, and contributing to Chisel and the FIRRTL intermediate representation specification and transformation toolkit. SiFive will also contribute and maintain Diplomacy, the SoC parameter negotiation framework.

    To learn more about CHIPS Alliance, please visit chipsalliance.org.

    SUPPORTING QUOTES

    Esperanto Technologies

    “Intellectual property for VLSI chip designs ought to be able to reap similar benefits as open source software has for years. We hope that the CHIPS Alliance will be a catalyst where all hardware designers feel comfortable both contributing and finding useful designs for their projects,” said Dave Ditzel, founder and CEO of Esperanto Technologies, Inc.

    Google

    “We are entering a new golden age of computer architecture highlighted by accelerators, rapid hardware development and open source architecture and implementations. Google is committed to fostering an open community of collaboration and innovation in both hardware and software,” said Dr. Amir Salek, senior director, Technical Infrastructure, Google Cloud. “TheCHIPS Alliance will provide the support and framework needed to nurture a vibrant open source hardware ecosystem for high-quality, well-verified and documented components to accelerate and simplify chip design.”

    SiFive

    “Semiconductor design starts have evaporated due to the skyrocketing cost of building a custom SoC,” said Yunsup Lee, co-founder and CTO, SiFive. “A healthy, vibrant semiconductor industry needs a significant number of design starts, and the CHIPS Alliance will fill this need. SiFive is excited to continue to work on and contribute to the RocketChip generator, Chisel and FIRRTL projects as we push the boundaries of open source innovation.”

    Western Digital

    “The data-centric universe continues to grow and expand in ways many of us never imagined,” said Dr. Zvonimir Bandic, senior director of next generation platforms architecture at Western Digital, a co-founder of RISC-V as well as CHIPS Alliance. “The CHIPS Alliance will provide access to an open source silicon solution that can democratize key memory and storage interfaces and enable revolutionary new data centric architectures. It paves the way for a new generation of compute devices and intelligent accelerators that are close to the memory and can transform how data is moved, shared, and consumed across a wide range of applications. By extending Western Digital’s commitment to the RISC-V architecture and instruction set, and teaming up with fellow industry leaders to form the CHIPS Alliance, we make another important stride forward toward unlocking the true potential of the data.”

    About the Linux Foundation

    Founded in 2000, the Linux Foundation is supported by more than 1,000 members and is the world’s leading home for collaboration on open source software, open standards, open data, and open hardware. Linux Foundation’s projects are critical to the world’s infrastructure including Linux, Kubernetes, Node.js, and more. The Linux Foundation’s methodology focuses on leveraging best practices and addressing the needs of contributors, users and solution providers to create sustainable models for open collaboration. For more information, please visit us at linuxfoundation.org.

    ###

    The Linux Foundation has registered trademarks and uses trademarks. For a list of trademarks of The Linux Foundation, please see our trademark usage page: https://www.linuxfoundation.org/trademark-usage. Linux is a registered trademark of Linus Torvalds.

     

  • Survey Finds Wide Gap Between DevOps Adoption and Success

    Survey Finds Wide Gap Between DevOps Adoption and Success

    A recent HBR survey shows there is work to be done to advance DevOps in organizations

    A survey of 654 subscribers to the Harvard Business Review found that while 86 percent of respondents said it is important for their organizations to be able to build and deploy software quickly, only 10 percent said they are successful at it.

    Conducted by Harvard Business Review Analytic Services on behalf of Google, the survey found nearly half (48 percent) of respondents said their organization always relies on DevOps methodologies when deploying software, while another 21 percent rely on DevOps selectively. An additional 8 percent said their organization plans to adopt DevOps practices.

    Among those respondents employing DevOps practices, the top benefits cited were increased speed to market (70 percent), productivity (67 percent), customer relevance (67 percent), innovation (66 percent) and product/service quality (64 percent). Despite those benefits, however, most organizations still have a significant amount of work to do in terms of implementing best DevOps practices—the three biggest hurdles organizations face are organizational silos (50 percent), legacy technologies (49 percent) and resistance to change (46 percent).

    Melody Meckfessel, vice president of engineering for Google Cloud Platform (GCP), said these issues are precisely why Google is sharing how it operates in the form of site reliability engineering (SRE) principles, which are a prescriptive set of guidelines for implementing best DevOps practices. Not every organization needs to implement those principles to the same level, but just about every organization can benefit from adopting a set of well-defined engineering principles, said Meckfessel.

    The challenge many organizations face is the fact that most of the IT staff are not engineers by trade. Most IT staffs are made up of administrators who tend to have limited programming skills. But Meckfessel said even IT organizations that don’t necessarily have a lot of engineering expertise still can benefit from automating operations based on the SRE principles.

    Many of those organizations will need to slow down the rate at which they are building applications today to be able to speed up tomorrow. It takes time for organizations to master the cultural change required to implement DevOps practices. But once they begin to master DevOps, the benefits to the organization compound quickly. Meckfessel said two of the most important attributes of DevOps is adopting an immersive approach to training and establishing a no-blame culture within the IT organization when it comes time to conduct post-mortems. That latter goal is especially difficult to achieve sometimes, because most interactions between developers and IT operations teams historically have been less than positive.

    Meckfessel also noted it’s critical now for organizations to incorporate cybersecurity into its DevOps processes as well, which requires not only additional training for developers but also closer alignment with cybersecurity teams.

    It’s now only a matter of time before the complexity associated with emerging technologies such as microservices based on containers force many organizations to dive deeper in DevOps. The real challenge now is charting a path of least resistance that also generates the maximum level of benefit for all concerned.

    — Mike Vizard

  • Google Rolls Out Additional Cloud Certifications

    Google Rolls Out Additional Cloud Certifications

    Google is moving to address a chronic shortage of cloud skills necessary to advance DevOps.

    Specifically, Google plans to add Professional Cloud Developer (GA), Professional Cloud Network Engineer (Beta), Professional Cloud Security Engineer (Beta) and G Suite certifications (GA) alongside Professional Cloud Architect, Professional Data Engineer and Associate Cloud Engineer certifications that already exist.

    Google also plans to localize its cloud learning and certification programs in Japanese, French, German, Spanish and Portuguese to make them accessible to more people.

    Rochana Golani, director of Google Cloud learning and enablement for Google, said Google is trying to increase the number of qualified engineers available to manage IT infrastructure as code. Most of the IT operations community is made up of IT administrators that don’t have much in the way of actual programming skills. The Google cloud certifications are all steeped in the site reliability engineer (SRE) principles that Google defined in 2003. As a more prescriptive instance of a DevOps practice, the goal is to make the skills needed to successfully transition to DevOps more widely accessible, said Golani.

    Google is especially keen to increase the number of networking and security professionals that have demonstrated cloud skills, as the critical role these functions play in cloud application environments is often underestimated, added Golani.

    Those challenges likely will only increase as organizations increasingly embrace microservices based on containers running on Kubernetes clusters that eventually will be tightly coupled to serverless computing frameworks. In fact, the first era of the cloud was defined by a comparatively simple replacement of virtual machines from on-premises IT environments with similar virtual machines residing on a shared public infrastructure. The next wave of the cloud is being defined more by so-called cloud native applications constructed using microservices. That approach is inherently more flexible from a business perspective, while paradoxically being more difficult to manage from an IT perspective.

    On the upside, however, DevOps in general and SRE disciplines more specifically enable IT organizations to operate more hand in glove with the business as it grows and evolves.

    Of course, none of that is achievable without the skills required to achieve that goal. Savvy IT professionals naturally will pursue certifications that enable them to earn higher salaries. But organizations that encourage their personnel to achieve those certifications should be able to at the very least engender more loyalty among IT professionals they help advance their skills.

    In the meantime, it should be apparent that transformation is all but inevitable. Not every workload will move into to the cloud. But the principles applied to managing workloads in the cloud will be increasingly applied within on-premises IT environments. Those on-premises IT environments may not be as programmable and resilient as a public cloud, but the need to process data locally is not going to go away anytime soon. The challenge IT organizations are facing today is how best to define a set of best DevOps practices that can span both environments.

    — Mike Vizard

  • Google Cloud Platform: The Basics for Absolute Beginners

    Google Cloud Platform: The Basics for Absolute Beginners

     Google Cloud Platform provides powerful tools and services. Why should you pay attention to one of the leading cloud service providers? More and more companies are moving to the cloud to help create software applications and manage data, and Google Cloud Platform is one of the many platforms available for cloud computing, an industry that is poised to break $159 billion in market value by 2020.

    There are four basic options for creating applications on Google’s infrastructure:

    • Google Compute Engine
    • Google App Engine
    • Google Kubernetes Engine
    • Google Cloud Functions

    What You Need to Know About Google Compute Engine

    Google Compute engine lets you create instances. These instances are virtual machines with their own set of hardware and software. Basically, your personal instance, or VM, is a virtualized environment that sits on top of the very real partitioned environment on servers in a Google data center.

    Google Compute Engine is an infrastructure as a service (IaaS). As such, you’ll be able to control networking, firewalls, disk storage and a lot of granular aspects of how, when, where and why your application works.

    The Essentials of Google App Engine

    Google App Engine is a platform as a service (PaaS). You don’t have to worry about the infrastructure—this is taken care for you. Your project will scale as necessary according to your needs. Simply vendor your APIs, add custom code, load it into the app engine and voilá! You’ve got yourself a working Google Cloud application.

    Simply set up billing, add some developers, put in a dash of code and you’ve got the gist of the full project arc for App Engine.

    Google Kubernetes Engine: The Absolute Basics

    Google Kubernetes Engine lets you create containerized applications with the open source Kubernetes. Containerized application management is just another way of talking about a kind of specialized VM management.

    With containerized apps, you don’t need a full VM to launch distributed software applications. In other words, it’s a way to boost memory and efficiency by having multiple containers sitting on top of a single VM instance. It’s a relatively new way of developing applications.

    Google Cloud Functions

    Serverless applications aren’t without their dedicated service, either. For all your serverless application needs, Google Cloud Functions has got you covered.

    Getting Started With Your First Project

    Setting up your first project within Google Cloud Platform can be done with relative ease.

    First, you’ll want to get an account with Google Cloud Platform if you haven’t already. This can be done in a few easy steps:

    1. Sign up with a Gmail account.
    2. Agree to the terms of service.
    3. Sign up for the free trial (one year of free membership and $300 credit toward your account).
    4. Enter your credit card information.

    After you’ve completed those steps, you’ll be prompted to an in-browser tutorial. Don’t skip this. It’s an easy way to get oriented to the platform right off the bat. Follow the tutorial so you can get a feel for how the developer console is organized.

    Creating Your First Project

    Now it’s time to create your first project, add team members and enable billing. This is fairly straightforward, but it’s important to get acquainted with these pages, dashboards and controls early on.

    To create a project, simply click on “create a new project,” then follow the prompts. Every Google Cloud Platform project needs a unique title so make sure you supply one or accept a generated one. Then scroll down to IAM policies. You’ll want to make sure you have the right privileges in place.

    After you’ve gone through a few of the tutorials and created your first project, open up the hamburger menu in the upper left corner. When you expand the menu you’ll see a long list of services. App Engine, Compute Engine and Kubernetes should all be visible.

    Conclusion

    There are four major options when deciding to employ Google Cloud Platform: Google Compute Engine, Google App Engine, the Kubernetes Engine and Google Cloud Functions. All are great options for starting your first cloud project.

    Google Cloud Platform is a service that gives you access to cloud client libraries and enterprise-grade hardware. As more and more companies move to the cloud, and more and more data is produced, it’s important to start investigating which cloud solution is right for you.

    With the Google Cloud Platform, you have many options, from building atop an infrastructure, a platform or building and managing containerized applications. What you choose to employ is completely up to you.

    — Charles Dearing

  • What Google’s ‘Digital Wellbeing’ Means for Every App Developer

    What Google’s ‘Digital Wellbeing’ Means for Every App Developer

    One of the big themes to emerge from Google’s recent I/O conference was Digital Wellbeing—the notion that technology should be a positive force in our lives, and that we need ways to disconnect from apps when they suck up too much time or distract us in other ways.

    Google is taking the idea seriously, and developers should, too. The fact that the company is encouraging people to spend less time with its services should be a wake-up call that every developer needs to think carefully about how they design, build and test their applications.

    To understand why Google’s digital wellbeing initiative is significant for all developers—not just those building for Google platforms—it’s helpful to look at some of the features announced in support of digital wellbeing, many of which are coming in the next version of Android.

    The first is the app dashboard, which will allow users to see at a glance how much time they spend on their phone, how frequently they use different apps and how many notifications they get. Android also will provide new options for silencing notifications, and easier ways to turn on do not disturb mode.

    Another new feature is an “app timer,” which allows Android users to set limits on how much time they spend in an app. When you reach your daily limit, the app icon is greyed out to remind you not to use it again. Google’s YouTube app is getting a similar timer that reminds you to take a break after an hour’s continuous viewing, or however long you choose to set it for.

    It was unthinkable a year ago that Google would encourage people to take a break from their smartphones, since mobile ads are one of the primary ways it makes money. But times have changed, and Google’s digital wellbeing initiative reflects a wider public awareness that technology is not an unbridled force for good.

    “We’ve reached an important inflection point in computing,” CEO Sundar Pichai said at Google I/O, “and it’s made us even more reflective about our responsibilities.”

    Microsoft CEO Satya Nadella expressed a similar sentiment at Microsoft Build, calling for greater responsibility in the technology industry. “None of us want to see the future that Huxley and Orwell imagined,” Nadella said, referring to dystopian authors Aldous Huxley and George Orwell.

    So what does this mean for developers? It means we need to have a greater awareness of the impact the technologies we build will have on users, and to provide the best possible experience with everything we do.

    That means we can’t bombard people with alerts and notifications just to make them spend more time in our apps. People are tired of the constant nagging, and those tricks now work against you. We also need to maximize ease of use, so users can accomplish what they need to do quickly and move onto the next task. Don’t be the app that makes people think, “I waste too much time in this software; I’m setting the timer.”

    Most of all, developers need to provide the very best user experience they can. There were 3.5 million apps in the Google Play store at the end of last year, so there’s no shortage of choice about which to use. If your software is buggy or unintuitive or triggers too many alerts, your app will be among the first to be silenced or uninstalled.

    Google’s focus on digital wellbeing has raised the bar for all of us, and that’s a great thing. But it means developers need to redouble their focus on the user experience. People don’t unquestioningly love their phones anymore; there’s greater awareness of the downsides that constant app usage can bring, and developers need to think about this changing environment as they design, build and test their software.

    — Fred Stevens-Smith