devopsy.kinsta.cloud

Tag: database

  • DataStax Brings AIOps to Cassandra Database Management

    DataStax Brings AIOps to Cassandra Database Management

    DataStax today unveiled a private beta release of artificial intelligence tool dubbed Vector that provides advice on how to optimize clusters running the open source Cassandra wide column store database.

    Vector analyzes individual nodes and then compares behavior to other nodes in the cluster to identify opportunities to improve deployments.

    Ed Anuff, chief product officer for DataStax, said the goal is to enable IT teams to take advantage of AIOps for things such as improving performance and query techniques.

    Available for both instances of Cassandra downloaded by IT teams as well as the curated instance of Cassandra that DataStax makes available as DataStax Enterprise, Vector is intended to augment rather than replace IT teams managing Cassandra databases, said Anuff.

    Anuff said by employing algorithms and heuristics, Vector provides IT teams with predictive guidance, which they can determine whether to implement or not.

    As IT becomes more complex the need to rely on AIOps tools such as Vector is becoming more pronounced. The number of variables that could adversely impact the performance of applications deployed on Cassandra or any other database exceeds the ability of most IT teams to monitor manually. AIOps tools such as Vector essentially provide IT teams with a digital assistant to help optimize database deployments.

    Anuff said there will always be a need for a human to be in the loop, but each IT organization will determine the degree it trusts the recommendations being surfaced by AIOps tools such as Vector. In some cases, IT teams will use those recommendations to inform frameworks to automate the management of databases.

    Given the commitment to ruthless automation by IT organizations that have embraced DevOps, Anuff said DataStax expects DevOps teams to be at the forefront of embracing AIOps. In many cases, those DevOps teams have already assumed management responsibilities for both applications and the underlying databases on which they depend.

    Of course, AIOps is still in its infancy. However, in the wake of the economic downturn brought on by the COVID-19 pandemic interest in AIOps as a means to reduce the total cost of IT is rising. Many IT teams are becoming smaller as organizations downsize, while others are limiting headcount. AIOps provides a way to manage IT at scale without necessarily having to expand the size of the current IT staff.

    Regardless of the motivation, AIOps is here to stay. Many of the rote tasks that conspire to turn IT into drudgery should soon be eliminated. That may reduce the number of entry-level jobs that historically have been available. However, as schools adjust to this new reality, it may also mean that new entrants into the field may be able to take on more complex challenges sooner than later.

    In the meantime, IT professionals might want to consider what part of their job might be better handled by a machine and plan the next step in their careers accordingly.

  • MongoDB, Storj Labs Work to Reduce Data Security Costs

    MongoDB, Storj Labs Work to Reduce Data Security Costs

    MongoDB Inc. and Storj Labs announced today they have jointly created a backup service for the open source MongoDB database that leverages a global network for excess storage capacity.

    Storj Labs has created a Tardigrade Decentralized Cloud Storage Service based on an open source object storage system that is compatible with the S3 application programming interface (API) defined by Amazon Web Services (AWS). That object storage system distributes 80 slices of a file across a subset of thousands of nodes that each contain only a slice of the data required to reconstruct that file when needed. Only 29 of those slices are needed to reconstruct a file, which enables IT teams to secure data without having to manage encryption keys.

    Via a connector from Storj Labs, the Tardigrade Decentralized Cloud Storage Service now can be invoked via MongoDB DevOps Manager, which is made available in the commercially supported enterprise edition of the open source databases provided by MongoDB Inc.

    Storj Labs CEO Ben Golub said the Tardigrade Decentralized Cloud Storage Service provides a less expensive alternative to backup and recovery platforms based on the cloud, as excess capacity on devices ranging from servers to individual desktops is readily available, The challenge has been finding a way to marshal all that excess capacity in a way that makes it simple for IT organizations to consume, he said.

    IT organizations, Golub said, should expect Storj Labs to build additional connectors to databases and other platforms that require a less expensive approach to data protection. In the wake of the economic downturn brought on by the COVID-19 pandemic, he noted, more organizations than ever are open to innovative approaches to reducing storage costs.

    Of course, most IT teams are well-used to employing a range of data protection platforms to back up and recover data. The Tardigrade Decentralized Cloud Storage Service is likely to be viewed as one more option that might find traction at a time when the number and types of databases being deployed across the enterprise continue to expand rapidly. Developers are especially fond of adopting open source document databases that don’t require a database administrator (DBA) to get approval for a license from a central purchasing organization. It’s usually not too long, however, before developers look for help from central IT to manage all those databases, especially after an application is deployed in a production environment.

    Naturally, not all those applications are of equal value, so given the current economic climate, there might be a greater willingness to at the very least experiment with alternative approaches to data protection.

    Regardless of platform, however, the one thing that is clear is data protection alongside the rest of IT infrastructure is being driven more by APIs. As such, responsibility for data protection is also being pushed further left toward DevOps teams, who have the expertise required to continuously automate back up and recover using APIs. That might not always sit well with traditional IT administrators who have been using tools based on graphical user interfaces (GUIs) for decades. Nevertheless, IT times are clearly changing.

  • Hasura Launches Beta of GraphQL-Based Remote Joins Tool

    Hasura Launches Beta of GraphQL-Based Remote Joins Tool

    Hasura today announced the public beta of Remote Joins, which makes use of an open source GraphQL query language for application programming interfaces (API) to join disparate data sources residing in different repositories.

    Company CEO Tanmai Gopal said Remote Joins extends a GraphQL engine the company has developed around a project that is now hosted by the GraphQL Foundation, an arm of the Linux Foundation. The GraphQL Foundation counts among its members Airbnb, Amazon Web Services (AWS), Apollo, Coursera, Elementl, Facebook, GitHub, IBM, Intuit Hasura, Paypal, Prisma, Shopify and Twitter.

    Originally developed by Facebook, GraphQL has emerged as the basis for a more streamlined alternative to REST APIs. GraphQL provides a complete description of the data made available via an API and is organized in terms of types and fields rather than traditional endpoints. In addition to making it more clear what data is available via that API, developers can employ types to avoid writing manual parsing code.

    Hasura is now taking that concept a step further by enabling IT organizations to join data residing in different databases and microservices to enable that data to be queried as if it were a single data source. The core Hasura engine automatically generates the GraphQL engine, which enables those remote joins to be created in a few minutes.

    Gopal said joining data across a complex portfolio of modern applications based on microservices and legacy monolithic applications is becoming increasingly difficult as the number of microservices in an IT environment continues to proliferate. The Hasura engine addresses that issue by providing data access and flow tool base on a GraphQL type system for managing APIs that can be applied to GraphQL APIs as well as REST APIs or other protocols, he said.

    The join, authorization logic and consistency checks are all managed by the Hasura engine. Hasura also provides data change management, observability, auditing, security and performance management capabilities. That approach makes it possible for data sources and APIs to evolve independently while still allowing organizations to query data flexibly, noted Gopal.

    As DevOps and DataOps processes continue to converge, IT organizations will need to find a way to advance application and API development in a way that doesn’t result in data access becoming a major bottleneck. Too many DevOps teams are rapidly building applications and APIs only to find themselves waiting to be given access to data. GraphQL could provide a foundation on which IT teams can manage DevOps and DataOps processes more collaboratively.

    It’s not likely IT teams will be replacing existing REST APIs wholesale overnight. However, it’s also apparent that REST APIs may not enable IT environments to scale as efficiently as GraphQL, which is the primary reason why Facebook developed it. Of course, it’s still early days as far adoption of GraphQL is concerned. However, with more IT teams under pressure to innovate faster, interest in a more streamlined approach to building and managing APIs is likely to gain additional momentum.

  • Datical Embraces Liquibase Identity

    Datical Embraces Liquibase Identity

    Datical today announced that it will now operate under the moniker Liquibase to better reflect the open source tool it provides for automating database releases.

    At the same time, the company is promising to make it easier to upgrade from the Liquibase Pro edition of its software to Liquibase Business, a new edition that provides enhanced visibility into code changes across the pipeline to improve workflows across teams.

    Dion Cornett, formerly president of Datical, has been appointed CEO of Liquibase. Current Datical CEO Derek Hutson will remain an advisor to Liquibase.

    Company CTO Robert Reeves said since Datical took control of Liquibase in 2012, the open source project has steadily gained traction to the point where it now makes sense to rebrand the company. Since 2012, there have been more than 15 million downloads of Liquibase, he said.

    The company now offers four editions of Liquibase ranging from Liquibase Enterprise, formerly known as Datical DB, and Liquibase Business to Liquibase Pro and Liquibase Community, a free offering. Liquibase Pro and Liquibase Community are aimed at developers, while Liquibase Enterprise and Liquibase Business are aimed at IT operations teams, said Reeves.

    Previously, developers were able to upgrade from Liquibase Community to Liquibase Pro via a single click. The company is now committing to providing the same level of frictionless upgrade path between Liquibase Pro and Liquibase Business, he added.

    In general, Reeves said there is a much greater appreciation for database schema change management and version control among organizations that have adopted best DevOps practices. Many database providers sometimes promote their databases as being platforms that developers can simply download and manage on their own, he noted; however, from a practical perspective, DevOps teams need a tool to consistently manage both database schema updates spanning multiple types of databases.

    Historically, databases were managed by database administrators. However, in recent years developers have been downloading a variety of open source databases to build applications that in most cases are simple to set up and don’t require someone to approve a purchase order. As those databases proliferate, however, DevOps teams need a tool to manage databases that have proliferated across the environment. Liquibase provides a tool for managing databases that is designed from the ground up for DevOps workflows, Reeves said.

    Most organizations are still working through the respective roles DevOps teams and DBAs will have as DevOps processes are adopted more widely. As IT environments become more complex, there is a clear need for more automated approaches to managing fleets of databases. In the wake of the economic downturn brought on by the COVID-19 pandemic, many organizations may not have the IT staff that would allow them to manage databases in any other way.

    The paradox, of course, is that the time and effort to automate database management often requires the expertise of members of the IT team who are many times too busy keeping the IT lights on to really think through how to automate the update process once and for all.

  • DevOps Chats: Liquibase, Open Source and Databases, with Datical

    DevOps Chats: Liquibase, Open Source and Databases, with Datical

    Datical has been a leader in DevOps for databases since even before the term was used. What many may not realize was that the open source Liquibase project was a major contributor to Datical’s success. So it should be no surprise that in filling the open president’s seat, they would tap an open source veteran.

    Dion Cornett, has open source credibility from his time at Red Hat and MariaDB. In his new role, he is responsible for marketing, sales and product. He is determined to see the same kind of success with open source business models he has been part of before, succeed at Datical.

    In this discussion we hear from Dion on his plans and views.

    As usual, the streaming audio is immediately below, followed by the transcript of our conversation.

    [soundcloud url=”https://api.soundcloud.com/tracks/771223528″ params=”color=#ff5500&auto_play=false&hide_related=false&show_comments=true&show_user=true&show_reposts=false&show_teaser=true” width=”100%” height=”166″ iframe=”true” /]

    Transcript

    Alan Shimel: Hey, everyone, this is Alan Shimel for devopsy.kinsta.cloud. You’re listening to another DevOps Chat. We’ve got a nice DevOps Chat for you today, happy to introduce you to the newly minted president at Datical, Dion Cornett. Dion, I didn’t mess that up, did I?

    Dion Cornett: You didn’t, and thank you very much for the introduction, and I’m very happy to be on your podcast today.

    Shimel: Thank you, Dion. So, listeners of our podcast are no strangers to Datical. You know CTO Rob Reeves is a frequent guest here, and you know, we make fun of him and his Big Lebowski sweaters, but it’s always refreshing to hear Robert’s take on DevOps and databases, the DevOps market in general, and pontificating on anything Robert likes to pontificate about.

    But it’s also nice to have a fresh face from Datical. So, first of all, welcome aboard.

    Cornett: Thank you.

    Shimel: And I always like to give my audience a flavor for who’s talking. Dion, would you mind—I’m not asking you to read your bio, but would you share a little bit of your background for the audience?

    Cornett: Yeah, you know, I think what is more pertinent to Datical, I’d say if I had to characterize myself as that, I’m an open source guy. I bleed open source, I believe it’s a better production model for software, you know, this combination of a meritocracy of ideas and transparency leads to more elegant code.

    And so, you know, my background has been one of sort of having a career that aligns with this philosophy. So, I spent nine years at Red Hat in a couple different executive roles, run North American non-named account sales, and then ran global strategic alliances. I moved from there to take over sales at MariaDB, an open source database company that was—

    Shimel: Sure, [Cross talk] well.

    Cornett: Yeah. [Laughter] It wasn’t quite so popular in 2013, but we literally grew that business sevenfold in three years. That set me up for a CEO job. I did do something outside open source, but it was sort of a big data challenge, a company called ReachForce, that we successfully sold to another company in this space called Leadspace.

    And then, I was actually taking some time off, and introduction through a board member, spent some time with Datical, and just sort of fell in love with the concept. So, you know, my work at Red Hat, which obviously covers the very foundations of sort of IT into all the challenges as it relates to data at MariaDB and really sort of seeing the struggles that go on in building and maintaining a big, powerful database and how central that is to virtually any application. And you look at some of the top banking or telecom type customers we have, you know, data is the core of some of the value that many of these companies create.

    And then, even at ReachForce, right, big data company in the martech space, data is just paramount. It’s all maintained in databases and trying to update, upgrade, you know, maintain those databases in the context of fast changing application requirements is a non-trivial exercise.

    And so, when a company came along that had this combination of the open source that’s sort of in my DNA, and then solving this challenge around database change management, it just seemed like a great fit.

    Shimel: Absolutely. So, you know, another thing let us get out of the way early on, Dion, is—in today’s world, when we appoint a president, you know, there was a time, as we were talking, CEO and president were one person. And then, from a governance point of view, that’s a bad model. We have, for the most part, split off CEO from president. And presidents have various responsibilities. Some are more of a COO, some are more of a CRO, you know, meaning responsible for all revenue kind of activities.

    Just, what—where is your, you know, what are you responsible for at Datical?

    Cornett: Yeah, so, formally, my responsibilities are sales, marketing and product. And sort of the genesis here, and I have a great relationship with Derek, the CEO of the company. We’ve been on the job for a month now and I think we’ve teamed well, even through my recruitment process and through that.

    But Derek has built an amazing culture here. And this is one of the things you never know when you start a new job is just sorta, you know, what is the synergies between the team, and Derek’s a very effective leader, he’s built a great culture. In terms of the depth and the value of the solution that Datical has, it’s sort of undisputed, right? We’ve got multiple high six figure customers and even a seven figure customer. I mean, these are solving demanding problems in big enterprises, and the company’s been very good at that.

    I was brought on for a very specific charter, right? How do we better engage and collaborate with our community to push innovation out to the market faster? And as your listeners may know, Alan, the real foundation for Datical is an open source project called Liquibase. Liquibase is—and one of the things I also did at Red Hat was, I ran the venture portfolio there. I’ve looked at literally hundreds of open source business plans, made a dozen plus investments and sat on a few boards of companies that Red Hat invested in. And Datical was sort of a top 10—or Liquibase was sort of a top 10 project as you look at the thousands and thousands of open source projects.

    And so, how do we work with that community, again, in a collaborative fashion that we’re really pulling innovation, pushing it back, and driving as much value into this challenging database problem as we can?

    Shimel: Absolutely. So, you said a lot, there. Let’s pick ‘em off one by one.

    So, first of all Liquibase. It’s something we’ve mentioned in past podcasts and in past content type webinars and stuff that we’ve done with Datical. But—and maybe you’re gonna change this, and I hope you will, Dion. I know in my mind, when I think of Datical, I don’t think Liquibase. And though Liquibase has a tremendous following, I just don’t—it seems like, and I’m not blaming anyone or anything like that, but I just, I don’t connect the two. And maybe that’s part of what your mission needs to be.

    Cornett: Absolutely. We want this connection to not only exist, but be robust. Again, we’ve got a great team here, great culture, great people. We wanna use that to be excellent stewards of the community.

    You know, part of what motivates people here is, this is a big challenge to solve. You sort of commented earlier about sort of the maverick kinda style that Robert has, and I think it’s properly so, right? We’re breaking some glass. We’re trying to create a new mold, a new direction for how and a very important development of how IT operates.

    And so, that’s Liquibase. And, you know, Datical is a set of capabilities built on top of that. We need to make it clear, though, that that core is our foundation, and that, you know, we wanna be, again, as good of stewards as we can of that Liquibase community, 15 million plus users, and you will see this become much more of our identity, in everything we do, how we act, behave, where we invest and what we do to try to push value forward into our customers.

    Shimel: Excellent. So, Dion, let me throw another thing along the open source stuff out at you. You know, as I spoke to you earlier, I have my own views of where the market is, you know, builders, the democratization of IT, as I call it.

    I also have a view of where the open source movement, you know, the whole open source community is. And that is, you know, we’ve moved from the cathedral and the bazaar to what I call the big brother stage of open source where, seemingly, every open source project was kinda sponsored or in fact owned by a single vendor. And that kind of inhibits collaboration on an industry basis, right? It’s like, Oracle wasn’t gonna contribute to MS SQL. I mean, MS SQL was far from open. [Laughter]

    But let’s say, once Oracle bought—

    Cornett: MySQL.

    Shimel: MySQL. Look, that was the reason Maria and a host of others popped up, right? Because people weren’t sure that Oracle was gonna play nice, frankly.

    But that’s old. The model is what I call the foundational era of open source, where we have, like, the Linux Foundation or the Eclipse Foundation or the Cloud Foundry Foundation. You know, they’re these not-for-profits that are, they quote-unquote own the open source project to the community’s benefit.

    I’m not aware of a foundation in, like, the Liquibase kind of playing field. Are you, or is that—

    Cornett: No, you know, and I’m very familiar with the foundation model. Yeah, obviously, Red Hat was sort of a pioneer in this with Fedora.

    Shimel: Sure, yeah.

    Cornett: And we actually implemented a foundation model at MariaDB as well. And so, even if you look today, MariaDB.org is a separate legal entity—

    Shimel: Yep.

    Cornett: Separate president and set of engineers. And the real purpose there is to provide confidence to the users that, regardless of vendor acquisitions as you discussed, that the capabilities of that project will go forward.

    And we don’t—again, I’m a month on the job here—we don’t have a foundation today with Liquibase.

    Shimel: Right, I’m not asking you to—don’t say anything you can’t [Cross talk]. [Laughter]

    Cornett: Yeah. [Laughter] It’s certainly a consideration. [Cross talk] But it’s, you know, it’s a measured one, right? So, what are the trade-offs, what are the value? And as you talked about the different phases of open source, I think it’s—it’s unfortunate that there was a point where open source was sort of a, you used the phrase democratization of technology; that was a phrase also used by Matthew Szulik, the former CEO of Red Hat.

    Shimel: Uh huh.

    Cornett: And he also talked about this fashion statement thing where people, open source is a success. I mean, everything you use on the Internet, IT today is open source for the most part. So, it’s clearly won, right? That battle’s sort of over, and people said, “Hey, I wanna glom onto this.”

    And so, there was sort of a group of companies that attempted to co-opt the branding around open source without really understanding the nuance of what it meant to be a steward of the community and collaborate well. And in some ways, I think that sort of provided a skewed vision. And we’ll avoid that at Datical. You know, again, we’re gonna be good stewards of the community. We recognize that it’s gotta be participatory. Most of the community members are people that have jobs at companies you would recognize in IBM or SAP or other places like that. And they’re taking time to be part of this community, because it provides some value to what they’re doing and their professional life.

    And so, we’ve gotta recognize that we have to contribute to that value, attempt to sort of usurp what they’ve done, and go off and monetize it without the, never work, because if it really is important functionality, someone will just do a fork and monetize that around.

    It’s really about providing these synergies—and I used this phrase earlier where you have a meritocracy of ideas and that sorta lifts everyone up, right? So, we can provide higher value to customers that need the specifics of an enterprise offering or as we have brands, as you say, Liquibase Pro. And then the people that are, they’re part of the community for their specific needs that they’re getting multiples of return also on that investment for the problems they’re trying to solve.

    And it really is sort of this, community is the perfect word for it, because it really is this joint effort to make everyone better off.

    Shimel: Great. Dion, we’re running out of time. I wanted to give you one more thing to chew on. I want you, in your mind, to fast forward six months, 12 months, take your pick—what effect have you had on Datical?

    Cornett: You know, I think that we have to look at—so, part of what we’re doing is, we’re taking a blank sheet and we’re saying, hey, if we were gonna start today and look at the trends in the market, cloud, microservices, NoSQL, NewSQL, you know, these are all things that weren’t necessarily the case when Liquibase was originally founded by Nathan 10 years ago, what would it look like today versus where we are? And where are the gaps between the perfect DevOps solution for database and what we have in the community today, and sort of how do we help guide and steward that into a place that really solves the problem that we’re facing and we’re going to face as the IT infrastructure continues to evolve?

    And so, I think, if we can look back in a couple years and say, “Look, this has been the evolution of the product, and it’s much easier to use, it’s much easier to implement, the time to value is relatively quick,” and it’s really allowing, at the end of the day here, we’re trying to let database keep up with all of the DevOps advances that have been made around the application. If we see the database not being the drag on application, timeliness, speed, effectiveness that it can be today, then I know we’ve been successful in sort of this re-instillment, re-engagement in the community, and pushing that innovation into the product.

    Shimel: Yep. I agree with you. Dion, I promised I’d get you out of here in 15 minutes. I know you have another appointment. But I want to invite you back. Maybe we’ll do a video next time and we can dig in a little deeper. And, you know, I love talking about open source, as do you, it appears, and our audience does, too.

    Cornett: [Laughter]

    Shimel: So, I think it’s a great topic for us to continue. I wanted to wish you all the success in the world, though, with Datical. It’s a great bunch of folks over there, and you’re a great addition to the team.

    Cornett: Oh, thank you very much. I appreciate the kind words, Alan. And yeah, I look forward to future conversations. It’d be great.

    Shimel: It certainly well. Alright, Dion Cornett, new president at Datical, joining us here on DevOps Chat. This is Alan Shimel, and you’ve just listened to another DevOps Chat. Have a good day, everyone.

    — Alan Shimel

  • DataStax to Support Multiple Distributions of Cassandra

    DataStax to Support Multiple Distributions of Cassandra

    DataStax today announced it will provide support services for any distribution of the open source Cassandra database via a subscription offering dubbed DataStax Luna.

    The company also announced it has made available two additional free tools in the form of DataStax Apache Kafka Connector for integrating Cassandra with the open source real-time streaming engine, and Bulk Loader to make loading and unloading data into Cassandra faster and simpler.

    Based on a wide column store architecture, Cassandra has emerged as a popular NoSQL alternative to traditional distributed databases because the column store on which Cassandra is based can also be organized into rows made up of multiple columns.

    Previously, DataStax only supported its own distribution of Cassandra alongside an enterprise variant through which it makes available additional features. There’s also a managed service, available in beta, through which DataStax will run Cassandra on a public cloud on behalf of customers.

    Created at Facebook, Cassandra is currently maintained under the auspices of the Apache Software Foundation (ASF). DataStax was subsequently created to provide a curated distribution of Cassandra alongside the distribution made available by ASF. Cassandra also makes use of Cassandra Query Language (CQL) as an alternative to SQL for processing queries. DataStax is now the largest contributor of code to the upstream Cassandra project.

    DataStax CTO Jonathan Ellis said that as the adoption of Cassandra has increased, there is now enough critical mass of multiple distributions of Cassandra to support. DataStax is now offering support via community and enterprise editions of DataStax Luna.

    Adoption of DataStax largely has been driven by developers. However, as the number of instances of Cassandra deployed increases, IT teams are finding they don’t have the expertise required to manage yet another database platform on their own. DataStax Luna provides a subscription service through which IT teams can rely on support from DataStax regardless of what distribution or version number of Cassandra they are running, said Ellis.

    In many cases, Cassandra databases are being managed by DevOps teams rather than traditional database administrators (DBAs). However, it’s not clear whether more DBAs will be expanding their roles to include managing alternatives to traditional relational databases. Many organizations are now coming to terms with the need to support multiple types of databases. In fact, in many cases, the management of databases on which modern applications are built has been incorporated into a larger DevOps team.

    Regardless of who manages the database, Ellis said the most important thing for any organization embracing Cassandra to understand its data model. As organizations look to process analytics alongside transactions in real-time, Cassandra affords IT organizations a lower-cost alternative.

    When it comes to databases, there is no shortage of alternatives. The days when organizations standardized on a single database are, for all intents and purposes, over. The challenge now is defining the best DevOps practices that will be required to manage multiple database engines as instances of each type of database continue to proliferate.

    — Mike Vizard

  • Should Your Company Use DBaaS?

    Should Your Company Use DBaaS?

    From SaaS to IaaS, companies are rapidly moving to the cloud to gain huge agility and cost benefits. So, is it time for your company to take advantage of database as a service (DBaaS)? Here’s a high-level discussion to help you along your decision path.

    First, it’s important to distinguish DBaaS from simply running a database on a cloud-based server, such as Amazon AWS or Microsoft Azure. While you can certainly gain some cost and agility benefits from running your database on a virtual machine, you’ll still need to provision and maintain the database, that is, you will need server operations expertise.

    With DBaaS, the cloud provider also provisions and maintains the database, eliminating the need for your organization to maintain server operations expertise in-house. DBaaS manages the entire environment—physical and virtual hardware, operating system, networking, replication configuration, etc.—so you can stay focused on software development. With the click of a button you can set up the database and then connect your application with a simple API call. And you’re ready to go.

    The speed and cost advantages of this approach can be huge, but there are also some limitations, so it’s important to assess whether your company’s resources and your project’s requirements align with the current capabilities of the various DBaaS offerings.

    Reasons to Use DBaaS

    Lack of Expertise: As suggested above, if your technical team is mainly composed of software developers who won’t be able to manage the complexity of provisioning a database, you should consider DBaaS. In addition to handling the management of the database, the DBaaS provider will offer additional services, such as automatic backups of your data. In this worry-free environment, you can focus on optimizing your software instead of constantly worrying if the database is functioning properly.

    Short-Lived Databases: If you need databases to serve as back ends for continuous integration testing, DBaaS will allow you to rapidly provision the databases, populate them with test data and then delete them when they are no longer needed. Having the database provisioned for you makes it far faster and simpler to build the required automation for the testing environment.

    A Microservices Design: DBaaS can be particularly helpful, even in a production environment, if you are using a microservices design composed of a very large number of databases. Even if you have the internal expertise, you may not want to invest the time and effort in deploying all those databases. DBaaS can get you into production significantly faster.

    Reasons Not to Use DBaaS

    While the above benefits are appealing, you likely won’t want to use DBaaS in the following scenarios.

    Database-Centric Applications with Complex Topologies: To ensure ease of deployment, DBaaS providers limit the ability to deploy complex environment topologies. This reduces the flexibility DBaaS customers have. For example, let’s say that for disaster recovery, you have a primary synchronous replication cluster in one region and a secondary synchronous replication cluster in a different region, with an asynchronous relationship between them. This relatively complex topology is outside the scope of most DBaaS offerings.

    IO-Bound Workloads: If your database has IO-bound workloads, you may need very powerful hardware, including high-end CPUs, terabytes of memory and very fast storage. In this case, you may not find a DBaaS provider that offers a configuration that will meet your needs.

    Understand Your Options

    If your database requirements align with the capabilities of DBaaS, you now have the ability to eliminate significant operational complexity and develop, test and deliver your applications faster and at a lower total cost. However, make sure you spend time carefully investigating today’s DBaaS offerings, including Amazon RDS, Google Cloud SQL and Azure SQL Database, which all have different features and focus, including deployment models, performance trade-offs, instance types and more.

    If your needs don’t fit current DBaaS offerings, you’ll need to provision your databases yourself. If you don’t have the expertise in-house and can’t afford new hires, a database consultant may be a cost-effective strategy for getting the support you need.

    — Tyler Duzan

  • BMC Extends DevOps Reach to Db2 Databases

    BMC Extends DevOps Reach to Db2 Databases

    BMC Software is extending the reach of its campaign to bring DevOps to the mainframe to the IBM Db2 databases that run on the venerable platform.

    John McKenny, senior vice president, ZSolutions Strategy and Innovation at BMC, said BMC AMI DevOps for Db2 makes it possible for DevOps teams to automate changes to the Db2 schemas as part of a pipeline being employed on a Jenkins continuous integration/continuous development (CI/CD) platform.

    As part of its effort to bring DevOps processes to the mainframe, McKenny said BMC is trying to bridge the historical gap between DevOps teams and database administrators. Many DevOps teams have tried to bypass DBAs that they often perceive to be to slow to keep up with agile development processes. In mainframe environments, however, Db2 databases remain the primary source of data for applications. BMC AMI DevOps for Db2 automatically discovers and determines database schema change requirements, streamlines the review and approval process and implements those changes, he said.

    BMC AMI DevOps for Db2 is part of a portfolio of BMC automated mainframe intelligence (AMI) tools that leverage a policy engine to enforce best practices for database schemas. Those tools play a critical role in enabling organizations that continue to rely on mainframes to capture those best practices at a time when many of the IT professionals who defined those best practices are retiring, said McKenny, adding BMC will evaluate whether it will add support for other mainframe databases such as IMS based on demand.

    It’s unclear how much or even whether organizations that rely on mainframes are embracing best DevOps processes. Providers of tools for mainframe environments such as BMC are betting digital business transformation initiatives will require those organizations to replace their current application development and deployment tools and processes with DevOps processes, which are more agile. The challenge, however, is overcoming the perception that existing waterfall approaches to application development and deployment result in more reliable software. As a result, many organizations that rely on mainframes are trying to determine which methodology to employ based on the nature and criticality of the application workload.

    At the same time, BMC is investing in various forms of artificial intelligence (AI) to automate IT operations, otherwise known as AIOps. Naturally, BMC is not the only provider of IT management frameworks with similar ambitions, but it contends that after decades of providing tools to manage both mainframe and distributed computing environments, it’s in a better position than its rivals to aggregate all the data required to drive the models that are at the core of any AIOps platform.

    Of course, it remains to be seen to what degree IT organizations are comfortable with AIOps. Beyond the perceived threat to their jobs, many IT professionals are skeptical AI models are sophisticated enough to automate complex IT processes reliably. The one thing that is clear is that with the simultaneous rise of DevOps and AIOps, the way IBM platforms have been managed is about to change utterly.

    — Mike Vizard

  • Lessons Learned From the Salesforce Outage

    Lessons Learned From the Salesforce Outage

    Everyone has bad days. Every company has been through some kind of outage due to a buggy database deployment. Even the best of the best, with highly trained staff, world-class best practices and well-thought-out processes make mistakes. On May 17, Salesforce.com had a bad day.

    What Happened?

    The company deployed a faulty database change script that broke permission settings in production and gave users read and write access to restricted data. This opened the door for an unauthorized employee to steal or tamper with the company’s data. As a result, Salesforce needed to take large parts of its infrastructure down to find and properly fix the issue. The outage lasted 15 hours, 8 minutes. According to Gartner’s cost-of-downtime formula ($5,600/minute), this outage cost approximately $5 million. Plus, since so many companies rely on Salesforce, it was a very visible and embarrassing outage. (Just take a look at #SalesforceDown and #permissiongeddon on social media.)

    Salesforce had to shut everything down because of the way databases work. It’s not as easy as pulling a single application. Who knows how many Salesforce employees worked like mad to take the whole database down, find the offending database script, and restore everything—all because of one change script. That’s not a fun way to spend a weekend.

    What Does This Tell Us?

    Historically, Salesforce customers have experienced very little disruption in service. On the day of the outage, many loyal customers were tweeting about how rock solid the service has been, and that’s impressive.

    That being said, this outage should be a wake-up call for users to realize their dependency on their platform, which has become a more integral part of how we conduct business. I’ve heard anecdotes of entire offices being unable to complete work that Friday.

    The customer reactions show that they clearly have their stuff together over at Salesforce. What this outage shows is less about any shortcomings of this company specifically; but rather that everyone has blind spots, no matter how robust the testing process is.

    Lessons for IT professionals:

    • Don’t forget the database. Unfortunately, this problem is all too common. There are countless cases of a database change script being executed in production and causing unexpected issues. This indicates that the database is often the forgotten part of the software test and release cycle. Unless companies start to seriously consider the database when it comes to change management, things like this will continue to happen.
    • Automate everything. Manual efforts fail. Humans make mistakes. Companies such as Salesforce that handle sensitive customer data should not put it on the internet unless they have automated every aspect of the system, including the database. Companies need a robust DataOps process, one that includes production-like data and automates the validation database changes.

    Lessons for end users:

    • Establish a continuity plan. Organizations that depend on systems such as Salesforce need to have a business continuity plan to continue operations if they go down. This includes syncing Salesforce tasks and calendar with Office365, Exchange or Google Apps. However, since Salesforce has become the system of record for most companies that use the technology, having a backup of the data will be very difficult—after all, the point of a company using the platform in the first place was to avoid hosting its own data.
    • Demand accountability. Additionally, users can and should demand more accountability from software vendors. This can take the form of demanding a refund, but also customers need to speak to their account manager to understand what steps Salesforce is doing to make sure this does not happen in the future. It’s 2019 and companies update software all the time without taking down their production systems at the end of a quarter. Moreover, companies also can demand that companies don’t change production systems, outside of necessary security issues, during business hours.

    The bottom line is that, while the mistakes that led to the Salesforce outage are very costly and highly visible to customers, they are also entirely preventable.

    — Robert Reeves

  • 4 Steps to Introduce DevSecOps to Database Development

    4 Steps to Introduce DevSecOps to Database Development

    Although DevOps was implemented within application development, there is a growing awareness that including the database within DevOps processes is vital to success. We live in a data-driven world and many front-end applications use a database at the back end to collect, process and store customer and user data.

    The faster development of applications that DevOps encourages also results in more frequent updates to databases. By excluding database development from DevOps, the database becomes a counterweight to the speed of releases that otherwise can be gained.

    This is already being recognized, as demonstrated in the 2019 Database DevOps Survey from Redgate, which found that 77 percent of application developers are now responsible for database development and 75 perent build database deployment scripts.

    All of which is welcome news, provided DevSecOps for the database is also taken into account.

    While DevSecOps is relatively new (the term was coined in 2012 by Gartner analyst Neil MacDonald), the need for it is a widely accepted facet of DevOps because DevOps radically changes how companies tackle security in application development. Whereas security used to be reviewed and certified at the end of long development cycles with a scheduled window, that window has now disappeared. Hence the rise of DevSecOps, which bakes security into the development process, with tools such as static code analyzers and open source code vulnerability scanners testing code as soon as changes are made.

    In the same way that the database must be included within DevOps if the process is to be seamless, companies also need to ensure that DevSecOps is applied to the database. In fact, given the importance of protecting personal data in today’s world, ensuring privacy and security is imperative for both compliance and public reassurance.

    In 2017, MacDonald continued his work on defining what DevSecOps is when he set out 10 steps that companies need to follow to protect applications during development. All of these are equally relevant to DevSecOps for the database, with the exception of the second step: Quit trying to eliminate all vulnerabilities during development. Many databases contain personal data, much of which is sensitive, so every vulnerability during database development has to be addressed and resolved.

    Looking in detail at Gartner’s list, I’d say there are four key steps that particularly apply to DevSecOps for the database:

    Identify the Vulnerabilities

    At the center of DevSecOps for the database is the data, which has to be protected from both internal threats such as negligence or accidental exposure, and external threats such as hacking. The biggest problem with data, however, is its size and sprawl. Data is growing at an annual rate of 58 percent, according to IDC, and is now stored in a multitude of different places.

    To identify what the vulnerabilities are, the data first needs to be located and classified and its risk identified. Data mapping exercises such as this can be done by hand or with a third-party tool, but it’s important to create a record of every database, including copies used in development or testing, to gain a real understanding of where data is stored and who has access to it. Classifying the data then can be undertaken to identify which data is sensitive and at risk, such as credit card details, and therefore needs to be protected.

    Remove the Vulnerabilities

    As we’ve seen, the recommendation to not try to eliminate every vulnerability in application development doesn’t apply to the database. Where data has been identified as sensitive and at risk, access must be limited and it cannot be widely shared.

    Developers, however, often need a copy of the production database in their development, test, or QA environments to ensure changes will work once deployed. However, those same production databases invariably contain sensitive data that needs to be protected.

    One solution is to create a version of the production database with a limited dataset of anonymous data. However, this means it is neither realistic, nor of a size where the impact on performance can be assessed. A second solution is to take a copy of the production database and mask the data manually, but this will age quickly as ongoing changes are deployed to the production database.

    Therefore, data masking measures such as pseudonymization, encryption, anonymization and aggregation should be adopted, and companies can use third-party tools to help mask database copies, yet also provide development teams with test data that looks—and behaves—in the same way as real data.

    Introduce Secure Coding 

    While application development typically uses an imperative programming language such as C# or Java, database development relies on a declarative language, such as T-SQL for SQL Server, which is less strict and where the coding standards vary widely.

    This has resulted in developers who code in T-SQL having their own styles and preferences. With multiple developers writing updates, this often ends up with code that is confusing, difficult to understand and with errors that can cause deployments to fail.

    To overcome this, code should be written in the same style, with a secure approach that follows set standards, put in place by the database administrator. Introducing this secure coding mindset at the point code is written will prevent problems later down the line.

    Automate Where Possible

    The ninth point in Gartner’s list is to implement strong version control on all code and components, with developers checking their changes into a common repository during the development process, preferably at the end of each working day. As a result, everyone has access to the latest version of the application, and it is always clear what was changed, when it was changed and who changed it.

    This is just as true for the database code, which also can be version-controlled, preferably by integrating with and plugging into the same version control system used for applications.

    Importantly, once version control is in place, processes further along the development pipeline can be automated. Continuous integration can be introduced, for example, to automatically test changes as soon as they are committed and ensure they are not breaking changes.

    Data has been described as the 21st century equivalent of oil, with its use central to success in every organization. Given its sensitive nature, protecting it is equally vital. That’s why all businesses and teams need to ensure they are not only including the database within DevOps, but within DevSecOps as well.

    — Matt Hilbert