devopsy.kinsta.cloud

Category: Blogs

  • ‘Old Man Yells At Cloud’

    ‘Old Man Yells At Cloud’

    These days, software programmers are increasingly estranged from hardware. Most millennials and Generation Z developers have never had to mount physical servers. They have never been lulled by the gentle hum of a server room chugging along in synchronous harmony. Today, you don’t even necessarily have to build your own PC or worry too much about disk space.

    Instead, cloud-native software developers conveniently offload the server burden to cloud service providers (CSPs). Hosting in a cloud, like AWS or Azure, for example, makes a lot of sense — it allows programmers to hone in on their own specializations. Elastic computing models are cost-effective for startups, and CSPs typically offer a vertical stack with helpful integrated platform-as-a-service (PaaS) and software-as-a-service (SaaS) offerings. However, what are we sacrificing?

    Well, first, research shows the cloud model becomes less cost-effective as a business scales. Second, as more companies transform into technology platforms, cloud usage could better enable their competition. Third, more direct access to bare metal could be an attractive option for certain service providers to increase ownership and control workload optimizations.

    These arguments are bolstered by large companies such as Dropbox, Facebook, Walmart, and others that chose to cut their cloud allegiances in favor of building their own data centers or adopting colocation (in other words, running your own hardware in someone else’s data center, also known as “colo”).

    I recently met with Zach Smith, managing director, bare metal, Equinix, to understand the current state of hardware-as-a-service and the future of hybrid server models. He believes that in the coming years, improved interfaces that access low-level infrastructure will arm IT with better consumer choice that brings them closer to the metal. Yet, to enable this, the hardware world will need to take some lessons from the upper stack regarding transparency, open source community, programmability and improved developer experiences.

    Sidestepping the Cloud

    Public clouds arose as a convenient one-stop-shop to host entire software application environments. As a result, “we have a generation of developers who will never physically touch that hardware,” Smith said.

    The cloud infrastructure services market was worth $39.8 billion in Q4 2020, estimates according to estimates from Statista. Having witnessed the “movement toward verticalization of hyperscale clouds,” Smith now sees the infrastructure-as-a-service (IaaS) market as an industry of high margins and escalating fees. This is echoed by others who seek to reduce rising cloud expenses.

    Though cloud computing usage is growing considerably, the data center isn’t dead. Instead, computing models are morphing and becoming hybridized. “By 2025, 85% of infrastructure strategies will integrate on-premises, colocation, cloud and edge delivery options, compared with 20% in 2020,” predicts David Cappuccio, distinguished VP analyst, in a Gartner report.

    As hybrid multi-cloud approaches emerge, Smith notices a resurgence of attention being paid to infrastructure. Some massive service providers are starting to construct their own data center estates; others, such as Box, now own and lease servers from data center providers.

    Benefits of Bare Metal

    As mentioned above, there are a few reasons to move back to the metal. First, it may simply be more cost-effective for large deployments. In his well-researched article, Antoine Beaupré compared monthly recurring costs for colocation versus an AWS bill, and found the break-even point rested at $100,000 per month, or 150 Amazon m4.2xlarge instances. His research also found that I/O-heavy applications are more costly in the cloud than via colocation.

    “While many services simply launch in the cloud without the organizations behind them considering other options, large content-hosting services have actually moved back to their own data centers,” Beaupré said.

    In addition to cost savings, automating colocation with a programmable layer could also offer more fine-grained control over computing optimizations. “I believe what we’re creating is an operating model for a rapidly changing hardware substrate,” said Smith. He described how bare metal opens up a neutral infrastructure substrate to control physical networking aspects such as cross-connects, IP transits and peering — a benefit Smith half-jokingly called ‘NetDevSecOps.’

    Another argument is that investing in hardware could, ironically, bring sustainability benefits. A report conducted by ITRenew estimates that, of the total embodied energy required within a server’s entire lifespan, 77% is used to manufacture the hardware itself. In contrast, the operational energy used over a server’s lifetime only accounts for 22%.

    Thus, there is an environmental benefit to reversing the trend of a make-dispose economy continually producing new servers. Instead, hardware ownership could encourage a more circular economy. “A circular economy aims to end our take-make-dispose economy, which is a very linear approach, and instead, keep products at their highest possible value for as long as possible,” according to the ITRenew report.

    Learning From the Upper Stack

    How can we build a fundamental non-opinionated infrastructure platform? According to Smith, the industry needs “hardware and networks that are programmable and usable by a developer generation.” To Smith, changing firewalls at a kernel level, altering how the runtime works or controlling the IP stack, should all be operational via API commands.

    Yet, hardware accessibility is still rather opaque and, historically, not as developer-friendly. “How can we take abstraction and community-driven ideas from up the stack?” Smith asks. To enable these changes, he believes the hardware world should model itself on the upper stack in a few key areas:

    1. Transparency: The hardware world is not as easy to comprehend. “When you get to the physical side of switches and routers, it’s still arcane,” Smith said. Increased transparency in functionality and ownership will be required to turn servers into something all developers can operate.
    2. Open-source community: In the same way the software world is built on strong developer communities, hardware will require an open community to normalize things like firmware, boot order and support for differences between drives.
    3. Programmability: “Self-service and programmability are requirements for most things these days — there’s no difference with physical infrastructure,” says Smith. Take the rise of the API economy — today’s mantra is, essentially, if something is possible in a UI, it should be possible through an API. Similarly, if it can be done in a server, it should be controllable with an API.
    4. Quality developer experience: Self-service, programmable infrastructure is just the beginning — interfaces to operate infrastructure require good developer experience. Smith feels that hardware requires an improved buying experience and better life cycle management capabilities. This could be enabled by open source tools like Tinkerbell, a provisioning and workflow engine supported by Linux Foundation.

    Back to the Metal — A Platformer’s Dream?

    All this is not to say the public cloud isn’t valuable – it absolutely is. The cloud has been necessary and has proven resilient enough to support online business and digital life as we know it throughout the pandemic and for years prior. “In 2020 and beyond, most organizations are going to be transforming into a completely cloud-based environment,” said Werner Vogel, CTO, Amazon, during an online AWS summit.

    But, even if most organizations are in the cloud, that doesn’t mean all of their assets will be. Service providers, especially, may cling to hybrid models for more control, to avoid supporting the competition and, perhaps, even to save money. Private data centers could also increase security. In an era where raw computing power is becoming a new asset (quite literally, in the case of cryptocurrency), the server rack is starting to look sexy again.

    “The magic happens when software and hardware work together,” said Smith. But, giving access to physical technology to a generation that has never touched it will require some ingenuity. As Smith says, this could be done by “automating colo” through more usable, standardized interfaces for hardware management.

    “We will see a future in next 10 years where operated software is separated from cloud infrastructure,” Smith said. That’s a bold statement, as such a dramatic change would, in effect, usurp the current cloud provider IaaS market share, altering the distribution and economic model associated with global software.

    While this dream may seem far-fetched, if more platforms decide to cut out the cloud middleman and go it alone, it could materialize.

  • An Eternal Truth

    An Eternal Truth

  • Atlassian Extends Open DevOps Ambitions

    Atlassian Extends Open DevOps Ambitions

    At an online Atlassian Team 2021 event this week, Atlassian launched a two-pronged effort to provide organizations with more insights into software development workflows constructed using tools from multiple vendors.

    Suzie Prince, head of product for Atlassian, said an Open DevOps initiative will tighten integration between the Jira project tracking and management tools provided with Atlassian and a wide range of tools commonly employed by DevOps teams. The goal is to not only make it easier to correlate issues that are listed in Jira with artifacts found in various software platforms, but also to collect data from DevOps tools to automatically update Jira.

    That data will then be employed to inform the key performance indicators (KPIs), based on graphs that Atlassian provides within Jira, to keep track of value streams. Rather than having to acquire a separate value stream management (VSM) platform to track software development progress, the insights needed to understand the impact of, for example, a delay to one project becomes a dividend of using Jira.

    At the same time, Atlassian is moving to making it easier for other stakeholders in an organization to track software development projects. A Jira Work Management application is specifically designed for business users that require a higher-level tool to track projects. At the same time, Atlassian revealed this week it has acquired ThinkTilt, a provider of low-code application development tools optimized for Jira.

    Atlassian for the first time this week revealed that half of Jira users are not software developers. As organizations increasingly appreciate the degree to which they are now dependent on software to drive digital business transformation initiatives, Atlassian is adding a set of project tracking applications designed specifically for users that need to consume Jira data in a more intuitive fashion.

    Prince said Atlassian is committed to making it simple to replace one DevOps tool integration for another with a single click. Rather than requiring organizations to standardize on a single DevOps platform, Prince added that developers will be able to continue to rip and replace tools and platforms as they see fit without compromising visibility into application development workflows.

    In addition, the integrations being provided by Atlassian will reduce the time and effort required to configure all the tools that make up a DevOps workflow, from roughly 60 steps today to about six.

    As DevOps workflows become more pervasive, the way organizations operate is fundamentally changing. Formal product launches are giving way to a steady stream of software updates that add new capabilities to any platform with little to no fanfare. End users ranging from sales teams to custom support personnel now need to track updates to platforms more closely as new capabilities are continuously added.

    Of course, not every organization has the same level of maturity. However, as competitors start to add new features to rival offerings, it is now only a matter of time before most business processes – that increasingly revolve around software updates – begin to evolve and change. As the pace of software development continues to accelerate, the pressure on legacy business processes will only continue.

  • Make Metrics Matter: Measure What’s Important

    Make Metrics Matter: Measure What’s Important

    In a data-driven society, organizations rely heavily on sets of metrics to evaluate how they’re doing on a particular task, initiative or resource allocation. Metrics are particularly important to DevOps teams, whose very function is tied to concepts like collaboration and continuous improvement.

    But, while metrics can help drive efficiencies, they can also drive DevOps teams down the wrong path. Just because you’re measuring specific things doesn’t mean you’re measuring the right things. And if you add more metrics, it doesn’t necessarily mean you’re creating more value; you may just be spinning your wheels faster and getting nowhere.

    The key is choosing the right metrics that drive the right outcomes for your organization. But how do you choose the right metrics? And how do you measure the true value you’re creating? Organizations that successfully navigate this challenge will not only build successful DevOps cultures but also improve their overall competitiveness in an increasingly demanding business climate.

    Be Measurable – and Be Actionable

    First, choose metrics that are objectively measurable. Eliminate individual subjectivity from your metrics palette. Rather than ask “How beautiful is my UI?” focus on measurable data such as, “Is the UI accessible to people with disabilities?” or “Did it drive more clicks?”

    It is important to choose metrics that are actionable, not just “vanity metrics.” Back in the 1970s, McDonald’s used to update the signs outside its restaurants with actual numbers: “3 Billion Served;” “4 Billion Served.” Now the signs read, “Billions and Billions Served.” That is the ultimate vanity metric – it’s not easily auditable and not actionable. How many billions? Should McDonald’s be aiming for trillions? A good example of a DevOps vanity metric is server uptime. In fact, it’s actually a contraindicative metric: with cloud instances playing such an important role, you could make the case that low server uptimes are actually preferable.

    Metrics should focus on inter-team and intra-team performance and outcomes. DevOps is all about cross-functional teams — development, test, operations and information security, among others — working together to deliver business value. So, rather than focus on individual metrics like “lines of code,” focus on outcomes like “cost of customer acquisition,” “new customer signups” or “overall revenue.” It may not always be obvious how to move the needle, but that is where experimentation comes in.

    Avoid the Noise

    Focus on as few metrics as possible and maintain a high signal-to-noise ratio. When everything is a priority, nothing is. If you’re presenting your metrics to the board of directors and you have 50 different graphs on the screen, you’re focusing more on how the presentation looks than on showing real, tactical information. Choose metrics that align with your current goals and be ruthless about ensuring those metrics are accurate, timely and focused on a strategic goal.

    Expect goals – and the metrics you track – to evolve over time. You may want to shorten feature delivery time, but to do that, you may start with measuring things like build time, regression testing time, deployment time, etc. You will be impacting the feature delivery metric but doing it in a roundabout way. The key strategy is to pick an outcome you want to achieve. Once you knock down that problem, you can keep that metric around, but you’ll want to push it down. You can put an alert on it so if it ever crosses a threshold where you don’t want it to be, you can act on it.

    Engineering organizations should focus on four to five metrics that they actively are attempting to improve at that time, so they can achieve those outcomes. As they achieve those outcomes, they can retire them or put those metrics in the background. Then, they can pick the next one, identify the next outcome to achieve and assign some new metrics around that.

    Align Metrics With Your Priorities

    Metrics can be situational. If you work in the network ops center, there are going to be some things you want to have up all the time. You’ll want to know the second a denial-of-service hits or the second error rates start going up. But for a software delivery pipeline, you can’t just throw out a bunch of metrics and hope the right ones will find their way to the forefront. You want to focus on metrics that align with your priorities today, next week, next month, this quarter or this year.

    So, which metrics are most important for a DevOps organization to pursue? A series of metrics identified by the Google DevOps Research and Assessments (DORA) team has emerged as the industry standard to determine how successful a company is at DevOps. These measurements include deployment frequency (DF); mean lead time for changes (MLT); mean time to recover (MTTR) and change failure rate (CFR).  These metrics show how well development teams deliver better software, faster, to their customers.

    I would add a fifth metric: mean time to discover. From the time a problem is injected into a project, how long does it take for us to discover it? Did our organization discover it or did the customer? From there, we can strategize about the cheapest possible way we could’ve caught a particular bug ourselves. Could we have written a five-line unit test that takes 100 milliseconds to run that would have caught that? Or was it more complicated? Was the UI misaligned so that it was impossible to catch that automatically?

    Different segments of the software delivery process will be concerned with different sets of metrics. While the Dev/CI practice focuses on development lead time, cycle time and work in progress/technical debt, the QA team pays closer attention to idle time, defects discovered and escaped, and the aforementioned mean time to discover. While the deployment group aims to improve deployment frequency and change success rate, operations gets jazzed from studying the cost and frequency of outages.

    Metrics are useful tools for DevOps organizations. They can push teams to be more efficient and more accountable. But to truly optimize the value they’re getting out of people and particular initiatives, it’s important to choose metrics carefully. Recognize that metrics aren’t a one-size-fits-all mechanism, and that more metrics doesn’t always equate to better performance. Choosing the right metrics – and reevaluating them on a consistent basis – is the best path to long-term improvement.

  • Successful Digital Transformation: It’s About Strategy

    Successful Digital Transformation: It’s About Strategy

    While having a mature digital enterprise helped organizations to better survive – and even thrive, in some cases – the pandemic, that’s not the primary reason why organizations will continue to invest in their digital transformation efforts. According to new research from consultancy Deloitte, businesses will continue to invest in their digital transformation capabilities to innovate rapidly and to survive competitive battles in the marketplace.

    Deloitte’s survey found that enterprises believe such ongoing investments in digital transformation will continue to deliver benefits, such as enabling them to innovate more quickly, modernize their business-technology systems and become more resilient. The pandemic proved an unfortunate test of such business resilience, with more than 75% of respondents saying that their digital capabilities helped them better meet the business challenges posed by the pandemic.

    The survey also found that enterprises plan to increase their digital investments and to do so considerably. Respondents indicate, on average, planned investment of nearly $13 million on digital transformation in the next year — a 15% increase from the prior year’s spend.

    These findings mirror those of another survey released this week from software provider Rimini Street, which found digital transformation is one of the top five priorities among CFOs, and that 71% of CFOs believe that such investments are essential to the success of their businesses. Further, 77% of CFOs said they would work with their CIOs to help find funding for digital transformation efforts — if there was a strong ROI demonstrated.

    Perhaps most noteworthy from the Deloitte survey is that, while digital platforms and digital transformation efforts are essential, they are not a competitive advantage by themselves. It’s what businesses do with those platforms that matter.

    Additionally, most Deloitte respondents see plenty of competitive upheavals ahead: most respondents feel it is most likely that their main competitors will be a startup or current digital native competitor in as little as five years. Less than one-third of respondents believe that their current competitors will remain competitors five years from now.

    To win funding for digital transformation and innovation initiatives, the Rimini Street survey suggests CIOs will most likely succeed by not pushing innovation for its own sake, but by seeking out initiatives that will generate a strong return. In that survey, CFOs made it clear that optimizing existing technology, revenue-generating endeavors and improving processes and staff efficiencies were their top three priorities.

    In that race to continuously optimize business technology systems, many CIOs in Deloitte’s survey also see a downside. More than half of those questioned said that the rapid pace of change is harmful to both their organization and their customers, and 75% said the rapid change had created significant upheaval in business leadership or their board during the past 24 months.

    Whether enterprise business and technology leaders want a world with such rapid change doesn’t matter: it is the world in which they must compete. And those enterprises that succeed in the next few years will be those that not only make the right investments in digital transformation, but those that figure out the best ways to innovate and connect with customers by leveraging those investments.

  • The Value of Attention

    The Value of Attention

  • JFrog Tightens Integration with Cloud Edition of Jira

    JFrog Tightens Integration with Cloud Edition of Jira

    JFrog this week announced that its DevOps platform, delivered as a software-as-a-service (SaaS), can now be integrated with the Jira project tracking and management application, delivered as a cloud service, from Atlassian.

    Casey O’Mara, vice president of worldwide business development for JFrog, said this bidirectional integration makes it possible to link a Jira issue, such as a bug fix, to the binary or build residing in the JFrog Artifactory repository that is part of the JFrog DevOps Platform.

    Available via an application that can be downloaded from the Atlassian Marketplace, this integration extends an existing alliance between the two companies, said O’Mara. There is, for example, already an integration between Atlassian Bitbucket Cloud for continuous integration and JFrog Artifactory repository.

    Creating a link with a Jira issue will make it easier to collaborate by, for example, making it easier to understand why a specific version of an artifact, such as a container image, was created, added O’Mara.

    Artifactory is already able to collect issue key messages from Git that logged alongside builds. The Artifactory dashboard for build information displays a link to each of the referenced Jira issues in the Issues tab. Now, users of Jira can link to the builds in Artifactory.

    Jira remains the dominant project tracking and management application for software development. As more organizations appreciate the degree to which they now depend on software, O’Mara noted the number of stakeholders in an organization using Jira is starting to increase, as application development schedules are tracked more closely. Atlassian is now making Jira more accessible as a cloud service to help foster that increased adoption, noted O’Mara.

    Atlassian is now making a case for transitioning its customer base to the cloud. In the third quarter of this year, it plans to discontinue new server license sales in addition to increasing prices for on-premises products. Ironically, the company reported preliminary results for its most recent third quarter that indicate revenues of between $566 million to $572 million. That’s $85 million more than originally forecasted, because demand for on-premises products has increased in anticipation of their unavailability later this year.

    In general, both Atlassian and JFrog are strong proponents of a best-of-breed approach to DevOps. JFrog, in its most recent quarter, reported $42.7 million in revenue, representing a 39% gain. Cloud revenue in that quarter grew 69% to $9.8 million, representing 23% of total revenue.

    It’s unclear at what rate organizations are shifting to the cloud. The number of workloads moving into the cloud has certainly accelerated in the wake of the economic downturn brought on by the COVID-19 pandemic. However, the bulk of data continues to reside in on-premises environments, and many organizations, because of performance, security and compliance concerns, don’t have any immediate plans to migrate all their applications to the cloud.

    Regardless of where DevOps platforms reside, there will always be a need to integrate them. It just may be easier to achieve that goal when all the tools that make up that workflow are running in the cloud.

  • Data Needs and Challenges with CData

    Data Needs and Challenges with CData

    The data businesses and applications need are everywhere and access through a multitude of technical interfaces, including APIs, database queries, data layers like ODBC, cloud services and SaaS applications (to name a few).

    In this episode of TechStrong TV, Jerod Johnson, technical evangelist at CData, joins Mitch Ashley to discuss the challenges of meeting the data needs applications require to support digital and transformation initiatives.

    The video is immediately below, followed by the transcript of the conversation. Enjoy!

    Transcript

    Mitch Ashley: I’m joined today by Jerod Johnson. Jerrod is technology evangelist with CData. Jerod, good to be talking to you today.

    Jerod Johnson: Yeah, no, it’s my pleasure to be here.

    Ashley: Well, as an old ex-DBA, data is always kind of a passion of mine, interest of mine, so I’m anxious to hear more about CData. Would you introduce yourself and tell us a little bit about what CData does?

    Johnson: Yeah, sure. So, like you introduced me, my name is Jerod Johnson, I’m a Technology Evangelist for CData Software. I’ll start with CData. So, we call ourselves a leading provider of data connectivity solutions. In short, what that means is, we make it easier for every user in an organization to connect to their data, which lets them focus on the job of finding value from that data or building applications on top of that data instead of having to focus on the integration, instead of tasking IT teams to build connectors or pipes or flows, whatever the case is, in order to work with the data that they need to drive business.

    Ashley: Mm-hmm. You are number two on my list of what are the reasons people come to IT when they’re building apps.

    Johnson: Yeah.

    Ashley: And that is, security, data, and other apps.

    Johnson: Yeah. [Laughter] 

    Ashley: So, always need data. Well, tell us a little bit about yourself. I’m curious—how did you become a technology evangelist? How did you—

    Johnson: Yeah, so, I actually spent six years teaching before I got into software. So, my education background was in teaching and computer science, and spent six years in a classroom teaching math and computer science, and just decided that we needed a change, for myself and my family.

    So, I started at CData, I was actually a member of the support team, which meant I was answering e-mails and helping solve customer problems. And then I moved into engineering, and spent some time there, but it really became obvious that there was a need in the company for somebody with a skill set that I had, being able to speak to people, being able to present information, even write technical documents. 

    And so, I was pulled into the marketing team, and I’ve been doing that since 2016 for the company. So, writing technical documents, giving talks at trade shows when those used to happen, through webinars.

    Ashley: When we were [Cross talk], right? Hopefully soon, hopefully soon.

    Johnson: Yeah.

    Ashley: Interesting. You’re not the first person I’ve talked to who’s come out of an education background and moved into technology and, in your case, moving into an evangelist role. That whole skill of being able to explain sometimes very complex concepts, sometimes not so complex, but you may not necessarily have the context or the background for a certain thing that you need to learn or understand—

    Johnson: Sure.

    Ashley: Or even explain to non-technical folks, right? So, a great thing to have, especially in the DevRel world that we all live in now of really focusing on how we work with service and kinda think like a developer.

    Johnson: Sure.

    Ashley: Because that’s the people who are using it, whether they’re business developers or people inside IT. Let’s talk about sort of the data challenges, and I’m curious, maybe you’ve seen some things over the last kinda year plus, too, with the acceleration of digital transformation projects.

    Johnson: Yeah.

    Ashley: It seems that one of the challenges always is data sprawl—it’s everywhere, and every application is creating data, consuming data, sometimes creating data exhaust that’s unused or somebody else might be able to use it. It’s no small feat just to find where it is, more or less find out how you can get to it and use it.

    Johnson: Yeah, no, absolutely. I think there have been studies done, I think programmable web is probably the best place to look if you’re looking for a list of APIs. And I think that they have 22,000 public APIs listed at this point.

    Ashley: Wow.

    Johnson: So, I mean, and each of those APIs, you know, there’s data behind it. So, you have that kind of explosion, the API explosion. On top of that, you’ve got private APIs and then databases, data warehouses, these traditional stores. So, data is everywhere.

    One of the things that we’ve found is that, as data explodes, as businesses transform digitally, you get the opportunity to pick the best service for your needs. So, you know, you’ve picked your CRM application because it meets your needs the best. You’ve picked your ERP solution because it meets your needs the best.

    The consequence of that is that your data is now spread everywhere. So, your CRM data is somewhere, your ERP data is somewhere, your warehousing data is somewhere else. And so, there’s this, as you said, this data sprawl where data is everywhere.

    And what we do is, we make it all look the same. So, we like to say that CData lets you see the world as a database. So, we’ve created technologies that allow users to connect to their data, no matter where it is, as if they were connecting to a database. And so, I’ll drop some buzz words, some acronyms here—so, JDBC, ODBC, ADO.NET, these are all database connectivity standards that we’ve produced that now can connect to things that are not databases.

    And so, you know, what that does is, it really democratizes data access. Now, anybody can get to data, from any of the tools or applications that they’re using. So, whether you’re looking at an IT team or a DBA or something like a citizen analyst or a data scientist, it doesn’t matter. As soon as your data looks like a database, anybody can work with it.

    Ashley: Mm-hmm. That’s interesting you say that citizen data scientist—it’s not just developers. Of course, developers are very familiar with ODBC and other connectivity layers for data access, which are fantastic for developers, but you have to have some pretty specific development skills to be able to [Cross talk] this thing.

    Johnson: Sure.

    Ashley:  A lot of this data is sitting on a Dropbox file server in some cloud application, you know, we signed up for a CRM system—well, guess what, I need to connect that with my workflow for when an order comes in. And, you know, as good as Zapier and things like that are for doing kinda nice integrations for automation, that’s not even gonna cut close to solving these kinda problems.

    And you mentioned, too, making it look like a database. I’m curious, from my experience, it’s how you get to it and making that understandable.

    Johnson: Sure.

    Ashley: But it’s also understanding the data and what it means and how it’s represented, because a customer is not necessarily defining the attributes similar in a CRM that they might be in an order system, there may be some variations in just the format of how we capture that data.

    Johnson: Yeah. Yeah, so, you know, what we do is, everything that we make sits on top of the API. So, your API is gonna have endpoints, and those endpoints are gonna expose fields for entities.

    So, let’s take a customer, for example. When you hit a customer using one of our drivers—hit the customer, well, now, it’s a table, it was an API endpoint. What we’ve done is, we’ve kinda tabularized that data model. So, now, every customer is a row in a database table and then each of the fields for that customer becomes a column.

    And then, you know, what we do from there is basically free you to map the data from one system to another as you need, in the tool that you’re using. So, we don’t force a model on you, we’re not trying to create one universal picture of a customer across your organization. We’re gonna let your business specialists do that part. We just make it easier to see, and because it looks like a database, the schema, the metadata, however you define it is really self-describing. 

    Ashley:  Mm-hmm.

    Johnson: You know, if you can talk to a database, you can point your tool. So, let’s say you’re looking at something like Tableau. When you point it to a database, it’s gonna show you all of the tables that are available and then all of the columns that are available for each of those tables.

    Well, we do the same thing, but for something that isn’t a database. So, it could be a NoSQL store like MongoDB or Couchbase or even Amazon S3. It could be a SaaS application like NetSuite, or even Dynamic CRM. So, whatever it is, you get that full tabular model where you can see exactly what’s there and then you can decide how best to work with that data.

    Ashley:  I mean, think about the advantage of, if you don’t have, necessarily, all the programming skills you might want, to go get that data, just not having to use an ABI, but use the driver instead.

    Johnson: Right.

    Ashley: And so, just get that to me in a table form that I can—now, I can compare how one customer is _____ and one, I’m just making that example, to another customer set of attributes and other applications.

    Johnson: Yeah.

    Ashley: Sometimes it’s hard enough just defining who is—what a customer is. [Laughter] 

    Johnson: Sure.

    Ashley: I remember that modeling. But now, you can see what the data is without writing a whole bunch of software and translation and trying APIs and learning specifics about this particular service or this cloud application or whatever it might be.

    Johnson: Right. Yeah, yeah. And that’s one of the things that we really offer is, we free you from that integration problem. We free your developers, we free up IT resources. So, APIs are constantly changing. I don’t know if you’re familiar with the Facebook API, but I think they give a new version, like, every three minutes. I think it’s realistically like every three weeks, but it feels like three minutes when you’re a developer.

    Ashley:  I try not to be familiar with the Facebook API. [Laughter] 

    Johnson: Yeah, which—that’s fair. [Laughter] But with our drivers, our engineers are the ones that are keeping up with those changes. So, every quarter, we release a new product, and that includes all of the API changes that have happened on the back end. So, the only thing you have to do is literally install a new driver into your system. There are no changes—no breaking changes to the API, we merge everything as much as we can. And it really, as I said in the beginning, it lets your business experts focus on your business, whatever that is.

    Ashley: Mm-hmm. I’m curious, too, is it primarily developers that are using CData and the drivers that you have? How many folks may be from a non-super technical environment, maybe some maybe a little technical skills and might use CData?

    Johnson: Yeah, so, that’s a good question. I think it’s a pretty solid blend. So, you know, one customer story might be a mom and pop shop, they wanna be able to look at their QuickBooks data in Excel, they wanna be able to work with it, print it out, edit it, even, and update it that way. They don’t like the QuickBooks UI, they’re familiar with Excel, and so they simply install our Excel add-in, and now they’re connecting directly to QuickBooks from Excel, with full read and write capabilities.

    We can get more complicated than that with customer stories. We have one customer who was dealing with that data sprawl, the data silos where their data is everywhere, and they needed a solution that allowed them to replicate all of their data into the data warehouse that they were already using for BI and analytics. And so, we have a tool, CData Sync, that allows them to do that. So, connectivity—solving the connectivity problem and solving the replication problem, so they get automated, incremental replication of all of their business data into one place for BI and analytics.

    And then, you know, I touched on this before, but we solved that BI and analytics problem in other ways. So, you know, you have data scientists that are familiar with, that really love their reporting tools, whether that’s Tableau or Power BI or any other tool on the market. They don’t wanna have to change tools just to be able to connect to the data. And with our connectors, that’s exactly what they get to do. They get to stay in the tool that they’re already using.

    Ashley: Mm-hmm.

    Johnson: But when it comes to developers, this is where we kind of get into our most complex customers. Ultimately, the customer is an organization that’s building a data-centric tool or a platform, and their developers, you know, they’re the ones responsible for maintaining the connections to data from those tools and platforms. So, you know, you’re building the best BI tool on the planet but if it only connects to SQL server, is it really the best? So, we’ve had companies come to us and use our drivers to build products. And what it really frees up there is one-off API integration.

    So, if you’re building a tool and you’re connecting directly to the API from your tool, integrating against Salesforce looks one way, integrating against NetSuite looks another way. Integrating against Google BigQuery looks a completely different way. But as soon as you use our drivers, it all looks the same. If you can develop a tool that can talk to a database, you can develop a tool that can talk to anything.

    Ashley: And one of the situations I’ve run across several times is through M&A, when you start bringing companies together through acquisitions and of course—

    Johnson: Right.

    Ashley: – you know, you need information, you need data to operate. I mean, just imagine combining a sales organization across two or multiple companies together and ones in Salesforce and one’s in HubSpot and one’s in homegrown whatever.

    Johnson: Sure.

    Ashley: And, you know, getting that data quickly, you don’t wanna have to go through a whole conversion and, “Okay, now everybody’s in Salesforce” or whatever your tool of choice.

    Johnson: Right.

    Ashley:  Maybe that’s where you go, but you may live six months, two years, where you need data out of multiple systems to see your pipeline, see the sales bundle, see, you know, how we’re closing the deal and what’s the conversion rate and all those kinda things for. And tools like CData are extremely valuable in getting to data very quickly to get value out of that [Cross talk].

    Johnson: Right, yeah. I mean, that’s, legitimately, our own personal story. So, when we first started as a business, we had a homegrown CRM, and then we started incorporating HubSpot. And now, you know, as we grow, as we get bigger, those systems don’t work for us anymore, and so now, we’re using Salesforce. And we’re in a place right now where we are concurrently using all three, and actually using our own technology internally to build reports, to do analytics, and then, as part of that process, to migrate data from one system to another—so, moving our data from our internal CRM into HubSpot and then into Salesforce.

    So, because the drivers are standard based, because they make things look like a database, it’s incredibly empowering for IT teams and DBAs to work with the data and to either consolidate it or migrate it, or leave it where it is, but build up connections to it in a way that lets the other data experts, the C suite, whoever needs to see the data, see it quickly and easily.

    Ashley: That’s a good point. We don’t always wanna take on the work of combining things, migrating things. If you can leave data in place, that makes sense to do.

    Johnson: Yeah.

    Ashley:  Boy, what an asset, because now you can get to it, but—okay, just manage it the way you were. And it’s great that you could relate to that example that I brought up where we didn’t coordinate that, at all.

    Johnson: Yeah.

    Ashley: I’m curious, how about the security? What comes with data, security is always super important to access control, things like that. Where in the stack does that fall, and how does CData play in those roles?

    Johnson: Yeah, so, we will leverage whatever security is available from the API. So, you know, as one example, Salesforce users have different permissions, whether that’s permissions to access different entities, permissions to only read entities, to write entities, whatever the case is. 

    So, when you connect to Salesforce using any of our connectors, you choose the user that you wanna connect through and then everything that they can do, you can now do through our connector. Anything that they couldn’t do before, you can’t do. And then we support the full gamut of authentication protocols. So, if an API supports OWASP, we’ll support it. If an API supports other authentication schemes, even—so, like, your Microsoft based products, on premise, we do NTLM. If you’re looking at Kerberos authentication, we can manage single sign-on—just whatever is available there, we allow.

    And then the protocols that the API uses for security—so, SSL, the encryption, all of that is supported as well, so.

    Ashley: I think an advantage you have, since you’re not pulling the data out of those systems and storing it somewhere else, it’s still data that’s living there—

    Johnson: Right.

    Ashley: – all the security systems, the access controls permissions, policies, et cetera, are still within those applications, however they’re managed, either standalone or collectively across those. So, you’re not having to devise some new data permission scheme from that, right?

    Johnson: Right, right.

    Ashley: That’s gotta be a huge plus.

    Johnson: Yeah.

    Ashley: Cool. What do you see as—while we have some time left, here—I’m interested in what you see as sort of emerging applications. You know, we’ve seen a lot of acceleration with digital transformation.

    Johnson: Yeah.

    Ashley: Digital engagement with customers, automation—there’s many more things happening, and of course, those both need data and create data.

    Johnson: Yeah. So, you know, I think one of the things that we’ve seen of late is just this migration to the cloud, where everything is in the cloud, your data’s in the cloud, you’re doing BI in the cloud. So, through products like Tableau Online or the Power BI service, instead of Power BI Desktop, Google Data Studio, AWS QuickSight, whatever the case is, you know, you’re working with data from a cloud platform now instead of something on premise.

    So, what we’ve created is, we have a cloud platform that creates a cloud to cloud connection. A lot of the tools that exist today, you know, even if you wanna go from a cloud platform to a cloud data store, you’re still required to put a gateway on premise somewhere. You know, this is the case with a lot of the Power BI connectivity. And, you know, if that doesn’t—why would you go down from the cloud, to on prem, back up to the cloud in order to work with your cloud data? And so, that’s—the CData Connect Cloud Platform is what it’s called, but it allows live, real time cloud to cloud connectivity. 

    And so, you know, that’s one of the, kinda the emerging trends that we’ve seen is people wanting to just do everything in the cloud. And it makes sense, especially—you know, I think about this past year, right? Everybody’s working remote, you know, you don’t have an on premise server sitting somewhere that can run your platform, you have to move it to the cloud so that everybody that’s working from home can access it, and that’s just one example of this cloud migration that we’ve seen.

    Ashley: Nothing like a pandemic—I say that knowing the human tragedy side of it, all due respect, but—

    Johnson: Yes.

    Ashley:  – to really force some very aggressive changes in our business and how we operate and some really interesting things. Is there a particular area of, you mentioned cloud and more usage of cloud services—what’s sort of the next frontier, if you will, that CData is tackling of how to make this data problem easier?

    Johnson: Yeah, so, I think—you know, we’re thinking about that cloud integration problem and trying to look for ways to help that. But you know, an area that we’ve started pushing into recently is kinda low code developers. So, I had mentioned you have your IT teams and your full stack developers that are used to working with data, that are used to writing code. But something else that we’ve seen crop up is this low code application development—whether that’s in a cloud platform or an on premise platform. And it’s great. You can build a workflow or an app using a drag and drop menu. These platforms tend to be intuitive. And now, your citizen analysts can build a workflow to combine data to get what they need into their BI tool.

    The problem that we’ve seen there is connectivity to data from these platforms, and so, that’s kind of an area that we’ve been pushing in an integration kind of marketing side is reaching out and trying to connect with low code and shoring up our integration with these platforms.

    Ashley: Yeah, and by the way, that happens to be an area of research that we’re working on with CData, and we’ll talk a little bit more about that here coming up in a few weeks—some really good, interesting findings from that research.

    And that was really interesting to Accelerated Strategies, the analyst firm that I run, because there is so much interesting in low code/no code right now.

    Johnson: Yeah.

    Ashley: And of course, it’s so broad of what that means. It could mean a spreadsheet to somebody, you know, as well as low code or platform, you know, there’s a number of platforms for creating applications and accessing data through a CData kinda combined platform. So, it’s really interesting, to me, of kinda who is using low code technologies to create applications.

    And I think the world used to view it, or maybe currently views it as, those are the business users, non-technical. That’s certainly a population, but it’s more and more coming into IT is what I’m seeing.

    Johnson: Yeah, that’s cool. I mean, as somebody who was a developer, was a support engineer, it’s fascinating for me to get my hands on a low code platform and figure out, you know—well, what exactly can I do with this?

    Ashley: Mm-hmm. Yeah, it’s interesting technology. Another way to solve problems, another technology, right, to build the applications.

    Johnson: Right.

    Ashley: How about the mobile side of this? With so many more mobile apps, we’re almost in an era, maybe we’re in the era of, “I’d like fries with that, I’d like a mobile app with that.”

    Johnson: Sure.

    Ashley: Or maybe it is mobile first. Does that change the dynamics of some of the data needs or challenges?

    Johnson: So, I think once you start looking at mobile applications, you’re looking for lightweight applications, right? You can’t—I mean, phones are impressive now, right? Like, I think about the amount of data that I can store on my phone versus the first smartphone that I got, you know, the amount of space that I have for apps. But at the same time, you know, embedding a software library or a driver or whatever the case is into your app, it doesn’t always make sense. Especially if you’re building an app that supposed to be able to connect to hundreds of different data sources.

    Ashley: Mm-hmm.

    Johnson: You know, are you gonna install 100 different 7 megabyte drivers with your app? Like, is that really the best way to go? And that’s another place where our cloud platform makes sense. So, you can be on your mobile app, you can connect—you can build an application that connects to a cloud platform that then allows users to connect to data from anywhere.

    Ashley: Mm-hmm.

    Johnson: So, you know, using the same technology that we have, it’s just consolidated into one single platform. So, you hit that platform and now, you know, you can pull in Salesforce data, you can write back to HubSpot, you can dump a report as CSV into AWS S3, all through one platform.

    Ashley: And you know, so many of those apps aren’t necessarily greenfield apps, right? All that data, maybe even the transactions processing, much of it is happening in other apps, other cloud locations.

    Johnson: Right.

    Ashley: So, when you’re building a mobile app, you know, you’re not inventing everything from the bottom up, right?

    Johnson: Right.

    Ashley: You’re merging a lot of existing resources—functionality as well as data. So, it seems like no better—what better of a kind of application where making it easier to get the data? Because, you know, mobile developers don’t like writing database access code, right? They like building interfaces and doing cool interactions and useful interactions with customers.

    Johnson: Right, right.

    Ashley: So, when you can let them focus on that and really create great experiences with that data, that’s a win/win for everybody, it seems like.

    Johnson: Yeah, no, I agree.

    Ashley: Well, cool. I really enjoyed talking with you, and where can folks find out about CData? What kinds of abilities do they have to check you out, from free accounts or sandboxing or things like that?

    Johnson: Yeah. So, CData.com is our website, and you’ll find all the information that you need there. For any of our installable products, we offer 30-day free trials, and they’re fully functional trials. You know, we’re not rate limiting, we’re not doing anything like that. And then with our trials, you get access to our support team, which we’ve been called world class, we have testimonials on our website if you really wanna dig into those.

    Ashley: Comes from a man who was on the support team, so.

    Johnson: That’s right. [Laughter] 

    Ashley:[Laughter] No, I’m sure it’s really great.

    Johnson: And then, you know, for the Cloud Platform, I think we offer a two-week trial, and so, you can kinda kick the tires on that as well. So, CData.com/drivers is where you wanna go if you’re interested in an installable driver. CData.com/connect is where you can go if you’re interested in that Connect platform, that Cloud Platform for connectivity. And then, you know, you can find us on LinkedIn, you can find us on Twitter, we’re @CDataSoftware. You can find us on Facebook. A lot of our posting there is upcoming events or recent articles or other news announcements, so.

    Ashley: Yeah, and I have checked out your drivers, and you know, an analogy like a Swiss Army knife falls apart very quickly, because you have so many technologies, so many platforms that you interface with—it’s about 50 Swiss Army knives. [Laughter] 

    Johnson: Yeah. So, I’m just gonna plug this—so, I think we’re 250 plus different data sources across 11 technologies. So, somebody out there is doing the math right now in their head to figure out how many SKUs we’re sitting on right now.

    Ashley:  The number of permutations—well, good. It’s really been good talking with you, and I look forward to having you all on again and talking some more, I think, about the low code/no code space and some things that are happening there, so we’ll look forward to that.

    Johnson: Yeah. Alright, yeah. Thanks, Mitch, it’s been my pleasure.

    Ashley: Great. Jerod Johnson, who’s Technology Evangelist with CData.

  • The Pandemic’s Impact on Mobile App Security

    The Pandemic’s Impact on Mobile App Security

    Mobile device use has continued to grow throughout the global pandemic, surpassing 5.19 billion mobile phone users worldwide over the past year. As a result, companies have been launching mobile apps at an incredible pace to capitalize on the abundance of potential users.

    The rapid pace of mobile app development, however, has led to a sudden uptick in privacy and security issues as well. As a result, many mobile developers are starting to recognize the need for a more proactive and streamlined approach to app security by adopting a DevSecOps mindset.

    Mobile Security In The Spotlight

    Mobile apps have become an enticing target for criminal exploitation because many developers don’t have the time to prioritize security measures like obfuscating their code, encrypting sensitive information or monitoring for live attacks after publishing an app. In fact, the FBI has reported a 50% surge in mobile banking usage, which has led to an increase in banking trojans and other attempts at financial fraud. Moreover, studies have shown that nearly every sector and companies across industries are facing cyberthreats, with 55% saying that being compromised had a lasting impact.

    With regard to government responses to COVID-19, the release of contact tracing apps intended to help contain the coronavirus were recently in the spotlight for their lack of adequate security measures. These crowdsourced solutions gave governments more widespread reach for warning individuals that may have been exposed to the virus, but many mobile app developers had to launch them quickly, and security was not made a priority. Some contact tracing apps, like many other mobile apps on the market, failed to protect user privacy because they relied on the phone’s GPS, harvested personal information or sent data to a centralized database.

    In an effort to make these contact tracing apps more secure, build trust and gain more user adoption, Apple and Google jointly released an Exposure Notification API. One year into the pandemic, mobile developers can use this API to build more secure contact tracing apps without compromising personal information. While the majority of contact tracing apps have adopted the API, this issue has revealed broader concerns about mobile application security and the need for DevSecOps in the industry.

    DevSecOps For Mobile Developers

    DevSecOps is an approach to application security that integrates security processes throughout the entire development pipeline. For mobile apps, this means scanning for issues during development, testing, deployment and following up on threats after release. By creating a secure software development life cycle (SSDLC), organizations can leverage automation to improve application security without burdening developers.

    A key aspect of DevSecOps is catching issues earlier in the SSDLC, which is often called “shift left.” This is a proactive approach to application security where app developers use code hardening tools to protect their apps and code scanning tools to detect and remediate issues during development. By preventing vulnerabilities from reaching later stages of the SSDLC, the cost and effort involved with remediating security issues is much lower. While mobile app developers have prioritized being first to market in the past, many companies are now recognizing that a shift left of security efforts aligns with this goal.

    Besides the development phase of the pipeline, it’s crucial for mobile app developers to implement security processes in the testing phase, as well. Both static and dynamic automated testing tools can detect vulnerabilities in the source code and execution of mobile apps to prevent issues from reaching production. With a DevSecOps approach, mobile developers can integrate these tools directly into the continuous integration and continuous delivery (CI/CD) pipeline to minimize the manual effort involved with improving their security posture.

    While many mobile app developers have had to learn the hard way by experiencing a security breach, the pandemic has motivated others to take security more seriously. As people spend more time at home and shift to digital means for everyday tasks like shopping and banking, they’re using mobile apps far more frequently. Prioritizing security as well as speed by adopting DevSecOps methodologies allows mobile app developers to better protect their users and intellectual property as well as build greater trust in their apps without sacrificing time-to-market.

    Enterprises may have more mature security programs in place, but mobile app developers are quickly catching up. The pandemic has put mobile security issues in the spotlight and motivated app developers to adopt DevSecOps. As mobile apps continue to become more critical for businesses, the maturity of security programs within the mobile app industry will likely improve even further.

  • Next Up: Portable CI

    Next Up: Portable CI

    One thing that we’re very good at in IT is breaking new ground, coming up with ideas that work astoundingly well, getting broad adoption, then standardizing them for interoperability. When a new technology (looking at you, cloud) has problems with standardization and interoperability, we find a way around vendors’ reluctance to give you freedom of choice. For cloud, for instance, we found another great technology – containers – that greatly limits our lock-in to a given cloud vendor’s fare. Of course, whether standards or workarounds are the answer we arrive at, there are always wrinkles. Vendors are certainly not motivated by interoperability; they want your regular cash infusions, and interoperability could be more of a drain on them than a feed of new customers. Particularly in today’s environment, where vendors sometimes think that doing “to” customers is an acceptable substitute for doing “for” customers. We have to work around things like “embrace and expand,” which is still very real, and deploying containers to cloud A versus cloud B still has a bit of vendor-specific work, but we have enough to get it done.

    As continuous integration (CI) has become critical to an increasing number of organizations’ IT processes, the time has come to do the same with CI tools. Standardization is a good first goal, but it is more difficult here than in many areas, because the topic is so broad. An app might be built in one tool and tested in another, or it might be built via the use of shell scripts while tested with a UI tool, etc. It is not an easy solution to build, but it is an idea whose time has come – particularly with the cloudification of the space. We now have full-on CI systems that are in the cloud, and which hook you in with custom instructions. We’ve already had one large DevOps vendor essentially tell their entire user base, “We’re moving you to the cloud, unless you pay these inflated fees,” and others that never offered options for traditional architectures at all.

    Yet, we know apps move from in-house to cloud (and though it is uncool to mention, move back), and organizations are motivated by cost containment and mobility more than by what their chosen vendor wants them to do.

    So, it is time for some bright company or OSS team to put together a standard and a tool that implements said standard. It will be a lot of work, to make portability between CI tools that came from widely disparate roots – some from build, some from test, some even from deployment. But we’ll be needing it. A while back, everyone was a Puppet shop. Now, it seems, everyone is an Ansible shop. In simple terms, everyone will inevitably be a [something else] shop, because nothing lasts forever. So, a tool providing CI mobility between tools will definitely have a market – smaller today, but eventually large. We don’t need Yet Another CI Tool, we need that portability.

    I’m sure it’s coming. History does offer glimpses of the potential future, and we’ve seen this cycle before. I’m just saying, the time is now, not five years from now. We can target the OS of our choice and the cloud vendor of our choice, using containers to simplify the differences in host OS/vendor. The ability to use the CI tool of our choice is the logical next step.

    I haven’t heard of a project or company working on this yet; if you have, drop me a link in comments or email (I prefer comments, so others can look at your suggestion too, but whatever works for you), and I’ll take a look. It would be cool to support this kind of effort, when it eventually comes about.