Author: Mike Vizard

  • CodeSee Unveils Enterprise Codebase Visualization Platform

    CodeSee Unveils Enterprise Codebase Visualization Platform

    CodeSee this week launched an enterprise platform based on its codebase visualization and mapping tools that makes it easier for developers to navigate and learn how an application was developed.

    CodeSee CEO Shanea Leven said this capability makes it simpler to understand how a codebase is constructed, including the application programming interfaces (APIs), without having to read each line of code. Organizations can use the CodeSee Enterprise platform to review code in addition to making it easier to add new developers to a project, she said.

    The CodeSee Enterprise platform makes it possible to automatically detect, visualize and link to the code used to create every connection between the services that make up an application. That capability is especially useful for any security review of an application before or after an application is deployed.

    A code automation feature codifies and automates the code knowledge that previously could only be passed from one developer to another, Leven noted.

    It takes a significant amount of time for developers to pore over a codebase to understand where they might contribute. The larger a codebase is the more challenging it becomes to make meaningful contributions.

    By definition, application development in the era of microservices is now a group effort. The days when a lone developer cared for an entire application are long over. The issue is finding a way for teams of developers to more easily collaborate when most of them may have never actually met in person.

    The amount of focus on developer productivity has increased substantially since the start of COVID-19 pandemic spurred organizations to accelerate a wide range of digital business transformation initiatives. Developers are generally challenging to recruit and retain, especially individuals familiar with the latest cloud-native platforms for building and deploying applications. Making it possible for developers to more easily contribute to multiple application development projects has never been more critical.

    Of course, most developers would rather spend more time writing code rather than studying it. The amount of time devoted to understanding codebases and maintaining development environments far exceeds the amount of time spent writing code. Many developers are not always going to be able to take advantage of advances in application development without the aid of a DevOps team to provision new tools and platforms.

    One way or another, the rate at which applications are being developed and deployed needs to accelerate. An uncertain economic outlook is increasing the pressure on business leaders to digitally transform processes in a way that increases both revenue and profitability. The application experience is especially critical because customers are not inclined to patiently wait for organizations to finetune their digital processes over an extended amount of time.

    Less clear, however, is the level of stress individual developers might be willing to tolerate. There’s no doubt most of them are sympathetic to the needs of the organizations that have hired them but that does not necessarily mean they’re willing to continuously work overtime to achieve goals that might not be attainable using the tools they have at hand.

  • VMware Embraces Graph Technology to Manage Multiple Clouds

    VMware Embraces Graph Technology to Manage Multiple Clouds

    At the VMware Explore 2022 conference today, VMware launched a portfolio of multi-cloud management tools, dubbed VMware Aria, based on a graph-based data store that captures the resources spanning multiple clouds and their relationship to one another.

    Accessed via a VMware Aria Hub platform previously known as Project Ensemble, VMware Aria Graph provides a common definition of applications, resources, roles and accounts in near-real-time across a multi-cloud environment that can consist of millions of nodes.

    VMware CEO Raghu Raghuram told conference attendees the Aria Graph is at the core of an effort to enable IT organizations to move beyond “cloud chaos” to embrace a next-generation “cloud smart” approach to managing distributed computing environments that span multiple platforms.

    Based on an event-driven platform, the VMware Aria Graph aggregates data pulled from public application programming interfaces (APIs) into a data store residing on that cloud. The data store is based on a federated model so there is no need to move data into a central repository. Instead, VMware is making use of GraphQL to launch queries across a federated data model. In addition, IT teams can discover dependencies and analyze historical configuration data to discover the root cause of any issue. All operational data is now aggregated in a way that creates a single source of truth.

    VMware has also provided a set of API services that can be used to integrate the Aria Graph data store with third-party platforms.
    In general, the company is committed to accelerating “the great replatforming” of IT environments spurred by digital business transformation initiatives, said Raghuram. The challenges IT organizations are encountering are roadblocks that involve everything from a lack of skills, the weight of existing enterprise applications and fragmented operations and cybersecurity models, he noted.

    The VMware Aria graph data store will enable the company to automate a wide range of management tasks, Raghuram said. That includes using the right cloud for the right application and governing and accessing them at the lowest cost possible, added Raghuram. The ultimate goal is to enable a unified developer experience that provides a frictionless multi-cloud experience across a consistent set of enterprise infrastructure, he said.

    VMware offerings that leverage the VMware Aria Graph include VMware Aria Guardrails for enforcing IT policies, a VMware Aria Migration tool for automating assessment, planning and execution using the VMware HCX platform and a VMware Aria Business Insights tool that surfaces business insights using machine learning algorithms. The rest of the VMware Aria portfolio is anchored around extensions to existing platforms in the form of VMware Aria Cost powered by CloudHealth, VMware Aria Operations and VMware Aria Automation.

    VMware also announced today that its existing VMware Cross-Cloud Service offering for deploying its stack of software across multiple clouds has been extended to add support for the Microsoft Azure cloud.

    Overall, VMware claims there are now five million developers in its ecosystem that collectively launch 350,000 projects every week. That equates to roughly 35 new applications a minute, according to Raghuram. All told, VMware claims there are 85 million workloads running globally on its platform.

    The bulk of those workloads today still run in on-premises IT environments. In fact, Raghuram noted that cloud platforms still only account for $413 billion in IT spending each year, which is only 10% of the total IT spending that reaches $4.4 trillion each year.

    VMware, which is expecting to complete its acquisition by Broadcom in the next few months, is betting that as more of those workloads move to the cloud, the need to centrally manage heterogenous IT environments will play to its strengths.

  • Automation has Become an Absolute IT Necessity – Techstrong TV

    Automation has Become an Absolute IT Necessity – Techstrong TV

    Vijoy Pandey, vice president of emerging technologies and Incubation for Cisco, explains why automation has become an absolute IT necessity. The video is below followed by a transcript of the conversation.

    Michael Vizard: Hey, guys, thanks for the throw. We’re here with Vijoy Pandey, who’s vice president of emerging technology and incubation for Cisco. We’re gonna be talking about all things related to automation. Vijoy, welcome to the show.

    Vijoy Pandey: Thank you, Mike, pleasure to be here.

    Michael Vizard: Right now, everybody is struggling, looking to find people, hire people, retain people, and this was an issue before COVID came, but it seems like it’s much worse these days, and there seems to be no end in sight. So, what is your best advice to folks about how to think about managing talent these days?

    Vijoy Pandey: Yeah, I think that’s a pretty important point you raise there, Mike. Just like everybody else, we’ve been seeing not just news articles around the great resignation, but also we’re feeling that inside of Cisco as well. We know that our partners, our customers, all of us are feeling that pinch, where it’s incredibly important to retain top talent, but it’s also very hard to find talent, given the trends that we’re seeing around us.

    Anything about cloud computing, anything about AI, all of those things. So, we think of the problem in a very different way. We think – and we’ve done a bunch of surveys inside of Cisco – what we realized was given the global nature of events – we know some events unfolding right now – and all of the geopolitical events happening, all the pandemic, people really are realizing that life is too short.

    It’s not a pithy statement here, it’s actually people believe that life is just too short, and they actually want to do meaningful work. They want to make an impact, not just to the team that they are with, nor the company that they’re with, but even a global impact with the work that they do.

    But they also want to grow and learn, and make sure that they are on a path themselves towards meaningfully improving their own lives. So, I think those are pretty important characteristics that we have gathered, and as organizations and as teams we need to harness that desire to make both local impact as well as global impact as well as figure out how to drive meaning into everybody’s personal lives – yours, mine, our teams’, and so on and so forth.

    So, there’s a bunch of things that we are doing inside of Cisco to make sure that this meaning, or meaningful impact, is nurtured. One of the things that we keep on saying is that yes, there is a business to run, but there’s also an avenue we need to give our employees and our teams to innovate, to look at impact beyond the run-of-the-mill business that any corporation needs to do.

    So, typically what we end up doing is we end up taking problem statements that we face, let’s say, as a team or an organization, and we start splitting it into buckets. So, think about buckets that deal with run-of-the-mill business, because that, you need to absolutely take care of.

    But also, there are buckets that you need to innovate around or do meaningful work around. So, we have a portfolio management system inside, at least on my team, that says let’s take 60 percent of your time and make sure that that’s allocated towards run-of-the-mill business as usual projects, and then maybe 30 percent of the time which is future-looking and adding new features or looking at new markets and so on and so forth, and maybe 10 percent of your time is actually towards growing yourself, nurturing yourself, or making some meaningful impact on the global stage. So, that’s actually how we’ve been viewing the problem.

    Michael Vizard: Early on, I think a lot of people were intimidated by automation, and there was concerns that people might lose their jobs. Now, I wonder if we’ve flipped that entirely, where people are saying hey, I don’t really wanna work for organizations that don’t have automated IT process in place because of your points – there’s just things that I don’t wanna spend my day doing anymore. So, has the whole conversation flipped, as far as automation is concerned?

    Vijoy Pandey: I would say the conversation is flipping as we speak, again, yeah, going back to the previous comment, yes, people don’t want to spend time on repeatable tasks, on things which a computer can do better than you and I can do. I have this constant struggle with my kid nowadays as to why do I need to learn grammar when there’s Grammarly? But that’s a different matter.

    But I think it’s a similar mindset going through every individual in the workforce, as to am I really adding value in the workplace, and am I being more productive, which in turn translates is the organization being more productive, which in turn translates to is the economy and the country being more productive.

    So, I think the bump towards productivity is definitely a goal that people are more concerned about, and you hear the other side of this, which is, as you pointed out, is the digitization of the workplace is the digitization of all economy. That’s in progress.

    We’ve called that using various phrases – software eating of the world, or the API economy – whatever term that you want to use. But we are moving towards a world where software is critical in driving everything that we do. Software in terms of automation, or even beyond that, is definitely key to whatever we do.

    Michael Vizard: Not too long ago we all thought we’d be running around by flying cars by now, and that IT would be easy. We’ve seen neither, and it looks like IT is getting more complex than ever. So, my question to you is is IT getting to the point where it is just too complex to manually deal with and we need automation, because we are seeing all these microservices, serverless frameworks.

    There’s all our existing monolithic applications, it’s distributed all the way out to the edge now. Have we reached some sort of inflection point?

    Vijoy Pandey: Yeah, I think to your point IT is definitely way more complex than it was even a decade ago, and I think the crux of the matter is that the way you are building applications and all of the digitization that we talked about is actually taking place in a very, very distributed manner.

    So, as an application developer, first and foremost, we at Cisco like to say that the application is your brand. So, if I am a bank, how my customers interact with the apps that I build defines me, my company, my bank, as the brand. Because if bank A has more capabilities in their software, their mobile app, versus bank B, I will migrate towards bank A.

    So, I think the application is becoming the front end to everything customer experience is all about. So, if you take that notion and play it out, that means the application developer that is developing those applications is under extreme pressure to deliver velocity behind features, do deliver trust with the data of the customers, to deliver uptime and security and so on and so forth.

    So, which means that they are picking and choosing all of these APIs that exist anywhere to make their job faster, easier more secure, more trustworthy. So, if you think about that kind of an environment, as a developer, I’m picking APIs from cloud providers, SaaS providers, I have legacy systems, I have transaction systems, I have edge devices, in the case of a bank. It might be sitting at a branch office.

    I have all of these APIs and systems that I need to interface with just to build one particular app or feature. So, in this kind of a distributed environment, think about the IT – more IT people, right? It’s like I’m trying to build a platform that caters to these kind of demands, and so it is definitely getting more complex, it’s getting more distributed, and every provider in the distributed environment has their own semantics, their own lifecycle management behind their APIs and their infrastructures.

    So, automation – I mean, it’s just humanly impossible to not just deploy but even manage and observe such a highly distributed – and secure, by the way – such a highly distributed system. So, automation is key to doing all of this, a direction automation, to me, is discoverability.

    It’s intent-based, it’s security, which is there’s no longer a perimeter of security, and it’s observability across all of these end points, API to mainframe. That is key to moving forward.

    Michael Vizard: Do you think as we go along that organizations – at least some of them I talked to, it’s almost like a chicken and an egg kind of issue. They have so much legacy stuff that can’t be automated that they can’t invest in automation, and they can’t necessarily buy all new platforms tomorrow. So, how do we get over the automation hump to kind of get from point A to point B without necessarily breaking the IT budget?

    Vijoy Pandey: Yeah, that’s a great point. I think some of that, I’ve dealt with in my previous lives as well. The way to do that is – and there’s no questions about it, that for some period of time, there is going to be more chaos and more money spent than before that time or after that time.

    So, we just have to acknowledge that as vendors, as customers, as organizations, that for some period of time, there’s going to be turmoil. That’s the way, at least, I’ve handled it in the past, where you say let’s put milestones, where we say maybe we look after a pocket of the organization, and say we’re going to build this out in a highly automated, distributed, cloud-first, API-first manner.

    This part of the application, this part of the infrastructure goes out and does that. To do that, we’re going to do a clean slate process here. So, we’ll think about – we won’t get ourselves bound to organizational boundaries, skill set boundaries – that is very hard to do. Most people’s jobs, peoples’ skill sets, organizational structures, all of that.

    But we have to try in some pockets, and then give ourselves strict guidelines and metrics, and try and meet those metrics. If we meet those metrics, the resulting outcome is that I will be spending less moving forward than I’m spending today.

    So, I might spend a little bit more in getting maybe SREs or software engineers, and marrying them up with IT folks or architects, and getting something going. I’ll give myself a timeline of let’s say a year or 18 months; it cannot be longer than that, otherwise it’s a science project. 

    But after those 18 months, these are the metrics I’m going to come back to. I’ll get X percent improvement in productivity, and Y percent reduction in head count, let’s say. We shall measure ourselves to that objective, and I’ve seen that work.

    If it works – and start with something simpler. But go all the way, don’t go 80 percent. Let’s not do 80 percent automation or 80 percent of the use cases. Hundred percent automation, 100 percent of the use cases for this pocket of your environment, and then grow that out over time.

    Michael Vizard: We, of course, hear a lot about artificial intelligent and AI Ops. Are the machines gonna save us from ourselves? What’s the reality of AI?

    Vijoy Pandey: To me, AI is a tool, just like all of the other tools that we’ve talked about, which is we’ve talked about automation or infrastructure as code, we’ve talked about cloud native and serverless and microservices. Each one of these tools, programming and scriptings tools.

    So, all of these tools bring in something to the table, and infrastructure as code brings in consistency and intent-based infrastructures, and serverless and microservices, or cloud-native, bring in immutable architectures.

    The fact that you can grow and scale out and do all of those things, and availability goes through the roof and velocity goes through the roof. Similarly, AI and ML bring in insights to the table where it was humanly, again, impossible to do that just by looking at the vast variety of data and the deluge of data coming our way.

    If we could just take all of that – and it’s all unstructured. So, if I could take all of that and somehow put it in a structured database and somehow make sense of it as an individual, sure, we don’t need the AI or ML. But if I need to take all of that unstructured data and make some sense out of it, there’s no other way out, and I do need AI and ML.

    I mean, I do need some statistical mechanism first, to figure out and make some sense out of it. I do need AI/ML, maybe deep-learning, to make some sense out of it. But then I need to marry that tool with subject matter expertise, I need to marry that tool with what I as an individual, or as humans, we’ve learned in the past, because AI doesn’t have that context.

    Yes, we can do learning, all sorts of learning, but it doesn’t always work in all kinds of environments. We need to marry that with subject matter expertise, with some boundary conditions, and then we need to marry it with actions. Because AI/ML is gonna give us insights, but somebody needs to take actions on the insights.

    So to me, it’s a tool in the toolbox, and we need to leverage it when it makes sense. We need to rely on subject matter expertise when it makes sense, and eventually things will get to a point where we need to go to the next level of abstraction. 

    So, it’s always a game of moving that level of abstraction higher and higher, and that’s what we are after. So, maybe someday there will be flying cars and robots that will take all our jobs, but not right now.

    Michael Vizard: How do I bridge this divide in IT that maybe we don’t talk enough about, but there’s clearly a DevOps community that’s very programming-centric. The use APIs. Then there’s traditional IT administrators that use ITSM tools. They tend to be more graphical in nature. As we move forward with automation, do you think that those two camps are gonna converge, or how will this all play out?

    Vijoy Pandey: In my opinion, I think they have to converge. I think each one of those pieces have their space in the hierarchy of things. One of the things I learned pretty early on in one of my previous lives is what good is a dashboard if you cannot take an action on it?

    So, if you think about all dashboards that exist today, eventually, there is data being presented in a certain way about a certain problem, so that somebody can look at that and take an action. If you think about infrastructure as code, it’s exactly that.

    It’s basically let’s remove the dashboard, and let’s figure out what goes – triggers that, that allow me to take those preset actions. Let’s try and remove the action of a human looking at a dashboard, thinking about it, and taking some action on it.

    Now, again, I’m describing utopia, but I think that’s where, as engineers, as operations folks, we all need to move towards. To me, dashboards are a good start when the problem is less well understood. As we understand the problems better, we need to move towards code.

    Dashboards will still have their place in helping us understand more and more complex problems, but as problems are better understood, they need to move towards code. And that’s the way at least I think through it.

    Michael Vizard: All right, so automation needs to be our new North Star, as it were. Hey, Vijoy, thanks for being on the show.

    Vijoy Pandey: Thank you so much, it was a pleasure, and I enjoyed our conversation.

    Michael Vizard: All right, back to you guys in the studio.

  • TidalScale Enables Hot-Swapping of Failed Servers

    TidalScale Enables Hot-Swapping of Failed Servers

    TidalScale has updated its namesake software for automating server management to include the ability to detect and prevent impending failures and enable IT teams to hot-swap another server.

    Version 4.1 of TidalScale also adds server wear leveling and preventative health checks that identify servers that should be replaced based on their remaining estimated life span.

    TidalScale CEO Gary Smerdon said that capability is especially critical given the high rate of DRAM memory failure that results in a high rate of downtime caused by server failures. The latest update to the company’s software-defined server management software reduces the failure rate in on-premises or cloud computing environments by a factor of 100 to ensure 99.999% uptime, he said.

    The company also announced that its software is now available on the Amazon Web Services (AWS) Marketplace.

    TidalScale employs machine learning algorithms to aggregate and virtualize CPU cores, memory and I/O across servers to present operating systems with a single software-defined server. That makes it possible to hot-swap individual servers to ensure availability.

    The single biggest cause of downtime in IT environments is memory failures; they are now 100 times more likely than they were a decade ago, noted Smerdon. In most cases, the root cause of the issues is the demands of modern applications running on legacy infrastructure that was not intended to support the latency requirements of those applications, he added.

    That issue can be especially acute in cloud computing environments where infrastructure is often shared by multiple applications, said Smerdon.

    It’s not clear at what rate cloud service providers are replacing legacy server infrastructure, but the ability to determine when to replace infrastructure gives IT teams an added measure of control over servers regardless of where they are deployed. The biggest issue, of course, is not so much replacing a server as it is determining which one actually failed. It often takes longer to identify a failed server than it does to bring a new one online.

    TidalScale is made available by both cloud service providers and manufacturers of servers used in on-premises IT environments, but it needs to be configured and deployed by an IT team. The overall goal is to not only increase system availability but also reduce the total cost of IT by managing infrastructure more efficiently across the life span of a server, said Smerdon. Some IT teams routinely replace servers at specified intervals, regardless of actual usage, to avoid downtime. TidalScale enables IT teams to determine which servers to replace based on the actual wear within the context of a self-healing server farm, he noted.

    Given all the things that might potentially go wrong in an application environment, not having to worry about server failures should enable DevOps teams to spend more time optimizing application performance.

    In theory, of course, modern applications based on microservices should be able to reroute calls to application programming interfaces (APIs) anytime a server becomes unavailable. However, over time it’s not uncommon for single points of failure to emerge as applications are updated. In the meantime, the majority of applications running today are based on legacy monolithic architectures that are much more susceptible to server failures that, unfortunately, are still far too common.

  • Five Great DevOps Job Opportunities

    Five Great DevOps Job Opportunities

    DevOps.com is now providing a weekly DevOps jobs report through which opportunities for DevOps professionals will be highlighted as part of an effort to better serve our audience.

    Our goal in these challenging economic times is to make it easier for DevOps professionals to advance their careers.

    Of course, the pool of available DevOps talent is still relatively constrained so when one DevOps professional takes on a new role it tends to create an opportunity for others.

    The five job postings shared this week are selected based on the company looking to hire, the vertical industry segment and, naturally, the pay scale being offered.

    We’re also committed to providing additional insights into the state of the DevOps job market. In the meantime, for your consideration:

    SimplyHired

    Kyndryl
    Remote
    DevOps Engineer
    $95,700 – $265,200

    Monster.com

    Randstad
    Remote
    Senior Cloud SRE
    $130,000 – $150,000

    Dice.com

    Staples
    Framingham, Mass.
    Senior Software Engineer I- CICD/DevOps
    $135,000

    Indeed

    Odyssey Information Systems
    Remote
    Senior DevOps Engineer
    $117,798 – $156,338

    ZipRecruiter

    Synthego
    Redwood City, CA
    Senior DevOps Engineer
    $113,000 to $165,000

  • OpenText to Create Software Powerhouse – Acquires Micro Focus for $6B

    OpenText to Create Software Powerhouse – Acquires Micro Focus for $6B

    OpenText this week announced it will acquire Micro Focus for $6 billion. The acquisition is part of an effort to reinvigorate a broad portfolio of information management, application development and testing, integration and cybersecurity platforms by accelerating the rate at which they are transitioned to OpenText’s managed private cloud.

    The acquisition is expected to close in the first quarter of 2023, and the combined company expects to reduce costs by an additional $100 million on top of the $300 million Micro Focus previously committed to cutting as part of an ongoing effort to return to profitability and growth.

    OpenText CEO Mark J. Barrenechea told analysts the ultimate goal is to enable organizations to accelerate their digital business transformation initiatives using a set of application programming interface (API)-centric offerings that reside on a private cloud.

    Digital transformation represents the largest opportunity in the history of computing, he said, but Barrenechea also acknowledged that a lot of companies are still struggling with making that transition. “There has to be a better way to get digital transformation right,” he said.

    The Micro Focus portfolio includes everything from a robotic process automation (RPA) offering, a recently-launched IT service management (ITSM) offering—dubbed Service Automation Management X—along with the Vertica database and LoadRunner testing tools. There is also a slew of cybersecurity platforms that include ArcSight, Fortify, IDOL Zenworks and Sentinel along with legacy COBOL application development tools and even the remnants of Novell.

    In effect, OpenText now has one of the broadest software portfolios in the industry after previously acquiring Documentum, Carbonite, Hummingbird and Zix. “The strategy appears to be to simply gather up all the pieces on the chess board,” said Mitch Ashley, principal analyst for Techstrong Research, an arm of Techstrong Group that is also the parent company of DevOps.com.

    Despite the disjointed Micro Focus portfolio, OpenText has no intention of selling off any of those offerings, said Barrenechea. Going forward, the strategy is to not only accelerate the transition to the cloud but also expand application license renewal services, he added. OpenText also sees an opportunity to expand the appeal of the Micro Focus portfolio into the small-to-medium business (SMB) market, noted Barrenechea.

    All told, it will take six quarters once the deal is closed to align the Micro Focus business model with the one OpenText currently employs, which Barrenechea said should result in growth starting eight quarters after the deal closes.

    There’s no doubt that, once the deal closes, IT organizations that rely on Micro Focus will be closely watching the level of ongoing investment in the portfolio for any signs of one platform being deemphasized in favor of another. It’s also probable that many customers will be relieved to see the stewardship of those platforms assumed by a company that is on firmer financial footing.

    Regardless of how the OpenText portfolio evolves in 2023, however, one thing that is certain is IT organizations have no shortage of alternatives to consider.

  • Apache Hop Integrates DevOps and DataOps – Techstrong TV

    Apache Hop Integrates DevOps and DataOps – Techstrong TV

    Matt Caster, chief solution architect for Neo4j, explains how open source Apache Hop projects advances integration of DevOps and DataOps. The video is below followed by a transcript of the conversation.

    Mike Vizard: Hey, guys, thanks for the throw. We’re here with Matt Casters, who is Chief Solution Architect at Neo4j and we’re talking about a new open source project called Apache Hop, that helps you move data round more easily. Matt, welcome to the show. 

    Matt Casters: Thank you very much for having me, Michael.

    Vizard: So walk us through what exactly is Apache Hop, what’s the problem we’re trying to solve, and it seems like we’re trying to move more data than ever, and when I was a kid people told me moving data was not necessarily a great idea, but we’re doing it anyway. So what’s the challenge and what are we trying to solve?

    Casters: Oh yeah, yeah. So that’s a great observation. I remember when everything was going to be the new mainframe, do everything in one box, right? And the opposite happened; we saw more and more services on prem, in the cloud, and all mixed up. See, I think, you know, more specialized services, in the cloud, on prem, graph databases, you know, advent of containerizations, virtual machines, and containers. So it’s become like a really complicated ecosystem out there and I think a lot of organizations are struggling to find expertise to service and cater to all those needs.

    I think what we’re trying to do is make the life of the developer easy by wrapping up user interfaces and easy-to-use tools around all these technologies so that you get like UMLI diagrams that you can work with and that can be easily understood by not only the experts but medium-level educated people on subjects, right? For example, if you’re working with a Kafka queue you might need to know how it works in general, but you don’t need to dive into the Java or Python APIs to get the best out of it. I mean that’s kind of it. 

    So lower the maintenance costs. And this comes down to the DevOps side of things. And while we try to make the life of the developer easy we position ourselves exclusively on the side of the organization. We’re not made to just make life fun for the developer, but we really try to make the life fun for the developer, but we really try to make life for the company or the organization that uses the software better. 

    So with that in mind you come to the conclusion that, okay, we need to also focus on configuration management, life cycle management, version control, unit testing, security investment, and that companies put into often many years of work of data orchestration, data integration, ETL work, right? So that is what I think Hop has been trying to focus on the last couple years by making sure that all these technologies are standard in the tool and easy to use. 

    Vizard: Do you think that we’re reducing the need for specialists, ’cause back in the day you had to have somebody who knew how to program with ETL and that capability and now with this we’ve got more of a visual tool, and maybe some people still prefer CLIs and APIs, but this is designed for developers to just easily handle this task themselves without any outside intervention, right?

    Casters: Yeah, that’s the lore. We fully acknowledge that it’s like a 95/5-percent kind of deal, right; you’re never going to do the last 5-percent without some scripting or coding. But solving the 95-percent is a big deal, it means that all of a sudden you open up possibilities to people that we would never have expected of using Apache Hop. Right? I’ve seen salespeople use it to extract the sales numbers from Salesforce. I would’ve never expected that in the past. It’s like, “Oh, they can figure it out.” “Oh, I can just enter the details: username, password.” “Oh, I can read this” and before you know it people start to use that. 

    Vizard: Citizen developers are here with us today. One of the things about Hop is that it’s lightweight and I don’t think people appreciate just how much heavy lifting went into ETL historically. So why does it matter that this thing is lightweight and metadata-driven and what’s the impact? 

    Casters: Well, at some point we saw the advent of LANDesk, you know, microservices, but also in its previous life, Kettle, before that was used was for plotting trains in more like an IOT settings. Call these Edge devices, maybe Raspberry PIs or very tiny devices. And there’s a real use case out there for a device that doesn’t have to do it or anything; it’s just trimmed down to the bare essentials for what it needs to do. Maybe it would be like measure temperature and air humidity and pressure and then send it off every 30 seconds or something, over some – using some Rescall. 

    And then this whole Hop library trims down to something like 30-40 megabytes and starts up fast and uses next to no memory. That is not in contrast to saying like, oh, maybe I run it on an AWS LAN in a small services. And it also gives us the ability to create new packets, right? You can only remove them, remove functionality, but you can add functionality quite easily as well. And that is really important. We’ve also already seen people implement machine learning or libraries or Python iteration and plug-ins for Hop. So this will be our next goal, is to create a marketplace around that plug-in kernel kind of architecture. 

    Vizard: Do you think that the functions of DevOps and DataOps are starting to converge more? Is this becoming more of a team sport or is DataOps just going to be folded into DevOps? What’s the relationship?

    Casters: Well, what I’ve seen personally is now that we have the ability to lock everything that a workflow or pipeline does into a graph, a Neo4j graph, it makes it easy to see where the arrow was, right? Can use graph algorithms to figure out not just where an arrow was, but figure out what the execution pat was in milliseconds. So the convergence of, “Oh, the developer has an easy time figuring out what went wrong, but also on Monday morning when you come in the office and see that the Saturday night run failed for some reason, or the monthly load of the data warehouse or whatever people are doing, they don’t have to troll through millions of lines of loading texts for a large workflow, but just see straightaway what went wrong, where it went wrong, saves a lot of time. 

    So is that DevOps? For sure. For sure DevOps. It’s also ease of development. There’s really like an overlap and we need to do better in providing a lot of these tools. They’re still too primitive in a lot of our software, As if locking text is enough, right? We can do so much more. [Laughs] We’re executing stuff; we know where it went wrong. Why guess? Why go through the effort? [Laughs] 

    Vizard: There’s a lot of focus these days on the volume of data that people are trying to work with, but it seems to me it’s also the speed in which that data is starting to emerge more, and people are trying to work in near-real time, and the data is going to be updated. We’re moving beyond batch. So are we really prepared for not just the volume of data, but the speed at which that data is starting to emerge. And for that matter, there’s a lot more types of that data, right?

    Casters: Yeah. So being from the Hop from the beginning opted to choose Beam to work as an abstraction layer so that we could execute on Spark, Flank, and Dataflow for the really fast use cases, but also, yeah, like real-time streaming, and just like playing, just streaming, Spark streaming and cloud Dataflow are really cool. And this comes along with the advent of all the streaming back-answer messaging services: Kafka, Pulsar, jewel clouds pops up. I think repertoire, right? I probably forgot like Azure. [Laughs] 

    But yeah, so these services have become really popular in making it possible to create cool Hop and Spoke architectures, where you can send stuff in all directions. Yeah, so it falls under the sort of like streaming category, and you can do all sorts of cool load-it-and-see exercises there. And opening that up is quite easy for us in Hop; all the pipelines are streaming, whether it’s in batch or not. So yeah.

    Vizard: So what’s your best advice to folks as they kind of look at these DevOps/DataOps challenges? What do you see as the best practice that people do the right thing? And for that matter, what kind of mistakes are people making that maybe they should avoid?

    Casters: So I think when it comes to these best practices, and then we’re talking about the classical list, put everything in version control. You know, keep your passwords and usernames and everything separate. Don’t hardcode anything. Like these are not specific to ETL data orchestration, but they’re genera programming best practices – development best practices. So even if it is like graphical programming, it still applies. Same with unit testing and integration testing.

    What we’ve seen in the past is that if you want to do, for example, unit testing, there’s a lot of plumbing that you need to put in place. So specialized Docker containers and so on and so on and so on. What we’ve tried to do is always look at the return investment of metadata. So what is the return investment for setting up like a lot of infrastructure and a lot of manual plumbing? It’s very low, right? So you see some benefit from a unit test like that, but it’s not great. So by building the integration testing and unit testing into Hop, into GUI, making it very easy to create, we’re saying like, “Okay, now you really don’t have any reason not to do it anymore. Right? [Laughs] Well, apart from laziness or not knowing what the value is. But at least those best practices are not different from any other development job that you might have.

    Vizard: All right, so we finally reached a point where we’re going to automate the orchestration of the data, just like the rest of any other DevOps process for that matter. So it’s onwards and upwards. Matt, thanks for being on the show.

    Casters: Hey, you’re welcome. It’s good to talk to you, Michael. 

    Vizard: All right. Back to you guys in the studio. 

  • Five Great DevOps Job Opportunities

    Five Great DevOps Job Opportunities

    DevOps.com is now providing a weekly DevOps jobs report through which opportunities for DevOps professionals will be highlighted as part of an effort to better serve our audience.

    Our goal in these challenging economic times is to make it easier for DevOps professionals to advance their careers.

    Of course, the pool of available DevOps talent is still relatively constrained so when one DevOps professional takes on a new role it tends to create an opportunity for others.

    The five job postings shared this week are selected based on the company looking to hire, the vertical industry segment and, naturally, the pay scale being offered.

    We’re also committed to providing additional insights into the state of the DevOps job market. In the meantime, for your consideration:

    LinkedIn

    CapitalOne

    Annapolis

    Director of Software Engineering – Risk Management and Compliance

    $190,000

    Dice.com

    Nigel Frank International

    Remote

    Senior Azure DevOps Consultant

    $165,000 per year.

    Indeed.com

    ROM Technologies

    Brookfield, Conn.

    DevOps Engineer

    $120,000 per year.

    SimplyHired

    Comcast

    Philadelphia, Penn.

    Senior Engineer Cloud Infrastructure

    $109,932 – $164,898.

    Monster.com

    Target

    Remote

    Lead Engineer – High Performance Distributed Computing

    $105,800 – $190,500

  • F5 NGINX Delivers Modern Management Platform

    F5 NGINX Delivers Modern Management Platform

    F5 NGINX has revamped an implementation of its management suite as part of an effort to make the platform both more accessible and simpler to use.

    Eric Braun, vice president of product management for F5 NGINX, said NGINX Management Suite 1.0 is based on a modular platform that makes it possible to dynamically scale instances of NGINX proxy software, application delivery services, application programming interface (API) management workflows and the company’s security offerings.

    That approach makes it possible to automate the deployment of F5 NGINX software without relying on manual processes that are prone to misconfiguration, Braun noted.

    The platform is based on a modern architecture that also makes it possible for administrators to enable development teams to self-service their own requirements within the limits of a set of policies defined via a portal, he added.

    The NGINX Management Suite replaces NGINX Controller, a more monolithic platform that is not based on an API-first approach to managing services, said Braun.

    The core modules of the NGINX Management Suite are an Instance Manager for the open source and commercial instances of the company’s proxy software, an API Connectivity Manager, App Delivery Manager and an App and API Security module. NGINX Management Suite will enable organizations to use NGINX App Protect Web Application Firewall (WAF) to protect applications and APIs while the App Delivery Manager module will be extended to enable configuring, securing, monitoring and troubleshooting of NGINX Plus when used as a load balancer.

    The revamp of the NGINX management plane is the latest in a series of efforts to modernize application and IT infrastructure management using a microservices-based platform that more easily scales as more resources are required. The more modular approach also gives IT teams more granular control over how various modules are accessed.

    It also makes it simpler for IT teams to deploy, for example, a service mesh or API gateway without having to add additional management platforms, noted Braun. That capability should significantly streamline the process of deploying those platforms because the need for a dedicated platform to manage them has been eliminated, he added.

    It’s not clear to what degree that latter capability will enable IT organizations to shift responsibility for managing various tasks further left toward application development teams. However, it is clear application development teams want to be able to self-service their own IT requirements and make services available without intervention from a centralized IT organization. In effect, IT organizations are trying to strike a balance between the benefits of centralization and the need to build and deploy applications faster within the context of a DevOps workflow.

    It may be a while before most IT organizations are able to consistently strike that balance. In the meantime, however, as IT management platforms continue to be modernized using APIs it should also become easier to integrate them. It will be up to each IT organization to then decide how many management platforms are going to be required, but at the very least, the various management silos that make up an IT environment today are finally starting to break down.

  • Postman Survey: Increased Usage of APIs Results in More Security Incidents

    Postman Survey: Increased Usage of APIs Results in More Security Incidents

    A survey of more than 37,000 developers and application programming interface (API) professionals published this week by Postman, a provider of a platform for developing and managing APIs, suggests that as more APIs are deployed, more organizations are seeing a rise in cybersecurity incidents.

    The survey finds more than half of respondents (51%) reporting that more than half of their organizations’ development effort is spent on APIs. About 20% also noted they are responding to API security incidents or breaches at least once a month at their organization. Overall, more than half (52%) said API security incidents occur less than once a year.

    Kin Lane, chief evangelist for Postman, said that while a lot of the concerns over API security could be classified as “theater” being put forth by providers of API security platforms, it’s clear there is a need for a zero-trust approach to building, deploying and maintaining APIs.

    The bulk of APIs are used primarily to interconnect internal-facing applications and systems, with a subset then used to integrate partners. Only a comparatively small percentage of APIs are externally facing. However, Lane noted many organizations fail to appreciate how quickly an internally facing API can become externally facing over the course of its lifecycle. As such, IT teams need the tools and platforms required to manage and secure APIs as they evolve, he noted.

    In fact, nearly three quarters of respondents (72%) said being able to build APIs remotely is very important to them. That would suggest best DevSecOp practices will need to be extended to API development teams that are often working from home.

    Despite those concerns, however, a full 89% said Investments in APIs will increase or stay the same over the next 12 months to drive, for example, digital business transformation initiatives. However, the survey also suggests most organizations are still coming to terms with API development and deployment. Only 8% of respondents identified themselves as API-first leaders, which Postman identifies as an elite group that produces APIs faster, deploys more often, has fewer failures and recovers sooner when failures occur.

    Lane said the survey makes it clear that while APIs are playing a crucial role in advancing business goals, there is a need for increased visibility as more APIs are deployed across the enterprise. That’s especially critical as more types of APIs employed also continue to expand. Most APIs in use today are based on the REST format, but there are also now APIs based on GraphQL, event-driven architecture and various messaging platforms. None of these newer APIs are going to supplant REST APIs any time soon. Rather, they are being applied to drive additional processes alongside REST APIs, noted Lane.

    It’s not clear how many organizations are adopting a design-first approach to building APIs before they write application code, but as more APIs are deployed, the way IT is managed is rapidly changing. Application services are essentially becoming more disposable so long as the API used to expose them to other applications remains stable. In fact, it’s now only a matter of time before most organizations fully appreciate the degree to which their entire IT strategy now revolves around their APIs.