devopsy.kinsta.cloud

Category: IT as Code

LF Networking (LFN) Adds to Open Source Portfolio

At the online Open Networking & Edge Summit + Kubernetes on Edge Day event LF Networking (LFN), an arm of the Linux Foundation focused on networking, today added several projects that promise to make it simpler to programmatically manage networking services based on open source software.

Walmart is contributing a L3AF project for managing network applications that make use of an extended Berkeley Packet Filter (eBPF) capability that allows applications to run in a sandbox at the microkernel level of an operating system. Intel and Aarna Networks, meanwhile, are donating the seed code from an EMCO project that provides a universal control plane to securely connect and deploy workloads across distributed computing environments that span everything from the edge to the cloud.

Arpit Joshipura, general manager for networking, edge and internet of things (IoT) for the Linux Foundation, said the L3AF project adds a control plane for managing those applications to the LFN portfolio that addresses load balancing, rate limiting, traffic mirroring, flow exporter, packet manipulation, performance tuning and other management tasks. Earlier this year, the Linux Foundation set up a separate eBPF Foundation to further advance development and adoption of eBPF across multiple operating systems.

At the same time, LFN also revealed today that Project Alvarium, a tool for measuring how well data is being delivered to devices that was originally created by Dell, and eKuiper, an existing open source lightweight IoT data analytics and streaming platform, are now part of the LF EDGE portfolio of projects.

Finally, an open source mobile edge computing project, dubbed Edge Gallery, that was originally created by Huawei has been contributed to the Linux Foundation.

Joshipura said the LFN continues to gain momentum as more vendors contribute core technologies that, ultimately, don’t provide them with enough differentiated value to continue to develop on their own. The LFN today also revealed that VMware and F5 Networks have joined the LF EDGE consortium as premier members, while mimik and Veena, Inc. have become general members of LFN and the Eclipse Foundation has become an associate member.

The LFN also provides a mechanism that promotes adoption of those technologies more widely than any one vendor can achieve on its own. The total cost of delivering next-generation edge computing platforms based on wireless 5G networks will also be sharply reduced as LFN projects continue to mature, said Joshipura.

In general, the rate at which various LFN projects are updated continues to accelerate, said Joshipura. The Akraino project for creating blueprints for edge computing platforms recently delivered its fifth update to address use cases involving smart cities, cloud-native multi-tenant environments and vehicular networks. An EdgeX Foundry project for IoT platforms, meanwhile, recently revamped its application programming interfaces (APIs) to make them more secure and enable more message-based communications. The Home Edge project will also soon issue a Drewberries update that will address code stabilization, scripting, APIs, data synchronization and security.

It’s now only a matter of time before these LFN projects become more integrated with DevOps workflows as IT teams seek to give developers more programmatic control over network services. The issue now is deciding not so much whether to integrate network operations and DevOps teams but rather determining to what degree those teams will need to fold into each other.

October 11, 2021
Closing the Education Gap for IoT Developers
The most common thing I hear from developers who are building internet of things (IoT) solutions is: It isn’t easy. And it’s not. Not because of a lack of skill—the developers I interact with are incredibly talented. There’s also no shortage of technologies to choose from for deploying an IoT solution. The challenge often comes down to a gap in available educational materials and learning support when in the development phase.

We’ve moved from the early stages of the IoT when there was a lot of talk but not a lot of application. Now, we’re coming into the “reality phase” where cheaper, large-scale IoT solutions are feasible. In this phase, developers are tasked with identifying, building and deploying technologies that they’re likely having to figure out as they go. It’s a tall order to expect today’s developers to know the ins and outs of all of the moving pieces required for an IoT solution, including the wireless technologies that help to enable them.

While developers are renowned for being effective at independent learning, they still need better access to educational resources and tools when it comes to IoT technologies. This is especially true for newer technologies that you won’t yet find in a textbook or college course.

One example of this newer IoT technology is low-power, wide area networks (LPWAN)—specifically using the LoRaWAN standard. It’s an area that I not only expect to become increasingly relevant for IoT developers but also one that can greatly benefit from more robust educational resources.

IoT Development Is Relatively New

Modern LPWAN technologies have only been around for the past decade, and LoRaWAN became an open standard when the LoRa Alliance was formed in 2015. In the grand scheme of things, that’s pretty new. It’s not a concept that’s widely included in higher education curricula and, although we see more inclusion each year, it can be difficult for developers to find peers who have experience in building with LoRa devices.

ABI Research forecasts that by 2026, LoRa will be the leading non-cellular LPWAN technology. This means that over the next five years more and more developers will need to understand the basics of LoRaWAN but may not have foundational knowledge or experience to draw from.

From what I’ve seen, that gap can lead to teams biting off more than they can chew. Without enough training or support, developers risk building LoRaWAN deployments that may have inadequate coverage or go over scope or budget or that simply don’t live up to their vision.

Complexity in Developing With IoT

The other major reason IoT technologies operating with LoRaWAN need more robust educational support is due to their complexity. Developing an IoT solution requires several different skillsets that well-equipped teams either have to account for in their staffing or—as happens in many cases—rely on a single developer to figure it out and wear many hats.

There are many aspects to building and deploying a LoRaWAN solution; for instance: Hardware engineering, device stack choice and application fit, gateway deployments and networking coverage, cloud considerations and application development involved for deployment. That’s a lot of moving parts.

So, if enterprises and IoT vendors are expecting developers to make large-scale IoT deployments a reality, we have to provide the proper resources and training.

Resources and Communities for LoRa Developers

As the industry works on ways to better educate developers about LoRa technology, there are some good places to start for developers who want to hit the ground running. Here are some resources where you can learn the basics, build prototypes, get advice from other developers and test and certify solutions.
- LoRa Developer Portal—A hub where IoT developers can learn, connect with peers and accelerate their builds with resources like tutorials, dev kits and a free network server. Helpful for every level of expertise, from learning the basics of LoRaWAN to creating a network to rolling out a full-scale commercial solution.
- LoRa Alliance—A nonprofit association where members collaborate and share experiences working with the LoRaWAN standard. They offers events, webinars, a resource hub and more.
- LoRa Basics Station —A GitHub Repository for the open source software that runs on gateways operating with the LoRaWAN standard.
- AWS IoT Core for LoRaWAN—A guide for AWS IoT developers, spanning coverage development and deployment from connecting gateways to monitoring and logging.
- Azure IoT Hub—Guide for connecting the ChirpStack open source network server using LoRaWAN to the Azure IoT Hub.
I expect higher education institutions to begin further integrating LoRaWAN into their curricula over the next several years, which will help to fill the education gap for the next wave of developers. At the same time, IoT vendors can step up and provide additional support to developers who are learning how to build with technologies like LoRa for the first time. This is critical for the success of large-scale deployments that all of us IoT believers hope to see across enterprise environments, cities, regions and beyond.
October 11, 2021
Your Tech is Only as Good as Your Infrastructure

A few weeks ago, one of my neighbors dug a ditch in his yard, hit a fiber internet line and knocked out my internet for eight hours. There I was, ready to start working, and I was cut off from my colleagues for an entire day.

That experience highlighted the importance of a good digital infrastructure—both at the office and at home—particularly as the pandemic has ushered in a hybrid work model. When the pandemic hit, organizations that were agile and had a strong digital infrastructure (including processes) were better able to quickly shift to remote work. Those that were not as digitally mature were forced to adapt.

Eighteen months later, it’s clear that the hybrid model is here to stay, making it even more vital that organizations ensure their workers have the right tech tools to work effectively—and it all begins with a strong digital infrastructure.

Just Because it’s On a Screen Doesn’t Make it Digital

Let’s begin with a definition of digital. I recently had a conversation with a business owner who proudly proclaimed, “We’re digital now.” In actuality, all he had done was put a PDF form on an iPad. Once his clients filled out the PDF, he still printed it and put it in a paper folder. That’s what I call an immature vision of digital, and it’s an extreme example of the ambiguity that often exists around the concept of digital transformation.

It’s not that paper is anathema to digital processes; in some industries (health care, for example) we want paper as a backup. For instance, if I’m a physician at your bedside and the power and/or Wi-Fi goes out and I can’t access your digital records, I need to be able to look at your chart and treat you. So there is a need for paper. But when considering the hybrid or remote work environment, the goal should be to move processes online.

Digital Infrastructure is the Key to Agility

As noted earlier, agile businesses are better able to adapt to changes like the move to a hybrid work model. Agile simply means the ability to move quickly and easily, the ability to move fast and make decisions in a fluid way. To do this, we must remove unnecessary steps so we’re able to focus on the end goal and the essential parts of the process.

This is where a strong (and reliable) digital infrastructure comes in. When you think about Maslow’s hierarchy of needs for digital infrastructure, the foundation is electricity and the internet, and then you build on that. I need computers and hardware and a network, and then I need software platforms and then I need an end-user interface.

But we’re getting ahead of ourselves; let’s go back to the basics. Once we have reliable electricity and internet, we need to look at other technology that can benefit the remote worker, like upgraded microphones and cameras for all those Zoom meetings.

These upgrades shouldn’t be an afterthought on the part of organizations; they are at the center of their employees’ hybrid experience and can be another key to success. It’s not enough to send employees home with a laptop and a monitor anymore. Employers must focus on ergonomics (standing desks, good chairs) and the hardware their employees need to be effective at home. If the employer doesn’t provide them directly, they might consider offering a stipend for employees to make the upgrades themselves.

(Organizations might even consider providing generators to workers who live in rural areas. I have one at my house. And once satellite internet service comes to my town, I’ll have a much more reliable digital infrastructure.)

Now we can start building on that infrastructure. Start with audio and video communication (Zoom, Teams, Dialpad). Then add chat (Slack, Google Chat) and a good customer relationship management (CRM) system (Salesforce, HubSpot). Where does your data live? Having a great CRM system is critical.

Using Digital to Put People First

Organizations ignore the importance of digital infrastructure at their peril. Not only are the days long-gone when employees stayed with a company for 30 or 40 years, but we’re now in the midst of the Great Resignation, where workers are leaving their positions in droves. So it’s incumbent upon organizations to listen to their employees and give them what they need to be successful in a hybrid environment.

And we’re not just talking about tech. Organizations need to keep employees engaged to really understand the pulse that’s around the organization because, as we all know, the culture and tone of an organization can go ebb and flow. It’s critical for organizations to really listen to what employees are asking for and to be flexible so that they can react and adapt to changing times.

If they don’t, a ruptured internet cable may be the least of their worries.

October 8, 2021
Microservices Adoption and the Software Supply Chain
Software development is undergoing industrialization, with more and more software rapidly assembled out of components and an emphasis on building automation around software validation and release processes.

Modern cloud-native software is no longer a monolithic application living in a single repo with the majority of its dependencies self-contained. It is integrated from third-party components provided by vendors, cloud providers and open source software (OSS) components, with as much as 90% of code coming from such dependencies. This allows developers to build applications much faster but makes maintenance much harder as it is no longer entirely under their control. If the third-party vendor makes a change to their API, developers are on the hook to update their applications before they break.

Each microservice interacts with hundreds of other microservices and each is built and released independently, making it hard to understand how to coordinate API changes across them. For example, if you want to make an API change to a microservice, what impact will this have on the consumers in the organization? Who will need to coordinate the change? When it was all within the same repository and released all at once, it was much easier to make such changes.

The Changing Nature of Technical Debt

What we continue to call technical debt is really the activities that are related to tending to and upgrading our software when third-party components are evolving or have common vulnerabilities and exposures (CVEs) and need to be upgraded.

These are tedious, repetitive tasks that usually fall to the most experienced engineers as they require technical expertise to do correctly. Such activities can paralyze engineering organizations and are a tremendous burden on engineers; that often leads to burnout. Up to 30% of engineering time is spent on technical debt. The perception that somehow developers were responsible for accruing this technical debt and are doing something wrong that prevents them from keeping up is hugely demoralizing and demotivating.

However, if we reframe technical debt as software supply chain management and stop blaming engineering for it, we can make maintenance more predictable and consistent. By taking steps like inventorying third-party components and determining how pervasive they are in the application (frameworks take more effort to maintain than a third-party API that you just call from one part of the application), an organization can arrive at a maintenance estimate.

These activities are highly repetitive across organizations, as everyone is integrating some subset of the same third-party components to create business value. This high level of repeatability points us to automation.

Specialized Developer Tools for Software Maintenance Don’t Exist

When developers write code, their hands often lag behind their thoughts. They think much faster than they are able to type the code. You often hear developers finishing a tedious task and complaining, “My fingers hurt.” When their IDE can autocomplete syntax, it helps reduce this lag to minutes.

When a major framework version is released, developers contemplate the codebase. Maybe they need to change this type to that, reorder some method arguments or change dependencies. The types of changes they need to make are enumerable and must be made almost immediately; again, the ability to implement them lags far behind their recognition of the problem. Often, this type of lag can represent months or years of work when dealing with a large codebase. Accumulate enough of this lag and the codebase can grind to a halt.

That lag kills the joy in development for most senior developers and leads to burnout. Upgrades/migrations amplify that lag by orders of magnitude.

However, there are few (if any) tools that focus on helping developers automate these remediations/upgrades/migrations. The IDE’s focus is rightly on helping developers write new code and maintain their own code, but code maintenance needs to be coordinated across multiple places across the codebase, within the same repository or across the repository boundaries. Often, developers have a pattern they want to change in their repository, but the IDE only suggests this as an improvement in their current location (one file). Other times, changes need to be coordinated across repository boundaries if developers want to change APIs and their consumers. That’s why this technology needs to exist outside of the IDE.

OpenRewrite

OpenRewrite is an open source project that offers semantic analysis and refactoring of code as standalone operations so that everyone can use and contribute over time; composing more and more refactoring operations that make whole framework migrations possible. It is integrated with build tooling and can be plugged into different workflows, from CI integrations to mass refactoring of multiple microservices/repositories. Similar to an IDE or other developer tools, OpenRewrite manipulates an abstract syntax tree (AST) representation of code, but this AST has special characteristics that allow it to transform this AST and generate code back to the standard text representation.

The OpenRewrite AST is produced by guiding the compiler through the first two phases of compilation to generate compiler type-attributed AST. This is then mapped to the OpenRewrite AST, preserving formatting and breaking cycles in the AST. The three unique characteristics of OpenRewrite AST are:
- Type aware, allowing 100% correct semantic code analysis and transformation.
- Style-preserving, so transformations produced are idiomatic within the projects they are applied to. So, the same transformation applied to multiple projects will potentially look different.
- Serializable, allowing to output the AST from the build and operate on it outside of the build en masse.
OpenRewrite ASTs is full of metadata about code. Visualizing the code itself as a highly connected tree of various vertices and edges, we can start to get an idea of the density of this representation. Syntactic elements alone are blue, type attribution is yellow and formatting information is red. If we were to just represent syntactic elements, the tree might look something like this.

Adding type attribution and formatting, the tree looks quite a bit denser.

A spanning tree of the data involved in this AST can be laid out in three-dimensional hyperbolic space and then projected onto the unit sphere similar to the way that two-dimensional space can be projected onto a disk—the famous Poincaré disk model. This three-dimensional approach was pioneered by Tamara Munzner at Stanford.

Every edge that isn’t part of the spanning tree is rendered as a lighter gray, and, as you can see, it is dense! In fact, to make this layout more tractable, we’ve omitted in this view an additional 600,000 vertices representing type information that make the syntax itself seem tiny.

OpenRewrite calls a single code search or transformation operation a recipe. It provides a number of building block recipes such as find method, change method, find transitive dependency, upgrade or exclude dependency. These recipes, in turn, can be composed into more complex recipes by grouping them together into a composite recipe. When the building blocks are not enough, a recipe can be written as a program in the same language as the code we want to transform, allowing us to encapsulate complex logic with the full expressiveness of the language already familiar to developers. We don’t need to learn a new DSL or programming language.

These building blocks abstract away many of the details to ensure that edits that we make to source code obey the original style of the project. This is possible for even complex changes like this automated migration from Spring Boot 1.x @ConditionalOnBean to Spring Boot 2.x AnyNestedCondition looks idiomatically consistent in the context of the project that it is inserted into:

It is important to note that OpenRewrite is not a replacement for an IDE. With software now composed of so many independent components, we need to start to remediate and manage it using new approaches that go beyond single lines of code to looking across entire code repositories.

Olga Kundzich, co-founder of Moderne, co-authored this article.
October 8, 2021
Tips for a Successful DevSecOps Life Cycle

A DevOps implementation, if done correctly, can do wonders for any organization that’s on the hunt for efficiency, productivity and speed. As per the 2020 survey conducted by Atlassian, 99% of survey respondents said that DevOps has had a positive impact on their organization.

However, it won’t do any good if security isn’t prioritized while adopting the DevOps best practices. That’s where DevSecOps comes into play.

Enterprises need to understand that leveraging the alignment of development and operations for workflow enhancement while ignoring underlying security issues is like trying to generate electricity from a broken solar cell on a cloudy night. Undoubtedly, the success of any DevOps initiative predominantly relies on general security practices and a robust security mechanism to mitigate risks—this is where DevSecOps plays a vital role.

There are several essential components of a successful DevSecOps implementation. Here’s how you can leverage them in your organization.

The Emerging Need for DevSecOps

DevOps can be simply defined as the next evolution of an agile development environment that bridges the gap between operations and development teams. Enterprises increasingly are turning to DevOps to boost their application development speeds and operational efficiency. Often, though, prioritizing development velocity means security concerns are left behind.

That’s why there’s an equally growing need for a secure, efficient yet still agile development environment. DevSecOps puts security at the forefront of the entire operational and development process, which safeguards good cybersecurity hygiene right from the beginning. Implementing DevSecOps directly benefits developers and operations teams working hard to deliver a flawless application in a short period of time. Moreover, this approach encourages enterprises to develop secure code and thus, secure applications. To do so, there are several resources and strategies to help implement DevSecOps into an organization’s software development life cycle (SDLC).

Secure Coding Practices

Secure coding is perhaps the most crucial aspect of ensuring stringent security right from the beginning of the SDLC. Developers who aren’t following secure coding best practices, security guidelines and maintaining compliance while coding the application are inviting many security risks, including breaches of sensitive data and exploitation of an individual’s identity.

Hence, an organization must focus on enhancing the skills of developers and ensuring that they are following security best practices while their application is in the initial stages of development. Moreover, establishing and adherence to coding standards are also crucial, as they help application developers to write clean code.

Leveraging Automation

As we know, automation is a driving force and a key aspect in DevOps; the same goes for DevSecOps. Automating security is vital to keep up with the pace of application code delivery, especially for more prominent organizations working on various code versions. Choosing the right automation testing tools can be beneficial for organizations that wish to speed up their operations and enhance the reliability of code when it comes to security. Organizations should strongly consider using at least static application security testing (SAST) tools that help developers identify potential vulnerabilities in code in the early stages of the development life cycle.

Early Stage Testing

The biggest mistake for any organization leveraging DevSecOps is testing the application only when it’s finally completed. Beginning early with automated testing offers numerous advantages, including identifying and resolving vulnerabilities in early development stages, a quick and inexpensive way to fix issues and saving valuable time during deployment.

However, testing during the early stages of the SDLC may also halt the overall DevOps development pipeline, so take that into consideration from a security perspective.

Implementing DevSecOps

Implementing DevSecOps isn’t a quick process and requires specific steps to get it right.

Planning and Development

A solid planning strategy for successful implementation must include test criteria, design, vulnerability models and analysis. Once the planning is complete, the next step is development. Development teams should initially evaluate the overall complexity and challenges of their current practices. Adding a code review system in the development process can help to encourage safe and reliable coding practices in a DevSecOps environment.

Build and Test

Now it’s time to build! You can use automated tools to help combine source code with machine code; many of those build automation tools offer security features like automatic vulnerability detection. Once the build is complete, rigorous testing is carried out through various automated testing frameworks that ensure the highest level of performance and security.

Monitoring

Monitoring is another crucial aspect of DevSecOps implementation to ensure your security mechanisms are performing up to snuff.

Undoubtedly, DevSecOps has revolutionized the way enterprises handle security, especially in challenging DevOps environments. Businesses that want to get to the next level of agility without sacrificing robust security should be implementing DevSecOps.

October 4, 2021
The Role of APIs in Mainframe Modernization
Mainframes are historically clunky systems. They are incredibly secure and can process impressive calculations, yet they are complex to manage and challenging to extract data from. This reality is at odds with modern cloud-native infrastructure, which emphasizes distributed computing and portable microservices. So, how can existing mainframes adapt to this connected, data-driven economy?

I recently met with Dr. Alex Heublein, president at Adaptigent, to talk about the state of legacy modernization. According to Heublein, many mainframes are here to stay for the foreseeable future. However, enterprises are continuing to get value from these systems by adding an integration layer on top of them. By using a REST API layer on top of a mainframe, enterprises could finally open up these systems. This could facilitate bidirectional communication with newer cloud-native services, enable phase migration and edge closer to real-time processing.

State of Mainframe Modernization

The software industry changes every day. “Nothing in the tech field is permanent,” said Heublein, yet some legacy technologies will persist into the foreseeable future. In terms of mainframe adoption, Heublein sees a couple of categories of users. First are the large financial services organizations, such as large banks and insurance companies, which are considerably invested in their mainframe architectures and which will likely continue to rely on mainframe environments for their policy management and core infrastructure. “I think there’s a lot of good reasons for that,” explained Heublein. “If you want to process billions of transactions a day, [mainframe] is still a good option.”

That said, regulations and market pressures are influencing a global movement toward open banking. For example, PSD2 mandates open banking in Europe and industry consortiums like Financial Data Exchange (FDX) are directing it in the U.S. market. These initiatives are encouraging financial services to transform their legacy environments and spurring cloud migration, especially for applications that aren’t mission-critical.

Heublein pointed out a few different patterns in enterprise mainframe modernization. One approach is that enterprises are rewriting or re-architecting old COBOL applications and keeping them on-premises. Or they are porting and re-platforming the application; moving a COBOL app to a Windows or Linux system, for example. Finally, some are shifting straight to the cloud. For solutions that don’t need to be on the mainframe, enterprises may shift toward cloud-native to free up capacity.

APIs Enable Legacy Modernization

Distributed cloud technology is helping businesses scale to the right level. Yet, it’s difficult to jump off the mainframe entirely, noted Heublein. “Very rarely can you take all your stuff and move it all at once,” he said. “It’s a tough pill to swallow.” It’s especially difficult to move core banking infrastructure to a distributed environment—an instant shift could introduce security risks. For this reason, he recommended a ‘phase migration’ approach in which individual components are slowly migrated. For example, an insurance provider may shift their claims engine, policy engine and ratings engine to the cloud one at a time.

As a result of phase migration, however, engineers must simultaneously support both on-premises and cloud-based applications. They must also ensure bidirectional communication so that these systems can exchange data. This is where a REST API transactional layer is necessary, Heublein explained. Such a layer could enable bidirectional communication between the mainframe and cloud-native applications, abstracting complexity when calling out to the modern world, or intercepting cloud data and filtering it into a format mainframe technologies can process.

To increase developer usability, abstracting complexity from the mainframe integration process will be the key to building out connectivity. This could involve a codeless development approach with a COBOL sub-routine to call external systems, Heublein said. Automatic restructuring of data will be necessary to enable fluent bidirectional communication, too.

Four Business Drivers of Mainframe Integration

So what is the end result of putting an integration layer on top of mainframe systems? According to Heublein, there are four key business benefits of using such an abstraction layer:
1. Orchestration: Software ecosystems often need to make multiple calls to different applications to collect and transform data. Yet, mainframe environments are very complex and challenging to integrate with. More usable integration fabric for mainframe systems could enable better orchestration of workflows in a developer-friendly manner. By using a REST interface, all apps could be calling the same interface, thus greatly simplifying orchestration between disparate services.
2. Sustained Adaptability: The world is rapidly accelerating, and enterprises need to maintain and modify these APIs very quickly. Organizations require “sustained adaptability” to adapt and survive in the digital world, said Heublein. Mainframe systems, however, were not designed for the 21st century. Thus, low-code integration layers are critical to empowering rapid development.
3. Data Transformation: Data is powerful for the business. Data is not only a commodity in and of itself but also is required for AI and machine learning. However, mainframes have a completely different way of encoding data. Thus, there is a need to transform this data into a modern format that businesses can consume. An integration layer between these systems can deliver an improved data consumption model.
4. Real-Time Processing: You may think online payments happen in real-time; however, payment apps like Venmo are really just an abstraction layer over bank transactions, which may clear as little as twice a day. In short, mainframes are not great at real-time processing. However, more usable integrations to this infrastructure could enable user-facing applications to interact more closely to real-time.
Another concern is security. Opening up mainframe environments that house core financial infrastructure must be done with extreme care—especially as API attacks remain the number-one rising threat in the software landscape. All organizations building out new integrations will require a secure layer from an outbound standpoint. Heublein said this often involves placing an API management layer or gateway in front of the integration layer. This system enables authorization and functions like a firewall between the internal and external world. It can also enable logging and prevent denial of service attacks, he explained.

Improving Agility for Legacy Systems

Legacy systems are entrenched in older technology and carry lots of technical debt, forcing operations to move at a slower pace. Maintainers are also culturally a bit different than the newer breed of cloud-native engineers. “Mainframe people are not … fast,” said Heublein.

DevOps folks want to build, test and promote projects to production very quickly and, arguably, this agility is necessary to satisfy rising digital innovation demands. Improving integration capabilities for on-premises systems could enable businesses to better utilize preexisting investments while also improving agility.

Low-code/no-code capabilities could also increase collaboration potential, Heublein added, which could bridge the gap between on-premises engineers and fast-moving business folks. When adopting low-code/no-code tools for mainframe integration, he cautioned against general-purpose citizen developer frameworks. Whereas universal integration layers can work well for many situations, he stressed that a very specific boundary box is best for building APIs for mission-critical applications.

How is your organization approaching modernization for legacy systems? Let us know in the comments below!
October 4, 2021
4 Tips To Plug The Cloud IT Skills Gap
Technical talent is few and far between. The programming world faces a developer shortage, and with this shortage comes a lack of cloud know-how. By now, most organizations have become multi-cloud, yet 86% of IT professionals believe a lack of these skills will slow down cloud projects.

As more teams transition from older architectures to cloud environments, the IT skills gap could pose a severe bottleneck to software development and deployment. Cloud servers require ongoing maintenance, and managing security and compliance is challenging, even for seasoned developers. Furthermore, it’s easy to overspend in the cloud—without keen insight into vendor nuances, it becomes challenging to select the most cost-effective package for the application at hand.

I recently talked with Joe Spurrier, chief technology officer and co-founder, cloudtamer.io, to see how teams can improve their cloud strategies while navigating a short supply of IT cloud know-how. According to Spurrier, increasing automation, sharing reusable templates, centralizing best practices and maintaining agnosticism are strategies to amplify cloud results. Below, we’ll expand on these areas to see how they maximize cloud benefits while retaining a lean engineering footprint.

Automation, Automation, Automation

“Automation is key to furthering your adoption and use of the cloud,” said Spurrier. The drive to automate comes as no surprise to many DevOps and SRE professionals whose goal is to automate their job away. When automating the cloud, infrastructure-as-code (IaC) is a fundamental necessity. “If you’re getting into cloud, you should probably have some scripting skills,” said Spurrier.

Automation can significantly aid scalability when setting up additional accounts since engineers must replicate similar operations, such as server provisioning, deployment patterns, monitoring and so on. Automation is also essential for shift-left testing and to increase observability into production environments. “It’s one thing to deploy an application via automation, but another to have tests in automation—that’s the next level,” said Spurrier.

Now that so many tools are API-driven, orchestration becomes programmable. A cloud service provider’s CLIs or SDKs could also be leveraged to hook into these integrations to construct such automation.

On the flip side, too much automation for high-level privileges could backfire. “Someone could break into a data center, steal a server, disconnect it, then run off,” Spurrier explained. Thus, automation will require a higher sense of security to ensure that every change is appropriately authorized and logged.

Reusable Templating

Along with automation, documenting operational processes to be shared amongst engineering teams is vital for maximizing efficiency in a lean talent landscape. The key here is to make the process self-documenting and reusable, stresses Spurrier. This will most likely involve someone on the team creating shared libraries and repositories for other team members to leverage.

One such example is Amazon’s golden AMI pipeline. Golden AMIs are virtual machines that teams can “standardize through configuration, consistent security patching and hardening.” Golden AMIs can also enable a repeatable process for performing monitoring, logging and even distribution to business units.

Reusable patterns are important for building CI/CD systems so that when you commit code, it is automatically deployed to preferred cloud providers. In general, Spurrier recommends building shared repositories that are easy to clone, with step-by-step instructions for usage. Sample applications that demonstrate components in different languages can also aid developer usability and save time.

Adopt Centers of Cloud Excellence

Another way to encourage better adoption is by creating a center of cloud excellence (CoCE). For an enterprise, having a cloud-centric project management group can be very helpful to organize knowledge and offer support. A CoCE could centralize operations, supply engineers with pre-configured containers and help distribute access to cloud accounts.

A CoCE is a good way to start to unite disparate teams around shared best practices, said Spurrier. Depending on your organization’s needs and workloads, a CoCE will focus on varying skills, he added. In the beginning, a company may have just one center for cloud knowledge. However, as adoption grows, dedicated centers may likely emerge around specific cloud providers, too.

Retain Agnosticism

Contrary to popular belief, multi-cloud steups aren’t always about lift and shift. Business units often adopt a particular cloud to suit specific workload requirements. For example, researchers may choose GCP for its data science functions, or Microsoft shops may prefer Azure.

In a multi-cloud world, management layers could help retain a modicum of vendor neutrality. For example, take Cloud Custodian, a tool developed at Capitol One and recently contributed to the CNCF. Using Cloud Custodian, engineers can run security checks across AWS, Azure and GCP and generate metrics for reporting. Such control and insights could empower teams to automate some of the multi-cloud management burdens.

When working with multi-cloud, it’s also good to consider agnostic tooling that can work across different providers, like Terraform or Kubernetes, Spurrier said.

Templating

Cloud transformation is ongoing at most organizations. With this rise, more cloud-native processes and tools are emerging. Yet, with talent in short supply, DevOps isn’t always fully prepared. To review, we covered four ways businesses can circumnavigate this issue to get an edge:
- Increased cloud automation is a necessary response to a slimmer labor pool. Though building automation requires upfront technical skill, it pays dividends in increased efficiencies.
- Reusable libraries and templates can rapidly configure new projects and do so consistently across business units.
- Consider maintaining a center of cloud excellence to centralize policies and offer support.
- Opt for agnostic tools. This helps to retain vendor neutrality and apply consistent controls across clouds.
Of these above strategies, Spurrier emphasized the importance of creating shared templates. When used right, a templating approach can be very effective, he says. “If you have a templating model, it should be easy for DevOps teams to migrate to new features. If you’re leveraging tools that set up templates, it’s easier to figure out how much infrastructure is going to cost.”
September 30, 2021
CloudBees Acquires Neuralprints to Shift Compliance Left

At the online DevOps World 2021 conference today, CloudBees revealed it has acquired Neuralprints to provide the core technology for CloudBees Compliance, a real-time compliance and risk analysis platform that it will roll out in the first quarter of 2022.

At the same time, CloudBees has enhanced the feature management capabilities of its namesake continuous integration/continuous delivery (CI/CD) platform to provide greater visibility into and control over feature flags that are now widely used to add new capabilities to applications.

Prakash Sethuraman, chief information security officer (CISO) for CloudBees, said CloudBees Compliance will make it possible for DevOps teams to continuously enforce compliance policies as part of an effort to better secure their software supply chains. Those DevOps teams will be able to enforce compliance across code, binary artifacts, data, identity and infrastructure environments in a way that provides developers and IT operations teams with instant actionable feedback to enable issues to be addressed long before an application is deployed in a production environment.

CloudBees Compliance is based on a mix of open source and proprietary technologies developed by Neuralprints and will be made available for on-premises IT environments as well as via a software-as-a-service (SaaS) application. The goal is to enable organizations to manage compliance as code as part of any effort to shift more responsibility for application security further left toward DevOps teams, said Sethuraman.

A survey of 500 C-level executives published today found nearly half (45%) admit that initiatives to secure their software supply chains are halfway or less-than-halfway complete. Nevertheless, 95% claimed their software supply chains are secure (95%), with more than half (55%) posting they are very secure.

A full 93% said they are prepared to deal with an issue such as ransomware or a cyberattack on their supply chain. However, nearly two-thirds (64%) admitted they are not sure who they would turn to first if their supply chain was attacked; with 58% admitting that, if they experienced an attack, they have no idea what their company would do. An equal percentage (64%) said it would take more than four days to fix the problem if they did experience an issue.

Almost all C-level executives (95%) said they think more about securing the supply chain now than they did just two years ago, and 92% said a security issue would impact their brand. A total of 83% said security issues cause their developers to drop everything to review code, with 82% noting that caused the organization to spend less time on innovation.

On the plus side, 95% of executives said container images are checked for high or critical vulnerabilities, while an equal number said automation access keys are set to expire automatically. A total of 92% said their company only accepts commits signed with a developer GPG key, while 90% said dependencies to trusted registries are limited at their organization, while 89% said administrative access to CI/CD tools is restricted (89%).

Of course, what C-level executives believe and what actually occurs inside an organization are not always one and the same. A series of high-profile software supply chain breaches resulted in a number of organizations reviewing their software supply chains. At the same time, the Biden administration has issued an executive order requiring all federal agencies to conduct similar reviews.

One way or another, most organizations will soon find themselves embracing a wider range of DevSecOps best practices to ensure the integrity of software supply chains no matter how secure they may think their software development platforms are today.

September 29, 2021
Red Hat Extends Scope of Ansible Automation Ambitions

During the online Ansiblefest, Red Hat announced the availability of a preview edition of Red Hat Ansible Automation Platform 2 that is intended to make it easier to automate IT processes at scale.

Richard Henshall, senior manager for Ansible product management at Red Hat, said previously most IT teams have been automating individual processes, but as organizations become more adept at creating playbooks to automate processes, they now need to find ways to manage multiple automations that are often dependent on one another.

Ansible Automation Platform 2 also enables IT teams to automate processes that span both monolithic as well as cloud-native applications based on containers using an automation controller, formerly known as Ansible Tower, that is more tightly integrated with Red Hat OpenShift, an application development and deployment platform based on Kubernetes, using Red Hat Operator tools. There is also an automation mesh capability that connects disparate automation components together and provides status checks.

Ansible Automation Platform 2 also introduces automation execution environments within Ansible Engine that make it possible to create self-contained automation spaces that can be easily replicated and repeated across an organization.

Red Hat also includes an automation content navigator to helps teams validate that automation content is working as it should across even the largest environments, along with updates to the Red Hat Insights analytics service that enable issues that might occur with automation controllers to be surfaced and eliminated by automatically applying policies.

Henshall said going forward, IT teams should expect to see Red Hat focus more on enabling events to be the catalyst for triggering automation. Today, most automated processes begin with an IT administrator triggering the process. However, as digital business transformation initiatives mature, it’s become apparent there will be a need to trigger automation when specific events occur.

Henshall said while there are still many organizations that are just starting to automate IT processes, there are also many that are entering the next phase of maturity. As such, they now require an automation platform that makes it simpler to manage automation at scale.

In general, Ansible is designed to make it possible for developers and IT operations teams alike to declaratively create automation without much programming expertise required. However, creating a playbook that works reliably still requires a deep understanding of the processes that are being automated. Most IT teams would be well advised to closely monitor processes when they are first automated to make sure they are performing tasks as anticipated.

Ultimately, it’s now a question of when rather than if most IT processes will be automated. DevOps teams, of course, have always been at the forefront of automation, but now it’s clear that as IT environments become more complex—with the rise of both the cloud and edge computing platforms—enterprise IT teams need to embrace automation more aggressively. The biggest barrier to automation is simply finding someone who has the passion to automate a process to lead the charge, said Henshall.

Naturally, most of those efforts will initially focus on repetitive, tedious and manual IT tasks, but the potential to automate IT processes at levels of unprecedented scale has never been greater.

September 29, 2021
Relyance AI Shifts Compliance Left Using ML Algorithms

Relyance AI emerged from stealth this week to unveil a namesake platform for managing privacy and data governance in real-time within the context of a larger DevOps workflow.

Fresh off raising $30 million in seed capital, Relyance AI co-CEO Leila Golchehreh said organizations can now quickly and accurately answer essential privacy questions and ensure compliance and governance by mapping requirements directly to their codebase. The platform provides transparency and visibility all the way down to the source code in real-time using machine learning algorithms, she added.

Relyance AI co-CEO Abhi Sharma also noted most legal and data protection teams today have limited visibility into digital operations. Developers and data scientists are constantly creating and updating applications. By the time compliance teams gather the data required, it’s obsolete, said Sharma.

The Relyance AI platform analyzes contractual requirements against the operational reality as determined by data processing in real-time. Compliance teams can then address privacy issues as they arise, added Sharma.

At the core of the Relyance AI platform are five integrated components that first inventory assets by connecting to application programming interfaces (APIs); generate universal records of processing activities (ROPAs) by scanning code and understanding how data is being processed; share alerts when issues are detected; provide views into data life cycles to highlight data subject access request (DSAR) exceptions and extensions and address vendor life cycle management to ensure third-parties are complying with policies.

Sharma said compliance, much like security, is shifting further left toward application development as organizations look to eliminate issues before they arise in a production environment. The Relyance AI platform is designed to be integrated with continuous integration/continuous delivery (CI/CD) platforms that are the core of modern software development life cycle processes, he added.

Undoubtedly, as the worlds of compliance and application development collide, there will be cultural challenges similar to what is now occurring with the rest of DevSecOps best practices that require more collaboration between developers and security teams. The challenge is compliance officers tend to be even further removed from application development than their IT security counterparts.

However, it’s now only a matter of time before compliance becomes code. Modern applications already make use of the Open Policy Agent, for example, to embed compliance rules in applications. AI platforms should make it simpler to identify compliance issues as they arise so that they can be addressed before any infraction ever occurs.

DevOps teams, of course, have a vested interest in managing compliance as code. It’s not uncommon for DevOps teams to race to build and deploy an application only to encounter a wave of compliance issues that need to be addressed by recoding elements of the application. Not surprisingly, those compliance issues often result in application delivery deadlines being missed. Machine learning algorithms provide an alternative to the current raft of documents that compliance officers create, but not enough other members of the IT team read those, much less understand them.

September 23, 2021