devopsy.kinsta.cloud

Category: Blogs

Tackling Audit Compliance as Code

Do the words, “It’s audit time!” make your stomach sink? If so, you’re not alone. Assisting with evidence collection for compliance audits around PCI DSS, SOC-2, ISO 27001, NIST and HITRUST is a drain on DevOps teams’ time and resources at companies of all types and sizes – time and resources that could be better spent driving innovation.

The good news is, vendors have taken notice, and are delivering new tools and platforms that use automation to eliminate repetitive manual tasks and dramatically speed the audit process. This is entirely in line with the DevOps philosophy of tackling various IT tasks and requirements “as code” to remove tedium and improve efficiency and orchestration. In fact, as this movement to improve the compliance process picks up steam, I believe it’s entirely likely the next evolution is the use of open APIs and online communities where engineers can build and share software to automate a wide variety of evidence collection for compliance.

But before we can get there, let’s understand where we are today and assess new opportunities going forward.

Audit Support: DevOps’ Favorite Task

As keepers of cloud environments, DevOps teams typically spend hundreds or thousands of hours each year gathering evidence for compliance audits. This tedious, manual labor takes significant time away from their primary job responsibilities. Large enterprises often have multiple, overlapping audits each year, with little break from the grind of evidence collection. Companies that make heavy use of the public cloud face additional challenges; the increased complexity means that evidence collection is more difficult, and the fast pace of software development and change management in the cloud (compared to more traditional monolithic application updates) makes it difficult for compliance to keep up. This contributes to an overall lack of clarity and confidence in the compliance process at the leadership level.

Audit Prep as Code

When I ran a cloud engineering team at a previous job, we realized the vast majority of repetitive, manual labor involved in evidence collection could be automated to save time and free up resources for other business needs. This automation also improved the completeness and accuracy of the evidence gathered, ensured that important areas weren’t overlooked and allowed for more regular assessment to prevent “drifting” out of compliance in the fast-paced cloud world. While automation helps with multiple aspects of compliance audits (such as evidence-to-control mapping and cross-walking evidence across common control frameworks), it’s the automation of evidence collection that primarily impacts DevOps.

In this scenario, automated collectors crawl your public cloud environments and connected SaaS systems to rapidly gather technical evidence such as specifics on user access, encryption of data at rest, key management, network segmentation, firewalls, vulnerability scan reports and more. The results are transformed into formatted reports ready for internal review, analysis and auditor review/approval. This, alone, can save thousands of hours annually for typical enterprises facing multiple audits. Even better, the data is considered more complete and accurate from a compliance perspective, because it includes time stamps and other metadata to ensure capture and processing integrity.

As mentioned, this is well aligned with the broader “X as code” mantra that underlies many DevOps automation initiatives – and why not? What could possibly be a more repetitive, manual task than capturing, renaming, organizing and describing screenshots?

The Scripting Curse

When confronted with evidence collection, an engineer’s first thought is usually, “I can write a script for that!” Scripting is a form of automation, but it has some limits in this scenario. First, scripting and doing the necessary QA shouldn’t be your day job; there are better things to spend time on. Second, given the explosion of cloud infrastructure and SaaS apps, there can be a lot of scripts to create. In fact, it’s arguably a never-ending task. Finally, while you understand your scripts, who’s doing the next audit? If it’s not you, are you going to explain and train someone else on all the work you’ve done? Scripting elements of evidence collection is an improvement over doing it manually, but is still only one piece of the larger puzzle.

Compliance is a Community, not an Island

So what’s the best way to automate evidence collection? Multiple vendors are tackling the challenge, from cloud platform providers to established GRC vendors and cloud compliance startups – and new tools have already begun to hit the market over the last few months.

But as discussed, a primary challenge is that there is a lot of complexity in most cloud environments, and tools that approach the puzzle from only one direction – say, a particular cloud platform – still leave a lot of potentially relevant systems untouched. In fact, when you take all of the variations possible with cloud infrastructure into account, this might, ultimately, be a challenge that is best tackled at the DevOps community level.

Consider this: adherence to industry standards for data handling, privacy and protection is not a competitive issue. We are all better off if everyone shares a basic commitment to complying with these IT security and compliance standards. Viewed from that perspective, the more we can lower the barriers to achieving this compliance, the broader the adoption will be.

What I hope to see is a DevOps community that coalesces around automating audit readiness. At the engineering level, this could entail sharing of custom collectors. For platform and SaaS vendors, it means ensuring open APIs that allow capture and extraction of relevant data. And the compliance tool vendors must design their collection engines, evidence libraries and usage policies such that customization and sharing is possible. Similarly, automation tools must not only pull data from standard systems, but support pushing data from custom systems or legacy infrastructure. In my view, this bottoms-up approach is the best way to stop wasting DevOps’ time on audit prep, while still allowing enough autonomy to do what works best for their specific environment.

In this future, then, perhaps an upcoming compliance audit won’t produce a collective groan from the DevOps team – or at least, generate a quieter one.

March 16, 2021
Measuring the Value of DevOps-as-a-Service (DaaS)

The late great W. Edwards Deming, statistician, professor, author, lecturer, and consultant said, “In God we trust. All others must bring data.” The point is, if you want people to trust your information, then you need to present measured data. This is true for countless circumstances. As a DevOps consultant, I am often asked by my clients what data should be used to show the value of their DevOps capabilities, in a way that is acceptable to the stakeholders of those DevOps capabilities. A lot of excellent research results, published by DORA and others, describes metrics that demonstrate the value that DevOps provides to application development. More and more enterprises are embracing the idea of providing DevOps capabilities, such as pipelines, tools, hardened containers and tech stacks, as a centralized service to support their many application teams. The question then becomes, “What metrics should be used to show the value of DevOps-as-a-Service (DaaS)?”

Before measuring the value of anything, you need to know two things: “What is it?” and “Why is it important?”

What is DevOps-as-a-Service?

In high-level terms, DaaS is DevOps capabilities made available to the users of those capabilities through portals and APIs. There is no standard set of services; DevOps capabilities offered vary from organization to organization. In my SlideShare, DevOps-as-a-Service (DaaS) Values, I illustrate a generic blueprint for DaaS.

The DaaS blueprint has four layers, as follows:

• Stakeholders – There are three types of stakeholders: business, app-dev users and DaaS teams.
• Service access layer – This layer consists of user portals, APIs, and access security.
• Service layer – The service layer consists of user services and management services. User services include things such as repositories, CI/CD pipelines, sandboxes, test data, configuration data, guides and training. Management services include service catalogs, issues and change requests, SLOs, SLIs, governance policies, experience monitoring and automated tasks.
• Resources layer – This layer includes application stacks, infrastructure resources, tools, infrastructure code and topology data.

Why is DaaS Important?

The value of DaaS, as perceived by different stakeholders, can be summarized as follows:

• Business – DaaS makes the business competitive.
• App-dev Users – DaaS improves application development.
• DaaS team(s) – DaaS improves management of DevOps capabilities.

What Service Level Objectives (SLOs) Demonstrate the Value of DaaS?

The SLOs need to be chosen to best fit the requirements of each application. The following are examples.

Business value can be demonstrated by:
ROI based on tracking cost savings relative to a benchmark comparing prior non-DaaS, trend cost per developer and per release.
SLO agility of services – Lead time for changes, and frequency of releases.
SLO reliability of services – change failure rate, time to restore services.
SLO governance, compliance and security scores.

App-dev user value can be demonstrated by:
SLO availability of DaaS, service level indicators (SLIs) depend on service components and architecture.
SLO service response time.
SLO experience scores.
SLO change request response time.

DaaS team value can be demonstrated by:
SLO scope of applications – number of apps supported, number of users per month.
SLO number of requests served per month.
SLO critical events per month.

How Does an Organization Develop DaaS Metrics?

Start by picking an application that is supported by DaaS. Do not try to boil the ocean and measure all applications at once. Then, appoint a DaaS SLO/SLI team to work on metrics development for the application. The team should include stakeholders from business, app-dev and DaaS support teams associated with the application.

DaaS SLO/SLI team members determine the SLOs of DaaS that each class of stakeholder values the most, and then decomposes the services to identify sources of data that will form the most important service level indicators (SLIs).

With the SLIs in hand, instrumentation, telemetry and analysis requirements can be determined and used to design monitoring agents, communication, analyzers, alerts and dashboards.

Then, roles, policies, playbooks and responsibilities will complete the solution. Test the solution for the chosen application. As you gain confidence in the solution, adapt and expand the solution for use with other applications.

Continue to monitor the progress and performance of the SLO metrics, and continuously improve them with experience and as newer technologies become available.

What This Means

DevOps-as-a-Service provides many valuable capabilities to business, app-dev users and DaaS teams. Metrics that demonstrate value to each class of stakeholder should be determined for each application that consumes the capabilities offered by each unique DaaS. A DaaS SLO/SLI team can determine the best metrics and system design for the implementation for an example application. The SLOs can be adapted and evolved for other applications in the organization, and used as a basis for continuous improvement.

March 16, 2021
IBM Deepens DevOps Insights on Mainframes

IBM today extended its effort to enable IT organizations that have mainframes to embrace DevOps best practices by updating the analytics tools it provides.

Tom McPherson, vice president for IBM Z Software, said a version 6.0 update to the suite of IBM application discovery and delivery intelligence (ADDI) for IBM Z tools surfaces more granular insights by leveraging a new version of Wazi Analyze that now analyzes code immediately after a change is made. That capability is critical, because most of the workloads running on a mainframe are highly tuned. Any issue that is likely to occur can usually be traced directly back to the latest software update made to the environment, McPherson said.

Wazi Analyze also can now be deployed as a set of containers, which McPherson said means IT teams can opt to run the analytics application on a platform other than the mainframe.

ADDI also has been updated to improve Assembler support by making it easier to analyze multi-level dynamic assembler macro calls. A set of application programming interfaces (APIs) for feeding that data back to a continuous integration/continuous delivery (CI/CD) pipeline has also been added.

Finally, IBM revealed it signed a licensing agreement with ITP Software Systeme GmbH, Munich (ITP) to enhance application analytics capabilities. That capability will be added to a future release of DevOps for IBM Z Hybrid Cloud, which includes ADDI and IBM Wazi Developer, to enable organizations to determine how best to modernize legacy monolithic applications as they transition to a microservices-based, cloud-native computing environment.

Naturally, not every monolithic application is going to be converted into a series of microservices. However, McPherson said, it’s clear there is a need for analytics tools that will enable IT teams to more easily determine which elements of an application are better suited to being reconfigured as microservices that could be deployed on, for example, a Red Hat OpenShift cluster.

For the last several years, IBM has been encouraging IT organizations that have mainframes to embrace DevOps best practices as part of a larger effort to position mainframes as another type of distributed computing platform. The overall goal is to lower the total cost of mainframe ownership by reducing the need to rely on specialists to manage mainframe application environments. Each year, more of those specialists retire. IBM wants to make it more feasible for organizations to replace those specialists with IT generalists, like those that typically manage distributed computing environments. At the same time, management of distributed computing environments is becoming more automated, thanks to the rise of DevOps.

Of course, mainframe processes have been highly automated for decades. IT teams managing mainframes may still rely primarily on waterfall-based processes to deploy applications, but the underling infrastructure is designed to scale up and down on demand as application workloads require. Convincing IT teams that, for decades, have employed the same processes to manage mainframes often requires a significant amount of cultural change.

Regardless of motivation, however, the winds of DevOps change are now blowing through mainframe environments just as fiercely as they have through the rest of IT.

March 16, 2021
5 Best Machine Learning Use Cases

Many industries are struggling with the ongoing COVID-19 pandemic, and the IT industry itself and the broader trend of transition to remote work over the last year revealed many areas where traditional approaches to managing businesses created unnecessary waste. Data science and its counterpart, machine learning, revealed that expansion in the ways technology can facilitate new ways of working is nearly limitless.

Machine learning uses powerful algorithms to discover insights based on real-world data. These insights can then be used to make predictions about future outcomes. As new data becomes available, machine learning-enabled programs can automatically adapt and produce updated predictions. As with any tool, machine learning is not a silver bullet. However, there are many situations in which the technology can outperform linear and statistical algorithms.

Here are five of the most common use cases where machine learning can make a big difference:

When engineers can’t code rules for certain problems. Many human-oriented tasks (such as recognizing whether an email is spam) aren’t solvable using simple (deterministic), rule-based solutions. Because so many factors may influence an answer, engineers would have to write and frequently update billions of lines of code. In addition, when rules depend multiple factors and when those rules overlap or need fine-tuning, it becomes difficult for humans to code precise rules. Fortunately, machine learning programs don’t require users to encode actual patterns. These programs only need proper algorithms to extract patterns automatically.

When you need to scale a solution to millions of cases. In finance, for example, you might be able to manually categorize a few hundred payments as either fraudulent or not. However, this becomes tedious or impossible when dealing with millions of transactions. As user bases grow, it’s no longer feasible for organizations to process payments by hand – end-users today want answers about their money in milliseconds, not minutes or hours. Machine learning solutions are effective at handling these types of large-scale problems with little to no human intervention.

When you can do it manually, but it’s not cost-efficient. There are situations in which in-house experts could process many requests quickly and accurately, but at a high cost. For instance, imagine you assess DMV forms for in-state and out-of-state car purchases to determine their validity before passing them on. In this situation, the business processes are well-defined, optimized and serialized. It may take only a few minutes to check each form thoroughly. But allocating so much manual labor to this work is likely not the best use of your budget. Machine learning, on the other hand, offers predictable, pay-as-you-go pricing for fully scaled operations.

When you have a massive dataset without obvious patterns. Consider this – you’ve successfully prepared a well-curated dataset and know the underlying problem you’re trying to solve. However, you don’t see any explicit patterns in the data, preventing you from encoding those validations. Plus, there are many typos, missing fields, and other human-caused errors with no validation in place. You may even know the data is poor quality and can manually determine every affected row. But you can’t see any actual connections between valid and invalid records. Machine learning algorithms can solve this problem. They can find hidden connections between data points that aren’t clear to humans. Tools like interpreting tracers can even describe how machine learning models arrived at their conclusion(s).

When you live in an ever-changing universe (adaptive). The world, and its problems, are always changing. A problem you solved yesterday can, today, easily mutate into something else entirely, rendering your previous solution inefficient or even useless. For example, if your organization processed medical appointment records to extract diagnoses, procedure information and billing codes, your rules might have to evolve constantly. However, you can’t make updates in real-time, 24/7. Meanwhile, incorrectly labeled items could lead to insurance rejections, huge fines and legal penalties. One major advantage of machine learning methods is that they can learn from data across the entire life cycle of your application – from the first line of code written to the moment when the model is finally shut down. Moreover, it’s important for production-grade systems to have feedback loops so that you can catch the moment when your model no longer solves problems correctly.

It’s important to remember that machine learning is a tool – it’s not magic. Machine learning models are, essentially, advanced math-based algorithms, which identify patterns in data and learn from them. However, when properly applied to the right use cases, machine learning can reduce the amount of time spent on error-prone manual IT operations, adding significant business value and greatly reducing IT costs.

March 15, 2021
How AI Will Drive Digital Twin 3.0
Testing is a critical task for digital twins, and AI will increase the efficacy of the testing model moving forward

Physics calculations may work perfectly well in theory. On a blackboard, academic science is pretty predictable (outside of the quantum realm, perhaps). Yet, nothing is manufactured in a complete vacuum, is it? When it comes to real-world settings, millions of factors could impact the state of a physical object — material, friction, temperature, pressure, altitude, wear… the list goes on.

With so many tangible conditions increasing the likelihood of deviation, it can be difficult to reproduce a digital twin that accurately represents real-world conditions. This is, in part, why some believe the next generation of digital twins will be more driven by artificial intelligence (AI). By leveraging neural networks created with sensor data, production could trust intelligence gathered from trends across data points instead of solely relying on theoretical models. This could enable better predictability, optimized operational performances and overall improved products.

To discover what the next generation of digital twins will look like, I recently met with Faustino Gomez, co-founder and CEO of NNAISENSE. Below, we’ll define what a digital twin is and why the concept needs an upgrade. We’ll also explore a few digital twin use cases to see how the next wave of AI-driven digital twins will be “learning directly from the data,” as Gomez describes.

What Is a Digital Twin?

No, we’re not discussing your cyber clone. Digital twins are a bit less transhuman than that. Put simply, a digital twin is a digital duplicate of a physical object or system. Digital twins are commonly used across many industries to monitor and test intricate apparatus. Industrial manufacturing, for example, often leverages a digital twin to monitor production and test the performance of equipment within a simulated environment.

Digital Twin 1.0

Some of the first digital twins are attributed to NASA. By reproducing mechanical objects locally, NASA could diagnose issues and test solutions for physical components 200,000 miles away in space, far beyond direct human intervention.

The digital twin concept also found roots in other mechanical devices, such as jet engines, enabling on-the-ground mechanics to diagnose issues in the air. Using digital twins, smart factories could mimic the behaviors of manufacturing robots to test behavior and identify potential bottlenecks.

Digital Twin 2.0 Adds More Data

Using a digital twin, engineers could represent complex interworkings of joints and parts within a computer screen — a 3D reflection of the device. As opposed to constructing physical models, virtual simulations significantly reduced the effort and resources required to test new concepts and diagnose problems. But, simulating reality in a black box wasn’t always easy.

Reproducing complex physical counterparts with a digital twin is wrought with complications that can be difficult to model in a standard CAD simulation, said Gomez. So, engineers began taking measurements from physical devices, such as vibrations or temperature, and integrating actual production data into the twin. This enabled operators to better forecast system performance by considering real-world elements. Yet, truly predictive forecasting was still lacking. That’s where AI comes in.

Digital Twin 3.0 Adds AI

Up until recently, digital twin and AI have been independent concepts. But with the advent of more sensors and greater data collection, integrating AI is becoming an inevitable evolution of the digital twin, said Gomez.

Adding an AI layer to a digital twin would involve training a deep learning model using hundreds of time-series data outputs from sensors. Generating this data would require capturing a chronological view of how systems work functionally for some time. “Then, take data and split it,” said Gomez. “Use some to train, some to test.” The process is not too dissimilar to Walk Forward Testing, common in financial trading, added Gomez.

Instead of using traditional statistics methods and standard physics, a neural net could unify data and find nonlinear relationships between data regardless of type. The end result could quickly identify cause-and-effect correlations between data sets (which simulated physics engines may miss) and predict future conditions based on new input.

Use Cases

With better intelligence, you get a more accurate depiction of possible outcomes. The use cases for crystal ball digital twins are numerous. Gomez described how 3D printing and factory maintenance could utilize digital twins:
- Additive 3d printing: 3D printing metal powder involves a complex thermodynamic process with high-intensity lasers. The process is often less than perfect, leaving cold and hot spots from layer to layer, but ignoring them could result in a defect. By analyzing each layer’s thermal displacements, a deep learning model could be trained on the intricate details of the process. This could be used to build a more refined process model, essentially enabling manufacturers to “pre-compute the part.”
- Predicting glass quality: Gomez described how a specialty glass company utilizes AI-driven digital twins. Their process model takes sensor readings at various chambers throughout its molten melt and cooling process, analyzing temperatures to maintain a uniform mix. Based on their controls, operators can predict what glass quality will be in the future. Essentially, it’s “a user assistance system based on a digital twin,” described Gomez.
As you can see in the examples above, a digital twin isn’t necessarily a single object — it could represent an entire process model for a large industrial operation. According to Gomez, having this digital twin could boost testing and R&D, improve development time, and decrease time to market.

Digital Twin 3.0 Forecasts the Future

For digital twins to become more effective, these simulations must be faithful to current and future real-world conditions. But, computing tests for situations involving wet chemicals, unstable materials, or thermodynamics is tricky. Therefore, to Gomez, predicting such dynamics really requires “learning directly from the process model,” he said.

The key thing here is a neural net doesn’t care what it’s monitoring. Be it torque, matter, friction, temperature, fumes or vibration, the model will learn regardless of what’s thrown at it. While the benefit here is you don’t have to know physics or chemistry, setting up this process requires much customization, which could differ significantly on the type of materials and process at hand.

The fusion of disparate data sets could bring interesting capabilities for testing and forecasting. Yet, quality and quantity of data is paramount, noted Gomez. Furthermore, these models won’t spring up in an instant — they must be tested and validated using real-world dimensions before deemed useful. While digital twin 3.0 is a promising step forward, more research will be required to ensure the predicted digital future reflects the physical outcomes.

AI-infused digital twins use “data from the process to predict what the process will be in the future,” said Gomez. This goes far beyond traditional 3D representations to produce more faithful, predictive digital twins.
March 15, 2021
Days of the Week

March 15, 2021
How to Know if You Need Multi-Cloud — and Succeed if You Do

There are many good reasons for running in multiple clouds, as well as times when multi-cloud environments are unavoidable due to organizational history or legal reasons. However, operating in multiple clouds adds complexity, increases operational overhead and can become expensive. Organizations should only pursue it when they have clear business reasons for doing so.

In this article, I’m going to cover some common situations when organizations can get value out of multi-cloud as well as strategies for success. Because multi-cloud is hard, the first question organizations should ask themselves is, “Do we really need multi-cloud?” So, the first order of business is to outline when technology leaders are better off avoiding multi-cloud altogether.

When Multi-Cloud is a Bad Idea

A single application and its data should never span multiple clouds, if it can be avoided. Here are some common reasons organizations adopt a multi-cloud approach when they would be better off avoiding it.

Short-term bursting (in hybrid cases). “I’ll just tap into a public cloud if my private cloud suddenly needs extra capacity.”

Disaster recovery. “Let’s run this application on AWS and Google in case one of them disappears overnight.”

Staying cloud-agnostic. “By running on multiple clouds, we are no longer locked in.”

In all the cases above, network latency and egress cost will hurt your application and your business. In the best-case scenario, you will spend a lot of time and effort prematurely optimizing for unlikely scenarios.

Unless you’re perfectly prepared, please, don’t go there.

When Multi-Cloud is a Necessary Evil

There are also plenty of times when multi-cloud is a drag on the engineering organization and a drag on the business — but can’t be avoided. Here are totally legitimate reasons to have a multi-cloud setup that will still require more effort to maintain than operating in a single cloud.

Mergers and acquisitions. Some organizations can end up operating in public clouds and private clouds as they acquire other companies. The effort required to migrate everything to one cloud provider often isn’t worth it, so everyone continues using the same cloud they used pre-acquisition.

Legal. Depending on the type of data you gather and the jurisdictions you operate in, you may have to worry about where your cloud provider has regions available. There will be times when you need a data center in multiple jurisdictions; none of the cloud providers are in all of them. In those cases, you don’t have much choice, you need to be multi-cloud.

If you need to use multiple cloud providers, either for legal reasons or because of mergers, it’s best to focus on making multi-cloud management as painless as possible — while still accepting that it will not be as smooth an experience as running all workloads in a single cloud.

When Multi-Cloud is a Solid Strategic Move

There are also some very good reasons to adopt a multi-cloud approach on purpose. For example:

Facilitating engineering speed. Engineering speed is paramount. Companies who focus on using multi-cloud to become cloud-agnostic and avoid lock-in often sacrifice development velocity, but if the organization instead uses multi-cloud to allow everyone on the team to use the tools and environment they are most comfortable with, it can increase velocity.

Best-of-breed tooling. The cloud providers’ offerings are not totally identical. Multi-cloud can let organizations use the best tools from each cloud provider; for example, machine learning technology from Google, compute from AWS and business intelligence from Azure.

Better customer experience. Especially if latency is a big concern, a multi-cloud approach can help organizations get physically closer to users, providing a better customer experience.

There is a common thread here: When evaluating whether or not you have a legitimate need for multi-cloud, the key question is whether you will be fulfilling a psychological need, a theoretical need (in other words, something that you do not actually need at this moment but might at some future date) or a concrete legal or technical need that you can easily describe. If there are no concrete legal, technical or business reasons for pursuing multi-cloud, don’t. There are costs to pursuing multi-cloud, so organizations that won’t actually reap benefits from it shouldn’t attempt it.

How to Succeed

Multi-cloud is hard. Here’s how to increase your chances of success so you can get the business benefits of better tooling or better customer experiences from your multi-cloud deployments.

Use a cloud management platform. Getting a single pane of glass for all your deployments is critical to managing multiple clouds without pulling your hair out. This abstraction, however, comes at the cost of supporting very deep and cloud provider-specific use cases, for example, using obscure service X.

Standardize workflows with Terraform and Ansible. You need to have processes that are as repeatable and predictable as possible. Even with standardized workflows, you still have to adjust for each environment, but the more standardization the better.

Use Kubernetes. Kubernetes lets you abstract the infrastructure layer and move workloads around more easily, even across clouds. The caveat here is that you have to adopt microservices and deal with the increased complexity of managing Kubernetes and its underlying infrastructure.

Each of the above approaches has its merits and flaws. They are not mutually exclusive, and you should not treat them like they are. Be prepared to mix and match depending on your needs.

Weigh the Costs and Benefits

Sometimes companies are not realistic about their technical capacity or what their needs are. They want the ability to do the latest thing they read about without thinking about whether or not that technology or approach will provide business benefits.

Multi-cloud adds a huge amount of complexity to your deployment. If multi-cloud truly is essential, however, use tools and platforms that hide that complexity from users and can simplify the management process.

March 12, 2021
Languages and DevOps: Traditional

For quite a while now, all the cool kids have been going on and on about “full stack development,” “API first,” and a ton of similar methodologies/practices. And yet, a huge swath of applications are still targeted at a single system to do a specific job. There are fat clients, there are internal servers, there are standalone apps. All are still in development, and the hype machine doesn’t account for them much, though they’re pretty pervasive.

We still have to maintain them, or write yet another. They’re not going anywhere, because there are some problems for which they are the right solution. So let’s talk about them.

For clarity, I’m defining traditional as, “Targets a specific OS/hardware platform, and is not primarily web-based.” We’ll exclude mainframe, because the toolsets and integration with DevOps are still different enough to warrant their own blog.

Online forums are full of people trying to re-target traditional apps (perceived Windows 10 issues caused many to want to move from Windows, and the growing Linux trend of making distros into beta – or even alpha – testers has many wanting to move from Linux), while others are just trying to keep them running as the world changes underneath their feet.

Easy Integrations

While these dev tools are the ones that Agile grew up around, that makes them well-integrated into Agile and DevOps. The feeds to/from compilers and various build tools are the same as other spaces we’re looking at (or, more precisely, the others are the same as traditional), so integration of AppDev is generally not an issue. Integration of Ops is harder, but exists. The various DevOps tools with an emphasis on operations (I like to call it devOPS as opposed to DEVops) are available to traditional development, and you can certainly deploy through them, but they’re like the second cousin to “full stack.” Good Enough rules the day, and a little work will be required. Thankfully, tools like Ansible and Puppet smooth the way a bit for deployment, not caring much if you’re deploying a traditional app or a web service, as long as you tell them what to do to successfully deploy.

You got to move it, move it…

Portability is increasingly becoming an issue. The rise and fall of “preferred” Linux distros, the costs of being a Microsoft shop … there are a lot of reasons to want to move your app around. The best first step to this (and the only step some will need, IMO) is containerization. Put it in a container, and your underlying hardware can run on whatever OS the corporation settles on, while your apps will run in containers on the OS they need. For most organizations, this will only be a stop-gap; a way to quickly move servers away from one OS and onto another, but the apps will have to move fully to the new OS, eventually. Microsoft has Windows Subsystem for Linux, which can help move from Linux to Windows, and Linux has Wine to help run Windows apps on Linux. Both have come a long way toward interoperability, but neither is perfect – or even easy. It will take a bit of work, but far less than rewriting for the target OS top-down. Tools like Cygwin are also available to get your Linux app running on Windows if you don’t want to use Microsoft’s solution. In the end, you are going to have to modify code, but you are porting an application, so no surprises there.

Consider the future

A place I worked in the late 90s/early 2000s had an OS/2 machine that was critical infrastructure. We were not allowed to touch the box. I think if the admins had their way, we wouldn’t have been allowed to look at the physical box. Of course, we virtualized it just as soon as we could in a step similar to my “containerize” advice above. Now is a good time to consider writing your apps with an eye toward portability, so you can remove this type of system from the equation. That RHEL 5, or Windows XP app should really be modernized or eliminated at this point. The tools for cross-platform development are out there, from Mono to Java to Xojo; platform-dependent bits can be isolated for rapid retargeting. It’s up-front work that pays off in spades when the time comes to change the targeted platform.

You. Rock.

Whatever your language of choice, in the end, it will likely be your build tool that links it to DevOps. Your biggest concern for DevOps will probably be portability in the near future. Meanwhile, keep rocking it. You’re doing the work that needs doing, even if it isn’t the part of DevOps everyone is currently buzzing about.

March 12, 2021
Container Security

March 12, 2021
ServiceNow Adds Low-Code Tool to Platform

ServiceNow today made available an update, dubbed Quebec, to its Now platform that adds a set of low-code application development tools.

Marcus Torres, general manager for IntegrationHub and vice president of platform product management at ServiceNow, said the Creator Workflows tool leverages existing App Engine and IntegrationHub capabilities in the platform to make it possible for both end users and professional developers to build custom applications faster using a visual development tool dubbed App Engine Studio. A set of templates that provide a set of pre-built workflows is also included.

The goal is to make it easier to develop applications that would otherwise not be built because professional developers, using existing tools, would simply not have the time, Torres said.

The Quebec release of the software-as-a-service (SaaS) platform is also used for IT operations management (ITOM), adding artificial intelligence (AI) capabilities in the form of ITOM Predictive AIOps that surfaces potential issues before they become a major issue. It is now also easier to set up the virtual agents that ServiceNow makes available to enable conversational AI, and there is now an AI search capability built into the ServiceNow portal. ServiceNow gained these capabilities via its recent acquisitions of Loom Systems and Attivio.

Finally, ServiceNow has added process optimization and workforce optimization modules, along with an engagement messenger module that extends the self-service capabilities it provides to third-party portals, and a universal request tool that enables agents to collaborate and transfer tickets across departments while keeping employees informed of the status of their ticket.

Thanks to the general enthusiasm for digital business transformation initiatives being accelerated in the wake of the COVID-19 pandemic, reliance on low-code tools to build applications has significantly increased. The challenge organizations encounter, however, is that a low-code tool can build a relatively simple application. However, development means may need to drop down to a lower level of abstraction to address more complex issues as an application continues to evolve. The Creator Workflows tool from ServiceNow is designed to provide development teams with that option.

In general, ServiceNow is making a case for building applications on the same platform that many IT organizations already employ to manage IT. Many organizations already take advantage of workflows that ServiceNow has built to extend its core platform to address a wide range of uses cases involving, for example, human resources applications. The addition of the low-code tool will make it easier for IT teams to create applications that address use cases unique to their organization.

Of course, there is no shortage of options when it comes to low-code and no-code tools these days. IT teams will need to decide to what degree platform-specific low-code tools make sense versus adopting low-code tools that enable them to build applications that can be deployed anywhere. Regardless of the path forward, however, it’s clear the number of applications that need to be deployed is dramatically increasing with each passing day.

March 11, 2021