devopsy.kinsta.cloud

Author: Gary Stevens

Best of 2021 – From Agile to DevOps to DevSecOps: The Next Evolution
As we close out 2021, we at devopsy.kinsta.cloud wanted to highlight the most popular articles of the year. Following is the ninth in our series of the Best of 2021.

As the workforce shifts to remote locations, networks are becoming more diffuse as the edge gets farther away. At the same time, the number of devices and applications on those networks continues to rise. Some estimates suggest that, by 2030, there will be 15 connected devices for every person on the planet.

The rapid expansion of the remote workforce and their associated application needs has created a significant challenge for both application developers and the IT professionals tasked with deployment and maintenance of applications. Finding the right balance between speed of release and stability and reliability of applications – never an easy task – is paramount for the success of any organization.

Agile and DevOps evolved to help organizations strike this balance, providing the best possible customer experience in the shortest realistic time frame. But have Agile and DevOps fulfilled that promise?

The Promise of the Agile Manifesto

While Agile as a methodology has been known and followed for nearly two decades, the ways in which organizations have applied Agile have been as varied as the organizations themselves.

The Agile manifesto set out four basic values when it was first written in 2001:
- Individuals and interactions take precedence over processes and tools.
- Working software takes precedence over comprehensive documentation.
- Customer collaboration takes precedence over contract negotiation.
- Response to change takes precedence over mindlessly following plans.
Agile processes are applied across a range of development activities, from development of standalone applications to development of cloud-native business applications. The Agile Manifesto also contains several more principles that further explain what the values mean when put in practice.

These Agile principles can be condensed into a few areas of focus:

Prioritize the customer over the product

Agile development requires continuous customer engagement throughout the entire development process, with the end goal being frequent product delivery based on specific customer feedback. While this can present challenges for an Agile team, particularly when customers demand changes late in the development cycle, Agile maintains that the best way to elevate customer experience is to make the customer part of the development process.

Trust Agile Teams to Self-Organize and Self-Manage

Agile focuses on smaller teams that remain in constant communication with customers, as well as with each other. These teams should include both developers and those with knowledge of the larger business requirements being met through development. For these teams to work effectively, trust is key. Agile posits that self-organized teams, given leeway to pursue their projects in the way they see fit, result in the shortest development cycles.

Keep it Simple

Agile teams require simplicity. This correlates with Agile’s disdain for overuse of process. Agile seeks to minimize the total amount of work required, thus minimizing overall development time.

Agility Leads to Agility

Just as Agile teams continuously improve their product and the customer experience, they also constantly improve themselves. Constant evolution of teams and the processes they employ (remember, teams before processes) improves team agility itself.

While the Agile values and principles seem simple, effective application of the values and principles is another story entirely. Indeed, at times, the implementation of Agile development has violated the very first value of the manifesto by focusing on which tools and processes should be used. But, in an environment where more users are clamoring for more products and services, implementation of Agile methodologies is not just important, it is imperative.

Building on Agile with DevOps

The precise meaning of the term itself and the scope of DevOps remain a source of debate. At a high level, DevOps is the marriage of the development cycle with post-development activities, such as deployment and maintenance.

According to open source contributor Barbara Ericson of Cloud Defense, “Instead of IT operations and software development being siloed off from each other, DevOps breaks down the traditional boundaries that previously existed between them in order to achieve continuous integration and continuous delivery (CI/CD) of quality software features and applications to end users.”

Just as with Agile, the effective implementation of DevOps requires cultivating a DevOps culture within the organization. However, Agile and DevOps are also not mutually exclusive. That is to say, DevOps is not merely a replacement for or an evolution of Agile. Although Agile and DevOps differ in their structures and approaches, they can work in unison to achieve the ultimate goals of speedy deployment and enhanced customer experience.

While Agile and DevOps share common goals, they have not always agreed on how to achieve those goals. DevOps differs in many respects from Agile, but, at its best, DevOps applies Agile methodologies, along with lean manufacturing principles, to speed up software deployment.

One area of particular tension between Agile and DevOps is that the latter relies heavily on tools; in particular, when it comes to the automation of testing and deployment processes. But DevOps can overcome the resistance of Agile developers to tool usage simply by applying Agile principles themselves.

Effectively, DevOps proponents must convince Agile teams that dogmatic adherence to the underlying principles of Agile is actually inconsistent with Agile in the first place. Ironically, Agile developers who insist that the process is always bad actually violate Agile principles by refusing to acknowledge the benefits offered through change, another Agile principle.

The challenge is to have the Agile development teams trust in the automation efforts of DevOps, while at the same time encouraging the DevOps team to consider the business goals of deployment rather than pursuing speed of deployment for its own sake.

With constant communication between the Agile team and DevOps team (another Agile principle), development can achieve a degree of comfort with DevOps tasks and processes. This means that testing and deployment automation can proceed quickly, resulting in a decreased time to market, overall.

Ultimately, Agile and DevOps can work together most effectively when they recall their shared goals: smaller change increments based on feedback resulting in faster delivery cycles.

Continued Evolution: DevSecOps

As networks expand and businesses continue to integrate cloud-based services (in many cases, building multi-cloud environments), security issues become more and more complex. One area in which Agile is lacking is in integrating security into the development process. Unfortunately, in an Agile environment, application security is often an issue addressed after development, rather than as part of the process. Enter the next iteration of DevOps: DevSecOps.

DevSecOps revolves around three basic principles: ensuring data security while minimizing inconvenience for users in accessing data, using development tools that identify risks as early as possible in the development process and ensuring data encryption using technologies such as secure sockets layer (SSL) and virtual private networks (VPNs).

While DevSecOps and Agile may also disagree about the priority of tools, security must become an integral part of any development project. This is why Agile, DevOps and DevSecOps must all work together to ensure not only rapid deployment but secure deployment.

The Next Evolution

Agile and DevOps were both important evolutions of the software development process, but DevSecOps is the next evolutionary step. Despite a few disparate characteristics, Agile, DevOps and DevSecOps can and should work in unison to achieve their shared goal: the best possible customer experience and short deployment cycles.
December 24, 2021
Improving Mainframe Agility With DevOps

Mainframe workloads were on the rise even before the coronavirus pandemic, and transaction-processing mainframes became vital as lockdowns went into effect and every aspect of business and personal life occurred online. Mainframes are reliable, secure, highly available and affordable, and it is essential for organizations to ensure their mainframe can achieve peak agility to enhance profitability and improve customer experience.

There are four areas of focus for DevOps teams who want to improve mainframe agility: teamwork, cloud services, KPIs and automation. Let’s look more closely at each one.

Encourage Teamwork Between DevOps and Mainframe Ops Teams to Improve Agility

It’s vital that mainframe ops teams work seamlessly with DevOps teams to ensure adoption of best practices in both areas, where practical, and that those best practices are adhered to. No matter how skilled your DevOps team, they must always be open to learning from mainframe professionals, operational and cybersecurity experts.

To truly empower everyone on your team to achieve maximum mainframe agility, mainframe code must translate within the larger, multi-platform DevOps toolchain. If DevOps and mainframe code are not in sync, mainframe code could slow down software development lifecycles. Important processes, such as diagnostics, test execution and source code need to be integrated to enable DevOps teams to achieve better productivity.

Enterprise DevOps is shifting focus away from software development exclusively to encompass surrounding processes within organizations. Successful DevOps teams must be able to work with cybersecurity experts and mainframe ops teams throughout the coding life cycle to ensure that everyone is on the same page and applications are functioning optimally.

Decide Whether to Move Entirely to the Mainframe or Integrate Cloud Services with On-Premises Mainframes

Scaling your organization for growth by migrating entirely to the cloud is not necessarily the right choice for everyone. In some cases, moving away from the mainframe’s core systems can be costly and unnecessary.

Cloud-based services don’t have to be eliminated entirely, either. The cloud can and should be used for business processes that could benefit from real-time data updates, easy team collaboration and seamless integration with other software and tools. Mainframes excel at functions that require reliable, secure transactions.

Whatever decision your organization makes, it must be communicated and agreed to at all levels. A customer-centric approach is ideal here, where the needs of the business are considered, but a seamless customer experience is prioritized.

Establish Measurable KPIs

Achievable and measurable key performance indicators (KPIs) are essential to monitoring your progress as an organization. KPIs for mainframe functionality usually include an availability rate of 99.999%, which roughly translates to downtime of less than 5.6 seconds per year. KPIs also generally measure time windows for batch completions. Software delivery velocity, efficiency and quality should be evaluated, but make sure you’re not prioritizing one of these metrics at the expense of another.

There are more nuanced goals that need to be prioritized, too. Continuous improvements in software delivery are harder to quantify, but no less important. By fostering a collaborative environment where communication is key and teamwork is valued, your organization can create a better product or deliver better services for developers and customers.

Use Automation Whenever Possible for Mainframe Agility

Automated testing and AI-enabled processes can solve problems in real-time and free your DevOps teams to focus on larger issues. Mainframe ops teams can save their organizations money by relying on automated processes, which can also reduce personnel costs.

Despite the benefits of automation, mainframe code is often removed from automated processes commonly applied to DevOps toolchains. Javascript, for instance, is one of the most commonly used programming languages due to its platform independence. It’s an essential tool for DevOps developers to enable the smooth configuration and installation of applications.

But as infrastructures grow more complicated and surface areas expand to accommodate more users, operations teams can feel overwhelmed. With the continuous pressure to create new and better software on accelerated timelines, DevOps teams can unwittingly introduce vulnerabilities in Javascript coding as they speed through the software development and delivery pipeline.

Automated mainframe diagnostics and optimization tools are vital to improving both the customer and developer experience. Comprehensive and accurate data presented to mainframe ops teams in real-time can detect vulnerabilities and to find and fix potential issues.

Ultimately, the mainframe is here to stay, with 93% of software experts citing the strength, reliability and scalability of the platform. Mainframes form the backbone of most monetized applications and websites, so it’s no surprise that mainframe agility has become a top focus for many IT departments. No matter how sleek and flashy your application is, if customers have a bad experience at the transactional level of the process, it could mean lost sales.

January 8, 2021
Antipatterns That Hurt DevOps Implementations
Avoiding these DevOps antipatterns can help organizations reap the benefits of implementations faster

It’s a given that all organizations at some point will suffer from poor development practices when trying to find the perfect balance between software development and IT operations. The good news, however, is that implementing DevOps can help you improve these processes and get them synchronized and working together in harmony.

However, understanding and mapping out the core issues and the common DevOps “antipatterns” that are holding you back from successful DevOps implementation is essential. This article will focus on the most common DevOps antipatterns that organizations tend to fall into and how to avoid or overcome them.

What Are DevOps Antipatterns?

At first glance, DevOps may sound like a pretty straightforward concept. At its core, DevOps is about bringing together separate but functional areas and bridges the gap between software development (Dev) and IT operations (Ops) with the ultimate goal of delivering software faster and with better stability.

That said, DevOps antipatterns can best be defined as ideas that are counterproductive to the DevOps culture. They include:
- Blame culture: In this kind of culture, blaming and punishing people is the norm when mistakes are done. It can happen at an individual or organizational level.
- Root cause analysis (RCA): RCA is a process that involves identifying the “root” cause of a problem or an event, as well as the appropriate actions to take to avoid a recurrence. It’s an iterative and continuous process that must be performed until data is exhausted or all organizational factors have been identified.
- Human error: Human error is often cited as a root cause in RCA. This is also common in “blame culture” and someone has to be held accountable for mishaps and mistakes.
- Silos: An organizational or departmental silo defines the mentality of teams that don’t share their knowledge with other teams within the same company.
4 DevOps Antipatterns and How to Break Them

Wrong Perception of Your DevOps Team

The perception you have of your DevOps team matters. It’s essential to consider the team as part of the entire process. Neglecting to provide them with structure, resources, and a positive working environment can increase your chances of DevOps failing. Furthermore, making DevOps teams more productive requires proper implementation of processes and tools.

How to Break It

A successful DevOps implementation requires more than just bringing people together. Structuring the team correctly and adding processes is vital if they are to succeed. Providing the necessary resources, technology and information security they need will get the best results.

Practices such as continuous integration (CI) and continuous delivery (CD) are both concerted processes that can be used to bring development teams together in alignment so software products are delivered quickly, operations are more simple to manage and no downtime is present during deployments that are detectable to the end user.

Poor Choice of Technology

Technology plays a significant role in IT; however, the rapid development of new technologies has made it challenging to separate essential and overhyped tech trends. For instance, DevOps itself is a powerful digital transformation tool as it plays a critical role in new software development and problem-solving. Nevertheless, many of the latest IT trends should not be worth your attention unless you and your team are willing to implement it correctly.

How to Break It

This antipattern can be broken by top-level and project managers who make the decisions regarding technology choices. Asking questions such as, “What does the organization need?” or, “How well will this technology work for us?” will help when choosing what tech is right for your organization. With thousands of tools and technologies available, you need such a mindset to make the right decisions.

Unclear Responsibilities in Teams

You may have come across this problem in many other non-IT projects. However, in both IT and DevOps teams, the consequences can be severe if solutions aren’t implemented quickly. If team members aren’t clear on their roles and responsibilities, this can lead to poor decision-making and unnecessary interference in other people’s positions.

How to Break It

First of all, you need to clearly define roles and responsibilities for everyone in the team. A clear definition of roles boosts productivity. For example, having a data scientist in your DevOps team can play a huge role in business success due to their very specific data analytics skills, techniques and tools. In fact, 89% of business managers agree that hiring a data professional will help to improve success at their respective organizations. There just needs to be active and consistent communication between team members for everyone to conduct their own roles and responsibilities efficiently.

Every team member should follow the strategies and plans in place so timelines can be monitored and effective top management decisions can be made. When implementing complex DevOps architectures that require teams and software to work together, poor decision-making will be an issue if roles are left unassigned.

Wrong or No Code Deployment

To put it bluntly, there’s no DevOps without code deployment. With this being the ultimate result, DevOps makes no sense if it fails. This is quite a challenge for several organizations, as having Dev and Ops teams working in harmony can seem impossible.

How to Break It

As mentioned above, code deployment is often a challenging task for DevOps teams. However, organizations can break the resultant antipattern by integrating the units correctly to enable a collaborative environment, a great strategy for successful DevOps implementation.

Such an environment will also make code deployment easier by reducing the need for a series of meetings and reviews, which ultimately delay teams from getting code to deployment.

NoSQL’s Key Role in Database DevOps

DevOps processes need to work in a synchronized manner to be successful. Issues with the database can cause it to slow down and affect the delivery of database changes. However, NoSQL’s unique approach is designed to simplify the process of DevOps implementation in the following ways:

Faster scalability

Many small companies fail to maintain peak performance with greater workloads. A more significant workload calls for better tools that can help maintain peak performance, subsequently making it easier for organizations to scale out.

More efficient technical support

Having technical support that doesn’t hold you back is key to DevOps success. The right NoSQL solution enables you to solve problems before they become full-fledged, expensive and time-consuming.

Faster adaptability

The changes that new codes bring for databases can be challenging. NoSQL makes it easier to deliver solutions to client requests, as it organizes them using dynamic indexing. Therefore, it verifies all queries before answering them with the appropriate index.

Platform testing

Testing is one of the critical steps you need to go through before actual implementation. Databases should be flexible enough to allow you to test queries without interfering with the real data. Testing helps to determine how new changes will impact the development process.

Conclusion

As you can see, there are many ways to implement DevOps systems in your business to deliver products faster, offer more technical support and automate essential tasks.

All of the requirements we have covered are necessary if a business is to scale and grow. Businesses need to ensure the correct strategies are in place, roles in the company have been clearly defined and common antipatterns have been identified and eliminated.
December 1, 2020
DevOps’ Role in Fixing Software Vulnerabilities
From the beginning, application development has required that software developers deal with bugs, vulnerabilities and other issues. But problems encountered under the DevOps model tend to be more manageable since the updates and fixes required are usually smaller. The reason for this is that the process of maintaining and improving DevOps-related software never stops. In other words, problems that do need to be corrected are typically tackled faster under the DevOps model.

In looking at the DevOps cultural shift that has fast-tracked the software development process while making security part of that process, it’s critical to consider how the DevOps process impacts software development, the issues around hackers taking advantage of vulnerabilities in code and why companies need to protect their infrastructure that incorporates speed and security.

How DevOps Practices Impact Software Development

DevOps practices impact every level of the software development process. In the past, software developers would churn out new app versions, sometimes every few years and sometimes every couple of months.

This afforded developers sufficient time to go over their code with a proverbial fine-tooth comb. It also allowed them to devote more time to quality assurance efforts, which helped to minimize vulnerabilities that hackers could use to their own advantage.

But the rise of DevOps has drastically revolutionized the way software is developed and delivered to customers. Instead of releasing new versions of apps perhaps yearly, software developers operating under the DevOps model can release updates and fixes quickly and on an ongoing basis.

Facilitating the software development process over the course of the DevOps life cycle are tools such as Jenkins or Docker that allow the automation of many procedures and applicable processes.

Quite simply, DevOps practices positively impact software development. The benefits can be broken down into three categories, namely technical, cultural and business, and as can be seen below:
1. Technical Benefits: Ongoing software development, less complexity to take on and faster resolution of problems or issues.
2. Cultural Benefits: Teams more fulfilled and productive, greater worker engagement and more professional development options.
3. Business Benefits: Speedier delivery of features, more durable operating environments and better collaboration.
It goes without saying that DevOps has a major and even revolutionary impact on the process that software development teams use to create applications as well as to fix vulnerabilities. It proves that doing things faster doesn’t have to mean proceeding in a haphazard manner.

Issues Around Hackers Exploiting Code Loopholes

Implementing the right processes and procedures can cut down on exploitable vulnerabilities. But software developers understand that hackers wake up each morning scheming up new ways to exploit code and infrastructure, which is why the struggle never stops. One of the issues customers are justifiably concerned about is privacy.

The answer is to take DevOps to the next level. In other words, software developers need to adopt the DevSecOps model, which refers to implementing open source security procedures in the beginning stages of the software development life cycle. Open source security measures are essential for DevOps security teams because of the fact that they are lower cost, more innovative and allow teams to make the most of their continuous delivery processes. This includes, but is not limited to, code analysis, vulnerabilities assessments and compliance monitoring.

When software teams purposefully and intentionally implement security practices within an Agile framework, they will be able to safeguard the integrity of their codebase. This can be accomplished with continuous evaluation, testing and updating.

Why Businesses Need to Maintain Fast and Secure Infrastructures

More companies are investing in protecting their infrastructure than they ever have before, as infrastructure protection spending this year could reach $17.5 billion worldwide, up from $12.6 billion in 2017. So while there is crystal clear evidence that more companies recognize the need to protect their infrastructure from nefarious actors, they are also, in fact, doing something about it.

The consequences of failing to maintain a fast and secure infrastructure can be disastrous. Consider the following statistics that show some of the problems that can materialize: On average, it takes 206 days just to identify a data breach, and over 30% of companies encountered cyberattacks on operational technology infrastructure last year alone.

So what can software development teams do to help organizations maintain secure infrastructures at a time where hackers are tirelessly persistent? While customers expect software to be delivered post-haste, they also expect it to be secure.

Software developers need to commit to DevOps and DevSecOps to ensure that the software they develop and deliver to customers is free of exploitable vulnerabilities that could jeopardize their infrastructure, scare off their clients and even potentially wreck their operations altogether.

Customers also have a role to play to keep their infrastructure safe. They should, for instance, have policies in place to govern how their workers use the software, roll out updates and fixes as soon as they are available, and use the right tools to protect their infrastructures.

Conclusion

By leveraging the DevOps model to make security part of the corporate culture and by training team members so that everyone adopts coding habits that are rooted in security, and by automating security in the development process, software development teams can deliver safe products quickly.

At a time when customers expect deliverables fast, software development teams need DevOps so they can go from the development stage to the delivery stage without needless delay. The DevSecOps model is one in which security is purposefully implemented at each stage of the software development process, not simply a value-added component.
October 28, 2020
DataOps and DevOps: A Match Made in Heaven
The adoption of IT technologies has increased exponentially over the last few decades. Nowadays, the need to create quality web applications while simultaneously managing their extensive data effectively is a major concern for any organization. As such, there have been continuous efforts to come up with better approaches to software development, data analysis and data management.

In order to meet the needs of today’s internet users, technology companies have intensified their efforts and are increasingly making use of all their available resources to find progressively efficient ways to create web applications within a shorter time-space and with superior quality, and to find more effective ways to integrate data-driven decision-making.

In order to accomplish these feats when it comes to application development as well as data pipelining management, most of the larger organizations are now following a set of practices, DevOps and DataOps, respectively. Within this environment, developers must create applications that can function across various hosted providers, each with its own specific administrators and requirements.

This can only be accomplished by looking past the traditional storage capabilities. Instead, developers and organizations they work for are creating an underlying data fabric that facilitates what they’re trying to do, and any cloud provider they want to do it with.

An Introduction to DevOps and DataOps

DevOps and DataOps are a set of practices that relates to software development and data management, respectively.

These practices can be applied to a single project or can be followed at an organizational level. DevOps emphasizes the automation of the processes of the different teams (software development team, quality assurance team and IT operational team) so that they can easily complete their relative tasks (i.e., design, code, create, check, deliver, deploy, operate and monitor).

DataOps, on the other hand, can be described as DevOps that includes data analysis, which empowers businesses to create processes to meet the needs of users throughout any data usage life cycle.

DevOps emphasizes the cooperation of the software development team (including testers) with system administrators (operational teams). In a similar way, DataOps relies on the cooperation of teams of data engineers, data scientists and other data practitioners who manage data-related processes (i.e., data storage, data processing, data decision-making, data archiving or data delivery).

The concept underscores both communication and collaboration between the different teams within the same organization, ultimately removing all obstacles for seamless integration between the practices and the end product, including cross-device compatibility.

The combined team can make the project more visible and deliver faster results. It also increases the morale of the company by showing that, when it comes to project management, people will still be above processes and resources.

Benefits of DevOps

The benefits of DevOps include:
- Transparency — By regular communication and coordination with the development team, the QA team and the IT operations team will be updated with the latest project details and will be more aware of the processes of other departments.
- Quicker and high-quality deployment cycles — On any great DevOps platform, the pace of deployment will be improved by applying DevOps practices. The scope of the teams will be limited (i.e., for a small task/module) for a deployment period, as such their efficiencies will increase.
- Faster bug fixes — Because tasks are smaller and continually distributed on the appropriate servers, the development team can receive input very easily from the QA team, allowing the development team to provide much faster resolution of any non-working components. Application stability would be a by-product of this method, because the cause behind any bug can be established very easily.
- Flexibility — Because agile methodology insists on managing the raised change request, teams aligned with DevOps will be able to manage any unplanned work more effectively.
Benefits of DataOps

Now, let’s take a look at the benefits of DataOps:
- Faster data processing — When different teams cooperate and interact on a daily basis and work on micro databases rather than on all at once, they not only avoid data overwhelm, they also accomplish much faster processing of data at all the different stages.
- Quicker problem solution — Similarly to DevOps, businesses can benefit from DataOps in multiple ways. DataOps data is processed in batches as this helps teams to recognize issues related to a batch of data faster as the emphasis is limited.
- Better data security — We can protect our data in the cloud by using a range of safety features such as two-factor authentication and virtual private networks, the best of which provide a greater level of security than is available merely through your ISP. As data security professionals are in constant communication with data integrators and data analysts in a DataOps scenario, data security at each process level is made easier.
- Improved resource output — When the team conforms to the basic principles of DataOps, seamless communication and transparency between the different teams will help the performance of each team member and lead to better resource output.
The Lessons We Can Learn from eharmony

The eharmony membership pool consists of singles in the U.S. as well as more than 200 countries around the world, making up about 5 million visitors per month.

The company is continuously bringing out new features such as their most recent one-on-one video communication feature aimed at helping their members to maintain meaningful connections in the current era of social distancing. But the company has come a long way since its debut in 2000, learning very valuable lessons along the way.

Overcoming Several Challenges

eharmony is leading the online dating industry through innovation by continuously improving its two key applications — Singles and Matching. But they’ve had to make significant changes on their journey to remain at the top. Their challenges included:
- The IT team needed a considerable amount of time to refresh the database, requiring the involvement of DBAs, storage specialists and network administrators.
- The labor-intensive process of creating, scanning and storing database snapshots made it difficult to create test datasets for new releases. As a result, QA often carried out tests using stale data.
- Database improvements triggered development freezes, which hindered the team’s ability to deliver new features and functionality.
- Their data volumes were increasing constantly, with several Oracle databases requiring between 20 to 70 TB of storage.
Combining DevOps and DataOps

eharmony has also implemented a self-service data management solution called Delphix, which enables the individual teams to develop and update environments as needed.

Today, their refreshes can be completed within an hour, providing accurate and relevant information to development teams on demand.

Other solutions included:
- They have saved over five thousand person-hours in just three years by automating database provisioning, refresh and rollback tasks. In the process, this has also saved the company over $3 million in operational costs.
- Through compression and virtualization, eharmony has reduced their database storage footprint to a fraction of the original size, reducing their storage requirements by almost 80%.
- As development teams adopt agile methods and more frequent releases, demand for non-production environments is expected to increase, further increasing storage cost savings. eharmony has also eliminated data related defects by providing accurate, up-to-date test data. Unrestricted access to test environments significantly improves the ability of QA teams to identify issues before they reach production.
Why DataOps and DevOps Are a Match Made in Heaven

It is easy to see why DataOps and DevOps go together so well. DevOps calls for continuous software monitoring, testing and deployment. DataOps are inherently modular because they are designed to execute a single function. Software that is scalable and easily fits into the DevOps framework.

Using DataOps in the DevOps process will help to fully recognize the benefits of continuous integration by validating the implementation promised by DevOps. These tools can be combined to optimize computing in a way that should be used during development whenever possible.
May 26, 2020
The Increased Role of Colocation Providers in Data Centers

You would be hard-pressed to find something more complex than a modern data center. A state-of-the-art building stocked with all the necessary software for a major company might run into billions in costs if you decide to build and operate your own from the ground up.

Even though many pundits had predicted data centers would become a thing of the past due to the ever increasing adoption of SaaS, as we previously predicted, data centers are still popular with companies and evolving for 2020. One phase of that evolution is for businesses to box up all of the gear in their data center and install it with what is called a colocation provider, or “Colo” if you’re looking for a snappy acronym.

“Going Colo” refers to renting space in a building, usually a rather large building, and moving all your physical servers and network hardware there. You’ll have neighbors who are doing the same thing because this is just a fancified way to say you’ll have a landlord.

The Colo market is getting big in a hurry. The global market was $31.5 billion for this type of data center arrangement in 2017, but is expected to expand to at least $62.3 billion by 2022. What a company, large or small, has to ask itself is if this type of arrangement is a good thing for DevOps. Let’s ponder that.

It’s Cheaper When You Don’t Own the Building

If you remove the possibility that you could be thrown out of the space at a moment’s notice (a worst case scenario that a good rental agreement should address), the idea is a no-brainer from a cost savings perspective.

For an enterprise, let someone else take on the risk of construction and infrastructure maintenance. For a small-to-medium-sized business, the cost of building and monitoring a modern data center makes it impossible.

The unfortunate reality is that, when you do own the building, someone has to take care of it. In a perfect world, every company would have a dedicated IT staff; if you’re Microsoft, Apple, Google or Facebook, that certainly is the case.

For much of the tech world, the choice to maintain a data center onsite often results in personnel being frequently dragged out of their role in DevOps and asked them to do something related to the data center, often a task they are not qualified for. This creates holes and inefficiency in the development process. That’s a bad thing.

Good Infrastructure Means Better Uptime

Guess what’s also bad for DevOps? The team’s least favorite scenario—besides an unexpected meteor strike—would be system crashes and downtime related to hardware issues. Downtime can be incredibly expensive for a multitude of reasons besides the obvious—employees sitting around twiddling their thumbs. There is also the scenario of actual lost income. As Amazon found out the hard way, the number can be astronomical.

With no internet connection, business payment systems, no matter how sophisticated, are likely to be non-functional, though some will queue payments and run them through when the system goes back online. This is why it’s a good idea for businesses to find ways to speed up transactions from clients and customers, such as automating invoices and customizable invoicing tools that allow multiple options for straightforward payments.

This is also another area where going Colo can be a huge help to development. Think about it. In return for scratching a monthly rent check, you get diversified and redundant power sources, the most reliable internet on the planet and enough bandwidth to strangle an elephant.

The bottom line is that moving equipment to a Colo space should be accompanied by a significant reduction in downtime. If it’s not, you need a different landlord. Downtime is the enemy of product/service development progress. A modern Colo arrangement means you don’t have to spend time running down to the local Dollar General in search of an extension cord when one flames out. Okay, you really shouldn’t be using extension cords from a dollar store in your data center, but you get the idea.

Secured Data and Premises

The only type of downtime worse than that caused by insufficient infrastructure is when data is breached either through hacker online trickery or physical penetration of the premises. The latter happens more than you might think. Bad guys decide to go old school and break into a Colo center in the middle of the night to physically drag servers out of the rack and spirit them away.

With the intense global focus on thwarting hackers, DevOps is already in the process of evolving into DevSecOps as we speak. The reason is simple. Development teams are realizing that a focus on security from the start reduces the chance that a project gets interrupted mid-development resulting in—you guessed it—downtime.

With a Colo provider, let them worry about security.

In a Word, Scalability

With the everyday lexicon expanding to include cloud storage possibilities, the idea of scalability is at least somewhat familiar to the masses and certainly to businesses.

Google actually started out in a colocation data center before their data collection grew to approximately the size of the Milky Way galaxy. For IT in general and DevOps in particular, scalability is the Holy Grail because no company expects to stay the same size forever.

A smart DevOps team, especially one that works in any of the growing number of SaaS business models, will put the bug in management’s ear about the benefits of Colo long before the need arises.

The worst time for an upwardly mobile small or medium-sized company to start thinking about making the switch is after you’ve already outgrown present resources. For example, when working with inadequate bandwidth, a peak time data spike can slow development as local computers struggle to find enough bandwidth to function.

Another dirty little secret is that Colo providers typically get better internet rates than a single company, so there’s that. With the cost of a spike spread across all tenants, the actual cost to you will be considerably less than trying to manage your own data.

Final Thoughts

The bottom line is that the service provided by a Colo center should reduce the strain on a company’s IT budget and resources. Common services provided to a tenant are support personnel, who monitor and troubleshoot systems and issues before they grow into a big, fat calamity. Money you don’t spend providing basic service to an onsite data center is money you can funnel toward doing your company’s main work.

There’s one more thing a Colo site provides that shouldn’t be undervalued, and that is peace of mind, because you have removed much of the stress, cost and personnel drain that accompanies running your own data center. Farm it out. Let someone run it who hires staff with the particular skills needed and has a vested interest in keeping you happy.

— Gary Stevens

April 8, 2020
Comparing Three Approaches to Multi-Cloud Security Management

Moving to a multi-cloud environment can bring many advantages, but it also brings huge challenges in managing both the everyday operations and security of multi-cloud infrastructures.

To meet these challenges, several approaches to multi-cloud management have arisen. The most common can be roughly grouped into three categories: Cloud Management Platforms (CMPs), Infrastructure as Code (IaC) and cloud-native abstraction. The differences between these approaches reflect the differences between top-down and bottom-up cloud management, as each management process suits a different set of companies.

In this article, we’ll take a look at all three approaches, explain what the advantages and disadvantages of each are and then explore whether new approaches to multi-cloud management are required.

CMPs

CMPs are the highest level approach to cloud management—the approach that is furthest away from the native code that your clouds run on. The rise of CMPs has come as the result of a simple observation: public cloud services are redundant and overlapping, and can therefore be managed through an abstraction layer that hides the complexities of what is happening at a cloud-native level.

By using a CMP, admins do not need to understand the differences between AWS and Microsoft public clouds (for instance), but instead use a consistent interface to manage both.

This has some huge advantages, primarily when it comes to improving cloud security. CMPs allow IT operations teams to implement a common security layer within multi-cloud environments, and apply the same identity and access management (IAM) processes across all the clouds they are working with.

This is particularly useful with cross-cutting applications that interfaces with multiple, critical portions of your cloud storage to help simplify, and which requires highly complex user access policies. This is ultimately one of many reasons why security breaches of cloud-based data are so rare, as 99% of businesses in the UK that rely on cloud-based software have never experienced a security breach.

IaC

IaC is a second approach to multi-cloud management. This approach arose in response to utility computing and second-generation web frameworks, which gave rise to widespread scaling problems for small businesses.

Administrators took a pragmatic approach: they modeled their multi-cloud infrastructures with code, and were therefore able to write management tools that operated in a similar way to standard software.

IaC sits in between the other approaches on this list, and represents a compromise solution. It gives more fine-grained control over cloud management and security processes than a CMP, especially when used in conjunction with SaaS security vendors whose software can apply a consistent security layer to a software model of your cloud infrastructure. This is important because SaaS is growing rapidly in popularity, with 86% of organizations expected to have SaaS meeting the vast majority of their software needs within two years.

On the other hand, IaC requires a greater level of knowledge and vigilance than either CMP—or cloud-native approaches. If admins are knowledgeable enough to write code that can accurately (and reliably) model multi-cloud infrastructures, IaC can be a powerful way to improve DevOps security. These software models, however, will require constant monitoring to ensure that changes in the cloud-native layer do not introduce vulnerabilities into management tools.

Finally, IaC can be significantly more efficient, when it comes to computing and network resources, than a CMP. Because IaC software runs closer to the cloud-native layer than (most) CMPs, it can execute common tasks much more quickly.

Cloud-Native Abstraction

The final approach to multi-cloud management is to manage your environment using only cloud-native tools. This approach has grown in popularity recently, as developers grew increasingly comfortable with working with containers and Kubernetes directly. Cloud storage providers have responded by offering increasingly complex tools designed to work directly on the cloud-native layer, and the storage provider you use should come with these tools.

Cloud native application management has several advantages, and several disadvantages. The most obvious advantage is that cloud-native tools give admins a much greater level of control over their multi-cloud environments than the other approaches on this list. Because these tools run natively, they are also typically faster and more efficient than the other approaches we’ve mentioned above.

On the other hand, saving computing time comes at the cost of increasing the workload for administrators. If you are working across a large number of clouds, managing each separately using each cloud’s native applications can quickly turn into a full-time job. In addition, managing multiple and complex access and usage policies at a cloud-native level can make it difficult for admins to keep on top of how their IAM processes are running, and increases the risk of mistakes.

Is It Time For a New Approach?

All of the approaches above are a different solution to solve the same problem, namely the challenges caused by rapid adoption of multi-cloud technologies. Each, as we’ve seen, has a primary advantage when compared to the others. But in truth, none of the approaches above represents a completely adequate solution to managing security in multi-cloud environments.

The choice of which approach to take must therefore be a pragmatic one. For companies who are blessed with the knowledge, talent and time to manage multi-cloud infrastructures at a cloud-native level, this is likely to be the best solution, especially at a time when cloud waste is booming. If you take this approach, however, you should also be aware that implementing it will require putting in place thorough and rigorous code auditing processes in order to ensure that any small IaC elements that run on top of cloud-native tools do not introduce common security vulnerabilities into your network.

For small to medium size businesses, or those running hugely complex multi-cloud systems, CMPs will be a safer bet. The reduced efficiency of these services–both in terms of computing resources and capital investment–is likely to be more than made up for by the reduced load on the admins who spend their days working with them, and the reduction in the number of mistakes that are made.

Ultimately, it’s apparent that a new approach to multi-cloud management is required. One that combines the flexibility and speed of cloud-native with the ease of CMPs. But at the moment, it’s difficult to ascertain what that will look like.

— Gary Stevens

April 1, 2020