devopsy.kinsta.cloud

Tag: devops

SoulCycle Flexes Its Muscles with DevOps and CI/CD
When SoulCycle wanted to improve application development and deployment, including improving services for customers across its 80-plus indoor cycling centers in the United States and Canada, the company turned to Mark Sost, a veteran developer and engineer who worked at several media companies, including Hearst.

Over the last two-plus years, Sost, who now holds the title of senior director of engineering at SoulCycle, has built-up the company’s engineering and DevOps teams, increasing the headcount from eight or nine developers to about 40 employees now.

The first major project was revamping a legacy, LAMP-based stack used to support applications to focus more on a modern platform that could take advantage of DevOps methodologies, as well as continuous integration and continuous delivery practices (CI/CD).

Sost and his team dubbed this new platform Podium. However, Podium is more than a development platform, Sost explained.

“Podium became the ethos and the methodology, as well as the architecture and application stack for re-platforming Soul’s technology stack in total,” Sost said in a telephone interview. “We moved from that monolithic to more of a service-oriented architecture. As such, we started to break out functionality on multiple applications.”

The re-engineering also meant investing in new infrastructure, as well as other cloud-based technologies. This includes a multi-cloud approach.

While most of the legacy applications have now been moved to Amazon Web Services, SoulCycle also uses Google Cloud Platform for Podium and all new developments. Sost also takes advantage of Kubernetes and Docker for container orchestrations, as well as investments in other managed services.

Other tools in the DevOps arsenal include Jenkins to run unit tests; DataDog and StackDriver for monitoring and analytics for workflows; combing Travis CI, Cloud Builder and GCR for CI; and technology from startup Harness, which provides CD-as-a-service. Sost called Harness the “last-mile” tool needed for application deployment.

“For Harness, that’s CD for us,” Sost said. “That’s continuous delivery to our various Podium environments. We’re leveraging it for delivery to all our Podium environments. Everywhere from UAC (user account control) during the code review cycle to QA after merge, to production, load test and everywhere in-between.”

This revamped approach to application development now means that the SoulCycle engineering team has created approximately 30 different apps over the last 24 months, this includes internal apps used within the company environment, and ultimately to customers using the facilities and equipment.

Additionally, Podium helps create a more unified architecture to help ensure that the entire team—whether it’s DevOps, QA or the production team—can paint a 360-degree view of the problem that needs to be solved and what the implications are for the business once the solution is put into place.

This is what Sost and his team call “Podium principles.”

“When we say that we’re building things with Podium principles in mind, we’re talking about the fact that everything is version-controlled,” Sost said. “We have full test coverage. We have everything in terms of Kubernetes manifests to infrastructure automation written down and version control checks it as part of source code. That applications are conforming to more of that 12-factor methodology and that we’re really focused really heavily on developer empowerment.

“I think for us, that really is a shift that Podium has introduced; a move from writing the code as an application developer and then throwing it over a wall to an operations person—a QA person—to test it and deliver it somewhere,” Sost added. “Instead, we provide a developer the tools and the patterns of principles to go about controlling more of their application development life cycle all the way through to delivery.”

The results have helped improve the overall SoulCycle experience for customers, which includes:
- Reducing deployment time by approximately 80 percent.
- Instead of one hour, deployments now take one developer 10 to 15 minutes.
- QA verification has been reduced by approximately 80 percent.
- It now takes three QA engineers one hour to verify, compared to three to six hours previously.
At the same time, Sost believes that the Podium approach gives developers more control over their projects and lets them focus on writing good code that creates apps people want to use.

“It’s more, ‘Now, I know what’s involved with getting my application to people,’” Sost said. “Because at the end of the day, engineers are just concerned with writing things that people want to use. … This is how we’re going to give you more control of the code and applications that you’re writing, so that you can see meaningful results quicker, without having to wait for someone.”

What’s next for Sost and his team at SoulCycle? The answer: Machine learning and a bit of automation to the DevOps process.

Here, SoulCycle is using Harness tools to log more data and infrastructure monitoring into the platform, which allows the algorithm to learn more about the various development projects. This can make the code better and eliminate errors that come with constant development and deployment.

“We’re letting it get smarter over time around how it can detect anomalies and understand whether our code was successful because there are going to be times where things slip through unit testing or slip through integration tests and there are even going to be times where they slip through load tests, where things like 99 percent are just not going to cut it,” Sost said.

— Scott Ferguson
March 1, 2019
How SRE Creates a Blameless Culture

People in tech love to praise failure as essential to innovation. But at far too many organizations, when failure happens, it is punished. When an IT system goes down, an individual or a team is immediately ID’d, blamed and shamed. This is counterproductive. Blame damages companies by creating a play-it-safe atmosphere and stifling innovation; even worse, when mistakes are made, they are often hidden.

Fortunately, more and more organizations are recognizing the damage that blame can do and embracing a blameless culture. This is a big shift, yes, but companies can make the move to blameless—and make it stick—if they bring site reliability engineering (SRE) best practices to their technology teams.

Why Blame is Bad

I’ve worked at several fast-growth companies that suffered under a culture of blame. At one, we were scaling quite rapidly. Change was constantly being introduced to the infrastructure, which meant that it broke all the time.

Because we had customers running their core business on our systems, our reliability requirements were very high—so high that for a while we stopped feature development altogether, in fear that new features would bring our systems down.

This was difficult enough, but the culture of blame made it worse. There was no psychological safety net and fingers were pointed constantly. There was an us-versus-them mentality—engineers versus non-engineers, developers versus operations, developers versus developers. Developers dedicated way too much mindshare to protecting themselves and their domain.

In a blame-rich environment, developers who should be delivering concrete results spend too much of their time in meetings trying to justify their work. There is a culture of retribution for mistakes. This leaves little room to learn and grow from failure because developers get harassed by management when they do make mistakes, or they get fired.

Your best people—those empowered to make decisions—see their responsibility undermined and go elsewhere. Creativity and innovation slow to a crawl. Meanwhile, competitors run ahead.

Why Blameless is Better

In a blameless culture, everyone feels safe and no one is afraid to make mistakes. It’s a psychologically secure environment, where true DevOps can happen. Developers feel confident enough to express their ideas and take chances. Development folks and operations folks collaborate well, and everyone is aligned on the problem.

The work environment is positive and workers are assumed to be doing their best. When mistakes happen, people aren’t blamed. Instead, any error is viewed as a manifestation of an underlying vulnerability in the systems. Attention is focused on fixing those vulnerabilities and, as a result, systems are constantly improving.

How to Get to Blameless

Let’s say you just had an outage and your database crashed. Don’t start by placing blame on a team or individual. Rather, start by conducting a postmortem. Ask questions and find the answers: Why did the database crash? Because Alex pressed the wrong button. Well, why? Because we didn’t have a system in place to enable automatic checking or a review process. So why did we not have that?

Now you’re getting to blameless and you’re reaping the benefits. You’re having a productive conversation and you’re focused on fixing your systems, not affixing blame. It’s not about Alex pressing the wrong button; it’s about how your organization is not set up to ensure success. It’s about this vital opportunity to learn from a mistake and get better. Don’t fire Alex. Put a solution in place so the problem doesn’t happen again.

How SRE Can Help

To get to a truly blameless environment, you need to implement SRE best practices. Why? Because SRE creates ultra-scalable and highly reliable software systems, by building in automation and tooling—moving the human element out of the picture and focusing on the systems.

One of the founding principles of the SRE movement is blamelessness. By implementing an SRE team and establishing SRE practices, your organization can move from the traditional operational model of break-fix to an environment that applies a development approach to IT operations, with the goal of improving reliability via automation, continuous integration and delivery.

For this to happen, you will need a SRE champion to quarterback your SRE effort. You will also need organizational buy-in to blamelessness. It has to be practiced top-down and have the full support of all leaders in the organization.

Tooling is critical, too. Look for tools and dashboards that can quickly do postmortems and find action items that need attention, that can identify the most commonly impacted services, products and customers, and the contributing factors that most often lead to incidents. The right tools will help you resolve incidents faster, by providing a clear overview of metrics related to incident identification and the actions that need to be taken.

You want to move fast and be innovative. But can you do it reliably? With a blameless culture, you can. You’ll have a safe environment that attracts and retains the best talent. You’ll move quicker, because you’ll be focused on fixing systems that break and finding out why—not the blame game. By delivering a higher degree of reliability, you’ll build greater trust with your customers.

You’ve heard the saying: Failure is not an option. In a blameless environment, failure is always an option—because it means that systems are always improving and innovation is always happening.

— Ashar Rizqi

February 27, 2019
FinOps Foundation to Help Rein in Cloud Costs

A new consortium has been formed to promote best practices for controlling cloud computing costs.

Announced at the Cloud Economic Summit, the non-profit FinOps Foundation counts among its founding members Atlassian, Autodesk, Cloudability, HERE Technologies, Just Eat and Nationwide.

Cloudability co-founder J.R. Storment said the goal is to bring financial accountability to variable cloud spending that can easily spiral out of control if not monitored carefully. The adoption of best DevOps practices makes IT organizations more agile, but not every organization has the financial controls in place to ensure spending on cloud services is aligned with business outcomes, said Storment.

These days engineers can use their corporate credit card to spin up virtual machines in the cloud. The FinOps Foundation is making a case for more accountability over the spending.

Gartner estimates worldwide cloud spending will reach $360 billion by 2022, which is five times the compound annual growth rate for overall IT spending. The unfortunate truth, however, is that most finance organizations don’t have a lot of visibility into how those dollars are being allocated.

To coincide with the launch of FinOps, 451 Research also published a survey of 300 IT and finance executives in the United States and the U.K. that finds more than half (57 percent) of large enterprises worry about cloud costs daily, with 80 percent admitting poor cloud financial management has had a negative impact on their business.

In fact, the 451 Research report makes it clear most finance departments have little visibility into cloud spending. Almost three-quarters (72 percent) of finance executives acknowledged they have no formal reporting capacity between departments. More than half (51 percent) of respondents admitted to occasionally overspending and 68 percent said they are only alerted to overspend too late. A full 82 percent of finance respondents rely on cloud vendors, spreadsheets and manual processes to track cloud costs.

Despite much of the variable cost hype surround cloud computing, 73 percent of the finance respondents in the United States and 81 percent in the U.K. still treat cloud as a fixed capital expense rather than an operating expense.

To help finance teams better understand how cloud spending is being allocated, the FinOps Foundation will provide access cloud economic research and related articles, FinOps training and meetup groups. Early access to the forthcoming “Cloud Financial Management Strategies, Creating a Culture of FinOps” book published by O’Reilly Media will also be provided.

As IT organizations increasingly employ multiple cloud platforms to run different classes of workloads, understanding the true cost of choosing to use one cloud over another will become more important. At the same time, IT organizations have a greater appreciation for the need to move workloads between platforms as costs rise. Just because an application workload was created in the cloud doesn’t mean that workload should be deployed there. Many long-running applications are less expensive to deploy in an on-premises IT environment. The challenge IT organizations arguably face going forward will be integrating streams of real-time financial data into DevOps processes to make more economically informed decisions about how to allocate their cloud resources.

— Mike Vizard

February 26, 2019
Top Trends in DevOps for 2019

DevOps is poised for healthy growth over the next several years, with analysts large and small weighing in with forecasts. For instance, IDC predicts the worldwide DevOps software market will reach $8 billion by 2022, up from $3.9 billion in 2017. Similarly, Grand View Research estimates the global DevOps market will hit $12.85 billion by 2025, registering a CAGR of 18.6 percent.

Below are some of the most influential trends driving growth in 2019.

DevOps Instrumentation in AI and Machine Learning will Increase Productivity Gains

Web-based forms are the nemesis of process automation. Everyone dreads manual forms, but that’s what organizations use when they want to capture data from customers, employees or partners.

AI and machine learning technologies are mature enough to reliably process voice, video, text and images. That means data-enriched forms could be filled out simply by making a phone call, taking a video, taking a picture or some other natural activity. The analog inputs from the external world are captured digitally and automatically.

Of course, the error rates in such conversions would depend on how well the AI models have been trained and how well we use the real-world input to enhance the model. DevOps-level instrumentation needs to be introduced in the pipeline. That way, either the customer or the person in-loop will be able to take these errors as inputs for enhancing the accuracy of the models.

Key Management is Key for DevOps

In 2019 and beyond, data usage will be closely monitored as GDPR becomes a reality and similar privacy laws are passed across the globe. This is when data tagging becomes critical. Therefore, tools related to data tagging and master data management will become crucial for tracking the origin of data usage.

Sensitive data will need to be encrypted in rest and in motion. Along with data encryption comes the important responsibility of managing encryption keys. Not only should the keys be stored securely, they also may need to be rotated. Key management services that offer such functionality will become commonplace in the DevOps world.

Container Adoption Increases with Data Locality Demands

Data locality is driven by countries that mandate data must reside within their geographical boundaries. Enterprises using SaaS or PaaS will use country-specific public clouds or even private clouds. As a result, critical data and applications will have to be deployed across geographies. Deployments will become more complex due to geographical expansion as well as agile development methodologies.

In response, container-based development, release and deployment models will become commonplace within the enterprise. While containers offer many advantages by creating siloed workloads, they also increase the demand for sophisticated monitoring tools and technologies that help consolidate the view of these applications.

The Compiler is the New King of the Data Center

The demise of Moore’s Law is not helping the CPU keep up with increasing workloads in the data center. To bridge the performance gap, modern hardware such as GPUs, FPGAs and ASICs will infiltrate the data center.

Developers will start using higher-level languages such as OpenCL and CUDA, and the compilers will generate code to run on the various hardware, depending on deployment-level input including workload, hardware configuration and data size. DevOps practices will have to start befriending compilers so that code can be generated just in time, very close to the point of deployment.

Technological advancements, security requirements and legislative changes will impact the professionals within this sector, which is growing at a rapid pace that could see it double within five years. As developers prepare to manage a higher volume of sophisticated data, an increasing amount of DevOps solutions will enter the market to further automate processes, accelerate deployments and address the above trends.

— Shailesh Kumar Davey

February 26, 2019
‘DevOoops’ Moves: Poorly Defined KPIs
DevOps’ ultimate aim is to create efficiencies in the software delivery process. But, it doesn’t always work out that way. Sometimes DevOps initiatives lead to mistakes—something we lovingly call a “DevOoops.” CloudBees recently polled some colleagues and came up with five examples of a DevOoops in action. The following is a discussion submitted by product marketer Juni Mukherjee about poorly defined key performance indicators (KPIs).

Scenario:

How we define KPIs can change the game. For example, one quality engineering team defined the number of tests executed per sprint as a success metric. Humans are driven by incentives and the natural tendency of the team was to add more and more tests without even considering archival of outdated ones. Think of this: We could have a larger impact with fewer tests and, more importantly, fewer tests lower the test cycle time. So, instead of sheer quantity, we should focus on the coverage and effectiveness of tests. Similarly, a release engineering team defined the number of releases per sprint as a success metric. The number of releases reflects on velocity for sure. However, releases move bits from point A to point B without assessing the value added to business. It is critical to tie a business KPI (such as the number of new customers acquired, percentage of revenue increase, etc.) to velocity, such that we know we are investing in valuable speed and not suicidal speed.

Management consultant Peter Drucker boiled his whole philosophy down to the following few words: “If you can’t measure it, you can’t improve it.”

Successful DevOps leaders take this saying to heart. To understand where they’re going, how to get there and ultimately how far they’ve come, these leaders know how critical it is to establish KPIs that fit the overall objectives of their DevOps transformations. KPIs provide appropriate and continuous insight—and they plot a route to help you get to your destination.

The big question is: Are you measuring the right things? If not, you could be spinning their wheels.

To avoid this “DevOoops” scenario, you need to start by establishing KPIs that not only align to your objectives but can also be established so that they create a continuous feedback loop. As the project progresses, as you mature and learn more, you will likely need to maintain, monitor and possibly adjust and update these KPIs.

What would be an appropriate test coverage goal? Teams all want to improve quality. But by how much? Seventy percent? Eighty percent? One hundred percent? This is an example of a KPI that needs to be considered in the context of your application as well as measured against your real objective.

You can have 70 percent test or code coverage, but if that’s just treated as a measure to test as much code as possible, you may miss the mark. If you are covering 70 percent of the wrong code and the 30 percent that’s missing is critical code, you’ve put in significant effort with the risk of little or no return on quality level.

In terms of your goals, increasing and monitoring test coverage may be an indicator of quality, but it may not measure your actual quality of work. In this case you want a KPI that’s measuring the outcome rather than the activity. Look at what is severity 1 and severity 2 defect rate in production. You can see code coverage get to 90 percent, but if you’re not seeing a decrease in severity 2 defects, you won’t really be seeing an improvement in our code coverage metric.

Takeaways:
- You’re not implementing DevOps just to say you’re doing it; you need to develop goals and objectives for your initiative and align to them.
- Once goals are defined, you need to align KPIs and success metrics to those objectives. Properly defining these success metrics and KPIs will dictate what gets prioritized when building pipelines.
— Brian Dawson
February 22, 2019
Smashing the (DevOps) Stack for Fun and Profit

Most organizations that I deal with have a similar story for the promulgation of DevOps in their data center. The story is similar for bringing in most new technology and techniques: A team that usually does a new project without any attachments—relatively small compared to projects at the center of the business—adopted DevOps from the start. A smaller (by far) number shoehorned DevOps into an existing project as a first taste. Some decided to go “all-in” and “convert” IT to DevOps. This is a painful method; I can’t say I recommend it. I suspect my friends at those organizations wouldn’t recommend it, either.

When I say “DevOps” in this sense, I am talking more about process and procedure changes, even cultural changes, rather than tools. My experience is that most companies that have been around longer than the last spring shower started adopting the tools of DevOps before they started looking at ways to streamline the process. And that isn’t a terrible thing at all. It doesn’t matter what marketing spin is used to sell the product; if it makes your life better or allows you to react faster, implementing it is worth consideration. I just wanted to be clear what we’re talking about here.

Those companies that started small generally grew small, also. It is a big deal to move that 20-year-old codebase with a ton of processes that touches everything into DevOps when compared to saying, “New projects will use DevOps.” This, too, is in line with what we’ve always done. Iterative replacement has always been the case, where the large, important systems are concerned. In a couple of industries I am familiar with, I’d even argue that those large, important systems weren’t completely upgraded to Customer Information Control System (CICS), so don’t hold your breath on seeing them completely upgraded to DevOps. But that is a different blog.

Once a critical mass has been hit, these shops start referring to themselves as a “DevOps shop” or “DevOps first shop.” This is the point I want to talk about today (though it took me a bit to get here).

You’ve got a bunch of apps or a couple of application portfolios that are now running under DevOps. In most organizations, the tool sets are not a perfect match. Indeed, in some organizations the DevOps toolchains look like they were assembled on opposite day. Teams tend to be more standardized, but even team composition varies from project to project, as do communications.

This is the next point of inflection in DevOps growth. You have it in your grasp to jump again, from amazing to truly amazing.

Time to smash the DevOps stack.

Get the teams together, build a list of each system used at each step in the process for each app, each manual intervention, each communications channel—essentially each difference between DevOps teams—and start to look for synergy. If one team is using ChatOps, check out what benefits they perceive from doing so and if ChatOps fits all teams. Does one team have a standard process to roll from continuous delivery to continuous deployment? Check it out. If you have several disparate DevOps teams, it is almost guaranteed that one team will handle DevSecOps better than the others. Take a look at what/how they’re doing and promulgate it to less security-intensive projects as needed.

Basically, build the corporate process/toolchain from what is successful. That sounds like simple advice, but it isn’t. Teams often love their tools and processes, and are certain their choices are better than others. It takes work to get to a common understanding and there are always exceptions in a complex environment: “Well, yes that works, but for our app, we have this thing over here that makes it not possible/feasible/affordable.” It requires extra work, but that’s always true. Because the environment is complex and has been for a long time, these issues have always existed; you’re just dealing with them with new processes.

Keep kicking it! Keeping the virtual lights on is as important as keeping the physical lights on in today’s environment. Unless your company is listed in “haveibeenpwned” emails, or your website has been suffering catastrophic outages, you are kicking it.

— Don Macvittie

February 22, 2019
Will ITIL V4 Bring Order to a DevOps World?

It is difficult to dispute the importance that the ITIL (Information Technology Infrastructure Library) approach has had on the world of ITSM (IT service management). After all, the best practices established by ITIL V3 have come to be the accepted way ITSM providers conduct business—so much so that many other IT processes have come to look to the ITIL framework to bring order and control to complex processes.

Yet, ideologies such as DevOps have struggled to adapt to the somewhat rigid doctrine of ITIL V3, instead favoring Agile methodologies to meet the rapid deployment of updates and the iterative cycle of software development. Perhaps, the Agile-versus-ITIL differences can be attributed to how life cycles should be managed and how the management of those life cycles impact both development and operations in the IT space. Nevertheless, change is coming to the world of ITIL, which may help to make the framework more suitable for DevOps practices, and perhaps make ITIL and Agile symbiotic in the field.

The ITIL V4 framework is scheduled to be released in Q1 2019, some two years after it was announced by AXELOS at the Fusion 2017 conference. “ITIL is the most firmly entrenched ITSM framework in the infrastructure of large, complex organizations, with many—if not most—Fortune 500 companies using it,” said Cathy Kirch, president of itSMF USA, the organizer of FUSION 2017 and the leading independent professional forum for IT Service Management professionals in the United States. “Currently, ITIL and complementary practices, such as DevOps, Agile and Lean, are used successfully in conjunction with each other; however the next evolution of ITIL will promote and underpin dynamic collaboration.”

The More Things Change, the More They Stay the Same

ITIL V4 doesn’t propose to completely rewrite the ITIL V3 narrative; in fact, the core elements are expected to remain the same. ITIL V4 will simply include new material on integration with additional best practices aimed at mitigating some of the biggest criticisms around ITIL V3, the less-than-ideal relationship with the ITSM community and failure to keep up with recent trends in software development and IT operations.

In essence, ITIL V4 will address those criticisms by shifting V4 to a community-driven initiative and integrating other practices such as Agile, DevOps and Lean into traditional ITIL best practices. That means ITIL may become complimentary to those other frameworks and begin to embrace other services management best practices in the context of new technologies, such as serverless, containers, microservices and multi-cloud.

With the DevOps community picking up steam, the Agile Manifesto and Lean development displacing ITIL V3 in some practices, it will be interesting to see what impact ITIL V4 will have on the IT industry as a whole. While it is far from ITIL’s last gasp, ITIL V4 will face significant challenges in a DevOps world that is constantly changing. However, with more than a million ITIL V3 certificate holders worldwide, ITIL V4 will definitely garner a great deal of interest when it hits the streets on Feb. 28.

— Frank Ohlhorst

February 20, 2019
IBM Adds Automated Testing Tool for Mainframe to DevOps Portfolio

IBM has added an automated testing tool to the portfolio of DevOps tools it has developed for mainframes.

Sanjay Chandru, director of Z DevOps for IBM, said IBM Z Open Unit Test adds a continuous testing capability to the continuous integration/continuous development (CI/CD) platform based on an instance of an open source Git repository developed by Rocket Software that IBM launched last year. The automated unit testing tool automatically records and plays back test data for both batch-oriented applications written in COBOL or PL/I and CICS transaction processing applications.

Chandru said IBM Z Open Unit Test is unique in that it eliminates the need to employ traditional debugging tools. Developers can generate test cases for COBOL CICS programs, as well as “stub out” CICS to achieving environment independence from CICS runtime during test case execution. A stub is an object that holds predetermined data, which is provided in response to calls made during tests. That approach eliminates the need to have objects that answer with real data. That means Z Open Unit Test operates without adding the overhead of a debugging tool, said Chandru.

Any test case and the resulting program created then can be stored in any source code management (SCM) system.

IBM is making a concerted effort to make mainframe a natural deployment target for any DevOps team, said Chandru. It increasingly has become apparent that millennials in particular aren’t biased toward one platform or another, as long as each platform is equally accessible, he noted, adding that as organizations embrace best DevOps practices to modernize their application portfolio, IBM wants to make sure the mainframe remains relevant.

IBM launched the z14 mainframe in the fourth quarter of 2017. According to the company, 92 of the world’s 100 biggest banks use its mainframes, which handle most global credit card transactions alongside billions of ATM transactions and annual passenger flight bookings. While use of mainframes overall isn’t growing, the platform isn’t going away anytime soon.

IBM has also enhanced the cryptographic capabilities of the z14 to make it one of the most secure platforms available and continues to invest in an instance of Linux for the mainframe. That latter capability makes the mainframe a more natural target for applications being developed and deployed using DevOps processes, said Chandru.

Of course, the biggest challenge when it comes to DevOps and mainframes is not necessarily the technology. Most mainframe applications have been developed using waterfall processes for decades. Getting those teams to embrace DevOps processes represents a significant cultural hurdle for any IT organization.

At the same time, however, DevOps teams who ignore mainframes are not taking advantage of an existing platform investment that has performance and security capabilities that are still unmatched after more than five decades. The real challenge is finding a way for old mainframe dogs to learn some new DevOps tricks while exposing proverbial whippersnappers to the capabilities of the most powerful mainstream platform ever built.

— Mike Vizard

February 19, 2019
IBM Extends Cloud Reach to Include Integration

At the IBM Think 2019 conference this week, IBM unfurled an IBM Cloud Integration Platform that pulls together a range of tools in a single development platform that the company promises will reduce the amount of time it takes to deploy applications by making it easier to reuse integration code.

Hillery Hunter, vice president and CTO for IBM Cloud Infrastructure, said by bringing all of its primary integration tools into a single cloud platform, IBM is effectively making an integration platform-as-a-service (iPaaS) environment available for the first time.

IBM this week also formally rolled out IBM Services for Multicloud Management, a portal through which multiple clouds can be managed using software-as-a-service (SaaS) applications developed by ServiceNow, along with a set of IBM Services for Cloud Strategy and Design through which IBM will advise clients on how to architect the right holistic cloud strategy. The premise of both those services is that organizations soon will be rightsizing workloads to run on whatever cloud or on-premises IT platform that best suits them. The portal developed with ServiceNow provides access to dashboards and analytics designed to enable IT organizations to analyze cloud spend and manage workloads deployed on any platform, including Amazon Web Services (AWS) and Microsoft Azure.

IBM is also extending the cloud services it provides to include the IBM Cloud Hyper Protect Crypto Service, which makes use of IBM LinuxONE mainframes to provide encryption key management built on the only FIPS 140-2 level 4-based technology.

Hunter said IBM’s approach assumes organizations have already bought into best DevOps practices and API management to provide the means through which multiple clouds will be managed. The challenge many of those organizations are now wrestling with is implementing processes that determine where a workload not only should be developed and deployed but also moved as its characteristics evolve. To provide that analytics, IBM is employing machine learning algorithms running on distributed instances of Kubernetes to continuously track changes and updates to cloud platforms, said Hunter.

IBM previously outlined a strategy under which it is trying to meld DevOps and ITIL-based processes to make it easier to manage workloads running on any cloud or on-premises IT environment. While the growth of cloud computing continues at an exponential rate, IBM executives this week took pains to remind conference attendees that 80 percent of enterprise-class workloads still run in on on-premises IT environment. In fact, IBM’s rationalization for spending $34 billion to acquire Red Hat is based on that fact. However, while many workloads will continue to run on-premises, IBM is clearly making a case for centralizing the management of all those platforms on IBM Cloud.

It most likely will take the better part of the next decade to determine where different types of applications may wind up running. Predictions concerning the percentage of application workloads that will run in the cloud continues to be one of the most hotly debated topics in the IT industry. Whatever the outcome, however, it’s 100 percent certain more than one cloud platform will be involved.

— Mike Vizard

February 14, 2019
Python Benefiting From Increase in DevOps Use

DevOps and Python have developed a mutually beneficial relationship.

In a recent survey released by the Python Software Foundation and software development company JetBrains, researchers found that more and more developers are using the program language for an array of versatile projects, including DevOps and machine learning.

In fact, Python developers told the researchers that DevOps is increasingly part of their professional development work.

“In 2018 we had significantly more respondents specifying they’re involved in DevOps (an increase of 8 percent compared to 2017). In terms of Python users using Python as their secondary language, DevOps has overtaken web development,” according to the “2018 Python Developers study,” which is based on responses from 18,000 developers in 150 countries.

Over the last two years, Python has grown more important to developers thanks to its flexibility, especially when it comes to web development. It’s no surprise that as other projects have gained popularity in the enterprise—DevOps and machine learning—developers have started adopting the language to other disciplines, especially as businesses invest more in application development.

In TIOBE’s February 2019 index, Python now ranks third behind favorites such as Java and C, edging out C++ for its current position.

Digging into the numbers, the survey finds that 43 percent of developers are using Python for DevOps projects, as well as system administration and writing automation scripts. That’s an increase from the 35 percent who reported the same types of uses for the programming language in 2017.

The report notes that these numbers reflect a combination of Python used as either the primary or secondary language for various projects. As the report notes: “DevOps/system administration/writing automation scripts has moved into first place among Python users using it as a secondary language.”

Machine learning is another up-and-coming category, with 38 percent of developers using Python for these projects compared to 31 percent last year.

Overall, data analysis remains the No. 1 use case for Python developers, with 58 percent reporting that it’s their main focus when it comes to development.

In second place is web development, according to the report.

In terms of the top cloud platforms for Python developers, Amazon Web Service is the top choice, with 36 percent of respondents reporting that they use AWS. From there, it’s Google Cloud Platform (29 percent) Heroku (26 percent), DigitalOcean (23 percent) and Microsoft Azure (16 percent).

About a third of those surveyed responded that they don’t use any cloud platforms.

When asked about running code in a cloud production environment, 47 percent responded that they use virtual machines, although 40 percent of Python developers reported that they prefer containers. Another 28 percent use platform as a service (PaaS) and 21 percent are taking the leap to serverless computing.

Additionally, when developing for the cloud, about 35 percent of those in the survey report that they use Docker containers.

This increase use of Python within DevOps, as well as other connected areas such as serverless computing, does not surprise Anshu Agarwal, the CEO and co-founder of Nimbella, a cloud-agnostic, serverless cloud platform for developers. She noted the language’s use within a wide range of projects, specifically machine learning and AI, are driving this change.

“Python developers are driving serverless requirements in new and unique ways, primarily because of the interest in data analysis, ML and AI,” Agarwal wrote in an email. “These data analysis and ML/AI-centric serverless use cases are different than request/response style serverless functions which run for much shorter durations. Python functions in serverless may have much longer durations (10s of minutes or hours), and consume a lot more memory. For such functions a different kind of serverless framework needs to be adopted that can handle long running workloads.”

When it comes to continuous integration systems, about a third of respondents don’t use any. However, when they do, Jenkins is the most popular at 25 percent, followed by Gitlab CI and Travis CI, with 18 percent each. For configuration management, 20 percent reported using Ansible, while 9 percent used custom tools. Puppet, Salt and Chef also appeared in the list.

Finally, about two-thirds of Python developers (69 percent) use Linux as their main operating system of choice for development. Microsoft Windows comes in a distant second at 47 percent and Apple’s macOS in third at 32 percent, according to the study.

— Scott Ferguson

February 14, 2019