devopsy.kinsta.cloud

Category: DevOps Practice

xMatters Looks to Meld DevOps and ITIL

With the launch today of a flow design tool from xMatters, the divide between DevOps and ITIL-based approaches to managing IT is narrowing.

xMatters Flow Designer is a drag-and-drop graphical tool that either developers or IT operation teams can employ to create incident management workflows on top of xMatters incident management software. Doug Peete, chief product officer of xMatters, said those workflows feed data into a variety of third-party IT management tools to enable IT operations and DevOps teams to consume the same data in different tools—for example, in an instance of ServiceNow for IT operations and as a message in Jira or Slack for DevOps teams.

Peete said the goal is to enable organizations to develop a more consistent approach to incident management. A recent survey of more than 1,000 IT professionals conducted by xMatters and Atlassian found that while 81 percent of developers and operation teams share information and tools, only 17 percent provide open access to information without time restrictions. The survey also found that 50 percent of respondents said the tools, processes and steps they employ vary from incident to incident.

Most of those organizations have very little in the way tools that enable them to effectively conduct a post-mortem analysis of those incidents, added Peete.

The arrival of the xMatters Flow Designer tool comes at a time when organizations are trying to find ways to meld the highly structured ITIL processes that most IT organizations rely on today and the inherently more agile approaches based on best DevOps practices. The most recent iteration of the ITIL framework incorporates some DevOps principles, but it’s apparent the two approaches to managing IT will be employed side by side in many organizations for many years to come. Developers these days tend to prefer DevOps methodologies, while IT administrators tend to appreciate the consistency of an ITIL framework that was designed to address IT governance concerns. By providing a set of workflow tools that span both those approaches, Peete said IT organizations can more easily meld those two approaches on their own terms.

At the same time, an extensible approach to incident management makes it possible for IT organizations to incorporate new tools into those workflows as their IT environment evolves. That capability also should make it easier to integrate incident management workflows across multiple organizations as well as incorporate a new IT environment as a result of acquiring another company.

Regardless of the path chosen, it’s clear that as organizations increasingly realize how dependent their business processes are on software in the age of digital business transformation, the more likely it is they will embrace DevOps practices. However, that doesn’t mean organizations need to abandon ITIL; there are plenty of IT processes that benefit from a highly structured approach. In fact, arguably the most important thing is to not let a philosophical debate over the merits of DevOps versus ITIL wind up becoming a major distraction that creates two isolated cultures who refuse to collaborate with one another.

— Mike Vizard

April 9, 2019
TypeFox Ties Cloud-Based IDE to GitHub

TypeFox has launched a GitPod.io service that melds the integrated development environment (IDE) based in the cloud from TypeFox with the open source GitHub repository.

Sven Efftinge, co-founder of TypeFox, said this level of integration will make it easier for developers to navigate multiple projects from which they pull code from GitHub into an Eclipse-based IDE.

In addition, when developers open a Gitpod.io workspace, they will find that any build involving an update to an artifact housed in GitHub will have been automatically completed, said Efftinge. That will reduce substantially the amount of time developers spend waiting for build to finish, he noted.

TypeFox collaborated with Google, Ericsson, Arm and Red Hat to develop the open source Eclipse Theia project, which provides a foundation for building IDEs in the cloud using TypeScript, CSS and HTML for a desktop-like user experience. At the highest level, Theia consists of a front end running in a browser or in the local desktop application and a back end running on any host or locally within the desktop application. Both the front end and back end communicate through JSON RPC over WebSockets. TypeFox then leveraged that project to build its own cloud-based IDE.

Gitpod.io further advances the collaboration capabilities that are inherent of cloud-based IDE by adding the ability to capture and share snapshots of the development environment.

Efftinge said following initial support for GitLab, the next effort will be focused on integrating Gitpod.io with the GitLab repository, which is used widely in traditional enterprise IT organizations.

Regardless of the repository employed, more attention needs to be paid to reducing the daily friction developers encounter when developing applications. While best DevOps practices go a long way to accelerating application development and deployment, there a lot of wasted steps during the development process that, if eliminated, would increase developer productivity substantially. Of course, the more productive developers become, the more applications there are moving through the continuous integration/continuous development (CI/CD) pipeline.

It’s still a bit unclear to what degree application development will move to the cloud. It’s certainly easier to streamline development process in the cloud. But many developers remain attached to individual workstations because they may not always be in a location that has reliable access to the internet. However, as application development—especially in the age of microservices—becomes more of a team sport, it’s clear that more application development is being pushed into the cloud. Pushing application development into the cloud also provides the benefit of reducing the number of high-end PCs and workstations an organization may need to acquire for its developers.

Whatever the path chosen, it’s clear repositories such as GitHub and GitLab are now at the center of application development. Very few developers these days are writing raw code. Most of them are combining artifacts stored in repository to build their applications. The challenge organizations now face is bringing some consistency to what otherwise could easily become a chaotic process.

— Mike Vizard

April 9, 2019
How to Modernize Mainframe Applications
It’s easy to understand the current disillusionment around the term “digital transformation,” as it has been thrown around vaguely as the ultimate answer to business innovation without much recognition of the nuances of technological change.

When it comes to mainframe systems, the issue here most likely can be traced back to the word “transformation.” The perceived challenge inherent in the transformation of a vital business process is that it involves a paradigm-shifting, instantaneous “big-bang” overhaul of IT, with no recognition of the complications involved in modernizing a complex web of intertwined systems and data.

It is this perceived challenge that has seen large enterprises reluctant to change and modernize legacy technology, as the pressure to throw out the old and bring in the new is seen as simply too risky. Horror stories, such as the infamous failed IT upgrade at UK retail bank TSB in 2018, have dampened the spirit of many enterprises to modernize with the perception that a big transformative leap may be required.

Many businesses seek a solution to this “all or nothing” conundrum and would benefit from modernizing legacy IT in small steps—carefully considering the modernization of infrastructure and applications as separate processes. This more graceful approach delivers gradual benefits while reducing risk and acknowledges that modernization is a continuum, not a destination.

What is the Reality of a Modernization Process?

Applications developed to support business requirements are never static; they change along with the business. Modern development environments, infrastructure and DevOps processes have benefited from tremendous innovation in recent years, enabling IT to more rapidly evolve applications to support these changing business requirements.

There is mounting pressure to integrate mainframe applications, which house critical business processes with these approaches. The aim of this modernization is to enable legacy applications to take advantage of the innovations spearheaded by the modern world of open source—and, therefore, respond more rapidly to the needs of the business. To achieve this, organizations would benefit from a two-phase approach:
1. Modernizing IT infrastructure: Moving legacy applications, in their original formats, to an environment where they can benefit from the scalability benefits of cloud deployments, exploit open source solutions and significantly reduce cost once legacy systems are replaced.
2. Modernizing mainframe applications: Rewriting or transcoding into modern languages only those legacy applications or programs that are most in need of modernization, while enabling them to communicate with those still in legacy languages.
Infrastructure Modernization

The foundation of infrastructure modernization is the movement of legacy applications, written and compiled to run on the IBM Z mainframe, to the Linux operating environment running on x86 architecture computer, either on-premises or in the cloud. Using conventional methods to re-host these legacy applications into x86/Linux requires all programs that make up a legacy application to be recompiled and, in many cases, changed. This approach has often resulted in budget overruns and a high risk of failure.

If an organization is able to move these applications and data to a modern environment without the requirement for recompilation or reformatting, this migration risk is significantly reduced.

Once moved to a Linux environment, and despite remaining in original languages, these applications can now benefit from a vast array of tooling to optimize their ongoing exploitation. They can take advantage of cost-effective scaling options, modern monitoring, high-availability tools and many other facilities available on Linux on x86. For those legacy applications that continue to be maintained, modern development environments are now available, meaning their management is no longer restricted to a diminishing mainframe workforce.

Application Modernization

The goal of many companies is to have their entire application portfolio based on modern languages or packages. Where legacy applications are concerned, attaining this means converting the individual programs from COBOL or PL/1 (and, in some cases, even more arcane languages) into modern languages such as Java. In so doing, an organization can tap into much larger pools of development talent and tooling. Furthermore, the door to an entire world of open source innovation is opened, which alters entirely the trajectory of application development for these legacy applications.

Yet modernizing mainframe applications in this way, using conventional methods, is not a process to be taken lightly.

Infrastructure vs. Application: The Chicken or the Egg?

When looking to modernize an application, many organizations question whether to start with the infrastructure or the application itself. The answer is infrastructure.

Of course, the organization could attempt to first modernize the applications in place. At a glance this method may seem the least disruptive, but very quickly the effort would run into many of the challenges previously highlighted that have caused the mainframe to be abandoned as a contemporary platform for application modernization. Using a modern programming language such as Java is one way to enable modernization with those programs being completely rewritten. With the whole process conducted on a legacy mainframe, costs and resource contention become practical issues. Organizations that run mainframes carefully control access to mainframe resources due to the punitive costs of exceeding licensed capacity; indeed, this resource contention point is another reason for companies to move ongoing development off the mainframe.

When taken collectively, it is easy to understand how these factors have conspired to thwart most modernization initiatives that choose to modernize applications first. By modernizing the infrastructure first, the issues identified above are circumvented.

Once mainframe applications and data are re-hosted on modern, open environments on x86, Linux and the cloud, the real process of application modernization, as outlined above, can begin.

The modernization journey is not easy, but continuing to depend on a declining skill set and diminishing vendor community in the mainframe world is no longer an acceptable alternative. By breaking up the modernization journey into manageable pieces, moving forward over time to the desired end state is the most realistic approach. After all, to quote a Chinese proverb, “A journey of a thousand miles begins with a single step.”

— Dale Vecchio
April 8, 2019
Autonomous Testing’s Role in Agile DevOps

DevOps testing, as an Agile methodology, is key to delivering testing and integration on a continuous basis, including continuous feedback, to enable better collaboration and quality.

For too long, testing has progressed by small incremental steps, creating a quality assurance (QA) winter. We think it’s time for a revolution, so we have written and published our manifesto for change. It’s a future where an intelligent test agent handles all testing needs so organizations’ resources are freed up, thus saving them time and driving revenue. This future means intelligent testing is a function everyone on a given team can contribute to—and, above all, allows for quality to be embedded in everything an organization produces.

How do we get there? Apply these perspectives and view testing strategically:

Testing Shouldn’t Be an Afterthought

QA should be a C-suite responsibility. No one laughs at the idea of a chief compliance officer, so maybe you should consider appointing a chief quality officer. Successful companies are the ones that adopt the mantra of test early, test often and test in depth. This means testing becomes a key strategic requirement in your organization. If testing needs more resources, make them available. If there is tension between your test and dev teams, try merging them. If you want to release new features frequently, look into continuous testing.

Testing Must Be Cloud-First

Up until now, testing has been done in-house using old servers, begged resources and outdated infrastructure. We believe testing should be run in the cloud, allowing you to leverage an almost infinite resource pool. Testing done like this will be faster, more efficient and cheaper.

Testing should not be the poor man of the software development process. Companies that engage with their testers produce the best and most reliable software. Those that treat testing as an afterthought risk bugs, production delays and, ultimately, their reputations.

Agile DevOps Requires Autonomous Testing

DevOps has become critical to the success of many companies. Without DevOps, many of the services end users take for granted would simply fall over. Social media sites and apps such as Facebook and LinkedIn would fail if they constantly went offline. If content delivery services such as SoundCloud, Netflix and Hulu kept dying mid-track, you’d soon cancel your subscription. People also expect messaging services to be 100% reliable.

DevOps straddles the border between developing software and running it in production. It is what you get when you apply the Agile approach to software development to deploying services in production.

Autonomous testing is the outcome of intelligent test automation. With autonomous testing, you now have an intelligent test agent (ITA) augmenting the work of your test and DevOps teams. This ITA is like the perfect regression tester—focused, tireless and driven, but still intelligent. Using this ITA means changes can be tested and deployed completely automatically. Without this, the job of an agile DevOps engineer would be infinitely more difficult.

Intelligent Automation Just Works

Classic test automation is dumb, and like all dumb systems, its growth and utility are limited. In this era of AI, test automation should be intelligent, too. There are few areas of software development where it is easier to apply AI—and even fewer where the rewards are so obvious and immediate.

Put simply, well-designed, intelligent automation just works. Intelligent automation means tests become less brittle, so no more time wasted on unnecessary maintenance. Productivity increases because test failures are automatically diagnosed and classified. New tests become easier to create and test flows are easy to reuse.

— Paul Clauson

April 8, 2019
Gremlin Brings Chaos Monkey Testing to Spinnaker CD Platform

Gremlin this week announced it will integrate its hosted chaos engineering service with continuous delivery platforms starting with the open source Spinnaker project developed by Netflix.

Chaos engineering traces its roots to a resiliency testing philosophy that posits applications should be able to keep functioning regardless of any service or infrastructure failure. Most IT organizations, however, don’t have the expertise required to build the tools required to testing resiliency based on those “chaos monkey” principles that randomly turn services off to see if an application fails.

Matt Jacobs, principal software engineer for Gremlin, said the decision to integrate with Spinnaker is being driven by the need to infuse chaos monkey resiliency testing directly into the application development and deployment pipeline at a more granular level. That capability is now a bigger requirement than ever because as organizations embrace microservices to build the next generation of cloud-native applications, it’s simply not possible for DevOps teams to manually track every dependency, he said, adding resiliency testing based on chaos monkey principles is just another type of testing that should be automated as part of a larger continuous integration/continuous delivery (CI/CD) platform to ensure an application doesn’t fail completely.

Integrating with a Spinnaker project that has drawn support from Google, Microsoft and Oracle provides the easiest path to achieving that goal. Spinnaker is also now a core element of a Continuous Delivery Foundation led by Alibaba, Anchore, Armory, Autodesk, Capital One, CircleCI, CloudBees, DeployHub, GitLab, Google, Huawei, JFrog, Netflix, Puppet, Red Hat, SAP and Snyk.

The Gremlin service is accessed via a user interface and comes in two forms: a free edition and an enterprise edition, which comes with support from Gremlin. While the concept of chaos monkey testing has been around for some time among the most advanced DevOps practitioners, enterprise IT organizations, which tend to be more conservative when it comes to introducing anything that might deliberately break their applications, will need a considerable amount of hand-holding. Gremlin is designed to learn the application environment in a way that introduces chaos in a controlled fashion before an application is deployed in a production environment.

On the positive side, resiliency testing based on chaos monkey principles tend to accelerate the rate at which IT staff learn how an application has been architected, said Jacobs. That can prove especially invaluable when training new members of a DevOps team, he noted.

Ultimately, the decision to embrace resiliency testing comes down to how proactive an organization wants to be when it comes to preventing downtime. When mission-critical applications are offline, there is a direct impact on the amount of revenue an organization generates. There are plenty of functional and performance testing tools, but the real difference between having an issue and a catastrophic event that requires DevOps teams to get out of bed in the middle of the night often comes down to how truly resilient any given mission-critical application really is.

— Mike Vizard

April 5, 2019
Cloud Visibility Remains a Top Challenge for Many Organizations
A number of enterprises have a love/hate relationship with cloud computing. Because of its low upfront costs, the expense for cloud is lower, especially in the early years. But historically there has been another cost: the challenge of obtaining good visibility into applications, users and network traffic. I thought advancements in cloud management and security tools in recent years had largely overcome these concerns. Turns out, I was wrong—or, perhaps, market perception hasn’t caught up to reality.

A recent survey from Dimensional Research, which is based on responses from 338 IT professionals, found that fewer than 20% of IT professionals believe they have complete, timely access to data packets in public clouds. In private clouds, the situation is somewhat better, with 55% reporting adequate visibility. Unsurprisingly, on-premises data centers had the most positive response when it came to data visibility.

The survey authors said the results show companies have low visibility into their public cloud environments, and the tools and data supplied by cloud providers are insufficient. They identified some of the challenges such lack of visibility can cause, including the inability to track or diagnose application performance issues and monitor and deliver against service-level agreements, as well as delays in detecting and resolving security vulnerabilities and exploits.

Some of the findings regarding cloud visibility from the survey:
- 87% of respondents expressed fears that a lack of cloud visibility is obscuring security threats to their organization.
- 95% of respondents said visibility problems had led them to experience an application or network performance issue.
- 38% cited insufficient visibility as a key factor in application outages and 31% in network outages.
Interestingly, 99% of respondents identified a direct link between comprehensive network visibility and business value. This shows IT performance contributes directly to the business bottom line. Here are the benefits respondents cited regarding visibility:
- Monitors and ensures application performance (60%).
- Enables better threat identification (59%).
- Helps to identify security indicators of compromise (57%).
- Helps identify performance degradation.
- Enables threat protection tools to identify malicious traffic by source.
- Enables security monitoring tools to detect indicators of compromise.
- Enables the monitoring of traffic at every link of their network.
A full 86% of respondents also stated visibility was important for network and application performance monitoring, and 93% stated it was valuable for security. Cloud consumers said predicting application performance was a key challenge.

Still, the lack of perceived visibility isn’t holding back public cloud adoption, with 84% of organizations having increased their cloud usage in the past year. While 3% of those surveyed had a moderate decrease in cloud usage and 13% remained unchanged, 63% experienced a moderate increase and 21% experienced a significant increase.

As organizations engage in digital transformation efforts and continue to turn to public clouds, organizations can’t afford to lose visibility. This is crucial for all of the business benefits of cloud traffic visibility the survey respondents cited, such as attaining superior application performance, meeting SLAs and shrinking the time it takes to detect and remedy security relation issues.

But here’s an unpopular opinion: There’s not much of an excuse for having poor visibility into cloud workloads anymore. There once was—that’s for sure—but today there are many tools made available by cloud service providers and third-party security vendors.

Today, it’s a matter of will and finding the right toolsets, putting them into place and making sure that security and operations teams have the software controls and the training they need to use them, so the organization can attain the level of visibility necessary.

— George V. Hulme
April 5, 2019
Using Jenkins Configuration as Code
Using the following code, we can configure Jenkins Master to integrate Sonar, Nexus and Jira and manage them as a code. Code as configuration is a plugin in Jenkins, which is very popular and simple to use.

First, we need to install plugins from Jenkins Market place.

Manage Jenkins > Manage Plugins > Advance

After installing, got to Manage Jenkins and you will find Configuration as Code settings.

After opening this link, it will give us the Jenkins Configuration as Code screen.

Here, you have two steps:
1. URL or Path.
2. Export the Existing configuration.
Provide URL or Path

We have two options for storing Jenkins.yaml file:
1. Any directory Jenkins is running
2. Version Control (such as Git)
If we go with option one, then we need to place Jenkins.yaml file in JENKINS_HOME directory, place that directory path and click on “Apply new configuration.”

If we go for version controlling, we need a place for the Jenkins.yaml file and then we need to open the file in a raw format in GitLab, as shown below:

Now, place that link and click on Apply new Configuration.

Export the Existing Configuration

Export the current configuration into Jenkins.yaml, click the Export Configuration button to download the Jenkins.yaml file.

After opening this file, you may see some errors. Keep in mind, this is still in under development so the export function is now working properly.

We can write Jenkins.yaml file in simple YAML format.

Plugins

This is what the YAML format looks like:

Let’s move to next step tool’s installation.

Tool installations

This is what it would look like during the tool installation:

Tools Integration

The next step is the pipeline job configuration.

Setting Up Job with JCasC

Now, let’s bring everything together and see what happens in Jenkins:

After applying the new configuration, it will show the date, time and path of the configuration.

If there is an error in syntax format, it will throw the error as shown below:

Rest API Method to Submit the Configuration

Here, officially we have JCasC. There is a way to apply changes through CURL.

Here we have two CURL commands:
1. Validation
2. Applying the new
Validation

Here, I am using Jenkins YAML to store in version controlling tool with public repo:

Response will be as shown below:

Applying new Configuration

— Santosh Rahul Goru
April 4, 2019
Using Product Experimentation for Continuous Improvement

Organizations are embracing product experimentation as a tool for continuous improvement

Companies that embrace product experimentation are able to capture highly accurate data about markets and product ideas and gain intelligence about market trends. As a result, these companies are able to continuously deliver innovation and measure business outcomes.

Industry leaders such as Netflix, Instagram and Facebook have used product experimentation to maximize a positive customer experience, increase their user base and elevate their market positions to the highest of levels.

Let’s say you want to be one of those companies—or at least you and, hopefully, your team, want to experiment with product features in hopes that your company will embrace experimentation on a larger scale to achieve continuous improvement.

Great. But now what?

Just thinking about learning the basic concepts of experimentation and figuring out where to start can seem like an overwhelmingly daunting task. Fortunately, there are a few tried and true steps you can take to get on the path to product experimentation for continuous improvement.

Measure First

“You have to walk before you can run.” That old adage certainly applies here. Rather than attempting to jump headfirst into experimentation, organizations first must ensure they are measuring things. Are you tracking things such as users per day, trends, dates and conversions? The idea here is to become data driven, because before an organization can run effective experiments, it first must gather quantifiable data; understand what this data means for overall growth, user retention and long-term projection; and then base business decisions on this data.

Buy or Build

No matter the size and scope of the experimentation throughout the organization, even if it is just one team or department, you need a platform or tool to experiment on. Many of the most progressive organizations that have based their growth and continuous improvement on experimentation have built in-house solutions to facilitate this. However, this can be time-consuming from the start and the platform will continue to require attention as it evolves. This evolution also requires a specific skill set to make it happen—specifically, data science and engineering—to ensure the experimentation platform is statistically sound.

For most, investing in a robust tool or platform built specifically for experimentation is the best option. Most experimentation platforms allow you to plug and play, which provides the right features and capabilities to get an experimentation started quickly.

Hypothesize

The next step is to create a testable hypothesis and then record and analyze the data during testing. It’s important to begin thinking in terms of hypotheses, as framing plans and ideas in this manner makes them easier to reuse. In other words, being able to say, “Based on this, we predict we can move that metric,” makes your processes very repeatable. Getting into this mindset and having a standardized format—in terms of how things are designed and reported on—makes the lives of everyone on your team easier, as they can simply replicate similar experiment designs. And they will have fewer excuses not to.

Keep It Simple

Now that you have quantifiable data and some hypotheses to work with, you are ready to dip a toe into the experimentation pool. It is always a good idea to start with a single simple experiment first, no matter how small. For example, experimenting on a known pain point in your product is a nice place to start. Once you can successfully run an experiment that allows you to fix something in your product that you know is a problem, then you are ready to move on to bigger things such as revenue creation.

Choose the Right Metrics

As your team’s product experimentation efforts become more sophisticated, choosing the right metrics—and more importantly, your overall evaluation criteria (OEC)—should become a top priority in continuous improvement. Your OEC comprises the most important metrics of your experiment and it’s a measure of long-term business value or user satisfaction.

In choosing which metrics to use, there are three essential characteristics of an effective OEC to consider.

The first of these is sensitivity. This means the metrics you have chosen should be adjustable due to small changes in business value or user satisfaction, which helps ensure the result of an experiment is clear within a reasonable time period. For example, let’s say a customer only buys something from your website once or twice year. That’s not a very useful metric for an experiment since it would be difficult to influence.

Next is directionality. Your metrics should be moving in one direction or another based on whether your overall business value increases or decreases. That said, the direction some metrics take can be ambiguous. For instance, say you are attempting to measure customers’ engagement with your support team. An increase in that metric could be a positive thing because it might be an indication that more people are engaging with your core product. On the other hand, a decrease in this metric could be an indication that your core product has become easier to use and therefore now requires less support.

The third characteristic to consider is understandability, meaning the metric should be easily understood, both in terms of its direction and its definition, by your entire organization. Moreover, you should ensure it ties the experiment directly to business value.

Of course, much more will need to be done to transform your organization into a place that truly embraces an experimentation culture and reaps the benefits that it brings in terms of customer experience and beyond. But these steps should set you and your team down the right path.

— Sophie Harpur

April 4, 2019
Apply Balance and Critical Thinking for Greater Agile Success
In an ideal world, implementing DevOps practices and Agile methodologies would be the key building blocks organizations need to realize rapid, high-quality releases.

The ability to iterate on a product with a high volume, velocity and level of quality is the ultimate goal of the modern enterprise; every CEO wants a faster time to market. But keeping business practices agile is much easier said than done. The term “agile” is broad. While The Agile Manifesto does a good job of defining the concept, there are myriad technical and non-technical aspects to the idea. At the end of the day, simply saying you are “agile” isn’t some silver bullet that delivers faster cycle times, a better product and more satisfied users.

Instead, success with Agile methodologies means thinking more critically about how they are deployed and if your current DevOps practices are strong enough to support that strategy.

What is Agility, Really?

In practice, agility is a complex combination of things, perhaps best understood by what it’s not. Being agile with a product is not simply thinking about how fast you can complete a collection of features. Yes, it’s great when the backlog becomes smaller, faster. But Agile, when applied correctly, is really about having a true understanding of the user’s business problem and a real idea of their envisioned, desired outcome—and what it might take to achieve that outcome.

This requires a shift in thinking when it comes to the product backlog. Too often, backlogs are viewed vertically, meaning features are built that may not have independent value. As a result, release efforts grow in development effort and elapsed time, limiting the ability to quickly release in small, incremental stages. As an example, consider search functionality as a feature set. On its own, search isn’t necessarily an independently valuable feature. What are users searching for? Why do they need this information and what are they doing with the results? A vertical view of agility fails to ask these questions and focuses only on the myopic feature set at hand. Consequently, even the most mature Agile practices are limited in what they can accomplish. Who really cares if you can get a feature out of the backlog quickly if it doesn’t matter to users?

Agile thinking should instead be applied horizontally; the user’s journey is the center of everything. What do they need to accomplish? What are they unable to do now and how quickly can we give them that ability? In the search example, this means thinking beyond simply building a search function and instead developing a new product release that eases a specific pain point of users. Quick and iterative changes based on direct user feedback is the goal.

So Where Does DevOps Come In?

Agile is hardly a foreign concept for software engineers—86 percent use an Agile methodology (such as scrum) day to day. But the tools, processes and practices of DevOps need to be honed constantly to support iterative development, testing and release of code.

As with the development of any product, software developers have much to gain from building and testing functionality in small, iterative releases. Smaller tests allow for more intensive studies of accuracy and problem-solving before allowing an entire organization to rely on a solution. While these pilots may not be fully rolled out, the teams involved must be integrated to include voices from a variety of stakeholders, including but not limited to:
- Architecture
- Experience design
- Development
- Security
- Quality assurance
- Operations
Teams must also keep these DevOps best practices front-of-mind when developing a software solution:
- Employ microservices. A solution built from these separately deployable services that perform specific business functions is much easier to scale out or expand. Microservices also remove single points of failure (SPOFs) by preventing issues in one area of an application from affecting others.
- Blend continuous integration (CI) and continuous delivery (CD). These two DevOps strategies support each other in much the same way as integrated human teams that consider stakeholder needs. While CI automates the build and testing processes, CD automates the deployment cycle from development to production. The automation behind both results in a vastly increased deployment frequency.
- Use infrastructure-as-code. Modern cloud platforms allow for infrastructure to be software-defined. This allows DevOps practitioners to bring best practices from the field of software engineering to bear against systems challenges. Think about the new features AWS announced for CloudFormation this year, for example: configuration drift detection, dynamic references, macros (functions with iteration support) and linters. All of these are in support of treating infrastructure orchestration as a software engineering activity, allowing DevOps teams to use one common set of tools and practices.
- Refine observability. As architecture becomes ever more complex, the practices of monitoring and logging are evolving. No longer can you independently monitor the uptime of a server and simply log OS and app issues. Application performance management (APM) becomes paramount, measuring aspects of the user experience as they flow through the app, as does identifying and publishing relevant business metrics. The distributed nature of microservices and the various cloud services enabling them, demand new tooling to measure potential bottlenecks in the form of distributed tracing. And the DevOps best practice of “hands-off production” (e.g. no shell or attachment access) means you’ll need to provide real-time visibility into the health and inner workings of your application via log shipping, adding your centralized logging solution into the critical path of application troubleshooting.
- Focus on communication. This is perhaps the most beneficial point for your team and should not be overlooked. The cross-functional, collaborative teams supported by both Agile and DevOps culture break down traditional silos and unite members around a common goal. Formal structures around these teams should start early and continue through retrospectives to assess and celebrate everything you’ve accomplished in a short time. You’re not working in a vacuum, and this project isn’t the only one your team will ever produce. Keeping careful records of successes, failures and areas for improvement will only make your work more efficient and Agile in the future.
Agile is the perfect process for the modern tech landscape—releasing quickly in small bites that are then perfected based on customer feedback. In living up to that ideal, teams who turn to Agile in managing their projects shouldn’t take a set-it-and-forget-it approach to their processes. Instead, bolstered by DevOps tools, culture and practices, they should embrace the ongoing refinement and reflection required to support both customer needs and business goals.

— Adam Hanson
April 3, 2019
Integrating Maven and Nexus for Continuous Delivery with Jenkins
This article demonstrates how to integrate Maven with Nexus repositories and deploy snapshot and release artifacts automatically during the build process. It also helps to understand version uptick using Maven.

In this article we’ll cover:
- Creation of a hosted repository for snapshots and releases.
- Creation of a proxy repository pointing to Maven Central.
- Creation of a group repository.
- Integration of Maven with Nexus repositories.
- Configurations in the project pom.xml file.
- Configuration of the Jenkins release build.
Proxy Repository

A proxy repository refers to a remote repository. The initial request for a component is forwarded to the remote repository. The component is then retrieved and stored locally in the repository manager, which acts as a cache. Further requests for the same component are fulfilled from the local storage.

Hosted Repository

A hosted repository is a repository that stores components in the repository manager as the authoritative location for these components.

For release, follow the same configuration done for snapshots except Name, Version policy and Deployment policy.

Group Repository

The combination of multiple repositories of the same format into a single item is a repository group.

This feature of the Nexus Repository Manager lets developers rely on a single URL for their configuration needs.

Create a new Maven (group) repository and configure it like:

Integration of Maven with Nexus Repositories

Nexus repositories are integrated with Maven using the Mavens settings.xml file. This file is usually present in ($HOME/.m2, in case of Jenkins /var/lib/Jenkins/.m2) for user-level configuration. Configurations in this file are applied for all the Maven builds initiated by the user.

There are two ways Maven can utilize the Nexus/other artifactory in general. One is to download the dependencies and use the artifactory as mirror during the build, and the other is to upload the built artifacts into the repositories.

Each repository in the Nexus and the access credentials are configured in this settings file, so that the repository can be referred in pom.xml and utilized to build the project artifacts and upload them.

<?xml version=”1.0″ encoding=”UTF-8″?>

<settings xmlns=”http://maven.apache.org/SETTINGS/1.1.0″ xmlns:xsi=”http://www.w3.org/2001/XMLSchema-instance” xsi:schemaLocation=”http://maven.apache.org/SETTINGS/1.1.0 http://maven.apache.org/xsd/settings-1.1.0.xsd“>

<servers>

<server>

<id>nexus-snapshots</id>

<username>admin</username>

<password>*******</password>

</server>

<servers>

<servers>

<server>

<id>nexus-releases</id>

<username>admin</username>

<password>*******</password>

</server>

<servers>

<mirrors>

<mirror>

<id>central</id>

<name>central</name>

<url>http://hostname:8081/repository/maven-group/</url>

<mirrorOf>*</mirrorOf>

</mirror>

</mirrors>

</settings>

Configurations in the Project pom.xml File

The following configuration is for downloading the dependencies from maven group repository:

<project …>

<repositories>

<repository>

<id>maven-group</id>

<url>http://hostname:8081/repository/maven-group/</url>

</repository>

</repositories>

</project>

And this is the configuration for publishing snapshot and release artifacts to Nexus:

<project …>

<distributionManagement>

<snapshot>

<id>nexus-snapshots</id>

<url>http://hostname:8081/repository/maven-snapshots/</url>

</snapshot>

</distributionManagement>

</project>

<project …>

<distributionManagement>

<releases>

<id>nexus-releases</id>

<url>http://hostname:8081/repository/maven-releases/</url>

</releases>

</distributionManagement>

</project>

Add the tag to the SCM for the Maven build:

<scm>

<connection>scm:git:http://gitlab.com/user/project.git</connection>

<url>http://gitlab.com/user/project</url>

<developerConnection>scm:git:http://gitlab.com/user/project.git</developerConnection>

</scm>

Run – clean deploy in Maven goals for Jenkins snapshot build.

Configuration of Jenkins Release Build

In the Jenkins build configuration, enable the a Maven release build with required release goal. Also, make sure to select Preselect ‘Specify SCM login/password,’ which is required when performing Maven release with authentication.

After the snapshot build completion, the below inputs are needed to perform the Maven release. Here, username and password refer to the GitLab admin credentials.

Conclusion

By following the above steps, Maven and the Nexus artifactory can be integrated at the user level and the Maven execution for that user inherits this setting. A similar setting can be done at the system level with the Maven global tool setting.

These steps help project teams with one-time configuration of snapshot and release artifacts upload to Nexus, and a version uptick of pom.xml in GitLab. Similar settings can be performed for other artifactories such as JFrog.

— Mohamed Azeem
April 2, 2019