devopsy.kinsta.cloud

Tag: process

Automation Relation
One thing that is absolutely true: This latest iteration of IT improvement is about automation. Whether making DevOps more automated, working on any of the various “X-as-code” movements or implementing AI, the idea is to create what one vendor portrayed as a “human-free zone.” A space where automation does everything that needs doing.

And the one question we all need to be asking now is simple: Where do we want that human-free zone in our organization?

The answer is inevitably, “In more places than you think,” for a wide variety of reasons. Let’s cover what you should be thinking about when answering this question – because this change is already upon us and we’re doing a lot more without so much as an approval.
1. Anywhere that compliance is on the line and the action doesn’t risk crashing systems.
2. Anywhere the process being performed is so rote that any member of the team can do it and everyone thinks of that process as a distraction.
3. Anywhere that time is critical. Think of incident response to a DDoS or zero-day attack.
This is a very basic starting point. From here, we want to move into “anywhere we have a static list of steps to perform.” So, the above points would include things like, “If a new employee is in the marketing department, they belong in the Marketing security group. If they are further working on marketing for new products, they belong in the Marketing for product development group.” But eventually, you want to get to, “This app communicates on port 12345, so that port must be accessible, run the script to open that port.” The decision about “Should this port be open?” can then be shifted left to have more involved discussions about options and necessity. Our current world already does some of that; automation will make it standard.

And that’s what we’re getting. We’re in a world where some standardization is enabling automation, and automation is demanding that we figure out more standardization. And it is all good. There is a lot we do in IT that a trained monkey could do, and we need to let a trained system do it. The more time we have, the more problems we can solve … So let’s keep freeing up that time.

And keep rocking it. Your time is valuable precisely because you keep the organization running. Freeing up that valuable time just makes you a greater asset to the organization and makes your job less boring as a bonus.
April 12, 2023
How to Solve 3 Significant Development Challenges
As products, systems, and software grow in complexity, so does the development process. This can lead to an increased risk of flaws and can result in expensive and potentially reputation-harming mistakes or recalls.

These new complexities have shifted the product development process entirely, requiring optimization from end to end. Engineering and design teams need solutions that answer the most common challenges they face in this process. This means being provided purposeful, structured collaboration tools; connecting distributed team members; and accurately capturing and facilitating feedback, decision-making and context for requirements under review.

Navigating Development Challenges

To do this, teams developing complex products, systems and software must first overcome some significant obstacles. To help navigate this journey, let’s examine some of the most prominent product development challenges engineers are facing today.

Outdated Legacy or Document-based Solutions

The traditional approach to managing risks and requirements is highly manual, operating in an old mindset in which numerous spreadsheets and versioned documents are shared via email. This method has serious drawbacks, including no guarantee that data is up to date. This results in valuable time lost deciphering different versions or painstakingly making changes manually to achieve cross-team alignment.

Additionally, without a clear path of traceability—which can be difficult to achieve for those who rely on static documents—teams often end up referencing older and often outdated versions of requirements. This inevitably leads to misalignment, increased risk and costly rework.

The solution? Uplevel your traceability practices. Moving on from outdated legacy solutions to a modern requirements management tool is the first step to bolstering creativity, productivity and collaboration.

Compliance, regulations and standards processes

Product, systems and software development has become much more complicated over time. As a result, so has product risk and the accompanying regulatory compliance required.

To ensure the long-term success of organizations developing these complex systems, software and products, strong collaboration between teams, knowledge of applicable regulations and effective requirements management platforms are necessities. These tools provide a simple path and single source of truth for defining, verifying and validating key regulations. However, despite numerous industry studies indicating that requirements issues are a prominent factor of project distress, not everyone is convinced their team needs to focus on requirements development and management. Further, they’re unsure of how it will all pay off.

To address these challenges, companies need to streamline their quality and risk management strategies. This can be done by incorporating detailed traceability across the development life cycle—from the high-level user needs and systems requirements through to validation and verification.

Effective Review Cycles

Review cycles are often time-consuming and fragmented. For example, when approaching requirements management manually and with disparate documents, a cycle can be repeatedly delayed due to versioning issues and lack of a single source of truth updated in real-time. It also reduces team visibility throughout the review process; individuals end up spending hours per week in review meetings just to ensure everyone is on the same page.

To up-level this process, it’s necessary to have:
- One source of truth for requirements and tests.
- A straightforward way to send items such as requirements, user stories or test cases for review.
- Best practices for each major role involved clearly outlined (i.e., reviewer, approver, moderator).
- Real-time collaboration within a shared, dynamic requirements management platform.
- An outline of a formal approval process to capture and record sign-off.
Collaboration is Key to Development Success

At the end of the day, collaboration is at the core of all product development work. The ability to effectively work together is critical to ensure innovation doesn’t stop. In this era of rapidly accelerating change, structured and strategic team collaboration is the means for improving the product development process for all involved. By updating your legacy traceability solutions, establishing effective review cycles, and streamlining your regulations, compliance and standards processes, your team will be well on its way to a much more effective and unified product development process.
November 6, 2020
Why the DevOps Trinity is Key for Digital Transformation

As digital technology continues to disrupt and transform businesses across industries and around the world, the ability to deliver high-quality software rapidly will make the difference between survival and extinction for many companies. DevOps seeks to change the dynamics in which operations and development teams interact. Ultimately, successful adoption of DevOps and the process of continuous delivery are likely to determine whether an organization thrives or fails in the digital age.

To build a sustainable DevOps culture and achieve true DevOps success, organizations need to consider the DevOps Trinity—people, processes and tools—and understand how to connect all three:

People and Culture

The core tenet of DevOps is a culture of collaboration and shared goals, which eliminates finger-pointing and promotes teamwork, with everyone working together in support of the same outcome—reliable software quickly delivered to production. In essence, the goal is to enable the different stakeholders in the software development and delivery process to reach the mutual understanding and empathy needed to drive alignment on shared goals. To achieve this change, teams need to have trust, honesty and responsibility.

Process and Practices

DevOps aims to achieve predictable, efficient, secure and sustainable operational processes. Using a DevOps approach, simple processes become increasingly programmable and dynamic. Automation supports this objective by using technology to streamline the steps within the process.

Tools and Technologies

Some categories are more essential in a DevOps toolchain than others, especially continuous integration (build) and infrastructure as code (configure) tools. The common elements of the toolchain include applications for coding, building, testing, packaging, releasing, configuring and monitoring. Often, rather than a single tool, there are sets of tools (or toolchains) of multiple tools, that make up the process.

A sustainable DevOps organization cannot be achieved without all three components of the DevOps Trinity.

For those looking to learn more about people, processes and technology needed for successful DevOps implementation, join us in San Francisco Sept. 16-19 for DevOps World | Jenkins World. With this year’s conference theme “TRANSFORM,” attendees will learn how organizations are transforming software delivery, adopting DevOps and returning business value to their organization from DevOps initiatives.

Don’t miss Viktor at DevOps World | Jenkins World! Be sure to register today for 20 percent off your registration using the special devopsy.kinsta.cloud code just for our readers: JWDEVOPSCUST

— Viktor Farcic

September 12, 2018
Creating a Continuous Integration Delivery Pipeline
This article will provide an example delivery pipeline used through continuous integration, as well as the DevOps tools involved in the automation of building, testing and deploying code through your software development life cycle (SDLC).

Background

These days, it’s almost impossible to go anywhere without hearing about DevOps. In short, DevOps is the effort to improve communication and synergies between development organizations and operations organizations—hence the term, “DevOps.” The main focus is to improve the efficiency, quality and speed to market within the software development world.

Today, you also may hear and see a lot about continuous integration and continuous delivery/deployment. There are differences between the two for sure, but for simplicity’s sake, this blog focuses on breaking down the different stages of the process. This way you can get the basics and hit the ground running with a delivery pipeline in your organization. The delivery pipeline can be broken down into a few major buckets of work, or stages, as mentioned below.

In my opinion—and, again, trying to keep things simple for now—the stages can be broken down as follows:
1. Source Code Control (Management)
2. Build Automation
3. Unit Test Automation (could also include Integration Testing here as well)
4. Deployment Automation
5. Monitoring – not included in this discussion, and can be added at any time
I have outlined the progression of these stages in Figure 1 below. Let’s review each of the stages in a little more detail.

Figure 1 – Continuous Integration Delivery Pipeline Sample

Source Code Control (Management)

Background

Source code management, or source code control, certainly is not a new topic. This has been around for decades and has evolved over time. In a nutshell, an organization stores its code in a source code control system or repository so it can be tracked, maintained, versioned and audited. Developers should not store the code on their laptops or virtual machines and trust that will suffice for managing the code.

Possible Tools
1. Git is probably the most widely used SCM system out there. It is an open-source system.
2. Subversion (SVN) has been around for quite some time. It is also an open-source system and is heavily used in many organizations, but there has been more of a push toward Git.
Git and Subversion comparison can be found here.

Build Automation

Background

Once a source code management system is in place and actively being used by a development team, the team will need to be able to compile and build their code. This is probably the first step in the whole chain of continuous integration events—it’s what gets the ball rolling. The code needs to build cleanly before an organization can even think about deploying out to its environments for testing and production.

Possible Tools
1. Gradle is an open-source build automation system used by top companies including Netflix, Google and LinkedIn.
2. Maven is another open-source build automation system.
Gradle and Maven comparison can be found here and here.

Unit Test Automation

Background

Developers unit test their code to ensure the functionality they are building works as expected. In an ideal world, the development team should be saving these unit tests so they can be reused and put into a regression test bed.

Possible Tools

There are multiple tools out there for helping developers unit test their code. Many of these tools are open-source and can be used freely.
1. JUnit is an open-source unit test framework pretty widely used.
2. CA DevTest allows for the automation of unit testing, as well as a few other bells and whistles, such as service virtualization.
Deployment Automation

Background

For the last stage in the process, delivery teams need to deploy their code/applications out to various test environments and, of course, production. To reduce errors and overhead in the deployment process, while increasing speed to market, this step can be automated through a variety of tools and methods.

Possible Tools
1. IBM Urbancode uDeploy allows users to model a process and orchestrate their deployment. This process then can be repeated across all an organization’s environments, and tweaked for each environment as needed.
2. Ansible is an open-source IT automation tool. It can be used for everything from configuration management to product installation to application deployments.This tool is rapidly gaining acceptance and momentum in the DevOps community.
Conclusion

This was simply a quick overview for companies to get started down the path of continuous integration and the DevOps world. Figure 2 is an example of what the whole flow looks like from committing the code to repo to deploying the code to an environment, to help put things into perspective. It’s easy to see how the tools interact with each other on the journey to production.

Figure 2 – Continuous Integration Process Flow

About the Author / Seth Gagnon

Seth Gagnon is a Middleware / Infrastructure / DevOps architect with more than 12 years’ experience in the field. Please visit sethgagnon.com for additional related articles. Connect with him on LinkedIn and Twitter.
October 6, 2016
Delivering DevOps’ Business Value

DevOps is an interesting space to be operating in right now. More than just culture or a set of tools and processes, DevOps delivers speed and agility that can be truly meaningful for the business. Unfortunately, for most large enterprises, the path to the benefits isn’t always well-known and isn’t assured.

Over the last five to 10 years, a number of DevOps tools and solutions have emerged: Puppet, Chef, Jenkins, Git, SaltStack, Elasticsearch … and the list keeps growing. Originally, these tools were sought by developers and system engineers for their ease of use and low cost of ownership: You could download it, use it the same day and solve your specific problem. The issue for enterprises, however, is that nature abhors a vacuum; the problem one developer addressed with Chef is being fixed with Puppet by another developer in a different business unit. Group A is using Jenkins, Group B is using Chef Delivery. In short, after five to 10 years of technology adoption by developers and system engineers, the tooling landscape in the DevOps space is packed to the rafters with competing technologies, all of which are solving similar problems with overlapping features.

So why should this matter to you at all? Without careful planning, consideration and execution, your business will derive little value from this DevOps exercise.

Here are a few common pitfalls we see far too often:

Too Many DevOps Tools Doing Too Few Things

Building Rube Goldberg devices looks neat on paper, but those mousetraps fail very often in real life. The more tooling and solution handoffs you have, the greater your likelihood of failure: the less likely you are to be able to scale horizontally or vertically, the more toolsets you need to maintain, and the more skills you need on your team. Convoluted messes do not yield speed and agility. If it feels wrong, it probably is. If it is overwrought, look to remove complexity from the landscape.

Reintroducing Risk and OpEx Spent on Menial Tasks

As an organization you’re onboarding risk, hidden costs and a lot of the things you just spent the last 10 years automating away: tribal knowledge, lack of governance, lack of compliance, lack of third-party auditability and reporting. Many enterprises spend millions of dollars making it so their admins and developers could spend time on more valuable workstreams, just to throw them back into the salt mines of making continuous integration (CI), continuous delivery (CD) and infrastructure-as-code “work” over umpteen toolsets. The goal isn’t for DevOps to “work”; the goals need to tie to some tangible (i.e. measurable) business benefits. Anything can be made to work, but far fewer IT programs deliver value. Be the people who deliver real value.

Taxation Without Representation

All of the stakeholders need a seat at the table. Whether that means forming a SWAT team from operations (all infrastructure domains and security), development, QA or a permanent DevOps matrix organization, all vested interests need a say in how the process and tooling develops for all impacted domains (which is just about everything) across all environments (i.e Dev, Test, QA, Production). Far too often we see one set of stakeholders acting as the service provider for another group of stakeholders playing the role of customer. This is a solution for and by Development and Operations.

Congratulations, you’re having a baby. Let’s make sure it’s not ugly, okay?

About the Author / Jesse Langhoff

Jesse Langhoff is co-founder and CEO of Raindance, a cloud and automation consultancy serving enterprises nationwide. With over 15 years of experience in leveraging technology to address business challenges, Jesse works with customers to determine the optimal path to meeting their goals through the use of automation and solutions provided by the leading vendors in the space. Areas of expertise include strategy and assessments, private/hybrid cloud services, process automation and orchestration, DevOps and operations as a service. Jesse holds an MBA from University of St. Thomas – Opus College of Business.

August 3, 2016
DevOps is ITSM Best Practice!
What boggles the mind is that opponents of DevOps use risk as an excuse why DevOps would not work in their environments and why using the “traditional ITSM” approach is better and safer.

When I start asking them about their transition practices, specifically the alignment between transition, change, test and release models, I soon find that these models are either very skimpy or, most of the time, non-existent.

So here are the facts: You don’t want to do DevOps because you don’t want to do what ITSM best practice dictates!

ITSM best practice recommends building and aligning change, test, release, deployment and transition models that clearly define transition/change boundaries, practices, artifacts, acceptance criteria, roles, responsibilities and authorities, making transition practices repeatable, less risky and more reliable.

Is this not exactly what you need to do in DevOps automation? If you have not built the models and made sure they are tried and tested, you can’t automate them—and if you have built these models and made sure they are tried and tested (repeatable), then why not automate them? It’s a no-brainer.

The reality is that by using DevOps practices, organizations have far better control and manage risk far better than what opponents and detractors of DevOps have in their non-DevOps environments.

This may not be best practice, but we have for years recommended that during a post-incident review the following questions should be added to the agenda:
- Will we do this again? If so,
- Will we do it in the same way?
- Can we define an implementation approach based on the change just completed and the lessons learned, and can we reuse this approach safely in future (build change, release and test models)?
- If we can,
  - Build the models
  - Define this type of change as a standard change
Some say that this is not best practice, as “standard changes” are low risk. This, in fact, is not true!

ITIL® defines a standard change as follows (ST 4.2.4.7): “A standard change is a change to a service or other CI for which the approach is pre-authorized by change management, and this approach follows an accepted and established procedure … Every standard change should have a change model that defines the steps to follow … Authorization for each occurrence will be granted by a delegated authority … The crucial elements of a standard change are defined triggers, tasks are well-known, documented and proven, authority is given in advance … The risk is usually low and always well-understood.”

We have for years argued that standard changes are ideal candidates for automation. By extending the post implementation reviews, you identified safe candidates for DevOps automation!

About the Author/ Johann Botha

Johann Botha is the CEO of getITright, a DevOps Institute Registered Education Provider that will run its first African DevOps course May 23 in Johannesburg. Visit www.get-it-right.com for more info.
May 19, 2016
Six Honest Serving Men of DevOps !

“I Keep Six Honest Serving Men” is a famous poem by Rudyard Kipling.

Take help from these honest men in a brave and dangerous new DevOps world:

I keep six honest serving-men
(They taught me all I knew);
Their names are What and Why and When
And How and Where and Who.
I send them over land and sea,
I send them east and west;
But after they have worked for me,
I give them all a rest.

These six questions have been very useful in every new DevOps assignment: Ask the right questions early in the assignment and you can worry less about irritating others. It’s also the time to ask seemingly stupid but pointed questions.

You will find that these questions are of great help to the team in finding better ways of working or to eliminate unnecessary process steps and bottlenecks. And, as a new member, you gain a better understanding of the product and approach.

So, be bold in those retrospectives and interactions, and challenge any biases of the incumbent team.

January 7, 2016
Lean for DevOps: Is it OK to move the banana?
There is an urban legend about a Lean deployment which, in the process of applying the 5S to a workshop, drew lines on the desk to indicate where each worker should keep their banana.

5S, for the uninitiated, is a set of principles by which Lean practitioners organize a workspace – which might apply to a virtual, shared workspace, or Sharepoint repository just as well as a back office space or factory floor.
- Sort – identify and get rid of unnecessary items
- Set in order – organize what’s left so that everything has a logical place – including deciding on the optimum spot for banana placement
- Shine – tidy up and clean your new workspace
- Standardize – document and put boundaries around your newly organized workspace so that it’s obvious where everything should go – including drawing a line around that banana
- Sustain – establish systems, tasks and behaviors to ensure your work place remains neat, organized and free from clutter
These are great principles to apply to any shared workspace – and standardization will speed your adoption of DevOps principles. Standardizing the way we organize information allows for fungibility, so a Quality Manager can switch teams at short notice and quickly pick up the way their new team works, and access the information they need. Standardizing storage in a way is the principle behind Docker, and other modular code concepts. Standardization can reduce errors: if user requirements are captured in a consistent way it is easier to see at a glance if any essentially information is missing before we start work. And standardization ensures consistency, for example, applying a pre-determined decision tree and criteria to enhancement requests will allow prioritization decisions to be made quickly and equitably.

But you can have too much of a good thing, Creative activities and problem solving tasks are often intuitive or require people to take a variety of approaches until they find one that fits. A checklist just may not help here. Furthermore, one of the aims of DevOps is to increase flexibility and speed to market. So if your standard start to get in the way of that, you’ve lost your way.

As a Lean Coach I have struggled with the amount of churn I see in requirements during the development lifecycle: at first glance it reads like rework. But it is, in fact, a healthy feature of an iterative, responsive process. Many IT departments have sought to manage churn by trying to standardize it out – setting deadlines for new intakes, and arduous change control procedures. As soon as you layer governance on top of a process, you create a need for a shortcut, and an expedited route.

I recently ran a Lean workshop for an IT team during which we identified that 30% of change requests were going through an ’emergency’ process. Clearly the standard process wasn’t working.

DevOps requires standardization on an ‘as needed’ basis. This includes a standard way to handle rapid change, and variation. That standard approach must be light touch governance: make the right way the easiest way. And I’ll stick my neck out and say that standardizing where a worker keeps their snack is a step too far.
September 28, 2015
DevOps – what does “good” look like?
I’ve blogged a lot about my definition of DevOps and what it means to me.

I’ve always thought it much larger than “Dev” and “Ops” and that there isn’t really an end state – it’s an ever changing idea that will adapt to meet the needs of the business.

That said, I’m asked quite often what “good” looks like? How do you know you are on the right path?

For me DevOps is a primarily a cultural idea. If the culture is right then the technology and people will follow. I talk to a lot of people who see this word “DevOps” and reach out straight to technology. DevOps isn’t another word for automating Cloud servers…

In order to an effective “DevOps” organisation I think you need 4 main pillars:
1. Culture – does the culture of your company promote collaboration, self improvement etc?
2. Process – you need to have processes that support different silo’s working together to achieve a fast pace of change
3. People – if you don’t have the right people then it doesn’t matter how great your technology is. People and culture are very linked but I’ve tried to separate them here to demonstrate qualities that individuals hold vs values that the company holds
4. Technology – what are the building blocks you need to be a mature DevOps enterprise?
The following is very much my view of what “utopia” looks like. The truth of the matter is that DevOps means different things to different people depending on what you are trying to achieve. For example, you absolutely don’t need to use public Cloud to achieve DevOps. However, I do think it’s a good pattern to promote working on high value tasks (rather than undifferentiated heavy lifting**) and using a common change process between development and Ops teams.

This is where I want my company to get to…

Culture
- Measure everything, always
- Individuals have empathy for the rest of the team (i.e. they don’t pass the buck)
- Shared goals and incentives
- Don’t reward the fire fighter, reward the fire preventer
- Reward innovation and challenging the status quo
- Don’t punish people when they try something new but fail
- There is no IT and “business”. IT as much “the business” as the sales people.
- Seeking to break down silo’s
- “It’s not my job” doesn’t exist
- “Its my server/code/network/database” doesn’t exist
- Individuals are empowered to make decisions
- Top-up management rather than top-down
People
- Restlessnes (relentlessly looking to improve themselves, others, processes etc)
- Good technical ability across a broad skill set (understand as much of other people’s jobs as possible)
- Everyone can code and use version control
- Everyone understands the test triangle
- Dev-test pairing
- Organised in small product focused teams rather than technology silo’s (align teams to the business not the technology)
- Common incentive schemes
- Favour automation and repeatability above anything else
- CIO/CTO are DevOps biggest guardians & SME’s and seek to destroy anything that affects that culture
- Natural face-to-face influencers rather than endless emailers
- Natural sharers of information
- Take an interest in their specialism outside of work (i.e. go to conferences and take part in the wider community)
Process
- Agile
- Continuous Integration
- Continuous Delivery
- Lean
- Fail-early, fail often
- Release management team are facilitators of change not guardians of change (i.e. they try and aid change rather than stopping/slowing it)
- All change (I mean all) goes through the pipeline from left to right (dev, test, acceptance, production)
- Knowledge sharing and “just enough” documentation is part of the process
- Measuring success and failure is part of the process
- Retrospectives are part of the process
Technology
- TDD/BDD everything (including Puppet etc)
- Everything is in version control (code, automated tests, server config etc)
- Release automation tooling
- Convergent desired state tooling
- Cloud (private or public but the ability to spin servers up on demand)
- The same trending, monitoring & alerting solutions available through nonprod & prod
- Application Performance Management
- Service Virtualisation
- Continuous Integration
- Continuous Unit tests
- Continuous Service level tests
- Continuous GUI tests
- Performance testing
If you can get half way to delivering all of that then I think you should be in a very good place indeed.

It’s a long hard journey if you are a traditional Enterprise. Nothing worth achieving was ever easy though!

I’m outta here for what proved a fantastically successful 2014 (personally and work). 2015 is a massive year too for me but good luck in whatever you have planned ahead.

Happy New Year to you all!

J
xx
January 6, 2015
DevOps, Growth Hacking & other ways to rule the world

DevOps will disrupt management styles, speed the end to ancient IT practices, and flatten away processes that don’t focus on doing more with less. But DevOps is not the only dev-led movement that will impact and reshape the High-Tech industry. So called “growth hacking” will also disrupt organizational charts by bringing development tools, methods and practitioners into roles previously filled by marketing, sales, and customer success pros. As DevOps and growth hacking become more mainstream, the inevitable result is developers ruling the world.

(more…)

June 18, 2014