devopsy.kinsta.cloud

Tag: devops team

  • 5 Time-Saving and Problem-Solving Technologies for DevOps Teams

    5 Time-Saving and Problem-Solving Technologies for DevOps Teams

    DevOps has transformed almost every aspect of software development, taking us from the dark ages of big-bang deployments every 12 months to the reality of releasing software multiple times per day. However, DevOps professionals have to overcome several hurdles to successfully implement and sustain the DevOps culture within the organization. Let’s review some of these challenges—and five technologies that can help.

    Cultural and Organizational Differences

    One persistent challenge for DevOps teams in a mature setting is navigating cultural and organizational differences. These differences often manifest in varied working styles, communication methods and priority management across teams. For instance, development teams might prioritize rapid innovation, while operations teams focus on stability and security, leading to conflicts. Managing these diverging priorities requires constant negotiation and alignment, which can be time-consuming and complex. 

    Furthermore, as organizations grow and incorporate diverse teams, including remote and international members, ensuring cohesive communication and understanding becomes even more critical. This diversity, while beneficial, brings challenges in terms of aligning on common goals and working practices, often leading to delays and misunderstandings that impede efficient workflow.

    Complexity of Automation

    In mature DevOps environments, the complexity of automation can become a significant day-to-day challenge. As the infrastructure and application landscape grows, the automation scripts and tools that were once efficient can become unwieldy and difficult to manage. Keeping these automation processes up-to-date with the constantly evolving technology stack is a never-ending task, requiring significant time and resources. 

    Additionally, the integration of new tools or updates to existing ones often introduces complexity, leading to potential errors and downtime. This complexity not only slows down the deployment process but also requires DevOps teams to invest time continuously in learning and adapting to new tools and methodologies. Ensuring that automation is both robust and flexible enough to handle a dynamic environment is a delicate balance that DevOps teams must constantly maintain.

    Cross-Team Collaboration

    Effective cross-team collaboration is vital for DevOps teams but can be challenging in day-to-day operations. The interdependence between various teams–such as development, operations, QA and security–requires a high level of coordination, often hindered by differing goals, tools and terminologies. Communication barriers can lead to misalignment on project objectives, causing delays and inefficiencies. 

    Moreover, in a fast-paced DevOps environment, keeping everyone on the same page regarding updates, changes and issues requires meticulous attention. This is compounded when teams are distributed geographically, as time zone differences and cultural nuances add another layer of complexity. Regular synchronization meetings, clear communication channels and shared tools are necessary, but managing these effectively without causing meeting fatigue or information overload is a continuous challenge.

    Consistency and Standardization

    Maintaining consistency and standardization across various projects and teams is a significant challenge for mature DevOps teams. As the number of services and microservices increases, ensuring that each adheres to the same set of standards for coding, security and deployment becomes increasingly difficult. Inconsistencies can lead to significant problems in integration, testing, and deployment, slowing down the entire delivery process. 

    Furthermore, as different teams may prefer different tools and practices, creating a standardized environment that accommodates everyone’s needs without compromising on efficiency or quality is a complex task. Regular audits, effective governance and a balance between flexibility and standardization are essential to manage this challenge, but they require constant effort and vigilance.

    5 Lesser-Known Technologies That Can Help

    While there is no one technology that can completely alleviate the above challenges, there are several tools that can help. Let’s review five technologies that can make the life of a DevOps team just a little bit easier.

    Chaos Engineering Platforms

    If you’re part of a DevOps team, you’re probably all too familiar with the panic that ensues when a system fails unexpectedly. Chaos engineering platforms aim to save you from such unforeseen surprises by embracing the philosophy of ‘breaking things on purpose.’

    Chaos engineering involves intentionally introducing failures into systems to understand how they respond and to identify potential weaknesses. By doing so, DevOps teams can proactively find and fix issues before they manifest in production, thereby saving significant time and preventing potential downtime.

    Platforms like Gremlin and Chaos Monkey offer controlled environments for running these chaos experiments, providing teams with valuable insights and recommendations. In the long term, this can not only save time but also instill a sense of confidence and resilience in your systems.

    Application Dependency Mapping

    Application dependency mapping (ADM) is a process that helps DevOps teams visualize and understand their applications and their dependencies. It is a crucial technology that aids in quickly diagnosing and rectifying application performance issues, thereby saving time.

    ADM provides a holistic view of the application architecture, showing how different components interact with each other. This comprehensive visibility is invaluable during troubleshooting as teams can easily pinpoint problematic areas and rectify them without having to sift through mountains of code.

    Furthermore, ADM is crucial when planning for application changes or upgrades. Understanding how changes in one component may affect others helps in preventing potential disruptions, leading to smoother and faster software delivery.

    Policy-as-Code Solutions

    Policy-as-code is a practice where policy definitions are codified and managed as code files within version control systems. This approach offers a host of benefits that can save time for DevOps teams.

    Policy-as-code solutions enable automatic enforcement of policies, eliminating the need for manual checks. This significantly reduces the time taken to ensure compliance and allows teams to focus more on developing and delivering quality software.

    Moreover, since policies are managed just like any other code, they can be versioned, tested and reviewed before being deployed. This allows for a more streamlined and error-free policy management process, saving valuable time and effort. Tools like Open Policy Agent and HashiCorp Sentinel are commonly used to implement policy-as-code in DevOps environments.

    Service Mesh Technologies

    Next on the list is service mesh, a dedicated infrastructure layer for managing service-to-service communication in a microservices architecture. Service mesh technologies offer a range of features that can save time for DevOps teams.

    One of the primary benefits of a service mesh is its ability to offload common communication concerns from individual services to the mesh itself. This includes functionalities like load balancing, service discovery, and traffic management. By doing so, it saves DevOps teams the time and effort required to implement these functionalities within each service.

    Additionally, service mesh technologies provide detailed observability of the service communication, making it easier for teams to monitor, troubleshoot and optimize their applications. Istio and Linkerd are two common open source options for implementing service mesh.

    Codeless Test Automation Tools

    Codeless test automation tools represent a significant shift in the way DevOps teams approach software testing, offering a more efficient and less time-consuming method. These tools allow the creation of automated tests without the need to write extensive code, making test automation accessible to team members who may not have deep coding expertise.

    Using a graphical user interface, codeless test automation tools let users design tests by simply dragging and dropping elements into a workflow. This approach not only accelerates the test creation process but also simplifies the maintenance and updating of tests as applications evolve. Additionally, these tools often integrate seamlessly with various CI/CD pipelines, ensuring that automated tests can be easily incorporated into existing DevOps workflows.

    With the speed at which technology is advancing, it’s crucial for DevOps teams to stay updated on the latest tools and technologies that can help them optimize their processes. The technologies listed in this article can significantly save time for DevOps team and make complex DevOps workflows a little smoother.

  • Addressing Skills Gaps in Your DevOps Team

    Addressing Skills Gaps in Your DevOps Team

    Skills gaps refer to the disparity between the skills that employers need and the skills that employees possess. These gaps can occur at an individual level, in a certain organization or across an entire industry or sector. Either way, they hinder productivity, stifle innovation and lead to missed opportunities.

    Understanding skills gaps is crucial for anyone looking to advance their career or improve their organization’s performance. By identifying skills gaps, organizations can take steps to close them by helping their teams become more proficient and versatile.

    Skills gaps are not always negative. They can also present opportunities for learning and growth. If an organization knows its skills gaps, it can proactively give employees the necessary skills and knowledge. This can result in increased employability and job satisfaction and foster a culture of continuous learning and improvement.

    The Importance of Having a Skilled DevOps Team

    One area where skills gaps can have a significant impact is DevOps. The DevOps model requires a broad range of technical, operational and soft skills, making it particularly susceptible to gaps.

    Having a skilled DevOps team is vital for several reasons. First, it enables faster, more efficient software delivery, which can give your organization a competitive edge. Second, it promotes a culture of collaboration and continuous improvement, leading to higher-quality products and services. Third, it increases job satisfaction among team members who can see the direct impact of their work on the organization’s success.

    Unfortunately, skills gaps in DevOps teams are common and can hinder these benefits. They can delay software delivery, create friction between team members and lead to subpar products. Therefore, identifying and addressing these gaps should be a top priority for any organization using the DevOps model.

    Common Skills Gaps in DevOps Teams

    Technical

    When it comes to DevOps, technical skills gaps are often the most noticeable. These can include a lack of proficiency in coding, system administration, cloud computing and other technical areas.

    Gaps in this area can have a significant impact on a DevOps team’s performance. They can slow down software delivery, lead to more bugs and errors and make it harder to implement new technologies or practices. Moreover, they can create a divide between team members, with some feeling overwhelmed or underqualified while others feel overburdened.

    Operational

    Operational skills gaps in a DevOps team can be just as detrimental, if not more so, than technical skills gaps. These can include a lack of understanding of the software development life cycle, project management or the ability to monitor and troubleshoot systems effectively.

    Gaps in this area can lead to inefficient processes, missed deadlines and poor-quality products. They can also create tension within the team as members may struggle to understand their roles or how their work contributes to the overall goals of the project.

    Soft Skills

    Finally, let’s talk about soft skills gaps. These are often overlooked in the tech industry, but they are crucial for a successful DevOps team. Soft skills include communication, teamwork, problem-solving and emotional intelligence, among others.

    Gaps in this area can lead to miscommunication, misunderstandings and conflict within the team. They can also hinder collaboration and continuous improvement, which are at the heart of the DevOps philosophy. Therefore, addressing these should be an integral part of any DevOps skills gap strategy.

    Identifying Skills Gaps in a DevOps Team

    Define What’s Required

    The first step is to outline what’s necessary for the DevOps team. These could range from specific technical skills such as coding or system administration to soft skills like problem-solving and communication. A clear understanding of the skills required for each role within the team is crucial. This should not be a static list but rather a dynamic one that evolves with the changing needs of the industry and the organization.

    Evaluate Current Skill Levels

    Once the required skills have been defined, the next step is to evaluate the current skill levels of the team members. This can be done through performance reviews, one-on-one meetings or skills assessments. It’s important to be objective in this evaluation and focus on the skills rather than the individual. This process will help identify where the team stands in terms of what is necessary and which areas need improvement.

    Identify the Gaps

    The comparison of the required versus current skills will help identify the gaps. These are the areas where the team’s skills do not meet the requirements. Identifying these gaps is not an indictment of the team’s abilities but rather an opportunity for growth and development. It is crucial to communicate this to the team to maintain morale and motivation.

    Prioritize the Gaps

    Not all skills gaps are created equal. Some may have a more significant impact on the team’s performance and the organization’s goals. Therefore, it’s important to prioritize the gaps. Consider factors such as the impact of the gap on the team’s productivity, the complexity of the skill and the resources required to address the gap.

    Create a Skills Matrix

    A skills matrix is a visual tool that maps out the team’s skills against the requirements. It provides a clear picture of where the skills gaps lie and can help in planning how to address them. A well-designed matrix can also identify hidden talents within the team and highlight areas for cross-training.

    Four Strategies to Address the Skills Gaps in Your DevOps Team

    1. Internal Training and Development Programs

    One of the most effective ways to address skills gaps is through internal training and development programs. These can be tailored to the specific needs of the team and can provide in-depth knowledge and hands-on experience. They also provide an opportunity for team members to learn from each other and strengthen their collaboration skills.

    2. Hiring and Onboarding Skilled Professionals

    While training existing team members is important, sometimes the skill gaps may be too large or the need for the skill too urgent. In such cases, hiring new team members with the required skills can be a viable option. A well-structured onboarding program can help these new members integrate into the team and share their skills.

    3. Leveraging External Training Resources and Certifications

    External training resources and certifications can also be an effective way to address skills gaps. These can provide industry-recognized validation and can often be tailored to the specific needs of the team. In addition, they offer an opportunity for team members to expand their network and learn from industry experts.

    4. Cultivating a Learning Culture within the Team

    Finally, this is not a one-time task but an ongoing process. Cultivating a learning culture within the team can help in this regard. Encourage continuous learning and make resources available for team members to upgrade their skills. Recognize and reward efforts to learn and grow. This will not only help address the current gaps but also prepare the team for future challenges.

    Conclusion

    In conclusion, skills gaps in a DevOps team can be a challenge but also an opportunity. By identifying and addressing these gaps, we can ensure that our team is equipped to deliver high-quality results. As professionals, it’s our responsibility to lead this process and create a culture that values continuous learning and skill development.

  • How to Overcome Resistance to DevOps Implementation

    How to Overcome Resistance to DevOps Implementation

    The number of businesses adopting DevOps to make software releases happen faster without compromising on software quality continues to grow. Within the next five years, the DevOps market is expected to reach almost $13 billion. If you are among those who’ve decided to make a shift to DevOps, you are likely to experience resistance coming from the departments involved in a software development life cycle (SDLC), and even from end users, at the start of your DevOps journey.

    In this article, we will outline the reasons why your employees may resist the DevOps approach and show how to overcome their resistance.

    About DevOps Approach Implementation

    Among the benefits DevOps brings are the fast provision of new infrastructure, quick and reliable delivery of software updates, fewer post-release errors due to the implementation of continuous testing and the alignment of testing and production environments, etc. DevOps implementation requires efforts in establishing tight collaboration between the development, QA and operations teams, as well as adopting new tools and methodologies, such as containerization, IT infrastructure automation, continuous integration and continuous delivery (CI/CD) tools.

    A new approach calls for project managers, developers, infrastructure management, QA and operations teams to leave their comfort zones and get used to new working conditions. However, each stakeholder has reasons to resist DevOps implementation and requires a tailored approach to overcome them.

    The Resistance of Project Managers

    Why They May Resist

    Project managers (PMs) are the first to face the changes when the DevOps approach comes into force. Their resistance originates from a range of new duties, new responsibility and knowledge they need to add to their playbook, such as:

    • Persuading development, QA and operations teams involved in SDLC in the DevOps necessity.
    • Establishing constant collaboration between developers, the QA team and the operations team.
    • Getting familiar with DevOps-specific tools and techniques (CI/CD, continuous monitoring, version control tools, etc.) as well as cloud computing technologies like AWS, Azure, Google Cloud Provider (GCP).

    How to Overcome Resistance

    PMs should become DevOps evangelists as they will be the ones who will contribute the most to overcome the resistance of other parties involved. A CIO should provide a PM with sufficient training on all DevOps-related aspects. Such training should include the management of the development, the QA team and the operations team in the context of close collaboration between them. It should also provide the PM with the understanding of DevOps methodologies and tools–containerization, infrastructure as code (IaC), continuous integration and continuous delivery (CI/CD), cloud technologies, etc.

    The Resistance of Developers

    Why They May Resist

    Developers may resist DevOps adoption due to the following reasons:

    • They are not excited by the necessity to work with infrastructure engineers closer and deal with operational issues if the operations team is not competent enough to address them on their own.
    • They are comfortable with the technology stack they’ve chosen for a particular project and not accustomed to caring about how familiar the other project team members are with these tools and technologies.

    How to Overcome Resistance

    PMs should embrace their soft skills to establish a mutual understanding between developers and infrastructure engineers. They should also make sure infrastructure engineers can work with the tools and technologies developers may use within the DevOps approach, e.g., Docker, Jenkins, by providing them with relevant training.

    The Resistance of a QA Team

    Why They May Resist

    A QA team may feel concerned about a faster SDLC DevOps implementation, since it entails less time for testing. Test engineers now need to acquire new skills prescribed by automation across the entire SDLC. In particular, they face the need to learn new tools for automated testing (Selenium, Appium, fMBT) and API testing (JMeter, Postman).

    How to Overcome Resistance

    To make QA engineers less risk-averse, QA managers in cooperation with PMs, should provide them with appropriate training (tools for automated testing and API testing) and show the advantages DevOps may bring. An opportunity to increase test engineers’ qualification level, less effort on testing activities than required with manual testing are among such benefits.

    The Resistance of an Infrastructure Department Manager

    Why They May Resist

    In the pre-DevOps times, an infrastructure department manager was responsible for controlling the design, configuration and delivery of infrastructure for development, testing and production. With DevOps on board and the IaC approach implemented, the infrastructure manager may feel as if they lose control over the IT infrastructure provisioning and management. What’s more, the scope of the infrastructure manager’s responsibilities extends greatly since they’ll need to run the infrastructure department as a competency center.

    How to Overcome Resistance

    A company’s CIO should explain the infrastructure manager their new responsibilities upfront. The infrastructure manager should stay assured that they change the area of control, not lose it. Within DevOps, the infrastructure department manager has to transform their approach to work. They must be able to help a PM correctly arrange the development, QA and operations team for each particular project taking into account project complexity and size. The infrastructure manager must be able to proactively deal with the cases when the team members involved in software development lack qualification for a project, and not wait until the lack in teams’ competencies results in the project prolongation, for example.

    The Resistance of Operations Engineers

    Why They May Resist

    With DevOps in place, the operations team will have to coordinate their efforts with developers and the QA team from the very beginning of the project, as well as regularly report to the PM on the tasks completed, which is not the usual work approach. To be able to work in the DevOps environment, operations engineers will also have to get familiar with a range of IT automation, version control and other tools. What’s more, the operations staff’s responsibilities will now include the provision of the information on the IT infrastructure functioning to business and technical stakeholders.

    How to Overcome Resistance

    To help an operations team adapt to the DevOps culture, the infrastructure department manager should deliver training to make operations engineers familiar with new tools and technologies they will need to use, e.g., version control systems (Git, Team Foundation Server), software deployment automation tools (Chef, Puppet), etc. A PM, in their turn, will need their soft skills to increase the operations team’s willingness to work in close alignment with development and QA teams.

    The Resistance of End Users

    Why They May Resist

    It may sound surprising, but even the primary beneficiaries of the DevOps approach implementation may resist the changes DevOps brings. There are two reasons for it:

    • The time available for software acceptance testing gets reduced due to a faster SDLC.
    • Acceptance testing becomes automated, and end users hardly trust automated tests.

    How to Overcome Resistance

    To dispel end users’ DevOps-related doubts, PMs must do their best to earn their trust by establishing strong communication with end users. If, for example, end users know software testing covers critical software functionality and the probability of post-release errors is minimized, their confidence in software reliability and the overall trust in the DevOps approach rises.

    In a Nutshell

    Each party involved in software development has its reasons to be suspicious when the DevOps approach is coming on board. New duties, close collaboration, the necessity to apply new tools and other DevOps-related changes may make project managers, developers, QA engineers, IT infrastructure managers, an operations team and even end users resist the new approach.

    To overcome the resistance, a company’s CIO should maintain open communication with each party involved in SDLC, explain their new duties and listen to their concerns. It’s essential to make sure each team member realizes what changes and advantages DevOps brings into SDLC. To ensure they have the necessary knowledge and skills, appropriate training should be provided to project managers, developers, QA team, IT infrastructure manager and the operations team.

    — Andrei Lipnitski

  • How Product Managers Can Take Back Control with Feature Flags

    How Product Managers Can Take Back Control with Feature Flags

    Product managers live in a high-speed world and are always focused on releasing the most impactful features in the shortest amount of time. Since users have grown accustomed to regular UI updates and bug fixes completed in near real time, this frenetic pace at which product managers work will only increase.

    The process of pushing out new features or software fixes can be very stressful on product managers, and the one thing they crave to keep their sanity and ensure a successful release is control. Luckily, that’s exactly what feature flags provide.

    Dialing Up Feature Flags

    At their core, feature flags give development teams the ability to turn features on and off without deploying new code. Think of them as dials in the cloud. When the dial is attached to any feature in a product–whether a new one or an old one–it controls who sees it, who has access to it and how quickly it is delivered to an entire customer base.

    Feature flags let organizations break the tyranny of the Thursday night release by segmenting it into smaller bits to control exposure and enable a continuous release motion. You might start by turning on a feature to only internal users or beta testers so you can test in production to make sure nothing has been missed. From there, you have the freedom to ramp up the feature to your customers gradually before going to 100% availability.

    With this controlled exposure, development teams can lessen the panic and anxiety around the weekly release schedule. Having the ability to completely turn off features means feature flags also increase application stability and remove the need for hotfixes. It’s less often necessary to assemble a war room if there is a problem when you can simply turn off the feature in seconds. By working feature flags into the release process, teams take control of the who, what and when in terms of releases. They can also dictate types of users, demographics, locations and scale the release of the new feature.

    Feature Flags and Product Managers

    Outside of the DevOps team, feature flags can also be really powerful for product managers. Historically, the release of a new feature meant product managers had to coordinate with cross-functional teams to ensure the release was executed successfully. This meant making sure marketing was ready to publish a blog post about the feature, the sales team knew how to use and sell the new feature, and the product marketing team had documentation prepared for how the feature set works. Of course, all of this had to be perfectly synchronized with the engineers’ timeline for releasing the feature.

    With feature flags, product managers now essentially have the keys to the kingdom. It gives them the power to turn the feature on for some or all of their customers without waiting for engineers to complete their next deploy. Not having to coordinate that extra piece of the schedule gives a lot of power and time back to the product manager.

    A Deeper Look at Customer Feedback

    Another aspect of a product manager’s job is collecting feedback. They need to be able to answer questions related to business KPIs, such as “How is this feature performing for my customers?,” “Are customers able to do the action the feature is intended to perform?” and “Are we generating more revenue from the feature?” Feature flags can also help in this area.

    As product managers ramp up a feature, they are simultaneously shortening the feedback cycle. Since feature flags allow for segmentation, where one customer group sees one set of features and another customer group sees a different set of features, and since key metrics can be captured into separate buckets for each group, product managers can now run controlled experiments as part of the release cycle. Seeing KPIs aligned with the different customer groups helps them understand which features are performing the most successfully.

    Creating the Ideal Product Roadmap

    As product managers embrace usage measurement tied to feature rollouts, entire teams can better align with agile best practices to continuously push out the smallest incremental set of new features. By understanding the performance of every feature, you’ll be better equipped to iterate and refine your ideal product roadmap.

    — Dave Karow

  • The Power of a Game: What I Learned from Facilitating The Phoenix Project Simulation

    The Power of a Game: What I Learned from Facilitating The Phoenix Project Simulation

    I recently had the opportunity to become certified to lead a simulation based on the best-selling IT book, “The Phoenix Project.” The Phoenix Project simulation is not a product as much as it is an experience. The realistic scenarios, challenges and pain points embedded in the game built a creative tension in a safe environment and allowed everyone (including me) to discover new insights into applying DevOps to improve flow throughout the value stream—from the business request through IT and to customers.

    DevOps and Agile practices are built on the back of Lean principles that go back decades before the Toyota Production System. As someone who attempted to apply Lean to IT long before DevOps came into existence, I have always said that DevOps is about creating an environment where it is safe to observe, experiment and learn based on outcomes. I was fortunate to meet with Gene Kim many times while he was writing “The Phoenix Project.” We always talked about Lean thinking, Lean systems and Lean tools. Those discussions eventually morphed into The Three Ways—the core principles of DevOps.

    Alignment with the Business Changes the Feel of Work

    The simulation put people in a position where it was easy to see and feel the impact of not being aligned with business and not being coordinated with all the elements of it (AppDev, engineering, IT Ops, change management, etc.). In each round, the team was able to reflect and learn from the experience and experiment with new ways of working. What we all experienced was profound: The DevOps principles, systems and tools are effective only when the team directly experiences the frustration of a broken work system and they work together to solve the problem.

    It was amazing to see how quickly people took on the personas of the functional role they had been assigned. The woman who took on the role of retail operations—you may remember Sarah in “The Phoenix Project”—became rather aggressive as she demanded results. The CISO—John in the book—constantly nagged people about SOX-404 issues. And of course, the CEO was a royal pain to everyone. Although we all knew this was a simulation game, everyone reported the stress and tension they felt as we embarked on round one with no real plan of how we were going to transform into a DevOps team.

    You Can’t Improve What You Can’t See

    Among the chaos of trying to get the work done, it became instantly apparent that the team had no visibility of all the work or priorities. True to life, there was more work that our people could handle so we needed a way to see the work, compare it to our capacity to do the work, identify trade-offs and then work with the business and other stakeholders to set priorities. Once the team was able to accomplish this (through some trial and error) we all could feel the alignment of purpose, see the smiles on people’s faces as we created value and feel what collaboration between silos is really like.

    This discovery was just the tip of the iceberg. As a coach and facilitator, I was able to ask the team questions that positioned them to reflect on what had just happened and see within themselves additional changes that needed to take place to improve the quality, speed and volume of our deployments. Some of the other topics, which were experienced and later expanded on, include:

    • A recipe for coordination among silos.
    • Making time for improvement, learning and technical debt.
    • DevOps without Value Stream Mapping can be hazardous to your health.

    At the end of the day, DevOps is about creating an environment where it is safe to observe, experiment and learn based on outcomes. The Phoenix Project Simulation creates an environment to give everyone that experience. From there, they return to work and begin their journey.

    — Mike Orzen

  • The DevOps Role: Not a One-Person Job

    The DevOps Role: Not a One-Person Job

    This is actually a follow up of the article I wrote a little while ago on LinkedIn titled “DevOps is NOT Jenkins.” In that article, I ranted about how many people and companies confuse automation tools with DevOps. Now I want to address another misconception that I come across quite often: “DevOps is a role.”

    When managers tell me they have all bases covered because their team consists of developers, designers and a DevOps engineer, many questions come to my mind. Having the awareness to at least assign that role in a team is already something. Quite a few engineering managers think that by giving an engineer a DevOps title is enough, but nothing could be farther from the truth. As I mentioned in the LinkedIn article, DevOps is a culture, a process and a philosophy. Adding a DevOps engineer to the team is like declaring someone on the team to be the scrum master and assuming agile is in place. These days, Agile is widely adopted and usually it is not seen as a role, but as a proper methodology. (Although it never surprises me when I encounter a team that is micromanaged while a scrum master is on duty …)

    Like Agile, DevOps should be practiced by the team or organization as a whole. At the core of all engineering we should not only address features but also how those things will run in production. And let’s not forget quality assurance, security and compliance when designing a feature or fix. Fixing an issue at the beginning is between 10 and 50 times cheaper than leaving it until all the testing phase is done already, according to a NASA study. “The study revealed that costs escalate in an exponential fashion.”

    I suspect that concentrating on just the feature or fix is so much more convenient because it is the immediate business value that is addressed by completing it. Thinking about how a new configuration variable will be injected in production or how this new function is going to be load tested probably takes away from focusing on a adequate and speedy completion. There is even a possibility that taking every possible aspect into account simply is too much to fit in one’s brain. I know I’ve had to force myself to not forget anything with checklists and tools.

    In Practice

    Every change, feature or bug fix should have a checklist to make sure all the bases are covered. Naturally, the requirements for the change have to be captured with user stories. But also make sure there is a user story for each of the non-functional requirements. Here are some examples of user stories addressing the various concerns:

    • As a security officer I want to be aware of any changes in potential security risks so I can adapt my (automated) security checks accordingly.
    • As a quality assurance engineer I want to be able to verify the functionality of this feature/fix in isolation and integrated in the product.
    • As an operations engineer I want the to be aware of any changes in the runtime contract so I can update deployment systems accordingly.
    • As a compliance officer I want to reduce to scope for which compliance regulations are in effect.

    There are plenty of tools out there that allow these user stories to be part of a template so it’s easy to embed into the team’s workflow.

    Runtime Contract

    Defining a “contract” for how the application is configured and run in the target environment when the application or microservices are developed removes many uncertainties at the design phase. The mere fact that such a contract exists forces you to either comply with it or negotiate an amendment, like with any contract. A runtime contract has to define how the runtime is configured and which elements are considered secrets. It also needs to describe how and in which format the service outputs runtime feedback around errors and informational messages. Metrics, startup and shutdown and health checks also must be considered. This is an example of what such a contract could look like.

    Broaden the Role

    After checklists and contracts have been established, a large part of the process is codified. I typically write these things down in a version controlled document structure so amendments can be easily tracked. However, this still doesn’t relieve us from the misconception that DevOps is just a role in the team. In fact, the DevOps culture within a team or organization probably should be guarded and or facilitated by a person or persons, much like the Scrum master is tasked with facilitating that process and methodology. Probably, the actual automation will be built in large part by the person tasked with the DevOps responsibility, but on top of that it is also their responsibility to raise and maintain awareness of all the aforementioned aspects of testing, deploying and running an application.

    As such, I believe that people practicing DevOps are required to be more than average communicators and well-versed in every step of the software delivery life cycle (SDLC). Almost all of the DevOps job descriptions that pass my eyes take great care to mention all kinds of technology and familiarity with clouds and programming languages, but don’t emphasize the communicative requirements. It’s time to take DevOps seriously and start reaping real benefits by embracing it as a methodology.

    I’d love to hear your opinions and feedback. Let’s start a discussion below in the comments or on LinkedIn/elexy.

    — Alex Knol

  • Is Your DevOps Team Ready For The Real World?

    Is Your DevOps Team Ready For The Real World?

    Ten years ago I had the biggest cultural shock of my career. It wasn’t witnessing poor collaboration across IT development and operations or whatever tech fiefdom you care to mention. Nope, these were minor issues compared to those facing me on a really interesting project – integrating Industrial control systems with IT management.

    Working with a team of product managers, coders and sysadmins, our task was to develop a standards-based application for monitoring a physical research facility. When I say monitoring, I’m not only talking about network and systems management, but also managing equipment operating in the  ‘real world’ – like manufacturing robots, pressurized compartments, control networks, smart meters, alarms and controllers.

    It was a new experience for the team. Not only were we faced with a new set of technical challenges, we had to quickly become familiar with foreign sounding terms (to us anyway), like programmable logic controllers (PLC’s), supervisory control and data acquisition (SCADA) and human machine interfaces (HMI’s). Add to this a very different set of stakeholders – like industrial and plant engineers, and the initiative started off resembling something closer to a multi-language speed dating session (only without translators) than it did a traditional IT development project.

    At the time projects like this were a rarity because the worlds of operational technology and traditional IT was seen as separate. Process control and electrical transmission systems ran on separate control networks with proprietary protocols, whereas IT ran on a business network and primarily handed back-office business systems – the two domains were as different as ‘chalk and cheese’.

    But all that’s changing courtesy of the Internet of Things and a software-driven economy.

    Today, organizations are realizing that standards, unified architectures and software applications can be used to connect and integrate the devices and sensors in the physical world with IT systems – the benefit – more seamless end-to-end business processing.

    Take for example manufacturing. If plant production equipment can be better integrated with consumer demand analytics, then optimum maintenance windows can be identified. Similarly in energy and utilities, integrating the physical world of smart meters with back-end IT customer information applications is a must for efficient billing.

    This convergence will drive new approaches to developing and operating applications. The DevOps movement with its focus on teamwork and collaboration to shorten delivery cycles while improving quality might just be the answer, but there are some serious challenges to overcome – not least cultural.

    You might think you’ve solved the great IT divide with DevOps, but that’s nothing compared to the cultural chasm you’ll need to bridge across the world of industrial operations and IT. And no amount of IT ego will help, which I learnt the hard way when I was once given a sharp lesson in what ‘real-time’ processing actually is business by a group of chemical engineers – boy was I humbled.

    Technology beat up aside I believe there are many ways DevOps can help address the challenges associated with connecting and integrating operational technologies – especially:

    • Standards guidance – since operational systems work in isolation and use proprietary systems and communication protocols designed for specific control system networks (e.g. Modbus and BACnet), DevOps teams can help provide advice on new standards and integration platforms.
    • Knowledge and Expertise – DevOps teams can provide expertise in the selection of new software platforms that are now available to build and run Internet of Things applications.
    • Security and Risk Management – as more and more software finds its way into sensors and operational control networks become connected to the Internet, DevOps and security specialists will have a key role in identifying and mitigating many new security risks.
    • Data Management and Analytics– connected sensors and control systems may result in the need to capture and analyse huge quantities of data. DevOps teams can help in areas such as application architecture and design, testing, network capacity and cloud automation.

    As operational and information technologies converge, I believe the purview of DevOps will increase considerably.  Now teams might include technology constituents from other areas of the business – like engineering and logistics specialists – each bringing new skills to the table that when coordinated with IT expertise in areas like mobility and cloud computing has the potential to drive huge efficiencies.

    Application management will change also. Development work will increase, but so too will need to bake quality and resilience into every release. This is especially important because as processes become more integrated (from the factory floor sensor network to a mobile enabled materials management system or from healthcare diagnostic equipment to patient records applications) any minor defect, security issue or performance problem can trigger more catastrophic business failures.

    When I look back to my own teams foray into managing ‘real world’ technology, I realize that while we never had DevOps to guide us we still used many principles. Success came when we built a cross-domain business team that maximized a unique set of technical strengths, but also understood each other’s weaknesses.

    So DevOps teams take note – as technology convergence accelerates, the goal for many industries won’t be just bridging the IT development and operations divide and releasing more and more software, it’ll also be about developing a new range of end-to-end digitized business processes – from the factory floor to the boardroom – that’s a bigger deal and where success will ultimately be measured.