devopsy.kinsta.cloud

Category: DevOps Culture

DevOps Needs Collaboration and a Safe Place for Success

When it comes to DevOps adoption, “[many organizations] tell a similar story,” said Scott Van Kalken, systems engineer at F5 Networks. It takes more time and effort to develop the right culture than you probably expect.

Van Kalken has a long history in tech, including development and operations roles, and has spent several years practicing DevOps. He is part of the open source and meetup communities, and organizes some of the largest DevOps events in Australia.

“DevOps typically starts in a corner of the organization, where people want to deliver high-quality software really quickly,” said Van Kalken.

When that team gets good results, the organization decides to scale up its DevOps efforts, but tradition and silos get in the way.

“At least two things are needed to overcome those obstacles and achieve DevOps success,” suggested Van Kalken.

Collaboration and Safe Places

The first thing you need to achieve DevOps success is a collaborative team environment. One of the central ideas of DevOps is you don’t throw software over the wall between functions, such as development and testing, so there needs to be an understanding and acceptance that everyone in the team is on the same side.

Second is the provision of a safe place to fail and the realization that failure is just an iterative step on the path to implementing the best ideas.

A financial services company that Van Kalken works with was adopting GitOps but needed to convince the service management team this was the right move.

“Initially, it was quite challenging because service management thought it [GitOps] was uncontrolled chaos, even though that isn’t the case,” he said.

Its support was gained after running a series of workshops that showed the underlying processes were actually the same, even though the tools used to implement them were changing.

Service management is still the final gatekeeper, but it now approves releases in the repository rather than on paper.

Maturity

“There is a maturity in accepting change does involve risk,” said Van Kalken.

Things such as GitOps provide a quick and easy way of reverting to a previous release if something goes wrong. So the “safe place to fail” idea can be implemented in a way that protects the organization in a practical sense as well as staff members in a psychological (and career protecting) sense.

Another way DevOps can protect the organization is that frequent releases generally involve relatively minor changes, which in turn have a smaller blast radius in the event something goes wrong. Again, this is about having the maturity to recognize risk and dealing with it, rather than trying to avoid it in the first place.

A sometimes overlooked aspect of increasing the release cadence is the effect on users, who are being asked to cope with frequent changes.

Van Kalken pointed out that increasing the frequency does not necessarily mean multiple releases a day–it could be from two releases a year to three or four, if that’s what suits the organization.

Not all changes have a direct impact on users. Some are under the hood, improving performance or addressing rarely-encountered bugs. But when users’ interaction with the system is changed, he suggested canary deployments as a way of checking the acceptability of the new approach among a larger pool of users than those brought into the development process, before it is released to the entire user population.

“This approach also has a place in DevSecOps as another way of limiting the blast radius,” he said.

Accepting Failure

Perhaps the biggest challenge to an organization’s culture is the adoption of chaos engineering because if you’re going to kill a container or flood a network with data in order to check that the wider system can cope, you absolutely need to be in a safe place to fail.

You also need to realize everyone involved–including developers, security people and those involved in the business side–wants to achieve high availability, and that means designing systems that can handle (partial) failures.

It isn’t enough to do this in pre-deployment testing. Ongoing testing provides confidence that production systems will continue to cope with such failures.

“It’s pretty cool when you get it right,” said Van Kalken. However, “your culture has to be OK with this iterative approach.”

“DevOps is really just a methodology to achieve a better outcome,” stated Van Kalken. That involves collaboration, iteration and embedding security and other considerations up front. But the wider organization needs to understand and tolerate this, and the risk that goes with it.

“Education, awareness and experience can all contribute to dispelling the traditional view that risk and failure are inherently bad,” he suggested.

Are the Metrics Appropriate?

A related point is that whether one outcome is better than another, depends on the metrics adopted. “[Metrics are] probably the biggest things organizations need to change,” said Van Kalken.

For example, a customer who ordered a pizza doesn’t care whether a particular system involved in taking the order, cooking the pizza and then delivering it, achieved a certain uptime target. They just want the pizza to arrive promptly.

When that doesn’t happen, he believes a collaborative approach is needed to determine why the delivery was late, exactly what caused the problem and how to quickly roll out a fix to stop it recurring.

When it comes to practical advice, Van Kalken emphasized the need to get people together early in a project, otherwise there is a risk that sub-groups will demonize each other. In addition, you get more buy-in and everyone accepts they have to help with the pedaling, not just sit back and be passengers.

“If people aren’t talking, you have a bad culture,” he said. But he acknowledged it can be difficult to get past existing adversarial relationships. Breaking the ice between teams that haven’t previously engaged with each other can also be tricky.

Returning to an earlier theme, he stressed the importance of adopting an analytical, rather than punitive, approach when failure occurs. Everyone involved must know the organization accepts risk and understands the consequences.

— Stephen Withers

October 1, 2019
Common Pitfalls of Software Development and Tips to Avoid Them

In business, no one wants to plan on failure. However, when it comes to software development, failure is more common than most managers want to admit. In particular, many managers and executives fear large software-based projects are especially likely to fail due to the complex nature of these projects and the nuanced reality of the deliverables.

While there is no doubt a software development plan is complex, there is no reason to be so readily consigned to failure. In fact, it might be argued that software is so important to your operations that failure should be viewed as unacceptable. However, this aggressive approach in theory needs to be backed up with proper execution in practice. That is why you need to know the common pitfalls that befall most software projects. This is the only way to make sure your company avoids them.

Vague Project Requirements

One reason so many projects fall short of the mark is that these projects fail to define the mark adequately. Your software development project cannot succeed until you know what you really want it to do. This means your project specifications need to be clear from the very start. Your goals should be defined first, and then these goals should be consistently reiterated to team members throughout the process. This includes the scalability of the project. The development team needs to know what features should be included in order to produce a final software solution that actually meets your company’s needs.

This is all part of developing a versatile software development plan. A good software development plan will anchor your project from start to finish. A poor one will inevitably leave you vulnerable to failure.

Unrealistic Timeline

The timeline matters to the company, which is why some companies set aggressive timelines as part of their software development plan. However, while these timelines sound good in planning sessions, they may not make much sense in the daily development process. It is pivotal to think carefully about a timeline in order to account for development, testing and more. All parties should be consulted in the development of the timeline. This includes high level executives, project management and developers. This varied input should allow the project to accommodate the needs of management alongside the needs of developers. This is a form of risk management because an insufficient timeline will lead developers to cut corners, leading to a lackluster result.

Unbalanced Staffing Commitments

As you define your software requirements, you will need to think about the team members who will bring your plan to fruition. If you do not assign enough people to the project, production will fall behind. However, if you put too many people on the project, things can start to get messy. Overstaffing a project can easily lead to miscommunication, and it is harder to keep a large team of people unified. This can also lead to redundancy and a lack of consistency if too many duties are spread out. For these reasons, it is important to assess your staffing situation correctly from the start. Be sure to include your project management team in the final staffing decision.

Insufficient Testing Procedures

In all your planning, it can be easy to neglect testing. The reality is testing should be a critical part of your plan, accounting for a significant amount of your timeline and resources. Testing is important because it allows the development team to iron out any problems with the software. All software will have bugs in its early versions, and these bugs can be caught and fixed with sufficient testing. Testing is the only way to make sure your software works the way it needs to work. If you implement deliverables that do not perform, then your workers will be flummoxed, your consumers will be disappointed and your security could be at risk. Therefore, allot plenty of time for testing throughout the entire development plan, allowing for each component to be tested prior to going live.

Poor Hands-On Leadership

If you really want to make sure your project goes well, then make sure you have good people in place to lead it. Your project management team is absolutely critical to successful software development. A good project manager will internalize all parts of your plan. This includes the budget, the timeline, the goals and the staff. Your project managers need to facilitate communication, manage schedules and oversee adjustments. This key leadership role is your software development plan in action. Make sure you hire the person with the right skills and the right experience. Then, make sure that person knows exactly what your goals are to achieve success.

These pitfalls are common for a reason. However, with a little foresight, it is possible to keep your project and your company from disaster.

— Shane Zilinskas

September 30, 2019
Chaos Engineering for ITOps

Chaos engineering (CE) is the discipline of experimenting on a system in order to build confidence in the system’s capability to withstand turbulent conditions in production. This approach is becoming commonplace in software development and operations (DevOps) practices. But how would its application extend to ITOps? CE for ITOps offers a similar framework for stress-testing a technology platform to understand its weak points and performance pitfalls under heavy pressure.

CE tends to be used primarily in DevOps during bug testing: setting up experiments to run software under different conditions, such as peak traffic, and monitoring how it functions and performs. This becomes increasingly necessary in cloud-based systems where failure to understand extreme load responses could result in runaway cascade failures or, worse yet, spinning up thousands of extra nodes handling error conditions while not doing any actual work. These same principles, applied to IT operations management (ITOM), help define a functional baseline plus tolerances for infrastructure, policies and processes by clarifying both steady-state and chaotic outputs when extremes are reached.

Applications in IT

The theory of CE in DevOps gained early traction at Netflix as they moved from physical to virtual infrastructure, with the team that implemented it on AWS breaking off to form Gremlin. However, CE is not typically used in ITOps, because ITOM has historically been separated from development (generally, IT monitors system dynamics, and when a problem occurs, engineering change management or ITSM is brought in to remediate the issue).

With the growth of containerization in cloud applications today, IT infrastructure looks more like development environments than classical multi-tier architectures. But the limitless scale of the cloud means failures can also be limitless: Microservices are well-served by testing elasticity and scalability, data flows and resiliency through stressing the system to the edge of its tolerances and fixing their shortcomings before a public crash.

One, Two, Three…Chaos

Implementing chaos engineering for ITOM provides a systematic approach to identifying weaknesses in a microservices-world. In a monolithic environment, you have visibility into performance and event metrics that may be lost with microservices designs. As a result, the need for operational insights becomes even more critical when scaling to unknown workloads.

Netflix’s Chaos Monkey grew out of CE principles from their own cloud-native community, meant to address the gaps in common dev tools’ abilities to manage extreme complexities. This methodology is extendable to infrastructure and helps to set guardrails on platform behavior as a whole.

Here are five fundamental steps to follow in order to bring this thinking into your team’s ITOM.

Define the Current Steady State

Performing baseline analysis is a standard concept in capacity planning, upgrade strategies and other high-impact functions. Start with something relatively simple (and small) so you don’t get overwhelmed by the data, or risk interfering with the business if something goes wrong (such as security Red Teaming). For example, monitoring CPU and network utilization, which are common bottlenecks in any IT shop.

Define Optimal Conditions

There’s how your system generally operates, and then there’s how it should operate; these typically aren’t the same thing. CPU utilization and network latency are always affected by application efficiencies, hardware conditions and a host of other factors. Create a standard that outlines what engineers should expect on a normal day, on an easy day and on a very hard day. These are the control groups, and the extreme day will be the stress test.

Form a Hypothesis

Where will the system break? If you’re running an application scenario such as doubling the peak traffic that even your worst day so far has seen, will your CPU maintain optimum utilization (or will the container provisioning engine smoothly deploy additional nodes) as in the variable control groups, or will it spike so severely that processes grind to a halt because there isn’t enough memory or network bandwidth left to manage the load?

Execute a Real-World Event (But Contain the Blast Radius)

Do something extreme, such as taking down a firewall that severs connectivity to one internet service provider. This will confuse the application as it tries to respond to requests with repeated failures, ramping up CPU processes as errors return from a dead network endpoint. Log events will mount, filling the database and saturating the backbone.

Validate the Hypothesis

What happened? Monitor utilization and network throughput during the test and see where the system fell over. Is it what you expected, or did something never previously considered take place? Did new chaos erupt from the fissures in your infrastructure? Stabilize, document and remediate.

Never Stop Not Being Afraid

Stressing a system to its absolute max—and a little bit further—to see where things go wrong allows you to understand steady-state behavior and error-handling, so you can fix it before something breaks in new and unexpected ways. What do traffic spikes look like? What are real-world events and their impacts on your organization?

CE is not just for DevOps. It should be a systemic practice for load-testing (out of your comfort zone) to the point of failure. It’s a responsibility for more than microservices deployments and applies to all sorts of disciplines within the IT organization.

— Michael Fisher

September 30, 2019
The Secret to Digital Transformation is Human Connection

We know digital transformation is no longer an option for today’s organizations–it’s a requirement. End-users have come to expect an easy, digital-native experience, whether they are ordering a pizza or automating an entire network infrastructure, which is why businesses and governments are throwing trillions of dollars at new technologies and related services.

Organizations must build and deliver exciting new software applications, faster than ever before. They must continuously innovate, adapt and stay ahead of the never-ending demands of digital data, experiences and channels. These demands have contributed to the meteoric rise of cloud computing, DevOps methodologies, open source technologies, predictive analytics, AI and many other innovations.

However, even with all these investments, more than half of all new software or IT projects still fail. In fact, according to a study by Forbes Insights, 75% of executives say they’re still waiting to reap tangible benefits from disruptive technology. Here’s why: Technology alone does not solve digital transformation. It doesn’t address some of the most fundamental issues that inhibit or enable transformational success nor the most critical dependency, which is people.

To truly transform, organizations must go beyond just technology to embrace the human elements of this evolution. We’ll look at six factors driven by the human side of the business, and how each of these can help, not hinder, digital transformation when properly addressed.

Organizational Culture

One of the biggest challenges for companies as they embark on digital transformation is the needed cultural change. The technology decisions, while daunting, are easier compared to the cultural shift organizations must undertake to truly become digital leaders. Top down mandates won’t work, because for it to be successful, digital transformation needs buy-in from all involved. People need to connect to the change and feel they can have an impact, or at the very least, know their job is safe amid the change. Also, to achieve the speed of change and innovation, organizations need to empower everyone to make decisions, not just a select few.

Embracing an open culture can help with this. In an open culture, digital leaders look at their organization more like a community of individuals, all contributing to something bigger than themselves. Anyone can participate, lead and suggest a change or improvement. This methodology helps for the best ideas to win out, regardless of where they come from. Employees feel part of the process and ultimately, it empowers them to work toward the shared goal. Let me be clear, this isn’t a democracy or a lack of clear decision making power. However, it is using more open decision frameworks that move everyone toward a common goal, which brings us to the next big human factor.

An Inspiring Vision with Metrics of Success

It is often said to start with the end in mind. With digital transformation, this is absolutely true. For digital transformation to be successful, everyone in the organization needs to understand, and ideally agree with, the destination. Visions are not just words—they are driving the priorities, investments and decisions made by every team and employee. Without a clear vision, organizations can’t create an executable business strategy. The key is to make sure you have a vision that clearly describes what your organization will be like in several years–and then make that vision measurable.

All good visions come with goals attached to them. Yet according to research from Gartner, 50% of organizations have no digital transformation metric or way to measure progress for digital transformation. Set up clear, ambitious targets–stretch goals the entire company can work toward. These metrics allow organizations to clearly check on how they have progressed and where they still need to get to.

Cross-Organizational Teams and Processes

Another cultural barrier to digital transformation is siloed behavior and workstreams. Often, teams refuse, or just don’t know, how to break out of their silos. Digital transformation requires collaboration and breaking down boundaries, as data, insights, applications and processes need to flow seamlessly throughout an organization. It all goes back to having a vision and an open culture. Without a culture of trust and a shared agreement on the end goal from across the organization, you won’t be able to break down these silos.

With the vision and culture in place, adopting DevOps and agile processes is a great way to initiate this. It helps bring together business, technical and other people from across an organization to solve a specific problem or figure out how to better work together. Often, this takes a courageous leader, who doesn’t measure their value by the power or budget they hold, to broker these cross-organizational relationships and new teams.

People Development

In talking to IT and business leaders all over the world, they are faced with a common challenge: skill set gaps. This includes both soft skills, such as communications or management, and hard skills, such as technical capabilities. According to Red Hat’s survey on Global IT Trends & Priorities Research, the biggest blocker to an organization’s digital transformation efforts is people. More specifically, the survey found IT talent or skills gap was preventing organizations’ efforts from being successful. Technology and digital processes are changing so quickly, it’s hard to keep up. This is creating a digital skills gap in organizations across industries and around the world.

Rather than hoping to hire people with the skills and talent you need, I encourage organizations to develop the people you have. Take your loyal employees who already fit your culture and believe in your vision and invest in them. Also reward them for taking risks. Digital disruption does not happen by standing still or encouraging small moves–it takes big bets. Leadership must also be willing to stand up for employees who try new ideas and push the boundaries.

User Experience Focus

According to CareerFoundry, which manages a Web UX design school, the cost of a bad user experience is high. Really high. Its research estimates bad user experience costs e-commerce companies $1.4 trillion in lost revenue.

As we focus on the human connection part of digital transformation, user experience takes the front seat and is a top priority. We need to start with the user. What are they trying to do? What is the pain we are hopefully solving? Too many times organizations start with the cool technology and slap on the user experience as almost an afterthought. By starting with the user, projects are more likely to give users the feeling they are interfacing with a human being rather than a digital thing—and ultimately, will succeed because of it.

Personal Connections

One of the downsides of the digital era is we don’t have enough face time. We believe video conferences provide the same benefits as face-to-face meetings. We put our trust in digital handshakes instead of the real thing. However, to succeed at a company-wide digital transformation, you can’t rely on digital communication alone. Ironic, right?

One key reason for making face time is to build trust. As humans, we need to see someone, look them in the eye, shake their hand, respond to their cues. All of this helps us trust the other person. To put your organization through a major transformation takes trust–across the organization, but especially of the leadership. This means leadership needs to get out there and talk to employees, customers, partners and users. Take the time and the effort to make your transformational efforts relate to the humans doing the work and buying your products or services.

The Bottom Line

The most common mistake organizations make is believing technology alone can solve their problems. Technology is still needed for a successful digital transformation. In fact, I am not in any way advising we all sit around and have a lovely conversation and hold hands to solve these massive technology issues. However, technology must be combined with human connection.

In other words, it’s not just what you use to transform, but the how and the why of your transformation. The human elements require hard work across culture, hiring, people development, leadership and communication.

Human connection is the killer application for digital transformation.

— Margaret Dawson

September 26, 2019
Top 3 Challenges of DevOps Adoption

Customer expectations continue to relentlessly rise as new technological opportunities emerge. Today, organizations demand products and application features are delivered and implemented wherever and whenever needed. Everywhere you look, execution is at risk of falling behind imaginations and business imperatives. That is, of course, unless DevOps is starting to wield a strategic and philosophic influence.

DevOps has been one of the hottest enterprise trends for some time now, and a crucial concept in helping companies satisfy customer demands by bringing teams together to fuel better collaboration and innovation.

According to Deloitte, organizations adopting DevOps see an 18% to 21% reduction in time to market.

By removing the waste that impedes software development, operations are streamlined, and businesses can react faster to market demands. At the same time, automation enables companies to align business objectives with processes to immediately recover from IT failures. Deloitte also reports the speedy, DevOps-influenced, customer-friendly release of new products and services led to a 20% revenue increase.

While that is all good, DevOps is still a new topic to many. Large scale adoption is generally slow. It is important to note DevOps is not simply the implementation of new tools. It is about rebooting deep-rooted cultural habits, transforming old processes and ensuring all changes are made for the right reasons. Don’t embrace DevOps for the sake of it. Only proceed if you have clarity and purpose.

Breaking of the Deep-Rooted, Cultural Habits

DevOps is not a team. It is a culture of collaboration that yields a set of best practices and operational cultures. Sometimes companies wanting to adopt a DevOps approach are met with resistance from employees uncomfortable with change. Leadership needs to come from the top. Executives must support employees in dismantling the status quo and breaking down silos hindering successful implementation. Motivation rises when results are positive, concrete and observable. There is no magic formula but, if you get the right leadership and culture in place, chances are DevOps will land with resonance. Gartner reports 88% of businesses believe team culture is one of the top impacts on an organization’s ability to scale DevOps.

Changing Well Defined Processes to More Efficient Ones

The best way to identify inefficiencies is to map processes and recognize what is and isn’t working. Start by looking for waste–areas where resources are being exhausted without delivering real value. A silo-influenced lack of communication can significantly slow down businesses and there is a risk each department will think DevOps is not their responsibility. Some may be resistant to implementing change, which will cause unnecessary lags within the business. It is vital to understand it is not just the responsibility of software developers to ensure the processes work for them. DevOps is a process that brings development teams and other IT stakeholders together to achieve one common goal: delivering faster, higher quality work to market.

Beware of DevOps for DevOps Sake

With DevOps, companies can streamline and automate every task. That doesn’t mean it’s right for every element of every business. Often, companies will pick a couple of departments to trial a DevOps approach with–usually the development team. The temptation is then to continue scaling across the organization as soon as positive results are apparent. The issue here is that different departments all have different needs and challenges and, as we’ve already said, effective DevOps rollout requires an overhaul of processes and ways of working. Organizations need to be careful and think about specific business needs, instead of just being engaged in a rudderless trend chase. Know your objectives before engaging in sweeping overhauls. Keep learning and, above all, stay tuned to customer needs.

— Rodrigo Albuquerque

September 26, 2019
Implementing DevSecOps Goes Beyond Technology

While technology is crucial for implementing DevSecOps, it is the people, processes and culture that drive it forward. As recently as last year, a survey found 58% of technology leaders cited existing culture and lack of skills as hurdles to being able to embed security testing and evaluation within software development processes. That report found that only about one in four companies strongly agreed their organization’s culture and practices supported collaboration across development, operations and security.

A separate recent survey on DevSecOps skills, conducted in partnership with devopsy.kinsta.cloud, found 70% of respondents said the security education they received is not adequate for what their current positions require.

Plenty of similar statistics can be found, and they point to a recurring theme–education, training and support for developers and security teams are as critical as modern application security solutions in implementing DevSecOps.

Automation Can Help Overcome Challenges

Most developers want to create high quality, secure code. Often, the problem is developers have never had formal training in what secure coding is, in either traditional education or while on the job. The good news is more organizations are starting to provide that training for development teams. As development and release cycles speed up, developers need to find ways to incorporate security testing in less intrusive ways.

In addition, development teams are gaining some control over budgets directed toward application security tools. Legacy tools for testing code and assessing application security risk simply weren’t built for the speed DevOps testing requires. Whenever evaluating new security analysis tools for a development team, automation should be top of mind. Static analysis tests the application code for a broad range of security flaws, and it can be fully automated into both the IDE and continuous integration (CI) processes. In the IDE, it provides early security guidance and education to software engineers while they are coding by highlighting potential vulnerabilities and suggesting best practices.

Companies can also reduce friction by ensuring developers don’t have to go out of their way to run the security tool. Development teams should be able to view and triage the results and take advantage of plugins and/or APIs to integrate security tools seamlessly into toolchains they are already using. This means there will likely be no one-size-fits-all approach that can be dropped into any dev team; rather, take the time to understand how that team works currently, and work with them to introduce the security tools in the least disruptive way.

Shifting Mentalities

One of the biggest challenges is asking developers to do something they weren’t doing before. With DevSecOps, it may seem like dev teams are being asked to take on more work, but in reality, the organization is replacing unplanned work with planned work. Rather than address defects late in the release cycle, development teams are scanning for and addressing flaws early on. Such a cultural shift can be plodding, but developers and development managers should realize embedding security this way will be more efficient and more secure while also reducing their unplanned work.

Giving development teams more autonomy over security tasks can also be daunting for security practitioners, but in order to facilitate shorter development iterations, security has to relinquish some control and also reconsider how they think about risk tolerance within the context of the business. Historically, the relationship between developers and security practitioners has been adversarial, but DevOps and DevSecOps methodologies have made it more cooperative. Still, many security teams are uncomfortable with the thought of giving up control.

Adapting to Avoid Adding New Technical Debt

Even with the understanding that producing more secure code benefits both development and security, reducing technical debt is gradual and starts with integrating security directly into the development process. It may seem convenient to lump security findings in with technical debt with the intention of getting to it someday, but it’s not practical to let security debt pile up.

Organizations are decentralizing security work, particularly via security champions: One or two people on each development team who serve as the eyes and ears of product security, taking ownership of some of the day-to-day tasks that would have previously been bottlenecked on the security team. A security champion could help with security code reviews or coach others on their team on how to fix security issues. Sharing responsibility for security in this way can further bridge the gap between security and development teams, reducing the workload of the security team while allowing development teams to be more self-sufficient.

For developers, context switching has a productivity cost. If they find out about a security defect while they are actively working on that particular feature or section of code, it’s only minimally disruptive and can be fixed quickly. By contrast, if they discover the same security defect two weeks later, after they’ve moved on to a completely different feature, they’ll incur the cost of context switching back to the earlier feature before they can fix the flaw correctly. Over the long run, finding and fixing flaws early in the SDLC should not meaningfully delay time to market.

Here are some best practices in implementing DevSecOps in your organization.

1. Embed Security Processes Early in the Development Cycle

Ensure scans are baked in at every stage of the software development process and work with your development teams to integrate security within existing workflows and tools. By the time software is pushed to production, code should have been scanned multiple times and all vulnerabilities should be fixed. Research has shown the frequency of security scans during the development process correlates to fix velocity. Specifically, businesses that do daily scans fix vulnerabilities 11.5 times faster than the typical organization.

2. Give Security Responsibility Directly to Development Teams

Developers need to take ownership of the security of their applications. This is the only way to integrate security seamlessly into their existing workflows. The perception of someone coming in and forcing tweaks to their code can be discouraging, but that isn’t to say security teams should be hands off. They need to take an advisory role, working with developers to ensure secure code. Training is key here. It’s likely developers will not be acquainted with the latest security strategies. Security experts need to educate developers on known vulnerabilities, threats and tools.

3. Identify a Security Champion on Every Development Team

It’s equally important to find someone on the development team who can go beyond automated testing tools to serve as the eyes and ears of the security team. Rather than assign the role to a member who already has additional responsibilities, choose someone who shows an interest in security. Because engineers may switch teams often, it’s best to have at least two security champions on each dev team. If one moves on, the institutional knowledge remains on the team and a new security champion can be trained up.

4. Automate Security Scans

Security cannot be an inhibitor to development. If it is, developers will fail to implement the proper security policies. With automated tools in place, development teams can efficiently test code throughout the software development lifecycle (SDLC), ensuring more secure applications while delivering software on time.

5. Take a Data-Driven Approach to Tracking Progress

Showing progress is crucial, both for developers who want to see the impact security is having on their software and for the management team who needs clear insight into application security’s return on investment. Analytics with compelling visualizations supports this value by providing quick and intuitive data so organizations can assess their current state and take action to improve their risk posture. Sophisticated analytics allows organizations to target critical applications and fix critical findings by defining a custom security policy based on their specific risk tolerance.

— Chris Eng

September 25, 2019
DOES19 London: Effective Team Management Foments New Ways of Working

In the IT space it feels like we collectively begin asking and ruminating on the same question all at once, chewing on all the possible answers and outcomes. Recently, one such question has begun popping up with more and more frequency: “How can we help teams, and especially product teams, be more effective?”

While more research is coming out to help guide teams through new and iterative ways of working, moving on from static and traditional models is far from the norm. So, it may come as no surprise that insights into different ways of structuring product teams would be a prominent topic at Gene Kim’s DevOps Enterprise Summit 2019 (DOES19) in London.

There, I sat down with Manuel Pais, an independent IT consultant focused on team interactions and the co-author of a new IT Revolution Publishing book, “Team Topologies.”

In our interview, Manuel shares some takeaways from his book as well as guidance on the importance of managing team “cognitive load” and how organizations can begin rethinking their approach to software delivery when it comes to cross-functional teams.

In addition to our interview on the Digital Anarchist set, I am also happy to include a video of Manuel’s presentation alongside co-author, Matthew Skelton.

If you’re interested in hearing from other IT and DevOps leaders, be sure to watch the other 30 video interviews from the conference up on the Digital Anarchist site.

Don’t forget! DevOps Enterprise Summit Las Vegas 2019 is happening Oct. 28-30 at The Cosmopolitan Hotel. Be sure to register here if you haven’t already and join us for the premier event for technology leaders of large, complex organizations implementing DevOps principles and practices. You can also preview the event by watching presentations from past DevOps Enterprise Summits.

— Alan Shimel

September 24, 2019
Upping IT Skills: IT Pros Cite Significant Challenges

Anyone who works within enterprise technology learns very quickly that if they want to stay relevant to the workforce, they’ll need to continuously learn new skills. But it’s certainly hard keeping up with new skill training while also keeping pace with day-to-day work demands.

This is probably why a survey earlier this year from IT management software provider SolarWinds found that 70% of all technology professionals surveyed do not feel completely confident that they have all the necessary skills they’ll need to successfully manage their IT environments over the next three to five years.

A certain amount of fear that one’s current skillsets may become antiqued by technological change is worthwhile, but that apprehension is only useful if technology professionals turn that energy into action. According to a more recent SolarWinds survey, 60% of technology professionals haven’t actively pursued learning a new skill or completed a certification in the last six months.

The survey, Building Confidence for Tech Pros of Tomorrow, is based on an international survey of 177 technology professionals within both the public and private sectors. It was fielded in August 2019.

The survey found roughly 50% of the technology professionals who did start a certification process did not finish. The primary reason was lack of time.

Enterprises, aware that there exist challenges when it comes to finding the staff with the skills they need, are making efforts to mitigate their staffing challenges. These efforts predominately involve internal trainings, with 35% of respondents citing such training. Additionally, 54% of respondents said full-day in-person workshops provide the most value for the time spent.

The survey also found soft skills are becoming increasingly important, with 46% of respondents stating interpersonal communications skills are the most critical for career growth. Other top skills were project management (56%) and public speaking/presentations (52%).

Interesting, the majority of technology professionals surveyed said, while they are confident speaking with business leaders, they don’t believe they are adequately equipped with the support and tools they need to be effective at their work. According to the survey, the top two requirements for technology professionals to become confident managing current and future environments include increased budget/additional resources (64%) and more support from IT or business leadership (48%).

Finally, and which could be a sign of the increased reliance of enterprises on software, the survey found application performance management (APM) skills are increasingly necessary for technology professionals. Survey respondents cited application support, troubleshooting and monitoring as tasks they are increasingly responsible. And 53% said user experience monitoring of applications is an additional skill of theirs that they are required to develop.

— George V. Hulme

September 23, 2019
Online Experimentation: With Great Power Comes Great Responsibility

Recently we had the honor of giving a talk at this year’s Turing Festival. The festival, which is held in Edinburgh, is one of Europe’s biggest cross-functional tech conferences and is named after the late Alan Turing, who invented the Turing Machine. This invention cracked the Nazi Germany Enigma code, helped to end World War II and is the conceptual foundation of the modern-day computer.

So much of the technological advancements we benefit from today are a result of Alan Turing’s amazing achievement. But with the great power that Alan left us comes great responsibility. The responsibility we decided to speak on at the festival is ensuring you are not being misled by data.

Engineering teams are delivering faster than ever before. In the 1990s, a new feature would be released every few years, whereas today, apps such as Netflix see a new feature released hundreds of times a day. However, we need to make sure the value of the software we are delivering is not suffering at the cost of this increase in speed.

A great way to build this confidence and put out Turing-worthy features is by utilizing online controlled experimentation. Product managers can now utilize controlled experimentation across the full stack of their codebase. Experimentation can change the way in which we deliver new features. Instead of focusing so much on the speed at which new features are being released, it allows us to fine-tune the feature to meet the needs and demands of users.

All this speed of delivery doesn’t mean companies are getting it right 100% of the time–apps still crash and websites still fail. For example, we’ve seen companies such as Salesforce, US Bank and even Instagram suffer downtime following new feature releases. Online experimentation allows for DevOps teams to better understand the impact the upcoming feature release will have on their users.

Online experimentation requires you to split your traffic and randomize this traffic in an unbiased way, serving a new feature to a proportion of this traffic while the rest of your traffic does not experience the new feature. You can then compare the metrics of both these user groups to make an informed decision on whether to kill the feature or ramp it up to 100% of your users. You can run an experiment to a specific user segment, such as a demographic or market by utilizing feature flags within your code.

Online controlled experimentation also allows for real signals to be differentiated from noise. By noise we mean the randomness that is always apparent in the data due to using a sample to represent the whole population. The statistics from online controlled experimentation allow us to determine if the differences in the metrics are unlikely to be due to noise alone, in which case we can be confident our feature really did have an impact. This is how using online controlled experimentation decreases the chance of mistaking noise for a real, meaningful signal. In other words, it allows us to control and limit the chances of a false positive–a result that falsely indicates the change you introduced impacted your metrics.

We could spend ages discussing the benefits of using online controlled experimentation for feature releases (and we did at Turing Fest). However, the bottom line is experimentation allows you to be an outcome organization rather than an output organization. Taking the time to conduct an online controlled experiment means you will truly understand a new feature’s impact on your users and therefore you can be more confident in making your product decisions.

This article was co-authored by Sophie Harpur, product manager at Split. Sophie is passionate about making the complex world of experimentation accessible to everyone with core engineering, growth, design and product principles in mind. After graduating from the University of Strathclyde, Sophie worked for Skyscanner before eventually joining Split.

— Lizzie Eardley

September 23, 2019
5 Key Skills Developers Need to Master in Order to Thrive

The promise of the cloud, in its essence, allows the IT department to focus on generating business value rather than spinning cycles fixing and managing infrastructure. However, the cloud is not a one-size-fits-all proposition, and many early adopters are finding it challenging to move to the cloud. There are well-documented cases of huge budget overruns, far in excess of prior on-premises costs, making estimating costs in the cloud versus the status quo a key skill of any C-level IT executive nowadays.

This has ushered in an era of microservices and cloud-native applications. The symbiotic relationship between the two has enabled new architectural patterns such as microservices, while the cloud itself has enabled cheap experimentation and throwaway infrastructure. This means developers today can focus less on the infrastructure itself and more on dedicating additional resources, time and skills to iterating and deploying new code, which was traditionally only available to big organizations with limitless budgets. In addition, this evolution has created opportunities for a crop of new developers who come from non-traditional IT backgrounds to supplement the tech skills gap that’s currently crippling the industry.

Here are five invaluable skills I believe every developer, whether just entering the workforce or currently in it, should cultivate.

Adaptability Is Key

This goes without saying, but developers today must be polyglots. Ten years ago, a developer would only have to reason about one process, in one language, connecting to a database. Today, with the advent of microservices, there are many different stores of data and many applications, written in many languages, that all need to coordinate with each other over the network.

Different development teams are generally responsible for their own microservices, which in turn interact with other teams’ microservices. Careful thought is required when designing APIs through which they interact. Backward compatibility, availability and graceful failure all become crucial design decisions.

More than ever, adaptability is a key skill. In the past, developers might only have needed to learn one language and one database, and chances are that skill set would have been all they needed throughout their entire career. Today’s developer will know Java (which is still by far the most popular back-end language), but the rise in newcomers such as Go is driving the development of a new wave of cloud-native software, such as Kubernetes and Docker. It’s critical that developers keep up with new languages and database technologies.

Communication Is Critical

Communication is vital. Traditionally, developers have not been known for their communication skills, but with distributed teams and developers working from home becoming more common, working in isolation is no longer an option.

Because of these changes, it’s easy for a developer to become overwhelmed, which is why it’s also important to learn how to schedule time and how to be able to look after oneself physically. Taking breaks, sleeping well and exercising regularly is vital to preventing burnouts.

Always Look for Ways to Improve Your Technical Skills

It’s now easier than ever to improve your technical skills. The most important technologies are open source, meaning you can download everything you need and learn at your own pace. Many online videos and other materials can be leveraged as well; you no longer have to wait for your employer to approve funding for training.

But crucially, the easiest way to pick up technical skills is to practice them in a business setting. This may mean moving jobs to find the right fit. Another way to become familiar with how distributed teams work is to contribute to an open source project. You can contribute to these projects in many different areas, not just coding. Contributing to documentation, for example, is an area that offers a low barrier to entry.

Success in Humility

There are certain traits I always look for when building development teams. First and foremost, I always look for people who are able to clearly express ideas and cooperate with other members of the team.

Humility is a key requirement, because teams that cannot come to a consensus are doomed to failure. It is no use having an intransigent individual who will insist his or her ideas are best; you will waste many hours coming to conclusions with such people. Quite often, hiring a rocket scientist into your team can upset the entire balance.

Don’t Be Afraid to Ask Questions

One particular incident very early on in my career demonstrated another key skill: Never be afraid to ask questions, even if it is the same question you’ve asked previously. At the time, I was working as a mainframe operator for a book sales company. All of the day’s orders were stored in volatile memory; at the end of the day, they would be applied to a batch process that would calculate stock, print invoices and send instructions to the warehouse for customer book deliveries. This was in the time before the internet, so orders were entered manually by the data entry team who worked through order forms sent in the mail. Approximately five people entered these orders, and they usually totaled around 100-200 a day.

I was working on overseeing the batch process that night and was a few weeks into the job. There was an initial step in the batch where manual input was required to start it. However, I encountered an error message and it offered a number of options with increasingly cryptic descriptions. I had no idea which option to choose. Rather than ask the senior person working with me, I simply made a guess. The option I chose ended up wiping the volatile memory holding all the day’s orders. While this caused the company to lose a significant amount of money, I learned a vital lesson: Never, ever guess and always ask for help.

Developers are today’s innovators. With the advent of the cloud, concerns around infrastructure are minimized. In most cases, platforms to run an organization’s business software will already be available. The days of a developer having to worry about selecting a database and working out sizes of servers will be gone. Instead, a developer’s key skill will be in writing the business software and identifying which cloud platforms to use in order to deliver it. Armed with that knowledge and the desire to keep learning, today’s new wave of developers will only become more adept at delivering value over existing business processes.

— David Brimley

September 20, 2019