devopsy.kinsta.cloud

Tag: culture

  • Management and Culture

    Management and Culture

  • Smashing the (DevOps) Stack for Fun and Profit

    Smashing the (DevOps) Stack for Fun and Profit

    Most organizations that I deal with have a similar story for the promulgation of DevOps in their data center. The story is similar for bringing in most new technology and techniques: A team that usually does a new project without any attachments—relatively small compared to projects at the center of the business—adopted DevOps from the start. A smaller (by far) number shoehorned DevOps into an existing project as a first taste. Some decided to go “all-in” and “convert” IT to DevOps. This is a painful method; I can’t say I recommend it. I suspect my friends at those organizations wouldn’t recommend it, either.

    When I say “DevOps” in this sense, I am talking more about process and procedure changes, even cultural changes, rather than tools. My experience is that most companies that have been around longer than the last spring shower started adopting the tools of DevOps before they started looking at ways to streamline the process. And that isn’t a terrible thing at all. It doesn’t matter what marketing spin is used to sell the product; if it makes your life better or allows you to react faster, implementing it is worth consideration. I just wanted to be clear what we’re talking about here.

    Those companies that started small generally grew small, also. It is a big deal to move that 20-year-old codebase with a ton of processes that touches everything into DevOps when compared to saying, “New projects will use DevOps.” This, too, is in line with what we’ve always done. Iterative replacement has always been the case, where the large, important systems are concerned. In a couple of industries I am familiar with, I’d even argue that those large, important systems weren’t completely upgraded to Customer Information Control System (CICS), so don’t hold your breath on seeing them completely upgraded to DevOps. But that is a different blog.

    Once a critical mass has been hit, these shops start referring to themselves as a “DevOps shop” or “DevOps first shop.” This is the point I want to talk about today (though it took me a bit to get here).

    You’ve got a bunch of apps or a couple of application portfolios that are now running under DevOps. In most organizations, the tool sets are not a perfect match. Indeed, in some organizations the DevOps toolchains look like they were assembled on opposite day. Teams tend to be more standardized, but even team composition varies from project to project, as do communications.

    This is the next point of inflection in DevOps growth. You have it in your grasp to jump again, from amazing to truly amazing.

    Time to smash the DevOps stack.

    Get the teams together, build a list of each system used at each step in the process for each app, each manual intervention, each communications channel—essentially each difference between DevOps teams—and start to look for synergy. If one team is using ChatOps, check out what benefits they perceive from doing so and if ChatOps fits all teams. Does one team have a standard process to roll from continuous delivery to continuous deployment? Check it out. If you have several disparate DevOps teams, it is almost guaranteed that one team will handle DevSecOps better than the others. Take a look at what/how they’re doing and promulgate it to less security-intensive projects as needed.

    Basically, build the corporate process/toolchain from what is successful. That sounds like simple advice, but it isn’t. Teams often love their tools and processes, and are certain their choices are better than others. It takes work to get to a common understanding and there are always exceptions in a complex environment: “Well, yes that works, but for our app, we have this thing over here that makes it not possible/feasible/affordable.” It requires extra work, but that’s always true. Because the environment is complex and has been for a long time, these issues have always existed; you’re just dealing with them with new processes.

    Keep kicking it! Keeping the virtual lights on is as important as keeping the physical lights on in today’s environment. Unless your company is listed in “haveibeenpwned” emails, or your website has been suffering catastrophic outages, you are kicking it.

    — Don Macvittie

  • Want a Successful Software Build? Just Say No

    Want a Successful Software Build? Just Say No

    It is a truth which should be universally acknowledged that the most important skill for a customer-facing job in what we laughingly call “information technology” is a cast-iron ability to say no—and make it stick.

    Empty suit

    The misguided desire to please other humans by attempting to deliver on their every request has sunk more projects than any units mismatch or catastrophic misconfiguration—even more than the justly feared Friday afternoon deployment.

    The key to success is an agreed definition of what success is, and how we will all know and measure whether we have achieved it. Let’s take the car as an example.

    French Cars Have Always Been Weird

    These days, we would consider a car to be woefully unfinished if it were lacking upholstered seating for at least four adults, a roof and windows to keep the elements out and some form of in-car entertainment—an FM radio as a bare minimum, probably a CD slot or USB socket as well. This is on top of seat belts, airbags, anti-lock brakes that are probably power-assisted and more lights and safety features than you can shake a stick at.

    Nicolas Cugnot did not bother with any of these features. His design didn’t even have four wheels. It not only didn’t have an airbag in the steering wheel, it didn’t even have a steering wheel. It was controlled with a ridiculous tiller arrangement that unsurprisingly completely failed to catch on. And yet, his vehicle was a success by the stated criteria of the project: It made it much easier for the French army to haul ammunition around.

    Nicolas Cugnot's steam dray from 1769
    Nicolas Cugnot’s steam dray from 1769

    Admittedly, he was also lucky enough to be constrained by the technology available to him. Even Louis XIV might have balked at adding a four-piece chamber ensemble to provide an appropriate musical soundtrack for the driver. But the key point is this: Monsieur Cugnot stuck to his brief, and concentrated all his efforts on the features necessary to achieve it.

    Bringing Apple Into It

    In somewhat more recent times, we have Steve Jobs (emphasis mine):

    People think focus means saying yes to the thing you’ve got to focus on. But that’s not what it means at all. It means saying no to the hundred other good ideas that there are. You have to pick carefully. I’m actually as proud of the things we haven’t done as the things I have done. Innovation is saying no to 1,000 things.

    One reason why Steve Jobs is so famous is that he actually said those thousand nos—often loudly, rudely and profanely. Steve stands out in IT because most technologists have an irrepressible instinct when faced with a question to start to speculate about it, bikeshed possible solutions, build a quick minimum viable prototype and, before you know it, you are armpit-deep in feeping creatures, all of which you are now somehow responsible to nurture and feed. Not being a technologist himself but more of an ideas person, Steve was perhaps better able to take a step back and focus on the end goal.

    Let’s Get Concrete

    Now, this should not be taken as a license to adopt all of Steve Jobs’ methods (Lesson One: Be a complete arsehole to absolutely everyone in your life), if for no other reason that even Steve got fired for it, and from the company he founded at that. When someone comes to you with a request for just one leeeeetle feature, you and I probably can’t get away with screaming “NO!” and slamming the door in the requester’s face. The polite and career-prolonging way to deal with the problem is to ask, “Why?”

    Ideally, in the process of explaining the rationale for whatever the request is, some other solution to the problem will be identified. A good way to improve the odds of this happening is to involve other people in the discussion. Extending the audience in this way creates opportunities for someone to interject helpfully with something along the lines of, “but Carol, we have a perfectly good hammer over there; why are you trying to use a screwdriver to batter a nail in?”

    Peter Falk as Columbo

    Sometimes, you may need to adopt an iterative approach, which sounds impressive, but simply means waiting until the end of the first explanation, nodding thoughtfully, and then asking, “Yes, but why?” Among people who watch more television than me, this is known as the Columbo approach: “Help me understand, what is it you are trying to do here?” and just keep doggedly repeating the question. Eventually enlightenment will arrive—or the requester will get tired and go away, which is also a perfectly valid win condition.

    This method will also help with the tricky situation where the most important question is the one which is not being asked. There is often an implicit question behind that one, where the solution is not to give a glib answer, but to think critically about the proposed or effective solution and ask the questions yourself: “Why are we trying to solve for this?” Or, “What is this thing you are proposing actually meant to do?” And finally, “What was the original problem, anyway?”

    People have a tendency to focus on the issue that is right in front of them, even when the real problem is in an upstream step of the process. They observe or experience something about X as the identified problem, and so default to solving for X directly without considering the broader landscape. This leads to convoluted, inefficient and often high-friction solutions—Rube Goldberg/Heath Robinson machines (delete as appropriate to whichever side of the Atlantic you are on), where a simple connecting pipe is all that is required. An artillery piece will make holes in the ground, but there are a couple of collateral impacts you may want to consider—and maybe offer a spade instead.

    Man standing in front of giant crater
    All I needed was somewhere to plant some flowers!

    Unfortunately even the Columbo Method is not entirely guaranteed to work, because on some rare occasions the underlying request is actually valid and cannot be summarily dismissed. This is when you bring out the big guns and start ranking the urgency of what was already agreed as the scope of the project (a nail hammered into the wall sturdily enough to support that lovely picture) as opposed to what is being requested (a fully motorized picture rail, capable of supporting up to 10 paintings on the scale of Guernica and turning them to optimize lighting conditions throughout the day while also keeping them out of direct sunlight). Nobody is saying that the second option would not be nice, but it certainly won’t be done in time for supper.

    And that, children, is how you say No. Go forth, and do ye likewise, and may all your projects be huge successes. And, most importantly, not require me to come in and clean up after you.

    — Dominic Wellington

  • 7 DevOps Lessons in 2018: The Path to DevOps Success in 2019

    7 DevOps Lessons in 2018: The Path to DevOps Success in 2019

    We all learn from mistakes, right? But it would be better if we learned faster and didn’t repeat them in our future exercises. And this is exactly what we will discuss here in this blog.

    For a while now, we’ve been overwhelmed by the idea of the DevOps principle. We love how the collaboration between development and operations provides room for automation, continuous integration/delivery and enhanced time-to-market.

    But what about the tiny shortcomings we overlook in the process of continuous delivery?

    Listed below are some of the don’ts—or, rather, lessons learned from our DevOps implementation in 2018:

    Gauging Employee Capacity Over DevOps Delivery

    When Carmen DeArdo, Tasktop’s senior value stream management strategist, posed two questions to a crowd of 200 people, they had no answer to give. What were the questions, you might ask?

    • How many treat their delivery pipeline tooling as an integrated product with a product owner and an architect?
    • How many have any measure of how long it takes for them to deliver a feature or defect?

    Companies treat their delivery pipeline tooling as different components along the delivery chain. And, when it comes to measuring time taken, all that matters is getting the product to the finish line.

    These components are important if companies need to avoid obvious mistakes on the path to success. Consider the above questions the next time you set out on your DevOps mission. Within this aspect, measuring team happiness is as important as an integrated delivery pipeline tooling, and time is taken to introduce a feature.

    Companies need to strike a balance between managing work and achieving team happiness. This is important if the company has a long-term vision. Unhappy employees burdened with work don’t deliver the expected results. A sustainable pace helps companies ensure that employees don’t succumb to pressure.

    Revise the Management Strategy

    Organizations need to encourage their managers in a way that they are not put off by the DevOps process but rather motivated to build and maintain their team.

    Most organizations emphasize giving managers incentives to deliver expected results, maintain quality, expand team size, set immediate goals and rake larger profits. However, this strategy could backfire when applied to a DevOps implementation.

    Companies instead should encourage their managers to create an environment where the teams grow and become flexible, not only contributing to that process but also adding value to other processes, hence, the organization’s overall success. This thought process will lead to a chain of positive outcomes.

    Upskilled and Cross-Skilled

    Gone are the days when specializing in a particular skill was rewarding! The competitive market is looking for a workforce that is upskilled and cross-skilled. In short, employees need to be a package of multiple skills, with an emphasis on soft skills.

    Being multi-skilled not only supports transformational goals but also supports an organization’s goals. When the lack of new skills becomes a constraint, companies must invest in hiring individuals to propel transformation, which is equivalent to investing in automation.

    Automation ≠ DevOps Success

    Somewhere between adopting the DevOps culture and being DevOps-equipped, professionals have shifted from the purpose of DevOps to automation.

    In the process, people have lost control over the overall management of DevOps. Either professionals are in a situation in which the code is not built in the intended way or that the automated part is managed but the steps required for automation are not.

    Organizations instead should refocus their attention on the scalable nature of DevOps, building a new, fully automated process rather than automating an end-to-end process over time, as it is difficult and also time-consuming. Hence, automation does not equal the DevOps mission accomplished!

    From Idea to Blackboard

    DevOps once was more an idea of a team discussion rather than full-fledged process. But we saw 2018 transform DevOps from a mere idea to a part and parcel of everyday business—so much that it defines the way businesses plan their finances, regulations and organizational strategy as a whole.

    Additionally, DevOps experts in 2018 emphasized the need for value delivery than just focusing on details of technical practices. This can be achieved by constructing a managerial framework that bridges the gaps between business and technology.

    Relieve Your DevOps of Old Tech

    Last year also witnessed DevOps professionals moving toward new technology such as containers, Kubernetes and serverless. Why? It’s obviousthese technologies enable DevOps to deliver its promises. Trying to innovate the DevOps process or even innovate its end product with old technology will only take you backwards in the DevOps process.

    In 2019, we should realize that holding on to old technology will mean limiting your organization’s capabilities and impact.

    Identifying the Right DevOps Adoption Patterns

    While there are several DevOps adoption patterns out there, some organizations choose a central DevOps team enabling standard CI/CD tooling. The approach is in sync with DevOps thought leaders’ approach of reducing global complexity.

    However, this approach fails at catering to the needs of different teams. With the focus only on addressing CI/CD tooling, many centralized teams struggle with adoption.

    In another approach, companies seek to engage ambitious and autonomous tools to use DevOps tools and practices.

    This approach tends to match the common DevOps belief, ‘First prove that new work methods fit the enterprise, then spread.’ However, these teams are only successful in delivering projects with high levels of automation involved but have quickly failed to convert team-level experience to company-level.

    A 2019 DevOps adoption approach should focus on having high levels of team autonomy and reducing global complexity.

    Here are some tips to help you achieve this:

    • The central architecture team collect and share recommendations about languages, libraries and tools.
    • Teams consider these recommendations and experiment to see if there’s an opportunity for greater gain.
    • If things aren’t working, teams can change change and undo dead-end technology decisions/ideas.

    Now that we are ready with our lessons, it’s time to go live! Are you ready for your DevOps 2019 showdown?

    — Veritis

  • Culture, Not Changing Much in DevOps

    Culture, Not Changing Much in DevOps

    One of the things that has bemused me from day one of the DevOps movement is how culture is viewed with an odd dichotomy. It is simultaneously DevOps’ greatest strength and its greatest weakness.

    Of course, there is a simple and reasonable explanation for this phenomenon. DevOps wants to change culture. Cultures that are resistant to change are typified as “bad” and those that are not are typified as “good.” It is never that simple, but that’s how the portrayal goes.

    After a decade, you would think we’d have massive IT culture change supporting DevOps initiatives at one level or another (somewhere on a scale from pilot project to corporate standard) pretty much everywhere.

    But data seems to indicate that this is not the case. Disclaimer here: I fall more on the “automation” side of the “What is DevOps, really?” question. I don’t think that matters here, since I’m pointing at data from the DORA “State of DevOps Research Report,” which is 1,900 of you all speaking, not me.

    Image Credit: DORA State of DevOps Report

    Most of the coverage on this report is by people who have bet their entire career on DevOps—a solid bet, but one that can lead to stilted coverage, which takes all the positives and hoists them high. And there is a lot of positive to be found. But I’ve noticed a trend of not looking at the data with a critical eye. The bit that I’d like to focus on today is what DORA labels “Firmographics”—a breakdown of respondents by title.

    Only 27 percent of those who responded were on a DevOps team. Presumably because Google is a sponsor of DORA, it included Site Reliability Engineering with DevOps teams, and that’s a fair inclusion, since SREs by definition are doing DevOps.

    Consider that data point representing “culture change” in a world where the respondents are, by self-identification, doing DevOps. Some will argue this is the wrong way to look at it, but one of the issues that culture people bring up constantly is breaking down silos and organizing around product or product portfolio instead of development and operations. But it seems more as though organizations are knocking holes in silos and communicating through them—which means we should pay more attention to that style of DevOps and quit harping on organizational change. Again, this fits well with my world view, which has always been “That specialist in a shared bit of hardware is not moving to a DevOps team.” The EMC specialist (for example) adds value to the entire org because of their domain-specific knowledge, not because of their deep understanding of the entire DevOps toolset. Once you’ve carved out exceptions for storage, networking and platform specialists, moving everyone to a DevOps team is pretty moot.

    I won’t conjecture why only 27 percent are moved into dedicated teams, nor why that number did not go up from 2017. I have a predilection toward “because it’s not necessary,” but likely there will be two camps when this discussion gains steam because more reports show a low rate of change—Group A will be, “Because those people are resisting change,” while Group B will be, “Because those people don’t feel they need change.” Guess what? That will be the same group of people, and that angle of discussion will be irrelevant. The important bit will be, “Are you completing everything you need to get done, and what is your structure?” That’s where the meat of the discussion will need to occur. Because by deduction, 73 percent of people participating in the DORA survey are doing DevOps while not being on a dedicated (eg: reorged) team.

    And that’s perfectly okay, as long as they’re achieving the goals, that’s what matters. The how is useful when deciding to move in a certain direction, but delivering is the thing IT should—and generally is these days—be gauged on.

    — Don Macvittie

  • DevOps Transition: The Cultural Commitments and Challenges

    DevOps Transition: The Cultural Commitments and Challenges

    Enterprises can achieve certain levels of success by filling a need in the market, hiring skilled teams and utilizing cutting-edge tools and technologies. However, to set themselves apart and chart a course for long-term success, they need to address the values, beliefs and behaviors that make up organizational culture as carefully as they address business decisions. It’s culture that ultimately informs the decisions made about how a company evolves its concepts from idea to value, so when you want to make a change and grow, that’s where you need to start.

    One area where culture is particularly important is the introduction of new practices and processes applied to software engineering and the role that a DevOps strategy plays within organizations. DevOps encourages a mindset of communication, collaboration, integration and automation among software developers and IT operations, so it’s more than just a new set of tools; it’s a cultural shift. This change doesn’t happen immediately or automatically, it requires deliberate steps and actions to be taken, and calculated decisions to be made.

    The shift also requires commitment. A recent Gartner report, “New Insights into Success with Agile in Digital Transformation,” found that organizations must be prepared for a commitment of three or more years to ensure success when moving to a more agile development process. Teams with less than a year of experience with a new development process show a success rate of 34 percent, while that rate jumps to 64 percent with between one and three years of experience. After three years of agile experience, the success rate skyrockets to 81 percent.

    New Technologies Accelerate Momentum, not Create It

    Implicit in the decision to move to DevOps is the desire to develop faster. To do that, organizations often begin the process by saying to themselves, “We need to automate things.” They buy a variety of testing automation tools, production delivery tools and anything they think will make them more agile. However, if they don’t take steps to prepare the people in the organization, things can get worse, not better, because the new elements can create isolated teams centered around “campfires” where separate teams use the various new automation tools individually.

    When teams are undergoing transformation, we often see them go through a roller coaster of up-and-down results. This was illustrated in the DORA report, “Accelerate: State of DevOps 2018: Strategies for a New Economy.” When the transformation begins, teams quickly identify wins and automation helps lower performers improve, creating an uptick in overall performance. However, as the process moves along, automation increases test requirements, which are still dealt with manually, and a pile of technical debt blocks progress. At the same time, the mounting technical debt and increased complexities cause additional manual controls and layers of process around changes that, more often than not, slows work. Teams generally don’t see an upswing in results until putting in relentless efforts toward improvement, leveraging expertise and learning from their environments, which finally results in a jump in productivity.

    This is a moment when organizations need to lead with their brains, not their pocketbooks, and stay committed to the process. To minimize the risk of failure, they must address the people and the process with the same rigor and commitment they address the technology. In fact, evolving the culture and process associated with how people work is the biggest challenge in adopting DevOps. People resist change and no amount of technology makes change easier.

    Any initiative that begins with just buying software or technology to make the organization agile is a mistake. The best way for a business of any size to launch a DevOps initiative is to get people comfortable with the vernacular and the process before changing anything. I’ve seen organizations go so far as to change the name of existing processes just to get teams thinking differently before the real change is made.

    Putting in Tools Before a Plan is Like the Cart Before the Horse

    After seeding the terminology and idea of change, the next step has to be tactically thinking through how the organization will evolve. It’s often thought that tools will make an immediate impact, but not enough planning is done to integrate the tools before being taken out of the box. Basically, they end up like a bunch of kindergartners at recess: Everyone has their own toy to play with but they do it alone and no one works together.

    Before putting in new tools and changing processes, there needs to be understanding around where the gaps and inefficiencies are so KPIs and metrics can be mapped back to tell where progress is being made. You can’t manage what you can’t see, so you have to develop visibility into the process to understand where those inefficiencies currently are. When that is achieved, remaining constantly vigilant to understand the constraints that exist while establishing new processes and using the new DevOps tools will allow you to continuously improve.

    It needs to be known that the reward is worth the risk. Making the move to DevOps can’t be thought of as a decision or a choice, it’s an imperative for organizations that want to remain competitive. Every organization needs to become a software company and to do that they need to move to DevOps and make continuous improvement part of their DNA. Organizations can’t fear the move, they just need to be diligent with the transition, monitoring and follow up when adopting DevOps.

    — Bob Davis

  • 4 Questions to Ask a Web Developer During an Interview

    4 Questions to Ask a Web Developer During an Interview

    Web development is a fragile process. You need talented web developers to make truly beautiful sites with seamless user experiences and fast loading speeds. Without experienced web developers, creating a website that has both form and function is exceedingly difficult.

    Web developers have a great deal of responsibility. They understand design and they understand how to implement that design using programming languages, tools and frameworks. How do they piece all of these skills together? How do you assess whether they have these skills?

    There’s no formula for the interview process that guarantees great hiring decisions. There are a few questions you should consider appending to yours, however, as they can help you get to know the web developer a little better.

    Here are four questions to include in your interview with a qualified web developer:

    1. What programming languages, tools and frameworks are you most familiar with?

    Web developers are usually proficient in a number of programming languages and are adept at wielding frameworks. They must have an in-depth knowledge of HTML, CSS and web scripting languages such as JavaScript. If the role you’re hiring for includes back-end programming, they also will need to know how to program databases by using PHP or even Python.

    Content management systems such as WordPress can also be extremely helpful for web developers. Bootstrap and frameworks also may be helpful when building web pages.

    You’ll want to ask more about the technologies, programming languages and frameworks they like using. Don’t limit your questions to strictly the technical requirements of the role. Assess the developer’s passion and knowledge.

    1. How do you stay relevant as a web developer?

    Technology is constantly evolving. Web developers have to have strategies to stay up to date on the latest updates, frameworks and trends. There’s no wrong way to keep up self-education. A potential web developer candidate might cite online courses as a way to stay skilled and knowledgeable.

    Web developer meetups, professional web development classes and sites such as StackOverflow and GitHub can help developers stay informed and in conversation with other developers.

    1. What are some mistakes you’ve made in the past as a web developer? How did you react to those mistakes?

    Web development is a complex process. There are lots of little things that can go wrong during development. It’s not uncommon to make mistakes even as an experienced web developer. However, it’s still important to assess how a candidate responds to tricky situations.

    Do they show themselves compassion when they make a simple mistake? Do they create actionable plans to remedy mistakes when they appear? Are they resilient or easily defeated? Are they prone to blaming others or do they take responsibility for their actions?

    It’s important to know how a candidate will respond to stress especially if you expect your projects to be lengthy or complicated.

    1. What would you like your role to be?

    It’s easier to get to know a candidate if you know the activities they like to perform. Are they interested in leading a team? Do they enjoy helping design wireframes with the design team? Do they take great pleasure in optimizing back-end processes?

    Ask the web developer candidate about their own likes and how they can best fulfill their duties as a member of your company’s team. You’ll find out a great deal about their personality, preference, and work style with questions that probe into their own likes, dislikes, and dreams.

    Conclusion

    Great web developers are rare. Even though it’s hard to attract the top web developers on the market, it’s not impossible. By asking the right questions, you’ll capture the imagination of the best web developers and, hopefully, gain some insight into their own creative and technical processes.

    Delve into a few technical aspects of the job, gauging their familiarity with popular programming languages and tools. Make it a point to ask web developers about past projects and performances. See how they self-assess and gauge their capacity for compassion, collaboration and resiliency. Continuing education is also important for web developers as trends often change.

    Be sure to also delve into their personal preferences and opinions as this will help you better understand how they work and if they are the right fit for your organization.

    — Greg Robinson

  • DevOps is Failing, But Here’s How to Succeed

    DevOps is Failing, But Here’s How to Succeed

    As the Data Economy continues to gain momentum, many companies have turned to DevOps in hopes of achieving agility and velocity to compete and win. Whereas it’s no secret that DevOps can have a major impact on agility, what’s not widely known is that most DevOps projects actually fail. In fact, a simple search of “DevOps failures” returns a range of recent results from minor issues to major fiascos. So, what are the warning signs that an initiative is failing to live up to expectations, and how can you fix it before it’s too late?

    Focus on People, Not Tools

    There are several types of potential problems that undermine the efficacy of DevOps, and the most common one—which is the easiest to overlook—is failing to recognize that DevOps is a changing of processes. So, as with any kind of process change, it can only be effective when it’s embraced and adopted by all people involved. Organizations that are successful are able to not only shift processes, but also win the hearts and minds of workers and get their buy-in early on.

    When leaders approach DevOps as a tools-first methodology without addressing outdated legacy systems or the underlying IT cultural issues required to make newer tools work for their organization, projects have a tendency to either fail or fall short of expectations. In today’s competitive digital business landscape, DevOps teams need to be able to deploy multiple test environments multiple times a day to collaborate and test effectively.

    But deploying DevOps technology is often the easiest part. The real problem is trying to use the same tired data management techniques with those new tools. If marrying new tools with old structures (or systems) were simple, most companies today would already have DevOps approach in place. For initiatives to be successful, though, requires a significant shift in thought from leaders and workers alike to achieve both short and long-term development goals.

    Slow-to-Change Culture

    DevOps is all about using rapid iterations and collaboration without requiring formalized meetings to move forward, broker alliances or seek permissions. Given that DevOps is a people-driven process at its core, you know you’re in trouble when your teams are holding routine meetings lasting 30 minutes or longer. If you are doing this, it means the organization is still operating in a siloed manner, which undermines DevOps efforts.

    Creating a successful initiative takes clear communication, because even though everyone wants the same result, they often want said result on their own terms. Introducing DevOps efforts into a well-established organization isn’t easy, and often there can be culture shock and even pushback from rank-and-file workers all the way to mid-management or senior leaders.

    With that in mind, to build a culture that enables agile development, it’s important to communicate objectives clearly and act with discipline early. There should be a well-defined explanation of the objectives, metrics and goals by which your DevOps initiatives will be measured across the organization. It’s not enough to adopt DevOps practices just because it’s the buzz right now; rather, you need to be able to understand how it can help you rebuild your business process and achieve specific development goals—or else your success is not likely.

    You can’t evolve a company culture overnight, but with the right approach and commitment to the DevOps journey, you can lay the groundwork for organizational change and achieve more agility.

    Inefficient Workflows (Data Friction)

    DevOps initiatives are often dependent on applications, which are powered by business data, but getting that data to the right teams can create an enormous amount of drag on the workflow. We often call this data friction. This is a prime example of data friction, or the tension between demand for data access and the constraints that stand in the way.

    This commonly leaves DevOps teams in one of three places, each with their own drawbacks: using stale and polluted data sets, stopping Dev and Test to wait on data provisions and refreshes, or just developing on purely synthetic data (punting the data problem to Test). To avoid returning to the same constraints, we need to rethink the way our teams access data in a way that minimizes the current friction so that we’re no longer choosing between quality, security and agility.

    Data should flow securely, freely and at lower cost, eliminating the friction around data. Synthetic data should be used for development because it is the correct data to use for development, not simply because it is the fastest. Taking a DataOps approach—tackling the data constraints around people, process and technology—is the only way to ensure you are eliminating data friction, rather than shifting it to someone else.

    Conclusion

    Companies set themselves up to repeat mistakes when they try to throw even more technology at the problem, without considering how existing infrastructure, culture and tools will work together in a DevOps strategy. At the end of the day, a tool is only as good as the teams who use it, and DevOps as a methodology should always be considered a continuous cycle of improvement. It’s time to broaden our perspective on the aspects of data management that we can improve upon to also include the people behind the tools. The future of DevOps depends on our own team’s flexibility and willingness to continually improve our own processes.

    — Adam Bowen

  • DevOps & DataOps: Catalysts for Organizational Transformation

    DevOps & DataOps: Catalysts for Organizational Transformation

    Over the past few decades, use of the internet has increased exponentially. Nowadays, the need to build quality web applications and managing its huge data effectively is a huge concern of any organization. Thus, there has been continuous exploration for finding better methods of software development and data management.

    To meet the need of today’s internet users, many technology companies have stepped up and are continuously utilizing their resources to find more efficient ways of developing web applications in less time and with better quality, and are finding more effective ways to manage and analyze its data.

    To achieve the same in application development and data pipelining management, many big organizations have started following a set of practices—DevOps and DataOps respectively.

    In this article, we’ll take a look at what DevOps and DataOps are, and then we will focus on the ultimate question: Are they really worth to implement in an organization, and will following the same set of practices really improve the application development and data management—and consequently the growth—of an organization?

    DevOps and DataOps: An Introduction

    In simple words, DevOps and DataOps are a set of practices for development of software and management of data respectively, for a particular project or to follow at an organization level.

    DevOps emphasizes on automation of processes of various teams—software development team, quality assurance team and IT operational team—so as they can complete the relevant tasks (i.e. plan, code, build, test, release, deploy, operate and monitor) of application development quickly. Also, the these teams will be able to maintain the higher quality of software application by delivering/publishing software in smaller parts with higher deployment frequency while keeping the cost low, thus following the agile methodology more seamlessly.

    DataOps, in comparison, can be defined as DevOps with data analytics.

    As mentioned earlier, we know that DevOps emphasizes the collaboration of software development team (including testers) with system administrators (operational team). In a similar manner, DataOps weighs on collaborating the teams of data engineers, data scientists and other data professionals who are handling the data-related processes (i.e. store data, analyze data, deriving decisions from data, archive data or deliver data).

    The concept emphasizes both communication and collaboration between various teams of the same organization. The combined team can have more visibility of the project and come up with faster results. It also will improve the morale of the organization by proving the fact that people will be always above processes and tools when it comes to management of projects.

    DevOps DataOps
    Automation and Monitoring of Software Development Automation and Monitoring of Data Pipelining/Management
    Combination of

    –       Software Development

    –       Quality Assurance

    –       Technology Operations

    		 Combination of

    –       Data Engineering

    –       Data Integration

    –       Data Quality

    –       Data Security/Privacy
    Collaboration of

    –       Software Engineers (Developers)

    –       System Administrators (Operational Team)

    –       Testers

    		 Collaboration of

    –       Data Engineers

    –       Data Scientists

    –       Data Analysts

    –       Data Professionals
    Typically aims at

    –       Shorter development cycles

    –       Increased deployment frequency

    –       Quickly moving from Dev –  to/from Test – to Production

    –       More dependable releases

    		 Typically aims at

    –       Automation of Storage Workflows

    –       Monitoring the Operational Flow

    –       Optimization of Data Analytics

    –       Creating Infrastructure that will enforce data quality rules

    Now that we’ve gone through the basic details regarding DevOps and DataOps and what they are doing for software development and data management respectively, let’s take a look at some of their potential benefits to see if they can really act as a stimulator for an organizational transformation.

    Benefits of DevOps and DataOps to adapting Organization

    There are numerous benefits of adapting the defined processes of DevOps and DataOps in project execution and running the organization. Here are some of the key benefits for applying the practices of DevOps for applications development in an organization.

    Benefits of DevOps

    • Transparent Processes and Culture across Organization – By continuous communication and collaboration between the development team, QA team and IT operations team for every deployment cycle, each employee will be updated regarding latest information of project and will be more aware regarding the processes of other departments.
    • Quicker and Higher Quality Deployment Cycles – By applying the set of practices of DevOps, especially the shorter development cycle, will boost the frequency of deployment. As the focus of teams will be limited—i.e. on small task/module—for a deployment cycle, quality of each deployment will improve.
    • Faster Resolution of Bugs – As tasks are smaller and continuously deployed on relevant servers, the development team will be given feedback very quickly from the QA team, thus enabling the development team to provide much quicker resolution of any non-working functionality. Application stability will be a byproduct of this process, as the reason behind any error can be identified very easily (because particular deployment cycle of an error is known).
    • Flexibility to Handle Unplanned Work – As agile methodology insists on the handling of the raised change request, teams streamlined with DevOps will be able to handle such unplanned work more efficiently.

    Benefits of DataOps

    • Faster Processing of Data – As various teams are getting collaborated, communicating continuously and working on micro databases rather than all at a time, processing of data at various process stages are done much quicker.

    • Quicker Solution of Problems – Similarly to DevOps, in DataOps data are being processed in bunches rather than at once. This approach can help to identify issues pertaining to a bunch of data more quickly, and their resolution can be derived more quickly, as the focus is limited.
    • Better Data Security – As the data security professionals are in continuous touch with data integrators and data analysts, securing data at each process level becomes easier.
    • Improved Resource Output – When the team adheres to the fundamentals of DataOps, seamless communication and transparency between the various teams will aid in the performance of each team member and will lead to better resource output.

    Implementation Procedure of DevOps and DataOps

    Defining the processes for implementing DevOps and DataOps is simple, but implementing same processes do require tremendous dedication of teams. Let’s focus first on the general steps for implementing DevOps processes:

    • Collaboration, communication/seamless integration between formerly siloed teams, i.e. software development team, quality assurance team and IT operations team
    • Breaking down the major modules/big development tasks in particular smaller tasks
    • Processes for DevOps should be followed for each deployment cycle/each smaller tasks
    • Tools – There are various tools available in the market to accomplish processes of DevOps, and there are some techniques for continuous development and performing agile operations‚ such as Kanban, Scrum, etc., which should be followed for seamless application deployment.

    With DataOps, there are no specific software tools available in the market to implement DevOps. But there are some frameworks and its supporting tools that can assist the DataOps approach for agile-based project management.

    • Data storage and analytics infrastructure should be redefined, i.e. cluster-based or redundant data storage approach is recommended. Also some latest data analytics techniques should be adopted.
    • Use of microservices-based architecture and software that can allow blending of various type of data, i.e. structured and unstructured, are also recommended
    • Implement ‘built for purpose’ database engines – By doing such, accessing and analyzing large quantities of data become much easier (these are the same type of database engines that drive Google Search, YouTube etc.)
    • Supporting tools – Tools that aid to agility of project and collaboration between teams i.e. ETL/ELT tools, data curation and cataloging tools and log analysers should be used. For data blending, software tools such as Map Reduce, HDFS, Kafka, Hive and Spark can be used

    Conclusion

    All IT organizations have been managing software application development and its relevant data for a long time. But right now, we are in an era where managing the same with velocity and quality can make your organization stand above the rest of peer companies. Adopting DevOps and DataOps demand sheer dedication and focus of the teams, but the output after implementation will be very positive and such followed standards will definitely transform the organization constructively.

    — Vikash Kumar

  • How to Fix ‘Dark Debt’ in Your IT Organization’s Culture

    How to Fix ‘Dark Debt’ in Your IT Organization’s Culture

    Poor decisions, bad metrics, lack of communication can accrue dark debt in any company

    Buying a low-quality microwave meal from a gas station, kicking a sock under the bed before guests arrive, jaywalking … We all take shortcuts. But unlike enterprise software delivery, these decisions rarely cause any collateral damage. The ramifications of taking shortcuts in IT, however, are much more serious.

    These decisions—i.e. “technical debt,” the “rework” debt owed after opting for a convenient solution instead of a better approach that takes longer—has implications on your company’s ability to grow and adapt to change. Fortunately, since the 1990s, when the term was coined by Ward Cunningham, we’ve learned how to identify and manage most forms of technical debt, understanding how to limit its impact and manage the interest.

    Your organizational culture can accrue debt, too. For example, a company that finds it easier to hire a homogeneous team up front might discover the costly cultural trade-off of hiring diversity later. When it comes to this “cultural debt,” we are much less disciplined. We often make decisions in the dark, without understanding how they are going to play out over the long term. “Dark debt” builds up over time and creates a toxic work environment in which people are not set up for success.

    Common issues indicating dark debt include:

    • Failures lead to finger-pointing.
    • Metrics incentivize undesirable behavior.
    • Lack of candor.

    Ways to Combat Dark Debt

    Call Out Psychological Safety Indicators

    Failures are inevitable, whether we are discussing power outages, data breaches or false missile alerts. A “safe to fail” mentality promotes learning and support adaptability. If employees don’t feel safe telling the boss bad news, the latter might be the last person to know what’s going on—not a good situation for a company working to disrupt an industry, or one expecting to stay listed on the Dow Jones.

    A Net Promoter Score-like survey can help identify areas where employees experience dissatisfaction. Here are some sample statements for your NPS survey that are based on Westrum’s organizational culture model as introduced in the “2014 State of DevOps Report.”

    • On my team, failure causes inquiry and not blame.
    • Our leadership is open to hearing bad news.
    • In my organization, failures are learning opportunities and messengers are not punished.
    • People on our team trust one another.

    The survey results will shine a light on the forces shaping your culture. Instead of finger-pointing, you can learn to address issues earlier and ask questions such as, “Why did it make sense for someone to do that at that time?”

    Measure Thoughtfully

    Tell me how you’ll measure me and I’ll tell you how I’ll behave. Gaming metrics are fairly easy to do, especially when an employee is incentivized to hit an unrealistic target or avoid embarrassment. Take activity metrics, for example (lines of code, number of bugs): These metrics focus on keeping people busy. But busyness does not equal business value delivered. People can be 100 percent utilized sprinting from meeting to meeting while making scant progress on business goals.

    Instead, consider metrics that reveal progress toward important business objectives or ones that help you make better business decisions. Flow metrics such as flow load, flow time, flow efficiency, and flow velocity reveal trends that measure four key areas: workload, speed, waste and throughput (respectively). These metrics are helpful in numerous ways, such as:

    • When others imply that things take too long, teams work inefficiently or people aren’t fully utilized, it’s useful to test their opinion against the data.
    • Having a balanced set of metrics allows you to see how changes in one area impact other areas. If the amount of work-in-progress (flow load) goes up, things will take longer to do (because of context switching and conflicting priorities), which results in slower speeds (longer flow times).

    Think about the behaviors that occur in your organization by how metrics incentivize people. What we measure impacts people because people value what is measured. Think twice before automatically defaulting to free and easy metrics available in your existing toolset.

    Build Social Capital

    Ideas start out flawed and incomplete. High levels of social capital, debate and discussion are the means by which ideas flourish. Social capital is what enables people to collaborate more effectively. The absence of social capital makes it impossible for people to speak openly (due to the lack of trust), resulting in withheld feedback and invisible work.

    Not much happens without discussion. When we engage in debate, our capacity to see each other’s perspectives is realized. TED talk speaker Margaret Heffernan’s insight on social capital is compelling: “Time compounds social capital.” The longer teams work together, the more trust, helpfulness and candor they accrue. Longstanding teams, therefore, benefit from increased productivity and reduced risk.

    Unlike the costly and time-consuming transitions observed in project-managed work, where people disband after dumping their work on the operations team before starting another project, organizing teams around a product allows the people who developed, tested and delivered the functionality to not only stay in their area of expertise, but also grow and gain from social capital.

    — Dominica DeGrandis