devopsy.kinsta.cloud

Category: Enterprise DevOps

Shouldn’t DBAs Be Part of the DevOps Inner Circle?

DevOps is changing the culture of IT. This movement is bringing application development and operations teams to the same table and the same time, working to understand and overcome the challenges facing the entire application delivery process—not just their piece. Development and operations teams are forging new bonds and alliances in the spirit of accelerating software release cycles.

However, while quickness to market is paramount to success, businesses can’t forget about the internal implications of accelerated release cycles. With that in mind, there is one conspicuously empty seat within the “DevOps Inner Circle,” and that’s the database administrator’s (DBA).

DBAs hold the keys to one of businesses’ most important assets: the database. But database change innovation has not kept pace with agile development methodologies and DevOps tooling that enable continuous delivery of software updates and changes. The vast majority of enterprises are still updating databases the same way they have been for the better part of the last three decades: through manual script review, validation, execution and individual heroics. As a result, application release cycles ultimately slow down at the last mile—the database.

It’s time for a better approach. It’s time databases and their stewards, DBAs, be part of the DevOps Inner Circle. Including DBAs can create better, more efficient teams; save time and money; and even help companies create better solutions for their customers’ needs or their own.

Shattering Silos

One of the founding principles behind DevOps is the breakdown of silos between development and operations teams. Product managers can easily translate customer needs and communicate that feedback to developers in a way they understand—using epics and themes, for example. This helps bring everyone on the same page so that application updates or changes can be quickly coded, tested and delivered. But before application updates can be released to users safely and effectively, they have to be translated into the changes that will update the database. This is where the DevOps breakdown occurs. DBAs often have to rely on manually intensive, timeworn, error-prone processes and tools to make sure the right changes are made to the database.

By inviting DBAs into the DevOps Inner Circle, database updates that go against business policy and will ultimately lock the database or “break the build” (think inserting a new column with a default value when the row count exceeds a million) can be found faster and addressed by the development team well before that change reaches the database. Having DBAs involved in release cycles sooner can help ensure developers are coding with corporate compliance in mind, ultimately making changes to the database smoother while adhering to guidelines that DBAs have typically policed.

There was a time when databases needed protection from changes made by creative problem-solving in development, but DBAs simply cannot stand against the deluge of changes necessitated by accelerated cycles. Inviting DBAs into the DevOps Inner Circle enables teams to consider the implications of new changes and updates across the entire technology stack. Breaking down silos and making teams more aware of catchpoints in the last mile of the release process lets them anticipate and avoid database issues—which means faster releases and fewer headaches for everyone.

Saving Time (and Time is Money)

Faster release cycles are good for business, and not just because they get applications in front of end users sooner. Continuous delivery cycles are designed to cut costs associated with application development considerably by saving man-hours. Still, while development timelines keep getting shorter, the workload and time to deploy database changes gets longer.

Enterprises can have multiple database configurations, so each change needs to be checked for compatibility. As the complexity of data repositories grows, so too does the time it takes to review and approve changes. In a 2015 IOUG survey on database manageability, 41 percent of respondents said changes took a week or more to approve. With multiple change requests coming to DBAs each month, each week or maybe even daily, it’s clear that sheer manpower can’t keep up with continuous delivery forever. Relief for DBAs is nonexistent without automation.

Remember our earlier example, where we ran the risk of locking the database by inserting a new column with a default value when the row count exceeded a million? Database automation software protects against outcomes like this by forecasting the impact application changes will have on the database. DBAs can use forecast reports generated by automation software to pinpoint any issues without expensive hours or days of searching.

As part of a DevOps team, the DBA equipped with automation can do more than save time and headaches for themselves. Protecting against errors that could break the build saves teams from spending time to find and fix the source of a breakdown, and also saves a business from losing revenue while the fix is made. If so much time, money and potential frustration ride on DBAs’ ability to implement changes at the speed of continuous delivery, why are they still on the outside of the DevOps Inner Circle?

Freeing Up Time for Innovation

Flashes of inspiration rarely come about when you’re just trying to keep your head above water, and for so many DBAs, that’s the reality of work. Today, enterprises fight a war of attrition with the database, throwing manpower at the final hurdle of the release cycle. In addition to the attractive benefit of saving money on man-hours, bringing DBAs into the DevOps inner circle and automating some of their tasks also creates time and space for innovation.

The relational database is nearly a 50-year-old innovation, one that changed the way people and computers interface through data. After so much time, and with so much on the horizon, the next innovation seems imminent or perhaps even overdue. The advent of quantum computing may bring with it a fundamental change to the way databases interact with one another. It’s certain to bring a shift in the way data is stored and retrieved, though at this point experts expect only the most complex business processes will benefit from quantum computing. So, while future tech remains out of reach, DBAs will be vital in the effort to innovate faster, more streamlined ways for dealing with zeros and ones.

The first step to innovation, then, is addressing the needs of DBAs alongside those of customers and developers. Breaking down walls and increasing efficiencies between existing DevOps teams and DBAs will give teams more opportunity to create novel solutions to both their users’ problems and their own. Add another seat to the Inner Circle, save time and money, get quality applications to market even faster, and create innovative solutions to the problems of today and tomorrow.

About the Author

Derek Hutson joined Datical as CEO in 2015, bringing more than 15 years of leadership experience in enterprise software. Before joining Datical, he led multiple companies through extended periods of rapid growth and also held several executive roles at IBM. There he served on the leadership team for IBM’s acquisition of Telelogic, and was responsible for due diligence and integration of the acquired sales functions globally. He currently serves on the board of Hart InterCivic and Charity Dynamics. Other board or advisor positions have included CoreTrace (acquired by Lumension), Innography, StreamStep (acquired by BMC) and Phurnace (acquired by BMC).

February 17, 2016
An Inconvenient Truth about DevOps
In a company’s early stage, the focus is on producing features for the customer. When the focus is tangible value, it’s natural for a software-as-a-service (SaaS) startup to fill its ranks solely with application developers. But when should a company invest in development operations, and at what level? If it has enough capital to hire four developers, should one of those four be DevOps?

Over the last few years we’ve continued to wrestle with this question. The value from an app engineer was so tangible, but DevOps was something else. Though no visible product was created, DevOps produced many accelerants. Incorporating development operations promised to make the other engineers faster, more effective and happier, but we always feared a sacrifice in features. The decision was similar to setting up robust automation testing—the value accrues, but the upfront cost is a hard pill to swallow.

We did it too late, and learned some interesting things along the way.

The Benefits/Advantages of DevOps

The benefits to incorporating development operations early on are many. As an immediate ROI, DevOps will add a lot of agility to your team by allowing your other engineers (and your quality control team) to focus on what they do best: create your product.

What is interesting is that the role always exists implicitly within your development team, but is more effective when explicit. If added early, this position can accelerate your team. If not added in time, each developer will play the role in part. The biggest difference between the traditional engineering role and DevOps is the context required: It is possible for a DevOps person to not really know the business rules of your system and still add tremendous value. This means anyone skilled in the craft can be put to work, which allows for quicker onboarding.

To better illustrate this, let me start by explaining deeper the importance of DevOps in a startup (and in companies of all sizes, actually). By implementing DevOps practices you will enable quicker product iterations. This is vital for a startup, as much of the success of a new venture depends on meeting release dates and getting new versions of the product in the hands of your users as quick as possible. Also, configuration management (CM) tools provide an inventory of all your software and configurations as a living documentation. Your infrastructure setup will be more predictable, you’ll have a better understanding of your stack. Centralized logging solutions and performance monitoring tools will enhance your monitoring and alerting. In short, investing in DevOps in the beginning yields valued dividends over time as you scale.

How DevOps Has Impacted Our Business

Establishing DevOps deeply improved our release process. We used to release new versions of our product much less frequently, taking 30 to 40 hours per month of a developer’s time. Now we are able to release much more often—at least three times a week. In the past we would barely release three times in the same month! Now it takes just a few seconds of a developer’s time, only requiring the push of a button in our continuous integration/continuous deployment tool (CI/CD) Atlassian Bamboo. Builds are run and the application is deployed continuously to our test environments over the course of a few days. After successful rounds of testing performed by our quality control team, we are then ready to push our new release to the production environment by hitting the little icon named “Deploy.” It is enabling.

This successful implementation of a CI/CD strategy also added much more transparency to our pipeline. Now everyone can see in a central place all the metadata about a release, including what product version is deployed to which environment, what is being deployed and by whom.

Another result we achieved directly with DevOps was agility in setting up our cloud infrastructure. In the old days we would have to log in to our cloud provider control panel and manually spin up new instances, waiting long minutes for them to start only so we could then secure shell (SSH) into them and make all the necessary installations and apply the required settings. Now, we can simply a run a script in Ansible (our CM tool) that sets up an entire environment for us. And since it is much cheaper now to provision and configure new infrastructure, we are now able to create an entire new process for testing new epics completely segregated from our main infrastructure. This has greatly improved our development processes, allowing for more flexibility, speed and control when developing and testing new product epics.

DevOps Excellence

During the past year, we have come across different DevOps professionals and learned a lot about the good, the bad and the ugly DevOps engineer. More importantly, we’ve learned what makes a great one. Topping the list of attributes is extreme attention to detail. Managing an entire infrastructure isn’t easy; it has a lot of moving parts. Being able to not get lost in the details is essential.

A great DevOps engineer also possesses strong process orientation and a passion for automation. To be supportive to the engineering team, he or she needs to understand deeply the current engineering processes and think of ways to streamline and automate them. The questions the DevOps engineer should ask include:
- What is our build process?
- When do we deploy to QA?
- How do we notify the Quality Control team that a new version is ready to be tested?
You need someone who can challenge the enterprise status quo and help to define best practices for developing and deploying systems.

Finally, as happens with the definition of the word, there is no definitive answer on where a DevOps team or individual fit in the engineering organization of a startup. At our company, the DevOps engineers report to the Enterprise Architect, who reports to the CTO. In other organizations we have seen DevOps engineers report to the Head of Infrastructure (who reports to the CTO), or report to the Head of Infrastructure (who in turn reports to the VP of Engineering).

It really doesn’t matter where it fits in your company architecture. First, define your company and vision, then define your org chart. Just remember to include a box for your DevOps engineer from the beginning!

About the Author

Rafael Neves, head of Enterprise Architecture at ALICE, has more than 8 years of experience with technology, having worked in the financial industry both in the sell-side and the buy-side for most of his career. Prior to joining ALICE, he served as head of Engineering at investment advisory startup Monetar. Before that, he was the head of IT Quality & Process of Brasil Plural, engaging in prominent projects such as the technological structuring of the bank’s Broker Dealer in New York. In 2010 he became the head of Algorithmic Trading in the first electronic trading desk in Brazil at Agora CTVM, the leading brokerage house in the Brazilian Exchange which he joined in 2008. Rafael earned a Bachelor’s Degree in Computer Science from Universidade do Estado do Rio de Janeiro.
February 11, 2016
How DevOps Can Effect Culture Change to Improve Business

The rift between Dev and Ops teams has long been one of the main obstacles for organizations managing their IT capabilities. Overcoming it can be crucial for a successful business. In fact, every company should be able to achieve harmony by improving process automation, implementing better procedures and refining workflow regulations. This is where DevOps culture steps in.

To be successful, your DevOps approach must include a detailed understanding of your culture and learned behaviors that will influence the changes you want to see in your organization.

According to recent Gartner research, DevOps will only work when both Dev and Ops are willing to collaborate. They need to embrace change at a behavioral level to initiate and achieve culture change. If not, there is a 90 percent likelihood that any DevOps initiative will fail.

In discussing the importance of communication in implementing DevOps know-how with Consultancy UK, Gartner’s research director Laurie Wurster explained: “With respect to culture, DevOps seeks to change the dynamics in which operations and development teams interact. Key to this change are the issues of trust, honesty and responsibility. In essence, the goal is to enable each organization to see the perspective of the other and to modify behavior accordingly, while motivating autonomy.”

Theory vs. Practice

In reality, the existing autonomy of the operations and development teams is one of the most common reasons for a possible failure of the DevOps approach. While the business culture of developers is founded on innovation and breaking the status quo, life in Operations can seem unthinkable without a procedure-based routine.

Going further, these two teams can be pushed apart by their own executives. After all, handing out separate tasks and objectives without actually putting the Dev and Ops teams in a room together (even when working on the same project!) has been a traditional approach that dates back decades.

Deadlines and Objectives

With Dev and Ops each being unaware of the business value of the other, the departments end up sharing only two business factors: deadlines and objectives. This often leads to a blame game—explaining a delay or a failure seems much easier when you have someone else to blame and a team with an unknown value is always an easy target.

Putting a stop to this poor practice by initiating cooperation and reminding both they are part of the same organization is the best way to improve efficiency and speed of innovation. IT managers must build bridges and encourage Dev and Ops to discover new ways to accomplish their shared objectives.

DevOps for an Agile Reboot

Modern business has realized that while Agile improves speed and service delivery, it may have skipped some very important aspects of culture change resulting in the birth of “fragile infrastructure.” The term is used to describe the damage that can be caused by Agile’s ‘shortcut policy,’ in which teams may compromise to meet their deadlines.

Many see DevOps as the savior that comes just in time to influence business on a cultural level and unite with Agile toward excellence.

One of DevOps’s flagmen, Mike Kavis, vice president and principal architect at Cloud Technology Partners and a Forbes contributor, points out that “the solution is for dev and ops to collaborate on the environment specs and together build the appropriate automation scripts that can provision environments with a single touch of a button without the need of additional configurations.”

When implementing the DevOps approach, developers can be the toughest audience. They may remember all the arguments and rifts with operations, potentially caused by tentative coding leading to functionality issues within the environment—a destabilization every Operations team fears and detests. So, one question remains: How can you build this collaboration?

Incentivize Accordingly

As stated by Forrester analyst Kurt Bittner, IT companies of today are helping the cultural rift between dev and ops by incentivizing the teams to do different things—developers are prompted to deliver quickly, while operators are targeted to keep the services up and running.

Encouraged to pull in opposite directions, it is only natural that the two business units end up facing a rift, making cooperation seem more illusion than reality.

Because most companies set their priorities based in part on its incentives policy, it is important that those priorities agree with the company goals. In short, once you “go DevOps,” make sure your incentives policy is encouraging rather than hindering collaboration.

Set Shared Goals

DevOps supporters see DevOps as a cultural movement and a philosophy, rather than a framework or simply an approach.

“DevOps is a mindset of moving away from fragile systems that take too long to deploy to reliable software that can be deployed quickly. DevOps requires great collaboration and teamwork, not just within IT, but also with management and product,” Kavis notes.

Identifying a goal that can bridge the gap and make teams work together is a great way to initiate a change in a positive direction. As Bittner explains, “Part of that means measuring devs on stability, and part of that means measuring ops on innovation. But the best thing is to reward everybody on improving the customer experience.”

A recent case study presented by Orbitz Worldwide’s senior software engineers Greg Burton and Ori Rawlings at the Velocity Conference neatly demonstrated how setting a common goal for Dev and Ops can bring actual results.

The goal itself was simple but ambitious: to double the hotel search capacity across the whole website. While achieving it by working closely with operatives, the development team managed to get a hold of many issues plaguing the system on day to day.

In fact, being a back-end developer himself, Burton defined the term “feature guilt”—the belief among developers that any time spent in aiding operations to resolve outstanding performance issues is lost and unproductive, so everything that does not involve new features must be completed quickly.

Like many other professionals, Burton also believes change should be realized as soon as possible throughout the organization to prevent substantial losses for everyone. Working on a shared goal, such as improving the customer experience, for example, can help influence developers to appreciate that maintaining a system is as important as developing new features.

Change the Culture of Fear

As Bittner notes, “If agile was the opening act, continuous delivery is the headliner.” Once the culture change is favorable and the company is running at full capacity thanks to the DevOps approach, favoring innovation and experiments within the organization becomes one of DevOps’s main advantages. Still, allowing it to work might prove unattainable to some companies. What lies beneath is the fear of failure that can paralyze every manager and employee.

Randy Bias, director of the CloudStack Foundation and vice president Technology at EMC, speaks of “a culture of fear” that exists within organizations, which is driven by an “endemic” fear of failure.

This is caused by the widely adopted practice of heavily penalizing every form of failure instead of focusing on how to minimize the potential losses and still allow experimentation for the sake of innovation.

The real question is, How can you make this possible when your business is at stake? The secret to success, according to many industry experts, is creating a structure based on separate components that can malfunction without leading the company to bankruptcy.

As hard as this may be for companies relying on monolithic applications, it is best to adopt microservice architecture that will permit managers to allow developers freedom, even if this means paying the cost for the potential mistakes.

According to Mike Kavis, vice president and principal architect at Cloud Technology Partners, and Etsy’s CEO Chad Dickerson, it is essential to allow people to make mistakes as freely as reason allows and turn postmortems into productive and proactive meetings, not a blame game.

One of the greatest success stories of today’s business practices, the e-commerce platform Etsy is a great example of the benefits of DevOps practices and blame-free meetings.

Senior vice president of Technical Operations John Allspaw, who is in charge of Etsy’s “blameless postmortems,” told Business Insider: “Less-skilled companies name, blame and shame. It’s not about avoiding accountability. In a blameless culture, it’s easier for people to take responsibility for their mistakes and learn from them. Developers already feel quite guilty when they break the site.”

Seeing is Believing

One-on-one meetings with management, prearranged brainstorming and specially made presentations are no longer enough to effectively influence culture change. Encouraging communication between teams with the right incentives is vital, but modern management needs to push even further.

The best way to make developers understand operations and vice versa is to get them together and even share job descriptions.

Companies like OnShape, for example, have taken a big step in reorganizing on-call duty by making developers and operations actually work together. According to Paul Beltrani, webops at MC10 Inc. and former techops at Onshape, this has been a win-win game, as the change has helped both teams.

Developers have gotten a grasp of the after-effect of a faulty code, while operations have gained a better understanding of how the code works. Furthermore, making dev and ops share a responsibility has also proven to be less stressful, as developers and operations were effectively cooperating by exchanging knowledge to handle the issues.

CenturyLink goes even further in this direction, by transforming its dev and ops teams into product teams able to support the entire service.

“It aims to be light on management and process, with each product team assigned each aspect of the product with a laser focus,” explains Jared Wray, CTO of CenturyLink Cloud and head of the development center. “There’s no throwing it over the fence, and there’s no distractions between the two organizations.”

Patience is a Virtue

If you’re willing to transform your organization according to DevOps, you should invest all of your patience and good will; be ready to listen and improve.

As every business is unique in its organizational structure and services, and is counting on many individuals, there is no strict route to follow when implementing DevOps. Of course, there is something all professionals agree on: “Most people think they’re going to snap their fingers and it’s going to happen overnight by doing a reorg. But it’s a full culture shift. It took us three years to make this model, and we’re still evolving it,” Wray says.

According to Chris Nowak, senior vice president and head of DevOps Services at Bank of America, it’s natural to face initial resistance when presenting teams with new requirements and processes and the best way to handle it is with care. Nowak recommends the human approach: finding and encouraging opinion leaders who can influence change and “eventually help other teams let go and embrace a new way.”

So, setting clear goals is the first step to becoming a DevOps success and rushing change has only ever resulted in a temporary workaround. DevOps is gaining more and more popularity, making it possible to gather opinions and learn from the forerunners.

Industry experts have come together to found the DevOps Institute and share a common language through the DevOps Foundation course, so maybe now more than ever is a good time for your business to “go DevOps” and see the value.

February 10, 2016
DevOps Debates: Image-Driven or Build-Driven Delivery
In the last 12 months, Docker and the containerization paradigm has challenged the foundation of some of the software delivery principles. People are resorting to things that were unheard of 18 months back and are now using them to successfully deliver software faster and cheaper. One such thing is the debate around build-driven delivery vs image-driven delivery.

Over the last few decades, software has been packaged and built by a build system and tagged with a number; testers aligned their testing to the tag as the entity being tested and promoted them until it reached production. The build number was one of the most important tag for anyone to communicate about the status of the software in the delivery pipeline.

For some time now—primarily due to Docker—several teams are moved to a methodology in which the image defines the version of a software. Every change in application program and deployment program will together create a new image, which is tagged and used as the version of the software to be launched for downstream use and finally into production.

The debate then is, which option is better?

Let me take a shot at this, and will invite your thoughts on this as well.

For builds-driven delivery:
1. Builds helps software to be tagged irrespective of the target hardware and other deployment options. It bundles all dependencies to ensure the software works when deployed to the expectation. This provides the development team a good hand-off point to move on to the next set of work, or sprint.
2. Builds are much smaller in size, as the build bundles only necessary software needed by the application; hence, keeping them aligned with every change will not be a costly storage proposition.
3. Versioning of artifacts and bundling them at run time has given tools such as maven to build and link dynamically, collecting all dependencies from the cloud. This makes it easier to keep a thin build and further reduce the redundant data to be kept in each build.
4. Build repositories have matured over the years to provide a very easy way to version and mark dependencies so deployment tools can find new versions and automatically upgrade environments as needed.
For images-driven delivery:
1. Images takes the concept of build to the next level and ensures the variability in target hardware and deployment options are removed completely. It bundles the application as given in the build or built as part of the image creation and pushes the image as a build.
2. Docker registries have started to compare and contrast themselves into a build repository by providing similar options for end users.
3. The images-driven option provides a much better benefit of standardizing all your environments, removing the question,”It works in my environment; does it in yours?” If it works in the container in a developer’s laptop, it should work in production as well—everything is same, except for the size and number of resources.
4. Images are much larger than builds, given that it has to now carry the system software and the application base software within the image. This option was unheard of earlier, but with container images coming down to megabytes, this is not such a big issue today. With layering sizes can be reduced much further as you start peeling the onion of dependencies. Take a look at this nice article by Brian on this topic.
I guess the choice depends on where your application is in its life cycle, the questions teams should really ask are:

a) Do we need to change too much to pick an option for an incremental benefit?

b) Do I have the issue of environmental differences?

c) Can I merge these options to get a hybrid solution going?

But for me the larger question is, Which option would you choose, if you were starting from scratch tomorrow? I would love to hear your thoughts.

Finally, irrespective of which way you lean, image-driven solutions is one of the best way to bring DevOps culture to a team: If the image works, the team has won; it doesn’t matter whether it was ops code or dev code!
February 8, 2016
In DevOps, Business and Operations Metrics Both Matter

Proponents of adopting a complete DevOps approach to application delivery and deployment often note that DevOps includes a change in culture. And not just a change in lateral change in culture—improvements in the way teams across (and within) silos communicate and collaborate—but also horizontal change that permeates every aspect of the organization, from the top down.

That change means more than sharing information, it means sharing the right information. And in the case of convincing business stakeholders and even executive leadership that DevOps is the right approach to succeeding in this fast-paced, API and app economy, that means communicating its impact as it relates to their primary concerns.

Surveys such as Puppet Labs’ State of DevOps has done an excellent job of providing insight into the operational impact of DevOps. We’ve seen improved mean time to recovery (MTTR) and shorter lead times required. If you’re familiar with these measurements, then said improvements no doubt provide an excellent reason to at least begin adopting DevOps.

But what about executive and business leadership? They’re likely no more impressed by an improved MTTR than you are by a 10 percent reduction in average handle time or improved call resolution rate. To get buy-in from business and executive IT stakeholders, we have to provide proof of positive impact in their metrics. CA Technologies illustrated this divide in its DevOps Jigsaw survey, in which 86 percent of respondents consider “business stakeholder education” important, and yet only 33 percent have completed.

Communicating the benefits of DevOps in a meaningful way to non-technical people is an important part of that educational process. That could mean translating MTTR to dollars saved by reducing downtime, or improvements in app performance (response times) translated into productivity increases or reduced abandonment rates. That means focusing as much on measurement after delivery as it does measurement of delivery. It means crossing the chasm (and I know it’s a big chasm, trust me) between marketing and IT to understand abandonment and conversion rates and how they relate to performance and availability. It means making connections between IT metrics and business impacts and being able to connect the two and educate business leaders on how DevOps will or can make a difference.

It means, at a minimum, showing a direct and positive correlation between DevOps and a more agile IT environment and the metrics that consume the lives (and dashboards) of executives and business stakeholders. You don’t have to necessarily do the math for them (though that would be nice if you could), but being able to connect the dots between operational and business metrics should be considered minimum viable effort. And it’s a good place to start the process of affecting the cultural transformation required to sustain the benefits of a DevOps approach.

February 8, 2016
IT Governance at DevOps Speed: Time to Shift Gears
Technology plays an incredibly important role in the idea economy, where business innovation is dependent on the software that powers the business. Speed and agility in IT are key to a competitive advantage in this modern economy, in which small startups with disruptive ideas are competing with well-established enterprises and industries. Consider how Uber and Airbnb are challenging taxis, rental car companies and hotels, all based on the power of an idea and software. Yet, traditional enterprise IT struggles to deliver at the speed business demands.

Have you ever heard a business leader say, “Our IT teams are extremely responsive and very fast”? The typical response is quite the opposite. IT is commonly viewed as a bottleneck to business innovation and, in many organizations, the business simply avoids IT and instead creates some form of shadow IT.

IT Governance, the Old Way

The principles and practices of DevOps creates the potential for faster delivery and higher quality. However, one of the largest sources of friction is the traditional focus on IT projects and project governance. Consider this definition of a project from the Project Management Institute:

“A project is temporary in that it has a defined beginning and end in time, and therefore defined scope and resources. And a project is unique in that it is not a routine operation, but a specific set of operations designed to accomplish a singular goal. So a project team often includes people who don’t usually work together – sometimes from different organizations and across multiple geographies.”

Given that projects can be expensive investments, and because of limited resources, enterprises justifiably carefully prioritize and plan which ideas and initiatives should be approved.

Faced with limited resources, enterprises expend significant effort to prioritize, optimize and fund only the most important projects. But no matter how well-intended, this governance approach also struggles to respond to the speed and velocity required in the idea economy. Indeed, the typical portfolio-planning and project-management approach is often an underlying cause of dysfunctional IT behaviors such as excessive requirements, throw-it-over-the-wall deployments and questionable quality—none of which are pleasant.

Consider how the business approaches user stories and requirements management. In a traditional, project-driven world in which the project is a one-time opportunity, businesses often are given an exhaustive list of ‘must have’ features in their user stories. This behavior is understandable—given the ‘feast or famine’ nature project, funding businesses need to make sure their future needs are included, even if the need isn’t immediate.

Sadly, this introduces both technical debt and excess complexity from the very beginning. Especially, when you consider that only a fraction of those ‘must have’ requirements will ever be actually used. In fact, a recent IEEE paper noted that 28 percent to 45 percent of software features often go unused.

Consider how “project success” is typically measured. Project teams typically are evaluated on their ability to deliver on time and on budget, along with other various quality and satisfaction measures. The emphasis on schedule and cost can drive sub-optimal decisions, as deadlines and cost pressures loom. As a result, testing is often abbreviated, as the schedule compresses. Sometimes the scope is reduced to conform to the deadline. This routinely leads to challenged releases in which the service desk is left holding the bag, which ultimately degrades trust and teamwork.

After the project is completed, the project team often redeploys to other projects, depending on demand for specific skills. The accumulated knowledge and wisdom of the team is lost and there is no further incentive to invest in automation or process improvements because the team has disbanded.

The DevOps Shift

DevOps is a very different approach that requires rethinking how work is funded and governed. Fundamentally, a DevOps team is charged with delivering small, high-quality releases, very quickly. The team embraces fast feedback to remove bottlenecks and waste and utilizes relentless automation to make tasks repeatable and efficient. Given that the team will deliver dozens or more releases per year, it is incentivized very differently from the traditional “project” team. The DevOps team often “owns” the application in Development, Test and Production, and has a shared accountability for the value it delivers. The team reduces handoffs and delays and implement small, frequent changes in response to business demand, customer feedback and system performance.

The friction of traditional governance needs to be addressed using DevOps to enable the enterprise to thrive. One of the ways to remove the governance friction is a paradigm shift from projects as described above to products. It is vital to focus on value and business results along with employee and customer satisfaction. The key difference is that a product is funded for the long haul:
- A product team delivers ongoing value and the business can count on improvements to meet market demand.
- A product team does not form and disband depending on funding.
- A product team is focused on delivering value through the product and will naturally enhance and improve their processes. CIOs and senior leaders need to consider using the Product model as an alternative way to fund and govern their initiatives.
I’ve talked with several IT leaders who are exploring how to evolve the concept of the traditional project toward a longer-term product orientation. Some organizations explicitly make the change to products, and others are taking a more evolutionary approach. Regardless of the path, the need to evolve the funding model is clear.

Consider how product governance is different from projects. The product team, in partnership with the business, is funded for a sufficient period of time (year, half-year, perhaps quarter) depending on the business needs. The product team then delivers fast releases of business-driven innovation and rapidly updates the system based on feedback from continuously assessing the product. As the team remains intact, it is able to learn and improve while also accelerating delivery. Additionally, because the product team is able to deliver quickly, the incentive to overload releases with “nice to have” features is greatly reduced.

Here’s how the mindset is different:

Project                                                                         Product

Short time frame                                                   Long term commitment

Limited funding                                                     Predictable funding over time

Team disbands                                                      Team owns the product and results

No incentive to automate                                   Automation improves quality and speed

Incentives based on schedule/budget            Incentives based business value

This does not mean there won’t be projects in the future, but there will be a combination of products and projects in most large enterprises. Most likely, we will see projects will be the way new products are proposed, evaluated and conceived. However, once the product is launched, the product team then will operate under a product model and mindset.

About the Author/John Jeremiah

John Jeremiah leads the Digital Research Team with the HP Software Applications Delivery Management team. He is a software professional with over 20 years of experience including a variety of leadership roles with the U.S. Navy, IT consulting and Fortune 500 IT organizations. His previous roles span from application developer, project and program manager, Director of Testing Services, and Delivery Process and Methodology Director. Prior to joining HP, he was director of Testing Services and Project Delivery Services for Thrivent Financial for Lutherans, where he led the adoption of CMMI Maturity Level 3 process framework.
February 4, 2016
IT DevOps: One Size Does Not Fit All

As absurd as this may seem, some in IT may be longing for the good old days of the mainframe. After all, the hulking machine, with its dumb terminals and thousands of punch cards, gave IT complete control. No citizen developers or business analysts allowed.

Things got more complicated with distributed desktop computing and its different operating systems and requirements. Server-based web apps simplified things once again, but now mobile has made development more complex than ever. Instead of just updating the server, you also now have to update each mobile device—creating apps that work on each and getting changes out to the entire mobile population.

What’s IT to do? Applied properly, DevOps can help.

The Evolution of DevOps

Coming on the heels of agile development—a term first coined in 2001—is DevOps, which was first popularized in Belgium in 2009. Today, 15 years into agile development and six years into DevOps, it is becoming clear that these intertwined terms are quite broad, encompassing attitude and approach, platforms, tools and solutions. Agile development, with its Scrum meetings and quick turnaround cycles, did speed development but made deployment more difficult, because many organizations just couldn’t support fast cycle deployment.

DevOps, with its platforms, tools and practices emphasizing collaboration between IT and development, targets application delivery with continuous testing, ongoing feature development and ongoing maintenance to deliver apps faster. While the intention is well and good, the DevOps process must be task-specific to succeed.

Today’s development environments can be broken down into three categories: low-code, deployment-centric apps and, in a class by itself, mobile. Delivering apps in these three environments involves a broad range of skills and tools. Therefore, the approach and degree to which DevOps is utilized must vary as well—from virtually no DevOps in low-code environments to extended DevOps in mobile.

Low-Code = Zero Ops

In cloud-based, application performance-as-a-service (aPaaS), low-code environments, DevOps can take a back seat. With the vendor managing, maintaining and controlling the deployment environment, the citizen developer or business analyst is empowered to work on GUI development and other features and functions with little involvement from IT or development. In these environments, power users can easily push out low-code apps. A strong DevOps presence here is not necessary because the separate environments typically required to propagate apps does not exist. The business analyst simply makes the changes and pushes them through to production.

Here, IT is best utilized as gatekeepers, getting involved only when needed. We characterize these environments as zero ops—the less DevOps the better. This leaves business users, IT and development free to focus on their strategic areas and core competencies, funneling resources appropriately and enhancing productivity across the board.

Deployment-Centric = Controlled Ops

High-transaction, highly scalable, business-critical global apps fall into the deployment-centric category. The goal of DevOps in these code-intensive environments is to free developers from deployment issues, enabling them to focus on their key competency of writing code. Well-executed DevOps can effectively move apps from development to deployment, enabling tight collaboration between development and IT. In this environment, a well-managed DevOps process can make it easy for the DevOps team to deploy the apps and to manage the deployment moving forward, scaling apps automatically with rules that scale automatically based on usage.

That is why we characterize DevOps in these environments as controlled ops, a collaboration tightly controlled by development and IT.

Mobile = Extended Ops

From development to deployment, maintenance, monitoring and analysis, mobile has introduced new levels of complexity to application delivery. Developers have iOS, Windows and Android to deal with, not to mention multiple versions within those environments. Then there is the development environment itself—web, native or hybrid. Compounding it further are the devices—countless phones and tablets. And deployment options—public app stores, private app stores, file-sharing services. Finally, the end user is engaging the device in ways never before imagined, placing the need to build responsive and compelling user experiences front and center.

This environment calls for extended ops—a highly collaborative environment where development, deployment, maintenance, support and business stakeholders are all integral to an app’s success. In mobile, everyone needs to have a place at the DevOps table, as mobile apps are literally iterative apps—constantly on the move, morphing and changing, driven primarily by end-user feedback. The ability to make seamless updates and user-driven changes at record speed can spell an app’s success or failure. The collaboration, platforms and tools DevOps offers in this environment have never been so critical, as business users now must be involved in the design and development of the app and in analyzing how the app is used, which will dictate changes moving forward. Successful, extended ops welcomes business users, but does not involve them in the DevOps process itself—that’s for IT and development.

In Summary

DevOps can have an impact on the entire application delivery cycle. From improving deployment frequency—which can lead to faster time-to-market—to shortening lead times for new releases with lower failure rates, the goal of DevOps is to maximize the predictability, efficiency, security and maintainability of the simplest mobile app to the most complex transactional system. The key to successful DevOps is applying the right level of collaboration and employing the right tools to ensure that DevOps improves quality and productivity rather than hinders it.

About the Author/Mark Troester

Mark Troester is the Vice President of Solutions Marketing at Progress Software. Mark has extensive experience in bringing application development and data integration products to market. Mark previously led product marketing efforts at Sonatype, SAS and Progress DataDirect. Before moving into marketing, Mark worked as a developer and developer manager for start-ups and enterprises alike.
LinkedIn Google+

February 3, 2016
Are You Ready for Your DevOps Transformation?

You’re done with missed release deadlines, long release cycles and risky releases. If your objectives are continuous application delivery to achieve fast feedback loops and reduced waste, DevOps practices provide the means to reach them.

DevOps is all about agile transformation, which reaches beyond agile development into production to achieve continuous delivery. DevOps practices provide help for the necessary next step after agile development to leverage the full potential of agile and avoid massive post-delivery changes through the development teams, but before they are ready to go into production.

The challenge is to navigate that DevOps transformation—and make it happen faster and at less risk. The speed and direction of that transformation depends on individual circumstances—for example, the type of technology you have deployed, the shape of your organization structure and processes, and what your overriding business and IT requirements are.

Beginning the Transformation Journey

Only once you have assessed your organization’s current DevOps maturity and orientation can you begin to scale your DevOps implementation—and move faster on the transformative journey toward continuous delivery. In other words, to know where you’re going, you need to know where you are now.

By measuring your status across each of these dimensions on a scale of Level 0 and Level 4 (see diagram), you can benchmark where you are today and derive recommendations and measures for future improvements.

Culture and Strategy: DevOps requires a strong vision and strategy and its success will depend on the awareness, knowledge, skills and motivation of your employees. This pillar addresses the effectiveness and alignment of your awareness-building and enablement programs as well as your incentivation system with regard to agility in all involved units, dev and ops. It also includes a snapshot of the sentiment and motivation of people to make such a change reality.

Automation: This pillar addresses the maturity of your organization in terms of IT automation as an important prerequisite for continuous delivery. Both the application architecture and the DevOps automation tool chain are assessed. Relevant automation capabilities in application configuration, provisioning, deployment, test, infrastructure and application monitoring are included in the assessment. The maturity is assessed across the tree layers infrastructure, platform and application, and by technology stack—for example, your SAP platform, .NET or Java stacks—and then combined into a single level.

Structure and Processes: The current organizational structure and methodologies and processes in use within all involved units are assessed with regard to their effectiveness for continuous delivery. This focuses on their agile fit and maturity. Governance and how agile teams and processes are embedded into the broader organization are part of the assessment scope as well.

Collaboration and Sharing: Collaboration and sharing between development, operations, infrastructure and middleware teams are crucial to accelerate the delivery process and make it leaner. The assessment looks at particular sharing vs. redundancy of tools (e.g., tools used for deployment), information (e.g., the status of a package in the delivery pipeline), processes (e.g., an ITIL release process and a development release process) and functions (e.g., a release engineer in development and a deployment expert in production).

Now you have the facts at your fingertips. The DevOps Maturity Assessment Service presents all of this as documented findings, to include a series of recommendations on how you can scale and mature your DevOps activity.

January 28, 2016
DevOps and the Challenge with Services

At RackN, we’ve been working for years to make DevOps portable across platforms. For us, that means the ability to move between clouds and from cloud to metal. Of the several challenges to portability—composability, API heterogeneity, sequence, cross-system dependencies—making configuration management tools work with services has been particularly important, as significant parts of building systems rely on services that are not directly configurable.

Let’s look at DNS as an example. Whenever we configure services on a node, there are DNS implications. If we bring up a web server, we want to use a logical name for the node. If it’s a secure web server, that name should match our certificate. If we want to make that service discoverable, we likely want to add a logical name for that node, such as “clustermanager6.example.com” or similar. If you include names for BMC and rack location—not to mention IPv6 naming—it’s easy to image nodes with 10 names. Even cloud nodes have similar issues. That’s a lot of entries to manage!

The problem with all those DNS names, and similar services, is that they are NOT configurable from the node. They are services with APIs that require restricted authorization.

Services and Automation

DNS is just one simple example. System-level DevOps has many places where the configuration workflow needs to include external service actions. In cloud environments, generally that means either writing pre- or post-configuration steps that talk to the specific cloud API or using one of the cloud-specific template files. In physical environments, generally that means a lot of added manual steps or tightly constraining automation to inventory map files. No matter what, nodes don’t own DNS configuration—it’s a node external service with the API elements that are site specific.

All of these approaches make DevOps brittle and site-locked. Five years ago that was okay, when choices were more limited and scripts were hand-crafted by DevOps artisans. Today, we need our automation to be robust, portable and transparent.

We realized system automation needed to treat services interactions as equal to configuration. That means when we’re working on provisioning, we make sure service steps can be woven into the sequence of configuration steps. They must not be assumed correct or delegated to pre- or post-conditions because that makes it impossible to test and re-create working environments.

What can you do? We’ve found service registration tools (we like Consul.io) are essential and relatively easy to implement to help track services that we manage or proxy. More broadly, we’ve been working in the open on Digital Rebar to blend configuration and services into a single workflow.

The first step is to re-examine your configuration management scripts (and their prerequisites) for hidden service API interactions. Exposing those external dependencies will help you improve your automation repeatability and portability.

January 26, 2016
DevOps Debates: The Benefits of Tool Standardization
As my role demands, I work with various large enterprises looking at implementing DevOps to improve their software delivery cycle times. I have seen many strategies being adopted by them—some working, and some not working yet. I have pulled together some of the more common ones so we could debate them to see if these indeed are worth looking at. With that, I kick off my “DevOps Debates” series.

The first one on the list is a popular one: If you want to create anything in large volumes, you need to standardize the manufacturing process. That makes it easier to understand what is going on, control the quality and later change processes if necessary. This is happening in a similar fashion for software delivery processes and in DevOps.

Standardization techniques adopted in many large enterprises often include standardizing the toolset used by software developers. To do this, companies create a centralized tools group or software center of excellence (COE), which tries to find the best tool for the company and then implements and manages it for the teams. The intent is to streamline DevOps implementation across the board by having a common DevOps platform that hooks into these standardized tools, delivering common practices and reporting.

I have seen and heard a lot of emotional discussion on this. Here is my go, for and against the topic. Your thoughts are most welcome!

For Standardization:
1. Centralized teams can take a global view of how things are being used across the organization. As such, they are better placed to push best practices where necessary and learn best practices when they need to. This helps in picking the right set of tools.
2. Centralized teams can create a generic DevOps platform that can serve the needs of the organization across all groups. They then can help bring a DevOps focus and manage budget centrally.
3. Standardization helps organizations streamline license cost to optimize software use.
4. Small teams focusing on DevOps may not have the muscle power or the budget to create a momentum needed for adoption. Running a central body for such implementations helps.
Against Standardization:
1. DevOps is still evolving—standardizing may be detrimental to the innovation process that needs to happen within the tools and needs ecosystem.
2. Developers from various teams driving a DevOps implementation will be more successful; they can solve the issues “on the ground” rather than a central team trying to understand and then implementing a solution.
3. Some tools are good for a specific use and may not be used companywide. Standardization may reduce productivity.
4. Developers are the users of the tools—they need the freedom to choose what they want to use. Standardizing won’t allow them that freedom, which may upset them and, in turn, lead to decreased productivity.
Would that then mean,
1. While it is a good idea in the long run to standardize, is now too early to do it?
2. Would it be better to create a DevOps platform that can provide multiple tools support while standardizing the organization’s needs?
Thoughts?
January 20, 2016