Tag: ibm

Webinar: Canada Mortgage Housing Corporation breaks down IT silos and speeds up mainframe development

DevOps encourages adoption of lean and agile development practices irrespective of platforms. However in most organizations mainframe and mobile/web development teams are functioning in siloes, resulting in a plethora of technology platforms and tools, fragmented or manual processes, skills shortage causing unnecessary delays, rework and friction among teams, impacting organizations ability to respond to the business needs.

Join Jeff Blackadar, Application Development Manager at Canada Mortgage and Housing Corporation, Alan Shimel and IBM experts as they explore the various DevOps strategies, practices and solutions to increase collaboration, streamline processes and leverage modern tooling to attract next generation of mainframe developers to speed mainframe application development and maintenance.

Event recorded Wednesday April 22^nd, 2015

VIDEO

SLIDES

[seoslides embed_id=”24772d7de05d” script_src=”https://devopsy.kinsta.cloud/embed-script/variable-speed-devops-ibm-chmc/6919/” overview_src=”https://devopsy.kinsta.cloud/slides/variable-speed-devops-ibm-chmc/” site_src=”https://devopsy.kinsta.cloud” site_title=”devopsy.kinsta.cloud” title=”Variable Speed DevOps-IBM-CHMC” /]

Your Host:

Alan Shimel, Editor-in-Chief devopsy.kinsta.cloud, An often-cited personality in the security and technology community and a sought-after speaker at industry and government events, Alan has helped build several successful technology companies by combining a strong business background with a deep knowledge of technology.

About Panelists:

Jeff Blackadar, Application Development Manager, Canada Mortgage and Housing Corporation

A veteran at application development, Jeff leads a team that programs COBOL, VB.Net, Java and Enterprise Generation Language applications. He is dedicated towards implementing and upgrading of tools and practices to improve productivity and reduce the risks of making changes to mission critical mainframe applications.

Venkat Balabhadrapatruni, Senior Technical Staff Member, Chief Architect, Enterprise Modernization, IBM

Powered by extensive knowledge of z Systems, software design, development, delivery and enterprise modernization, Venkat is responsible for technical architecture and direction of Rational Developer for System z (RDz) and understanding requirements, emerging technologies, market trends and translating them into a technical strategy

April 28, 2015

Is DevOps a Competitive Advantage? A Debate

The idea of DevOps being a competitive advantage is not a new one. Service providers, software sellers, and internal IT frequently use this argument to justify all kinds of investments. But is there any real merit to this argument? Is DevOps truly a source of sustainable competitive advantage? Or is it simply just hype and a new way to sell a product / service?

Here are some of the more popular points of view from both sides of the aisle.

“Yes Absolutely”	“No, Go Take A Hike”
Speed is sexyIn the mobile, social, and web world customers expect speed and responsiveness. Bringing an idea to the market faster than your competitors is definitely an advantage and DevOps allows us to do just that.	DevOps is not the only fish in the seaAgreed that speed is sexy. But at this given moment there are multiple technologies / philosophies out there that claim to do the same – i.e. cloud, big data, analytics etc. What makes DevOps so different?
DevOps is hard to replicateWhile purchasing tools is easy, the real benefit lies in getting the culture and process right. Any organization that can get that right will be able to form a sustainable competitive advantage.	Same has been said about other strategiesRemember when outsourcing started. It too was touted as a sustainable competitive advantage. Now it is the de facto strategy for IT erasing any advantages that the early adopters had. DevOps, if it does take off, too will follow the same path.
New IT operating modelDevOps represents a new way for enterprise IT to operate. It breaks down the silo structures and unlocks the value trapped within the existing organization.	Enterprises are not start-upsJust because it worked in a start-up environment does not mean that it can work in a large enterprise.
Listening to customersSpeed is only one equation of DevOps advantage, the other is faster feedback loops so that you can determine that you are going in the right direction. Combine the two and you have a powerful competitive advantage.	Not an IT responsibility / competencyIsn’t that what most organizations already do? I have my entire business / product / marketing org. focused on just that. The last thing I need now is IT to jump in and muck it all up.
The new Software Delivery LifeCycle (SDLC)DevOps represents a new way for enterprise IT to operate. It breaks down the silo structures and unlocks the value trapped within the existing organization.	Just good marketingThis is just repackaging of an old wine into a new bottle…DevOps is just a new thing. Like all other fads this too will pass.

My Take

Allow me to be the quintessential consultant and speak from both sides of my mouth. DevOps is and is not a competitive advantage. Now before you close the browser and move on, let me explain.

DevOps in the short term is most definitely an advantage. Moving an organization from Traditional to a DevOps approach is not an easy task. Any corporation that can successfully achieve this transformation will be the first to reap its benefits – from faster speed to market to being more responsive to their customers. And the benefits are real! Ask Netflix and Etsy, who have been able to out innovate their competitors and gain a sizable lead in the market.

However, in the long term, as more and more firms invest in DevOps the advantage will continue to shrink. DevOps, if you strip all the marketing jargon and go down to the most basic definition, is about reorganizing the internal plumbing of IT (i.e. how IT performs the various tasks within the software delivery lifecycle). And while in the short term plumbing can allow you to do some neat stuff, at the end it’s only internal plumbing and rarely a source of lasting competitive advantage.

To see this is action, look at the automotive, financial, and then our very own Information Technology industries. In all three examples, the company that was able to transform was able to establish an early sizeable lead, reap the rewards, and the rest of the industry forced to catch up eventually.

In automotive industry, early investment in lean and TQM earned Toyota customer loyalty and a significant lead over its traditional competitors. And while the lead lasted for a decade (short time in an industry where product lifecycles are 4 – 5 years), competitors quickly started to invest and catch up. You can now argue that U.S. manufacturers are at par with Toyota in both quality and speed. Where the competitors have still not caught up is in perception.

In the 1970s it was not unusual for a home buyer to wait for up to 90 days to get a credit decision and a loan, as per “industry standards” in those days. Citibank rearranged its internal processes to give buyers a decision in 30 days. It took the industry by storm. Competitors eventually caught up and now 30 days is the norm.

And as for our very own, Information Technology industry…outsourcing was considered to be a competitive advantage for many of the first movers. Like other strategies, it too required a structural change to make it happen. Many of the initial pioneers were able to cut IT costs by over 30% and establish a cost advantage over their competitors. But within a decade competitors copied, banked the savings, and eroded the cost advantages.

In my opinion, DevOps too will follow a similar path. A few leaders will emerge (already happening), they will set the pace of change for the industry, and the rest will have to follow….whether they like it or not.

About the Author

Mustafa Kapadia is the Service Line Leader for IBM’s DevOps practice, a business advisory practice focused on helping large enterprises transform their software & application delivery. He has over 17+ years of experience in the tech. space, both as a service provider and a management consultant.

Prior to joining IBM, Mustafa was a management consultant with Deloitte’s Strategy & Operations practice. He lives in San Francisco Bay Area, is an avid blogger (when time permitting), a speaker, and adviser to start ups.

Connect with Mustafa LinkedIn| Twitter

April 21, 2015

DevOps and the Case for ROI to Executives
DevOps is hot and likely top of mind for your executives.

Every IT analyst firm now has a point of view (POV) on DevOps. International Data Corp. (IDC) for example believes that DevOps will be adopted (in either practice or discipline) by 80% of Global 1000 organizations by 2019 (read: IDC MaturityScape Benchmark Assesses DevOps in the United States).

DevOps events, conferences, web sites, meetups, and communities are proliferating like wildfire. For example, the devopsdays conferences started with a single event in 2009. Fast forward to 2014, through DevOps’ meteoric rise…18 devopsdays events were held at locations scattered far and wide across the globe.

The top IT vendors all have DevOps product lines and consulting services, and new technology startups and VC funding continually enter the space. Why? Organizations need to deliver more applications and systems faster with higher quality, at reduced risk and expense. Downtime has become untenable at any cost, and every second of time to recovery equates to lost revenue. DevOps addresses these challenges through improved collaboration, workflow, and automation solutions, based on proven Agile and LEAN principles and techniques, that improve speed to market and reliability. A DevOps approach delivers technical and business results to LOB leaders, the C-suite and to IT app delivery and operations.

Organizations need to deliver more applications and systems faster with higher quality, at reduced risk and expense. Downtime has become untenable at any cost, and every second of time to recovery equates to lost revenue.

If you are responsible for justifying investment in new mobile or cloud app platforms, or modernizing enterprise apps, or implementing a DevOps solution for continuous delivery, then our upcoming webinar will be of interest to you. When you ask stakeholders to change what they’ve done successfully for years, and adopt a DevOps approach, you must monetize the benefits and ROI for them and make the organization change real. Like any change in approach, investment and ROI are required before it will be adopted and scaled widely. And like any executives who sponsor or fund an investment, your executives will want to see a significant, monetized payback, and the quicker the better. As you know, senior executives in the C-suite love numbers, and those with dollar signs are even more endearing to them.

Like any change in approach, investment and ROI are required before it will be adopted and scaled widely. And like any executives who sponsor or fund an investment, your executives will want to see a significant, monetized payback, and the quicker the better.

Join Valerie Scott, Manager of the Shared Services Division at HM Health Solutions, and Peter Spung from IBM, as they discuss monetizing key DevOps benefits and associated costs. They will share a case study where cost benefit analysis frameworks were leveraged to justify DevOps ROI to senior executives.

Speakers: Valerie Scott, HM Health Solutions, and Peter Spung, IBM

HM Health Solutions invested in and adopted proven DevOps practices, such as: test-drive development, automated testing, streamlined code review, continuous integration, and continuous delivery via automated deployment.

These practices have been rolled out across dozens of applications and environments thus far, and will be rolled out across hundreds in the coming months. The results have been stellar thus far:
- a 67% reduction in resource demands on the release team & process,
- a 75% reduction in relative hours to repair defects per environment,
- and an 82% reduction in app deployment time
This is just the tip of the iceberg as Val will describe — she’s just getting started on HM Health Solutions’ DevOps journey! And they are not alone. These results are typical and reflective of many other client cases studies and success stories, as Peter will describe.

Please join us April 23 from 1 to 2pm EDT for what will be a very engaging and insightful discussion!

You can register for the webinar here!
April 16, 2015
Why do DevOps? Winning hearts and minds with a Lean boxscore

Change is difficult. It can be painful, disruptive, and costly. Making a case for change requires us to win over hearts and minds. In my experience, we need to win the hearts of the ‘do-ers’, the people whose daily work is going to be disrupted. And the minds of the executives, whose financial support and personal sponsorship we need to make change happen.

A Lean boxscore allows you to address both the hearts, and the minds. Boxscore is a simple tool which we use to compare the current state with the future state, and turn that comparison into a compelling case for change.

Step One: Current State Baseline
Let’s start with where we are today, by capturing performance metrics that tell an accurate story of today’s development lifecycle. This may mean looking beyond the project plan to the reality, by the way. By interviewing or workshopping with development and operations teams, we get a true picture of the way the development process currently performs.

We’re always interested in end-to-end leadtime, of course – from the moment the business provides a prioritized set of requirements, to the moment that functionality is in production and working as intended. We’re also interested in how much wait time there is within the process – how much time between steps, waiting for an approval, a developer, or a tester. We do this by creating a value stream map, capturing all those metrics and validated by the teams who work within this process every day.

We want to look beyond leadtime and wait time too. We capture the number of resources involved in each release, the number of handoffs, and the number of bugs caught at each stage of development and deployment. If we can get to it, we might capture the percentage of time a developer spends writing code for their current release, as opposed to firefighting issues from the last release, arguing over schedules, trying to secure an appropriate environment, or other ‘non value adding’ activities. We might also capture some usage statistics, or other measure of customer satisfaction, and any relevant performance indicators, like server response time.

Before moving on, it’s crucial to ask stakeholders: ‘is this a fair reflection of the way things currently work?’

Step Two: Future State Target
This next step requires some imagination, and some bold thinking. Imagine a world where you’ve adopted DevOps: you’ve removed the waste within the development lifecycle and instilled the ‘shift left’ principle in all your working practices, your ‘4 in a box’ teams are rapidly developing, testing and deploying code, Cloud-based environments give you immediate provisioning, and automation and smart tooling have taken the bellyache out of testing. What does that future look like? How quickly can you develop – if provisioning is immediate? How many bugs do you expect to put live – if testing is automated and your production environment matches your test environment? How many changes to scope – if you’re working with smaller batches and faster delivery?

We create a new value stream map, which details the way this brave new process will work in practice. This allows us to set sensible but challenging targets for our future state.

Before moving on, be sure to ask stakeholders: ‘are these targets achievable, if we make the changes we have outlined?

Step Three: So What?
Finally, we get to the money question. What’s it all worth? For each improvement there will be an impact. Many of these will be financial. Ask your stakeholder ‘if we can get this new functionality out to market 2 months faster, what does that mean for revenue?’ Or ‘if we reduce the number of defects in production by 90%, can we reduce our operating costs?

Putting a dollar value on the financial impact will win over those minds with a logical, fact-based argument. Now, investing in a DevOps pilot seems like something they can’t afford NOT to do.

And the case for change for hardworking developers, testers and production resources? Metrics like number of handoffs, bugs and rework, and batch sizes can easily be translated into ‘how much pain you have to live with each day at work’ to win those hearts over.

April 9, 2015
Think you’ve established Kanban? Why you might only be halfway there
As a Lean Coach working in IT I often hear development teams refer proudly to their scrumboards as an example of Kanban. To which I say, ‘that’s a great start… but it’s really only just the start’.

Let’s start with what a scrumboard is, and does. A scrumboard is a visual display of work, showing the progression of user stories in development. As you see some stories moving through the phases of development, and others getting stuck, you huddle to figure out how to get them unstuck. Now you’re using your visual display for visual management, which is a great Lean solution.

Change management boards, like the one used in The Pheonix Project, are another good example of visual management. The physical display of changes, shown by release date and system, is a great way to get a handle on the amount of change and risk you’re throwing into live production. When you start moving them around to limit the risk, again, you’re using visual management.

The word ‘kanban’ roughly translates as ‘signboard’, and a scrumboard really is a good start. But it’s only half the story. Until you limit the number of stickies on your change management plan, or the number of story cards on your scrumboard, you’re really only half way to kanban nirvana.

True kanban requires you to limit the amount of work in progress and to establish the pull of work. In a manufacturing plant, a kanban card is attached to each unit in production. When a completed unit is, say, taken off the shelf by a customer, the card is sent back into the production line to signal that there is now an empty slot on that shelf which needs replenishing. Limiting the kanban cards sets a constraint around the amount of work in progress – and this limit should reflect the capacity of the process. In the same way, you can limit the number of story cards, boxes on your scrumboard or spaces in your change management plan. When a scrumteam complete a story that frees up a slot for them to pull the next story into their workload: the visual display should reflect this by showing each team’s capacity. In addition to visual management, you can create rules around work in progress, requiring each scrumteam to limit their intake so they have no more than, say, two workpackages in progress at a time.

This is all well and good, but what do you do when demand from the business requires a greater capacity? Historically you’d have thrown that extra demand into the mix, asking your teams to do more with the the same amount of resource, maybe pushing back a release date to accommodate the extra work.

Kanban requires you to create additional capacity in the system to allow for the additional demand either by:
- rebalancing the load, for example switching some resources from a work package that is ahead of schedule
- removing or deprioritizing other work and putting work on hold
- increasing resources by bringing in new team members
All of these actions will generate some churn, so it’s going to be worth a tough conversation with your customer first. ‘Are you sure you want me to disrupt the current sprint, or the next release, to squeeze this in? How about we pick it up next week, when we’ll have some new capacity?’ By limiting your work in progress, you’ll be speeding up the flow of development into production, so you can make that ‘next week’ promise with increasing confidence.

So the next time you walk over to that scrumboard, why not take the next step, and limit the work in progress to truly establish kanban.
April 8, 2015
Webinar: How Nationwide Insurance is Adopting DevOps for Variable Speed IT Systems

To drive towards faster innovation, and also ensure the stability of the core business, the need arises for what is being termed as “two-speed or variable speed IT.” While innovative Systems of Engagement demand speed, agility and experimentation, legacy Systems of Record need governance and predictability. Two-speed IT enables adoption of DevOps capabilities by adapting them to the “speed” of delivery. It requires two-speed business planning and processes and even two-speed organizational structure and budgeting. Join Carmen DeArdo, Director of Application Development at Nationwide Insurance, Sanjeev Sharma, lead DevOps Technical Evangelist at IBM, and Lee Reid, executive IT specialist as they show how IBM’s solution is helping Nationwide adopt DevOps practices for variable speed IT systems.

This webinar is the first in the Variable Speed DevOps – A Deep Dive series.

Event recorded Tuesday May 5th, 2015

SLIDES

[seoslides embed_id=”be17733a1254″ script_src=”https://devopsy.kinsta.cloud/embed-script/webinar-how-nationwide-insurance-is-adopting-devops-for-variable-speed-it-systems/6643/” overview_src=”https://devopsy.kinsta.cloud/slides/webinar-how-nationwide-insurance-is-adopting-devops-for-variable-speed-it-systems/” site_src=”https://devopsy.kinsta.cloud” site_title=”devopsy.kinsta.cloud” title=”Webinar: How Nationwide Insurance is Adopting DevOps for Variable-Speed IT Systems” /]

VIDEO

Your Host:

Alan Shimel, Editor-in-Chief devopsy.kinsta.cloud, An often-cited personality in the security and technology community and a sought-after speaker at industry and government events, Alan has helped build several successful technology companies by combining a strong business background with a deep knowledge of technology.

About Panelists:

Carmen DeArdo, Nationwide – Director – Tools and Technology Leader

Carmen DeArdo, Director – Tools and Technology Leader, Nationwide Insurance, An ardent advocate of DevOps and a leading speaker in the DevOps and technology related events, Carmen is responsible for driving continuous delivery utilizing DevOps, Lean and Agile techniques across Mobile, Distributed and Mainframe and other technologies.

Sanjeev Sharma, IBM Worldwide Lead – DevOps Technical
Sales and Adoption, IBM Cloud Unit

Sanjeev Sharma, IBM Worldwide Lead – DevOps Technical Sales and Adoption, IBM Cloud Unit, is a 20-year veteran of the software industry with expertise in DevOps, Mobile Development and UX, Lean and Agile Transformation, Application Lifecycle Management and Software Supply Chains. He is a DevOps Thought Leader at IBM and speaks regularly at conferences. He has written several papers and is the author of the DevOps for Dummies book.

Lee Reid, IBM – Senior IT Specialist

Lee Reid, Senior IT Specialist, IBM, is a senior solution architect with focus on practical adoption of software engineering tools, software delivery solutions, application life cycle management, and DevOps. He has over 25 years of experience in software engineering, design, programming, testing, development, tools & methodology, team leadership, and strategic planning.

March 31, 2015
IBM’s Watson sits at the crossroads between DevOps and Big Data

While attending IBM InterConnect in late February I had an opportunity to sit in on a press conference. The focus of the press conference was primarily around IBM’s efforts related to hybrid cloud solutions, but the thing that caught my attention was a little tidbit mentioned almost in passing about an innovative way students at the University of Texas Austin had applied the power of the Watson supercomputer.

Robert LeBlanc, senior VP of IBM Cloud, and a handful of other executives shared IBM’s vision for hybrid cloud and big data analytics with the media. The press conference ranged from data sprawl, to data visibility, to security and performance. The group talked about a focus on developers and the importance of open standards and more.

Then there was the bit about Watson and the University of Texas. For those who may not know, Watson is a cognitive computing system. It is a supercomputer with an artificial intelligence component that enables is to analyze massive volumes of data, understand complex queries posed in natural language, and respond with evidence-based answers. It learns as it goes.

Watson University Competition

IBM has enlisted the support of companies and organizations around the world to help develop new applications for Watson and push the envelope of its capabilities. One of the programs that helps drive innovation with Watson is the Watson University Competition.

Students participating in IBM’s Watson University Competition were directed to identify and solve an industry-specific challenge using Watson. Students had to work as a team to identify and input relevant data into Watson in order to train the supercomputer to answer questions related to the issue being solved. The students must then develop an app and supporting business plan targeting their chosen industry.

The UT Austin team won the competition with an app called CallScout. The purpose of the app is to help Texas residents find information about social services and programs in their local area. Many Texas residents depend on state resources for housing, healthcare, transportation and other vital functions, but wading through government websites and other information is confusing and makes it difficult for residents to find the information they need. The CallScout app integratesl hours of service, route and map information and other relevant data, and it can automatically deliver push notifications when important details change.

“This is more than a school project for us – it’s about creating a sustainable business that addresses one of the key challenges we all face as Texas residents,” said Bri Connelly, team leader and undergraduate Computer Science student at the University of Texas at Austin. “The opportunity to directly impact citizens of our home state was a huge driving force in our work.”

Watson depends on DevOps

Watson is a powerful platform for analyzing and understanding information. More than any technology or platform that has preceded it, though, Watson also depends on constant evolution. IBM has embraced DevOps principles for developing and maintaining Watson out of necessity. Watson simply wouldn’t be possible and wouldn’t be able to keep up with the pace of ingesting and adapting to information using legacy development models.

The IBM Watson team manages the rapid pace of change using DevOps principles, which are an extension of Agile development methods. Carl Kraenzel, an IBM distinguished engineer and director of the IBM Watson managed services group, explained that effectively maintaining Watson means adopting continuous delivery and continuous availability. “If you don’t approach it the right way, change is a cost driver. It will add cost instead of adding value. DevOps helps us achieve continuous delivery, and DevOps helps us deliver continuous value.”

IBM is a resilient company. It doesn’t have the same dominant role in the tech industry that it once had, but it didn’t just sulk off into the sunset. IBM has adapted and evolved. IBM InterConnect was an excellent demonstration of the innovation and vision driving IBM right now—positioning IBM to play a significant role in the new cloud, mobile, DevOps future. Watson, and the work done by the UT Austin team and other students are perfect examples of what’s possible when DevOps principles and cutting edge technology are applied to help resolve the issues that face mankind.

March 26, 2015
DevOps Golden Rule: Writing More Code for Test Than For Production

Continuous improvement can help completely turnaround a staid IT organization into an innovation machine. But without effective test automation, that transformation is a non-starter. So says Gary Gruver, who has led DevOps journeys both at HP and Macy’s and now works as a consultant guiding other organizations through his firm, Practical Large Scale Agile.

Last month at IBM InterConnect, Gruver gave audience tons of great big-picture advice on leading DevOps from the top of the food chain. But perhaps one of the best takeaways when drilling down into the details was his emphasis on test-driven development.

“It needs to be reliable, it needs to be stable, it needs to be maintainable. i would say test automation is something that’s the most important if you’re going to transform your organization,” Gruver said. “And it’s one of the things that’s most often done wrong. If you do it right it’s extremely valuable, if you do it wrong it’s going to be a maintenance nightmare.”

He provided real world lessons he learned from his stints at Macy’s and HP, explaining, for example, a situation at HP where unstable tests and unstable codes on trunk had kept the team from checking in code for four days. With a $2 billion business “breathing down my neck,” Gruver admitted he was getting antsy at that point and found out that the instability prompted them to add even more tests to the process.

“So we weren’t getting any better because no code was making it in to get fixed and what we really needed is a process that lets the good code in and keeps the bad code out,” explaining that his team eventually settled on gated commits. “That’s where if the code doesn’t pass the system tests it needs to for red build, then it doesn’t make it into SCM and doesn’t make it on to the trunk.”

Meanwhile, when he got to Macy’s he was told there were 1,000 automated tests ready for use. But when he started marrying those tests with the burgeoning continuous delivery team, they found not a single one worked reliably enough to depend upon. He recommended to the audience to take a look at the architecture philosophy’s put forth in Jeff Morgan’s Cucumber and Cheese, which helped in his work at Macy’s, where by the end of his stint the IT department was running over 5,000 automated tests several times a day, versus 1,300 every 10 days.

He explained that he’s a firm believer that quality assurance should be brought together with the architects to make sure the automation framework is sustainable for the long haul.

“Get your lead architect on the development side to engage with your lead on the quality assurance side and think about how to architect your framework for your test automation where its maintainable,” he said. “If not, you’re going to die under the weight of test automation.”

If done right, an organization will very well likely write more code for test than for running its product.

“And that’s not a bad thing,” he says. “You’ll find when you write good test automation it forces you to write better code.”

March 19, 2015
Shift Left: Can you be left out?

One of the most popular terms bandied about in DevOps is “shift left”. I was at a workshop last month at IBM InterConnect conducted by Sanjeev Sharma and Mustafa Kapadia. They put on a great workshop which was a preview of an exercise they perform in the field for clients. A central theme of their talk was the idea of shifting left to address issues earlier in the development process.

iBM, Sanjeev and Mustafa (both of whom write here on devopsy.kinsta.cloud) are not the first or only people who speak about shifting left in DevOps. Derek Weeks of Sonatype wrote a blog post about it back in September of 2013. Almost a year to the day later, Dibbie Edwards of IBM (and the focus of a good article by George Hulme) wrote a good article about shifting left with continuous testing. Andrew Storms wrote a post here on devopsy.kinsta.cloud some months back about shifting security left in DevOps. Indeed shift left is generally regarded as a basic foundation of DevOps.

Before we dive into what this means to you, we should make sure that we all understand what shift left really means. For me looking at a standard flow of the development/deployment process is the best illustration of this. I think Andrew Storms used a good one in his article, so let’s repurpose it here:

Pay no attention for now to the before DevOps or DevOps rays for now. Take a look at the top line which is the pipeline. You can see the flow of an app from plan to code and all the way to operate. The flow runs from left to right. At its simplest shift left moves tasks further left. Security, continuous testing, continuous integration, continuous delivery are all shifted left.

In doing so potential issues are addressed earlier in the lifecycle. What is this a good thing? A chart from Dibbie Edwards article gives us the obvious answer:

As can be seen the earlier you can get to issues, the cheaper it is too fix them. That alone is a reason to shift left. But also concepts such as addressing security issues before it effects your customers is appealing. The same can be said for many other functions you can shift left.

For me though the question is by shifting everything left, do we just wind up with a big pile at the very left? How do we prioritize all of these left shifted functions? You can only shift left so far before you come to the beginning of the time line. This is not the birth of the Universe where you had a big bang which defies the laws of physics.

At some point can you shift too much left? This is a question I am going to pose to Sanjeev and Mustafa. I also pose it to you the reading audience.

I recognize that the concept of shifting left is not so efficient that everything that can be shifted is in fact shifted left. But as we get more efficient, more and more tasks are left shifted in the process. At what point do we reach the point of diminishing returns?

How do you feel about this? Can we shift too much left? I am interested in some of the different views on this. I am a big proponent of shifting security left, but are there other priorities?

Let us know your feedback.

March 19, 2015
IBM’s DevOps Journey

Companies that want to better address market demand and more efficiently deliver value through speed, quality, and agility are embracing DevOps. This includes retailers, insurers, defense contractors, and most certainly software makers.

Consider the DevOps journey within the IBM Software Group, which is unarguably larger than most enterprise software groups: currently 519 major software product titles, 36,000 software engineers, 116,000 virtual machines supporting development and testing, and 44,000 physical servers. “This is DevOps at scale,” said Matt Ellis, vice president mobile and cloud DevOps at IBM, recently at InterConnect 2015.

Getting teams and processes of that size – while cutting waste associated with outdated ways and duplicate toolsets – is no easy feat. But if it can be done, the benefits are substantial. For IBM, the efforts do appear to be paying off – if the results recently shared over the past six months at the DevOps Enterprise Summit 2014 and the IBM InterConnect 2015 conferences are any indication.

According to Dibbie Edwards, VP DevOps for hybrid, continuous engineering and application lifecycle management development at IBM, IBM has gone from spending about 58% of its development resources on innovation to about 80%.

What did that shift in focus do to IBM’s time frame? In 2008, IBM’s time to project initiation took 30 days. That was reduced to between two and three days in 2014. Groomed backlog was reduced from 90 days to one day and sprint test times from five days in 2008 to 14 hours today. Overall time to development went from 120 days to three, with time between releases reduced to three months from 12.

“IBM is seeing the value of DevOps, and as you heard themes through each one of those sessions, it’s really about speed,” said Edwards at the DevOps Enterprise summit this past fall. Speed has always been important, to be sure, but with the rise of cloud, analytics, mobile, and social, it’s become more important now than ever.

How did IBM begin its DevOps journey?

According to Edwards, it was systematic. The teams examined the work they were doing: all of their processes – from how they worked with their business stakeholders to how they were working with clients – and eliminated all of the inhibitors and friction points. It hasn’t been a swift journey – a couple of years – but it has been a meticulous one.

According to Edwards, during her talk at the DevOps Enterprise Summit 2014, Real-life Experiences Leading Large Organizational Transformation at IBM, the software group focused on uncovering every manual process that it could – things such as how the tests were being created, building more efficient testing tools, and building a page object framework in selenium that its developers could leverage. “That framework itself has allowed us to more than double the number of automated test cases that we have,” she said. “That very significant ability gave us the option to do our function verification testing alongside our development.”

With manual inefficiencies identified and senior leadership agreement to prioritize how processes would be improved and how much will be invested on clearing technical debt, the IBM Software Group then had a foundation on which it could build development processes based on DevOps principles and current business objectives. “We proceeded to build a groomed backlog based on those business priorities and on the size of the development work,” Edwards said. “And if the teams came up against work that they didn’t understand, they would run an exploration phase within their sprint to better understand what needed to be implemented.”

With that groomed backlog in place, IBM could then make more informed decisions at the end of each development sprint – and even pivot, if needed. “If we had a customer need or a business change where we need to shift the focus of the individual teams, we had the flexibility to do that,” she said.

That ability to pivot had been a big benefit for the IBM Software Group. “I recall just how many times we would just churn over these decisions and remake them and make them again,” she said.

Continuous Testing Automation

The culture at IBM has moved to full automation ahead: everything should be automated, and it’s an exception if it isn’t. “I think this is so critical. I saw a real shift in our team as it started to see the value of this and recognized that doing things manually just was not acceptable and really not sustainable, given the level of speed that we were trying to achieve,” says Ellis.

Previously, IBM completed testing in following sprints, because testing teams just could not keep up. Getting to this point was an effort that took focus on continuous improvement. “One thing I always share with the team is how critical learning at the end of each one of our sprints is. And then planning for the next sprint to eliminate the friction points and the areas where we were less efficient than we should have been,” she said.

Finally, IBM automated the testing of its more complex customer usage scenarios. “It literally used to take my team 30 hours to set up an environment to do the testing because we did everything manually,” she said. “We would have to set it up, break it down, and reset it up, because we had all the different databases configurations. By using patterns and automation, we were now able to deploy automatically within three hours.”

March 9, 2015